OSG Exporter - Texture seams somehow have alpha

[JvdBosch]

For a large project I migrated from 3ds Max Design 2011 to 2013. I installed the new exporter and some strange stuff happens when exporting.

There's a scene with only diffuse textures (houses), which I export using the setting displayed below. But somehow there's alpha present at the seams. Is there something wrong with the exporter? With texture clamping or something like this.

This is the problem. You can see behind the wall at the seams:


Wall texture is a standard texture. Object has a Map Scaler attached:


Export options used:


Any ideas??

[shivanangel]

Try moving your UVs a very small amount away from the corners to quick fix the problem.
Also, did you make the original texture in Photoshop and resize the image?
Many times, the edges of images that are not flattened first are given a feathering at the edges, which can result in poor seams at the corners. Are you starting with a different format than jpeg, such as PNG or TGA which has an alpha channel?

Do you get the same problem when you change your compression type from DX to no compression? It looks at first glance like a compression problem from the image...

[JvdBosch]

Originally Posted by shivanangel

Try moving your UVs a very small amount away from the corners to quick fix the problem.

There are no corners in the model. It's one big polygon. Within the polygon this happens.

Originally Posted by shivanangel

Also, did you make the original texture in Photoshop and resize the image?
Many times, the edges of images that are not flattened first are given a feathering at the edges, which can result in poor seams at the corners. Are you starting with a different format than jpeg, such as PNG or TGA which has an alpha channel?

Do you get the same problem when you change your compression type from DX to no compression? It looks at first glance like a compression problem from the image...

I used DXT1, which only has 1-bit alpha. The seams are clearly more than 1-bit alpha. The texture files are flat jpg/bmp files, with no alpha attached. This model worked fine in 3DS Max Design 2011...

[shivanangel]

Originally Posted by JvdBosch

There are no corners in the model. It's one big polygon. Within the polygon this happens.

So it is your texture that is causing this issue and it wrapping from one polygon to the next.

I used DXT1, which only has 1-bit alpha. The seams are clearly more than 1-bit alpha. The texture files are flat jpg/bmp files, with no alpha attached. This model worked fine in 3DS Max Design 2011...

I've noticed exporting issues with some image files when exporting from the native format to whatever techniques are used in 3D Studio Max to convert them to other file formats. Did you try this with no compression and see if you get the same result?
Just because it works in 3DS Max does not mean it will work outside of the application. Especially when you consider mipmapping and the potential for textures to be resampled during the export and import process.

[JvdBosch]

Originally Posted by shivanangel

I've noticed exporting issues with some image files when exporting from the native format to whatever techniques are used in 3D Studio Max to convert them to other file formats. Did you try this with no compression and see if you get the same result?
Just because it works in 3DS Max does not mean it will work outside of the application. Especially when you consider mipmapping and the potential for textures to be resampled during the export and import process.

Like I said, the exact same model with the same textures exported fine in 3DSMax 2011. Something has changed on either 3DSMax 2013, the exporter or both. I'll try to reproduce the issue with a simple max file, so the experts at WV can take a look.

It looks like the exporter is doing something weird at the edges of the texture images. I had something similar with tree textures, face mapped to a square polygon. At the edges of the polygon the transparency of the texture disappeared, so it looked like the trees were inside a box... So exactly the other way around than the wall, where a transparency appears at the edges of the texture.

[JvdBosch]

Here's an example to download.

It is just one tree, made with two crossed polygons and a simple material. When exporting a line appears at the edges of the texture. I believe this is the same problem as with the wall. Some weird alpha thing happens at the edges of the texture. In my opinion it has nothing to do with the polygons, but everything wih the textures.



Can someone take a look at this problem?

[Veleno]

With the brick wall texture, what format are you using? The transparency seems texture-based, so it is most likely something wrong with the texture itself. Bring the image in photoshop and check for an alpha channel. if there is one, delete it and export again.

If this is in a baked scene, edit the image output settings to disable the automatic creation of alpha channels in baked textures.

The problem wiith the tree can be solved by using multi-sampled alpha. This kind of artifact usually happens when one transparent object intersects with another.

 
import viz

viz.setMultiSample(4) 
#higher values increase quality but decrease framerate. 
#usually set between 2 and 16

viz.go()

tranModel = viz.add('model.IVE') #add a model, call it tranModel

GL_SAMPLE_ALPHA_TO_COVERAGE_ARB = 0x809E
transModel.disable(viz.BLEND)
transModel.enable(GL_SAMPLE_ALPHA_TO_COVERAGE_ARB)

viz.clearcolor(viz.GRAY)

[Veleno]

The above code has a typo - lines 12 and 13 say "transModel" instead of "tranModel", so be sure to fix that if you try it out.

[Veleno]

The edge problem with the tree is happening because you have tiling disabled on the diffuse and alpha textures. Anything beyond the 0-1 UV range becomes white by default.

This can be fixed by:
■ Re-enabling tiling on the model, in bitmap parameters
■ Preventing the texture from being used past the 0-1uv range in bitmap parameters (eg. set tiling to 0.99)
■ Set the wrap mode for the textures from within Vizard (see <multimedia:image>.wrap in the help docs)

By the way, if the texture used for the brick wall does not have an alpha channel, upload a copy of the wall and its texture and I'll take a look.

[JvdBosch]

Michael,

Thanks for your input and effort. However the problem still exists... I already had GL_SAMPLE_ALPHA_TO_COVERAGE_ARB enabled on the model (there are thousands of trees in the complete model). And tiling is enabled on the wall texture.

I've put together a zip of all files with a test script. You can find it here.

Both the wall texture and the tree texture exhibit the same type of problem at the edges of the texture. When exporting the same model in previous versions of 3DS Max there were no issues...

[Veleno]

Hi JvdBosch,

I took a look at the files in your model.

■ Tree Lines: Tiling is the cause of the lines appearing on the trees, though I see I wasn't very clear in my previous post what part of the model I was talking about.
■ Wall Transparency: Your wall texture, baksteen_CrRedGrain (Custom).bmp, has an alpha channel. The alpha is slightly gray at the borders, and this is causing the transparency issue. I'd guess the gray border probably came from re-scaling the image. I've attached a copy of the BMP with the alpha channel removed (.zip), along with a screenshot showing the problem. In Vizard, any alpha channel found in the diffuse texture is automatically used for transparency. (Bitmap_Alpha_Issue.jpg)
■ Multisampled Alpha: The code looks correct and the effect works correctly for me in the provided example. If this isn't working on yours, then my best guess at the moment is that it may not be supported by your graphics card. I've attached a screenshot of what it looks like on my workstation with an unmodifed version of your tree model (Tree_SampledAlpha_NoLineFix.jpg). When multisampled alpha is working correctly, you should see a screen door artifact when close to the transparent parts of a model (ScreenDoorArtifact.jpg). This artifact lessens at higher multisampling levels, though the banding generally stays.

[JvdBosch]

Michael,

WALL
Thanks for looking at the model. I see now there is indeed a alpha channel in the texture, my bad! This was no issue in previous versions of the exporter/MAX/Vizard using the same textures/models. That's why I didn't suspect the problem to be with the texture.

When I set the exporter to not export opacity maps and use DXT1 it still exports the alpha channel within the diffuse texture image? I guess this behavior has changed since MAX2013 or the new exporter or the latest Vizard? That could explain it, since I used this setting in earlier versions without this problem.

TREE
Regarding the tree, I'm using a GTX680 with the latest drivers. The multisample alpha was and is working fine. The lines are the biggest problem. You say tiling is the answer, so I enabled it. But then you see lines on the top of the tree from the repeating of the treetrunk. See the picture below.

Why is tiling an issue? It never was with previous versions... I find it peculiar to have to set tiling to 0.98 or so to solve a problem which did not exist previously. My complete model has several hundreds of objects with a non-tiled texture. There must be a better way to solve this problem.

Thanks,

Jaap

[Veleno]

Hi Jaap,

Wall:
■ Setting the exporter to DXT1 does appear to be converting alpha channels to 1 bit alpha instead of discarding them, which is odd. The compression ratio still appears to be correct for DXT1. I'm unsure whether this is functioning as intended.

■ Unchecking opacity under maps prevents max from exporting any maps placed in the Opacity slot of a material, which can also be used for transparency.

■ In general, I'd recommend avoiding alpha channels in maps that don't need them. When looking at a model in Inspector, the texture panel will show the format as RGBA if an alpha channel exists for the texture. If not, it will show RGB instead.

Tree:
There are a few ways to fix the issue:

■ Max Method: I've attached a maxscript that will loop through all scene materials (Standard and Standard contained in a multi-material), check for diffuse and opacity maps where tiling is disabled, and if so apply the above tweak. To use it, go to Maxscript -> Run Script... and select Jaap_TweakTiling_2012-01-03.ms.

■ OSGT Method: Save transparent file as OSGT. Open up the plain text model file in your favorite text editor, then use Find and Replace to replace all instances of CLAMP_TO_BORDER with CLAMP_TO_EDGE (case sensitive).

■ Vizard Method: Loop through all textures on the transparent model, changing all tiling that is currently viz.CLAMP_TO_BORDER to viz.CLAMP_TO_EDGE. See the Texture Wrapping entry in the help docs.

The way the exporter currently functions is consistent with how max handles them, at least as far as the diffuse texture goes. However, it does seem like it would be more beneficial, especially in cases like this, to include an option to repeat the texture's edge pixels instead.