HLSL effects

[nox771]

Has anyone figured out the HLSL effects?  I've been working with these settings, and overall I like the things that can be done, but I'm running into a couple problems.  I am running MAMEUI 143 as my test setup.

The first one I've read about elsewhere, and has to do with the ini files.  I set "hlslini hlsl" in my mame.ini file, and everytime a game exits it dumps it's settings into the hlsl.ini file (I'm not concerned with multiple games right now as I'm only testing things).  The problem is 99/100 times the "shadow_mask_texture" line gets corrupted, and the next time the game starts it tries to read that file (and its corrupted line) and it fails and dies with a black screen.  Is there some workaround for this problem?

Next thing I ran into has to do with the overall color and brightness.  Even if I turn the scanlines off, if HLSL is turned on (just curvature/pincushion) the whole screen gets noticeably darker (an undocumented tinted plexi effect?).  I'll try to attach shots below.  Darker shot is HLSL on (but with scanlines off), brighter shot is HLSL off - both using same brightness/contrast settings.  Does anyone else get this? or know what setting is the main cause of this?  I'm trying to counter the effect by boosting brightness/contrast settings, but if there is a better way I'd like to know.

[nox771]

I've made some progress here.  There seems to be some kind of internal data corruption on the shadow_mask_texture variable.  I've been looking through the source but I haven't located the problem yet. 

My original build was MAME143, 64bit on Win7, so I first grabbed the latest diff (143u1) and recompiled.  No help there (I expected as much, there was no mention in the changelog).  Then I figured it might be some kind of 64bit/32bit issue in the mingw libraries, so I recompiled as 32bit without the PTR64 flag.  Again no help.  Next I thought it might be some kind of overflow in the data structure, so I reordered the elements (and their initializers) to see if I could alter the behavior.  No change.

I noticed in my testing that the scanlines would show up, but the shadow mask was not there.  So it was apparently corrupted before it even got used.  I made the following edit in the source (in blue) to hardcode the path, then recompiled:

d3dhlsl.c - lines 892-896   
// experimental: load a PNG to use for vector rendering; it is treated
// as a brightness map
emu_file file(window->machine().options().art_path(), OPEN_FLAG_READ);
//shadow_bitmap = render_load_png(file, NULL, options->shadow_mask_texture, NULL, NULL);
shadow_bitmap = render_load_png(file, NULL, "aperture.png", NULL, NULL);

Bingo - shadow mask reappeared.  It still corrupts the shadow_mask_texture variable in the hlsl.ini file every time it exits, but now it doesn't matter since it is hardcoded - this fixed the "black screen hanging on startup" problem I was having.

As far as the overall darkness, I had to crank the gamma way up as well as tweaking contrast.  The two shots below illustrate.  The first one is my reference - HLSL off, brightness/contrast/gamma all set to 1.0.  Second one has the following settings:

brightness                    1.0
contrast                       1.15
gamma                        1.8
shadow_mask_alpha     0.10
shadow_mask_texture  aperture.png
shadow_mask_x_count 320
shadow_mask_y_count 240
shadow_mask_usize     0.093750
shadow_mask_vsize     0.093750
curvature                    0.05
pincushion                   0.05
scanline_alpha             0.25
scanline_size               1.0
scanline_height            0.5
scanline_bright_scale   1.30
scanline_bright_offset  0.6
scanline_jitter              0.0
defocus                       1.5,2.0

I'm still tweaking these.  If I get some time I may try to track down where that corruption is happening.

[Howard_Casto]

I'm no expert on what the hlsl effects are doing, but it's not corruption, that more darkened image is how they are supposed to look. 

The lower dot pitch on the monitor means a bigger "grid", which means more space between the pixels, which means an overall darker image. 

[nox771]

I'm no expert on what the hlsl effects are doing, but it's not corruption, that more darkened image is how they are supposed to look. 

The lower dot pitch on the monitor means a bigger "grid", which means more space between the pixels, which means an overall darker image. 

No, the corruption I'm referring to is not the image itself, but the shadow_mask_texture variable inside MAME (the one that holds the filename of the shadow mask).  Somewhere the variable itself or the hlsl_options structure is getting corrupted, which causes the shadow_mask not to load, and subsequent game restarts to crash (this is due to it writing the corrupted variable out to the .ini file and then attempting to read it back in on the next restart).

The darkness of the image is not due to the scanlines or shadow mask itself.  If you refer to my top post, the darker image has no scanlines in it at all.  It is ONLY doing pincushion distortion.  However the whole image is darker as if the whole thing were pushed through an alpha filter.  I'm not sure the source of this darkness, but it can be somewhat corrected via brightness/contrast/gamma settings.

[Howard_Casto]

Well just because the original hardware had a darkened image from the dotpitch doesn't mean that is how they are replicating the effect.  They might have just put a darken filter in there as it was easier.  

I have no doubt that the effects are still buggy though... better report the bit about the shadow mask.  


Update:  I can confirm this bug.... happened to me a few minutes ago playing xmvsf.

[scofthe7seas]

Have you tried this? :

Change

hlslini                   %g

To

hlslini                   

[nox771]

I forget the exact tests I ran, but at the time I did try numerous tests with all manner of HLSL configuration setups.  None of them could get the problem to go away (except for hardcoding which is not ideal). 

Due to time constraints I haven't been able to study this further, however I just checked the latest MAME release, and I notice now that the 143u5 release notes mention flags that have been added to control the HLSL ini file reading/writing behavior.  Those new flags might be an effective workaround for the corruption bug.

[scofthe7seas]

I first updated to u5. It was when I first started playing with the hlsl, and hadn't looked at before. I kept getting freezes, and crashes and thought something was wrong with my hardware. Recompiled back to u4, and no more problems. I have noticed with some games, at least the way I have the hlsl effects set up, it cuts off some of the screen sometimes. Not ideal.

[nox771]

I have noticed with some games, at least the way I have the hlsl effects set up, it cuts off some of the screen sometimes. Not ideal.

I've seen this also, but I haven't done enough testing to write any kind of definitive bug report.  When it does work it looks pretty good, but overall it still looks pretty buggy.

[Hunk_4TH]

Can anyone share their recommended hlsl settings for 143u5? I know this is a trail and error thing. I can't find a whole lot on which settings are recommended except for here http://forums.bannister.org/ubbthreads.php?ubb=showflat&Number=73239&page=1

[Hunk_4TH]

Can anyone share their recommended hlsl settings for 143u5? I know this is a trial and error thing. I can't find a whole lot on which settings are recommended except for here http://forums.bannister.org/ubbthreads.php?ubb=showflat&Number=73239&page=1

[nox771]

I think it really is trial and error, and the ideal settings really have to be set on a per game basis IMO.  The ones I found to be most important were the ones I listed in the 2nd post, but I haven't tried 143u5 yet so I don't know if the overall effect given by those settings has changed.  I found one in particular, defocus, can't be set too low otherwise the effect looks too hard edged and artificial.  Also make sure the shadow mask is working, I think scanlines + shadow mask looks better than scanlines alone.

Here are a couple more links:
HLSL MiniFAQ
MAME HLSL Effects

[Hunk_4TH]

Thanks for the reply and links! I'm sure hlsl will mature a lot more in the next releases.

[scofthe7seas]

Hey nox, I like your setting for adjusting the brightness. I found I needed it after adding a bit of convergence shift (probably more authentic that it would be a bit darker, but I wanted to see, so..) but I also found that if you leave contrast at 1.0 you get a better picture (in my opinion). Messing with the contrast washed out a lot of the color and details, merging what would normally be a fine gradient into pools of similar colors. I was taking pictures with my cell phone to try and find the closest color accuracy and brightness levels, and found the 1.8 gamma to be good for brightness, but the contrast was turned back down to 1.0

[nox771]

Yes I agree, it is best to leave contrast at 1.0.  However on my monitor (quite a few years old), I could not get the picture to match up in detail unless I tweaked the contrast a little bit.  IIRC, the problem I was having with respect to the sample image was darker areas turning completely black - for example, undercarriage of the car in the first pic of first post, the detail in that area is completely gone.  In any event, if you do get snapshots with different settings I'd be interested to see those.