How many modes are you adding to the EDID emulation? I ask because modes in the EDID emulation cannot be altered. Because EDID emulation is so helpful, you need it on, but adding extra modes will lock them and their timings, so you cannot adjust them.
After that, can you try altering the timings in ArcadeOSD - which is doing nothing here - then put those values into the range lines in mame.cfg, and see if game load up with adjusted modes?
The next step might be to redo your VMM config with different horizontal timings and see if that helps.