I'm reviving this old post because I've got some news.
I tried to change my previous video card (Radeon HD3xxx) with an ATI Radeon HD4550, to get rid of the "low pixel clock" limitation, and to see if that new video card could help with my flickering issue.
After running VMMaker, it generated 119 modelines and each and every interlaced modeline still flickers, as it did before. When it flickers it has an odd number of total vertical lines (e.g. 521 for 480x464i), if I increase that to 522 (via ArcadeOSD) it looks great (almost no flicker at all).
Of course, as explained above, SwitchRes recalculates the modeline on the fly so every game which triggers an interlaced resolution is still unplayable on my machine.
For now, I'm just preventing the resolution switch for vertical games (via vertical.ini) and playing them at 640x480i (which is my desktop resolution that I tweaked via arcade osd) but they look terrible - the stretch algorithm produces artifacts that make, for example, the text barely readable. Is there maybe some mame.ini configuration needed to improve the stretched image?
At this point, I'm suspecting it's my arcade monitor the main reason of this flickering issue so I'm here to ask: is it still true that there's no SwitchRes/Mame setting that I can use to work around this? I'm also starting to consider a recompilation of the GroovyMAME source after hacking the modeline generation algorithm, in order to generate an even total number of lines for interlaced resolutions.
PS: I'm using Windows XP. Would it make sense to try upgrading to Win7 to see if it solves the issue?