So I checked this out. It's kinda weird. Using the svgawin drivers, I find that with vsync on (which is on by default), there's tearing and stutters. Smooth with it off.
In SDL video mode, it does some kind of line doubling/scale thing. So for, say, DK: at magnify 1 (which is default in advmame setting), running horizontally, it automatically doubles the horizontal resolution (adv says 400x256; monitor says 1280x1024@85hz...my desktop is at 60hz...I don't know why advmame switches to 85hz), but simply runs a 31khz-like res if run vertically (adv says 224x256). On my multisync at XGA, magnify 1 is slightly fuzzy like mame is by default; however, it's slightly different on my PC monitor at SXGA - neither fuzzy, nor overly defined. Hmm.
If you change to magnify 2 (which is the setting it chooses for each game if you set it to 'auto'), it's double the original output and sharpens the graphics similar to RGBsharp in old mame, or prescale 2 in new mame. Hm. Higher magnifications/resolutions may work, though I've gotten some pretty weird behavior.
Scanline effects behave somewhat the same, though with some changes in brightness/gamma some games can look damn good. DK, or any of the black background games. Others just look as bad as before. You can't use them on magnify 1, of course, because the resolution is too low. However, above magnify 2, and you get weird behavior, so it seems magnify 2 + scan2horz is best for traditional scanline effects.
Vector games, for example Tempest, look best with display_intensity set to 3 (default is 1.5).
Svgawin drivers still don't work with my X800, and there doesn't seem to be any improvement in running native, so there's no need for me to use this version. Ah well.