Ah, maybe that's the issue then, isn't the D9800 actually a digital monitor? Could that be why it can do the range like it does and possibly how it's using the height/width to figure things out?
I'm almost sure that being digital must affect the results in some way, however your monitor seems to have an incredible range of tolerance. There's something I've been wondering, is if your monitor has some kind of autoadjust mechanism, so that when you set a resolution, it tries to autocenter, as pc monitors do. I suppose it's something necessary for multisync monitors, although I don't really have a proper understanding of what's going on from an electronic point of view. About height/width managing, I'd say LDC panels do recognize xres when fed from an analog source, so it's possible that this technology is present in newer CRTs.
It doesn't autoadjust like a normal computer monitor, at least not from what I can tell it doesn't do that. I do know that it's a definitely 100% fix though just having that 1 line less for the resolutions with 288 active 312 lines total, when it was 313 it could kick into more of the wide EGA type mode but it never does that now that there are 312 lines total.
So for us it would be great that hfreq was the value used by the monitor for making decisions, but maybe it's not so simple. Some months ago I tried to make modelines for a crt TV with a cheap digital chassis. Some resolutions were distorted for some reason. The same resolutions did work when removing some lines, keeping vfreq, others didn't. Things went definitely odd around 57Hz video modes, regardless the number of lines. I tried to find a rule for it (porches, hfreq, vfreq, vertical total,...) but couldn't, it seemed random, although consistent.
The lines seem to be a factor, but also from the slight change in vertical timing values used in that upper range near where 312+ lines occur it seems to keep it acting right. I get the feeling it might have been the lines, but also think it might still be the hfreq because it's interesting that it's right at 19Khz where that extra line pushes it over the edge and it's inconsistent there and above. It's almost at that point for the way a game like pacman turns out, we are right in the middle between CGA and EGA mode and if it's not the 312 line thing it might be the 19Khz point or a combination of both and how they interact and some other electronic variable from that combination.
Fortunately I think this is most likely the only type of monitor we have to worry about this on, and luckily we seem to have also possibly worked around it just with the extra range there at that in between point. It does seem to follow a pattern, there's only one missing from the ranges inside the EGA range but seems to be a 2Khz range outside each of the normal ranges from VGA->SVGA and CGA->EGA modes where it needs different timing values vertically.
# D9800/D9400
#
# CGA
$MonitorModes[scalar(@MonitorModes)] =
fill_mode("15250-18000,40-80,2.187,4.688,6.719,0.190,0.191,1.018,0,0",
$MonitorModes[scalar(@MonitorModes)]);
# CGA->EGA
$MonitorModes[scalar(@MonitorModes)] =
fill_mode("18001-20000,40-80,2.187,4.688,6.719,0.140,0.191,0.950,0,0",
$MonitorModes[scalar(@MonitorModes)]);
# EGA
$MonitorModes[scalar(@MonitorModes)] =
fill_mode("20001-29000,40-80,2.910,3.000,4.440,0.451,0.164,1.048,0,0",
$MonitorModes[scalar(@MonitorModes)]);
# VGA
$MonitorModes[scalar(@MonitorModes)] =
fill_mode("29001-32000,40-80,0.636,3.813,1.906,0.318,0.064,1.048,0,0",
$MonitorModes[scalar(@MonitorModes)]);
# VGA->SVGA
$MonitorModes[scalar(@MonitorModes)] =
fill_mode("32001-34000,40-80,0.636,3.813,1.906,0.020,0.106,0.607,0,0",
$MonitorModes[scalar(@MonitorModes)]);
# SVGA
$MonitorModes[scalar(@MonitorModes)] =
fill_mode("34001-38900,40-80,1.000,3.200,2.200,0.020,0.106,0.607,1,1",
$MonitorModes[scalar(@MonitorModes)]);
Sounds great, looking forward to that, will be nice to get this stuff easy to use for anybody, mainly the kernel patch and details on xorg.conf config
Also I've just gotten it so it'll override those 'invisible' default modes. Turns out I can add a modeline of 800x600 with xrandr, I just can't label it such, it must be '800x600xSOMETHING_EXTRA' basically can't be just '800x600'. Must be some odd bug in X, but doesn't matter because now I'm adding the xREFRESH as a string to each modeline generated for xrandr to use and now we can make those modes too. Basically the way I'm doing it, to avoid any conflicts with my desktop mode, is I have a single modeline in xorg.conf as 641x480, then everything generated by switchres, even 640x480 modes, won't conflict with my desktop and we can use them even if a different refresh rate is needed. I check and basically use that default desktop mode already in before launching switchres as a default in case no mode is specified by the game, but also align it to 8 removing the extra 1 pixel. So a user with a generated desktop resolution of 641x480 or 801x600 doesn't have any worries about conflicting issues, switchres should take care of all the details. I was always adding 8 pixels onto the horizontal width before this change, which now the focus is on the game being perfect and letting the desktop have 1 funny pixel extra. Also if a person does use 640x480 on the desktop then it'll revert back to adding 8 pixels to the width to make up for it. In all the calculations it always aligns it to 8 and I could try to force a single pixel there in those cases but seems unclean compared to just expecting the user to do that themselves for the desktop resolution.
I'm happy that old default modes annoyance has been overcome, I see it's convenient to use that 641x480 for the desktop so we make sure it's not mistaken with the others, it's clean.
VeS has started with the distribution. It's very likely we'll be harassing you with questions in the next days.
Sounds great, I just uploaded a new version yet with a ton of overhauls of how things are done, making it must easier to handle the modeline etc. Actually I think I fixed it so that Soft15Khz and AcradeVGA Windows support should work a lot 'better'. Basically now it should properly choose the right resolution, write the modeline to the file if specified to store them in, run the game with the exact resolution from the input resolutions file and also do all the correct checking for different command line args to mame for the difference between the original resolution. In the overhaul I just did, I broke apart the main body where I had a big if statement around stuff before and was able to reorder things much more logically and that was an natural outcome of the change. I had not been doing that check for the input resolutions early enough, before modeline generation, and in Windows do the command line processing for mame after all that and not try to use a new resolution that isn't in that original resolutions file (and write out to the new resolutions file the more precise to that game modeline generated by our engine).
I also got the way the monitor modes are filled and stored to be a bit more robust, I'm still not sure though what to add for lines parameters for each monitor range yet. Also I'm kind of seeing something odd sometimes in the logic of the virtualize function because of how it was built for interlace and I think it doesn't work well outside of that usage. When I try bigger resolutions outside the d9800 limits and remove my max parameters for width/height then the virtualize function can give me back some really wild resolutions, way too large and I need to look deeper into how that's done and also how to deal with when interlacing isn't set to enabled and it can't use interlace.