@MonMotha; Thanks for the in-depth explanation, but I can't quite figure it out.
What I could really do with is a real-time modeline generator that lets you test resolutions on the fly. That way trial and error isn't reduced to restarting the computer every time I alter a modeline slightly. I'll give powerstrip a go, but I've heard it's quite advanced.
xvidtune will do just that on Linux. No idea if there's something similar for Windows. Windows has historically been pretty limited in terms of the ability to use exact video timings, hence why Soft15k is fairly driver dependent.
Or even better, does anybody have 50hz modelines for any or all of these resolutions:
321x240
321x256
640x240
640x256
Any reason you need these at 50Hz? They'll be a fair bit lower than 15kHz (in the neighborhood of 12.5-13.5kHz), and some monitors may not like them. They'll also not be what the game needs to run in, so it will run slow or you'll have to use e.g. triple buffering to convert the framerate down. Are you trying to use them with a European TV via SCART or something? I'm told most European TVs will happily accept 60Hz timings.
Anyway, here's some GTF-derived calculated modelines for the resolutions you provided at 50Hz:
# 321x240 @ 50.00 Hz (GTF) hsync: 12.40 kHz; pclk: 4.17 MHz
Modeline "321x240_50.00" 4.17 321 305 328 336 240 241 244 248 -HSync -Vsync
# 321x256 @ 50.00 Hz (GTF) hsync: 13.20 kHz; pclk: 4.65 MHz
Modeline "321x256_50.00" 4.65 321 305 336 352 256 257 260 264 -HSync -Vsync
# 640x240 @ 50.00 Hz (GTF) hsync: 12.40 kHz; pclk: 8.33 MHz
Modeline "640x240_50.00" 8.33 640 600 656 672 240 241 244 248 -HSync -Vsync
# 640x256 @ 50.00 Hz (GTF) hsync: 13.20 kHz; pclk: 9.08 MHz
Modeline "640x256_50.00" 9.08 640 608 664 688 256 257 260 264 -HSync -Vsync
The 321xNNN modes are altered slightly because the VESA specs disallow odd horizontal active pixel counts. You may experience compatibility issues.
Note that the horizontal scanrates are quite low. I suspect most monitors won't work with these. There's somewhat of a way around this, but you'll probably end up with black bars at the top and bottom of the picture, and most TVs don't have an easy way to adjust it back out.
The dot clock is also quite low, and some video cards may not be able to do it.