Hi calm, I have attached some tests i did in the ModelineTest log. Basically anything 16khz, 25khz, and 32khz works.
Thanks for the logs! Curious monitor... it won't accept 15Khz but 16khz??
Try recalculating the modelines with VMMaker adding these lines to vmmaker.ini
   monitor_specs_0 = "16200-18000, 49.50-65.00, 2.000, 4.700, 8.000, 0.064, 0.160, 1.056, 0, 0, 288, 448"
   monitor_specs_1 = "25000-25800, 49.50-65.00, 2.910, 3.000, 4.440, 0.451, 0.164, 1.048, 0, 0, 480, 768"
   monitor_specs_2 = "31000-32200, 49.50-65.00, 0.636, 3.813, 1.906, 0.318, 0.064, 1.048, 0, 0, 576, 768"
... and set MonitorType = "CUSTOM"
Restart...
Then in mame.ini, add the same lines with this format
   monitor_specs_0   16200-18000, 49.50-65.00, 2.000, 4.700, 8.000, 0.064, 0.160, 1.056, 0, 0, 288, 448
   monitor_specs_1   25000-25800, 49.50-65.00, 2.910, 3.000, 4.440, 0.451, 0.164, 1.048, 0, 0, 480, 768
   monitor_specs_2   31000-32200, 49.50-65.00, 0.636, 3.813, 1.906, 0.318, 0.064, 1.048, 0, 0, 576, 768
and set "monitor cga"
(make sure to replace the default "modeline_specs auto" lines otherwise it won't work).
That should produce modelines that are between the ranges your monitor seem's to like.