Well I'll update my situation for those with similar monitor / situation to me.
I spent the entire weekend trying to learn as much as possible and read the official guides many times.. I know my way around vmmaker, arcadeosd and gm settings much better now. Also I found QMC2 frontend to be a really great gm-friendly alternative to MameUI
At first I did create my own monitor profile which seemed to work ok.. I got hold of the service manual and cobbled together a configuration based on the timings table.. though it had three completely different sets of timings based on 3 different resolutions, so I had no idea which set to use.. but somehow it still worked.
I set the ranges to avoid really low resolutions as they were not playing nice.
I was using super resolutions/mode generation to set up about 30 or so games, but I soon found that too many modes were over-riding my monitors internal geometry memory.. I'd align the screen for one game and it would throw out the settings for others etc. It wasn't limited to vertical amplitude either, horizontal shifting & amplitude also affected, sometimes extreme. A temporary fix was to add new doubled modes but I soon ran into same issue. Then I tried forcing tweaked modelines which worked for a few games, though most were too geometrically different to fix this way. Then finally when attempting add another new mode I got an overlapping label error and couldn't load the list into ram with vmmaker. I also could not delete the culprit mode because winmodelines doesn't work properly with windows 10
So basically I started from scratch, using standard crt 70/120 monitor config in case my custom one was bad.. and will try static modes only. I'll use CRU to add detailed resolutions and extended EDID modes, as for some reason the geometry settings are remembered for all of them doing it this way. When no slots are left I'll add more static modes with vmmaker. All is working well so far.
I think the super-resolutions approach is more suited to 15khz monitors and 31/96khz are better to stick with static modes for geometry memory reasons?
Anyway loving GM.. especially the frame_delay option