Hi Rattus,
It sounds like you've created all modes interlaced for some reason (maybe because you asked vmmaker to rotate the modes?):
Switchres: Searching for custom video modes...
Switchres: [ 0] 640x 480 @ 30i* : AMD ADL timing "640x480_30i 15.685000KHz 59.980881Hz" 13.050000 640 664 728 832 480 483 489 523 interlace -hsync -vsync
Switchres: [ 1] 664x 496 @ 60i : AMD ADL timing "664x496_60i 16.169000KHz 59.996288Hz" 14.100000 664 696 760 872 496 499 505 539 interlace -hsync -vsync
Switchres: [ 2] 768x 496 @ 60i : AMD ADL timing "768x496_60i 16.170000KHz 60.000000Hz" 16.170000 768 800 872 1000 496 499 505 539 interlace -hsync -vsync
Switchres: [ 3] 800x 496 @ 60i : AMD ADL timing "800x496_60i 16.173000KHz 60.011131Hz" 16.950000 800 832 912 1048 496 499 505 539 interlace -hsync -vsync
Switchres: Found 4 custom of 3 active video modes
A bit below, all 4 modes are shown as locked. This can be caused because you have manually disabled interlace, so GM can't select the only modes that exist.
So, start by creating a reasonable mode list. Run vmmaker again, and make sure you get some progressive modes shown in vmmaker's list. Then try again (and post a new log if things aren't still ok).