I was able to find a solution.
First of all i used this user_modes.ini
## Desktop ##
640 x 480 @ 60.000000 desktop
320 x 240 @ 59.014000 p64
321 x 240 @ 59.014000 am2r
640 x 240 @ 59.014000 am2r
## Super resolutions ##
2560 x 240 @ 60.000000 super
2560 x 248 @ 60.000000 super
2560 x 256 @ 60.000000 super
2560 x 264 @ 60.000000 super
2560 x 272 @ 60.000000 super
2560 x 280 @ 60.000000 super
2560 x 288 @ 60.000000 super
2560 x 296 @ 60.000000 super
2560 x 304 @ 60.000000 super
2560 x 320 @ 60.000000 super
2560 x 336 @ 60.000000 super
2560 x 344 @ 60.000000 super
2560 x 352 @ 60.000000 super
2560 x 360 @ 60.000000 super
2560 x 368 @ 60.000000 super
2560 x 376 @ 60.000000 super
2560 x 384 @ 60.000000 super
2560 x 392 @ 60.000000 super
2560 x 400 @ 60.000000 super
2560 x 416 @ 60.000000 super
2560 x 432 @ 60.000000 super
2560 x 448 @ 60.000000 super
2560 x 464 @ 60.000000 super
2560 x 480 @ 60.000000 super
2560 x 496 @ 60.000000 super
2560 x 512 @ 59.000000 super
2560 x 544 @ 55.000000 super
2560 x 560 @ 54.000000 super
2560 x 768 @ 60.000000 super
2560 x 800 @ 60.000000 super
Even though hd 2400 doesnt support it i used dotclock= 4 so i will be able to view the 320x240 resolution on arcade osd. I generated the modes with vmmaker, installed them and then after restarting i opened arcade_osd
From there i ve choosed 320x240 and by pressing "2" i edit the modeline like seen in the picture. The critical is to use a dotclock= 7.12
after testing i saved the new 320x240 custom. Is working great. I tried Am2r at 320x240 , Project 64 and zsnes at 320x240 and 640x240 stretched, and kega fusion at 320x240. Everything works great. Only Am2r needs first to choose from arcade_osd the custom created 320x240 resolution.
Of course, Groovymame works great as it used before for me.