Are you using version 1.13 of PSX?
Try running PSX from the command line thus taking Mala out of the equation. From the PSX directory, use something like:
psxfin.exe -f "C:\PLAYSTATION IMAGES\MedalHon.img"
...replacing the path to the images and the name of a game image with yours.
If that works fine in full screen, you need to post screen shots of your Mala configuration for PSX.