I'm not sure if the OP is understanding the help being given. I'll repeat what severdhed has said, but in a little different (technical) view, and a partial solution that will mostly work in mame/mess but no other app (AFAIK).
There are four different 'UP's: your up (user's up), the joystick's up, mame's up, and the game's up. Usually the four match, and all is well, so usually they can be veiwed as the same. There are exceptions, though. One is Qbert cab, where the joystick was rotated 45 degrees closckwise. What is up-right diagonal to you, is just plain up to the joystick, mame, and the game. Another is the OPs CP: what you think is up, the joystick disagrees. When you map your up into mame, mame sees the joystick's 'up-left' diagonal should be 'up', and sends the game up if and only if the joystick's up-left diagonal is sent.
Normally, a diagonal is when two ordinal directions are pressed at the same time. Again, there are the four different up-left diagonals: user, joystick, mame, and game. (From now on, uUp = user's Up, jUp = joystick's Up, mUp = mame's Up, & gUp = game's Up, u(UpLeft) = user Up-Left diagonal, etc) When the joystick's diags are mapped to mame's ordinals, mame's diagonals aren't possible. As mapped, mUp = j(UpLeft) and mRight = j(UpRight). So m(UpRight) = mUp + mRight = j(UpLeft) + j(UpRight), so would need jUp, jLeft, & jRight to all be pressed at the same time, and jLeft cannot be pressed at the same time as jRight.
This can be useful in rare conditions. Again, let's look at Qbert. If you only have a normal 8-way joystick, you can map j(UpRight) to m(Up), j(DownRight) to m(Right), ect, and it will work quite well.
However, in the OPs case, more fancy mapping needs to be done to get game diagonals. Mame can map key combos, multiple keys, and nots. Combo: j(UpRight), aka jUp + jRight pressed at same time, mapped to mUp means mame sees up only if both are pressed at same time. Multiple: jUp or mouseUp mapped to mUp means mame seees up if mouse goes up
or joystick up (either one, or both). Not: jUp Not jRight Not jLeft mapped to mUp = no diags on a normal joystick. Depending if uUp is j(UpLeft) or j(UpRight):
if uUp = j(UpLeft)
mUp = (jUp not jRight) or (jLeft not jDown)
mLeft = (jDown not jRight) or (jLeft not jUp)
mDown = (jDown not jLeft) or (jRight not jUp)
mRight = (jUp not jLeft) or (jRight not jDown)
if uUp = j(UpRight)
mUp = (jUp not jLeft) or (jRight not jDown) ...
etcIt's the same in actions in mapping in mame which ever way the stick is rotated: tab, Input (general), player 3/4, go to Up, enter, press u(UpLeft) then quickly u(downLeft) twice, wait. (pressing the same key twice makes it a not see
mame FAQ for more info.) Mame now should say "w not a" or something like that. Press enter again, press u(UpRight) then quickly u(DownRight) twice. You should have something like "w not a or d not s" (assuming wasd as keyboard directions). If you don't get the nots, or accidentally get a j(diagonal), move off of Up, go back to up and redo. Repeat for the other directions, do near diagonal then twice press the far diagonal on the same side, other near and twice press the remaining diag.
This should work okay, but some people find it doesn't work as well as it should.
ever1458, could you list the exact keys joysticks 3/4 and the directions they correspond to?
(Not sure, but it's possible that the joystick is not rotated and the OP expects it to be.)