If I read that right, you want to be able to play either Thor or Merlin on the left stick, and either Thyra or Questor on the right, but never more than two simultaneous players. If that's the case, you can configure the inputs in MAME for Gauntlet to use the same controls. You'd have to be careful never to start Thor & Merlin or Thyra and Questor at the same time, though, as then (I think) you'd get both characters running around, and sooner or later one of them would stick behind a wall and get you stuck.
On the other hand, have you looked at the cabinet plans for Gauntlet at
www.jakobud.com? The Gauntlet CP is really pretty small, compared to most 4p setups:
It's about 30" wide, by about 17" deep. For comparison, a Teenage Mutant CP is about 37" wide. Of course, you still need standing room for p1 &p4, so it may still be two big, but it's worth considering. You might be able to shrink it down a little more, if it's close. A cardboard mockup is your friend here.
Good luck!