With six buttons per player, something's gotta give. N64 controller had 10 buttons, analog stick, and a d-pad. So if you map your stick to the analog, and your start button to start, that still leaves 9 buttons and 4 d-pad directions but only 6 action buttons.
Several games you can play fine, but for example tetrisphere (a favorite) uses the d-pad which I didn't map to anythiing.
Mario Kart, Wave Race, etc. are fine.