How often, though? I am not assuming, but many people experience that they thought they needed 4 players before their build, only to find that 95% of their time, they only really have 1-2 people playing. A compromise I have seen done is have a couple wireless controllers that can be switched on for player 3 and 4. Your know what is best for your needs, but my suggestion is to build for the scenario that will fit your need for 80% of the time.