Honestly, I don't think there's much of a reason to use AVR at all for USB these days aside from the availability of legacy code. It's quite an underpowered architecture by modern standards, though it was great in its heyday. You can even get 5V ARMs, now, though 3.3V I/O is pretty well established at this point.
Modern ARMs are so fast that you can keep up with USB's timing requirements on high-speed (dozens of MHz) SPI I/O expanders without introducing any delays (sample on SOF and have the data ready in the EP buffer by the time you get your IN token), so you don't even have to worry about pin count if you want lots of discrete I/O. Using I/O expanders would also free up the native MCU pins to use the MCU's onboard ADC. Of course, you can also pretty trivially get ARMs in 144-pin QFP which are usually going to give you all the I/O you need.
There's certainly no reason to go about using multiple AVRs, I don't think. If you do want to use an AVR, you might look into the larger 90USB646/647/1286/1287. Not only are those 64-pin packages, so you start with more I/O, they have a true, native parallel bus on them. I actually have an arcade I/O board using one of those micros (a 646) plus a bunch of 74 series buffers for zero-wait-state access to the entirety of JAMMA + kick inputs and something like 32 discrete outputs, too. I would certainly consider it a legacy design, at this point, and would use an ARM if I were doing it now, though.
BTW, you might be aware that JVS uses USB cables, but it is not USB. JVS is actually RS-485 UART. The protocol spoken on top of the byte-framed UART is somewhat loosely defined, and I've never managed to get good, authoritative documentation on it. There seems to be a reason why it's difficult or impossible to substitute a JVS I/O board and game PCB from one vendor to another which may be part of why it didn't catch on as much as JAMMA.
Oh, I also noticed that you want to do two players worth of USB slave. This may complicate things a bit. You can find ARMs that have two USB controllers, but they tend to be on the large size. I've seen some -M4s though, I think. The easiest method would be to use a USB composite device and expose two HID interfaces OR to just do a HID composite device and have separate usages for player 1 and player 2. This won't let you plug them into separate USB ports, though, which may be problematic if you intend to use them with consoles (do any modern consoles realistically let you use "non-approved" HID devices?)