This is how I remember them but never payed attention to wether they moved or not, you would think that would be something I would notice!
Well when you played in the arcade those were probably metal CP's. On those the dust washers hardly move. Just look at how little it moves when it's laying on the joystick base (and you move the stick). Imagine putting 3/4" of wood between the dustwasher and the joystick base. The dustwasher will be further up the shaft and thus move a lot more. You will really see it move when you put it on top of a wooden CP.
Especially with the classics the artwork is right around the joystick. If you put the dustwashers up on the CP you will cover up the artwork. For instance in this prototype I made a few years ago:

The dustwasher would cover the green arrows and the word Move.
BTW I made my CP's a 2 layer design. I liked Oscars solution, but didn't feel like leaving the top layer secured only by buttons. I wanted to be able to remove the whole top layer without breaking the CP overlay and i didn't feel like routing out the entire CP top to fit the plexiglass in. So what I did was make it two complete layers. A top of 9 mm thick multi and under that a 12mm layer of multiplex. I flush mount the joysticks from the top of the 12mm layer (routed in). The 9mm layer has t-nuts through it so the screws bond the two layers with the joystick in between. This makes it look like the joystick is mounted in a 9mm panel from the top (it allows you to hide the dustwaser under the top layer) yet it has the strength of a full thickness piece of wood. Guess you could make the top layer thinner too, but I need some thickness for the t-nuts.
I used this approach mostly because of the mspac galaga reunion stick which otherwise needs a metal plate to be able to mount them, but I felt it works great for other sticks too.
In the prototype shown above I originally planned to cover it with a bent piece of lexan or plexi, but couldn't get the bent properly (so I ended up with an ugly cut in the overlay). In a new version I simply made the thinner layer of wood go on the top and front of a square box.