How to dampen button "clack" sound (not from switch)

[Martoon]

I got some Sanwa buttons (OBSF-30), and I love them (very responsive, with a really light touch).  The only problem is they make a rather loud "clack" noise when I tap them.  The switch in the button doesn't make any noise (it doesn't click).  It's the plastic buttons themselves.

Right now, I have the buttons mounted in a temporary control panel that's just a piece of 3/4" MDF with a couple support boards holding it up, sitting on my desktop.  I'm sure the clacking will be a little quieter when the bottom of the control panel is enclosed, but it could still be pretty loud.

Does anyone have any tips for how to mount these buttons in a way to dampen the "clacky" noise?  I know the MDF panel is acting as a sounding board to amplify the sound.  Would another material for the panel be better?  Or would there be some way to mount the buttons to dampen it?  Maybe a rubber washer or sleeve around the button (between it and the panel)?