50 grams.....je-sus. Whether you stay with that resistance, try different switches as mentioned.
I have experienced the greatest performance with Sanwa flat plunger buttons. Those are half-height. Longer plungers have greater 'sway'. No buttons are built to have straight travel, as that would require tighter tolerance, hence at best a modified design, possibly lube to prevent resistance, or different material(s) that is self-lubing.
None of that make sense, Mike A?