So one one ground and one power for the button and one ground and one power for the led. Four "connections" per one button?
Yes, for a single-color LED button like this you'll have 4 wires.
- Encoder input (microswitch NO tab) -- single wire to encoder
- Encoder ground (microswitch COM tab) -- usually daisy-chained
- LED power (one "stirrup" tab) -- usually daisy-chained
- LED ground (other "stirrup" tab) -- usually daisy-chained

I usually put the LED ground wire on the side with the large black tab. (bottom of the LED socket in this pic)

You may need to turn the LED 180 degrees so the polarity is correct.
PROTIP: Check LED polarity
before you install/wire the socket using the diode setting on your multimeter. LED will light dimly when the red lead is on the positive and black lead is on the negative.

Scott
P.S. How many LEDs are you using and where are you getting the power for them? Computer PSU Molex?