I want the light to be inactive when the switch is turned off and active when turned on.
I'd guess the bulb is wired between the tabs on either the left or right side of the switch.
If it stays on all the time, then it must be wired on the power cord side that gets power all the time.
You don't want to use that side for the power inlet then. You want to use the other side that only has power when the switch is flipped.
So.....
Swap the blue wires on the back of the switch. Then swap the yellow wires.
(or disconnect the switch, remove it from the housing, flip it over, put it back in, and wire it as pictured.)
If it doesn't work the way you want after that, then the switch just isn't wired in a way to do what you want.
I ripped the bulb out of mine.
