Well, the answer was "use the pliers". The button I was working on wouldn't budge, so I decided to try one of the other ones using the twist back-and-forth method and it worked fine, popping out in my hands without much trouble at all.
The first one was still stuck though, so I ended up unscrewing the nut up against the light assembly and cranked on it until the pressure popped the bulb-assembly right out.