Yeah, you'll want to use a wall-wart (an AC to DC transformer that plugs into the wall) for this. Snip off the end, bare the wires, and solder them to the battery terminals.
Just make sure you get one that has enough balls to power whatever you have in mind, or you'll cook your wall-wart.