Found it, removed it and soldered on a remotely located 2 AA holder that'll make changing the batteries easier in the future.
The only issue with the jukebox is a noticeably loud constant HUMMMM or BZZZZZZZZZ coming from the main power supply. Is this something I can fix? It's quite annoying at lower volumes.