If the amplifier's power cord is plugged into the "AMPLIFIER" socket on the power transformer assembly, try moving it to the "SERVICE" outlet instead. Rowe wires the amplifier outlet to be on all the time the machine is plugged into a wall outlet regardless of whether the machine's power switch is on or off.
The service outlet is controlled by the power switch. Now when you switch off the jukebox's power switch, the amplifier will also shut off and hopefully your noise will go away.