You still need an amp. Potentiometers won't amplify the signal for you.
If the sound is amped, then is it possible you connected the wires wrong? POTs have 3 prongs (usually). Maybe you're using the wrong two?
The way a potentiometer works is you have two leads separated by the resistive material, and one lead that works as a 'tap' to pick off the voltage from an arbitrary point on the resistive element. So if you have two leads connected, there are only two options. Option 1: connection to the input and output (not using the tap). This will function as a normal resistor, and the knob won't do anything so long as the tap is floating and unconnected. Option 2: Connection to either the input+tap or the output+tap. In either case, the potentiometer will function, but the direction you turn the knob to ramp up the resistance changes dependent upon which way you have connected it.
So you do have your pot connected correctly, since you are hearing a change with turning the knob. If I were you I would follow Ray B's advice and look into picking up a small amp. Of course, if you do have an amp hooked up, maybe you got a potentiometer that is way too big in resistance.. that would explain a small operating range as well.
Where did you decide to mount your volume knob anyway? I'm looking for ideas on where to put one myself.. it would sure beat running around to the back of the cabinet to adjust it from game to game!

Let us know how it turns out..
-88mph