I would say the 6 volt adapter will work just fine.
As long as it is regulated.
And you said it is 6.19 with no load attached, I would say it is very well regulated. I have seen non-regulated adapters (I call em "wall warts") run as high as 50% over rated voltage, with no load.
If you are really worried about it, just add a 6 volt zener diode and resistor to clip anything over 6 volts.
Here is a link to calculate the zener and resistor you will need:
http://www.opamplabs.com/zenerreg.htmI guessed the load to be about 20 ma (.020). It's probably a lot less.
with 6.19 volts in, 6 volts out, load of .020, the calculator says you will need
a 19 ohm 1/8 watt resistor with a 6 volt 1/8 watt zener.
should be able to get these at Radio Shack.
But I really dont think .19 volts is going to hurt anything. I cant tell you what to do, but if it were mine, I would try it with just the wall wart.
Good luck, and let us know what you come up with.