Well I took the trackball apart and from a tip from Andy, I swapped the x and y bearings.  Plus I put a small drop of 3-in-1 oil in each of the 5 ball bearings.  While the ball definitely is rolling smoother, it still is a little slow when tracking up now.  I'll give Andy a shout and see if I can get another shaft... maybe that shaft just isn't making good contact.