Alright, I purchased some "electronics cleaner" from Radio Shack, and used this to clean out the bearings. This was what someone suggested on a different thread. After using this on them, they all seemed to spin much better than ever before. I then used some 3-in-1 oil in each bearing. This slowed them down a bit, from when there was nothing in them, but they still spinned pretty well. And in the long run, I am sure having oil in there is best. I spun them on a drill for a few minutes each as well. Basically I just put a pencil and tightened it in my drill. Then I put the bearing on the tip of the pencil. I held onto the bearing, and turned the drill on. So anyway, I did this for a few minutes each. I put my newly cleaned, newly oiled, freshly spun bearings back into my trackball. It still does not spin correctly. When I give it a good push upwards, the cursor still goes up straight for a moment, and then drifts to the upper left. If I spin it downward, it drifts to the lower right. As shown in my picture that I originally attached. However the drift is much much much more severe than what I've portrayed in my picture.
I feel like throwing this trackball out the window...

So, I now have two entire trackballs, complete with bearings and encoder wheels and everything. I also have two more additional encoder wheels, and 6 extra bearings. With all of these items, I cannot get one single working trackball. No matter what combination of bearings, wheels etc, I use, I can't get it to roll right. The drift ruins everything. It makes games like Golden Tee and any bowling game virtually unplayable. I rarely, if ever, can get a straight drive/roll. I've tried using both trackball cases (with their included optical sensor things). Both do the same thing, so I know it's not the sensors that are the problem.
What am I doing wrong here guys? Do your trackballs go completely straight up and stay straight when you give them a good push? Should I just give up here? I am at a complete loss.