Trackball size varied by game. For example, Missle Command was a big trackball (about 4 or 4.5 inches), while Centipede was about 3", or maybe even a little smaller. I got the 3" because it seemed about right for most games. I love Centipede, and I just couldn't bring myself to play that with a 4.5" trackball.
It's just a matter of personal preference.