Well I couldn't wait so I pulled it here before work, turns out I was right. 
I cleaned them off when they where out but the mark remains so ideas are welcome.
That is pretty minor wear, and wear is expected to occur over use. You should see how rollers look after a couple years in an arcade (& before they were stainless). Check out these two links for pics of worn rollers:
Bob Roberts,
TB repair.
FWIW, the roller wears the fastest brand new, when the ball is resting on only one point. Once the rollers have a little wear, though, the ball is resting on a curved line, spreading the force over more area, and the speed of the wear drops very quickly.
The thing that's getting me is the idle roller (the diagonal one without a encoder wheel) looks to have more wear than the other two. If it is getting more wear, the ball might be loose and moving so it rubs on more than once spot on the roller. (This might be supported by the sound only when to the right and not to the left.) Or possibly you got a mismatched set of rollers, with the idle roller from a more used set. Is that just the pictures, or do it actually have more wear?