I've been kind of racking my brain on this one and I can't work out how to do it short of doing it by hand. I thought about using a hand planer or file but those would result in an uneven edge and shred the edge anyways. It needs to be cut.
I'd like to trim off an edge (because it's too deep and uneven). In the really bad sketch below, A shows a cross section, the red portion is what I want cut off. The gray/brown areas are what I want to keep. B shows the edge if you were looking straight on. It's curved.

I fiddled with a router, but since it's not on a flat surface, it's a little on the wobbly side and most likely not the brightest thing to be doing with a high speed tool.
So... any ideas? Is there a bit or tool that can let me use the brown surface as a reference and cut the gray area?