Yep, it will take +some+ of the paint off, but what you're really doing is filling in the minute grain pores and other imperfections in the wood.
You paint, the sand down a little, then paint again, which fills in more pores, and sand again, repeat, repeat.
That spray on primer filler helps because it's so thick, you shouldn't have to put on near as many coats before you get a flawless finish that you can really paint.