I don't mind subtitles. I don't even notice them after the first minute or two. And when I think back about a movie I watched with subtitles my brain has literally edited out the foreign language audio track and replaced it with English. I remember the people speaking English, in their correct voices, and the subs aren't there. Most people I've mentioned this to who watch foreign films say the exact same thing happens to them.
One potential problem with subtitles, though, is they usually have to be dumbed down in order to be read quickly enough. I watched the dubbed version of Crouching Tiger Hidden Dragon after seeing the subbed version probably three or four times, and I was surprised at how much detail was missing from the subbed version. However, as usual, the awful voice work made the dubbed version nearly unwatchable, so I won't be abandoning subs any time soon. The one exception to this was District B13, which by the way is one of the baddest ass action films you have ever or will ever see. They talk too fast in that movie and keeping up with the subs is practically impossible, and certainly makes it impossible to see anything else going on on-screen. My wife and I rented it and after about ten or fifteen minutes we started it over and watched it dubbed.