Depends on how the converter works. The answer is generally "probably not". While it would be possible to take specific steps when designing the upconverter to make it work, I've not found any that do (there are some valid reasons not to).
I wouldn't be surprised if someone has a device targeted specifically to this market to make it work, though.
You could use a non-scaling RGB to S-Video or YPbPr component adapter with a CRT television. Since that doesn't alter the resolution or timing, it should generally work as long as the TV doesn't lag (use a plain ol' analog SDTV to ensure this).