Here is what I suspect, but don't know for a fact....
Obviously each emulator itself has its own requirements, but you aren't talking about that...
My guess is that the more emulators, then the more ROMs to keep track of... The more ROMs, then the more data to manage... Plus each emulator needs its own graphics settings for the "wheel" etc...
Now, to me, it doesn't seem like it *should* be that big of a bog down, but again, I am only speculating anyway.