Your are out of luck unfortuantely.
Removing drivers doesn't increase speed in any way. I suppose you MIGHT get a little boost in terms of the time it takes mame to load, but that's about it.
Think of it this way. Let's say you have a bunch of pc games on your computer and one of them is running slow. Does removing the other games make it run any faster?