Your are out of luck unfortuantely.  
Removing drivers doesn't increase speed in any way.  I suppose you MIGHT get a little boost in terms of the time it takes mame to load, but that's about it. 
Think of it this way.  Let's say you have a bunch of pc games on your computer and one of them is running slow.  Does removing the other games make it run any faster?