do you also have the same problem when you use your regular pc keyboard, or is the problem only with your joystick?
some more info would be helpful, eg. what joystick is it, how is it connected to your pc, is it usb etc
ps. just as a test try turning off all the sync options in your video settings (eg. triple buffer, vsync...) and see if the problem goes away