It's called a drill guide, and many companies make them. The range in quality and price, so the one pictured above is not your only option.
Another option, if you have a drill press, is to make a simple guide jig. Get a piece of wood, drill through it with your drill press with the right size bit, then when drilling wood, line the wood your just drilled over the hole you need to make, clamp that wood over your hole, and drill by hand. It isn't 100%, and the jig won't last too many uses, but it can work in a pinch if you are worried about drilling a crooked hole.