More or less. The games are the actual code that was dumped (read: copied) by someone directly from the ROM chips on the boards inside the original cabinets. For this reason, they're called ROMs. So you feed a ROM into MAME and MAME emulates the hardware that the ROM originally ran on. To the ROM it looks like it is running on original equipment.
So yeah, you need MAME and you need the ROMs. However, the ROMs, with a few exceptions that almost certainly do not apply to you, are illegal to possess. They aren't especially difficult to obtain, but asking for ROMs here, or where to get them, isn't allowed here.
By the way, if you're just going to play a few games on your computer, I recommend MAMEUI. Regular MAME is run from a command prompt, e.g. , "mame.exe pacman.zip". MAMEUI has a built in graphical menu (the UI stands for user interface) that allows you to easily navigate and start your games. If you ever graduate to building an arcade cabinet you'll want to use regular MAME in conjunction with a frontend like MaLa or Hyperspin, which are two of many user interfaces made to look good on an arcade monitor or TV. Also, unlike MAMEUI, the frontends are also specifically designed to be controlled with authentic joysticks and buttons, rather than with a keyboard and mouse.