Download & unzip the b5 zipfile, and then download the b8 zipfile and extract into the same folder (overwriting mamewah.exe etc.)
Next run Mamewah and it will create some subfolders to get you started. It sounds like you need to take a look at the latest config.txt - it should tell you the file structure for layouts among other things. I expect you just need to rename your .lay file to layout.lay.