I put you script in the mame.cmd file, at the end of the CPS3....like this, but must have done something wrong as it didn't work.
It's part of an autohotkey script, it gets added to the .ahk script, not the cmd file
:cps3
cd C:\Program Files\AutoHotkey
autohotkey.exe C:\emulators\m2.ahk %1
Looks like the cps3 part of your .cmd script is pointing to a M2 ahk script?
Do you have an autohotkey script that launches cps3?
Can you give me detailed instructions please...I can get a little lost
Sorry, but I don't have the time to spend on it.
Oddly enough, the only free time I have is at work 
and I'm not installing ahk on my work computer.
If you can't figure out how to incorporate it into your current ahk script, or aren't using an ahk script to launch cps3, just copy and past this into notepad:
Esc::SoundSet, +1, , mute ;Mutes
Sleep, 1000
SoundSet, -1, , mute ;Unmutes
Then save it as mute.ahk (all file types)
Run it before you run cps3 and see if it works at all.
Since it doesn't have an exit, you'll have to right click on the autohotkey icon in the system tray or
use task manager to stop the script before making changes to the sleep time.
If you get it working the way you want, create a shortcut to it, then copy and past the shortcut into your start folder.
(google how to find your start folder if you need to)
That will make the script run every time your computer boots up and it will run all the time.
(every time you hit esc, it will mute for a second, no matter what programs are running)
It's the simplest way I can think of without getting into your ahk scripts.
If you are launching cps3 through an ahk script, please post a copy of that.