Here’s some info on how things work:
There is a difference between the Test mode on the config app and LEDBlinky running from HS – The config app lights all the mapped ports. The LEDBlinky GamePlay animation (running from HS) will only animate the buttons that are inactive while the active buttons will remain solid on. Active buttons are determined by the currently loaded emulator/game. For mame, active buttons are identified by the controls.ini and mame.xml files. But for other emulators, the active buttons are only identified by data from the LEDBlinky Controls editor.
If you turn on the Debug Log, run HS and a few games, then email me the debug.ZIP file I'll look it over and get back to you.