Think two lists: general & local (aka "this game"). This lets mame keep the defaults for all games even when editting specific games.
The general list is every input, the local is just the inputs included in the game. Usually the local list has everything set to "default", which means "look in the general list" (in pseudo code, of course). However, if the game driver sets the input to something else, then that local starting setting isn't "default" but whatever the driver sets. The general list defaults are also set in the source code, but there's only one for all games.
The ctrlr file changes the general list. The ctrlr file is read once for top to bottom, and any relevant changes are added in the order they are read. (This is different than the first ctrlr files, which had the priority you listed.)
Then the default.cfg changes the general list. (aka overriding the ctrlr file)
Finally the gamespecific.cfg changes the local list. (Almost always overriding the ctrlr and default.cfg files, see next paragraph for the exception)
The weird case is with driver set game specific inputs. For cfg files, only inputs set to something different from the starting settings are listed. In this case the starting input is not "default", but since it's the starting input, it's not included in the cfg file. However, since it's not "default", mame uses it instead of the general setting. To use the general setting, set that input to "default", which will be saved in the cfg file; sort of "un-overriding" the general list. And the only way to find them is to look at the source AFAIK.
Ignoring them, it's pretty easy.