Right, I've found the problem, actually two faults. You may call it a feature but I call it a bug

When you use the normal 'out of the box' layout (i.e. CTRL LEFT/RIGHT changes emulators) you can specify in the GUI config the location of the layout folder, in my case it was c:\arcade\mala\layouts. If you leave it blank it uses the mala folder where the executable is.
However, when you enable the tree it always looks in the mala folder and nowhere else, even if you modify the malatree.ini and update the layout folder ini entry.
The problem though, is when you define the tree it looks in the root and presents you with the layouts it finds in the mala root folder fine HOWEVER when mala starts up in tree view it looks in the layout folder and because your tree layouts are not defined there you get nothing.
Second, if you start malatree.exe directly (in the same way you start up the gamelist and layout editor), it wipes everything you've previously created and starts from a blank tree.
I couldn't send this to the author as there are no contacts on the website.
In summary:
1. You cannot change the tree vew layout folder or have it use the normal layout folder you specifiy
2. If you do have a different layout folder for normal layouts tree view fails to find your layouts
3. running malatree.exe directly wipes all your tree information