The USB specification allows for +5v at 500ma *per port*. The only way to insure that everything inside the panel gets enough power would be to place a powered hub in the panel... so you would need to run an additional power line to the CP.
(I was considering doing the same thing)