What kind of material is the panel? If it's wood you could use a hole saw bit the size of the button holes and cut out a piece from scrap glue in the hole. Then wood putty over it and sand it down. I've personally gotten by with just wood filler globbed in there then sanded flat.