Google Carpenter bee traps. You can buy them or make your own. As the owner of a log cabin in an area with plenty of carpenter bees, I've found these to be the only effective solution. Key is to place them in areas where bees tend to drill- if there are previous infestations, puff delta dust (or pump WD40 lol) in the holes-which are up to a foot long and always take a 90 degree turn from the opening. Once the queen/larvae are eradicated, plug the hole and place a trap there. Best done at the end of summer/early Fall. Next Spring when they return, you got em'. It's effing glorious. If you plug the hole and leave anything alive in there, the woodpeckers will come and peck out the whole channel, requiring repair of the wood. I made all these mistakes years ago not knowing the nature of it.