I think many people confuse carpenter bees and bumble bees. The former is a problem, the latter not in my experience. Carpenter bees themselves don’t cause troublesome damage. It’s not much to plug a hole, sand it and stain it. The problem as @CAPE mentioned is if woodpeckers catch on. Then the damage can be big. I’ve heard but never tried a fake owl. I’ve read they will keep woodpeckers at bay. Again I don’t know that. I have seen carpenter bee traps that catch them by the dozens. Whatever you try, if you can rid yourself of them you probably need to.