I think the reality is, if everyone pulled a permit for everything that needed to be done, nobody would ever get anything done.
I mean, my parents put up a pool 10 feet away from wetlands and it has sold 3 times since without any issues lol.
Obviously, any footprint changes, you should probably pull a permit