Exactly! Heck there were reports of asphalt being cracked and foundations suffering catastrophic damage. If you're going to use "improper building" as a reason to withhold an EF5 rating then basically the only chance of an EF5 rating is on the coastline in a hurricane zone.
Regardless, I've seen an EF4 up close and it's damage. I'm not sure how you can get much worse than that anyways, it's beyond anything you reasonably even expect.