Yeah understanding the strengths/weaknesses of models should certainly be factored in. One example of this is with forecasting temperatures and using MOS. We see this very often during the spring months, but MOS tends to struggle mightily when it comes to highly anomalous patterns. When we see an anomalously warm airmass in the spring, MOS can underdo temperatures badly. See this often in the heart of the country too...sometimes MOS will miss in places by like 10-15F
can't argue that