Oh man, that is close to a pounding there...but even if it isn't, the drive to Hadley from there takes you across Rt 2 in MA, right? I'd have to imagine it's not going to be the easiest travel in those spots around Gardner/Winchendon.
Only saving grace is the heaviest precip might be before that...so perhaps 4pm isn't quite as bad as it would be around midday or sometime in the morning.