It is the overnight lows.
The cold overnight temps keep the the plants dormant longer than the city itself. The urban heat island is particularly effected by the moist air.
I imagine that NYC will increasingly see longer growing seasons and increased sub-tropical plantings