You can always add layers for the cold, up to a point. There is only so much you can take off for the heat.
Here on the frozen tundra of Maine, hot is 85°F. If it gets much above that I don't ride. Luckily a nice summer day in Maine the brings 75 to 80°F for the temps. On the down side, summer is short. In a similar thread it was pointed out that cold brings snow, ice, salt, sand, etc. This is a greater impediment to riding than just the cold.
So, I guess I'll take the cold, down to about 35°.