-15 degrees is the coldest I've ever shot in (-35 wind chill)...I stayed outside for about 4 hours and then packed up. The side of my rifle was completely frozen from me breathing on it. I really, really do not like shooting in areas that are high wind and dust. I don't mind the wind, but when the wind is sand-blasting you and your gear, it can be a bit annoying. However, I will still go out when it's windy/dusty...the only thing that keeps me from going out is if the roads are closed (which is rare).
Like Dirty D said above, I like it to be on the cold and/or windy side...only the dedicated guys come out, and if the tacticool AR dudes come out, they stay out for about 30 minutes and pack up and go home when they are out of ZombieMax ammo.