Yes, the climate has fluctuated over geological time. Scientists know this.
The difference is a) the rate at which it is warming and b) the fact that all living things have adapted to a completely different climate.
Climate change means mass famines and suffering, mass disruptions in commerce as whole industries implode, increased conflicts over dwindling resources, and mass extinction. It's a big deal.
As far as agriculture, crops have evolved to certain soil types and climactic conditions. For examples, yield rates are going down for grains like wheat due to warmer nights, and the soil in Iowa in 2012 was too hot for seeding corn! It won't be a simple as moving the crops northward as the world gets hotter: soil types and ecological conditions are important for crops too.
As far as mass famine and suffering: whole countries will be wiped off the map. That means refugees. It means fewer resources for more people.
As far as more more violence associated with climate change: see dwindling resources and refugees. The US Department of Defense predicts that there will be armed conflicts over desertifying regions, especially Africa, which experience some of the worst climate change [1]. The USDOD is not made up of hippies.
As far as mass extinction: it's already happening. Due to CO2, the ocean is 30% more acidic than it was in the 70s, and the shell fish industry on the wet coast is dying [2]. (Shellfish are particularly sensitive to ocean acidficiation.) This is just the beginning, and the great coral reefs of the world are dying too.[3]
Basically, in my view, climate change is the single most consequential event in human history, and its scope will extend beyond humans too. The aggregate pain and suffering and the unimaginable loss will have a profound effect on the human psyche.
If anyone reading this is concerned, I urge them to join 350.org [4], which is a climate change activist organization. 350 is named after NASA scientist James Hansen's estimate for the upper bound of acceptable CO2 PPM for humanity to thrive in. We passed 400PPM in 2012; humanity evolved in the upper 200's. James Hansen was head of NASA's Goddard Institute before he retired to organize against climate change full time [5].
350.org's main initiative at the moment is divestment: getting major educational institutions to divest from fossil fuels as a social statement. To sign the petition for your school, go to http://gofossilfree.org/ [6] .
The climate change movement desperately needs conscientious, scientifically aware individuals to stand up!
[1] http://www.nytimes.com/2009/08/09/science/earth/09climate.ht...
[2] http://apps.seattletimes.com/reports/sea-change/2013/sep/11/...
[3] http://www.scientificamerican.com/article/ocean-acidificatio...
[4] http://350.org
[5] http://en.wikipedia.org/wiki/James_Hansen#Research_and_publi...
[6] http://gofossilfree.org/