In a sense, it's great that agriculture is only 3% of GDP. A disaster that would make agriculture much harder globally, would mean that agriculture would become a much _larger_ share of GDP, say 50%. And we would be worse off for it.
However, the studies quoted in the Wikipedia article take such considerations into account.
Measuring climate change impact by GDP is a good idea. It's not the only way to measure it, just one out of a number of useful ways.
For example, it is a useful way to figure out how much of our economic wealth we should be spending on fighting climate change:
US GDP seems to be about 60k USD per capita, or about 20 trillion USD total. Spending up to about 10% of that, or 2 trillion USD per year, on reliably fighting climate change might be worth while by that metric.
It also helps us to put into perspective the cost of programmes like the one talked about in https://news.ycombinator.com/item?id=23770718 and make it comparable to energy efficiency campaigns or carbon sequestration, etc.