Author Topic: Climategate Part Deux: A Tale of Code, Commentary, and Contumely  (Read 12459 times)

roo_ster

  • Kakistocracy--It's What's For Dinner.
  • friend
  • Senior Member
  • ***
  • Posts: 21,225
  • Hoist the black flag, and begin slitting throats
Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« on: November 24, 2009, 09:54:16 PM »
I put this in another thread because it deserves its own chewing over.  Here is the original Climategate thread:
http://www.armedpolitesociety.com/index.php?topic=21990.0



Folks, the emails are just the thin skin of scum on top of the s*** stew that is the AGW Theory Promotion Club.  The real turds are floating in the code & code comments

It is the sort of stuff you just can't make up.  If it were a book plot, you'd be shot down for going over the top and caricaturing the AGWers.

If y'all have read my comments on AGW and, specifically, the modelers and their practice of withholding data & methodology, you might recall I considered their actions unethical and questionable in the strongest terms.  Now, we have their own words (the emails) to damn them as unethical, grasping, lying, and engaging in illegal activities.  The code and code comments put to rest any doubts as to the natures of the AGW researchers (unethical, lying, etc.).

But wait, there's more!

The code comments show that the input data is horse manure unfettered by reliance on data integrity or fidelity.  The code comments also show that the model algorithms are also suspect.  I should back up...the few model algorithms that the coders understand are suspect.  The other algorithms, they have no effing clue about.  They built new versions of the model with algorithms they knew to be fallacious and used as climate input data into the new model the known-incorrect output data from the old model becasue they thought that they lost all the data taken from weather stations in the 1990s.  Or so they think, because they aren't sure just what data is in the input files.

For just a taste of the shinola stew, mash this link:
http://www.devilskitchen.me.uk/2009/11/data-horribilis-harryreadmetxt-file.html
Go ahead.  All of y'all who have done some coding and algorithm development will either be thoroughly entertained or need a recepticle to vomit up your disgust.

It is a digest of a poster named Asimov who is running through a README file.  Here is the link to the thread:
http://www.tickerforum.org/cgi-ticker/akcs-www?post=118625&page=13

I read the digest at devilskitchen.me.uk and had to check to make sure he wasn't making it up so moseyed on over to tickerforum.  Still not believing what these folks were doing with taxpayer monies, I downloaded Nick1911s link on rapidshare and read some of hte thing my own self.

Well, I extracted the README in question and have posted it at:
http://www.filefactory.com/file/a1ea9d1/n/HARRY_READ_ME.txt

Truly, I thought poorly of the AGW whores, but I never thought they would be so thoroughly mendacious and clueless.

The guy writing the README is not the original coder, but a maintainer/developer who is supposed to make it bigger & better.  I can sympathize with the difficulty of working on awfully written, scantily documented code, but I can not excuse his collusion with the AGW whores to keep all this under wraps.

If the story of this can be explained to the general public, there will be prosecutions and people will go to "pound me in the ass" prison.  Also, AGW theory promoters will be laughed out of any professional or academic milieu with any technical sophistication.  If...



My background includes basic physics training (baccalaureate) and ten years as an analyst working with models, data, algorithms, CONOPS, etc. while getting my MBA.  I use many different models to do different things and some of these models are rather large (and surprisingly well-documented).  Weather and atmosphere has always been of interest as it effects my employer's products.  Recently, I have delved into atmo modeling quite a bit more, as we are trying to replicate the physics of atmo in a place for which we did not have blessed data from AMSAA.  Using various tools (incl Modtran) and weather station data, we have come pretty darn close to reality WRT the physics of the atmo of one place on Earth during three months of the year.  I was able to get 6000+ hourly weather station readings (apiece) from several locations nearby.  (An earlier project used weather station readings every 15 minutes for several years to get the atmo right.  THAT atmo model was, in the technical verbiage of My Cousin Vinny, "dead on balls accurate.")

Anyways, to get the work done, you have to make and document assumptions as well as be able to explain and defend methodology.  Also, most contracts require that we give them all data if they want to delve through it.  I can understand the frustration of defending work when some yahoo is tossing rocks, but that is part of the deal.  Put on your big boy pants and man up.  Hiding data, lying, conniving to get your critics fired, and destroying data so folks can't assess your work is not the work of ethical men.
Regards,

roo_ster

“Fallacies do not cease to be fallacies because they become fashions.”
----G.K. Chesterton

RevDisk

  • friend
  • Senior Member
  • ***
  • Posts: 12,633
    • RevDisk.net
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #1 on: November 25, 2009, 01:12:55 AM »
For just a taste of the shinola stew, mash this link:
http://www.devilskitchen.me.uk/2009/11/data-horribilis-harryreadmetxt-file.html
Go ahead.  All of y'all who have done some coding and algorithm development will either be thoroughly entertained or need a recepticle to vomit up your disgust.

I feel mildly ill.  I'm not overly familiar with the whole global warming argument, but if this is a cornerstone of it...   That's very bad.   

Question, why aren't the folks who do current weather models and such in charge of developing the proof of global warming or whatnot?  They seem fairly competent, relatively unbiased and reasonably accurate. 
"Rev, your picture is in my King James Bible, where Paul talks about "inventors of evil."  Yes, I know you'll take that as a compliment."  - Fistful, possibly highest compliment I've ever received.

AZRedhawk44

  • friends
  • Senior Member
  • ***
  • Posts: 13,973
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #2 on: November 25, 2009, 10:10:48 AM »
I went and read the link.

Seems that Ian Harris is attempting to recreate a program, not recreate or falsify data.

I sympathize with that:  I'm re-writing a 4GL app to run as a chain of SQL stored procedures.  And end up with identical data sets to the old app, while not re-using the slow *expletive deleted*ss chain of cursors and looping logic in place.  Documentation for my old program does not exist, and code comments are "sparse" at best.

My methodology is unfortunately similar to his... run against a constant set of source data and  compare the results of the old code versus the new code.

The problem I see here is that he says "Meh" and "good enough" and "eff it, I don't have time" to data analysis that could derail our nation's economy and possibly that of the world.
"But whether the Constitution really be one thing, or another, this much is certain - that it has either authorized such a government as we have had, or has been powerless to prevent it. In either case, it is unfit to exist."
--Lysander Spooner

I reject your authoritah!

MicroBalrog

  • friend
  • Senior Member
  • ***
  • Posts: 14,505
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #3 on: November 25, 2009, 10:13:13 AM »
The problem is, the usual suspects are claiming the comments have been 'cherry-picked' and 'taken out of context'.

I begin to think the people who did ClimateGate did not do the job right in presenting it to the general public.
Destroy The Enemy in Hand-to-Hand Combat.

"...tradition and custom becomes intertwined and are a strong coercion which directs the society upon fixed lines, and strangles liberty. " ~ William Graham Sumner

makattak

  • Dark Lord of the Cis
  • friend
  • Senior Member
  • ***
  • Posts: 13,022
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #4 on: November 25, 2009, 10:17:04 AM »
The problem is, the usual suspects are claiming the comments have been 'cherry-picked' and 'taken out of context'.

I begin to think the people who did ClimateGate did not do the job right in presenting it to the general public.

The response to "cherry-picked" data is to release the full data. (You know, like the information from the FOI requests they've been illegally deleting...) Somehow I doubt the people at the CRU will do that...

Also, I read one theory that whoever did this may be releasing the data over time much like the ACORN scandal.

If that turns out to be the case, I eagerly await the next installment.
I wish the Ring had never come to me. I wish none of this had happened.

So do all who live to see such times. But that is not for them to decide. All we have to decide is what to do with the time that is given to us. There are other forces at work in this world, Frodo, besides the will of evil. Bilbo was meant to find the Ring. In which case, you also were meant to have it. And that is an encouraging thought

MicroBalrog

  • friend
  • Senior Member
  • ***
  • Posts: 14,505
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #5 on: November 25, 2009, 10:20:28 AM »
I do not imply that the East Anglia CRU people are in the right. I am however not sure if this was the right tactic to use.
Destroy The Enemy in Hand-to-Hand Combat.

"...tradition and custom becomes intertwined and are a strong coercion which directs the society upon fixed lines, and strangles liberty. " ~ William Graham Sumner

makattak

  • Dark Lord of the Cis
  • friend
  • Senior Member
  • ***
  • Posts: 13,022
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #6 on: November 25, 2009, 10:28:58 AM »
I do not imply that the East Anglia CRU people are in the right. I am however not sure if this was the right best tactic to use.

Fixed that for you.

It may not have been the best tactic. However, it allows for anonymity if the one who released it is still within the CRU.

However, although it may not have been the best tactic, it will likely be effective enough.

"Sunshine is the best disinfectant." I think enough light has been shed that they will be unable to continue to stall releasing data/emails that are the subject of the FOI request.

As much as the "mainstream" media tries to spin or ignore this, they are no longer the gatekeepers. Truth will win out.
I wish the Ring had never come to me. I wish none of this had happened.

So do all who live to see such times. But that is not for them to decide. All we have to decide is what to do with the time that is given to us. There are other forces at work in this world, Frodo, besides the will of evil. Bilbo was meant to find the Ring. In which case, you also were meant to have it. And that is an encouraging thought

AZRedhawk44

  • friends
  • Senior Member
  • ***
  • Posts: 13,973
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #7 on: November 25, 2009, 11:31:39 AM »
Quote
As much as the "mainstream" media tries to spin or ignore this, they are no longer the gatekeepers. Truth will win out.

Truthiness.  Definitely proves a modern day necessity for a Freedom of Speech that is not dependent in any way on a predication of Freedom of the Press.  Amazing what the innernetz can accomplish.
"But whether the Constitution really be one thing, or another, this much is certain - that it has either authorized such a government as we have had, or has been powerless to prevent it. In either case, it is unfit to exist."
--Lysander Spooner

I reject your authoritah!

CAnnoneer

  • friend
  • Senior Member
  • ***
  • Posts: 2,136
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #8 on: November 25, 2009, 12:27:22 PM »
That is a general problem with issues that become highly political - a cloud of misinformation is laid think onto the public, while the supporters of each camp tend to radicalize.

I have a good friend, who is also a scientist, and he is a most vociferous supporter of global warming theories. For my part, I am extremely skeptical of either view and want to see data that I can comprehend and analyze by myself, precisely because this is so politically charged an issue. In any case, here is a rundown of his arguments:

five years ago:
"Human-caused global warming is a scientific fact established beyond reasonable doubt. Further argument is not serious and can be downright damaging" (so essentially he bought Al Gore's ideas)

this month:
1. "Human-caused global warming was a fifty-fifty toss-up four years ago. Two years ago, it was mostly convincing, because we got more data. Today, with all the accurate data that we have by now, it is a scientific fact established beyond reasonable doubt." (I wish I had taped him 5 years ago)
2. "When I saw Al Gore coming out with his movie, I felt deeply saddened, because he did a disservice to the cause by associating himself with it. Then I knew people would reject the conclusions just because they are coming out of him." (so, the theory is right, but people reject it because of Gore)
3."It was a toss-up before, because people had sparsely distributed data - great data for a few locations for many years, but little data for most locations and only for a few years. Now we know better because we have a huge distributed array of data collection through satellites."

Just think about #3 for a minute. So, people out there make sweeping conclusions based on long-term data from only a few locations throughout the world, while the richest data is only a few years old! How does that make any sense mathematically?

In any case, I just wanted to share this to show that even smart, educated people, even with technical specialties close enough to the problems (he works at JPL), can get so far taken by the politics and emotionality that they completely lose touch. How much easier would it be for the majority of the public, who has to rely predominantly on the media for their information, and at least in part, for their conclusions.

PTK

  • friends
  • Senior Member
  • ***
  • Posts: 4,318
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #9 on: November 25, 2009, 12:41:05 PM »
This whole thing hurts my brain.
"Only lucky people grow old." - Frederick L.
September 1915 - August 2008

"If you really do have cancer "this time", then this is your own fault. Like the little boy who cried wolf."

GigaBuist

  • friends
  • Senior Member
  • ***
  • Posts: 4,345
    • http://www.justinbuist.org/blog/
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #10 on: November 25, 2009, 11:10:54 PM »
I'm skeptical of man-made global warming but the latest news isn't really solidifying that stance.

Without having actually viewed the code, and I am a programmer, I've seen two complaints:

1)  Sections where nobody knows what it does.

That's not really damning.  There's still a function somewhere in ID's source, I think for Quake, that nobody remembers what it does.  It's needed but nobody remembers how or why it got there.  Sounds similar.

2)  A "fudge factor" to make the data line up.

Well, Newton had to do that too.  It was something like 70+ years before anybody could actually measure the gravitational constant he used in his calculations and verify his "fudge" number was correct.

Waitone

  • friend
  • Senior Member
  • ***
  • Posts: 3,133
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #11 on: November 26, 2009, 08:32:38 AM »
I suspect alternative media will flap around for a short while trying to measure the scope of the problem.  Big Media will ignore it.  The elites who have staked everything on GW will try to get something on the books before the Great Unwashed wises up.  At some point someone will chirp, "Follow the money" and that will be when the real revelations will commence. 
"Men, it has been well said, think in herds. It will be seen that they go mad in herds, while they only recover their senses slowly, and one by one."
- Charles Mackay, Scottish journalist, circa 1841

"Our society is run by insane people for insane objectives. I think we're being run by maniacs for maniacal ends and I think I'm liable to be put away as insane for expressing that. That's what's insane about it." - John Lennon

cassandra and sara's daddy

  • friends
  • Senior Member
  • ***
  • Posts: 20,781
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #12 on: November 26, 2009, 10:40:35 AM »
did gore dump his carbon credits stock?
It is much more powerful to seek Truth for one's self.  Seeing and hearing that others seem to have found it can be a motivation.  With me, I was drawn because of much error and bad judgment on my part. Confronting one's own errors and bad judgment is a very life altering situation.  Confronting the errors and bad judgment of others is usually hypocrisy.


by someone older and wiser than I

roo_ster

  • Kakistocracy--It's What's For Dinner.
  • friend
  • Senior Member
  • ***
  • Posts: 21,225
  • Hoist the black flag, and begin slitting throats
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #13 on: November 27, 2009, 04:36:02 PM »
I guess I'm not the only one with a little analysis & SW experience to think this fishy.

There's this other fellow by the name of "esr*" who is somewhat worked up by what is contained in the code & comments.  Here is a digest of his remarks with links:
http://rebootcongress.blogspot.com/2009/11/eric-s-raymond-on-east-anglia-crus.html

http://esr.ibiblio.org/?p=1431
http://esr.ibiblio.org/?p=1436
http://esr.ibiblio.org/?p=1447
http://esr.ibiblio.org/?p=1465
http://esr.ibiblio.org/?p=1473

Lots of good, chewy information to gawk at, but the following kinda rolls it up in a nice with a bow on top:
Quote from: Commenter to esr and esr's reply
#  krygny Says:
November 25th, 2009 at 8:02 am

Wait just a second. Explain this to me like I’m 12. They didn’t even bother to fudge the data? They hard-coded a hockey stick carrier right into the program?!!

ESR says: Yes. Yes, that’s exactly what they did.


* Playing it cute here.  I am familiar with esr.
Regards,

roo_ster

“Fallacies do not cease to be fallacies because they become fashions.”
----G.K. Chesterton

MicroBalrog

  • friend
  • Senior Member
  • ***
  • Posts: 14,505
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #14 on: November 27, 2009, 04:43:21 PM »
There appears to be a massive gap between how people who already opposed the idea of AGW perceive this, and how the MSM perceives this in terms of importance. Have people failed at getting the message out, or is it going to gain momentum?
Destroy The Enemy in Hand-to-Hand Combat.

"...tradition and custom becomes intertwined and are a strong coercion which directs the society upon fixed lines, and strangles liberty. " ~ William Graham Sumner

Waitone

  • friend
  • Senior Member
  • ***
  • Posts: 3,133
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #15 on: November 27, 2009, 05:18:56 PM »
Quote
Wait just a second. Explain this to me like I’m 12. They didn’t even bother to fudge the data? They hard-coded a hockey stick carrier right into the program?!!

ESR says: Yes. Yes, that’s exactly what they did.
Sorta make me think they got the idea in their heads they would not be challenged.  Wonder why heavy hitters in the scientific community would think such.
"Men, it has been well said, think in herds. It will be seen that they go mad in herds, while they only recover their senses slowly, and one by one."
- Charles Mackay, Scottish journalist, circa 1841

"Our society is run by insane people for insane objectives. I think we're being run by maniacs for maniacal ends and I think I'm liable to be put away as insane for expressing that. That's what's insane about it." - John Lennon

roo_ster

  • Kakistocracy--It's What's For Dinner.
  • friend
  • Senior Member
  • ***
  • Posts: 21,225
  • Hoist the black flag, and begin slitting throats
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #16 on: November 27, 2009, 05:26:54 PM »
Sorta make me think they got the idea in their heads they would not be challenged.  Wonder why heavy hitters in the scientific community would think such.

What have ben two of the most potent motivators since history began?  Almighty Gawd* and the almighty dollar.




* I think you are making the mistake of considering these folks scientists and what they are doing as science.  Think of them as priests in the service of a higher purpose who are jealously guarding their Holy Writ from being defiled by unbelieving Deniers.  Actual science requires attention to detail when colecting data, documenting methodologies and assumptions and then disseminating it for others to replicate or disprove.
Regards,

roo_ster

“Fallacies do not cease to be fallacies because they become fashions.”
----G.K. Chesterton

roo_ster

  • Kakistocracy--It's What's For Dinner.
  • friend
  • Senior Member
  • ***
  • Posts: 21,225
  • Hoist the black flag, and begin slitting throats
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #17 on: November 27, 2009, 05:36:38 PM »
Waitone:

Here is a more thorough explanation by esr that may or may not make sense to you:

Quote from: esr comment in http://esr.ibiblio.org/?p=1447
#  esr Says:
November 27th, 2009 at 8:19 am

>Eric, you’ve already accused them of “blatant data-cooking” based on a rather perfunctory code review,

The code says what it says. It’s not my job to produce evidence that they disseminated it; given their willingness to destroy primary datasets, their bullying of skeptics, and their contempt for basic standards of scientific conduct, we’d be idiots to give them the benefit of the doubt about it.

>Then at least admit you leveled charges of criminal conspiracy without performing due diligence on the available evidence.

Look, try to keep up, will you? The accusation of criminal conspiracy isn’t on the basis of this code, it’s based on their planning to destroy primary datasets rather than allow them to fall into the hands of skeptics wielding FOIA requests. That is a criminal violation of both the U.S. and U.K. versions of FOIA. It’s a separate issue from what the code demonstrates about their willingness to cook data, and it’s a reason they shouldn’t get the benefit of the doubt about how the code was disseminated.

>You might also, for the sake of credibility, try to show us some daylight between your position and that of, say, James Inhofe

OK. If Inhofe actually believes that the entire scientific community is embroled in monolithic AGW conspiracy, he’s an idiot; I agree with that. What I believe is actually going on is a lot more complicated and ambiguous than that. There are a lot of players in this dance. I’ll round up a few….

First, the scientists. Most are caught up in, or struggling against, an error cascade of humongous proportions. What’s an error cascade? Somenbody gave one of the type examples upthread, over the mass of the electron. This is not conspiracy, it’s a result of a tendency to use seniority or authority as a shortcut when it’s technically difficult to evaluate evidence and socially difficult to be skeptical. All humans do this, even scientists.

Next, the Gaianists – term I made up for people in whom “Save the Earth!” has psychologically substituted for traditional religion (in more or less chiliastic forms). They mean well, they really do; they recycle as an act of virtue, they worry about composting and buying local produce — and they’re totally subject to being manipulated by the other players, which is important since most of the action is going on in democracies. They’re not usually manipulated directly by the scientists, except occasionally a very wealthy one (er, think dot.com millionaire) might get hit up for funding. The Gaianists aren’t a conspiracy; they’re not organized enough. There’s some overlap with the scientists at the non-chiliastic end of this group.

Next, the green-shirts. These are political hacks of all varieties who just love the ideas of more carbon taxes, more regulation, and the general expansion of state power, especially if they can posture as virtuous eco-saviors while they’re arranging this. They’re not a conspiracy either, just a bunch of careerists who compete for the Gaianists as a voting bloc. They sometimes behave a bit like a conspiracy, but only because their behavioral incentives tend to push them all in the same direction. Er, they’re not scientists. They’re Al Gore, or they’d like to be, only with political power too.

Any conspiracies in sight? Yes, actually…

Conspiracy #1: Most of the environmental movement is composed of innocent Gaianists, but not all of it. There’s a hard core that’s sort of a zombie remnant of Soviet psyops. Their goals are political: trash capitalism, resurrect socialism from the dustbin of history. They’re actually more like what I have elsewhere called a prospiracy, having lost their proper conspiratorial armature when KGB Department V folded up in 1992. There aren’t a lot of them, but they’re very, very good at co-opting others and they drive the Gaianists like sheep. I don’t think there’s significant overlap with the scientists here; the zombies are concentrated in universities, all right, but mostly in the humanities and grievance-studies departments.

Conspiracy #2: The hockey team itself. Read the emails. Small, tight-knit, cooperating through covert channels, very focused on destroying its enemies, using false fronts like realclimate.org. There’s your classic conspiracy profile.

My model of what’s been going on is basically this: The hockey team starts an error cascade that sweeps up a lot of scientists. The AGW meme awakens chiliastic emotional responses in a lot of Gaianists. The zombies and the green-shirts grab onto that quasi-religious wave as a political strategem (the difference is that the zombies actively want to trash capitalism, while the green-shirts just want to hobble and milk it). Pro-AGW scientists get more funding from the green-shirts within governments, which reinforces the error cascade — it’s easier not to question when your grant money would be at risk for doing so. After a few times around this cycle, the hockey team notices it’s riding a tiger and starts on the criminal-conspiracy stuff so it will never have to risk getting off.

Overall, is this conspiracy? No. Mostly it’s just people responding to short-term incentives, unaware that they’re caught up in an error cascade and/or being politically *expletive deleted*ed around. Nobody involved is what you could reasonably call evil…well, except for the zombies. It would be pretty evil if the hockey team had planned all this, but I’m not cynical enough to believe that. Not yet, anyway, but I haven’t read all the emails either.

Back to Inhofe: he wants us to think the zombies did it (which is half the reason I included them in the taxonomy) but I don’t buy that either. They’ve certainly had a major contributing role in the feedback loop, but they don’t run the scientists (I don’t think and certainly hope not) and weren’t responsible for the error cascade.

Is that enough daylight for you?

And as for “miraculous psychological antibodies”: I know what junk science looks and smells like, having seen and correctly diagnosed several previous outbreaks. And I have one quality which you may interpret as virtue or not, however you wish: contrarian stubbornness. When I see that the emperor has no clothes, it is close to psychologically impossible for me not to yell “Naked!” at the top of my lungs.
Regards,

roo_ster

“Fallacies do not cease to be fallacies because they become fashions.”
----G.K. Chesterton

roo_ster

  • Kakistocracy--It's What's For Dinner.
  • friend
  • Senior Member
  • ***
  • Posts: 21,225
  • Hoist the black flag, and begin slitting throats
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #18 on: November 27, 2009, 05:43:42 PM »
There appears to be a massive gap between how people who already opposed the idea of AGW perceive this, and how the MSM perceives this in terms of importance. Have people failed at getting the message out, or is it going to gain momentum?

Well the MSM is composed mostly of folks for whom AGW is a tenet of faith.  For them to report on it, they'll have to be dragged, kicking & screaming.  Hopefully the net can shorten the process and bring heat on them.  FTR, I can find NOTHING about it in my local news rag, the Dallas Morning News.

Think about, say, a R Catholic periodical that is composed of believers in RC doctrine & tradition.  Think they were the very first to break the RC priest gay pedophilia scandal?  No, it was the Boston Herald, a mostly atheist & progressive journal with an axe to grind against the RC Church.

From the day the scandal broke, how long was it before the RC Bishop, Cardinal, & Pope were willing to address it as fact and not mere axe-grinding?  Years.
Regards,

roo_ster

“Fallacies do not cease to be fallacies because they become fashions.”
----G.K. Chesterton

Waitone

  • friend
  • Senior Member
  • ***
  • Posts: 3,133
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #19 on: November 27, 2009, 07:41:26 PM »
jfruser,  very helpful explaining the dynamics at the worker bee and supporting infrastructure level.  What remains to be seen and exposed is the rat lines into the worlds of politics, finance, and industry.  Teams in these worlds call their plays a little more carefully than to rely on an enthusiastic group of "scientists" (thank you for drawing a distinction I had forgotten).  Galactic sums of money are in play here and those who engage in such games are not inclined to trust the outcome to chance or the good intentions of a loyal civil servant. 

No, there is a lot more to the story which will hopefully come out.  More about the linkages amongst industry, finance, and the ever present politician.  Years ago I worked for an international chemicals conglomerate during the time the environmentalist and some scientists "discovered" CFC's hurt the ozone layer.  After much ado the manufacturer of CFC's at the time agreed to phase it out all in the name of protecting the ozone layer.  Curiously the company had a replacement ready to go.  Environmentalists consider it to be a signature victory for their cause.  What actually happened was quite different.  Fundamental patents on CFC's were set to expire and hence, major new players were ready to enter the market with significant capacity.  I won't regale you with the brilliance of the market play at this point but let's just say what the company pulled off was within the capability of only a few company in the world.  What made the play work was a nexus of academia, environmentalism, government, business, and finance.  The same cast of characters playing around in (man-caused) global warming.

I shut up now.  There's plenty of opportunity to comment in the near future.
"Men, it has been well said, think in herds. It will be seen that they go mad in herds, while they only recover their senses slowly, and one by one."
- Charles Mackay, Scottish journalist, circa 1841

"Our society is run by insane people for insane objectives. I think we're being run by maniacs for maniacal ends and I think I'm liable to be put away as insane for expressing that. That's what's insane about it." - John Lennon

Hawkmoon

  • friend
  • Senior Member
  • ***
  • Posts: 27,273
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #20 on: November 27, 2009, 09:05:56 PM »
From the day the scandal broke, how long was it before the RC Bishop, Cardinal, & Pope were willing to address it as fact and not mere axe-grinding?  Years.

True ... but "decades" would have been a more accurate term.
- - - - - - - - - - - - -
100% Politically Incorrect by Design

Zardozimo Oprah Bannedalas

  • Webley Juggler
  • friend
  • Senior Member
  • ***
  • Posts: 4,415
  • All I got is a fistful of shekels
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #21 on: November 29, 2009, 01:46:44 PM »
A few interesting snippets from the ipcc-tar-master file:
Quote
IPCC WGI THIRD ASSESSMENT REPORT – (TAR)
GOVERNMENT/EXPERT REVIEW – APRIL-JUNE 2000
COLLATED EXPERT COMMENTS

....

General Comments
The idea that climate without human intervention can only undergo “natural variability”, and that “climate change” can only result from human activity is false and fallacious. It is in conflict with all that we know of evolution and geology. It is simply wrong to assume that “ climate change” automatically implies human influence on the climate.
This fallacy is embraced by the Framework Convention on Climate Change, but the IPCC (Footnote to “Summary for Policymakers. Page 1) claim that they are prepared to accept “natural variability” as “climate change”. They are, however, unwilling to accept the truth, which is that climate can change without human intervention.

This fallacy renders worthless several conclusions of the Report, notably, that  “there has been a discernible human influence on the climate”. The surface temperature rise, however “unprecedented” could be “natural”, and the entire Chapter  12 “Detection of Climate Change and Attribution of Causes”  should be rejected as based on a false premise.

An additional fallacy adopted by the IPCC is the claim that it is possible to “attribute”  cause and effect from a correlation.  Science can  assign probability levels of  the likelihood of a cause and effect relationship, but it can never attribute a cause and effect relationship with certainty.  This Report does not assign any quantitative probability levels to correlations.

The statistical treatment of data, and of model simulation is inadequate throughout the Report. The conventional use of 95% confidence limits for estimates is followed only for surface temperatures, but ignored elsewhere, where a single standard deviation (60% confidence) is preferred, or no indication of the level of confidence is stated. For example, in Figure 2.11 (Chapter 2, page 101) showing ocean heat, error bars are only one standard error. They should be doubled.

Linear regression is used throughout the Report without the necessary checks for linearity and autocorrelation corrections.

Models are particularly lacking in adequate statistical information of uncertainty or correlation. It is just not good enough to use qualitative and subjective estimates of “consistency”. “Goodness of fit” should be quantified. Projections from carbon cycle and climate models never have the necessary uncertainty information, and are therefore wholly unreliable.

I would like to suggest that the whole Report is checked over by a specialist in the statistical treatment of data and of  correlation procedures with a view to more scientific and  uniform treatment of data, and the overall provision of the standard quantitative measures of correlation and accuracy.

There is undue emphasis throughout on the importance of surface temperature trends measured by weather stations. Chapter 2 has 10 diagrams showing these data, but only one thoroughly confusing diagram (Figure 2.12) for other methods of global temperature measurement.  The fact that satellite and weather balloon measurements in the lower troposphere do not show a warming for the past 21 years suggests strongly that the surface data are influenced by proximity to human habitation, rather than by greenhouse warming. There is insufficient attention paid to the evidence that this is so, which is

  • A quadrupling of human population and increases in  human prosperity in the last century have led to a great increase in buildings, fuel consumption and vehicles in the vicinity of weather stations.
    Weather stations do not take any precautions against these influences
    Many remote weather stations do not show a warming
    Much recent warming has been from a rise in the minimum temperature rather than the maximum
    Recent warming took place mainly in cold climates, in winter, and at night
    Two thirds of the weather stations operating in 1975, mainly rural, have been closed down
    Many scientific studies have identified “urbanization” effects, but these have been underestimated, because “rural” stations are assumed to be free of such effects.
    Vegetation growing around stations usually increases,  but is rarely reduced.
    Airports have made a transition from  “rural” to “heat island”
    Surface temperature compilations make inadequate corrections for urban effects
A human influence on climate  from these effects is highly likely. An influence from emissions of greenhouse gases is yet to be established.

The treatment of scenarios is confusing. There is only one  reference (in Chapter 13) to a document describing the SRES set of IPCC scenarios  and incomplete information is spread around several Chapters. There is a whole Chapter 13 on “Scenario Development” which does  mention the IPCC  SRES scenario as authored by Nakicenovic, and “under review” but gives no further details. . Several Chapters give model results from the obsolete IS92a scenario, thereby “selecting” it, in defiance of the IPCC claim that they do not favour one scenario over another. There is evidence in Chapter 4 (in Table 4.12a) that IS92a has been revised; the earlier version being referred to as “SAR” As IS92a made exaggerated assumptions of climate and social parameters for the years 1990 to 2000, this is to be welcomed, but these revisions presumably were not  used in the various reported models, and it seems that the revisions were only made for  1990-2000, whereas they should have been made throughout.

There are two different versions of the SRES scenarios in different Chapters. Chapter 4 has A1, A2, B1, B2 only, whereas the rest have A1B, A1T, A1F1, A2. B1 and B2.

The assumed atmospheric concentration figures for carbon dioxide  for the SRES scenarios were included in the First Draft, but have now been deleted. Presumably you are ashamed to admit such absurd figures. Figures for all the other gases are given in Chapter 4 including ridiculously exaggerated figures for future methane concentrations.

Chapter 13 needs to be rewritten to include explanatory material on  the IPCC scenarios.

The scenarios are merely the personal opinions of their creators, who seem uninterested in  procedures for checking whether any of the scenarios agree with past or future trends.
Recent  unwelcome changes in greenhouse gases are ignored. Carbon dioxide emissions from combustion of fossil fuels have fallen for the years 1997 and 1998. The rate of increase of  atmospheric methane has steadily decreased over the past 15 years, to a current value of zero, yet you persist in projecting increases.

The Report should be a review of published work, There is too high a proportion of references to papers that are “submitted”, “in the press”. or “in preparation”. Chapter 12 has 21% of such papers. How can you expect us to comment on material we cannot consult?

47 out of 91 models listed in Chapter 9 assume that carbon dioxide in the atmosphere is increasing at the rate of 1% a year when the measured rate of increase, for the past 33 years, has been 0.4% a year. The assumption of false figures in models in order to boost future projections is  fraudulent. What other figures are falsely exaggerated in the same way?

The use of questions as headings to paragraphs (particularly in the Summary for Policymakers and Chapter 2) is inappropriate for a supposedly scientific document, and gives the impression of a public relations exercise.
Vincent Gray, Climate Consultant, New Zealand, (Exp.)

....

General Comments
In the previous Expert Review, I noted that Chapter 2 (Observed Climate Variability and Change) was, in general, a very high quality product; I did not review other chapters at that time.

While it is still the best chapter in the WG1 document, I find that the quality of  Chapter 2 has declined some, with comments detailed below.  By extension, the rest of the document suffers greatly.  The Policymakers Summary contains several rhetorically alarming statements that are sure to misused in the hue and cry that will result from the leaking of this document at a critical time in the American election cycle.  Some of the SRES scenarios  are simply politically correct fantasies that are logically inconsistent.

Ironically, the likely answer on the sensitivity of climate change has already been determined, unless IPCC wants to argue that the functional form of almost every climate model is wrong (I choose my words carefully here).  What no one knows is how technology will ultimately evolve.  But what we DO know is that any assumption of similar technology over the course of the next several centuries flies in the face of history.

Note that the Technical Summary and the SPM are reviewed after the Chapters.
Patrick  J. Michaels, University of  Virginia, USA, (Exp.)

....

General Comments

Executive Summary
This individual review paper focuses on IPCC's three most essential modelling and core parameter errors. The impacts on all modelling results would be so tremendous that if the TAR would be corrected for these errors, there would hardly be any more justification for it. So this paper addresses only few individual TAR fallacies, but focuses on the nondisclosed flawed science it is based on.

Solar impacts
Taking into account the impact of solar variability on global warming, best fit studies have revealed that solar forcing is amplified by at least a factor 4. By leaving out this 'Svensmark factor' and using an exaggerated aerosol cooling, IPCC maintains a CO2 doubling sensitivity of 2.5 °C that is about a factor 3 too high.

Carbon cycle
Our global Carbon Cycle Model reveals a half-life time of only 38 years for any CO2 excess. With present constant global CO2 emission until 2100, the temperature would only further increase by 0.15 °C. Scenario IS92a would end up with 571 ppm only. IPCC assumed that far more fossil reserves would be burnt than being available. Using a flawed eddy diffusion ocean model, the IPCC has grossly underestimated the future oceanic CO2 uptake. Hardly coping with biomass response, limited fossil reserves and using a factor 4 temperature sensitivity, all this leads to an IPCC exaggeration factor of about 6 in yr 2100. The usable fossil reserves of 1300 GtC burnt by 2090, merely cause 548 ppm – not even a doubling. The WRE 650, 750 and 1000 ppm scenarios, projected until 2300, are infeasible. Emission reduction is absolutely useless: the realistic temperature effect of Kyoto till 2050 will be only 0.02 °C.

Radiative forcing
The additional IR absorption (being evaluated here for CO2 doubling) is the energy source for global warming. HITRAN transmission spectra – the fringes being by no means saturated yet – can be used to compute this absorption, mostly occurring near ground. A simple radiative energy equilibrium model of the troposphere yields an IPCC-conforming radiative forcing which is here defined as the additional energy re-radiated to ground. Coping with water vapor overlap on the low frequency side of the 15 µm band, the clear sky CO2 forcing is considerably reduced to 1.9 W/m². With vapor feedback and for cloudy sky the equilibrium ground warming will be about 0.4 to 0.6 °C only – a factor 4 to 6 less than IPCC's 'best guess' for CO2 doubling.
The detailed paper titled "IPCC's most essential model errors" is in HTML, with 16 figures at http://www.microtech.com.au/daly/moderr.htm. I am a known contrarian (see John Daly's Website), and I suppose IPCC can hardly cope with my arguments. So as I basically do not consent with the TAR, please do not use my name within the listing of reviewers.
Peter Dietze, Germany, (Exp.)
I get the impression that this was not a hacker, but a disgruntled insider. It appears that author/company/etc data was removed from some files.

agricola

  • friend
  • Senior Member
  • ***
  • Posts: 1,248
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #22 on: November 29, 2009, 04:02:22 PM »
I suspect alternative media will flap around for a short while trying to measure the scope of the problem.  Big Media will ignore it.  The elites who have staked everything on GW will try to get something on the books before the Great Unwashed wises up.  At some point someone will chirp, "Follow the money" and that will be when the real revelations will commence. 

I must admit that this particular conspiracy theory does confuse me - if you look at the money, the influence and the history then it is quite clear that "the elites" are not on the "enviromentalist" / global warming side of this argument.  To take the UK as an example, its difficult to think of one occasion when the Government has acted in a way that suggests they are actually doing something to stop man-made global warming (roads still being built, data supressed to encourage new roads, expansion of Heathrow and new power plants etc); the Press (especially the Murdoch papers) have been very hot on the CWU leak issue and the concern of MPs on global warming is probably around 10% of the concern they express over their expenses.  Even the EU (firmly in the elite camp nowadays) see global warming as such a serious issue that they are sending Nick Griffin to Copenhagen.  The "elites" might bitch about global warming but they arent doing anything to stop it - indeed the main use of the theory appears to be justification for trips to agreeable destinations around the world for entirely futile talking-shops.

Furthermore, it is also noteable that there are a lot of people criticizing the CRU in an entirely dishonest, misleading and oddly choreographed way - the most well-known here being John Lott, in this piece which also features some especially good hypocrisy:

Quote
Science depends on good quality of data. It also relies on replication and sharing data. But the last couple of days have uncovered some shocking revelations. Computer hackers have obtained 160 megabytes of e-mails from the Climate Research Unit at the University of East Anglia in England. These e-mails, which have now been confirmed as real, involved many researchers across the globe with ideologically similar advocates around the world. They were brazenly discussing the destruction and hiding of data that did not support global warming claims. The academics here also worked closely with the U.N.'s Intergovernmental Panel on Climate Change.

http://www.foxnews.com/opinion/2009/11/24/john-lott-climate-change-emails-copenhagen/
"Idiot!  A long life eating mush is best."
"Make peace, you fools"

MicroBalrog

  • friend
  • Senior Member
  • ***
  • Posts: 14,505
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #23 on: November 29, 2009, 04:26:39 PM »
1. In America, the incident had been roundly ignored by the major papers. The same had occured in the Israeli media and, AFAIK, in the Russian media. That some right-wing papers and channels reported it does not spring make.

2. I don't know about Britain, but Israel's government has long implemented policies intended explictly to suppress the use of private automobiles (by the use of 80%+ taxes and similar devices). To say that no statist policies have been enacted in the West due to Global Warming scaremongering is disingenuous at best.

3. THe accusations against John Lott for data tinkering had not only never been meaningfully proven (certainly not to the extent to which the thing we're dealing with here is proven), they did not affect the body of his work in the same way in which the CRU leak seems to affect the work of the CRU employees.
Destroy The Enemy in Hand-to-Hand Combat.

"...tradition and custom becomes intertwined and are a strong coercion which directs the society upon fixed lines, and strangles liberty. " ~ William Graham Sumner

agricola

  • friend
  • Senior Member
  • ***
  • Posts: 1,248
Re: Climategate Part Deux: A Tale of Code, Commentary, and Contumely
« Reply #24 on: November 29, 2009, 05:00:59 PM »
1. In America, the incident had been roundly ignored by the major papers. The same had occured in the Israeli media and, AFAIK, in the Russian media. That some right-wing papers and channels reported it does not spring make.

Over here its been in every major paper.

Quote from: MicroBalrog
2. I don't know about Britain, but Israel's government has long implemented policies intended explictly to suppress the use of private automobiles (by the use of 80%+ taxes and similar devices). To say that no statist policies have been enacted in the West due to Global Warming scaremongering is disingenuous at best.

Then its good that I didnt say that.  Ironically, here in the UK today evidence emerged - via a Freedom of Information request - that the Government and BAA supressed evidence that did not support their argument to expand Heathrow:

http://www.timesonline.co.uk/tol/news/politics/article6936339.ece

Quote from: MicroBalrog
3. THe accusations against John Lott for data tinkering had not only never been meaningfully proven (certainly not to the extent to which the thing we're dealing with here is proven), they did not affect the body of his work in the same way in which the CRU leak seems to affect the work of the CRU employees.

Now who is being disingenous?  Lott made repeated claims based on varying sources, none of which gave a reliable result (either because the data was "lost", or because he was in error when assembling his own / other peoples data).  What do you think the anti-global warming crowd would make of a pro global warming survey for which no data exists, for which none of the people who carried it out can be identified and which was performed by someone who spent some of his time afterwards - when he wasnt repeating a claim for which he had no evidence (and attributed to numerous sources) - sock-puppeting on the internets?
"Idiot!  A long life eating mush is best."
"Make peace, you fools"