Why Climategate was not a computer hack.

Deus ex machina ...

Why Climategate was not a computer hack.

In the year since the Climategate material became public, I’ve read many articles on the effect the publication has had on the AGW debate. When it is referred to as a hack in them, I usually don’t bother reading any further. If the writer of the article is gullible enough to accept the silly proposition that it was a hack, then it won’t contain anything much in the way of insight; just the usual alarmist party line. If the writer does know better, then the article is disingenuous and the writer is being fundamentally dishonest.

The Climategate material proved to be explosive but I seriously doubt that prior to its publication, anyone would have thought it could be. None but the most hardened campaigners on the skeptic side of the debate had ever heard of the Climate Research Unit (CRU) of the University of East Anglia (UEA). It really wasn’t on anyone’s radar and neither was its email correspondence. There was simply no good reason to do a hack into their systems but in an effort to dispel that myth, I’ll sketch out the amount of work such a hack would have entailed.

The publication of the Climategate material represents a breach of security. For this to have occurred, one or more parties would need to have had gained access to the material. Such access could have been obtained legally by an internal party in the normal course of their duties or by an external party, who had no legal right to the material. The respective terms are internal or external security breach though in the context of Climategate, they are usually described as either the leak or the hack of the Climategate emails.

Industry numbers say that 80% of all reported security breaches are internal but I and most other people with knowledge of the area would say the real figure is nearer 90% or upwards. If one of the Great Whites out there in cyberspace comes after you, it’s because you have information or a particular dataset that is of real value to them and they’re prepared to work very very hard to get it. They have the patience of Job.

Despite what Hollywood and the movies would have you believe, pulling off a successful external hack is far from easy. It requires skill, talent, detailed technical knowledge and above all, patience. Hackers come in three flavours; script kiddies, ascendants and what I like to call the Great Whites. Script kiddies just find scripts laying about the internet and run them, hoping they’ll achieve whatever it says on the can. Ascendants are graduate script kiddies who’re learning to write their own scripts and are perhaps delving deeper into the manuals. They tend to trade scripts with each other and to share some of them with the kiddies for reasons of ego and status. It’s a King of the Kids thing. The overwhelming majority of them never graduate to Great White simply because it requires a massive amount of effort to master the technical requirements and, I would have to say, dedication. They also lack that last but most important ingredient; the nerve to go after hardened targets with a jail sentence attached as the punishment for failure.

The very few who make it to Great White drop off everyone’s radar and are never heard from again, except for their work but only when it’s detected. If I have to go looking for them, I usually start with their juvenile activities because that’s where they’ll have made the mistakes I can use to start locating them. The art of course, is matching the adult’s style with the juvenile’s exploits, their ‘fist’ if you will. That’s why I spend some time watching the ascendents I think are showing some promise.

If the Climategate breach was a result of a hack, then it would have to have been done by a Great White. This outline analysis of a classical frontal assault on an organisation should make that point. I’ve organised it into distinct phases, giving an insight into what each one is about. There are some things to bear in mind while reading this article. The intended audience is the general reader; no great knowledge of IT is assumed. Where it’s come down to technical accuracy or clarity, I’ve chosen the latter. It’s about technique rather than bits or bytes. It is not intended to be nor can it be used as a guide to hacking. Finally, it is not definitive in the sense that there are a myriad of other ways of achieving the same end.

Reconnaissance.

A well constructed attack will begin with a non-invasive reconnaissance phase for information. The objective of this phase is to build up a detailed view of the organisation; its departmental structure, where its buildings are located, who works in the organisation and their roles, who their external suppliers are and the services supplied, any other organisations they interact with and pretty much anything else that can be found out. Google is the prime attack tool here. It will be used in a totally exhaustive search to find every piece of information on the organisation. As each new item of information is found, it in turn is used to find out more. For example, when a name is found, an effort would be made to get that person’s resume or CV, especially for IT personnel. Their areas of technical expertise are a good guide to the exact type of systems running inside the organisation. Why recruit them otherwise? Slightly more intrusive “social engineering” techniques may also be used. Social engineering is essentially tricking information out of people and is an art form in its own right. For example, to obtain CVs one could set up a minimal but very discreet headhunting recruitment site and simply request the CVs (under the strictest of confidence, of course). That one nearly always works.

Mapping and finger printing.

The next phase is to build a detailed technical picture of all the networks and computer systems of the target organisation. This would include determining all services running and each service’s manufacturer and the exact software version, all network connections; internal and external and of course all communications protocols in use.

All computers have what are called ports. Think of them as doorways into and out of the computer through which packets of data flow. The standard Intel chip has sixty-four thousand of these and usually a service operates using one or more of these ports. For instance, email usually uses two of these ports, one for incoming and one for outgoing email. Some other services only use one. There are several methods used to map the internal layout of the target but they all rely largely on sending small ‘signals’ or IP packets to selected ports and examining the result. The IP packets transmitted may be standard or deliberately malformed to provoke a response.

Determining what services are running is done in a similar manner but something called banners can be a help here. When an external server, such as an email server, gets in touch with an internal server, they have to first make contact with each other and establish a communications protocol. At the start of the conversation, normally called handshaking, a banner displaying who developed the server’s software may be shown, thus giving away details of the software’s manufacturer and possibly its version. Even though the banners can be suppressed and although the protocols are of course standardised, there are other nuances in the conversation which can be used to identify exactly the software and its version.

Using these and other methods, the services detected would be “finger-printed” and the exact manufacturer and software versions determined.

Breaking in.

Now that all the technical details of the target’s systems are known, the actual breach can be attempted. It is the most dangerous phase since being noisy or clumsy will set off alarms. Like all internal work, it’s done in the middle of the night in the timezone of the target, allowing some time to recover from any mistakes. There are a number of ways of doing this but I’ll outline just two of the approaches. There are a lot more.

The classic technique is that since they now know the versions of the software, they consult the relevant manufacturer’s website to determine what security patches the version should have to cover loopholes. Armed with this information, they next try to exploit each vulnerability in turn, hoping the security patch has not been applied to the software. If just one works, they’re into the system.

The quality end of the market, tend to take a more difficult but safer approach. They obtain, usually by purchase, the relevant software, install it on a machine and proceed to find a new way to break into it. Having found the weakness, they’ll use it to gain entry to the target’s system. They never share the weakness they’ve found, of course.

If the break-in fails on a particular server, they’ll move their attention to a different one.

Concealment and promotion.

Once in, the next phase begins immediately because they need to conceal the break-in as soon as possible. They will install what’s called a “toolkit” or a “rootkit”, which is essentially a set of programs they can run inside the target’s systems. These are used to “climb the privilege ladder”, which means getting themselves an administrator’s account, the one with the most privileges. Having done this, they will create legitimate logon accounts for themselves and alter all audit logs to hide the break-in. A quick way of getting an administrator’s account, is to install a keystroke logging utility or modify the log in software and then create a minor problem with the server which will oblige an administrator to log onto it to investigate. When they log out, the intruder has his logon Id and password, which he uses to create a new administrator’s account. So, the system is now their bitch? No, not yet and not by a long chalk

Traversal.

All that’s been achieved so far is the Great White now owns a single server which is, to some extent or another, inside the organisation. The next step is to extend ownership or at least access to other servers. This is yet another very delicate technical gavotte whose precise steps I won’t burden you with but take it from me; it’s an even more difficult and time-consuming process. Paradoxically, system administrators pay more attention to what’s happening in internal systems than they do to perimeter systems. Anything strange occurring or anything new in the audit logs gets noticed, so even more care must be taken to make everything appear normal. It only ends when they’ve got access to the data they came in for and it’s been extracted but it isn’t over yet.

An orderly withdrawal.

The final phase is always the cleanup and it’s done very carefully for two very good reasons. Firstly, if confidential information is known to have been accessed, it loses value. Secondly, and just as important, any traces left behind of the break-in will be used in any attempt to find the Great White. They will back out of the target’s systems, server by server, altering logs and closing down any accounts they’ve created. Any code injections will be removed as will all the trip wires they will have strung across the systems. Any internal programs they’ve had to modify will be restored from copies previously taken. At every point during the run, they will never have used an IP address that can be traced back to them and they will never ever use any of those IP addresses again. Any identities stolen will be relinquished, never to be used again. The hard drive of the attack computer used will be extracted from the machine and smashed to bits before the machine with any attendant routers and modems is consigned to the nearest furnace.

All but the first phase of such an attack can be detected by firewalls and Intrusion Detection Systems (IDS). Their answer is to do the subsequent phases very very slowly. Typically, they will ping one of the ports they’re interested in of the available 64,000 on your server in a day. This will not set off any alarms. As I said, the patience of Job. All this concerted effort to get at one mail server? Then more traversal work, to get at the backed up emails from a decade ago on a different server? And then yet another huge effort to hack across from the operations area over to the development area to get at the program source library? Simply no way. An insider job.

Anyone who thinks all of the above effort was expended to obtain apparently innocuous material from an obscure unit of an equally academically obscure university, needs an introduction to William of Occam’s razor.

© Pointman

Related topic : A profile of the Climategate whistleblower.

Click for a list of other articles.

About these ads
Comments
57 Responses to “Why Climategate was not a computer hack.”
  1. Rastech says:

    Well put P. I don’t think whoever was responsible will ever end up in Court.

  2. orkneylad says:

    Occam’s razor is needed in an awful lot of places these days P, when did civilisation become so dumb?

    Great article.
    OL

  3. senter says:

    Why has nobody done this sort of analysis already? excellent.

    • woodNfish says:

      Assuming this is a serious question, the MSM lapdogs are not interested in investigating their pet AGW fraud. If they did so, they might have to admit the truth that they have been avoiding and concealing.

      The only real analysis is being done on blogs like this one.

  4. ThomasJ says:

    Plenty thanks for this most interesting article, Pointman!
    Hopefully, the insider(s) [at CRU] has/have somewhat of a connection to Mr. Assange…

    Brgds from Sweden
    //ThomasJ

    • Pointman says:

      Hi Thomas. Welcome.and thank you. If only the leaker had availed himself of Mr. Assange’s services, Climategate might have forced itself into the MSM sooner.

      Pointman

      • ThomasJ says:

        Well, let’s wait and see. I’ve got a kind of feeling that ‘who’s who’ will be known – and when I’d do everything in my might to get that/those person(s) appointed to the Nobel Peace Prize. (Rather tough task, though… considering the Norwegian(!) committe’s last couple of awards, ex. this years)

        Brgds from Sweden
        //ThomasJ

      • woodNfish says:

        Oh please! Assange and the CRU crackpots are fellow travelers. You don’t have to pay much attention to wickileaks to see it is simple a left-wing anti-capitalist, anti-American tool. They have no wish to do harm to anyone else nor to be honest about their motives.

  5. gcb says:

    Yep – there was no profit motive in the UEA event, so there’s no real reason for this sort of action. Most “hacktivists” tend to choose much more direct (and disruptive) methods of expressing disagreement.

  6. ZT says:

    Here’s what happened:

    From Eric at WUWT:
    :
    1) Pesky requests for emails
    2) Desire to delete emails, but
    3) Fear of losing something important, so
    4) Gather emails into archive then delete local copies and
    5) Claim the emails have been deleted, which is half true, but
    6) Shocking to some decent staff member, who
    7) Leaks the email archive

    • Pointman says:

      Hi ZT and welcome. With respect to Eric at WUWT, try asking your IT department for the last ten years worth of emails and see what they say …

      Pointman

      • ZT says:

        Hi Pointman – agreed – once the IT department had stopped laughing – they would tell me that they were terribly under resourced – and they needed more servers so that I could have 125MB of email in my Lotus Notes account.

        Hence, Eric and WUWT, suggests that the IT department were not involved at all. Jones (or Briffa) was getting irritating requests for emails, Jones carried out his ‘delete’ threat, deleting email from his personal archives, but he also made an archive of everything he ‘deleted’, and that is what was leaked. The fact that he had deleted things from his email client (Thunderbird or whatever) would enable him to say ‘sorry, I have deleted that’, and that would be at least somewhat true. Meanwhile he was scared of losing something actually important to his earth shattering research, and so made an archive of the stuff he was deleting. A person with scruples found this archive and decided to do the ‘right’ thing – upload it to realclimate!

      • Pointman says:

        Hi ZT. There is a certain delicious irony in the theory that it was Jones (or Briffa) doing an archive and delete job on their email client, the results of which were leaked but there’s a problem with it; the emails were not all to, or all from, Jones. The leaker had access to everyone’s emails.

        “We feel that climate science is, in the current situation, too important to be kept under wraps. We hereby release a random selection of correspondence, code, and documents. Hopefully it will give some insight into the science and the people behind it”

        Those are the words that accompaned the release of the Climategate emails. It was a “selection” and that can very easily be read as a threat too. Lord knows what other material they’ve got as well.

        Pointman

      • ZT says:

        Hi Pointman, There’s no ‘Reply’ button on your recent comment (beginning ‘delicious irony’) so I’ll reply here. To answer your objection – I’ll speculate that Jones made the original archive, and Briffa released it, having first added innocuous Briffa emails (to avoid casting suspicion on himself). Briffa is the majority recipient and sender in the email archive, yet Jones is the famous author of the incriminating ones. (This wouldn’t be the first time a lieutenant has decided to bring down a weak and or corrupt general). How does that fit with your analysis? (There is a breakdown of the names and counts of messages in the archive here.)

  7. Lawrie Ayres says:

    I always suspected a whistle blower but with no assurance other than a gut feel. Being computer illiterate I appreciate Pointmans easily understood explanation. Thank you.

    • Pointman says:

      Hi Lawrie and thank you. I worked very hard to earn that “easily understood” compliment. Your gut feeling is accurate, it was a leak all along.

      Pointman

      • Blackswan says:

        G’day Pointman,

        The fact that you “worked very hard” to make the complexity of this issue understood by we ‘non-techy’ types is much appreciated.

        To us it’s as mysterious as brain surgery – just as well we have ‘surgeons’ like you who know what they’re doing.

  8. John says:

    Having just read the above (very interesting by the way) whats the chances of Gary McKinnon doing what they say he’s done? John

    • Pointman says:

      Hi John. The chances are very good that he did what they said. He would be the archetypal Script Kiddie, just running stuff, collecting servers and not really knowing what he was doing.

      Pointman

      • UninformedLuddite says:

        AFAIK all of the stuff he accessed was available via empty or default passwords. he didn;t work too hard for his fame.

      • Pointman says:

        As you say Lud, he didn’t work too hard. Several agencies, not noted for their IT forensic skills, traced him directly back to his PC in England. We’re looking at Dorksville here, not Great White.

        Pointman

  9. John says:

    Mate, that is a fascinating article. My website was hacked recently but no damage done, so I assume it was just ‘because they could’. Does my webhost need to tighten their security?

    • Pointman says:

      Hello John and welcome. “Does my webhost need to tighten their security?”. I assume it was a defacing attack. The answer is yes. Whoever’s doing your hosting needs to apply all the security patches and provide the safe service you’re paying for. Just give them Hell and don’t be fobbed off with a lot of techno BS.

      Pointman

  10. Monbiot, Wednesday 7 July 2010

    The latest, and thankfully last, review of the emails hacked from the climatic research unit (CRU) at the University of East Anglia will do nothing to dam the tide of filth and fury.

    Guardian hacked climategate emails section

    The official line is, and will continue to be that this was a hatchet job.

  11. GlynP says:

    Congratulations! I can’t recall ever reading such a clear explanation – necessarily simplified – of a complex technical achievement. But, even as one unversed in the black arts of IT, my money was always on a leak. Odd that the Guardian should deprecate such whistle-blowing; they were certainly keen enough on Mr. Ponting, who leaked confidential information whilst employed in a Government Department. (Ah, but it was a Conservative Givernment at the time!)

    • Pointman says:

      Hi Glyn. The Guardian’s double standard knows no bounds; witness them falling over to plaster the Wikileaks material all over the rag and an absolute silence for months over Climategate. They were even zapping comments in CiF that mentioned the emails.

      Pointman

  12. Greg says:

    Hey,

    This is my first visit to your site, I followed a link from JoNova. I have to agree with the others, this was very well written and clear. While I’ve always thought a leak was more likely I certainly don’t have the skills to prove that feeling. So now, if the subject comes up, I’ll just point the other person to this page. :)

    And now, I’m going to read the rest of this site!

    • Pointman says:

      Hi Greg and welcome.Knock yourself out mate. Hopefully, even the most devout Greenie would get to the end of the piece and ask themselves; were some boring emails worth all that work?

      Pointman

  13. xmfclick says:

    Just came across your blog (whilst looking for a climate-realist equivalent of skepticalscience.com, which seems not to exist) and rather liked your layman’s guide to hacking. As an IT industry veteran I have always been highly annoyed by ingoramuses like Moonbat referring to Climategate as “hacking”. when anyone with an ounce of sense could see it was an inside job. I have always thought it had to have been pretty premeditated, too, as the range of material involved seemed very diverse — how long must it have taken to collect together that quantity of emails, raw data, source code and whatever other stuff, all of which was undoubtedly spread across a whole range of machines? It certainly wasn’t a five-minute spur-of-the-moment job.

  14. Charlie says:

    very cool article – totally agree with you – its not easy to get in and out undetected, and in a lot of cases, pretty much impossible without some kind of supporting ‘physical’ intervention or assistance. Either CRU security is embarrassingly light and the police forensics team is utterly hopeless, or its an inside job. Watson and Holmes would have sussed this one ages ago..

  15. Dave says:

    As someone who’s spent time on the other side of the fence to you professionally – let’s just say that I worked for a governmental (not US or UK) organisation who employed several great whites – I concur with almost everything you have written. The one place I disagree is where you draw a dichotomy and suggest it was either a hack, or a leak. The third, unmentioned, possibility has always struck me as by far the most likely vector: social engineering.

    Social engineers of the mass-market variety – stealing bank details and so-on – could probably have gained access to the UEA systems without too many difficulties, but they’re only the well-known face of phishing. The less well-known attacks are those aimed at specific systems, and they are basically done by highly computer-literate con artists. Aside from the usual social engineering attacks, they will also do things like engineering a (real life, apparently casual) social encounter with targets in which extra information is often easy to obtain.

  16. cremaster says:

    A really fascinating article, but it still doesn’t prove that it wasn’t a hack. Lots of problems are seemingly beyond the wit of man and yet they get solved. What you have described is no more than a standard piece of espionage, however complex. It is purely a question of who pays how much to whom.
    There are many interested parties (and I speak as an amateur AGW denier) who would benefit from a hack – pretty much everybody who defied the Kyoto protocol, for instance (more power to them).

    • xmfclick says:

      If it was a hack, i.e. penetration by someone from outside of UEA, the question then arises: How did they know that it would be worth hacking in? Who would put all the effort in to such a skillful and accomplished penetration, data collection from diverse locations and then withdrawal removing all traces, unless they knew there was something to go after? How would they have found out? To my mind, Occam’s Razor says inside knowledge was absolutely necessary. Dave’s idea about social engineering is kind of interesting, but again, how would a social engineer know enough in the first place to make him/her think CRU was worth attacking? After all, CRU has been described as an undistinguished backwater in a middle-ranking university in the arse-end of nowhere.

      • Dave says:

        “How did they know that it would be worth hacking in? ”

        Without knowing quite what would be found, people would have had a reasonable suspicion that something was being hidden because of the responses to FOI requests. That holds true whatever vector an outside attack came from.

  17. Pointman says:

    It’s tempting to say that UEA is just another shambles on the IT front but it’s simply not true. Their IT department provides extensive services to 14,000 students and lecturers. That is large-scale and industrial level computing which cannot be achieved without a professional approach to the task. They’ve also had a couple of decade’s experience handling attempts by students to monkey around with their systems. They know how to pin down systems and data.

    Pointman

    • AJC says:

      I guess that you are not too familiar with the development of “IT” in many universities.

      For a “research” group like CRU it is almost certain that its computing effort was provided, for many years at least, internally on research funded equipment and staff possibly as a “hobby” overseen by one of the long term researchers.

      Over many years centralised IT service provision would have lagged the group’s requirement. More recently the provision of IT for the masses will have been the primary focus of the IT service – not providing support for specialist research requirements

      So its quite probable that CRU have been “running” their own show for most of their history

      The quality of the code fragments which have leaked and the excuses about being unable to to resource even minimal change/version control with their datasets does indeed indicate that CRU “is just another shambles on the IT front”: it just shouts lack of professionalism – and I would guess that this applies more widely within UEA .

      • Pointman says:

        I actually understand university computer systems very well. The “development of “IT” in many universities” is similar to the development of IT an any organisation. When there”s ten or fewer employees or whatever, a ‘talented’ amateur can cope. When the numbers grow towards 50 odd and there’s no standardisation, it’s impossible. When it’s approaching the hundreds, the thing grinds to a halt. When you get into the 14,000, the whole organization has long ago ground to a halt unless you’ve got professional.

        The oldest rule of IT applies; the business owns the data but IT owns the systems. Anyone within a large organisation who wants to run their very own systems, no matter what their budget, becomes a board / governance problem. No IT department will provide any support to another department who insists on doing their very own IT thing and that’s always the end of that idea. Let’s get sensible here.

        Pointman

  18. Bishop Hill says:

    I think I’m right in saying that the School of Environment at UEA ran its own IT. There’s something about this in some of the papers on the Russell inquiry website.

    • Pointman says:

      Hello and welcome. I’ve never been able to ascertain for sure the extent to which CRU ran its own systems. UEA provides accounts and email services to about 14,000 people and it would seem reasonable to me that given those numbers, these facilities would be furnished and maintained by a central body. Within each faculty there would be systems being run which were specific to the area of study. If they were indeed running their own systems for email then the chances of it being a leak certainly increase.

      Pointman

  19. Seedtickinohio says:

    Thanks

  20. Verity Jones says:

    Belated thanks from me too. Simplfication and clear communication of technical issues is a real gift and I really apreciate the effort it can take. I was beginning to believe the hacker thing, but no more.

  21. Ally E. says:

    Wow, Pointman, everything I read of yours is astounding. I linked to this post from “So, was Climategate a hack after all?” and, as promised, have been given insight into the field of hacking. The way you explain it makes it very clear – even to someone like myself, with no knowledge of computers – that Climategate HAD to be an inside job or at least have inside connection. Never ever put down your pen… er… keyboard. Your writing is too valuable. :)

  22. M Simon says:

    their ‘fist’ if you will

    That is first rate. You would have do be deep into the history of comsec to know what that means. As a very old radioman – bravo.

Trackbacks
Check out what others are saying...
  1. [...] has been much interest in Pointmans excellent analysis of why the notorious Climategate email leak could not have been a hack, contrary to the frequent [...]

  2. [...] all questions Unis, pointman is your [...]

  3. [...] reading a post last month by Pointman Why Climategate was not a hack  I am thoroughly convinced it was a leak, or a least that there was an insider involved.  (Highly [...]

  4. [...] världen runt och inlagda villospår. Alla håller dock inte med om det resonemanget, se t.ex. Pointmans [...]



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: