Conservatives erase Internet history
The Conservative Party has attempted to erase a 10-year backlog of speeches from the internet, including pledges for a new kind of transparent politics the prime minister and chancellor made when they were campaigning for election.
Prime minister David Cameron and chancellor George Osborne campaigned on a promise to democratise information held by those in power, so people could hold them to account. They wanted to use the internet transform politics.
But the Conservative Party has removed the archive from its public facing website, erasing records of speeches and press releases going back to the year 2000 and up until it was elected in May 2010.
It also struck the record of their past speeches off internet engines including Google, which had been a role model for Cameron and Osborne’s “open source politics”.
And it erased the official record of their speeches from the Internet Archive, the public record of the net – with an effect as alarming as sending Men in Black to strip history books from a public library and burn them in the car park.
Sometime after 5 October, when Computer Weekly last took a snapshot of a Conservative speech from the Internet Archive, the Tory speech and news archive was eradicated.
Conservatives posted a robot blocker on their website, which told search engines and the Internet Archive they were no longer permitted to keep a record of the Conservative Party web archive.
The Internet Archive was unavailable for comment. But a fortnight after Computer Weekly started asking its San Francisco HQ for an explanation, the Conservative speeches have begun reappearing on its site.
CW had asked the Internet Archive to explain how the historic record of the lead party in the coalition that holds power in the UK could simply be erased.
The Conservative Party’s robot blocker forced the Internet Archive to remove the entire record of speeches and news it had collected, in 1,158 snapshots it took of the Conservative website since 8 May 1999.
The Conservative bot blocker listed all the pages barred for public consumption thus (excerpt):
Disallow: /News/News_stories/2000/
Disallow: /News/News_stories/2001/
Disallow: /News/News_stories/2002/
Disallow: /News/News_stories/2003/
Disallow: /News/News_stories/2004/
Disallow: /News/News_stories/2005/
Disallow: /News/News_stories/2006/
Disallow: /News/News_stories/2007/
Disallow: /News/News_stories/2008/
Disallow: /News/News_stories/2009/
Disallow: /News/News_stories/2010/01/
Disallow: /News/News_stories/2010/02/
Disallow: /News/News_stories/2010/03/
Disallow: /News/News_stories/2010/04/
Disallow: /News/News_stories/2010/05/
Disallow: /News/Speeches/2000/
Disallow: /News/Speeches/2001/
Disallow: /News/Speeches/2002/
Disallow: /News/Speeches/2003/
Disallow: /News/Speeches/2004/
Disallow: /News/Speeches/2005/
Disallow: /News/Speeches/2006/
Disallow: /News/Speeches/2007/
Disallow: /News/Speeches/2008/
Disallow: /News/Speeches/2009/
Disallow: /News/Speeches/2010/01/
Disallow: /News/Speeches/2010/02/
Disallow: /News/Speeches/2010/03/
Disallow: /News/Speeches/2010/04/
Disallow: /News/Speeches/2010/05/
Disallow: /News/Articles/2000/
Disallow: /News/Articles/2001/
Disallow: /News/Articles/2002/
Disallow: /News/Articles/2003/
Disallow: /News/Articles/2004/
Disallow: /News/Articles/2005/
Disallow: /News/Articles/2006/
Disallow: /News/Articles/2007/
Disallow: /News/Articles/2008/
Disallow: /News/Articles/2009/
Disallow: /News/Articles/2010/01/
Disallow: /News/Articles/2010/02/
Disallow: /News/Articles/2010/03/
Disallow: /News/Articles/2010/04/
Disallow: /News/Articles/2010/05/
For pages at these addresses, the Internet Archive reported: “Page cannot be crawled or displayed due to robots.txt”.
An administrator at the Internet Archive HQ in San Francisco said its guidance for lawyers explained the mechanism. That was that if a website, like Conservatives.com, put up a robot blocker, those pages it blocked would simply be erased from the record as a matter of etiquette.
The erasure had the effect of hiding Conservative speeches in a secretive corner of the internet like those that shelter the military, secret services, gangsters and paedophiles.
The Conservative Party HQ was unavailable for comment. A spokesman said he had referred the matter to a “website guy”, who was out of the office.
It wasn’t always going to be like this.
Such as when the prime minister first floated his groovy idea that the democratisation of information would transform politics, at the Google Zeitgeist Europe Conference, on 22 May 2006.
“You’ve begun the process of democratising the world’s information,” he told the Googlers. “Democratising is the right word to use because by making more information available to more people, you’re giving them more power.
“Above all, the power for anyone to hold to account those who in the past might have had a monopoly of power – whether it’s government, big business, or the traditional media,” said Cameron, who was then campaigning for power as leader of the Conservative opposition.
Cameron was going to make sure the information revolution would hold people like prime ministers to account, he said another speech on 11 October 2007, at the Google Zeitgeist Conference in San Francisco.
“It’s clear to me that political leaders will have to learn to let go,” he said then. “Let go of the information that we’ve guarded so jealously.”
Transparency would make public officials accountable to the people, said Cameron then. He was riding at the front of the wave that would wash us into a new world, and a new age.
Likewise the chancellor, who on delivering his landmark “Open Source Politics” speech at the Royal Society of Arts on 8 March 2007, declared his ambition was “to recast the political settlement for the digital age”.
“We need to harness the Internet to help us become more accountable, more transparent and more accessible – and so bridge the gap between government and governed,” said Osborne.
“The democratization of access to information… is eroding traditional power and informational imbalances.
“No longer is there an asymmetry of information between the individual and the state, or between the layperson and the expert,” said the Chancellor when he was campaigning for election.
If the Conservative Party had moved its speeches and news archive to a more convenient location it had managed to do it in a way that hid it from the search engines. It might before long end up at the Oxford University’s Bodleian Library, which keeps the official Conservative Party archive of really old stuff like speeches from the days before the internet.
The robot blocker – a robots.txt file – tells software bots run by sites like Google and the Internet Archive to bog off. The bots grab web pages for the benefit of plebs like those Cameron and Osborne claimed to be speaking for in those years before they were elected. The bots were what made the democratization of information possible. It was bots that inspired Cameron and Osborne. It was bots that were going to free us from serfdom in the way they said we would be. Without the bots you just had pockets of power and privilege for those in the know. Without the bots you just had the same old concentration of wealth and po wer there had always been, since long before the Internet Archive started taking snapshots of the Conservative website in 1999.