fantomNews™ — the ultimate know

Seems that the White House is doing its very best not to get indexed by just about every search engine under the sun, if their robots.txt file is any indication. Interpreted in SEO terms, they’re obviously tied up in a frenzy of privacy protection, trying to lurk far off from the madding crowd in the dark, like the shady spooks they are so very much in love with.

Of course, the fact that Google’s such a veritable spook outfit in their own right lends some not-too-subtle irony to this all, reminding us of the old Spy vs Spy joke and dagger funnies…

Ok, so here’s a FREE TIP for Mr Bush and his administrative entourage, straight from the black hat’s mouth: If you really don’t want your stuff to be found by puny, inquisitive citizens on the Net, why not simply password protect it, only allowing nerds with top security clearance in? (Hint: Matt Cutts should qualify. ) That’ll teach the riff-raff a lesson or two, yippie!

Then again, if you’re looking for an industrial-strength cloaking, er: IP delivery solution, may we make bold to point you here?

Note: Hat tip and all kudos to the good folks at Done! SEO who have pointed to this first: Check out the robots.txt for Whitehouse . gov 

(You’ve simply got to love their comment, too: “And this from the people that tell us, ‘if you don’t have anything to hide….’” LOL)

The reason why we’re offering the full file for download below is actually for keeping the records straight - these things are subject to change, and keeping an additional copy here on our system may serve for archival redundancy to aid researchers further down the road.

In any case we haven’t compared their version with this one which comes straight from http://www.whitehouse.gov/robots.txt. Some of the excerpts are pretty telling - I mean, who in their right mind would want to exclude their privacy policy from spidering?

Some gems:

User-agent: *
Disallow: /911/heroes/text
Disallow: /911/messages/text
Disallow: /911/patriotism/text
Disallow: /911/patriotism2/text
Disallow: /911/progress/text
Disallow: /911/remembrance/text
Disallow: /911/response/text
Disallow: /911/sept112002/text
Disallow: /911/text
Disallow: /ConferenceAmericas/text
Disallow: /GOVERNMENT/text
[…]
Disallow: /asia/2005/photoessay/china/text
Disallow: /asia/2005/photoessay/japan/text
Disallow: /asia/2005/photoessay/korea/text
Disallow: /asia/2005/photoessay/mongolia/text
Disallow: /asia/2005/photoessay/mrsbush1/text
Disallow: /asia/2005/photoessay/mrsbush2/text
Disallow: /asia/2005/photoessay/summit/text
[…]
Disallow: /baseball/2005/text
Disallow: /baseball/photoessay/text
Disallow: /baseball/text
Disallow: /beazley/text
Disallow: /bioshield/text
Disallow: /birdflu/text
Disallow: /blackhistory/text
Disallow: /budget/text
[…]
Disallow: /deptofhomeland/analysis/text
Disallow: /deptofhomeland/bill/text
Disallow: /deptofhomeland/text
Disallow: /dpc/stemcell/2007/text
Disallow: /dpc/text
[…]
Disallow: /easter/2006/eggsbystate/text
[…]
Disallow: /firstlady/newborn/text
Disallow: /firstlady/news-speeches/releases/print/text
[…]
Disallow: /firstlady/photoessays/gulfcoast-rebuilding/text
[…]
Disallow: /ghosts/text
[…]
Disallow: /halloween/text
Disallow: /healthcare/text
[…]
Disallow: /kids/baseball/quiz/old/text
[…]
Disallow: /missbeazley/text
Disallow: /mrscheney/news/text
Disallow: /mrscheney/old-site/text
[…]
Disallow: /news/releases/2007/text
[…]
Disallow: /omb/financial/text
Disallow: /omb/foia/print/text
Disallow: /omb/foia/text
[…]
[…]
Disallow: /president/american-flag/text
[…]
Disallow: /stateoftheunion/2007/text
[…]
Disallow: /vicepresident/news-speeches/text
[…]
Disallow: /help
Disallow: /sitemap.html
Disallow: /privacy.html
Disallow: /accessibility.html

Get the full current file here.

Subscribe!