fantomNews™ — the ultimate know

So maybe it’s yet another fad, just like one of the many that have hit the SEO industry consistently ever since it came about to be. Remember the “theme based search” debates? Reciprocal links? Directory submissions as the be all and end all of effective SEO? Domain age, registration span and Whois data as a purported search ranking factor? The “filthy linking rich” (Mike Grehan’s lovely phrase) versus the link havenots? Geo targeting as an SEO killer? Personalized search as yet another SEO nemesis? Not to forget that inane old hard-to-kill “white hat vs. black hat” controversy, still good for plenty of link bait to this very day…

Not saying that any of those disputes were entirely off the mark or a mere waste of time, as such debates go. Provided there’s a certain level of rationality involved and mere surmise (aka fantasies…) isn’t being touted as proven, measurabld fact they usually have their merits. Because yes, search has always been in a state of constant evolution, so what’s new?

Of course, the main issue bedeviling all such discussions is always the same: the minute people try to reduce what is essentially a pretty complex, heterogeneous set of SEO factors to a mere simplistic how-to recipe, possibly tagged to some fixed numbers pretending to be hewn in stone, “scientific” even (”you’ll need 25 RSS subscribers to make it in the SERPs”, and similar drivel), that’s when the proverbial feces will hit the ventilation.

Sure, this silly orgy of reductionism may be in full compliance with human nature, but that doesn’t really make it any better, does it? Or more effective. (Not to mention efficient…)

Currently, the SEO meme of the day is “bounce rates”, derived from behavioral metrics analysis, a slew of search engine patents documenting more than a fleeting interest in this field, a few (typically anecdotal and subjective) metrics (with sources usually being withheld), plus a boatload of entailing speculation of what it may all mean, followed up with tons of conclusions of the - you guessed it - how-to kind.

As a rule, the rationale runs somewhat like this: The more search engines know about you, the more personalized your search results are going to get (or: are actually getting already). Needless to say, a lot of confusion abounds in this field, starting off with the question how many Google toolbars and Google Analytics implementations etc. there actually are out there to base behavioral metrics on in any meaningful manner.

At closer look much of this contention is fairly immaterial, however: statistical demographics as used in market research, advertising, polls research etc. doesn’t really require an unlimited set of data to come up with fairly stringent and verifiable prognoses. It all depends on how granular you actually want (or need) your data to be.

What I haven’t seen being discussed at any length to date is the role AdSense may or may not play in identifying individual site visitors - after all, it’s a prime data conduit for Google. (With other contextual ad platforms probably providing their respective owners with similarly well targeted user data streams.)

Of course this isn’t all about the “user experience”, no matter what the search engines may want us to believe. Far more importantly, because that’s where the big money lies, it’s about laser targeting advertising, the more the merrier. The better focused advertising becomes, the greater obviously the competitive advantage for everyone involved: the advertising platform’s, its advertisers’, its publishers’.

But this isn’t about when we will actually see bounce rates and behavioral data being converted into critical ranking factors and in which precise manner: that, to date, is really anybody’s guess and only time will really tell beyond any reasonable doubt. For a general intro to some resources covering the issues at hand in some depth, please see the link list below.

What’s really more critical for us here, however, is the currently predominant perception that this is at least one major way search will be heading fairly soon. And  markets having that knack of usually getting what they’re crying out for, it only stands to reason that we may expect to see the first serious SEO Surfbot Nets being rolled out  in just a few months’ time.

Some terminological clarification may be called for here, lest misunderstandings – both willful and innocent – run rampant: after all, the term “botnet” itself is pretty ambivalent at best, with common usage lending it a decidedly malicious slant.

Seeing that Mr Cutts of Google is not beyond stooping to equate “black hats” with criminals or felons in a blanket manner these days, pretentiously assuming the garb of investigative agent, legal counsel, prosecutor and judge in personal union (see mins. 07:04-07:42 on this much touted video…), this is an eminently critical distinction to make!

Wikipedia actually differentiates it the best for a change:

Botnet is a jargon term for a collection of software robots, or bots, that run autonomously and automatically. The term is often associated with malicious software but it can also refer to the network of computers using distributed computing software.

While the term “botnet” can be used to refer to any group of bots, such as IRC bots, this word is generally used to refer to a collection of compromised computers (called Zombie computers) running software, usually installed via worms, Trojan horses, or backdoors, under a common command-and-control infrastructure.

Webopedia, by contrast, is focused entirely on the destructive malware aspect of the term:

A botnet refers to a type of bot running on an IRC network that has been created with a trojan.

So is the Free Dictionary:

(roBOT NETwork) Also called a “zombie army,” a botnet is a large number of compromised computers that are used to create and send spam or viruses or flood a network with messages as a denial of service attack.

In a similar vein, finally, AllWords:

A collection of zombies that are controlled by the same cracker; a collection of compromised computers that are slowly grown then all unleashed at once as a DDOS attack or used to send massive amounts of spam

By contrast, an SEO Surfbot Net (SBN) is not a malicious let alone an illegal operation. Yes, it qualifies for a “black hat” SEO venture in that it will probably violate most search engines’ Terms of Service (much like search engines will violate very many individual web sites’ TOS and, arguably, international copyright laws and treaties to boot), but that doesn’t constitute an illegal act per se in any except perhaps the most dictatorial jurisdictions like, maybe, North Korea…

Rather, its sole rationale will be to simulate human Web surfing behavior to furnish the search engines with appropriate behavioral data to “help” them rank those sites better the SBN in question is servicing. (Well, at least inasmuch as bounce rates etc. will actually constitute a material ranking factor.)

So what will such an SBN conceivably consist of? The basic setup should be pretty straightforward:

1. An extensive range of anonymous proxies - think mid to upper five digit numbers here if you really want to make a noticeable impact in a fairly competitive industry. This will obviously depend entirely on your targeted niche: by way of an off the cuff guess, search phrases such as “online realtors in upstate New York” or “scuba diving off Somalia” can likely make do with a few hundred clicks per month whereas search terms such as “money” or “sex” will probably require a bot traffic volume and, hence,  a proxy network, that’s significantly larger by orders of magnitude.
2. Configurable rule driven automated browsing scripts (your “bots” proper) that will daisywheel through your proxies to simulate human visitors when parsing the SERPs, spoofing different browser UserAgents, accepting cookies, sporting select appropriate referrers as required, etc.
3. Intelligent behavioral pattern master scripts for your bots ranging from set deployment times (e.g. “morning”, “noon”, “afternoon” etc.) through SERP clicking patterns (like going for different other sites first before hitting the real target link) to on site navigation.

Obviously, a rudimentary structure like the one outlined here leaves a lot to be desired by way of fine tuning to make it more effective. Here’s a few improvement factors that will push it to a higher level of efficiency:

• Geo targeting: E.g. if the web site you’re servicing will typically pull most of its traffic from U.S. surfers, you’d better bias your proxy network accordingly. By the same token, French web sites should be serviced primarily via French proxies, etc.
• Time zone compliant behavior: Don’t deploy your bot traffic at highly unlikely times of day relative to your proxies’ time zones. Similarly, you will want to account for public holidays, seasonal factors, etc.
• Virtual user profiles: By creating a set of typical user profiles (translatable into behavioral rules your bots will follow, with a reasonable amount of randomized deviance/variance implemented to avoid mechanical footprints) you will add that much more plausibility/credibility to your SBN’s performance. This would be in compliance with (i.e. mirroring) established browser market shares, usage of typos, etc.

There’s lots more, of course, but then this article isn’t intended to serve as a comprehensive engineering blueprint. Rather, it’s merely a general outline to set those of you thinking who might want to develop (or make use of) such an SBN.

As an aside: there’s alternate possibilities of conducting all this far more cost efficiently without leveraging any extensive proxy network at all, but discussing such technicalities at length here would go beyond the scope of this article which has turned out to be quite a bit longer than anticipated anyway.

Sources

So here, finally, is the list of promised resources (far from comprehensive and entirely subjective, to be sure) in case you’d like to read up this topic (minus the SBN aspect) in greater depth.

Quadzilla: “Confirmed: Bounce Rate is A Search Engine Ranking Factor”

Eric Enge: “Do Search Engines Use Bounce Rate As A Ranking Factor?”

David Leonhardt: “Do Bounce Rates Really Count?”

David Leonhardt: “Bounce Rate SEO Fallacies”

See the discussion at Sphinn as well: “Bounce Rate Fallacies”

Ian Lurie: “SEO 2009: Adapt or Die”

Ian Lurie: “Google 2009: The Painful Details”

For a (well argued) contrarian view, see -

David Harry (aka theGypsy): “Is Google REALLY using bounce rates as a ranking signal?”

Subscribe Feed!