Cloaking FAQS — Page 3
What is keyword density?A: One of the standard elements of web page optimization is Keyword Density: up until very recently the ratio of keywords to rest of body text was generally deemed to be one of the most important factors employed by search engines to determine a web site's ranking.
However, this basically linear approach is gradually changing now: as mathematical linguistics and automatic content recognition technology progresses, the major search engines are shifting their focus towards "theme" biased algorithms that do not rely on analysis of individual web pages anymore but, rather, will evaluate whole web sites to determine their topical focus or "theme" and its relevance in relation to users' search requests.
This is not to say that keyword density is losing in importance, quite the contrary. However, it is turning into a lot more complex technology than a simple computation of word frequency per web page can handle.
Context analysis is now being determined by a number of auxiliary linguistic disciplines and technology, for example:
- semantic text analysis
- textlexical database technology
- distribution analysis of lexical components (such as nouns, adjectives, verbs)
- evaluation of distance between semantic elements
- AI and data mining technology based pattern recognition;
- term vector database technology, etc.
All these are now contributing to the increasing sophistication of the relevance determination process. If you feel this is beginning to sound too much like rocket science for comfort, you may not be very far from the truth: it seems that the future of search engine optimization will be determined by what the industry is fond to term the “word gurus”.
A sound knowledge of fundamental linguist methodology plus more than a mere smattering of statistical calculus will most probably be paramount to achieve successful search engine rankings in the foreseeable future. Merely repeating the well worn mantra “content is king!”, as some of the lesser qualified SEO professionals and very many amateurs are currently doing, may admittedly have a welcome sedative effect by creating a feeling of fuzzy warmth and comfort. But to all practical purposes it is tantamount to whistling in the dark and fails miserably in doing justice to the overall complexity of the process involved.
It should be noted that we are talking presence AND future here: many of the classical techniques of search engine optimization are still working more or less successfully, but there is little doubt that they are rapidly losing their cutting edge and will probably be as obsolete in a few months' time as spamdexing or invisible text — both optimization techniques well worth their while throughout the 90s — have become today.
So where does keyword density come into this equation? And how is it determined anyway?
There's the rub: the term “keyword density” is by no means as objective and clear-cut as many people (some SEO experts included) will have it! The reason for this is the inherent structure of hypertext markup language (HTM) code: as text content elements are embedded in clear text command tags governing display and layout, it is not easy to determine what should or should not be factored into any keyword density calculus.
The matter is complicated further by the fact that the meta tags inside a HTML page's header may contain keywords and description content: should these be added to the total word count or not? Seeing that some search engines will ignore meta tags altogether (e.g. Lycos, Excite and Fast/Alltheweb), whereas others are still considering them (at least partially), it gets even more confusing. What may qualify for a keyword density of 2% under one frame of reference (e.g. including meta tags, graphics ALT tags, comment tags, etc.) may easily be reduced to 1% or less under another.
Further questions arise: will meta tags following the Dublin Convention (“D.C. tags”) be counted in or not? And what about HTTP-EQUIV tags? Would you really bet the ranch that TITLE tags in tables, forms or DIV elements will be ignored? Etc., etc.
Another fundamental factor generating massive fuzziness left, right and center, is the issue of semantic delimiters: what's a “word” and what isn't? Determing a lexical unity (aka a “word”) by punctuation is a common though pretty low tech method which may lead to some rather unexpected results.
Say you are featuring an article by an author named “John Doe” who happens to sport a master's degree in arts, commonly abbreviated as “M.A.”. While most algorithms will correctly count “John” and “Doe” as separate words, the “M.A.” string is quite another story. Some algorithms will actually count this for two words (“M” and “A”) because of the period (dot) is considered a delimiter — whereas others (surprise!) will not. But how would you know which search engines are handling it in which way? Answer: you don't, and that's exactly where the problems start.
The only feasible approach to master this predicament is trial and error. The typical beginner's inquiry “What's the best keyword density for AltaVista?”, understandable and basically rational as it may be, is best answered with the fairly frustrating but ultimately precise: “It all depends — your mileage may vary.” It is only by experimenting with keyword densities under standardized, comparable conditions yourself that you will be able to come to significant and viable conclusions.
To get going, here are some links to pertinent programs that will help you determine (and, in one case, even generate) keyword densities.
KeyWord Density Analyzer (KDA)
An all time classic of client based keyword density software is Roberto Grassi's powerful KeyWord Density Analyzer (KDA). It is immensely configurable and offers a fully featured free evaluation version for download. Find it here:
http://www.grsoftware.net/search_engines/software/grkda.html(Expect to pay appr. $99 for the registered version.)
Concordance
Concordance is a powerful client based text analysis tool for making word lists and concordances from electronic texts. A trial version can be downloaded here:
http://www.concordancesoftware.co.uk/(Expect to pay appr. $89 for the registered version.)
fantomas keyMixer™
Our own fantomas keyMixer(TM) is the world's first automatic keyword density generator, enabling you to create web pages with ultra precise densities to the first decimal digit. Read more about this server based Perl/CGI application here:
http://fantomaster.com/fakmixer0.html(Expect to pay appr. $99 for the registered version.)
How can I cloak themes?A: There is a marked tendency with major search engines to rank web sites according to their pages' overall or “theme” relevancy. Though this technology seems to be still very much in the making there is little doubt that it will play a major role within the foreseeable future.
So where does that leave us cloakers? The actual maths behind theme or context relevancy based algorithms is a very involved and complex affair. Generating theme based web pages according to all the rules in the book is bound to become a major industry in its own right some day, but this is well beyond the scope of this FAQ. (See a graphical overview of the basic mechanics at: http://contextbase.com/cbintro03.html.)
So for practicability's sake we'll assume that you have a fairly focused web site up and running which you wish to optimize on a theme base. One trick to do this is a technique we will term Topical Clustering or TC for short.
The mechanics of TC are fairly straightforward to start with:
- Determine your site's sub topics Determine which pages belong to which common set of keyword phrases. Most multi page web sites feature several sub topics at least. Make sure to distinguish between them, even if they do relate to each other on a meta level. E.g. if your site is focused on “second hand cars” as primary or meta topic, you will want to optimize it for the overall theme “automobiles”. However, you will probably have structured the site into several sub topics — each of which constitutes a Topic Cluster. Sub topics will typically be reflected in a site's navigation scheme: “vans”, “limos”, “Ford”, “Chrysler”, “vintage cars”, “Italian sports cars”, etc.
- Generate filler text files by Topic Cluster
Generate as many different filler text files per Topic Cluster as you can. And here, “different” actually means different! Merely switching some keyword phrases for their alternatives won't do.
Neither should your filler text files consist of mere dictionary lists — while this technique used to work fine until quite recently, expect it to lose its clout as search engine algos become more sophisticated, checking spidered pages for plausible linguistic syntax, punctuation and other natural language features plus, of course, for content.
So in a nutshell: use as much “real” and relevant text as you can come up with.
It is recommended to work from fairly long filler text files — while your mileage may vary, we tend to generate pages sized 10-15K for our clients' sites. The reason for this is that it gives you a wider scope for expounding on the targeted theme simply because you can boost semantic and context relevance more easily that way. - Keyword optimize your cloaked pages Next, you may proceed to optimize your cloaked pages by keyword density. (For an extensive discussion of this technique see “What is keyword density?”.) Using several differing but topically related filler text files, you will generate a set of TC pages.
- Cross linking cloaked pages
Cross link your content related cloaked pages by Topic Cluster so that they will point to each other. Don't forget to include at least one link to your index page on one or all of them. You may also want to link to at least one if not all of them by placing an invisible link on your index page.
Additionally, you may also create a cloaked contents page per Topic Cluster linking to your newly generated cloaked pages (and, again, to your index page). Finally, fine tune the keyword density of all pages covered here again if required. - Submit your topic cluster pages Finally, submit your TC pages to the search engines.
A: Cloaking web page meta tags follows a different procedure than the IP delivery and full page cloaking method discussed so far.
Server Requirements
To take advantage of this procedure you must be able to make use of Server Side Includes (SSI) on your web server.
Note IIS/4.0 users: The code presented here is an extended SSI expression which is not supported under IIS/4.0.
Meta tag cloaking is effected by excluding browser from viewing certain parts of a web page, specifically the header where meta tags are positioned by default.
Browsers are determined by their UserAgent variable. Once properly cloaked, it won't make any difference whether you read the source code online or whether you download it for viewing offline — the meta tag code will remain hidden, the browser will not be able to read it and will therefore not download it either.
Browser UserAgents
Here is a list of UserAgents as used by popular web browsers:
- “Lynx”: Lynx text browser
- “Mozilla”: Netscape browsers
- “MSIE”: Microsoft Internet Explorer
- “NCSA Mosaic”: Mosaic technology based browsers like Spry, Spyglass, etc.
- “Opera”: Opera browser
- “WebTV” WebTV's proprietary browser
Activating SSI
The .htaccess File
If your web server is not configured for SSI by default, you will need to upload a file named “.htaccess” (please note the period/dot at the beginning of the file name!) to your server directory. This can be done by Telnet or FTP.
The .htaccess file should have the following content:
Options Includes +ExecCGI AddType text/x-server-parsed-html .html
Note that many web servers will not require the specfication "Includes", meaning you can omit it altogether. However, since it won't do any harm to keep it in your file, we suggest you do not change the above entry. Thus, should you switch servers some day, you will not have to readjust your .htaccess file.
After you have uploaded the modified .htaccess file (MUST be in Ascii mode!), you're ready to go.
In the HEADER section of the web page whose meta tags you wish to protect, place the following code:
<!--# if expr="\"$HTTP_USER_AGENT\" != /Mozilla |MSIE|Opera|Lynx|WebTV|NCSA Mosaic/" -->
VERY IMPORTANT (1)
The above must be in one SINGLE line! As browsers tend to word wrap lines which are too long for display, we had to word wrap the example. So please take this into account when cutting and pasting the above code.
Under this first header entry, you may now add the actual meta tags you wish to protect.
When you are done, you must close the protected section with the last header entry, or the rest of your page won't be displayed either!
<!--# endif -->
VERY IMPORTANT (2)
If you have other entries in your page header (e.g. for an external CSS style sheet, an external JavaScript applet, etc.) you MUST place these OUTSIDE the protected area (but WITHIN the header tags) or they will not work unless you are operating with a browser sporting a UserAgent not included in the code above.
So what does it do?
The SSI code outlined above will determine the accessing browser by its UserAgent variable. If it is recognized, the system will skip the content within the exclusion tags, effectively preventing the meta tags from being displayed.
Search engine spiders not using common browser UserAgent variables (most don't) will still get to read the meta tags nevertheless, which is, of course, what you want them to do.
Limitations
The method outlined above may well qualify for “poor man's cloaking” — it is NOT an industrial-strength protection against code snoops, the more so as UserAgents can easily be forged (“spoofed”). But it will at least cover about 95% of all ordinary browsers and their users without putting an undue strain on server load and, hence, system performance.
Bear in mind, too, that meta tags are gradually losing in importance as many search engines have stopped indexing them because of massive abuse by keyword spamming (“spamdexing”) and irrelevant description tags in the past.
Other uses
This technique can also be used to prevent email harvester bots (address extractors) from culling email addresses from textarea fields, e.g. if you happen to display your ezine complete with editors' addresses in text format on your web site.
- What are Ghost Pages?
- What are Doorway Pages, then?
- And Hallway Pages?
- How are cloaked pages submitted?
- How about changing stealth pages?
- What are the mechanics of cloaking?
- What's a keyword switch?
- Isn't this really just a simple redirection technique?
- What about penalization?
- What is keyword density?
- How can I cloak themes?
- How do I cloak meta tags?
- Could you show me an actual real life example of cloaking?
- Do you have a tutorial showing how to generate stealth or phantom pages?
Featured as an ongoing multi part section in our fantomNews™ newsletter (formerly published as an e-zine, now an online weblog), we are offering you all the stuff the search engines don't want you to know, straight from the horse's mouth!
Learn the secrets of the pros – it's free!
Get your copy of “Stealth, Cloaking, Phantom Technology: How it works – and how it doesn't” and lots of other SEO / SEM information by downloading our complete e-zine archive here!
Info
fantomNews
fantomProducts
fantomTips
fantomServices
fantomFree
fantomOrders
fantomAffiliates
fantomOEM
fantomMedia
Special Deal
The all-powerful 100% automatic Shadow Domain™ generator for effective heavy duty industrial-strength cloaking.
This beauty generates 100% relevant and unique content for your Shadow Domains™, creating an unlimited number of highly optimized unique pages including site maps in fully customizable keyword density and page weight
Then submit them to the search engines and redirect search engine generated human traffic in realtime (i.e. without delay) by search phrase to any URL you wish
Read More …searchbotBase
Service
The world's largest database of search engine and other spiders, updated every six hours, seven days a week
The fantomas spiderSpy™ botBase Service is an indispensable backend tool for IP based search engine optimization software developers and agencies.
It is also a must have for server stats analysis tools and services to ascertain reliable spider access traffic data.
Read More …IP Delivery Program Highly efficient and reliable IP based cloaking technology for optimized search engine rankings.
If you're developing web content in highly competitive areas, this software is your only chance to achieve good to smashing positioning!
Read More …Keyword Density Generator Program Generates precise customizable keyword density for optimized search engine positioning.
The world's first tool of its kind, this cutting edge server based application automates the task of generating high precision keyword density for your gateway and stealth pages.
Read More …Engine Submission Program Fully customizable and user defineable server sided automatic search engine submitter.
Submit as many URLs as you like to any or all of dozens of search engines — and let the process run in the background, leaving you all the time in the world for other tasks.
Read More …