< Back To Search Engine Marketing Info

You've probably heard news reports regarding the trend toward what is generically referred to as paid inclusion or paid submissions. The press has often been critical of the search engines' move to "monetize" searching. The criticism comes from the long-standing belief that a search engine should deliver unbiased results similar to the way a major newspaper or publication tries to separate their ad departments from their news reporting. The success in which the mainstream media succeeds at separating the two is also a continual source of debate by many.

Since advertising revenues alone have not supported many dot com companies of late, the search engines have turned to other ways to charge for their services. Those of us in search engine marketing have long known the value of listings on the major search engines, assuming you achieved a high ranking. It was only a matter of time before the search engines realized that the results they were serving generate free, highly targeted traffic to potentially millions of Web sites. There's obviously real advertising value in those submissions. The question is how does a search engine profit from it while avoiding editorial bias?

Before discussing the significance of this trend and what it means to you, let me first explain some of the terminology. The search engines have fallen into three primary paid "submission" models: Paid Inclusion, Pay-Per-Click, and Pay-Per-Review.

1. Paid Inclusion - Most of the search engine "crawlers" have added a paid inclusion option to their submission pages. Crawlers are the engines that will use an automated robot, or spider, to crawl your pages. Engines like AlltheWeb, Inktomi, AltaVista, and Google are examples of spider or crawler-based engines. Each will attempt to index the entire content of your page as well as consider other factors like incoming links.

It should be noted that Google so far has refused to join the paid inclusion bandwagon. They prefer to avoid a perception of giving preferential treatment to Web sites that pay them versus those that don't. Instead, they wish to rank a page based on the merit of its content and links only. Whether they will stick to this policy over the long haul still remains to be seen. Note: Google recently announced a new pay-per-click twist to their AdWords text ad service. I'll try to cover this in more detail next month.

The idea that paying a search engine for inclusion will automatically increase your rank is a common misconception. All the major engines deny that using their paid inclusion options will increase your rank versus those sites that submit using the free option. The main benefit you receive by paying them is that you're guaranteed inclusion, and you'll be indexed much faster than submitting for free. Perhaps most importantly, the spider will crawl your site more frequently. For example, Inktomi promises to index you in 48 hours rather than waiting 3-4 weeks as with the free submissions. It will then continue to re-index your page every 48 hours after that.

Are paid inclusions worth the money even if you're a patient person? They certainly can be. For example, the key to bringing traffic to your site is not simply to be included, but to be ranked near the top of the results. This means taking advice from resources and applying it to your site design and promotional strategies. Being able to test new designs and techniques every 48 hours rather than every three weeks can give you a definite competitive edge.

Be aware that with paid inclusion, you pay for each URL you want included rather than paying once for your entire Web site. Since that can be expensive for large sites, rather than paying for inclusion of all URLs on your site, consider paying for just a few strategic pages. You could use these pages to test your optimization ideas and verify that they are working for you and generating traffic. Once you know for certain what works for you, you can apply those concepts to your entire site and submit the remaining pages via the free option.

If you have the budget, you can pay for inclusion of additional pages and measure your return on investment. Ideally this means measuring how many dollars you generate from each listing. However, if you don't have the means to track sales to each page, then analyzing the number of visitors to each page should also give you a good idea of its effectiveness. The question then becomes how much search engine traffic are you getting to each page?

The amount of traffic you receive to a page will vary depending on the keywords you target and the rankings you achieve. If you target a keyword that is searched infrequently, your traffic will be minimal. Furthermore, if you target a keyword that is TOO popular, you may find it overly difficult and time-consuming to achieve a top ranking.

WordTracker keeps a database on how frequently keywords are searched and can rank them by popularity. More importantly, it will help you find keywords that are searched for often but are not so competitive as to be next to impossible to achieve a top ranking. It's by far the best service of its kind that I've seen to date.

2. Pay-Per-Click (PPC) - Unlike crawlers offering paid inclusion options, PPC engines rank Web sites by the highest bidder. Whoever bids the highest for a given keyword ranks the highest. You don't actually pay for the ranking, but instead pay only for the number of clicks generated by that listing. In other words, if you bid 25 cents for the keyword "purple widgets," then your account will be charged 25 cents each time someone clicks on your listing and visits your site.

Overture (formerly named Goto) remains the king of all PPC engines. They chose years ago to develop an engine that is based on an entirely different concept than traditional search engines. Crawlers rank sites based on indexing the actual content of the page and certain other factors such as link popularity. Overture on the other hand reasoned that those willing to pay more for a top ranking most likely had a higher caliber of content to offer. The reasoning goes that the more successful and popular a Web site is, the more it can afford to pay. If a Web site is successful, then it has to be doing something right.

For some types of searches, PPC engines can yield reasonably good results. However, for many other searches, you'll not find the best matches to your query in my opinion. For example, you may have a site giving away hard to find, quality information for free. This may be a fantastic bargain for you as the consumer. However, that site may not be in a position to pay Overture 50 cents per visitor like their competitor. Their more successful competitor may have plenty of cash to spend but charges you more for the exact same information. In this type of scenario, you'd be better off searching an engine like Google who does not bias results by the amount they can pay.

Moreover, a specialized search site like MySimon.com will help locate vendors with the lowest prices for an array of products. In contrast, on Overture, you'll find only the vendors who paid the most to be there. You might reason that those who pay the most for those positions may have the highest overhead, and thus higher prices for the same product. This is not always the case of course because of economies of scale, but it is something to keep in mind when you shop.

The success of the PPC model thus far has not come from delivering the highest quality results to your queries as its proponents would like you to believe. Instead, their success has come from their ability to buy visibility on the most popular crawler and directory based engines that consumers tend to prefer.

The major crawlers are struggling to turn a profit right now. However, Overture has jumped in claiming to be their savior. Overture and some other PPC's are already making a profit, largely since they get paid for every click-through on every listing in their database.

Unlike the crawler engines, Overture doesn't have to worry about setting just the right price for each and every banner ad to maximize their profits. Instead, the PPC's auction style bidding allows the market to set prices in real-time to exactly what they're willing to bear. Better yet, the advertiser only pays for actual visits to their site. If you're a Web site owner, this advertising model is much less risky than paying by the impressions served. Most banner ads are sold by the cost per impression (CPM). Unfortunately, they make few guarantees that you'll receive a certain number of visitors from those impressions. The greater risk lies more with you, the Web site owner.

Overture's ability to share a sizeable percentage of the click-through revenues with its strategic partners led to more total click-throughs for its advertisers. The more click-throughs a PPC delivers, the more upside potential for the advertisers and for its search partners. This led to the signing of even more partners and more advertisers for Overture, thus more paid click-throughs. You'll now find Overture and other PPC listings near the top of the search results for most major engines. The other PPC's now face an uphill battle to dislodge Overture's leadership position.

The PPC engines make it clearly known that their listings are based on the highest bidder by displaying the bid prices next to each listing on their site. The non-PPC engines, however, have taken flack for doing a poor job of differentiating the paid listings from the unpaid. While some engines have kept the paid listings in a separate area of the page clearly labeled as "Sponsored listings," others have pushed the envelope by using vague terms like "Featured listings." Some have tried to an extent to blend the results in with the unpaid and theoretically unbiased search results. You may have heard how Ralph Nader's consumer watchdog group has filed a lawsuit against many of the major engines for deceptive advertising practices.

Thus far no legal ruling has been made as to whether the engines are breaking consumer protection laws. Whether they have crossed that legal line I cannot say. I'm not an attorney. However, I do believe that the consumer deserves to know whether a listing is a paid advertisement or is regular content. Ultimately, the courts will decide if the engines are doing a satisfactory job in this department.

I'm sometimes asked "why bother optimizing your page if others can simply buy their way to the top via these PPC listings?" Well, studies show that people tend to skip over those listings and click more often on the crawler-based search results. Therefore, you could have three or four paid entries above your free listing and still gain more traffic from the "freebie." It's unclear whether this is due to the engines doing a sufficient job of identifying those listings that are paid advertisements. If they were doing that, this would logically cause some people to tune them out like commercials. Alternatively, there may be another explanation for the lower click-through rate of the PPC listings. It may be due to fact that the paid placements simply are not giving the consumer results that are as relevant to their query as the unpaid matches. More than likely, it is some combination of the two factors and varies depending upon the keyword searched.

In the last year or so we've seen the conversion of major sites like Excite, NBCi, and Disney's Go.com from a crawler or directory engine to an Overture clone. This begs the question of whether Overture and others will continue their march until all the major engines serve only PPC listings. Although the dot com shake-out could force other engines to serve PPC results on their path to bankruptcy, I predict you will not see them all turn to this model.

How many more times have you found what you are looking for on Google or Yahoo rather than Overture? Only when an engine is fighting bankruptcy and can no longer afford to maintain their own content do they agree to serve PPC results exclusively. It's the next best thing to simply shutting off a popular domain name still receiving traffic. As soon as this happens though, the portal starts to quickly lose popularity as consumers realize the change.

For many types of searches, a crawler or directory-based engine simply does a better job than a PPC. So long as there is that continuing need by the consumer, the free-market will offer a reward to those that fulfill that niche. That's why we've seen Google, a relatively small privately held company rise from total obscurity to become one of the most popular engines today in just a few short years. They did an excellent job of fulfilling the need for crawler-based searches.

Conclusion? The PPC's can certainly be an effective supplement to your advertising strategy. They are also not going away since they are more efficient at bringing in advertising revenues for the engines. PPC's are also less risky to the Web marketer than using a CPM based advertising campaign. Therefore, they'll continue to be favored over CPM models by many advertisers. That's why search engine optimizers monitor your rankings on all the major PPC's and provides advice on how to best promote to them.

PPC's should not, however, be considered a replacement for a top ranking in the regular search results. That's because most consumers tend to see those Sponsored listings as advertisements, and thereby less trustworthy than the crawler-based results.

For more information about PPC engines see: http://www.paypermaster.com/ppm/index.html

3. Pay-per-review - This model has been adopted by two of the top human reviewed directory-based sites: Yahoo and Looksmart. Years ago you could submit to Yahoo and Looksmart for free. Unfortunately, Yahoo soon became backlogged in review requests to the point where the odds of you getting a review by one of their editors was slim. Many Web marketing veterans remember the days when they'd have to submit to Yahoo and wait six weeks in the almost desperate hope of a review and inclusion by a Yahoo editor.

It was often frustrating and maddening to get into Yahoo at this time to say the least. Quite often you would have to submit two or three times over a period of several months before being reviewed. Times change and now you can be reviewed in seven days guaranteed, although you'll have to pony up $299 for the privilege. Unlike the paid inclusion engines, the review sites don't even guarantee that you'll be listed. They only guarantee you'll be reviewed and considered for inclusion if you meet with their standards. Critics contend that you can't separate editorial freedom from the pressure to include a site when businesses are paying you $299 for the review. Those people have the expectation to be added to the directory in exchange for that fee. If you reject too many payers for reasons of editorial control, you risk a decline in those willing to pay you in the future. There's certainly a fine line they must walk.

A common fear from Web businesses operating on tight budgets is "Will free submissions disappear entirely?" These businesses have found the free listings an incredible value over the years and hate to see them disappear.

I've heard a number of people contend that it's now inevitable. Everyone will soon require paid inclusion or placement. However, there is always going to be valuable information on the Web from sites that are not going to be willing to pay for the inclusion of each page on their site. To stop crawling pages for free would mean reducing the size of their index from hundreds of millions of pages to just thousands. In this scenario, the consumer would no longer receive the comprehensive view of the Web that they have come to expect. They would then logically move onto another engine that does provide that service. I believe that's why we've seen AltaVista, Inktomi, AllTheWeb/Lycos, and others continue to offer both free and paid inclusion options rather than mandating the paid service.

Even with the PPC's, you see them drawing results from a free database such as Inktomi to supplement their paid listings. In the case of Yahoo and Looksmart we have seen them acquire a model of primarily reviewing just paid submissions, at least in regard to commercial sites. They say that they still review and include sites for free that their editors find worthy of consideration, although in my opinion it's questionable if they still devote much time to this anymore.

Yahoo and Looksmart continue to allow ways for non-profit sites to be reviewed for free which is good. They get away with doing few free reviews of commercial sites knowing that they offer a high degree of value to those sites. They know that their results are integrated into most of the major crawler and portal sites, thus yielding in most cases more than $299 of advertising value. This assumes you achieve good rankings of course.

Just as Google has continued to buck the trend regarding paid inclusion, Open Directory has chosen to do the same by not charging for its reviews. Open Directory (dmoz.org) was founded on the premise that a directory should be unbiased, comprehensive, and not ruled by the almighty dollar. However, building such a large directory normally takes large sums of money. Their creative solution was to create a network of volunteer editors to do the reviews for free. Each editor is in charge of one or more topics they have a special interest and expertise in. The idea is that they would do it out of a love and passion for their favorite interest. Amazingly, over 46,000 editors from around the world have volunteered their time to the Open Directory project thus far.

This system has allowed thousands of sites to submit and be reviewed for free. It has also led to the creation of a very comprehensive catalog of sites not based on whether they could afford to pay or not. Unfortunately, it has also encouraged many volunteers to join with their own agenda in mind. Open Directory continually battles complaints regarding how volunteer editors have demonstrated a bias toward their own Web sites or against those of their competitors. In Open Directory's defense, they do have an elaborate system in place to discourage, albeit not completely eliminate, such practices.

I also periodically hear someone ask, "Isn't the trend toward paid submissions making search engine optimizing obsolete?" My response is quite the opposite. In the case of paid inclusion, you still must optimize your pages to achieve top rankings. Submitting alone is still not enough whether you are doing it for free or paying for inclusion. The stakes are simply higher when you're opting to pay for faster or better service.