Search Engine Optimization Is a Negotiation But ? | SEO Theory

As complaints about Google?s business practices mount up again (despite reports that the Federal Trade Commission is about to let Google off the hook once again) it seems worthwhile to sort out the unrealistic expectations from the challenges of doing business with a predatory company.

Predatory business practices are not easily defined. That is, there is no theory that predicts when or how a practice slips into the predatory grade of consumer-unfriendly practices. But in business there are typically only two partners: the provider and the purchaser of goods and/or services.

The Searchable Web Ecosystem, however, has three partners: the Publishers, the Indexers, and the Searchers. This three-way exchange of value works differently from a typical two-way exchange of value.

To the Searchers the commodity is the content that is published and the providers are the Indexers.

To the Publishers the commodity is the searchers and the providers are the Indexers.

To the Indexers there is no commodity ? they are simply brokering exchanges between Searchers and Indexers.

However, (and Google is not the first search engine to do this) when an Indexer becomes a Publisher it competes directly with the other Publishers it is indexing; and when an Indexer simply republishes information originated by other Publishers it becomes a Searcher.

Google is therefore attempting to fulfill all three roles in the Searchable Web Ecosystem: it is Searching for information that it can Publish itself through its Index.

We see this happening in numerous types of queries: definitions, weather inquiries, product pricing queries (especially in travel and accommodation), business reviews, and more. When Google publishes answers directly for consumer informational queries it deprives Publishers who have originated that information of opportunities to engage with and sell to consumers. Of course, Google points out that these are informational queries ? oversimplifying the transaction between Searcher and (Information) Provider by pretending that the sales pitch is not of value to the consumer.

There are, of course, some queries where Google integrates its own merchant-friendly services into the search results so that if consumers want to take the step of initiating a purchase they can interact directly with merchants. But Google strives to eliminate the middle men ? the lead generation and/or affiliate commission Websites.

All three parties have a stake in this exchange. Let?s see if we can break down their pros and cons, strengths and weaknesses, and see where the equilibrium point should be.

Publishers Create Value And Compete With Similar Value

Two or more merchants may offer the same or extremely similar products and services. Consumers may choose between the merchants on the basis of price, performance, or brand loyalty.

However, in order to reach out to consumers some merchants build affiliate networks with Websites that are capable of reaching consumers through channels the merchant Websites are not designed to managed. 10-15 years ago there were more online channels of discovery for consumers than there are today. Through consolidation of technology and services most consumers tend to use Google and/or Bing-powered search tools for discovery.

However, after discovery consumers may use different channels to return to merchants including personal bookmarks, advertising they see across the Web, and social media outreach. Merchants therefore deal with customers in DISCOVERY or POST-DISCOVERY relationships.

Affiliates and lead generators tend to rely on search engine optimization to find new consumers in the DISCOVERY phase; but there are some aggressive third-party marketers who go after the brand-oriented consumers. The shadiest of these operators are trying to deceive consumers into thinking they are the brands, or at least closely associated with them.

So the search environment is confusing to the consumers: they must find merchants, pick from multiple merchants, and decide if they want to go through middle men.

The middle men can create unique additional value and ask for consumer engagement as a reward for creating that unique additional value or they can simply replicate the value provided by the merchants and just try to get a slice of the pie. I know from many discussions with merchants who were frustrated by their own affiliate marketing success that after they achieve brand recognition many of them do NOT want to be outranked by their affiliates in brand-related queries.

So merchants who ally themselves with affiliate marketers wants to have it both ways: they want consumers to come directly to them because they are known brands AND they want other Websites to help them find new customers.

From the merchant perspective, affiliates and lead-generators should not be playing in the POST-DISCOVERY market. Yes, you can argue for exceptions but this is the rule of the ecosystem, not a point-by-point breakdown of who does what.

The bottom line here is that consumers (Searchers) may indeed be confused by the similarity between merchants and between merchants and their affiliates. Affiliates should seek the best fit in the DISCOVERY market where they can complement the merchants? own efforts.

Searchers? Needs and Requirements Change

In some markets there is a real need for more than 3 major vendors of goods and/or services. Economies of scale make it easy for a small number of vendors to dominate markets with moderate quality goods and services. They eschew the higher costs and lower profits of refining goods and services to meet specific consumer needs. Hence, these types of markets spawn many smaller providers who will never scale up to become one of the BIG GUYS (although they are often bought out after demonstrating sufficient success).

Start up, scale up, and sell out is a common formula in manufacturing, entertainment, software, and many other industries. It?s just a part of the normal market cycle. Consumers are comfortable with this. They love and support niche markets. You can find thousands of Web forums dedicated to alternative products and services for lifestyles, health, nutrition, fitness, and many other areas of consumer interest. Consumers may want to save money, help the environment, manage health-related issues, support political causes, support charitable organizations, etc. They have many reasons to seek out alternative providers of goods and services.

Feelings of dissatisfaction and saturation contribute to searches for alternate providers of goods and services. Hence, many consumers change their searching habits. Either they refuse to buy from certain providers again or they just want to find something else to see what else is available. Hence, there will always be some segment of the consumer population that is in the DISCOVERY part of the market for most industries.

It is these DISCOVERY-minded consumers who create opportunity for alternative vendors, lead generators, and affiliates. If the lead generators and affiliates can help these consumers reconnect with known brands, it is by persuading them to look at the value provided by the known brands in different ways.

?I didn?t know I could do that with Super Gummy Shoelaces!? is the cry of the consumer who is reconnecting with a brand through an alternative information source. Of course, there is nothing to prevent a brand from creating the alternative information source. In fact, brands that use social media monitoring tools are trying to figure out what alternative information they should be supplying to consumers.

So while the consumer appetite for new information about existing brands is almost as insatiable as the appetite for new brands, consumers won?t care if the new information comes from the brands or alternative sources. I have not seen any research that looks at whether consumers would trust the information more if it comes from the brands or the alternative sources; my guess is there would be qualifying factors to favoring both groups as information sources.

Nonetheless, the message here is that consumers don?t need middle men if they can connect with brands in a satisfying way. The brands are beginning to realize this but they still have to be profitable. In fact, large publicly traded companies are expected to make profits, so their tolerance for unprofitable experimentation tends to be lower than the collective tolerance of groups of alternative providers (who individually lack the resources to tolerate much unprofitable experimentation).

Experimentation and discovery go hand in hand, but the process is much less expensive for consumers than for providers (merchants and middle men).

Indexers Must Provide Value and Sustain It

Indexers have an advantage over both merchants and middle men in the DISCOVERY part of the market as the Indexers are the brokers of information and exchange between consumers and providers. But Indexers have a distinct disadvantage in the POST-DISCOVERY market because consumers only use them for navigation to the providers.

For example, if I have never bought red shoes before and I want to find a provider of red shoes, I can use a search engine like Bing or Google and look for providers of red shoes. I expect to see a selection of manufacturers, retailers, and affiliates. I can pick and choose to my heart?s content and in that process of discovery I may click on a few ads.

So my discovery supports the Indexer by giving them some advertising revenue. My discovery may also support a few middle men if I click on THEIR ads. Ultimately, however, my discovery will only contribute to the sales of 1 provider.

The next time I want red shoes I will only use discovery again if I am unsatisfied with my first purchase or if I want to see what else is available. Otherwise as a satisfied consumer I will go back to the brand I chose. I may use a search engine to help me get there by searching for the brand Website. I may click on an ad for the brand or not.

As long as the Indexer shows me a proper organic result leading to the brand I am happy. The Indexer earns no money, the middle men are cut out of the transaction as well, and I continue to build my relationship with the brand.

Of course, being happy and satisfied with my search experience I will want to return to that Indexer the next time I search for something. The Indexer has become my Search Brand.

In this kind of POST-DISCOVERY transaction the consumer, Indexer, and brand all win and the middle men lose. There is simply no legitimate opportunity for an honest middle man in a POST-DISCOVERY transaction.

The Indexer, however, has to act like a brand while being a broker. This is because search engines have relatively small core audiences. You hear about the huge numbers of people who search on Google every day but the vast majority of those people (80-90% by some estimates) do not search exclusively on Google.

Search engines must compete for searchers and therefore the search engines have to offer unique and specific value. Google may offer a better DISCOVERY experience than Bing and Bing may offer a better POST-DISCOVERY experience than Google. Also, many smaller search services may offer better experiences than either Bing or Google depending on what consumers are searching for.

So where do the middle men ? the lead generators and affiliates ? fit into the search engines? Value Proposition? If consumers find only middle men in the POST-DISCOVERY transaction, how happy will they be with the Indexers? service? If consumers do not find alternative information in the DISCOVERY transaction, how happy will they be with the Indexers? service?

Google (and to a lesser extent Bing and Yahoo!) attempts to smooth the process for the consumer by blending DISCOVERY and POST-DISCOVERY information for consumer searches. They do this by republishing information that is widely available on the Web. While it?s true that no one buys weather information from Wunderground or Weather.com consumers still like these services because of the depth of information they provide. Hence, the Indexers are trying to siphon off just the consumers who want short, immediate answers without in-depth information.

And, of course, in the product queries where the Indexers are organizing preferred merchants they are trying to more efficiently monetize the DISCOVERY transaction but they are also looking for monetization of the POST-DISCOVERY transaction. This secondary monetization is unfair to merchants who now must pay for some limited forms of navigational search.

Organic navigational search remains free, but consumer-incentivized navigational search has proven to be a revenue-generating channel for Indexers. Consumers will buy products if they see pictures and prices, and Indexers can get a cut of that transaction if they provide pictures and prices to consumers.

In this scenario the Indexers become the middle men ? and we have seen that Google is very aggressive about interjecting itself between consumers and merchants. Google shortens the path between consumers and wedding dress by including the pictures and prices directly in search results; Yahoo! (powered by Bing) expects its searchers to click on the SHOPPING tab before showing them pictures and prices. Bing has not yet added a direct shopping channel to its search results.

The competition between search engines to increase the value they provide to searchers is thus bleeding over into the competition between affiliates and lead generators.

So Who Is Entitled To What?

The title of this article says that search engine optimization is a negotiation. Negotiation occurs between two parties who each possess unique value that the other wants. A middle man inserts himself in-between the two parties and he may or may not bring unique value.

Middle men sometimes perform necessary functions. Merchants who lack the expertise to organize their information may rely more on middle men to help present their best information in the right situation. Insurance companies deal directly with consumers but the independent insurance agent industry is not going away any time soon. There is so much information out there about insurance companies and products that many consumers naturally turn to independent agents to help them choose between providers (of course, independent agents cannot represent all providers ? the major brands all have their own agents).

Middle men can also bring unique perspectives to the transaction ? either by bundling products and services from different providers together (Amazon does a great job of that) or by offering supplemental value (often information). Consumers are comfortable with middle men who provide a clear and distinct value proposition.

Search engines can be middle men, though. We have no laws preventing search engines from encroaching upon the affiliate and lead generation markets. In fact, we can easily show that merchants use search engines as free (uncompensated) lead generation channels.

Being the channel means you can ask for compensation, does it not? After all, even monopolistic companies are allowed to charge a profit for goods and services they provide.

The issue thus comes down to who is allowed to participate in the negotiation process. I submit that anyone who offers unique value is entitled to participate. However, other factors may nullify or suspend the right to participate.

The Rules of Negotiation Through Search

Search engines never bothered to ask for anyone?s permission to crawl and index Websites. Although the benefit of being found in a search index is immediately obvious to anyone who generates a revenue that way, there are times when the search engines overstep their boundaries, either through too-aggressive crawling or by sending the wrong traffic to Websites.

Search optimization must address both of these issues by helping Websites to manage crawl and filter out or prevent undesirable search listings. The negotiation between Website and Indexer is concerned with how much crawling is acceptable (and what can be crawled) and which types of listings are mutually useful.

Search engines have no incentive to send you the wrong traffic; sometimes they send a lot of wrong traffic and that hurts their value proposition to searchers just as much as it hurts yours.

On the other hand, merchants never bothered to ask search engines what the limits of aggressive marketing can be. In fact, once the value of being found in search was fully realized marketers began seeking all possible placements for their Websites in search, causing much consumer confusion and frustration.

Every shoe salesman on the Internet wanted to be found in queries for ?red shoes? even though those queries didn?t provide much value for either the salesmen or the consumers. The salesmen assumed they needed to be found in vanity queries; the consumers didn?t want to see sales pitches in informational queries, however.

So consumers reacted to poor search results by abandoning search engines, by changing queries, and by avoiding aggressive merchants or leaving their Websites. This kind of misguided forced transaction is expensive for everyone in that it wastes consumers? time and creates ill will on their part and it wastes resources for Indexers and Publishers.

When search engines began laying down some ground rules merchants didn?t take them very seriously. To this day many online retailers and middleman marketers disregard or poorly understand the guidelines published by the search engines. The secrecy shrouding search algorithms complicates the negotiation. On the one hand it is nearly impossible for merchants to comply with search engine guidelines that are so vague as to be useless; on the other hand search engines cannot afford to give unscrupulous merchants the keys to their algorithms.

These negotiations do not occur in closed rooms or through signed contracts. Rather, the negotiations are conducted in the open in 1-to-many formats. Publishers must manage relationships with multiple Indexers and Indexers must manage relationships with multiple Publishers.

The Limitations of Negotiation Through Search

By trial-and-error we have settled thus far on a crude exchange of information between Indexers and Publishers: the Indexers look for robots.txt, XML sitemaps, and embedded meta directives (but they reserve the right to ignore any and all of these directives); the Publishers ask the Indexers questions and read the guidelines (but they reserve the right to ignore any and all of these hints).

The system works best for Indexers and Merchants and not-so-well for Indexers and Middle Men. Middleman Websites are treated as second class citizens by Indexers because the value and quality offered by the majority of Middle Men is unclear or sufficiently less than that offered by the Merchants that the search engines distrust the Middle Men.

The distrust is not due specifically to violations of search engines? guidelines ? brands violate the guidelines all the time. The distrust is due to the consumer reaction to the low quality and/or unclear value provided by Middle Man sites.

There are some really good lead generation and/or affiliate sites out there. They provide great information, a wonderful consumer experience, and they are genuinely valuable Websites. But the Searchable Web Ecosystem can barely distinguish between Primary and Secondary Providers ? hence, all the Secondary Providers must be treated as the same class regardless of individual behavior.

Merchants are not by nature necessarily any more reliable or honest than lead generators or affiliates. For example, many merchants who build up their customer bases through aggressive advertising infuriate those customers by being unresponsive, by overcharging customers, by setting up automatic charges without making it clear to consumers that these monthly billings will occur, and by frustrating the consumer experience in other ways.

As a consumer there have been times when I have wished for a middle man who would deal more honestly with me so I wouldn?t have to fight with merchant Websites that are obviously configured to extract as much money from consumers as possible regardless of what the consumer?s intentions are. And that illustrates a problem for the Indexers: they have to treat all the Primary Providers as the same class too.

People in the search marketing industry have invested a lot of time and energy in arguing that you need to create brand value, that search engines want to favor brands, etc. The truth is that there are a lot of shady brands out there. Brand recognition doesn?t make you honest, trustworthy, and dependable. Eric Schmidt was just talking like a damn fool when he suggested that favoring brands could help clean up the cesspool of the Web.

What Works Best Is Better Negotiation

The search engine optimization specialist must help the Publisher and Indexers find a mutually beneficial common ground. While I could say that an SEO should be a broker or mediator between the Publisher and Indexer, in reality we are advocates solely for the Publishers. An SEO cannot be anything BUT an advocate for a Publisher in this exchange or negotiation.

The reason is simple: Indexers have too much power in the negotiation. That is a result of market economics. Consumers gravitate toward the channels that are easiest to use and understand. Search engines are great DISCOVERY tools but they are also great NAVIGATION (POST-DISCOVERY tools). You cannot get to Amazon?s Website very easily through Twitter or Facebook.

Unfortunately for consumers Microsoft destroyed the competitive Web browser market. Now we are stuck with poorly designed browsers that provide no commercial incentive to developers to improve the consumer experience. In the late 1990s there were a small number of experiments in creating incentivized browser discovery channels. Those experiments created a poor user experience but after Microsoft inserted Internet Explorer between consumers and developers there was no opportunity left for creating a monetizable browser market.

We can point all sorts of fingers at people for destroying the competitive search market. But probably the consumers themselves were responsible for that. They preferred simplicity and too many people rushed over to Google?s side of the boat. What may happen ? what may topple Google?s dominance in Web search ? is a radical shift in consumer preferences for DISCOVERY, a shift that no search engine can anticipate or control. That shift may come in the mobile app market or it may come in some other form.

Meanwhile, search engine optimizers are faced with the challenge of maintaining relevance in an increasingly simplified world. We are Middle Men in the negotiation process. The negotiation is still defined in technical processes and the terms of the implied contracts are still poorly spelled out.

But the day may come when search engines and providers find a way to engage with each other more directly, to define mutually agreeable terms of interaction, and to manage their relationships efficiently without us. Such an innovation would happen piecemeal, incrementally ? and it will almost certainly be driven by the Indexers.

In fact, the process has begun but it remains to be seen whether the current outreach from Google and Bing will produce that kind of direct interaction between Publishers and Indexers. The reason why it could happen is that there are few enough major search providers that they can create unique rules for interaction that can exist side-by-side. The reason why it should happen is that everyone (but SEOs) will benefit.

So the only question for the SEOs in this negotiation between the present and the future is what role do we want to carve out for ourselves.

Source: http://www.seo-theory.com/2012/12/15/search-engine-optimization-is-a-negotiation-but/

last train to clarksville tim tebow taylor swift post grad arpaio carol burnett neil degrasse tyson neil degrasse tyson