The rise of Vertical Search Engines (VSEs)

There is a quiet revolution happening in Internet Search. Like a bunch
of tiny Lilliputians tying up a sleeping Gulliver, there is a new breed
of small, vertical-focused search startups in Silicon Valley and elsewhere
that are beginning to nibble at Google's search empire. In specific
domains like job search, shopping and health care, companies like
Simply Hired, Become.com and Healthline respectively are beginning to
attract users away by providing a richer user experience within their
specific niches.

Even though typical users tend to head over to one of the top search
engines (Google, Yahoo, MSN or AOL) by default - using search terms
as a starting point when looking for anything on the Internet - in reality,
most of these users are actually trying to solve a problem: find a job,
buy a car, shop for an IPod, arrange a vacation, and so on. Only a
fraction of search users are doing old-fashioned research, in the sense
of a text search among a bunch of documents - something that the
"single search box" that we're all used to, is very good at.

For the remaining majority of users, this problem-solving becomes
much easier when one applies a context, a solution-space, to limit the
scope of the search. Suddenly, you understand keywords better, and
you can apply metadata filters to aid in searching. The focus shifts to
returning trusted, authoritative results and to providing a whole host
of services, tools and community to go with them.

More specifically, a Vertical Search Engine can provide many distinctive
advantages in a given domain, over a general-purpose search engine.
Some of these are listed below:

• Authoritative results

  Since VSEs commonly use data from partner feeds, trusted sites
  and the user community to generate, qualify or complement crawl
  results, the overall data quality can be much higher

• Relevant results

  Results are specific to the vertical, and usually qualified by the
  focused deep understanding of that vertical

• Related non-search content, tools and services
  1. Contextually-rich content
  2. Specialized tools, integrated with the content
  3. Active communities
  4. Research, comparison and configuration tools
  5. Domain-specific filters
  6. Related non-text content (e.g. audio, video)
• The advantage of partnerships with related players in the vertical
  area (data feeds, relevant links, tools, co-marketing, co-branding)
• Specialized user experience, including UI metaphors and
  search algorithms
• Specialized metadata and integration with related data sets
• Suggestions and viral propagation
• From a marketing perspective, prospects are far more qualified; so
  the ad matches are a lot better

There are already a number of fairly successful, but small, companies
in this space in key domains, in addition to the three already mentioned.
A list of the more successful Vertical Search Engines is maintained
at the companion web site, www.SoftwareAbstractions.com .

It is not all roses, however. Even if they can provide a better user
experience in a specific context, the major challenge these players face
is the marketing problem of getting the word out. Given their relatively
small size and minuscule marketing budgets (compared to, say, Google
or Yahoo), the goals of significantly expanding their user base and of
staying in the collective consciousness - so that a potential user would
automatically think of them when attempting to solve a particular
problem in their domain - are not easy to achieve. Having the best
solution in the world does not help if users don't remember to come to
you. Of course, each of these VSEs will undoubtedly garner a loyal core
following that will hang on to every word in their announcements, and
evangelize these sites to their friends, but that's still a drop in the
bucket compared to the overall base of qualified users. This is especially
difficult if these engines solve problems that are common and difficult,
but occur somewhat infrequently for any particular user - such as
finding a job, buying a house or searching for a medication.

The likely possibility for some of the more popular VSEs is that they
will get swallowed up, either by one of the major search engines
making a vertical play, or by one of the major non-search companies
in their particular domain. Either of these outcomes would add the
financial muscle needed for large-scale marketing campaigns.

Google, of course, is already moving aggressively into this area with a
variety of initiatives: Google Co-op (trusted content from partners),
Google Base (uploaded data with attributes), Topic-specific searches,
integrated applications like Google Maps, and so on. And future
acquisitions or partnerships with specialized search engines in key
verticals seem to be a definite possibility.

Having said that, so far Google has always been a search generalist
at heart, not a problem-solver (although that could change), so it's
not clear how deeply they will get into this area. Even Google Base
seems to be a play focused more on acquiring structured, trusted
data feeds, rather than a pure vertical play.

Are VSEs the classic Long Tail  of Search? A vertical search engine
provides great depth in a given vertical, but is only useful for a
subset of users - so it fits the basic concept. Moreover, it is quite
possible that the total searches across the various key verticals could
eclipse the total searches at the top three general search engines
in the future. Whether that holds true will become increasingly clear
as these specialized engines become more popular.

What does seem clear, though, is that in terms of Search Engine
technology, we're still at Search 1.0. Or, as Frank Sinatra
sang, in his inimitable way - "The best is yet to come!"

---

Comments

[ncrawler]

What about directories like Yahoo, dmoz and
botw - where do they fit in?

[VK]

Very interesting article Nitin. I would be interested in your thoughts about Retrevo.

[NitinK]

Thank you for your interest, VK!

Very cool! I took a quick look at retrevo [searching for a Nikon L2, which I just bought :-)], and loved the manual showing up on the right side. I think the Preview functionality is a terrific feature. It's also great to see the integrated Blog Search, which is especially great for reviews!

I will have to take a more detailed look at retrevo - perhaps I'll post a review.

[ssoooo]

I think the vertical search engine's future is
chinese vertical search engine index www.ssoooo.com . When this website collage all
vertical search engine in their website. Then what's the different with vertical search engine and general search.

[Vidal]

Hi, I've just finished my new vertical job search engine JobGeni http://www.jobgeni.com that runs on Google AJAX Feed API. It's pulls the data from several major jobsites like indeed, simplyhired, yahoo hotjobs, monster and jobster.