Information Architecture Assignment

jeprej

Information_Architecture_For_the_Web_and_Beyond_----_Chapter_9._Search_Systems.pdf

Home >Information Systems homework help >Information Architecture Assignment

CHAPTER 9

Search Systems

The ultimate search engine would basically understand everything in the world, and it would always give you the

right thing. And we’re a long, long ways from that. —Larry Page

In this chapter, we’ll cover:

• Determining whether your product needs a search system • The basic anatomy of a search system • What to make searchable • A basic understanding of retrieval algorithms • How to present retrieval results • Search interface design • Where to learn more

Chapter 8 helped you understand how to create the best navigation system possible for your information environment. This chapter describes another form of finding information: searching. Searching (and more broadly, information retrieval) is an expansive, challeng‐ ing, and well-established field, and we can only scratch the surface here. We’ll limit our discussion to what makes up a search system, when to implement search systems, and some practical advice on how to design a search interface and display search results.

This chapter often uses examples of search systems that allow you to search various different types of information environments, ranging

211

Rosenfeld, Louis, et al. Information Architecture : For the Web and Beyond, O'Reilly Media, Incorporated, 2015. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/unt/detail.action?docID=4333758. Created from unt on 2023-02-19 06:01:55.

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

from the entire Web to mobile phone apps. Although these tools tend to index a very broad collection of content, it’s nonetheless extremely useful to study them.

Does Your Product Need Search? Before we delve into search systems, we need to make a point: think twice before you make your product searchable.

Your information environment should, of course, support the find‐ ing of its information. But as the preceding chapters demonstrate, there are other ways to support finding. And be careful not to assume, as many do, that a search engine alone will satisfy all users’ information needs. While many users want to search, some are nat‐ ural browsers, preferring to forgo filling in that little search box and hitting the “search” button. We suggest you consider the following issues before committing to a search system:

Amount of content in the information environment How much content is enough to merit the use of a search engine? It’s hard to say. It could be 5, 50, or 500 content items; no specific number serves as a standard threshold. What’s more important is the type of information need that’s typical of your product’s users. For example, users of a technical support web‐ site often have a specific kind of information in mind, and are more likely to require search than users of an online banking app. If your product is more like a library than a software appli‐ cation, then search probably makes sense. If that’s the case, then consider the volume of content, balancing the time required to set up and maintain a search system with the payoff it will bring to your product’s users.

Focus on more useful navigation systems Because many developers see search engines as the solution to the problems users have when trying to find information in their products, search engines become Band-Aids for poorly designed navigation systems and other architectural weak‐ nesses. If you see yourself falling into this trap, you should prob‐ ably suspend implementing your search system until you fix your navigation system’s problems. You’ll find that search sys‐ tems often perform better if they can take advantage of aspects of strong navigation systems, such as the controlled vocabulary terms used to tag content. And users will often benefit even

212 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

more from using both types of finding if they work together well. Of course, your product’s navigation might be a disaster for political reasons, such as an inability among your organiza‐ tion’s decision makers to agree on a system-wide navigation sys‐ tem. In such cases, reality trumps what ought to be, and search might indeed be your best alternative.

Time and know-how to optimize the search system Search engines are fairly easy to get up and running, but they are difficult to implement effectively. As a user of the Web, you’ve certainly seen incomprehensible search interfaces, and we’re sure that your queries have retrieved some pretty inscruta‐ ble results. This is often due to a lack of planning by the site’s developer, who probably installed the search engine with its default settings, pointed it at the site, and forgot about it. If you don’t plan on putting some significant time into configuring your search engine properly, reconsider your decision to imple‐ ment it.

Other alternatives Search may be a good way to serve your product’s users, but other ways may work better. For example, if you don’t have the technical expertise or confidence to configure a search engine or the money to shell out for one, consider providing an index instead. Both indexes and search engines help users who know what they’re looking for. While an index can be a heck of a lot of work, it is typically created and maintained manually, which makes it easier to implement. You could also provide access to a third-party search engine, such as Google’s. (While this is a cost- effective alternative, it has downsides: for one, search becomes separate from other means of finding, leading to a disjointed experience. For another, delegated search can’t generate the same data—and insights—from search analytics.)

Users’ preferred ways of interacting It may already be clear that your users would rather browse than search. For example, users of a handmade crafts site may prefer browsing thumbnails of cards instead of searching. Or perhaps users do want to search, but searching is a lower priority for them, and it should be for you as you consider how to spend your information architecture development budget.

Does Your Product Need Search? | 213

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Now that we’ve got our warnings and threats out of the way, let’s dis‐ cuss when you should implement search systems. Many information environments—websites, especially—aren’t planned out in much detail before they’re built. Instead, they grow organically. This may be all right for smaller systems that aren’t likely to expand much, but for ones that become popular, more and more content and func‐ tional features get piled on haphazardly, leading to a navigation nightmare. The following issues will help you decide when your environment has reached the point of needing a search system:

Search helps when you have too much information to browse There’s a good analogy with physical architecture here. Powell’s Books, which claims to be the largest bookstore in the world, covers an entire city block (68,000 square feet) in Portland, Ore‐ gon. We guess that it started as a single small storefront on that block, but as the business grew, the owners knocked a doorway through the wall into the next storefront, and so on, until it occupied the whole block. The result is a hodgepodge of cham‐ bers, halls with odd turns, and unexpected stairways. This cha‐ otic labyrinth is a charming place to wander and browse, but if you’re searching for a particular title, good luck. It will be diffi‐ cult to find what you’re looking for, although if you’re really lucky you might serendipitously stumble onto something better.

Yahoo! once was a web version of Powell’s. At first, everything was there and fairly easy to find. Why? Because Yahoo!, like the Web, was relatively small. At its inception, Yahoo! pointed to a few hundred Internet resources, made accessible through an easily browsable subject hierarchy. No search option was avail‐ able, something unimaginable to Yahoo! users today. But things soon changed. Yahoo! had an excellent technical architecture that allowed site owners to easily self-register their sites, but Yahoo!’s information architecture couldn’t keep up with the increasing volume of resources that were added daily. Eventu‐ ally, the subject hierarchy became too cumbersome to navigate, and Yahoo! installed a search system as an alternative way of finding information. In 2014, Yahoo! discontinued its browsable site directory altogether.

Your information environment probably isn’t as large as Yahoo!, but it’s probably experienced a similar evolution. Has your con‐ tent outstripped your browsing systems? Do your site’s users go

214 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

http://www.powells.com

insane trying to spot the right link on your hugely long category pages? Then perhaps the time has come for search.

Search helps fragmented sites Powell’s room after room after room of books is also a good analogy for the silos of content that make up so many intranets and large public websites. As is so often the case, each business unit has gone ahead and done its own thing, developing content haphazardly with few (if any) standards, and probably no meta‐ data to support any sort of reasonable browsing.

If this describes your situation, you have a long road ahead of you, and search won’t solve all of your problems—let alone your users’ problems. But your priority should be to set up a search system to perform full-text indexing of as much system content as possible, even across such traditional silos as company departments. Even if it’s only a stopgap, search will address your users’ dire need for finding information regardless of which business unit actually owns it. Search will also help you to get a better handle on what content is actually out there.

Search is a learning tool Through search-log analysis, which we touched on in Chap‐ ter 7, you can gather useful data on what users actually want from your information environment, and how they articulate their needs (in the form of search queries). Over time, you can analyze this valuable data to diagnose and tune your search sys‐ tem, other aspects of its information architecture, the perfor‐ mance of its content, and many other areas as well.

Search should be there because users expect it to be there Your product probably doesn’t contain as much content as Yahoo!, but if it’s substantial, it probably merits a search engine. There are good reasons for this. Users won’t always be willing to browse through its structures; their time is limited, and their cognitive-overload threshold is lower than you think. Interest‐ ingly, sometimes users won’t browse for the wrong reasons— that is, they search when they don’t necessarily know what to search for and would be better served by browsing. But perhaps most of all, users expect that little search box wherever they go. It’s a default convention, and it’s hard to stand against the wave of expectations.

Does Your Product Need Search? | 215

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Search can tame dynamism You should also consider creating a search system for your product if it contains highly dynamic content. For example, an online newspaper might be adding dozens of story files daily via a commercial newsfeed or some other form of content syndica‐ tion. For this reason, its team probably wouldn’t have the time each day to manually catalog its content or maintain elaborate tables of contents and site indexes. A search engine could help by automatically indexing the contents of the site once or many times daily. Automating this process ensures that users have quality access to the newspaper’s content, and the team can spend time doing things other than manually indexing and link‐ ing the story files as they come in.

Search System Anatomy On its surface, search seems quite straightforward. Look for the box with the search button, enter and submit your query, and mutter a little prayer while the results load. If your prayers are answered, you’ll find some useful results and can go on with your life.

Of course, there’s a lot going on under the hood. A search engine application has indexed the content of the information environ‐ ment. All of it? Some of it? As a user, you’ll probably never know. And what parts of the content? Usually the search engine can find the full text of each document. But a search engine can also index information associated with each document—like titles, controlled vocabulary terms, etc.—depending on how it’s been configured. And then there’s the search interface, your window on the search engine’s index. What you type there is looked up in the index; if things go well, results that match your query are returned.

A lot is going on here. There are the guts of the search engine itself; aside from tools for indexing and spidering, there are algorithms for processing your query into something the software can understand, and for ranking the results. There are interfaces, too: ones for enter‐ ing queries (everything from simple search boxes to advanced natural-language, voice-driven interfaces like Siri) and others for displaying results (including decisions on what to show for each result, and how to display the entire set of results). Further compli‐ cating the picture, there may be variations in query languages (e.g., whether or not Boolean operators like AND, OR, and NOT can be

216 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

used) and query builders (e.g., spell checkers) that can improve upon a query.

Obviously, there’s a lot to search that doesn’t meet the eye. Addition‐ ally, there’s your query, which itself usually isn’t very straightfor‐ ward. Where does your query come from? Your mind senses a gap that needs to be filled with information, but isn’t always sure how to express what it’s looking for. Searching is often iterative—not just because we don’t always like the results we retrieve, but often because it takes us a few tries to get the words right for our query. You then interact with a search interface, heading for the simple, Google-like box or, if you’re “advanced,” grappling with the advanced search interface. And finally, you interact with the results, hopefully quickly determining which results are worth clicking through, which to ignore, and whether or not you should go back and try modifying your search. Figure 9-1 shows some of these pathways.

Figure 9-1. The basic anatomy of a search system (image adapted from Search Patterns: Design for Discovery, by Peter Morville and Jeffery Callender)

That’s the 50,000-foot view of what’s happening in a search system. Most of the technical details can be left to your IT staff; you are more concerned with factors that affect retrieval performance than with the technical guts of a search engine. That said, it’s important that the team responsible for the environment’s information archi‐ tecture be part of the search system selection and implementation processes. The team must be prepared to argue strongly for owning at least an equal responsibility for selecting and implementing the search engine that will best serve users, rather than the one that runs on someone’s favorite platform or is written in someone’s favorite programming language.

Search System Anatomy | 217

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Choosing What to Index Let’s assume that you’ve already chosen a search engine. What con‐ tent should you index for searching? You can point your search engine at your content, tell it to index the full text of every docu‐ ment it finds, and let it do its thing. That’s a large part of the value of search systems—they can be comprehensive and can cover a huge amount of content quickly.

But indexing everything doesn’t always serve users well. In a large, complex environment chock-full of heterogeneous subsystems and databases, you may want to allow users to search the silo of technical reports or the staff directory without muddying their search results with the latest HR newsletter articles on the addition of fish sticks to the cafeteria menu. The creation of search zones—pockets of more homogeneous content—reduces the apples-and-oranges effect and allows users to focus their searches.

Choosing what to make searchable isn’t limited to selecting the right search zones. Each document or record in a collection has some sort of structure, whether rendered in a markup language like HTML or database fields. In turn, that structure stores content components: pieces or “atoms” of content that are typically smaller than a docu‐ ment. Some of that structure—say, an author’s name—may be lever‐ aged by a search engine, while other parts—such as the legal disclaimer at the bottom of each page—might be left out.

Finally, if you’ve conducted an inventory and analysis of your con‐ tent, you already have some sense of what content is “good.” You might have identified your valuable content by manually tagging it or through some other mechanism. You might consider making this “good” stuff searchable on its own, in addition to being part of the global search. You might even program your search engine to search this “good” stuff first, and expand to search the rest of the content if that first pass doesn’t retrieve useful results. For example, if most of an ecommerce site’s users are looking for products, those could be searched by default, and the search could then be expanded to cover the whole site as part of a revised search option.

In this section, we’ll discuss issues of selecting what should be searchable both at a coarse level of granularity (search zones) and at the more atomic level of searching within documents (content components).

218 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Determining Search Zones Search zones are subsets of an information environment that have been indexed separately from the rest of the content. When a user searches a search zone, he has, through interaction with the envi‐ ronment, already identified himself as interested in that particular information. Ideally, the search zones correspond to his specific needs, and the result is a better search experience. By eliminating content that is irrelevant to his need, the user should retrieve fewer, more relevant, results.

In Windows 8.1, shown in Figure 9-2, users can select search zones based on the type of content they are looking for (Settings, Files) and—somewhat awkwardly—by its location (Web images, Web vid‐ eos), with “Web” implying that the “Settings” and “Files” options refer to settings and files on your computer. (Note that “Every‐ where” is the default selection.) But what if the user wants to search for something other than videos or images on the Web? Or, inver‐ sely, wants to search for videos or images on her computer?

Figure 9-2. Search zones in Windows 8.1

Although both the search box and the search result screen in Win‐ dows 8.1 present a single and consistent user interface for all searches, behind the scenes the system is rendering results from two very different search zones: the user’s computer system in the case of

Choosing What to Index | 219

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

settings and files, and the entire Web (via Microsoft’s Bing search engine) in the case of images and videos.

You can create search zones in as many ways as you can physically segregate documents or logically tag them. Your decisions in select‐ ing your environment’s organization schemes often help you deter‐ mine search zones as well. So, our old friends from Chapter 6 can also be the basis of search zones:

• Content type • Audience • Role • Subject/topic • Geography • Chronology • Author • Department/business unit

And so on. Like browsing systems, search zones allow a large body of content to be sliced and diced in useful new ways, providing users with multiple “views” of the environment and its content. But, natu‐ rally, search zones are a double-edged sword. Narrowing one’s search through search zones can improve results, but interacting with them adds a layer of complexity. So be careful: many users will ignore search zones when they begin their searches, opting to enter a simple search against the global index. Users might not bother with your meticulously created search zones until they’re taking their second pass at a search, via an advanced search interface.

Following are a few ways to slice and dice.

Navigation versus destination Most content-heavy information environments contain, at mini‐ mum, two major types of pages or screens: navigation pages and des‐ tination pages. Destination pages contain the actual information you want: sports scores, book reviews, software documentation, and so on. Navigation pages may include main pages, search pages, and pages that help you browse the environment. The primary purpose of navigation pages is to get you to the destination pages.

220 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

When a user searches an information environ‐ ment, it’s fair to assume that he is looking for destination pages. If navigation pages are included in the retrieval process, they will just clutter up the retrieval results.

Let’s take a simple example: your company sells electronics accesso‐ ries via its website. The destination pages consist of descriptions, pricing, and ordering information, one page for each product. Also, a number of navigation pages help users find products, such as list‐ ings of products for different device types (e.g., tablets versus smart‐ phones), listings of products for different types of accessories (e.g., screen protectors, cases), and listings of different device manufac‐ turers (e.g., Apple, Samsung, LG). If the user is searching for Mophie iPhone cases, what’s likely to happen? Instead of simply retrieving the Mophie’s product page, she might have to wade through all of these pages:

• iPhone cases index page • External batteries index page • Apple devices products index page • Mophie products index page • Android products index page • Mophie iPhone products index page

The user’s search retrieves the right destination page (i.e., the Mophie iPhone product page), but also five more that are purely navigation pages. In other words, 83% of the retrieval obstructs the user’s ability to find the most useful result.

Of course, indexing similar content isn’t always easy, because “simi‐ lar” is a highly relative term. It’s not always clear where to draw the line between navigation and destination pages—in some cases, a page can be considered both. That’s why it’s important to test out navigation/destination distinctions before actually applying them. The weakness of the navigation/destination approach is that it is essentially an exact organization scheme (discussed in Chapter 6) that requires the pages to be either destination or navigation. In the following three approaches, the organization schemes are ambigu‐ ous, and therefore more forgiving of pages that fit into multiple categories.

Choosing What to Index | 221

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Indexing for specific audiences If you’ve already decided to create an architecture that uses an audience-oriented organization scheme, it may make sense to create search zones by audience breakdown as well. We found this a useful approach for the original Library of Michigan website.

The Library of Michigan has three primary audiences: members of the Michigan state legislature and their staffs, Michigan libraries and their librarians, and the citizens of Michigan. The information needed from this site is different for each of these audiences; for example, each has a very different circulation policy.

So we created four indexes: one for each of the three audiences, and one unified index of the entire site in case the audience-specific indexes didn’t do the trick for a particular search. Table 9-1 shows the results from running a query on the word “circulation” against each of the four indexes.

Table 9-1. Query results

Index Documents retrieved Retrieval reduced by

Unified 40 —

Legislature area 18 55%

Libraries area 24 40%

Citizens area 9 78%

As with any search zone, less overlap between indexes improves per‐ formance. If the retrieval results were reduced by a very small figure—say, 10% or 20%—it might not be worth the overhead of cre‐ ating separate audience-oriented indexes. But in this case, much of the site’s content is specific to individual audiences.

Indexing by topic The Mayo Clinic employs topical search zones on its website. For example, if you’re looking for a doctor to help with your rehabilita‐ tion, you might select the “Doctors & Medical Staff ” search zone, as shown in Figure 9-3.

The 88 results retrieved may sound like a lot, but if you’d searched the entire site, the total would have been 1,470 results, many dealing with topic areas that aren’t germane to identifying a physician.

222 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-3. Executing a search against the “Doctors & Medical Staff” search zone

Indexing recent content Chronologically organized content allows for perhaps the easiest implementation of search zones. (Not surprisingly, it’s a common example of search zones.) Because dated materials aren’t generally ambiguous and date information is typically easy to come by, creat‐ ing search zones by date—even ad hoc zones—is straightforward.

The search interface of the New York Times provides a useful illus‐ tration of filtering by date range (Figure 9-4).

Regular users can return to the site and check up on the news using one of a number of chronological search zones (e.g., today’s news, past week, past 30 days, past 90 days, past year, and since 1851). Additionally, users who are looking for news within a particular date range or on a specific date can essentially generate an ad hoc search zone.

Choosing What to Index | 223

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-4. There are many ways to narrow your New York Times search by date

Selecting Content Components to Index Just as it’s often useful to provide access to subsets of your site’s con‐ tent, it’s valuable to allow users to search specific components of your documents. By doing so, you’ll enable users to retrieve more specific, precise results. And if your documents have administrative or other content components that aren’t especially meaningful to users, these can be excluded from the search.

In the Yelp business listing shown in Figure 9-5, there are more con‐ tent components than meet the eye. There is a business name, oper‐ ating hours, images, a link to the business’s website, and some attributes that are invisible to users. There are also content compo‐ nents that we don’t want to search, such as the reviews and tips toward the bottom of the screen. These could confuse a user’s search results; for example, if a review included the name of a competing restaurant. (A great by-product of the advent of content manage‐ ment systems and logical markup languages is that it’s now much easier to leave out content that shouldn’t be indexed, like navigation options, advertisements, disclaimers, and other stuff that might show up in document headers and footers.)

224 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-5. Yelp’s business listings are jam-packed with various content components, some visible and some not

Yelp’s search system allows users to take advantage of the informa‐ tion environment’s structure, supporting searches by the following content components, among others:

• Business name • Categories (e.g., Burgers, American) • Ambiance and attire (e.g., casual, formal, etc.)

Choosing What to Index | 225

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

• Noise level • Location

Would users bother to search by any of these components? In Yelp’s case, we could determine this by reviewing search query logs. But what about in the case of a search system that hadn’t yet been imple‐ mented? Prior to designing a search system, could we know that users would take advantage of this specialized functionality?

There is another reason to exploit a document’s structure. Content components aren’t useful only for enabling more precise searches; they can also make the format of search results much more mean‐ ingful. In Figure 9-6, Yelp’s search results include category and list‐ ing titles (“Boulevard Burger,” “Burgers, Breakfast & Brunch”), snippets of reviews (“My wife & I came in last night for dinner...”), number of reviews, average ratings, and locations. Indexing numer‐ ous content components for retrieval provides added flexibility in how you design search results. (See “Presenting Results” on page 233 for more on this topic.)

This leads to a difficult paradox: even if users would benefit from such souped-up search functionality, they likely won’t ever ask for it during initial user research. Typically, users don’t have much under‐ standing of the intricacies and capabilities of search systems. Devel‐ oping use cases and scenarios might unearth some reasons to support this level of detailed search functionality, but it might be better to instead examine other search interfaces that your site’s users find valuable, and determine whether to provide a similar type of functionality.

226 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-6. Title, rating, and location are content components dis‐ played for each result

Search Algorithms Search engines find information in many ways. In fact, there are about 40 different retrieval algorithms alone, most of which have been around for decades. We’re not going to cover them all here; if

Search Algorithms | 227

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

1 A good starting point is Modern Information Retrieval by Ricardo Baeza-Yates and Berthier Ribeiro-Neto (Boston: Addison-Wesley, 2011).

you’d like to learn more, read any of the standard texts on informa‐ tion retrieval.1

We bring up the topic because it’s important to realize that a retrieval algorithm is essentially a tool, and just like other tools, spe‐ cific algorithms help solve specific problems. And as retrieval algo‐ rithms are at the heart of search engines, it’s important to note that there is absolutely no single search engine that will meet all of your users’ information needs. Remember that fact the next time you hear a search engine vendor claim that their product’s brand-new proprietary algorithm is the solution to all information problems.

Pattern-Matching Algorithms Most retrieval algorithms employ pattern matching; that is, they compare the user’s query with an index of, typically, the full texts of your system’s documents, looking for the same string of text. When a matching string is found, the source document is added to the retrieval set. So, a user types the textual query “electric guitar,” and documents that include the text string “electric guitar” are retrieved. It all sounds quite simple. But this matching process can work in many different ways to produce different results.

Recall and precision Some algorithms return numerous results of varying relevance, while some return just a few high-quality results. The terms for these opposite ends of the spectrum are recall and precision. Figure 9-7 shows formulas for calculating them (note the difference in the denominators).

Figure 9-7. Precision and recall ratios

228 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Are your system’s users doing legal research, learning about the cur‐ rent state of scientific research in a field, or performing due dili‐ gence about an acquisition? In these cases, they’ll want high recall. Each of the hundreds or thousands (or more?) results retrieved will have some relevance to the user’s search, although perhaps not very much. As an example, users who are “ego-surfing” will want to see every mention of their names—they’re hoping for high recall. The problem, of course, is that along with good results come plenty of irrelevant ones.

On the other hand, a user who is looking for two or three really good articles on how to get stains out of a wool carpet will be hoping for high-precision results. It doesn’t matter how many relevant arti‐ cles there are if you get a good enough answer right away.

Wouldn’t it be nice to have both recall and precision at the same time? Lots and lots of very high-quality results? Sadly, you can’t have your cake and eat it, too: recall and precision are inversely related. You’ll need to decide what balance of the two will be most beneficial to your users. You can then select a search engine with an algorithm biased toward either recall or precision, or perhaps configure an engine to accommodate one or the other.

For example, a search tool might provide automatic stemming, which expands a term to include other terms that share the same root (or stem). If the stemming mechanism is very strong, it might treat the search term “computer” as sharing the same root (“com‐ put”) as “computers,” “computation,” “computational,” and “comput‐ ing.” Strong stemming in effect expands the user’s query by searching for documents that include any of those terms. This enhanced query will retrieve more related documents, meaning higher recall.

Conversely, no stemming means the query “computer” retrieves only documents with the term “computer” and ignores other var‐ iants. Weak stemming might expand the query only to include plu‐ rals, retrieving documents that include “computer” or “computers.” With weak stemming or no stemming, precision is higher and recall is lower. Which way should you go with your search system—high recall or high precision? The answer depends on what kinds of information needs your users have.

Another consideration is how structured the content is. Are there fields, rendered in HTML or XML or perhaps in a document record,

Search Algorithms | 229

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

that the search engine can “see” and therefore search? If so, search‐ ing for “William Faulkner” in the author field will result in higher precision, assuming we’re looking for books authored by Faulkner. Otherwise, we’re left with searching the full text of each document and finding results where “William Faulkner” may be mentioned, whether or not he was the author.

Other Approaches When you already have a “good” document on hand, some algo‐ rithms will convert that document into the equivalent of a query (this approach is typically known as document similarity). “Stop words” (e.g., “the,” “is,” and “he”) are stripped out of the good docu‐ ment, leaving a useful set of semantically rich terms that, ideally, represent the document well. These terms are then converted into a query that should retrieve similar results. An alternative approach is to present results that have been indexed with similar metadata. In Figure 9-8, hovering over individual search results in the Duck‐ DuckGo search engine offers more matches for the search terms in the same domain as that particular result.

Figure 9-8. DuckDuckGo search results are accompanied by a link to “More results” within the same domain

Approaches such as collaborative filtering and citation searching go even further to help expand results from a single relevant document.

230 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

In the following example from CiteSeer (see Figure 9-9), we’ve iden‐ tified an article that we like: “Application Fault Level Tolerance in Heterogeneous Networks of Workstations.” CiteSeer automatically finds documents in a number of ways:

Cited by What other papers cite this one? The relationship between cited and citing papers implies some degree of mutual relevance. Per‐ haps the authors even know each other.

Active bibliography (related documents) Conversely, this paper cites others in its own bibliography, implying a similar type of shared relevance.

Related documents from co-citation Another twist on citation, co-citation assumes that if documents appear together in the bibliographies of other papers, they prob‐ ably have something in common.

Figure 9-9. CiteSeer provides multiple ways to expand from a single search result

There are other retrieval algorithms, more than we can cover here. What’s most important is to remember that the main purpose of these algorithms is to identify the best pool of documents to be pre‐ sented as search results. But “best” is subjective, and you’ll need to

Search Algorithms | 231

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

have a good grasp of what users hope to find when they’re searching your site. Once you have a sense of what they wish to retrieve, begin your quest for a search tool with a retrieval algorithm that might address your users’ information needs.

Query Builders Besides search algorithms themselves, there are many other means of affecting the outcome of a search. Query builders are tools that can soup up a query’s performance. They are often invisible to users, who may not understand their value or how to use them. Common examples include:

Spell checkers These allow users to misspell terms and still retrieve the right results by automatically correcting search terms. For example, “accomodation” would be treated as “accommodation,” ensuring retrieval of results that contain the correct term.

Phonetic tools Phonetic tools (the best-known of which is “Soundex”) are especially useful when searching for a name. They can expand a query on “Smith” to include results with the term “Smyth.”

Stemming tools Stemming tools allow users to enter a term (e.g., “lodge”) and retrieve documents that contain variant terms with the same stem (e.g., “lodging,” “lodger”).

Natural language processing tools These can examine the syntactic nature of a query—for exam‐ ple, is it a “how to” question or a “who is” question?—and use that knowledge to narrow retrieval. For example, Siri uses natu‐ ral language processing to figure out if it should trigger a web search or a bad joke (Figure 9-10).

232 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-10. Siri uses natural language processing to determine whether the user wants to do a web search, look at the weather app, or hear a bad joke

Controlled vocabularies and thesauri Covered in detail in Chapter 10, these tools leverage the seman‐ tic nature of a query by automatically including synonyms within the query.

Spell checkers correct for an almost universal problem among searchers and are well worth considering for your search system. (Look over your search logs, and you’ll be amazed by the prepon‐ derance of typos and misspellings in search queries.)

The other query builders have their pros and cons, addressing dif‐ ferent information needs in different situations. Once again, a sense of your users’ information needs will help you select which approaches make the most sense for you; additionally, keep in mind that your search engine may or may not support these query builders.

Presenting Results What happens after your search engine has assembled the results to display? There are many ways to present results, so once again you’ll need to make some choices. And as usual, the mysterious art of understanding your content and how users want to use it should drive your selection process.

When you are configuring the way your search engine displays results, there are two main issues to consider: which content

Presenting Results | 233

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

components to display for each retrieved document, and how to list or group those results.

Which Content Components to Display Display less information to users who know what they’re looking for, and more information to users who aren’t sure what they want.

A variant on that simple approach is to show users who are clear on what they’re looking for only representational content components, such as a title or author, to help them quickly distinguish the result they’re seeking. Users who aren’t as certain of what they’re looking for will benefit from descriptive content components such as a sum‐ mary, part of an abstract, or keywords to get a sense of what their search results are about. You can also provide users some choice of what to display; again, consider your users’ most common informa‐ tion needs before setting a default. For example, the Yelp iPad app allows the user to view search results as listings, a location map, or images (Figure 9-11).

Figure 9-11. The Yelp iPad app allows users to select three different ways of viewing search results: as listings, as locations on a map, or as images

When it’s hard to distinguish retrieved documents because of a com‐ monly displayed field (e.g., the title), show more information, such as a page number, to help the user differentiate between results.

Another take on the same concept is shown in Figure 9-12, which displays multiple versions of the same book. Some of the distinc‐ tions are meaningful: you’ll want to know which items are available in the library. Some aren’t so helpful; for example, you might not care as much about the cover.

234 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-12. Content components help distinguish multiple versions of the same book

How much information to display per result is also a function of how large a typical result set is. Perhaps you don’t have that much content, or most users’ queries are so specific that they retrieve only a small number of results. If you think that users would like more information in such cases, then it may be worth displaying more content components per result. But keep in mind that regardless of how many ways you indicate that there are more results than fit on one screen, many (if not most) users will never venture past that first screen. So don’t go overboard with providing lots of content per result, as the first few results may obscure the rest of the retrieval.

Which content components you display for each result also depends on which components are available in each document (i.e., how your content is structured) and on how the content will be used. Users of phone directories, for example, want phone numbers first and foremost. So it makes sense to show them the information from the phone number field in the result itself, as opposed to forcing

Presenting Results | 235

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

them to click through to another document to find this information (see Figure 9-13).

Figure 9-13. A Yellow Pages search doesn’t force us to click through for a phone number

If you don’t have much structure to draw from or if your engine is searching full text, showing the query terms within the “context” of the document’s text is a useful variation on this theme (see Figure 9-14). In this example, The Verge highlights the query terms by using a bold font within the sentence they appear in—an excel‐ lent practice, as it helps the user quickly scan the results page for the relevant part of each result.

236 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-14. The Verge bolds search query result instances in their sur‐ rounding sentences to show their context

Presenting Results | 237

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

How Many Documents to Display How many documents are displayed depends mostly on two factors. If your engine is configured to display a lot of information for each retrieved document, you’ll want to consider having a smaller retrieval set, and vice versa. Additionally, a user’s screen resolution, connectivity speed, and browser settings will affect the number of results that can be displayed effectively. It may be safest to err on the side of simplicity—by showing a small number of results—while providing a variety of settings that users can select based on their own needs.

We suggest that you let users know the total number of retrieved documents so they have a sense of how many documents remain as they sift through search results. Also consider providing a results navigation system to help them move through the results. In Figure 9-15, Reuters provides such a navigation system, displaying the total number of results and enabling users to move through the result set 10 items at a time.

In many cases, the moment a user is confronted by a large result set is the moment he decides the number of results is too large. This is a golden opportunity to provide the user with the option of revising and narrowing his search. Reuters achieves this quite simply by repeating the query in the search box.

238 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-15. Reuters allows you to jump ahead through screens of 10 results at a time

Listing Results Now that you have a group of search results and a sense of which content components you wish to display for each, in what order should these results be listed? Again, much of the answer depends

Presenting Results | 239

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

upon what kind of information needs your users start with, what sort of results they are hoping to receive, and how they would like to use the results.

There are two common methods for listing retrieval results: sorting and ranking. Retrieval results can be sorted chronologically by date, or alphabetically by any number of content component types (e.g., by title, by author, or by department). They can also be ranked by a retrieval algorithm (e.g., by relevance or popularity).

Sorting is especially helpful to users who are looking to make a deci‐ sion or take an action. For example, users who are comparing a list of products might want to sort by price or another feature to help them make their choice. Any content component can be used for sorting, but it’s sensible to provide users with the option to sort on components that will actually help them accomplish tasks. Which ones are task oriented and which aren’t, of course, depends upon each unique situation.

Ranking is more useful when there is a need to understand informa‐ tion or learn something. Ranking is typically used to describe retrieved documents’ relevance, from most to least. Users look to learn from those documents that are most relevant. Of course, as we shall see, relevance is relative, and you should choose relevance ranking approaches carefully. Users will generally assume that the top few results are best.

The following sections provide examples of both sorting and rank‐ ing, as well as some ideas on what might make the most sense for your users.

Sorting by alphabet Just about any content component can be sorted alphabetically (see Figure 9-16). This is a good general-purpose sorting approach— especially when sorting names—and in any case, it’s a good bet that most users are familiar with the order of the alphabet! It works best to omit initial articles such as “a” and “the” from the sort order (cer‐ tain search engines provide this option); users are likely to look for “The Naked Bungee Jumping Guide” under “N” rather than “T.”

240 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-16. Baseball-Reference.com displays search results in alpha‐ betical order

Sorting by chronology If your content (or your user’s query) is time sensitive, chronological sorts are a useful approach. And you can often draw on a filesystem’s built-in dating if you have no other sources of date information.

If your site provides access to press releases or other news-oriented information, sorting by reverse chronological order makes good

Presenting Results | 241

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

sense (see Figure 9-17 and Figure 9-18). Chronological order is less common and can be useful for presenting historical data.

Figure 9-17. The Washington Post’s default list ordering is by reverse chronological order...

242 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-18. ...as is CNET’s

Ranking by relevance Relevance-ranking algorithms (there are many flavors) are typically based on one or more of the following:

• How many of the query’s terms occur in the retrieved document • How frequently those terms occur in that document

Presenting Results | 243

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

• How close together those terms occur (e.g., are they adjacent, in the same sentence, or in the same paragraph?)

• Where the terms occur (e.g., a document with the query term in its title may be more relevant than one with the query term in its body)

• The popularity of the document where the query terms appear (e.g., is it linked to frequently, and are the sources of its links themselves popular?)

Different relevance-ranking approaches make sense for different types of content, but with most search engines, the content you’re searching is apples and oranges. So, for example, Document A might be ranked higher than Document B, but Document B is definitely more relevant. Why? Because while Document B is a bibliographic citation to a really relevant work, Document A is a long document that just happens to contain many instances of the terms in the search query. The more heterogeneous your documents are, the more careful you’ll need to be with relevance ranking.

Indexing by humans is another means of establishing relevance. Keyword and descriptor fields can be searched, leveraging the value judgments of human indexers. For example, manually selected rec‐ ommendations—popularly known as “best bets”—can be returned as relevant results. In Figure 9-19, the first set of results was associ‐ ated with the query “Ukraine” in advance.

Requiring an investment of human expertise and time, the best bets approach isn’t trivial to implement and therefore isn’t necessarily suitable to be developed for each and every user query. Instead, rec‐ ommendations are typically used for the most common queries (as determined by search log analysis) and combined with automati‐ cally generated search results.

244 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-19. A search of the BBC’s site retrieves a set of manually tag‐ ged documents as well as automatic results; the recommendations are called “Editor’s Choice” rather than “best bets”

Ranking by popularity Popularity is the source of Google’s popularity.

Put another way, Google is successful in large part because it ranks results by which ones are the most popular. It does so by factoring in how many links there are to a retrieved document. Google also dis‐ tinguishes the quality of these links: a link from a site that itself receives many links is worth more than a link from a little-known site. This algorithm, which is part of Google’s “secret sauce” for pre‐ senting search results, is known as PageRank.

Presenting Results | 245

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

There are other ways to determine popularity, but keep in mind that small sites or collections of separate, nonlinked sites (often referred to as “silos”) don’t necessarily take advantage of popularity as well as large, multisite environments with many users. The latter have a wide scope of usage and a richer set of links. A smaller system isn’t likely to have enough variation in the popularity of different docu‐ ments to merit this approach, while in a “silo” environment, little cross-pollination results in few links between sites. It’s also worth noting that, to calculate relevance, Google uses many other criteria in addition to PageRank.

Ranking by users’ or experts’ ratings In an increasing number of situations, users are willing to rate the value of information. User ratings can be used as the basis of retrieval result ordering. In the case of Yelp (see Figure 9-20), these ratings—based on users’ reviews of businesses listed in the system— are integral to helping users judge the value of an item, and form the foundation of an entire information economy. Of course, Yelp has a lot of users who don’t shrink from expressing their opinions, so there is a rich collection of judgments to draw on for ranking.

Most sites don’t have a sufficient volume of motivated users to employ valuable user ratings. However, if you have the opportunity to use this data, it can be helpful to display user ratings with a docu‐ ment, if not as part of a presentation algorithm.

246 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-20. User ratings fuel the ranking of these Yelp results

Presenting Results | 247

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

2 Susan T. Dumais, Edward Cutrell, and Hao Chen, “Optimizing search by showing results in context” (Proceedings of CHI ’01, Human Factors in Computing Systems, 2001, 277–284).”

Ranking by pay-for-placement Advertising has become the predominant business model for pub‐ lishing online, so it is no surprise that pay-for-placement (PFP) has become commonplace in many search systems. Although the previ‐ ous Yelp example showed results sorted by user rankings, the first result on the list actually has a lower ranking than the others; it owes its position at the top of the list solely to the fact that it is a paid advertisement.

If your system aggregates content from a number of different ven‐ dors, you might consider implementing PFP to present search results. If users are shopping, they might also appreciate this approach—with the assumption being that the most stable, success‐ ful sites are the ones that can afford the highest placement. This is somewhat like selecting the plumber with the largest advertisement in the yellow pages to fix your toilet.

Grouping Results Despite all the ways we can list results, no single approach is perfect. Hybrid approaches that combine different types of sorting—such as Google’s—show a lot of promise, but you typically need to be in the business of creating search engines to have this level of involvement with a tool. In any case, our information environments are typically getting larger, not smaller. Search result sets will accordingly get larger as well, and so will the probability that those ideal results will be buried far beyond the point where users give up looking.

However, one alternative approach to sorting and ranking holds promise: clustering retrieved results by some common aspect. An excellent study by researchers at Microsoft and the University of California at Berkeley shows improved performance when results are clustered by category as well as by a ranked list.2 How can we cluster results? The obvious ways are, unfortunately, the least useful: we can use existing metadata, like document type (e.g., .doc, .pdf) and file creation/modification date, to allow us to divide search results into clusters. Much more useful are clusters derived from

248 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

manually applied metadata, like topic, audience, language, and product family. Unfortunately, approaches based on manual effort can be prohibitively expensive.

In Figure 9-21, Forrester contextualizes the query “user experience” with roles such as “Marketing Leadership” and specific date ranges.

Figure 9-21. Forrester contextualizes search results for the query “user experience”

These clusters provide context for search results; selecting the cate‐ gory that seems to fit your interest best allows you to work with a significantly smaller retrieval set and (ideally) a set of documents that come from the same topical domain. This approach is much like generating search zones on the fly.

Acting on Results You’ve provided the user with a set of search results. What happens next? Certainly, she could continue to search, revising her query and her idea of what she’s looking for along the way. Or, heavens, she might have found what she was looking for and be ready to move on. Contextual inquiry and task-analysis techniques will help you understand what users might want to do with their results. The fol‐ lowing sections discuss a few common options.

Presenting Results | 249

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Call to action Some search results can be acted on directly, without having to jump through intermediary steps. In these cases, it is often desirable to include a call-to-action button or link along with individual search results. For example, the iOS App Store allows the user to “GET” apps directly from search results, without having to view the apps’ description screens and user reviews (Figure 9-22).

Figure 9-22. Search results in the iOS App Store include a “GET” but‐ ton (which lists the app’s price when it is not free)

250 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Select a subset of results Sometimes when you’re searching you want to take more than one document along with you. You want to “shop” for documents just like you shop for books at Amazon. And if you’re sorting through dozens or hundreds of results, you may need a way to mark the documents you like so you don’t forget or lose track of them.

A shopping cart feature can be quite useful in search-intensive envi‐ ronments such as library catalogs. In Figure 9-23, users can “save” a subset of their retrieval and then manipulate those results in their “shelves” once they’re done searching.

Figure 9-23. The San Francisco Public Library allows users to add search results to three “shelves”: “Completed,” “In Progress,” and “For Later”

Save a search In some cases, it’s the search itself, not the results, that you’re inter‐ ested in “keeping.” Saved searches are especially useful in dynamic domains that you’d like to track over time; you can manually re- execute a saved search on a regular basis, or schedule that query to automatically be rerun regularly. Note that the example in Figure 9-23 includes a “Save Search” link in the upper-right corner of the search results display; the user can name saved search sets for later retrieval.

Presenting Results | 251

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Designing the Search Interface All the factors we’ve discussed so far—what to search, what to retrieve, and how to present the results—come together in the search interface. And with so much variation among users and search technology functions, there can be no single ideal search interface. Although the literature of information retrieval includes many studies of search interface design, many variables preclude the emergence of a “right way” to design search interfaces. Here are a few of the variables on the table:

Level of searching expertise and motivation Are users comfortable with specialized query languages (e.g., Boolean operators), or do they prefer natural language? Do they need a simple or a high-powered interface? Do they want to work hard to make their searches truly successful, or are they happy with “good enough” results? How many iterations are they willing to try?

Type of information need Do users want just a taste, or are they doing comprehensive research? What content components can help them make good decisions about clicking through to a document? Should the results be brief, or should they provide extensive detail for each document? And how detailed a query are users willing to pro‐ vide to express their needs?

Type of information being searched Is the information made up of structured fields or full text? Is it navigation pages, destination pages, or both? Is it written in HTML or other formats, including nontextual? Is the content dynamic or more static? Does it come tagged with metadata, full of fields, or is it full text?

Amount of information being searched Will users be overwhelmed by the number of documents retrieved? How many results is the “right number”? That’s a lot to consider. Luckily, we can provide basic advice that you should consider when designing a search interface.

In the early days of the Web, many search engines emulated the functionality of the “traditional” search engines used for online library catalogs and databases, or were ported directly from those environments. These traditional systems were often designed for

252 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

researchers, librarians, and others who had some knowledge of and incentive for expressing their information needs in complex query languages. Therefore, many search systems at the time allowed the user to use Boolean operators, search fields, and so forth; in fact, users were often required to know and use these complex syntaxes.

As the Web’s user base exploded, overall searching experience and expertise bottomed out, and the new breed of user wasn’t especially patient. Users more typically just entered a term or two without any operators, pressed the “search” button, and hoped for the best.

The reaction of search engine developers was to bury the old fancy tricks in advanced search interfaces, or to make them invisible to users by building advanced functionality directly into the search engines. For example, Google makes a set of assumptions about what kind of results users want (through a relevance algorithm) and how they’d like those results presented (using a popularity algo‐ rithm). Google makes some good assumptions for web-wide search‐ ing, and that’s why it’s successful. However, most search systems, web-wide or local, don’t work as well.

For that reason, the pendulum may eventually swing back to sup‐ porting users who, out of frustration, have become more search lit‐ erate and are willing to spend more time learning a complex search interface and constructing a query. But for now, it’s fair to assume that, unless your site’s users are librarians, researchers, or specialized professionals (e.g., an attorney performing a patent search), they won’t invest much time or effort into crafting well-considered quer‐ ies. That means the burden of searching falls chiefly on the search engine, its interfaces, and how content is tagged and indexed. There‐ fore, it’s best to keep your search interface as simple as possible: present users with a simple search box and a “search” button.

The Box Your system is likely to have the ubiquitous search box, as shown in Figure 9-24.

Figure 9-24. The ubiquitous search box (in this case, from Apple)

Designing the Search Interface | 253

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Simple and clear. Type in some keywords (“lost iPhone”) or a natu‐ ral language expression (“Where can I find my iPhone?”), hit the Return (or Enter) button on your keyboard, and the whole site will be searched and results displayed.

Users make assumptions about how search interfaces work, and you may want to test for those as you design your own search system. Some common user assumptions include:

• “I can just type terms that describe what I’m looking for and the search engine will do the rest.”

• “I don’t have to type in those funny AND, OR, or NOT thingies.”

• “I don’t have to worry about synonyms for my term; if I’m look‐ ing for dogs, I just type ‘dogs,’ not ‘canine’ or ‘canines.’”

• “Fielded searching? I don’t have time to learn which fields I can search.”

• “My query will search the entire site.”

If your users have those assumptions and are not especially motiva‐ ted to learn more about how your system’s search works differently, then go with the flow. Give them the box. You certainly could pro‐ vide a “help” page that explains how to create more advanced, pre‐ cise queries, but users may rarely visit this page.

Instead, look for opportunities to educate users when they’re ready to learn. The best time to do this is after the initial searches have been executed, when the users have reached a point of indecision or frus‐ tration. The initial hope that the first try would retrieve exactly what they were looking for has now faded. And when users are ready to revise their searches, they’ll want to know how they can make those revisions. For example, if you search the eBay app for “watches” (see Figure 9-25), you’ll likely get a few more results than you’d like.

254 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-25. The eBay app’s search results provide opportunities to revise your search...

At this point, eBay’s search system goes beyond the box: it tells the user something to the effect of “Here are those 1,631,329 results that you asked for. Perhaps this is too many? If that’s the case, consider revising your search using our souped-up ‘Refine’ interface, which allows you to narrow your search. Or, select from a list of categories to narrow your results further” (see Figure 9-26).

Designing the Search Interface | 255

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-26. ...including the ability to refine your search by specifying various category-specific facets

In general, too many or too few (typically zero) search results are both good indicators for users to revise their searches; we’ll cover more on this topic in the section “Supporting Revision” on page 260 later in this chapter.

Consider how your search box is presented. The box can cause con‐ fusion when it appears alongside other boxes. Unless your system’s

256 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

search functionality truly requires more than one field—as is the case with many travel-related services—it is best to keep search limi‐ ted to a single box. (If more than one field is required, it’s important that they be clearly labeled, as illustrated in Figure 9-27.)

Figure 9-27. Kayak’s flight search form features clearly labeled fields

Consistent placement of the search box alongside other global navi‐ gation choices, along with the consistent use of a button labeled “search” that comes with that box, will go a long way toward ensur‐ ing that users at least know where to type their queries.

There are many assumptions behind that innocuous little search box, some made on the part of the user, and some by the designer who decides what functionality will be hidden behind that box. Determining what your users’ assumptions are should drive the default settings that you set up when designing the simple search interface.

Autocomplete and Autosuggest Autocomplete and autosuggest are widely used patterns for interact‐ ing with search systems. In both cases, a list of results is presented alongside the search box, preemptively prompting the user with possible matches based on the first few characters typed. These results are culled from search indexes, controlled vocabularies, man‐ ually configured match lists, or often all of the above. Displays range from very simple and straightforward text lists (in the case of auto‐ complete patterns) to popovers with highly customized layouts (Figure 9-28).

Designing the Search Interface | 257

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-28. Like many airlines, Lufthansa presents a list of airports that match the first few characters the user types into the origin and destination search boxes

This technique is very useful, because it helps users identify poten‐ tial matches based on partial or incomplete information. In some cases, it also gives them hints as to the way the system is structured, allowing them to make smarter searches by giving them the ability to explore the system right from the search box. Because of this, it has mostly supplanted the dedicated, advanced search mechanisms of yore.

Advanced Search In the past, many websites provided advanced search interfaces as crutches for underfeatured or poorly configured search systems. In stark contrast to the search box, advanced search interfaces allow much more manipulation of the search system and are typically used by two types of users: advanced searchers (librarians, lawyers, doctoral students, medical researchers), and frustrated searchers who need to revise their initial searches (often users who’ve found that the search box didn’t meet their needs). As search engines have improved, advanced search interfaces are increasingly focused on serving the former.

258 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

While they are less common today, advanced search interfaces pro‐ vide flexibility and power to users who understand the structure of the information they are looking for. For example, the US Congress website allows knowledgeable users to configure extremely sophisti‐ cated searches using Boolean operators (Figure 9-29).

Figure 9-29. Congress.gov allows advanced users to build complex searches using Boolean operators

If your system could benefit from advanced search, a good rule of thumb is to expose your search engine’s various heavy-duty search functions on the advanced page for those few users who want to have a go at them, but design your search system with the goal of making it unnecessary for the vast majority of searchers to ever need to go to the advanced search page.

Designing the Search Interface | 259

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Supporting Revision We’ve touched on what can happen after the users find what they’re looking for, when the search is done. But all too often that’s not the case. Here are some guidelines to help your users hone their searches (and hopefully learn a little bit about your search system in the process).

Repeat search in results page What was it I was looking for? Sometimes users are forgetful, espe‐ cially after sifting through dozens of results. Displaying the initial search within the search box (as in Figure 9-30) can be quite useful: it restates the search that was just executed, and allows the user to modify it without reentering it.

Figure 9-30. In the Netflix Android app, the query is displayed on the results page and can be revised and reexecuted

Explain where results come from It’s useful to make clear what content was searched, especially if your search system supports multiple search zones (see Figure 9-31). This reminder can be handy if the user decides to broaden or narrow his search; more or fewer search zones can be used in a revised search.

260 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-31. The iOS iTunes Store app search system shows you where you searched (i.e., “All”), and makes it easy to reach results from other search zones

Designing the Search Interface | 261

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Explain what the user did If the results of a search are not satisfactory, it can be useful to state what happened behind the scenes, providing the user with a better understanding of the situation and a jumping-off point should she wish to revise her search.

Explaining “what happened” can include the two guidelines just mentioned, as well as:

• Restating the query • Describing what content was searched • Describing any filters that might be in place (e.g., date ranges) • Showing implicit Boolean or other operators, such as a default

AND • Showing other current settings, such as the sort order • Mentioning the number of results retrieved

In Figure 9-32, the New York Times site provides an excellent exam‐ ple of explaining to the user what just happened.

Figure 9-32. All aspects of the search are restated as part of these search results

262 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Integrate searching with browsing A key theme in this book is the need to integrate searching and browsing (think of them together as “finding”), but we won’t belabor it here. Just remember to look for opportunities to connect your search and browse systems to allow users to easily jump back and forth.

As Figure 9-33 and Figure 9-34 illustrate, Barnes & Noble provides this functionality in both directions.

Figure 9-33. Searching leads to browsing: a search for “2001 a space odyssey” on the Barnes & Noble site retrieves categories as well as documents

Designing the Search Interface | 263

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-34. And browsing leads to searching: navigate to the “Movies & TV” section, and you’ll find the search box set to search that zone

When Users Get Stuck You can strive to support iterative searching with fully integrated browsing and state-of-the-art retrieval and presentation algorithms, yet users still will fail time and time again. What should you do when presenting the user with zero results, or with way too many?

The latter case is a bit easier to address, because in most cases your search engine provides relevance-ranked results. In effect, winnow‐ ing oversized result sets is a form of search revision, and often the user will self-select when he is ready to stop reviewing results. But it is still useful to provide some instruction on how to narrow search results, as shown in Figure 9-35.

You can also help users narrow their results by allowing them to search within their current result sets. In Figure 9-36, the initial search for hotels in New York City retrieved over 600 results; we can “filter by hotel name” for particular brands to narrow our retrieval.

264 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

Figure 9-35. Congress.gov provides advice on how to narrow down searches

Figure 9-36. Priceline.com allows users to search within the result set

At the other end of the spectrum, zero hits is a bit more frustrating for users and challenging for information architects. We suggest you adopt a “no dead ends” policy to address this problem. “No dead

Designing the Search Interface | 265

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

ends” simply means that users always have another option, even if they’ve retrieved zero results. The options might include:

• A means of revising the search • Search tips or other advice on how to improve the search • A means of browsing (e.g., including the site’s navigation system

or sitemap) • A human contact if searching and browsing won’t work

It’s worth noting that we’ve seen few (if any) search systems that meet all these criteria.

Where to Learn More Although this is the longest chapter in this book, we’ve covered only the tip of the search system iceberg. If this piqued your interest, you may want to delve further into the field of information retrieval. Some of our favorite texts are:

• Search Patterns: Design for Discovery by Peter Morville and Jeff‐ ery Callender (Sebastopol, CA: O’Reilly, 2010)

• Modern Information Retrieval by Ricardo Baeza-Yates and Berthier Ribeiro-Neto (Boston: Addison-Wesley, 2011)

• Concepts of Information Retrieval by Miranda Lee Pao (West‐ port, CT: Libraries Unlimited, 1989); this title is out of print, but you may be able to find used copies on Amazon

• On Search, the Series by Tim Bray, an excellent collection of essays on search written by the father of XML

If you’re looking for more immediate and practical advice, the most useful site for learning about search tools is, naturally, Search‐ tools.com, Avi Rappoport’s compendium of installation and config‐ uration advice, product listings, and industry news. Another excellent source is Danny Sullivan’s Search Engine Watch, which focuses on web-wide searching but is quite relevant to site-wide searching nonetheless.

266 | Chapter 9: Search Systems

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

http://bit.ly/on_search_the_series

http://www.searchtools.com

http://www.searchenginewatch.com

Recap Let’s recap what we learned in this chapter:

• Search is an important mechanism for finding information; however, it’s not a given that your information environment requires a search system.

• Although search may appear simple—just type some words into the search box—there’s a lot going on under the hood.

• Choosing what to index in your information environment is an important step when configuring your search system.

• There are many different types of search algorithms. • There are also various different ways of presenting results back

to the user. • All of these factors—what to search, what to retrieve, and how

to present the results—come together in the search interface.

Now we move on to discuss the final principle in our overview: the‐ sauri, controlled vocabularies, and metadata.

Recap | 267

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed

C op

yr ig

ht ©

2 01

5. O

'R ei

lly M

ed ia

, I nc

or po

ra te

d. A

ll rig

ht s

re se

rv ed