Every SEO will at one point encounter the content paradox: the more quality content your site has, the more useful it is, but the harder it becomes to find that content.
At a certain point, well-designed site navigation alone is not enough.
Not every product, article, or other content pieces can have its own submenu. And not every visitor will happily spend time exploring via category led journey.
Searchers are looking for a specific category or topic. Sometimes even an exact product or service. And they want to know within one click if you have it.
These visitors will immediately look for a site search box (also known as internal site search functionality).
Interacting with an optimized on-site search function is the digital equivalent of a passionate and helpful librarian.
You say what you’re looking for (type in a phrase) and it speedily shows you relevant options (provides a list of result pages from the site).
Sadly, many search boxes typify a librarian who sits disinterestedly behind the desk, giving a curt response of “over there” if you ask for information.
It’s a choice what you employ at the entryway to your content. A valuable asset that can drive up organic sessions, sitewide conversions and enhance brand image. Or a barrier between visitors and their desired content.
On-Site Search Tools
You can get a site search box “out of the box” thanks to a variety of tools – from open source options to focussed software as a service, to enterprise site search solutions costing over six figures.
Most content management systems and ecommerce platforms will have basic search functionality in-built, which can be further enhanced with plugins.
And some companies choose to custom build their own.
I won’t second guess which search application will best suit your website.
I will give you a list of best practice functionality your chosen platform should accommodate.
On-Site Search Box Usability
We’ve all been there. Thumbing on a mobile screen into a tiny search box that delivers no useful result.
Sadly, there’s no one right user interface for a site search box. A/B testing different variations will show which one generates the most queries and conversions with your visitors.
Having said that, there are a few best practices to abide by.
Don’t Make Users Search for the Search Box
The placement of your search bar will dictate it’s usage rate. Although there is no “best” spot for all websites, users typically look for the site search box in the top right or top middle on desktop and in an own line, the entire width of the screen, within the header on mobile.
Don’t hide it in a drop-down or a hamburger menu. Don’t place it too near other boxes, such as newsletter sign up field. Don’t have it as nothing more than a small icon that must be clicked for the text field to expand.
The input field should be big enough for most search terms. After all, it’s tough to correct errors or misspellings you can’t see.
The reasonable width will vary from site to site, but as a guide, typical queries will fit into a 27 character long search box. Feel free to go longer. Beware going shorter.
Position the search box on every relevant page in the same spot. I say relevant page because search boxes aren’t necessarily a global element.
They don’t belong on checkout pages as this can distract from conversion.
They may not belong in marketing campaign landing pages.
They likely belong on 404 pages, but rather than in the header, front and center to help the lost user to their desired content.
And then there’s the homepage. Should it have the standard search box placement? Or should it be featured above the fold?
Answer the question – can your visitors convert faster with site search?
If your website is geared towards discovery and exploration, or has a focussed goal like driving signups, the typical placement shared by the rest of the site should do just fine.
If, on the other hand, your user knows what they are looking for, putting search in the hero image on the homepage can reduce the number of clicks to conversion.
Have a Clear Call to Action
A best in class search experience starts with the form itself. It should be immediately obvious what the search box field does.
Often this is achieved by pairing a text prompt with a magnifying glass icon and/or a button labeled ‘Search’ or ‘Find’ – The label ‘Go’ has fallen out of favor under the presumption it isn’t as clear.
Use placeholder text within the search box field to simultaneously affirm that yes, this is, in fact, a search bar, and to guide visitors about what they can search on your site.
Encourage searches with:
- Prompts like “Search today’s news”, “Find events” “Enter product, code or brand”.
- Or open-ended questions such as “What are you looking for?”
Just be sure the placeholder text clears when the user clicks in the search box.
Finally, while some visitors prefer to click on a search button others will just hit Enter on their device when they’re done typing. Both of these actions should trigger the search function.
Improve Imperfect Input
Visitors won’t type your idea of a perfect keyword. Any search function must adapt not only to their terminology but also their inevitable failings.
A user’s poor input doesn’t excuse a site’s poor output.
On-site search must:
- Search case-insensitively by default.
- Understand singular, plural and other stemming variants.
- Be forgiving of common misspellings and typos.
- Accommodate synonyms, abbreviations and alternate terms.
- Recognize American and British English.
- Handle numbers and special characters.
- Master stop words such as “a”, “the”, “of”, “for”, “and”, “in” etc.
Whenever on-site search automatically corrects the search query, for example in the case of misspellings, show the corrected and original queries at the top of the page. Be sure to give a clear option to force a search with the original term.
Also, decide how you will handle the submitting of an empty search (i.e., the user has clicked into the box, but not typed anything before triggering the search function).
You could send them to an ‘all categories’ page. You could show an error message. You can simply not allow the form to submit. Returning an empty results page shouldn’t be the answer.
Inspire Input with Predictive Search
Predictive search engages visitors from the first characters they type by showing result suggestions.
This:
- Speeds up the search process.
- Improves the search input quality by guiding query construction.
- Offers clues to your site’s range of content.
Such autocomplete drop-downs can offer query suggestions, ‘in category’ results and/or show a few pieces of specific content, including images where relevant.
For example, if I type “bag”, predictive search can complete my query intent with query suggestions such as “handbags”, “backpacks”, “laptop bags”, “bags for men”, “bag organizer” and so on.
As well as show me category options such as “sports”, “fashion” or “travel”. And offer a few select product suggestions.
For better readability, use bold to highlight how the user’s typed query relates to the query suggestions.
Some brands choose to have the category options as a drop-down next to the search box input field. While this has been made common practice by ecommerce players, it does add an extra click to the search conversion.
Split test this against automated category suggestions, which don’t require additional user action.
For websites offering different content types, consider using iconography or labels in the search suggestions.
And finally, for usability, be sure that hitting ‘esc’ closes the search autocomplete and allow keyboard navigation of list suggestions.
Internal Search Result Page Usability
Deliver Search Results Quickly
Page load speed is always important, but especially so for on-site search.
Google’s research shows that if “search results are slowed by even a fraction of a second, people search less.”
If search takes more than one second to process, show a progress indicator or other useful animation to distract the user in the interim.
Reiterate the Search Query
The visualization of on-site search results can be just as important as the results themselves.
It is jarring to be on a brand’s website, do an on-site search, and land on a page that looks like a Google SERP.
Search result page should follow a similar style as your category pages. This is likely not a list of SEO page titles, meta descriptions, and URLs. But would include headings, user-focused titles, images, call-to-action buttons, and other rich content details.
In such a rich search results page experience, ensure the search query is present in multiple locations.
- Keep it within the search box to allow for easy refinement or amendment.
- As a headline above the search results along with the number of matches.
- In bold for any matching text within the results.
Improve Result Relevancy with Semantic Search
How many times have you typed a long-tail query into a search box for half of that query to be seemingly ignored?
Returning too many search results can frustrate users, who may begin to question why they even bothered to use the internal search.
What’s more, similar to Google SERPs, there’s often a marked drop-off after the first page of results.
Despite this, many sites make the mistake of setting their on-site search criteria too broad.
It’s of critical importance to display only relevant search results.
On-site search devoid of natural language processing treats words in a query as unrelated terms and retrieves results based on basic keyword matching. This method can easily misunderstand the user intent as it analyses the keywords input at face value.
The more sophisticated approach of semantic search analyses the context and intent behind the query. Such implementations deliver more relevant results, especially for long or complex queries.
For example, the search “duffel bags for the gym” should return only gym duffel bags. Not every item containing the word bag or gym.
Especially when the searcher has made the effort to be precise, be respectful and return results that match their complete search intent.
Or, at least, show the exact matched results first, followed by a subheading such as “Other items that may interest you” where broad matched results are displayed.
If semantic search is not an option, rather than a free text search box, you can implement constrained search.
Fields are populated by set drop-down menus and/or searchable queries are restricted to autocomplete suggestion. This limits visitors to search within the scope of the site architecture and ensures a quality result.
However, this also requires more effort on the part of the searcher, more specs in the user interface, and limits user intent analysis on search data.
Leverage Search Within Search
If a query is broad to begin with, so it returns a multitude of listings, help the searcher refine results by providing relevant filtering (to narrow) and sorting (to organize) options.
On the other hand, where the facets are communicated within the search query, an optimized on-site search will break down the search phrase to automatically apply filters and give an easy option for adjustments.
For example the search phrase “kid-friendly brunch near me” would apply filters for the attributes “child-friendly” and “brunch” and sort results based on distance.
The focus here is, again, relevancy.
Remember, every filter option you add is decision load on the user and is potentially another URL using crawl budget.
As the searcher may not convert on the first visit, encourage the user to save any multi-faceted searches. Or better, to opt in to receive email updates for new matches.
This not only makes it easy for the visitor to re-find engaging content, but as an additional benefit, by offering saved search functionality you encourage the creation of a profile.
9. Leverage the No Results Page
What if there are no results to display?
Firstly, check if your search function is returning results based on all website content.
- Not just meta data, but full copy.
- Not just in HTML, but also in PDFs and other content based file formats.
- Not just product pages, but also CMS pages such as return policies.
If this is the case and you truly have no relevant content, be sure to handle it more gracefully than the infamous “No results found” message.
Or worse blaming the searcher by telling them to “Check your spelling” (on-site search should autocorrect) or “Use more general terms” (on-site search should match against broader content).
If visitors believe you don’t have what they’re looking for they will leave. Worse, leave frustrated. Making it less likely they will choose to engage with your brand in the future.
Apologetically state there are no results available and propose a valuable next step. This could be in the form of alternative search suggestions, contextual category links, broader matching content, or details to contact support.
Some websites choose to show top searches, promote selected content or list all their categories. These options do not recognize the searcher’s intent. As such, are less valuable.
However, any next step is better than a dead end.
Action Internal Site Search Data
On-site search data holds a wealth of actionable insights for SEO.
And it’s incredibly easy to set up in Google Analytics.
The usage report helps you understand to what extent users took advantage of your internal search function and how effectively the search results created deeper engagement with your site.
Use these metrics to quantify the potential KPI impact of site search optimization.
It is common that visits with site search show significantly lower bounce rates, higher time on site, more pages per session, and more conversions.
The search terms report shows what people are looking for and more importantly if your site is delivering.
Use search terms to:
- Guide site content: Terms with high unique searches and high search exit rate show what content visitors want, that is not yet provided or that can’t be easily found. Use this as inspiration for article topics, product range offering, category additions, navigational elements or content to feature on your homepage.
- Mine for keywords: Terms searched on your site are likely correlated to those your target audiences entered in search engines. Use them as keywords for PPC and SEO.
- Improve filters options: Terms with Results Pageviews/Search higher than 2 show searchers are digging to find relevant results. This could indicate the need for additional filter options, show the query is matching too broadly or that other optimization of the search result page are required.
- Predict user intent: Add a secondary dimension of Refined Keyword to understand the motivation behind high search refinement rates. Use these keywords as dropdown options in predictive search and to improve semantic search.
NOTE: For high-quality data, no matter how the query is typed, work with your development team to ensure all search terms are tracked in lower case. Else you will find “Term X” and “term x” splitting data within your reports, such as in the screenshot above.
The search pages report shows where searches began (start page) and the search result page (search destination page).
Use search pages to:
- Optimize search box: Successful optimization of the search box interface should result in higher total unique searches from the impacted start pages.
- Identify navigational issues: Start pages where the search box isn’t prominently placed indicate pages where the visitors got lost in their navigational journey and turned to search for help.
- Be trendy: Search destination pages with high total unique searches and a low search exit rate show what your customer base is currently interested in. These trendy pages can be pushed by other channels, such as email or social media, with the knowledge they are likely to perform well.
In addition to the basic site search reporting, it’s best practice to set up a Google Analytics event to track when the ‘no result’ page is triggered. Example values could be:
- Event category as ‘search’
- Event action as ‘no results’
- Event label as the search query
Use this data to identify high volume problem queries and improve searcher experience.
SEO Benefits & Risks of On-Site Search
Crawling & Indexing of Internal Site Search Result Pages
You’ve probably read Google doesn’t like to index site search pages.
One often repeated source was an article by Matt Cutts in 2007 where he commented, “we’ve seen that users usually don’t want to see search results… in their search results”.
So the prevailing SEO approach was to prevent crawling and indexing of site search result pages with effective robots handling.
But this is 2019, and you’ve probably seen examples of Google ranking search results pages in search results.
There are many case studies whereby giving Googlebot full access to site search result pages has brought in a wealth of organic sessions.
But this doesn’t come without risks.
Exposing search results pages to Google opens up your site to generating an indexable page for any keyword. Meaning you can end up with user-generated spam pages.
This can create some “interesting” SERPs. Especially with dynamic page titles and meta descriptions.
Women being offered free shipping on “big stuff”…
Babies being keen on paints…
Kids getting out of homework…
Of course, these pages are unlikely to be triggered on Google in the wild.
The point is that while some relevant internal search pages might make it to the top and bring in users. There’s likely to be an iceberg worth of low-quality pages below the surface dragging down domain quality and using precious crawl budget.
So what’s a site to do?
Control Crawl Budget by Using Parameter-Based On-Site Search URLs
For the most part, on-site search result pages are realized by using the GET method as a URL parameter.
For example, www.example.com/search?q=search-term
SEO professionals who are strong proponents of indexing internal search result pages often encourage the optimization of the search URL by making it a static path.
For example: www.example.com/search/search-term
Both these URL structures use crawl budget, unless specified otherwise by robots handling.
To avoid this, some internal search implementations use the POST method to remove the search term and render internal search result pages on a single URL.
For example, www.example.com/search
The obvious downside of this approach is that the URL doesn’t have an opportunity to rank for the search term. What’s more, the search can’t be saved by the user.
So which is best for SEO?
John Mueller has said that it’s easy for Google to “get lost in the weeds” of internal search pages. That both crawling and low-quality pages could become problems. Although quality “is not always black and white” as there are ways to shape internal search pages to be seen more like a category page.
He recommends the parameter approach as it’s easier for Google to recognize it as “something that may vary or as something that a user is submitting”. This allows Google to optimize crawling.
In addition, parameter URLs provide flexibility whether you want to search pages to be crawled or not with the click of a dropdown within Google Search Console and Bing Webmaster Tools.
Avoid Potential Duplicate Content With URL Redirects
If you have an existing URL that satisfies the search query, redirect the user.
For example, if a user searches “blue widgets” and there is a category page for widgets and a filter for colors with blue as an option, send the user to that page.
There’s no reason to have both the search result page www.example.com/search?q=blue-widgets and the category page www.example.com/widgets/blue as each of these fulfill the same user intent – to see a list of blue widgets.
If relevant, also consider redirecting the search queries on an item level, such as when a user searches a specific product SKU. Being taken directly to a content page, rather than a single item search results page, accelerates conversion.
From an SEO perspective – Hijacking the search query, whether on the category or item level, limits the total number of search result pages to content the site taxonomy doesn’t already cover. Meaning crawl budget is spent on pages that wouldn’t rank without site search.
These redirects also prevent any possibility of duplicate content or risks of a ranking signal being split across pages.
Avoid Thin Content Issues by Handling ‘No Result’ Pages
Providing a valuable next step, as advised above, within the user interface already reduces the likelihood of any thin content issues.
But as a “no result” page offers limited user value, you may want to add a noindex, follow robots meta tag. This helps prevent user-generated spam pages showing up in Google SERPs and dragging down domain quality.
These three SEO actions ensure indexable on-site search pages are limited to queries your website taxonomy doesn’t cover and which offer relevant results – a worthy allocation of crawl budget.
Search Box within Google Search Results
Having on-site search offers the opportunity to claim more space in search engines. A Sitelinks Searchbox can appear as part of the SERP result, often for high volume branded queries.
Tell this search box to use your website’s search functionality within Google to have searchers sent directly to on-site search result pages.
Becoming eligible for this feature is relatively easy.
- Have a functioning internal search.
- Add the Website structured data element on the homepage and specify one SearchAction.
- Set a preferred canonical URL for your homepage.
Although, like all such snippets, being eligible doesn’t guarantee the box will appear.
On-Site Search Best Practices
Too often, on-site search user experience testing is limited to a quick check that the search box is appearing on desktop, then mobile, and that it renders results when a basic query is entered.
But a good looking search box doesn’t mean anything if the results aren’t helpful to the searcher.
User experience covers more than the interface alone. It’s the search functionalities interaction with the user – via the form, via search results pages and especially the logic behind the screen.
Having no site search function is better than having one that gives searchers a false impression of availability of content on your site.
As such, internal search experience testing should be rigorous to ensure results are both relevant and accurate.
Understand the state of your current search capabilities with the best practice checklist.
Being sure to test on a variety of devices to see ease-of-use and the load speeds.
✓ Search box is a reasonable width and prominently placed on all relevant pages.
✓ Search box effectively prompts user action through iconography, buttons, and placeholder text.
✓ Search box offers predictive search suggestions based on query intent.
✓ Search interface accepts both click and keyboard actions, such as enter to search, esc to close, etc.
✓ Search function accommodates casing, numbers, stop words, misspellings, synonyms, and other variants.
✓ Search function handles empty searches.
✓ Search function returns results for all document types and informational related keywords (e.g., “returns policy”).
✓ Search result pages offer a rich, on-brand experience while clearly reiterating the search query.
✓ Search result pages use semantic search to ensure relevant results for the query intent are shown.
✓ Search result pages offer relevant filters for broad queries, allow the easy removal of any filters for narrow queries.
✓ Search result pages offer to save multi-faceted searches.
✓ No results pages are set to ‘noindex’ and offer a valuable next step.
✓ If parameter-based URLs are used, the parameter is configured in GSC.
✓ A logic of URL redirects is in place to avoid duplicate content of search pages to categories.
✓ Google Analytics site search usage report KPIs are benchmarked on a regular basis and any significant shifts explained with an annotation.
✓ Google Analytics search terms and search pages reports are reviewed on a regular basis for insights into site content and design.
✓ Google Analytics event for the no result pages is in place and report reviewed on a regular basis.
✓ Website homepage is eligible for the Sitelinks Searchbox.
Now, ask yourself. Are you leveraging the full potential of on-site search?
More Resources:
Image Credits
Featured & In-Post Images: Created by author, May 2019
All screenshots taken by author, May 2019
OhNoCrypto
via https://www.aiupnow.com
Jes Scholz, Khareem Sudlow
