Microsoft has more than 300,000 employees working around the globe, and collectively, our employees use or access many petabytes of content as they move through their workday. Within our employee base, there are many different personas who have widely varying search interests and use hundreds of content sources. Those content sources can be file shares, Microsoft SharePoint sites, documents and other files, and internal websites. Our employees also frequently access external websites, such as our Human Resource websites.
At Microsoft, personas are commonly clustered based on three factors: the major organization within the company, the employee’s profession, and the employee’s geographic location. A Microsoft seller working in Latin America has different search interests than an engineer working in China, for example. As a result, it can be challenging to accommodate the markedly different interests of these two employees while enabling them to search efficiently.
In parallel, Microsoft separates search activities into two logical sets.
The first is enterprise-wide search, in which search users don’t know the exact source for the content they’re seeking but want to find appropriate content from across the company. The second is tool-specific search, when users search within Microsoft Outlook, Microsoft SharePoint, and other applications or tools they’re using, and they’re expecting to find the content they want within that application or tool.
Microsoft’s central search administration team focuses on enterprise-wide search, striving to make it as effective as possible. The scale of enterprise-wide search within Microsoft is 1.1 million enterprise-wide searches per month on our corporate SharePoint site plus Bing’s work vertical. Central search administration doesn’t currently configure search in other tools such as Outlook, internally developed applications, or individual SharePoint sites.
Microsoft Search is bringing enterprise-wide search capabilities to many different endpoints, such as the Outlook app, Microsoft Word, and Windows Search Box. Any enterprise-wide search in those endpoints is also of interest to search administration.
Based on the findings of several surveys of our employee community, we’ve categorized searches into two broad sets:
- Quick find (re-find). The employee knows that something exists and just wants to get to that content. This activity accounts for approximately 70 percent of our enterprise-wide search volume.
- Research (investigate). The employee needs information or an answer requiring some in-depth content reading, analysis, or review. This search type accounts for approximately 30 percent of our enterprise-wide search volume.
[Explore how we’re monitoring end-to-end enterprise health with Microsoft Azure. | Read more about instrumenting ServiceNow with Microsoft Azure Monitor. | Discover how we’re making content more accessible and searches more efficient at Microsoft.]
Enabling optimal search
Based on survey findings and other internal data on search activities, search administration set a primary goal: enabling employees to find the information that they need as quickly as possible.
Industry research conducted by firms such as McKinsey & Company, in its report “The social economy: Unlocking value and productivity through social technologies,” and Gartner, in “Improving the Employee Experience Improves the Customer Experience,” has revealed that that the average employee spends an estimated 6 to 25 hours per month searching for information. Employees use this information to perform current work, provide better-quality answers to customers, and other purposes. So, the benefits of search start with direct productivity gains and extend to indirect benefits that result when employees have the appropriate information available when they need it.
To deliver the best possible search experience, Microsoft search admin uses most of the capabilities that Microsoft Search provides via the Microsoft 365 admin center. These capabilities include:
- Bookmarks
- Q&A
- Locations
- Acronyms
- Connectors
Search admin also uses two additional mechanisms outside the admin portal—the SharePoint hybrid crawler and a custom content export/import process that our internal IT team—Microsoft Digital Employee Experience (MDEE)—built several years ago, before connectors were introduced.
Business impact of search
By aggregating a variety of telemetry, some of it custom developed, search administration has determined that nearly half of all searches benefit from one of the search admin capabilities. Our employees who receive such benefits average a one-minute faster search completion time than those whose searches don’t use those capabilities.
Across 1.1 million monthly searches at Microsoft, that time savings amounts to more than 6,000 hours a month of direct employee-productivity benefit. The indirect benefits that McKinsey & Company and Gartner research identified can multiply that productivity benefit, doubling or tripling it depending on how the calculation is performed.
At Microsoft, we achieve these results through an investment of 300 hours per month in search admin time.
To manage and continually streamline employee search activity across Microsoft, search admin conducts a broad range of activities on a daily, monthly, and quarterly basis, as depicted in the graphic below.
Daily activities
- Process bookmark requests: SharePoint Search and Bing provide feedback options that enable employees to submit bookmark suggestions. Review bookmark requests as they arrive and add to the collection if appropriate.
- Process support requests: SharePoint Search provides a form that employees can use to ask for help with their search. Review and answer search-support requests.
- Review new sites: Brief check of Microsoft’s internal news site for any new products, new internal sites, or other announcements that may warrant proactively creating a bookmark.
- Perform search health check: Review the search-health monitoring dashboard, note any major incidents, and appropriately forward notifications of failovers or other outages.
- Monitor connections: Review connector job logs. Resolve any errors and resubmit.
Monthly activities
- Review search log: Review logs to locate queries that have sufficient usage but produce sub-par organic results. Create bookmarks or Q&A blocks as appropriate.
- Fix broken links: Run a third-party link-checking tool to locate any broken links in the bookmarks. Replace or remove broken links.
- Update search help site: Update search help site to reflect new features and services.
- Reload acronyms: Obtain an updated list of acronyms from the corporate taxonomy. Remove the existing acronyms and replace with the updated acronym list.
- Review recurring events: Review bookmarks related to reoccurring events. Deactivate bookmarks in the off months and reactivate bookmarks for upcoming events.
Quarterly activities
- Tune bookmarks: Review usage metrics to locate bookmarks that aren’t receiving minimum usage. For us, minimum usage is three or more clicks per month, but that threshold could vary for other enterprises.
- Reload locations: Obtain an updated list of company locations from our real-estate organization, Microsoft Global Workplace Services. Erase the existing location data in the admin portal and replace it with the updated information.
- Review friendly URLs: Review the shortened links that receive the most usage and add any that meet our criteria.
Where to start
Because resources are always limited, lessons from the Microsoft experience can help organizations when they’re deciding where to invest search admin effort. The table below shows the user benefit we were able to measure associated with the activities described in earlier sections.
| Search admin activity | User benefit |
|---|---|
| Bookmarks and loading acronyms and locations from other sources | 45% of all searches utilize this information
Approximately 8,000 hours of benefit/month
|
| Connect more content | 3% of all searches utilize this information
Approximately 700 hours of benefit/month
|
| Search How-to site and Help Me Find It | 1% of Microsoft users visit the site/month
40 user requests assisted/month
|
Benefits of search admin activity.
Given these benefits, our experience leads to a sequence of three steps to take when beginning search administration, starting with the one that has the greatest user impact. The following paragraphs describe the scope of each step and the intended benefit.
1. Implement active-bookmark management. Bookmarks are the best way to support Quick Find (re-find) use cases. For instance, by defining a bookmark along with a friendly name and entering keywords for common misspellings, the search admin staff can provide a bookmark when a user attempts a quick navigation in a browser but misspells a term. Search admin can also ensure that the correct authoritative content is placed at the top of search results.
Other suggestions follow:
- In partnership with bookmark management, including acronyms and locations provides good value, but only if your search admin can import this content from other sources.
- Where bookmarks exist for a particular search term or topic, define a Q&A. The answer text lists the most common links to be followed and it describes and groups them in a logical manner based on the different personas. For instance, if cloud is a common search term, sellers might have a very different set of content they’re seeking than IT technicians would. So, creating an answer showing five to 10 potential links, with a description of the focus of each one and how they fit together as a whole, is beneficial.
2. Connect additional content. Microsoft Search, by default, covers SharePoint and Microsoft OneDrive and is working to enable additional content to be brought in from across Microsoft 365. There likely are many other content sources in any organization—internally developed applications, third-party apps, internal websites, and others. Identify the most important content sources for your company and create a way to add them to search. Multiple mechanisms can help you do this, from using the SharePoint hybrid crawler, to using Microsoft connectors, to creating custom connectors and entirely custom processes.
One way that Microsoft search admin identifies such content is from Microsoft Azure Active Directory authentication reports. Identify the application registrations that receive the most authentication activity. Another way is to ask users what they’re seeking when they search.
3. Provide search support. Within Microsoft, our search admin team has partnered with our internal library services so that any search feedback that indicates a desire for help with searching can enlist a librarian’s help with that search. That option is also available from the search self-help site.
Although search is generally intuitive, many aspects of it aren’t, such as using Keyword Query Language (KQL) to better target a query. Therefore, providing information to employees about how and where to search for popular types of content is appreciated. Especially with new employees, it’s valuable to provide a quick-reference card or equivalent resource to help them during their first weeks of employment.
While return on investment (ROI) on this service is low, users are very appreciative of it. In addition, a few times a year Microsoft has found that the content that the user accessed has directly and positively affected corporate revenue and customer service.
Active administration of search is worth doing for any large enterprise. For Microsoft, our investment of 300 hours per month yields a benefit of more than 8,000 hours of productivity gain per month. Exactly how to administer search will vary for every enterprise, but at a minimum, creating and maintaining bookmarks to popular sites and covering popular topics will provide significant benefit to your company.
Steps to take every day
We provide some minimum activity and related numbers used in the details that follow. Internally, Microsoft logs more than 1 million enterprise searches per month, and those searches use more than 80,000 unique search terms. You can adjust any of the thresholds to be more appropriate to your organization.
Process bookmark requests
SharePoint Search and Bing incorporate feedback options that enable employees to submit bookmark suggestions. Whenever one is submitted, it’s reviewed for:
- Accuracy – the suggested page provides accurate, current information that isn’t already covered by another bookmark.
- Access – the suggested page is accessible to all Microsoft full-time employees, at a minimum.
- Query usage – the suggested or potential keywords associated with the bookmark have been searched at least 20 times in one month before they’re added to the bookmark collection.
- There are exceptions, such as new products, technologies, and initiatives, for example, which may not currently appear in query logs but have the potential of becoming popular search terms.
- The SharePoint Search logs can be used to review the query frequency.
- Current visibility in search results is poor – the suggested page doesn’t appear in the first page of search results for the bookmark’s suggested or potential keywords.
If the suggested bookmark meets the established criteria, it’s added to the enterprise bookmarks.
Whenever possible, search staff capture the name of the content owner that the bookmark points to, because the staff sometimes need this information to perform future administrative work.
Process support requests
Within Microsoft, we’ve created a Microsoft Search self-help site to provide guidance on where to search for what kind of content. In addition, there’s an option to request help from our search staff to find a specific topic. Whenever those requests are received, the search staff perform their own search actions to find the desired content. Because the staff are very knowledgeable about how to use search, they typically find content that the employee hadn’t found. In sharing the results with the employee, the search staff highlight what kind of search they found most beneficial, so that the employee gains insight into how they might improve their search the next time.
Depending on the nature of the request, our search staff might also update a bookmark’s keywords or description to improve search results for the topic.
Review new sites
Review the Microsoft corporate SharePoint portal and internal newsfeeds for any new products, new internal sites, or other announcements. For each, perform a search to check the organic search results, and then create a bookmark if necessary.
Because of our established process for removing unused bookmarks, will typically create a bookmark for major new items if they appear to be of interest to a large portion of Microsoft.
Perform search health check
When current search-health telemetry is available from the product, search staff will review health metrics there. In the meantime, some custom telemetry capabilities have been created to enable monitoring of search performance. Any spikes or drops are reported to the appropriate team.
Additionally, search staff perform several searches on SharePoint and Bing manually, timing how long it takes for search results to display in each search platform and ensuring that all search functions (bookmarks, filters, and added content) are performing as expected.
Monitor connections
Search staff evaluate each search connector connection defined in our environment, making sure that the connectors are listed as healthy. So far, there have been few errors—much better performance than the SharePoint hybrid crawler, which has a high error rate. Highlights from errors that have occurred include:
- Permissions to the content source have been lost or expired. We resolve this by getting those permissions renewed.
- A new version of the connector has been received. This tends to be a problem unique to internal Microsoft, as the company is commonly an early adopter of such releases. In these cases, search admin typically must delete and re-create the connection.
Steps to take monthly
Review search log
This activity reviews popular search terms from the previous month and ensures that either good-quality natural search results appear or that a bookmark or other search aid already exists. Beginning with the search logs, the process includes:
- Excluding search terms that the organization has blocked. These include things like confidential and Attorney/Client Privileged.
- Excluding search terms that have a frequency below a certain threshold. Our current minimum threshold is 50; other companies may set a different value.
- Excluding search terms that are in quotation marks or that indicate testing or monitoring, such as aaaaaaa.
- Exporting the current bookmark list and all associated keywords. Exclude search terms that already exist as keywords.
- For the search terms that remain, consider the following:
- Perform a search using that search term and evaluate the results.
- If the results are fine as they are, no action is needed.
- If the content that’s the best fit for that search term isn’t positioned near the top of the first page, create a bookmark or add to an existing bookmark so that the term is positioned appropriately.
Fix broken links
Keeping bookmarks and other admin-configured links accurate is a constant challenge at Microsoft. User feedback, as described earlier, is one mechanism that we use to learn about these issues. Another is the monthly broken-links check process. That process includes the following activities:
- All bookmark URLs are exported to a plain text list.
- Search staff process the list through a third-party link checker and review each URL that returns an error code or redirect notification. The decision-making guideline follows; note that you may need refinements and exceptions for your organization.
If a “Not Found” error was returned. First, manually research the topic to determine if a replacement site has been created. If so, update the bookmark with the new URL. This manual research includes performing searches and investigating the results returned or contacting the site owner—if search staff has that information or can locate it easily. Otherwise, remove the bookmark.
If a “Redirect” error was returned. If the bookmark URL is a registered friendly name, ignore this error. If the redirect URL simply adds “/EN-US/” (or something similar), ignore the error. Finally, if the redirect URL only adds query parameters, ignore the error.
Otherwise,, manually navigate to the original URL, follow it to the new URL, and then validate that the new URL content matches the bookmark. If so, update the bookmark with the new URL.
If a “Restricted Access” error was returned. Ignore the error, because even though search administration doesn’t have access to the content, the URL exists.
Update search help site
To help our user community work more efficiently when they’re searching for information across the company, we’ve created a SharePoint portal specific to search. That portal has a few major components, which are structured as sections within the portal:
What to use when. This section provides brief descriptions of the major search tools available within Microsoft and the type of content that they encompass. This covers our primary enterprise-wide search portal, Bing’s work search, and other search tools specific to individual personas or divisions within the company.
How to use major search tools. This section provides tips on how to use major search tools (SharePoint, Outlook, and Microsoft Teams, for example). It also includes a subsection on how to make content more visible, targeted to content owners.
Help me find something. This section enables anyone in the company to ask the search-support team to help them find information on a topic. Employees can also use this tool to make suggestions or provide feedback.
Reload acronyms
At Microsoft, this activity involves using the SharePoint Term Store to maintain an overall corporate taxonomy. This repository includes a large set of acronyms, which are loaded into the search admin portal. Search admin staff manages acronym refresh as follows:
- Corporate exports acronyms from SharePoint Term Store.
- Search admin checks the layout, adjusting it as needed, and then imports it by using the acronym import capability in the search admin portal. This process performs a complete overwrite of existing acronyms, because it’s easier to eliminate incorrect or outdated acronyms than it is to selectively identify needed changes.
Review recurring events
Every company stages major corporate events that occur on a regular basis, perhaps annually, quarterly, or monthly. Bookmarks for these events are very popular in the weeks leading up to and following such events but aren’t needed outside of that timeframe.
Search admin maintains a simple spreadsheet of these events and their frequency/active dates.
- If an event is coming up and has reached its appropriate timeframe, search admin confirms that the event is still happening, locates the official information site, and creates a bookmark.
- If the event is coming up, but there doesn’t appear to be an official site or the event has been canceled, the spreadsheet is updated appropriately by updating or removing the event.
- If the event has passed and has reached its appropriate timeframe, search admin removes the bookmark. Disabling the bookmark is another option.
- It’s rare, but possible, for an event to become more frequent, such that the bookmark can remain active indefinitely. In that case, remove the event from the spreadsheet.
Steps to take quarterly
Tune bookmarks
When this telemetry is available from the product, search staff review existing bookmark usage data. In the meantime, it’s possible to export the search logs and merge them with bookmarks by using the keywords.
- Search admin only reviews bookmarks that have been active for the previous 90 days. This allows new bookmarks, especially for new events or content, to have time to generate usage.
- Bookmarks receiving fewer than three searches in the previous 90 days are removed.
When telemetry allows, search admin wants to:
- Remove bookmarks receiving fewer than three clickthroughs in the previous 90 days.
- Remove bookmarks that were displayed fewer than 100 times in the previous 90 days AND were clicked less than 25 percent of the time.
Reload locations
The Microsoft Real Estate and Facilities (RE&F) team maintains a database of all corporate locations, which search admin accesses via a Microsoft Excel add-in. Search admin filters the locations to only those to which employees are likely to travel—that is, locations that aren’t inactive or under construction.
The search administration portal has a location import capability, which accommodates loading this information. Because location loading is more complex than acronym loading, our steps are:
- Get the current list of official corporate buildings from our building management system.
- Reconcile to the current list in search administration.
- Make appropriate changes.
Review friendly URLs
Within Microsoft, friendly URL names are very popular, and we’ve developed an application to manage them. As a result, search staff can ask the friendly-name apps team for a periodic export that includes the friendly name and how often the name has been used in the previous 30 days.
Any friendly name meeting the following conditions will have a bookmark created for it:
- More than 1000 uses in the previous 30 days.
- Links whose destinations are:
- Associated with the company
- Information pages (versus files, download pages, or videos)
- Not already indexed or bookmarked
- URLs that provide a clear indicator of their title or purpose
