Tuesday, March 22, 2011

Enterprise Search what is it, why is it important, how do we measure it?

Search is important for knowledge management.
Knowledge management is the process of converting tacit information into explicit information.
  • This incorporates the concepts of content creation, content storage and content retrieval.
  • This embraces the ideas of content strategy.
  • Content strategy is the process of determining the important content within an organization.
  • Content is format agnostic, it includes text, speech, audio, video, media.
Challenges of knowledge systems are acquisition, curation and findability.
  • Acquisition is the process of bringing in new content.
  • Curation is the process of applying content strategy at the tactical level.
  • Findability is a descriptive term for enabling the finding of content, via navigation and search.
Information retrieval (search) is required to be able to find the methodology, the sample deliverables, the thought leadership, and other content for search dominate users, and for other users when navigation fails.
Goal of information retrieval is to provide the right document to the right person at the right time.
  • Right document might depend on the role of the user, the application launching the query, the profile of the user (area / subarea / country / city, the industry, service line / sub service line, service) or the time of the year.
  • Right person might not be the user launching the query.
  • Right time may be when they ask, before they ask, as it is created, when connected, or when on other devices.
Measure of information retrieval is the first page of results.
Right document must be on the first page of results.
Right document must be visible to the user as the right document through good title, good summary, and a clear relationship between the document and the users information need.
  • This requires a relationship between information retrieval and content storage to identify the right data for the title, summary, keywords, and other metadata. This includes incorporating the right fields in the storage area and the concept of a connector between the search engine and the repository.
  • This requires a relationship between information retrieval and content creation to ensure content that answers the users information need exists, and is findable. This is sometimes known as enterprise search engine optimization.
  • This requires a relationship between information retrieval and content strategy to ensure the best content is found. This could be relevance adjustments, query cooking, best bets, any tool to ensure the right content is moved to the top of the list.

This can be measured by:
  • Calculations of precision and recall leveraging an expert assessment of content and query.
  • Calculations of mean reciprocal rank of the best document for a query, leveraging an expert assessment of content.
  • Evaluation of frequent query results by users. (70% of first 10 results being “relevant” seen as good).
  • Penetration rate of user population (indirect measure, should be ~30%).
  • Feedback of end users via survey, feedback forms, interviews, etcetera.Feedback from primary stakeholders.