Keynote from Enterprise Search Europe

Six key drivers search quality from Ed Dale

This is the keynote I gave in London in May. I think the concept of having a vision for how you will improve search is important. You can see in this presentation my perspective of having a measured approach, reliant on content, that is simple and easy for the user to use.

Presentation from Enterprise Search Summit 2013

Enterprise Search Summit 2012

Enterprise Search Summit is an annual event held in New York City, now in its 8th year. The focus of the conference is on real world experiences. It is a forum where business owners of enterprise search technology can share experiences, and develop insight on the future of search technology.

 

I had the privilege of attending, and presenting, this year. I saw three main themes from the presentations I attended. 

1. We are not alone.
2. Semantic web technology is moving away from cutting edge and towards mainstream.
3. Mobile search is key for intranets to deliver value.

 

On the theme of "we are not alone", Kristian Norling of Findwise presented some initial results from a survey he conducted with Enterprise Search product managers. In the survey, the majority reported poor user experience with search, significant need for information retrieval, and minimal staffing.  He was kind enough to post his slides on Slideshare.

 

Results from the Enterprise Search and Findability Survey

View more presentations from Findwise

On the theme of Semantic Web concepts becoming mainstream, Tom Reamy presented on applying semantics to search text analytics - showing how to leverage semantic relationships with your search engine to provide better results. This is a mechanism to leverage the existing taxonomy and metadata within the content to provide business value and improve findability. These concepts are core to a Knowledge environment.

 

With mobile search, there were two great presentations. The first was by Colin Jeavons focused on the need for mobile search. By 2015 there will be 2 billion mobile devices across the globe. 

Should we deny our users the ability to use these devices access to our internal resources? 

The key differences are that mobile search is browse & go, not sit back and browse, it is expected to be precise based on location and expectations are being set by Siri for direct response to the users search queries.

 

The second mobile presentation was from Tyler Tate of Twigkat. He focused on direct UI changes required for mobile search, and was kind enough to post his presentation on Slideshare as well.

 

My presentation focused on practical steps any enterprise search team could undertake, with minimal budget, to improve the findability of their content. It focused on the challenges we faced, the solutions we implemented and the results of those solutions. I have shared this presentation on Slideshare, if you are interested.  

 

The material was well received. There were a number of thoughtful questions probing on how to implement the same solutions within other environments and a few people who were not at the conference have asked for the material based on the feedback from Twitter and LinkedIn.

 

Why Angry Birds is so successful and popular: a cognitive teardown of the user experience

This article takes a detailed look at a successful, engaging product and breaks down why it is engaging across 6 factors. The author discusses the use of response speed, short term memory management, visual design and cognitive theory within the game to keep and maintain interest. It would be interesting to hear from the game developers if they used this approach, or if they were guided by their intuition.

This is a great way to think about products and how to engage users through system and visual design. The article will require a few minutes to read but is worth the investment.
http://www.mauronewmedia.com/blog/2011/02/why-angry-birds-is-so-successful-a-cognitive-teardown-of-the-user-experience/

Enterprise Search what is it, why is it important, how do we measure it?

Search is important for knowledge management.
Knowledge management is the process of converting tacit information into explicit information.

• This incorporates the concepts of content creation, content storage and content retrieval.
• This embraces the ideas of content strategy.
• Content strategy is the process of determining the important content within an organization.
• Content is format agnostic, it includes text, speech, audio, video, media.

Challenges of knowledge systems are acquisition, curation and findability.

• Acquisition is the process of bringing in new content.
• Curation is the process of applying content strategy at the tactical level.
• Findability is a descriptive term for enabling the finding of content, via navigation and search.

Information retrieval (search) is required to be able to find the methodology, the sample deliverables, the thought leadership, and other content for search dominate users, and for other users when navigation fails.
Goal of information retrieval is to provide the right document to the right person at the right time.

• Right document might depend on the role of the user, the application launching the query, the profile of the user (area / subarea / country / city, the industry, service line / sub service line, service) or the time of the year.
• Right person might not be the user launching the query.
• Right time may be when they ask, before they ask, as it is created, when connected, or when on other devices.

Measure of information retrieval is the first page of results.
Right document must be on the first page of results.
Right document must be visible to the user as the right document through good title, good summary, and a clear relationship between the document and the users information need.

• This requires a relationship between information retrieval and content storage to identify the right data for the title, summary, keywords, and other metadata. This includes incorporating the right fields in the storage area and the concept of a connector between the search engine and the repository.
• This requires a relationship between information retrieval and content creation to ensure content that answers the users information need exists, and is findable. This is sometimes known as enterprise search engine optimization.
• This requires a relationship between information retrieval and content strategy to ensure the best content is found. This could be relevance adjustments, query cooking, best bets, any tool to ensure the right content is moved to the top of the list.

This can be measured by:

• Calculations of precision and recall leveraging an expert assessment of content and query.
• Calculations of mean reciprocal rank of the best document for a query, leveraging an expert assessment of content.
• Evaluation of frequent query results by users. (70% of first 10 results being “relevant” seen as good).
• Penetration rate of user population (indirect measure, should be ~30%).
• Feedback of end users via survey, feedback forms, interviews, etcetera.Feedback from primary stakeholders.

Google changes the search landscape, again

Google is now offering a new feature called Google Instant. As you type in your query, you start to get results instantly, from the first few letters. Here is a video demo of this feature. This feature is only available, currently, to registered Google users, using the latest HTML 5 compliant browsers, in the US. They will be expanding the coverage shortly. Computerworld covered the launch well. The New Idea Engineering team has some thoughts on how this works, behind the scenes.

I like how this new feature reduces the feedback loop in search. I particularly like how the feedback you are reacting to is search engine results pages, not just the suggested search terms from type ahead. Some bloggers are saying this makes SEO irrelevant. I disagree. For content creators, this means it is even more important to have your good content at the top of the right key terms and phrases search results. I am more likely to try different phrases and searches, since results are "instant". I am less likely to click on a result that does not have a strong "scent of information". A new result is only a letter away, why try something that does not appear correct? Having good content, content that looks correct to the user, has become even more important. Ensuring that this content is found by the user has become harder. The SEO challenge has changed, and increased in difficulty, but it is *not* irrelevant. This makes content even more important to your search results, not less.

What it does seem to effect significantly is SEM. My cost per impression just changed. Google will be rendering my ad more often, but users will be clicking on the ad less often - this should mean that I get more impressions for my cost, assuming I am paying per click. I think it makes the top line of SEM better - I, as an advertiser, get more value for my money. Assuming exposure offers some value. If my only value comes from clicks, the top line stays the same.

What I like about Google Instant is the fact that this leverages Google's traditional strength, SPEED. It does it in a way that Microsoft will have trouble responding to - due to the infrastructure costs that speed requires in terms of local servers to reduce response time. Google has already made that investment. I do not think Microsoft has to the same extent. This innovation leverages Google's massive dataset of user behavior as well. It puts the work on the computer, and not the user - which is always the right place to put it.

I'm not sure how much this will impact the enterprise search arena, however. Most enterprises are unwilling to support the infrastructure needed for millisecond response time in the search results page, both in terms of network and in terms of hardware. I wonder if the GSA is going to support this feature, and if so what they will specify for the network and hardware requirements.

Great model for improving expert reviews

Conducting Expert Reviews Using the VIMM Model

View more documents from Michael Rawlins.

Business value of social networking tools

The question keeps being raised about the business value of social media tools. As we work with facebook, twitter, yammer etcetera - what value are we driving? I just got a free book on search engine interfaces, through twitter. If I was not on Twitter, I would not have this interesting book to read months prior to publication. Imagine that these networked tools did not exist. So I can only interact with the people near me, in close physical proximity. I talk to people in Cleveland. I sell to the people in Cleveland and I learn from the schools in Cleveland. I'm doing pretty well for myself. Someone I know tells me - you should go to New York, they have more people, and more opportunities to sell, and more clubs, more museums, more of everything. Is it worth it? What business value will I gain from going to New York?

Until I go to New York, I cannot understand the difference in scale. Once there, I see many different distractions - shows, plays, clubs, museums, what have you. But there is also much more opportunity there. I think social networking tools are like New York - if you are there, you can't understand why anyone would not be there. If you are not there, you really can't see why anyone would want to be there.

Yes, there is a lot of non-work related things that happen on these tools. There is a lot of non-work related things that happen in the office, from the water cooler conversations to the quick coffee run. But if you are not taking part in the conversation, you will miss the serendipitous moments that happen because you are part of the conversation. Like when a water cooler conversation leads to a new idea or a shorter process. Similarly there are things that happen in Twitter or Facebook that will add value.

This is only a half formed analogy, comments are welcome and needed. Anyway, I need to get back to work.

The burger joint and the mini

So, the burger joint. It is a hotel restaurant done up to be a small hole in the wall place. They got the atmosphere right, and the burgers were great. What made this the best dinner I have ever had in NY though, was the company. Because the place is small, recently reviewed in the New York Times - it was packed. I ended up sharing the table with a couple who had been married for 32 years, but were now divorced, and a girl from Long Island now living in the city. This random meeting of total strangers made the dinner great fun. The burger was good, but the company was better.

The mini surprised me. I have not played with one before, since our local Apple store would have to order one for me. It was fast. It did not bog down playing two movies, one in HD and one in SD. Each application opened quickly, ran smoothly and was very responsive. Even with iTunes playing music and the HD movie running. All on 2 GB of RAM and a 2.0 ghz processor.

Something on this Mac was configured differently, and it had a right click on the mouse. Not sure how, since the whole top moves, but if you push on the left it reads a left click, right reads a right click.

I think I may bite the bullet and finally buy a Mac. Only a couple decisions left to make. I need to decide if I want the wireless Apple keyboard, or if I should go with the Logitech dNovo keyboard with the built in click wheel. And, do I want to put in the 4 GB right away or wait till I need a speed boost? Apple RAM is a rip off, but the thing is hard to open.

Enterprise Search Summit 2009

Attendance is way down, the economy seems to be hitting this event very hard. I am not able to blog from the conference itself as the wireless is very flaky and there are not many places to plug in. This is the first time I have seriously considered the need for a netbook. Something very small, with a very long battery life would be helpful right now.

I'm heading out now to try the Burger Joint at Le Parker Meridian, then maybe off to the Apple store to look at the Mini again.

Search, Scent and the Happiness of Pursuit

Jared Spool gave a key note on search and e-commerce. There were a number of interesting points he made. In his studies of e-commerce sites, he found that the most successful sites, used search the least. Search is our tool to be used when navigation fails. Improving the "scent" of the links on the main page, or within the navigation hierarchy, will decrease the need to use search.

UIE collects 2000 pieces of data per participant. In their studies they found that the #1 predictor of success was the # of pages the user visited between the home page and the sales cart. Fewer pages led to greater success. The best way to improve scent is to build the trigger words into the link titles, or the description of the tool. To find the trigger words, examine the search logs - those words are the words the users are scanning the pages to find. Referring again to his studies, he found that no one only went to search. Everyone started by scanning the page, looking for categories or links that contained the trigger words. They only went to search when nothing could be found.

An interesting point he made, if you can follow the users click path and know when they leave the page through search, then you can know when the trigger words dried up. Adding a link on that page that takes the user to the search term will increase success for that one trigger word.

Search works well for known item location. So it is easy to find "The Princess Bride" or "Tom Clancy". Search does not work well for less concrete items like "an inexpensive yet high quality SLR" or "Novels written by Nobel Prize for Literature authors". To solve this issue he recommends using editorial capability to improve content finability.

Looking at 77 installed versions of search engines on e-commerce sites, the same vendors that were the best, were also the worst. The difference is the implementation, not the technology. Focus on doing a good job on configuring the installation.

Emergent Social Search Experiences

This was a moderated panel discussion around social search. The panel discussed the definition of social search, why to do it and how.

They defined social search as social tagging.

You do it because it enhances the findability of an object.

There is a psychological burden of asking a question within an organization. "Didn't we hire you to know that?" There is also a cognitive load on asking and posting within an organization as people take the time to format the question well. People don't share because they don't want to tip their hand.

The Steve museum project explores leveraging social tagging to help identify and find art. They build the tools internally, but examples are on the web at www.steve.museum.com. They included a statistical analysis tool set to help tell when tags were useful. They found that the tags were useful for 80% of the time.

In a study with ACM, they found that when authors post tags themselves, they are precise but they are not very broad. They recycled the authors tags, added them to a pseudo controlled vocabulary and found more use of the tags that were created, and more tagging on the documents as a whole. Providing the leverage to reuse the tag, increased the usage of the tags. People saw the reuse as a value, giving them a reason them to tag in the first place.

In the Steve museum project, they had a two part study where some users know that they are helping an organization and others did not see a connection to a organization. The users who knew and saw the value they were adding were more than twice as likely to help.


Vendors:
ConnectBeam seems to have closed. They discussed it as a good product, but there does not seem to be any new products, and people have left the firm.
"Knowledge Plaza is interesting." No other data given on the product.
Arrdvark - ask a question, be routed to someone to answer it. Most of the QA is private.
Hunch is a system to find previous questions.

Panel sees these tools as a way to pull tacit knowledge from users.

Another interesting slidedeck ...

http://www.slideshare.net/lrosenfeld/marrying-web-analytics-and-user-experience

Search Analytics

This presentation was an interesting overview of leveraging your metrics to help improve search and the overall web site. Lou spoke to working with metrics from the bottom up and from the top down.

Bottom up, he suggests, is working with the information and asking questions. Do not get caught up in specific statistical analysis methods, instead just look at the metrics and see what questions you have. Examples of these are:
- What are the most frequent search queries?
- What are the most frequent results clicked on?
- Which pages are referring the most queries?
- Which pages are launching the most 0 hit queries?

He suggested that you should look at your search results to create better metadata. This involves looking at your top 50 to 100 queries, developing a set of categories that these queries might fall into, then seeing what percentage of your queries fall into each category. Once you have the values for the categories, you can also look at the attributes.

You can also scan your data for content types that people are searching for. If you track this data over time, you begin to see the seasonally of the queries and the adaptations that the enterprise makes to its environment.

He suggests digging deeply into the failure states, specifically for the pages within the site that are not working. Look for where the *content* is failing as well as where the search is failing.
His last suggestion was to leverage your best bets to become a A-Z listing for the site. He showed an example from the University of Michigan where the list was of the terms the best bets were created, paired with the best bets. Seems a simple, quick win.

Top Down is the process of thinking about the larger business issues and seeing what the metrics can tell us about them. So if the bottom up is thinking about the most common queries, top down would be the question "can users find the items they are looking for, with these top queries?" His actual example was e-commerce conversion for the top queries.

The slide deck will be available at SlideShare.net, but here is a similar one from a few years ago

Enhancing Findability and Usability through taxonomy and search

This presentation was on the specific IA methodology of the presenter and the tools she uses in her state to ensure a consistent taxonomy across all of the web sites. The IA methodology was consistent with the best practices we employ with our CHS PM's. There was a succinct definition of IA - the goal is to have information be findable, manageable, and usable.

The methodology starts with establishing the goal of the web site, it's users, and it's content. They then use this information to create a taxonomy, a labeling and naming structure as well as the navigation of the site. This process is similar to the one outlined at http://www.finance.gov.au/e-government/better-practice-and-collaboration/better-practice-checklists/information-architecture.html. She referenced this checklist as a good example for anyone to follow.

Social Search for Knowledge Sharing

This was a presentation on a interesting community of practice development program run by the UK government. The project created a NING / Lotus Connections like environment for the 410 local community governments. These represent a workforce of 2.1 million people who deliver 700 common services to the population. The services range from marriage to road improvements and a total budget of 160 million pounds.
The goal of the project was to enable knowledge sharing across the local governments. This would allow a person in Chichester to learn from a similar project completed in Yorkshire. In addition, the groups would create a sense of community across the UK on various topics of interest. The tools provide a blog, a wiki, a library, a discussion forum, and search. There is no governance of the social networking application, communities have formed around the Cornish language as well as around road improvements.

The search implementation is interesting. It allows a community to identify the top 20 web sites for the community. It leverages Exalead's external Google like search engine to then search those web sites via a typical search box. From the results screen the user can further narrow the search by eliminating any of the web sites from the results. This gives the community the ability to dynamically create a custom domain of public web sites. This is similar to the functions EY has embedded within the Community HomeSpace product for internal search, but focused instead on the greater world wide web.

They are now able to leverage the social intelligence of the entire group, the "wisdom of the crowd". The feature has been taken up by 15% of the communities and the self reporting is very positive. Overall they find that the strength of the community reflects the skill of the community manager. A good community manager who keeps the conversation alive, ensures that discussion posts are followed up on, that the content added to the site is fresh, and who "weeds and feeds the garden" will have the strongest communities.

Enterprise Social Bookmarking at MITRE

This presentation was an overview of the Onomi product development and deployment at MITRE.

The initial project goal was to understand the use of social bookmarking within and enterprise. Specific sub goals were on social bookmarkings ability to provide an environment for knowledge sharing, ability to feed expertise finding or user profiling, ability to form networks and to enhance the value of information retrieval.

Features of Onomi:

• Based on a open source product called Scuttle
  
• Similar interface to del.icio.us
  
• People create the bookmarks with a title, URL, description
  
• Bookmarks can be kept private
  
• RSS feeds are available
  
• It is integrated with e-mail subscription service
  
• Incorporates broken URL checking
  
• Incorporates a people network section
  
• Related tags
  
• Related users
  

Some interesting data points on the Onomi implementation:
-83% of the tags are from external web sites.
-86% of the tags are shared
-50% of the company uses the software
-23,676 bookmarks total in the system
-130,310 total tags
-16,371 unique tags

The search engine indexes all of the public bookmarks in the system and presents the bookmarks as search results. The result indicates which tags have been applied to the term. The integration is done via XML feeds. It is still not 100% complete, a separate capability to search only tags has not been created. They integrate the bookmarks and tags into a separate search ala OneBox on some of their different enterprise searches experiences. They are planning on leveraging the items that are bookmarked for relevancy improvements in a future release. The bookmark search results include a presence awareness to allow users to IM or call the author of the bookmark.

An interesting capability they have is around best bets. They enable anyone to identify a bookmark as a potential best bet. This automatically populates a database of potential best bets which is searched. Results of the search against this database are presented similar to our EY Professionals Recommend. It is different from our approach of specifically identifying promotions for key words.

Free keynote

The Conference is May 12 & 13 at the Marriot in NY.
Anyone who registers using this link ( https://secure.infotoday.com/forms/default.aspx?form=ess2009&priority=COVVIP3 ) will have the option of joining us for one or both keynotes for free, as well as visting our exhibits hall.

In case you want to include details, this year's keynotes are:

May 12 Keynote: Harness Information to Deliver Enhanced Business Performance
9:00 am – 9:45 am
Ramesh Harji , Head of Information Exploitation, Capgemini UK
Despite the billions of dollars invested in information technology, organizations are still failing to realize the latent potential of their information. To be successful, organizations need to take a different approach—one that views information as a critical business asset, not an afterthought. Organizations must put exploiting information at the heart of the way they do business. A recent Capgemini research report concludes that poor information exploitation is costing the U.K. economy over $100BN/year in lost profits, representing an average 29% suppression of business performance per organization. Better information exploitation is one of the last bastions: an opportunity to both grow revenue and improve profitability.

May 13 Keynote: Improving Security Through Information Awareness
9:00 am – 9:45 am
Wim van Geloven, VP Information Technology, National Coordinator for Counterterrorism, The Netherlands
Approximately 20 agencies in The Netherlands are involved in combating terrorism. To boost the effectiveness in understanding terrorism and serious crime, four organizations decided to join forces in a unique venture, dubbed Improving Security by Information Awareness. The Program strives to improve the quality of intelligence and investigations within the public security sector. The approach revolves around the collaboration of the different agencies, operates from the perspective of the operational cases, and defines the IT case from these basic principles. Over the past 3 years, the Program invested in pan-organizational change, as well as IT techniques and also the (scientific) development of methods and techniques that improve the exchange, presentation, analysis, and storage of large quantities of data. This keynote will address the background and problems faced, the technological and organizational challenges, encountered, and the way in which this new approach has reformed information awareness.

Enterprise Search Summit 2009

As a speaker, I can offer you a $200 discount on the Gold Pass or Full Two-Day Conference Pass, with the option to attend the Showcase for free.

For this offer, please use this URL: https://secure.infotoday.com/forms/default.aspx?form=ess2009&priority=SPEAK3 file://localhost/forms/default.aspx.

Or use discount code SPEAK3 when registering.

Look forward to seeing you there!

Enterprise 2.0 can be funny!

How Great Ideas Are Killed

View more presentations from someguy05896.