The path to discovery

Sarah Hickman Auger, product manager, Innovative Interfaces

Good discovery tools are important to provide efficient access to the constantly expanding collection of research information in various different formats. Until all the resources are behind a single search box, there are going to be holes in search results. Discovery tools provide a way to gather everything together – articles from academic journals, conference papers, dissertations and theses, print materials, local digital assets, archives, and more; they can help ensure researchers don’t miss anything relevant to their work.

Libraries need to maximise the return on their investment in scholarly content by ensuring researchers are exposed to, and are able to access, all the collections relevant to them. However, staff resources have not grown in proportion to the size and diversified nature of the collections they are curating. Libraries must support article-centric research across vast and varied content by a population that is, by definition, transient. Opportunities for educating these researchers about available resources or the best practices for discovery are limited; also, many users do not see themselves as needing instruction to support their research activities.

However, even as web-savvy users are more skilled in evaluating the quality and bias of online content, they are less aware of the breadth of their library’s resources. Additionally, libraries observe that while their users recognise the superior value of library resources, those same users are often unwilling to sacrifice the convenience and user-friendly nature of freely available internet resources. Libraries face the challenge of remaining relevant to users who have great needs but little patience for anything more time-consuming or complicated than a web search engine.

With discovery tools a single search box offers a clearly defined starting point that is inviting to users of all levels. Local collections – both physical and digital – are highlighted alongside subscription content in a single search interface. Central indexes with massive coverage assure researchers have access to specialised sources for their specific disciplines as well as results across disciplines for very narrow topics.

Critical features of good discovery tools include relevance and currency, and known title searching, which must work equally well for articles and books. They also need to provide breadth and depth of coverage. All library collections need to be represented – physical, digital, local, subscription, and openly available – and it’s not enough to just ‘scratch the surface’ with thin metadata.

Full-text and subject indexing aids in discovery of relevant articles, and community tagging with emerging or highly-specialised vocabulary makes materials more findable. Facets allow users to start broad but quickly focus, using the literature in real-time to refine their research questions. Related searches and recommended materials suggest alternative discovery paths. Citation tools make it easy to manage sources as they are discovered. Finally, direct delivery of full-text whenever possible, combined with integrated request processing or direct consortial borrowing, ensures discoveries lead to access to the actual content required.

It must also be easy for users to access content once they discover it, through direct access to full text online or downloadable e-books or other digitised content. This also includes integrated requests for print materials – even those that are not owned by the local library. It doesn’t matter how good or how many features discovery offers if it’s a burden for staff to keep up, or if the contents are always stale – even by just a few hours.

Another important requirement for discovery is that it works out of the box and that it can be customised and configured in response to user feedback and changing needs. Having everything in one place is key. This means not just bibliographic information, but also user account data, course information, event details, and options to incorporate locally developed or third-party tools and services.

A major challenge of putting discovery tools on top of existing library structures is the requirement to integrate systems and services that are built and supported by different entities. Discovery is designed to integrate content from different providers, services from different vendors, and even locally developed tools. Attempting to integrate everything into a single, unified user experience presents issues. Fortunately, the trend toward APIs gives us a common framework to approach these challenges. A service-oriented approach allows discovery tools to bring together diverse content stores spanning bibliographic content, enriched content, and the digitised materials, as well as various niche services such as citation management software, live reference chat, social networking features, local finding tools and more. Outward-facing APIs from discovery systems allow libraries to integrate discovery features into existing services.

Another key challenge is that of managing authentication. It’s critical to protect publisher rights but also critical to provide authorised users with seamless access to that protected content. Integrating with existing authentication and verification schemes is important to enable users to navigate across platforms and view all the content they discover.

By integrating with industry-standard solutions and building and using APIs, discovery providers can deliver solutions that work for end users as well as university IT departments.

Additionally, libraries do not want to continue to maintain multiple public interfaces. It’s important that discovery tools are robust enough to completely replace traditional OPACs (rather than simply sit on top of them).

In the future, people will be looking for discovery tools that will allow them to ignore their legacy OPACs with the assurance that everything anyone really needs to do can be accomplished within the discovery interface.

John Law, vice president of discovery solutions, Serials Solutions

Users are often confused by the complexities of navigating library websites and resources and can bypass the library in favour of the convenience of Google and other open web search engines. As a result, the importance of the role of the library as a gateway for locating information has diminished and the library is increasingly dis-intermediated from the research process. To help meet users’ needs, libraries need to provide systems that look and function more in line with Google, Amazon and other open web search engines.

When Serials Solutions introduced web-scale discovery to the market four years ago, with the launch of the Summon discovery service, it was with a mission to return researchers to the library. The Summon service is a digital front door to the library, designed to meet the needs and expectations of today’s users.

Libraries are often first thought of as repositories for information. Increasingly, there has been a focus on the unique value they provide to institutions through their interactions with researchers, students, and faculty. Libraries are turning to technology to provide users a compelling place to start their research, and offer the ability to scale their services to reach more users, both virtually and in person.

We are finding that discovery services also allow librarians to become facilitators of learning. Librarians help users assimilate more fundamental and meaningful information literacy skills. It is important to bring librarians into the discovery experience so they can provide expertise and guidance to users directly when and where it is needed.

We hear from libraries that Summon is creating a positive and measureable impact – by exposing users to all of the library’s resources, including research guides, recommendations, librarians and more. Librarians are able to guide users to better research outcomes, and libraries are finding user satisfaction with the library and usage of library resources is increasing.

Discovery is much more than a single search box; services should deliver an experience that matches user expectations developed from searching the open web. A discovery service should employ the latest technologies and design techniques to deliver a user experience that is familiar, fast and easy to navigate.

A discovery service’s underlying architecture is very important. Features such as speed, unbiased relevance ranking, dynamic user display options, granularity of facets, ease of navigation across result sets, ability to pre-scope searches, discipline-scoped searching, recommendations and suggested searches, data-driven design features and important contextual guidance are all dependent on the architecture.

The unified index architecture of the Summon service allows users to search and navigate across almost all the library’s resources. In building the Summon index, we use our ‘match and merge’ technology to combine rich metadata and full text from multiple sources to make items more discoverable.

A discovery service should provide contextual guidance to advance the research experience for both novice and experienced researchers, and provide greater opportunities for librarians to engage with users in new ways. We offer librarians opportunities to impact and improve the overall discovery experience through localised recommendations, automated search guidance, and live reference help.

Putting discovery tools on top of existing library structures delivers a service that is not optimised for discovery and creates challenges that don’t exist in services developed explicitly for discovery. The Summon service is not a repurposed database platform or next-generation catalogue. It’s a built-to-purpose discovery engine, providing librarians opportunities to scale their services to connect with more users.

User behaviour and research needs change constantly, and librarians and researchers will expect library discovery services to evolve accordingly. Both the library and vendor community will need to continue to learn about how users do research and utilise that knowledge to drive development of discovery services and further improve the research process.

Clive Wright, senior director of discovery products (SaaS) for Europe and Latin America, EBSCO

When researching a topic, users are faced with an enormous number of possibilities but simply want to find the best research results in as few steps as possible. Libraries have a plethora of databases, journal collections and books, including premium information that cannot be found on the web, but the user is not always familiar with where the pertinent content is located. A good discovery tool allows the user to perform a single search of a library’s entire collection completely and efficiently, quickly find the most relevant and valuable content available, and accurately link to the full text immediately.

Libraries face the challenge of competing with Google for the attention of users and need to be able to differentiate themselves from Google. They also have to fulfil the needs of all levels of researchers – allowing both inexperienced and expert users to search according to their skills and be presented with the most relevant information.

Libraries must select discovery tools that use the power of subject indexes, make use of subject headings for the most appropriate relevancy in result lists, and integrate with catalogues, institutional repositories and library management systems.

If constructed properly, a discovery service will expose the most relevant and valuable articles to the end user. While end users may not care how the best results appear on the top of the list, they do care that the results are precisely suited to their needs. Whether they get a million results or a hundred results is not important to the user; they just want the first page of results to provide the most useful articles, books or images. The relevance of results must be obvious and immediate.

There are hundreds of millions of potential records so algorithms determining relevancy are increasingly sophisticated and significant. An algorithm focused on subjects in detailed indexing and metadata will reveal the best results. Additionally, if a discovery service can work seamlessly with ILS vendors, libraries can evaluate discovery services independently of their ILS and choose the best option.

Sophisticated API options and partnerships between discovery vendors and ILS vendors allow libraries to use the discovery service they feel provides the best content and relevance-ranking algorithm via a familiar interface. Users can begin their search in the ILS interface or the discovery service interface. The goal is to create seamless options for libraries. However, making such partnerships happen is not easy, given the underlying competitive stances. Integrated library system vendors are likely to partner only with discovery service providers that are not direct competitors in the area of ILS.

Jeffrey Penka, director, global discovery and syndication services, OCLC

Often, academic and research library collections are vast, disparate and provide separate interfaces for subsets of the collection. The search mechanisms also vary from tool to tool. If users are unable to navigate the resources provided by their library, they will turn to the general web and look to resources that are generally available and not necessary from scholarly publishers.

Libraries face the challenge of being visible to potential users and staying relevant to current users. Users expect services to integrate around their needs, which requires libraries to serve users in their preferred workspaces, like e-learning portals and the web, rather than assuming the users will come to the library or library site.

These challenges are further complicated by cuts in library budgets, which require the library to understand their users’ content needs and usage patterns more effectively.

A good discovery offering not only integrates a library’s resources into a single search interface and search tool, but will also provide the capability for libraries to weave their content and services into other services the library or institution purchases or users elect to use.

OCLC’s strategy is to combine discovery services like WorldCat Local with syndication activity through partnerships with consumer service providers in academic search (such as Google Scholar and Microsoft Academic), citation management (such as EasyBib) and others.

This approach means that library resources are included automatically in millions of research workflows – even though the researchers themselves didn’t originally start their search on a library website.

Users expect a simple search box to return high-quality results and allow access in some form to those results. OCLC is working with libraries and commercial partners to ensure their collections are being discovered and accessed outside the library, thus re-enforcing the renewed visibility and relevance of the library.

Such tools also need to connect discovery and delivery or fulfilment. Finding a description of an item with no options to obtain the item is more frustrating than not finding the item at all.

Given OCLC’s content-neutral position, we have been able to establish partnerships with many library service providers, content publishers, and consumer web services. These relationships have helped build an index representing over a billion items including electronic/licensed, print, and digital items held in libraries worldwide.

Embedding discovery where the user is already working is also important. Users often begin their discovery process outside the library. Library users, learners and researchers also expect to find the information they need through whatever device they’re using – be it laptop, desktop, mobile or tablet.

Good discovery tools should always support mobile-optimised views for a wide range of devices. A full discovery offering must support the library’s ability to integrate with other tools that the library purchases, the patron chooses to use, and the university/community employs (MOOCs, eLearning portals).

Tamar Sadeh, director of marketing at Ex Libris

When they were first introduced, discovery tools revolutionised the role of the library in presenting and publicising its collections to users. Whereas researchers’ previously needed to search in multiple destinations for relevant items, library users can now access all items – from the physical collections on the shelf and the library’s unique collections, to articles published online – with ease and speed. A discovery system changes users’ perception of the library’s information system: usage statistics show an increase in the number of searchers and the number of downloads of scholarly content once our Primo tool has been implemented.

Discussions with our users have revealed features that are most valuable to librarians and users. The first is quality and breadth of relevant content. In addition to offering library local collections – the library catalogue, digital repositories, teaching aids, and other local collections – a discovery system serves as a gateway to global information. This includes journal articles, e-books, newspaper articles, reviews, patents, and legal documents that are harvested from primary and secondary publishers and aggregators and from open-access repositories. While the breadth of information is important, the quality and relevance of the materials is crucial. The library needs to determine which collections should be included in the search scope of its users, and how they should be presented. We promote the discoverability of open-access materials through our initiative of enabling libraries to add their institutional repositories to Primo Central. We also maximise the visibility of open-access materials in several ways, such as exposing the availability of open-access articles published in hybrid journals.

Discovery systems offer an information landscape that includes hundreds of millions of scholarly materials but do not require users to demonstrate high searching literacy. This means that queries often yield a large result set. Discovery systems must be capable of presenting the most relevant results first. We are investing a lot of effort in the Primo ScholarRank relevance ranking technology. This technology takes into account many factors that help determine which results are most relevant to a specific searcher, including a document’s scholarly significance, and the researcher’s discipline and academic degree.

In many cases, a query is not specific enough and the user needs to refine the results – either by adding search terms or by using faceted navigation (focusing on one or more subsets of the result list). Using the option to select some facets and exclude others, results can be narrowed down.

Such systems must also incorporate an effective link resolver enabling users to reach online content seamlessly. Primo integrates with the Ex Libris SFX link resolver and others, and implements stringent quality testing on the dynamic, context-sensitive links to millions of resources. It also supports item-level, pre-validated linking whenever possible.

We have services to enrich the search experience by serendipitous discovery, based on usage data analysis of millions of researchers.

Discovery systems are most effective when they integrate with existing systems in the institution – for example, course management systems.

In order to reach users as conveniently as possible, the search box must appear in a range of contexts, such as departmental web pages and recommended reading lists. One of the major requirements of our customers is to replace their traditional OPAC and integrate the OPAC functionality into the new discovery layer.

Ex Libris enables Primo to be offered as the discovery layer for almost any ILS, through the Primo APIs. Access to library services such as loans and hold-requests via the Primo interface is in operation at hundreds of Aleph, Voyager, and Alma institutions worldwide. Some Talis customers will soon switch to offering OPAC via Primo, and BIBSYS is developing OPAC via Primo for their ILS. Other systems such as SirsiDynix Symphony and Horizon are supported through mashup.

Institutions have begun to expect their discovery system to offer a mobile interface for users who connect to the web via mobile phone or tablet computer.

The next trend that we foresee is a demand for a personalised search experience. The ability to tailor results enables a discovery tool to serve each researcher in his or her own context. Services based on analytics are likely to provide further insights that will help researchers locate publications of interest.