Title of project: Literature Searching service evaluation

Lead library: Integrated Library Service (ILS)
Wirral University Teaching Hospital NHS Foundation Trust
www.whnt.nhs.uk/library

Summary: The Literature Searching service provides clinicians and non-clinicians with a summary of the available evidence on their chosen topic. This may be in response to a question about patient care, to support audit and research activities, to prepare for a meeting or project, to support educational activities such as Journal Clubs, or simply to keep up to date in an area of interest.
The library service decided it would be a valuable exercise to evaluate this service to try to measure its impact on organisational activities and decision-making.

Alignment to local, regional and national drivers: The results of the evaluation highlight the ways in which the Literature Searching service is aligned with key local and national drivers.
Local drivers
During 2009/10, the key goal of the Trust was to improve the delivery of excellence in quality and safety. The Literature Searching service can support this goal by providing high quality evidence to support the clinical and business decision-making necessary during the improvement process.
National drivers
The Literature Searching service supports the QIPP (Quality, Innovation, Productivity and Prevention) agenda.
Quality: The evidence on which clinical and business decisions can be based is from high quality information sources.
Innovation: The Literature Searching service is expanding outside of the library to provide evidence searches ‘at the elbow’ of the clinician or manager, e.g. at ward rounds or team meetings.
Productivity: Evidence searches are carried out by trained Librarians, saving clinicians and managers valuable time. The results of searches are summarised in an easily digestible format promoting quicker and easier access to the evidence base.
Prevention: Where appropriate the results of a Literature Search can include ‘horizon scanning’ whereby areas of potential risk and examples of good practice in other Trusts are identified. The service also contributes to the reduction of knowledge related risk.
The white paper, Equity and Excellence: Liberating the NHS, emphasises the importance of improving healthcare outcomes and patient experience. The mobilisation of best evidence, which can be supported by an efficient Literature Searching service, is crucial to the delivery of high quality services.

Project team: The service is provided by the Clinical Librarians and Site Librarians within the Integrated Library Service.

Key Audience: The service is available to all staff within the following Trusts:

• Wirral University Teaching Hospital NHS Foundation Trust
• Clatterbridge Centre for Oncology (via Service Level Agreement)
• NHS Wirral (via Service Level Agreement)

The service responds to search requests on clinical topics as well as non-clinical.

“The story”: The Clinical Librarian Service implemented a pilot evaluation to capture data from everyone who received Literature Search results between 1st Jan 2009 and 31st March 2009. This was carried out as part of a wider regional project which involved collaboration with other Clinical Librarians in the North West. The aim of the pilot was to develop a generic evaluation questionnaire which could be localised and implemented by other library services in the region.
The pilot achieved a 62% response rate to an electronic evaluation questionnaire that was created using Survey Monkey. The pilot was considered a success and minor amendments were made to the evaluation questionnaire.
It was decided that everyone who received literature search results would be sent a link to the electronic evaluation questionnaire after a period of three months, in order to give them time to digest and utilise the information provided. This would hopefully allow us to measure ‘actual impact’ rather than ‘intended impact’.
An evaluation of the period 1st March 2010 to 31st July 2010 showed that search results received during that period had high impact on a range of Trust-wide activities, including direct patient care, management decision-making, guidelines development and educational activities.

Benefits of this service for the organisation: Evaluating this service allows users to contribute to the improvement of the service. It also allows the organisation to gain a further understanding how this service is contributing to Trust-wide goals and objectives and aligned with national NHS drivers.

Benefits for the library:

• Evaluating the Literature Searching service has many benefits for the library service:
• Impact evaluation: Assessing the impact of our services is an ongoing driver for library and information services in the NHS.
• Improving services: Feedback gathered as part of the evaluation will assist us in evaluating the quality of the existing service and identify areas for improvement.
• Demonstrating value: The evaluation process provides formal feedback on how the library service adds value to the wider organisation.
• User engagement: Evaluation allows us to involve service users in adapting and improving services.

Evaluation: The results indicate a high level of satisfaction with the delivery of the service. 100% of respondents either ‘Agreed’ or ‘Strongly Agreed’ that the search results satisfied their information needs.
92.8% of respondents either ‘Agreed’ or ‘Strongly Agreed’ that their search results were received on time, although one respondent commented that their “results [were] received in good time but not as soon as hoped for”.
However, 100% of respondents would recommend the service to a colleague. One respondent commented:
“Excellent service, good quality information really helps when researching new ideas.”

Impact assessment: The results signal a number of ways in which the service impacts on clinical and non-clinical activities.
Enhances existing knowledge of healthcare professionals
92.3% of respondents either ‘Agreed’ or ‘Strongly Agreed’ that the search results revealed new information to them, and 84.6% either ‘Agreed’ or ‘Strongly Agreed’ that the search results substantiated what they already knew. This demonstrates that the service has a role in providing new evidence to users, but additionally in acting as a quality control mechanism that enables users to corroborate their understanding with reference to the evidence.
The role of the service in highlighting gaps in the available evidence is often an equally useful outcome. One respondent said the service was “very useful as I thought I was looking in the wrong places for information but the service confirmed the lack of information available on the subject”. Another commented:
[The service] “enabled me to use information that I would otherwise have thought was out of date by confirming there was very little on the subject matter”.
Having a trained Librarian undertaking the search also enabled users to be confident that a thorough and systematic search had been conducted. One respondent commented that:
“[The service] identified work which I had not previously found, reassured me that I had not missed any other relevant work.”
Encourages usage of additional library services
As a result of receiving literature search results, 21.4% of respondents requested articles through the library, demonstrating a positive impact on usage of other library services.
53.8% of respondents said that using the literature searching service had prompted them to seek training in the use of electronic / library resources.
Promotes an organisational culture of evidence based practice
The evaluation data identified that search results are not used in isolation by the individual but are shared among a team. 71.4% of respondents shared the information with colleagues within the organisation, demonstrating that the service facilitates the dissemination of evidence into practice amongst the wider healthcare community.
Additionally, 78.6% of respondents obtained the full text of articles for further reading, indicating that the service promotes increased reference to the evidence base.
Supports a range of Trust-wide activities
The wide range of clinical and non-clinical activities that are supported by the evidence provided by the Literature Searching service demonstrates its far-reaching impact on many areas within the Trust.
Saves time
100% of respondents said that the service saved them time in finding the information they needed. This is a key benefit of the service; where searches are conducted by a librarian the clinician or manager can spend the time saved on patient care or service delivery.
Improves information handling skills
The service also demonstrates a positive impact on developing the information skills of the healthcare professionals who utilised the service. 92.9% of respondents said the service had improved their awareness of evidence-based resources. 100% of respondents said the service has improved their ability to access resources. 71.4% said the service had improved their own searching skills.

Feedback: Informal feedback suggests that in most cases literature search requests occur as a direct result of a promotional drive or marketing activity, following Librarian attendance at a team meeting, a conversation with a member of library staff or another ‘outreach’ activity.
Open-ended responses from the evaluation questionnaire reveal the various ways in which the service impacts on the organisation:
“Very positive. Improved knowledge, evidence based care and supported decision making on the ward. Invaluable.”
“The service has contributed to knowledge and the development of clinical care.”
“Improving best practice and evidence based knowledge for me and share on my work area.”
Comments also highlight the role of the service in supporting business decision-making within the Trust:
“[The evidence provided was] used towards guideline development / local practice / CIP targets”
The service has also resulted in spin-off involvement in other service improvement projects:
“Victoria Kirk has been an amazing asset to our team this year and has helped us attain PDU accreditation. Her input with helping us review the latest evidence based practice has had a major input on the service and staff development.”
Search results were used to support educational activities:
“This service was able to simplify and quicken my search that enabled [me] to give an up-date and informative journal club presentation.”
“It has given a head start for me to work on my dissertation, so has increased my motivation.”
“Helped with literature search for the purpose of a presentation, therefore
saving time and improving my resources.”
“The results have been very useful as I am currently working on a systematic review for an MSc dissertation.”

Lessons learned: The literature searching service is a highly valued service that contributes to decision-making with the organisation. Awareness of the service needs to be raised within the organisation to encourage usage.
Where searches are conducted by a librarian rather than a clinician or manager there is:

• A positive impact on time
• An improvement in the quality of information retrieved
• The potential for significant cost savings

Sustainability;next steps:

• We will continue to evaluate the literature searching service.
• We will endeavour to promote the service more widely.
• We will continue to develop the skills of our library staff to deal with the wide range of searches that are received.
• We will share the results of the evaluation within the wider organisation, with a view to more formal reporting in the departmental quarterly report.

Project output: One of the key resources produced as a result of the evaluation project was a generic literature search evaluation questionnaire that can be adapted locally by other library services within the North West. The project also generated impact assessment data which can be used for a range of marketing and reporting activities.

Timeframe: Evaluation data reported here was from a five month period from 1st March 2010 to 31st July 2010.

Contact: Victoria Kirk
Job title: Clinical Librarian
Telephone: 0151 604 6223 (internal 8610)
E-mail: Victoria.kirk@nhs.net

Date case study completed: 1st September 2010

Tagged: Impact, Literature searching, North West