Evaluation of S12 Solutions

Overview

Apollo Health Innovations completed a comprehensive evaluation project for S12 Solutions, an organisation focused on digitising the mental health act assessment process within the NHS. The project encompassed the evaluation of implementation and outcomes for statutory forms, S12 Solutions' training and support services, evidence generation of product co-design, and patient and public involvement. This case study highlights Apollo Health Innovations' expertise in conducting evaluations, making it a suitable partner for evaluating HWC.

Project Objectives

1. Evaluate the implementation and outcomes of statutory forms in Sussex and Surrey.
2. Assess the effectiveness of  S12 Solutions' training and support services.
3. Generate evidence of product co-design and patient centricity.
4. Involve patients and the public in the evaluation process.

Methodology

In the S12 Solutions evaluation, Apollo Health Innovations employed a mixed-methods approach, which involved the use of both quantitative and qualitative data collection and analysis techniques. This comprehensive approach allowed Apollo to gain a deeper understanding of the program's effectiveness, user experiences, and areas for improvement. The mixed-methods approach for the S12 Solutions evaluation included the following components:

Pre- and Post-Implementation Surveys: Apollo designed and administered pre- and post-implementation surveys to various stakeholders, including Approved Mental Health Professionals (AMHPs), Section 12 doctors, claims processors, and nurses receiving H3 forms. These surveys collected quantitative data on key performance indicators and user satisfaction, allowing Apollo to measure the impact of S12 Solutions on the mental health act assessment process.

User Feedback Surveys: Apollo co-designed user feedback surveys for participants who had completed product training and those who had accessed S12 Solutions support services. These surveys provided valuable qualitative insights into user satisfaction, knowledge and comfort with the platform, awareness of support resources, and recommendations for future improvements.

Discovery Conversations: Apollo conducted discovery conversations with S12 Solutions staff, commissioners, and site implementation teams to gather qualitative information about the program's development, implementation, and outcomes. These conversations provided valuable context and insights that informed the evaluation process.

Workshops and Interviews with Patients and Carers: Apollo organized workshops and interviews with patients or carers who had experience with mental health act assessments (detained or otherwise). These sessions provided an opportunity for patients to discuss their perspectives on the product's processes, content, language, and delivery, as well as to share areas of success and recommendations for improvement.

The qualitative data collected through user feedback surveys, discovery conversations, workshops, and interviews were analysed using thematic analysis. This technique allowed Apollo to identify patterns and themes in the data, providing a deeper understanding of user experiences and the impact of S12 Solutions on the mental health act assessment process.

By combining both quantitative and qualitative data collection and analysis methods, Apollo Health Innovations was able to provide a thorough and well-rounded evaluation of S12 Solutions. This mixed-methods approach enabled Apollo to identify areas of success, challenges, and opportunities for improvement, ultimately contributing to the ongoing refinement and success of the S12 Solutions program.

Key Findings

1. The evaluation framework and data collection tools were successfully designed and deployed for pre and post-implementation assessments in Sussex and Surrey.
2. User feedback surveys for training and support services highlighted satisfaction with the quality of training and areas for improvement.
3. Evidence of co-design activities and outcomes were collected and documented, showcasing S12 Solutions' commitment to user-centred product development.
4. Patient involvement activities provided valuable insights into the platform's processes, content, language, and delivery, leading to recommendations for improving patient experience.

Outcomes and Recommendations

The project led to several recommendations for S12 Solutions, including improvements to training programs, increased promotion of self-referral routes, and the adoption of best practices for future co-design initiatives. Additionally, the written report on patient and public involvement activities demonstrated the platform's responsiveness to patient needs and its person-centred approach.

‍