Project report: Capturing innovation to accelerate improvement

Published: 14 September 2023 Page last updated: 14 September 2023

Our report to the Department of Science, Innovation and Technology (DSIT) on the Capturing innovation to accelerate improvement project.

Contents

Executive summary

People matter most. People and quality need to be at the heart of any innovation.

There is nothing new about trying new things. Innovation has long been at the heart of health and social care and must undoubtedly be part of how we meet the changing needs and growing pressures seen today.

In November last year, the Department for Business, Energy and Industrial Strategy (BEIS) awarded Care Quality Commission (CQC) a grant of £118,004 via the Regulators’ Pioneer Fund (RPF) to deliver a project called Capturing innovation to accelerate improvement. The RPF is a grant-based fund aimed at UK regulators and local authorities to help create a UK regulatory environment that encourages business innovation and investment. The current £12m round is being delivered by the Department of Science, Innovation and Technology (DSIT).

Over the course of this project, we set out to understand the experiences of providers and innovators who have considered developing or adopting new ways of working. Some demonstrated how they had embedded new tools or systems, others reflected on what that process might look like for them. Understanding this helps us fulfil our purpose: to make sure health and social care services provide people with safe, effective, compassionate, high-quality care and encourage care services to improve. It also helps us deliver on our strategic commitments to encourage and champion innovation that benefits people.

Working with people from across the health and social care sectors, we explored what barriers to innovation might exist, what we could do to mitigate these and what opportunities existed to improve their experience. We also undertook focused test and learn tasks to ask registered providers and system partners their thoughts on specific issues. These included their use of data in the innovation process, the legacy of COVID innovations and looking for ways to encourage more conversations around innovation within CQC.

There was a notable appetite for clarifying the regulatory landscape so that health and social care stakeholders could engage with the right organisation at the right stage of their innovation journey. Innovators need more information and clarity on our regulatory approach to innovative practice, including safety and risk management implications. Innovators are unsure where to find information about innovation requirements and support. This extends to organisations building a culture where each part of the system, including CQC, has a shared understanding and approach to innovation.

With providers working across the country with different populations, different levels of support and in different local circumstances, we heard that trying something new can feel challenging. Encouragingly, people working in services and innovators recognised the opportunity created by our new approach, including the new single assessment framework and the quality statement that looks at learning, innovation and improvement. In particular there was significant discussion about the value of looking at innovation on a system level created by our new assessments of integrated care systems. There are unique opportunities now to improve impact on innovation through our new role in systems and the single assessment framework.

It became clear that the ability to innovate does not have a one-size-fits-all solution but depends upon a thriving improvement culture. Effective innovation relies on an improvement culture; a culture where the people feel supported to try new things and have the space to reflect on progress and setbacks.

People told us that successful innovation is not built around technology, large budgets, or distinct teams. People working in services need clarity, an enabling environment, and a continued focus on outcomes. Effective innovation support requires us to develop our approach to using our impact mechanisms, particularly the way we work with stakeholders and the way we use softer influencing skills in relationships with providers and systems.

Through the generous input of people engaging with this work, this report presents a series of important reflections for both CQC and the broader health and care sector. Where the project allowed, we have explored what we can do as an innovation-enabling regulator and have already initiated change. Where more consideration and effort are needed, we have outlined next steps and considerations to help support more innovation and better outcomes for people.

Background

The health and care sector has long been a source of, and in many ways a result of, innovation, from the development of treatments to how they are delivered and the shape of services. As the regulator of health and adult social care in England, we are committed to supporting a health and care system that supports effective innovation that leads to more people receiving high-quality care.

There are many reasons to promote and support innovation within health and care. We know that the system is struggling to meet demand, and innovation can create new ideas to address these challenges. New ideas, and new technologies provide opportunities for higher quality care, for tackling inequalities and for improving outcomes. We know there is strong government support for innovation in health and care, with recognition of the potential benefits to people receiving care, those providing it and to the economy.

Our strategic commitments and work so far

We have committed in our strategy to accelerating improvement in health and care. Our 2021 strategy says, “Innovative practice and technological change present an opportunity for rapid improvement in health and care. We have a role in creating a culture where innovation and research can flourish.” This takes a bolder approach to both innovation and improvement. It commits to encouraging and championing innovation, and to working in partnership with stakeholders to develop a coordinated, effective, and proportionate approach to regulating new innovations and technology.

Since publishing our 2021 strategy, we have been developing our approach to innovation. This includes the launch of the AI and digital regulations service, a collaboration with the National Institute for Health and Care Excellence (NICE), the Medicines and Healthcare Products Regulatory Agency (MHRA) and the Health Research Authority (HRA) to improve the regulatory pathway for AI and data-driven technology. We will soon roll out our new Single Assessment Framework, which will guide our regulatory assessments of both health and social care services. This includes a quality statement showing our expectations around innovation, improvement and learning cultures. We have published examples of innovative practice, including People First which shared best practice and examples of innovation in urgent and emergency care. We have significantly developed our understanding of the nature of innovation and the principles behind high-quality innovation from previous projects supported by the Regulators’ Pioneer Fund, including Enabling innovation and adoption in health and social care: Developing a shared view. We undertook sandbox trials for Digital triage in health services and Machine learning in diagnostic and screening services and published an update on Regulatory recognition and sharing of innovative practice by NHS GP providers to reduce health inequalities.

Aims of this project

There is more to do to ensure a supportive and effective regulatory approach to innovation. Building on this previous learning, this project aimed to:

  1. Explore and articulate the role of an innovation-enabling regulator
  2. Pilot innovation supporting interactions between regulator and health and care system
  3. Develop and pilot products to share learning and resources on innovating well, with greater impact using a range of regulatory impact mechanisms
  4. Build learning from previous projects into tangible products and innovation in our regulation

Key terms in this report

Regulatory impact mechanisms. This refers to research on the mechanisms by which CQC can affect provider performance (Kings Fund, 2018). This research found 8 different ways to create change. A summary of the impact mechanisms is given in Appendix 2.

Innovation. By innovation, we mean both invention (creating new ideas, products, services, or model of care) and adoption (implementing what has worked elsewhere). Our report on Enabling innovation and adoption in health and social care: Developing a shared view has more information.

Improvement culture. Culture is known to be important in facilitating improvement. A recent rapid literature review undertaken for us by independent research company SQW about improvement cultures in health and care settings defines characteristics of an improvement culture and the conditions in which improvement thrives. This includes psychological safety, high staff empowerment and engagement, commitment to being caring and person centred, being a learning organisation and compassionate, diverse and inclusive leadership.

How we carried out the project

The project funding allowed us to recruit 2 colleagues working full time on this project (a Strategy Manager and a Communications and Engagement Manager). We delivered the project through 4 workstreams:

Workstream 1: Stakeholders in innovation

We worked with internal and external partners to identify stakeholders and potential collaborators for this project. This included providers of health and care, systems, Academic Health Science Networks (AHSNs), people using health and care services and organisations representing innovators.

Workstream 2: Innovation roundtables

We invited stakeholders to a series of 4 innovation roundtables between March and June 2023. We met with 10-15 people at each event (49 attendees across 4 events, made up of 42 different people), including representatives from across the health and care sector and people with lived experience. They acted as subject matter experts, contributing to our conversations, helping us generate and develop ideas for pilot activity and providing crucial feedback on ideas, draft products and activities.

Workstream 3: Innovation Case studies

Building on positive feedback from previous innovation case studies, we sought examples of current innovation to develop a refreshed set of case studies on our website. We used these as a basis for our roundtable conversations. They also helped us to decide how best to use the examples alongside our impact mechanisms, to support innovation that leads to higher quality care.

We published a national call-out for case studies through our website, provider bulletins and in speeches delivered to relevant audiences by CQC leaders. Our internal working group triaged case studies based on previously published innovation principles and appropriateness for this project. A total of 7 case studies were included either in roundtable discussions or our pilot activity. 5 are being featured in a new case study publication for our website.

Workstream 4: Develop and pilot a suite of innovation learning products and activities

We used our learning from the early stages of the project, and our previous evidence on our impact mechanisms, to develop a suite of 9 test and learn pilot projects. Each aimed to build on suggestions from our roundtables and evidence collected within this project, to test approaches to using our impact mechanisms to support innovative practice.

Our learning

Themes from roundtables

Roundtables provided rich opportunities to discuss and explore key issues.

Roundtable 1 (March 2023) explored the characteristics of an innovation-enabling regulator and generated ideas for pilot activity. Key themes in the discussions were:

  • An innovation-enabling regulator must focus on outcomes for people using services, rather than the specific products used to deliver those outcomes.
  • Innovation and improvement cultures in providers and systems are important in creating an environment where effective innovation can flourish. Therefore, effective regulation should support and identify those cultures. This could create change and support innovation through our anticipatory impact mechanism (if providers change their behaviour in anticipation of regulatory activity).
  • Participants strongly support and encourage the sharing of examples and information (using our informational impact mechanism).
  • A regulator that supports innovation needs to embody innovation and improvement culture themselves.
  • There are views, which may or may not be accurate, about our role in innovation that can act as a barrier to innovation.
  • This feedback influenced our findings on the characteristics of innovation supporting regulation and was used to shape pilot activity (pilots A, B, C, G, J).

Roundtable 2 (April 2023) considered 3 case studies in more detail. Building on the feedback from the first roundtable about perceptions of CQC acting as a barrier to innovation, participants reviewed drafts of an innovation ‘mythbuster’ and co-produced content. Participants also completed a journey mapping task, focused on the potential touchpoints between innovators and regulators to help us focus our innovation activity. This activity fed directly into a pilot journey mapping product (pilots B, H). The journey mapping task highlighted 3 challenges – addressing inequalities, getting ‘back on track’ after the pandemic and ‘longstanding’ innovation barriers (including resourcing, funding, and ageing populations). These challenges formed the basis of the agenda for coproduction at Roundtable 3.

Roundtable 3 (May 2023) focused on the regulators’ role, or potential role, in addressing barriers identified in the journey mapping. This raised questions about the impact of the COVID-19 pandemic and what we can learn, and the importance of collaboration with partners who have expertise in innovation and evidence base. This led directly to pilot projects exploring these issues (pilots D & E).

Roundtable 4 (July 2023) gave an opportunity for participants to comment on emerging findings and potential next steps for the project and our innovation approach. Feedback on findings was positive, though attendees asked for more nuance about their meaning for CQC in practice. All participants reviewed the pilot activity to reflect on how they did or did not address issues raised at earlier roundtables to inform our overall learning. This roundtable contributed to all pilot activity.

Innovation suite pilots

We used the feedback, discussions, and subject matter expert input from our roundtables to develop a suite of 9 pilot activities. These short-term pilots were designed to test how we could use our impact mechanisms to support innovation which would lead to high-quality care. The pilots were intended as rapid test and learn activity to provide feedback and learning to inform our future approach.

Pilot A: Sharing innovation case studies (informational and lateral impact mechanisms)

Aim and rationale

We had positive feedback on previous case study sharing. This pilot provided an opportunity to refresh this resource, while exploring the best way to share it to support innovation that drives improvement. Roundtable participants spoke about needing examples to be accessible and applicable across sectors, while avoiding endorsement of products.

Pilot activity

Case studies submitted in response to a national call out and identified internally were reviewed for consistency with previously published principles of effective innovation. Eight were identified for publication. Potential mock-up formats were shared at roundtables with feedback informing the final format.

Outcome

Refreshed case studies in agreed format are being prepared for publication, with ongoing feedback mechanisms embedded. Learning and feedback about format options has been shared with content designers to inform future case study publications.

Learning

Case studies shared using our informational impact mechanism can be helpful and inspirational but are not sufficient innovation support on their own. Sharing case studies also uses our lateral impact mechanism by promoting shared learning among providers and systems.

Pilot B: Our role at key stages in the innovation journey (anticipatory and relational impact mechanisms)

Aim and rationale

We received consistent feedback that innovators wanted greater clarity and understanding of our role and requirements in the innovation journey.

Pilot activity

Roundtable 2 participants completed a journey mapping task to begin this work. We developed this along with the ‘mythbuster’ work to clarify the stages of developing an innovation. We also identified relevant information and guidance from CQC or other agencies which would be needed at each stage. This was tested with roundtable participants and via Citizenlab, our online participation platform, and improved based on their feedback.

Outcome

Constructive and supportive feedback has been given on drafts and the journey map will be published on our website. Feedback suggests this could improve transparency around our expectations, and support options for innovators.

Learning

Journey mapping uses our informational impact mechanism to provide transparency around our expectations. It also utilises our relational impact mechanism by being a tool that our inspectors can share with providers and help improve consistency of messaging. In the future, co-producing this with a wider range of stakeholders would help ensure we focus our attention on the most significant or challenging touchpoints between providers and CQC.

Pilot C: Innovation-supporting conversations with corporate providers (relational impact mechanism)

Aim and rationale

We heard in previous projects, and in our roundtables, that those involved in innovation receive inconsistent messaging about innovation and regulation. We aimed to understand conversations currently taking place, and to pilot supporting our colleagues to have innovation-supporting conversations.

Pilot activity

Our Corporate Provider Team (CPT) work with the largest adult social care providers. Corporate relationship managers meet regularly with national head offices, preparing for these meetings with updates from relevant local inspectors. They agreed to pilot a set of questions about innovation to support them in capturing feedback from inspectors and corporate providers for a 3 week period in July 2023. These questions asked providers for examples of innovation in their service, what barriers they face to innovation and how CQC may help overcome these. The questions we asked are given in Appendix 3.

Outcome

Seven corporate provider conversations took place, and we received feedback from the CPT, who explained why the additional questions were being asked. Corporate providers welcomed conversations about innovation and the CPT reported that this approach prompted improved discussion about innovation. In several of the corporate provider conversations during the pilot, providers explicitly named commissioning practices as a barrier to innovation and some described variable or insufficient support from local authorities and integrated care systems. Providers suggested we could support innovation by encouraging integrated care systems and local authorities to engage more with providers. They also reminded us to consider the positive aspects of innovative practice and not only the risks.

Learning

We can use our relational impact mechanism to gain insight into innovation taking place. However, it was difficult to gauge the likely impact over a longer period and there were challenges in feeding these innovation insights into CQC’s wider learning. While the conversations prompted interesting and relevant discussions, they significantly extended the meetings, and the CPT are not best placed to use the information gathered.

There are opportunities with our new powers to assess local authorities and integrated care systems, and with the innovation quality statement in our new single assessment framework, to encourage innovation support using our systemic and anticipatory impact mechanisms.

Pilot D: Innovation since the Covid-19 pandemic

Aim and rationale

Roundtable attendees told us that the COVID-19 pandemic enabled innovation to progress, but that this has not been sustained. This feedback came particularly from people representing the adult social care sector. We sought to explore this hypothesis to understand how learning from the pandemic could support ongoing innovative practice using our impact mechanisms.

Pilot activity

Over 300 case studies were shared with us and published during 2020. We selected a random sample, with some adjustment for the greater representation of adult social care examples, and sent those we could contact a survey. In the survey, we asked whether they were still using the innovation they had described, why that was, and what they could share about working as part of the local system. We contacted 42 providers to ask about 46 case studies and received 5 responses. The questions we asked are given in Appendix 4

Outcome

Among the small sample of respondents, most (4 of the 5) were still using the innovation. One respondent told us there had been a return to pre-COVID arrangements that impacted negatively on the services people received. While few trends could be seen in individual questions, across the survey there were reflections on the benefit of being able to respond quickly to changing pressures and the positive response from colleagues. Those who responded to questions about the impact on working with system partners (3 of the 5) told us about improving outcomes through collaboration, sharing best practice and influencing ICS strategy.

Learning

While the response rate was low, this pilot set out to gain qualitative insight into the experience of individual innovators rather than statistical analysis. The conversations initiating this pilot and the feedback from those contacted reminds us of the importance of developing and maintaining improvement cultures that allow innovation to flourish. The feedback received also highlights the value of continued conversations, review and evaluation even after a project is delivered.

Pilot E: Collaboration with others: the AHSN network (stakeholder impact mechanism)

Aim and rationale

Feedback throughout the project consistently called for us to work with system partners to reduce risks of duplication or misalignment, and to ensure partners’ capabilities and activities complement one another. Our work on identifying innovation stakeholders (workstream 1) identified opportunities for collaboration which are not currently well-used. For example, AHSNs have a remit to ‘spread innovation at pace and scale’ though we do not yet have consistent routes for collaboration on innovation-related issues with them.

Pilot activity

Yorkshire and Humber AHSN offered to explore this with us, opening conversations about how CQC and the AHSN’s priorities might align and what the benefits and challenges might be. Initial constructive conversations have taken place and will continue beyond the duration of this project.

Outcome and learning

It is too early to confirm the outcome of these conversations. Relationships have been strengthened and potential collaborations are being explored.

Pilot F: Using the right data to inform our work (informational impact mechanism)

Aim and rationale

We were given leads on existing data which we did not currently access that could contribute to our assessments on innovative provider culture.

Pilot activity

We initiated discussions with relevant partners. We also considered and discussed ideas around practicalities, benefits and challenges with roundtable participants.

Outcome and learning

It was not viable to take this forward in the remaining timescale due to the complexity of discussion, but there is willingness from partners and discussions will continue. Learning and content will be shared with our data and insight teams.

Pilot G: Highlighting positive stories (informational and lateral impact mechanisms)

Aim and rationale

We know that we must support our colleagues’ awareness of innovation and support our own improvement culture to effectively meet our commitments on improvement and innovation. Current sharing of positive stories internally, and opportunities to share examples externally, is not systematic and so opportunities may be missed.

Pilot activity

Discussions took place with regional communication and engagement colleagues internally to explore options for identifying positive innovation examples which could be shared internally and externally.

Outcome

A system to ‘tag’ reports that are reviewed by the regional communications and engagement team and record where they include an example of innovative practice has been agreed. This will facilitate a more systematic review of information in this particular process, that could allow routine sharing.

Learning

This change has now been implemented but it is too soon to assess impact. This change has the potential both to support our improvement and innovation culture and knowledge, and to support our use of our informational impact mechanism to share learning.

Pilot H: Are innovators using existing data sources to inform innovation? Is it embedded into the innovation process? (Informational and anticipatory impact mechanism)

Aim and rationale

We aimed to identify how innovators are using existing data sources and measurement to inform innovation, to help identify any gaps in the journey mapping work and to provide greater clarity and signposting support. Measurement is crucial to improvement methodology and therefore is likely to support high-quality innovation. Roundtable participants had spoken about the importance of data in understanding health inequalities that innovation might help address.

Pilot activity

A survey was shared via our provider bulletins to explore how innovators use measurement and approach data.

Example questions are included below with full questions given in Appendix 5:

  • What are the specific data sets that you use?
  • Have you used any particular external guidance to inform, drive or shape your innovation?
  • Do you consider health inequalities when developing new ways of working or innovations? If so, how?
Outcome

We had 15 respondents. Of these, almost all said that they embedded data in their approach to innovation (14 out of 15), with most reviewing data in the earlier stages of their journey. Those using data to inform their innovation gave examples including local demographic data, datasets from local authorities, internal data sources (such as risks and incidents), data on people’s views and feedback, and population health management data. Seventy-five per cent of respondents (10 out of 15) said that they consider health inequalities when developing new ways of working or innovations, using data to provide baseline information and to assess what would be useful as outcome measures.

“Last year we started a new patient service group in an area with high deprivation, we used local data to inform this.”

However, understanding and priority given to health inequalities was mixed, with one respondent stating that “we have never experienced any health inequalities during the last 26 years of providing care.” Respondents told us that data was helpful across the entire innovation journey, including development, testing and adoption.

“Data should be at the heart of an innovation project, to ensure that the innovation is appropriate for the local area, can have measurable impact and is evaluated effectively.”

Learning

There is no consensus on what data or information would support the innovation journey. This small sample suggested a significant minority (20%) do not use data to support innovative practice. We could potentially do more through the informational impact mechanism and through our approach to implementing the new Single Assessment Framework quality statement on innovation to promote evidence-based improvement and innovation approaches (anticipatory impact mechanism). While some innovators see the potential links between effective use of data and measurement to understand health inequalities and take innovative approaches to addressing them (or to ensure innovation does not reinforce inequality) this understanding is not universal.

Pilot I: Spotlight on positive external/provider messaging (informational and lateral impact mechanisms)

Aim and rationale

Participants told us that we should share more learning on innovation. We were given many suggestions on ways to do this that could increase learning and resources on innovating well with greater impact.

Pilot activity

We explored how communication channels could be used to better support innovation and whether a stand-alone innovation bulletin may be suitable. We mapped the planned communication activity for the month of July, hoping to run some trials to make a recommendation for future working. We identified a number of barriers, including challenges in ensuring our own data is stored and organised in a way that allows us to extract examples of innovation.

Outcome and learning

The strong interest expressed indicates there is appetite for further work through the informational and lateral impact mechanisms to share and encourage greater innovation. We updated our data approach and introduced a system to ‘tag’ CQC reports that are reviewed by the regional communications and engagement team. The team uses this to record examples of innovative practice.

Less helpful regulatory behaviours

In addition to telling us the importance of regulatory action to support innovation, we heard consistent messages about what we should not do. Specifically:

  • There was less support for using our anticipatory, organisational, and directive impact mechanisms to support innovation:
    • Directive (telling providers what they need to do about innovation, or enforcing that they do it): This is unlikely to support or maintain improvement cultures that create the conditions for effective innovation. This would also be inappropriate and outside of our scope.
    • Organisational: Different organisations will have different structures to support innovation, and we does not recommend management structures, nor specific innovation products.
    • Anticipatory: There was concern this might inadvertently act against high-quality innovation. We heard that some providers might pause work on innovative practice in anticipation of regulatory activity (such as an inspection). It is important that all parties understand innovation as part of an improvement culture.
  • Detailed work to understand or support specific innovations (for example, specific technologies) is unlikely to be helpful.
  • We must avoid duplication of others’ work or repetition. Effective alignment and engagement with other system partners (using our stakeholder and systemic impact mechanisms) is important.
  • While there was strong support for guidance alongside the new single assessment framework, there was little interest expressed in us developing any new tools or resources to aid in the innovation journey. People told us there are enough tools already available, but they needed more signposting and guidance on when and how to use them.

Challenges and limitations

We are confident that this project has effectively supported us to become a more innovation-supporting regulator. There have been challenges and we have identified some limitations to our findings.

  • Delivery against timescale: There was a very short period (4 weeks, including Christmas) between finding out we had been successful in our bid for the project and commencing delivery. This made timely recruitment challenging, posing challenges for delivery. This was managed by recruiting internally and providing additional input from other colleagues in the early stages. In line with a test and learn approach, we shared draft ideas and products with participants at early draft stage and held our roundtables as co-production events with development work taking place during the sessions.
  • Face-to-face events: With many events now held virtually, we were unsure how face-to-face events would work. While we sometimes had attendees drop out on the day, people who did attend said that they valued the face-to-face conversations.
  • Scale of pilots: This project was designed to deliver small-scale pilots to support rapid learning about our approach. This has been successful, but we must remain mindful that small-scale activity can give skewed evidence and some surveys sent to all providers had low response rates. As we use this learning to develop our ongoing innovation approach, we will need to continue embedding learning and evaluation to gather stronger evidence to assure our approach.
  • Evidencing impact: At the end of the project, the team is confident that the learning has already started to have an impact on our approach to innovation-supporting regulation. This has been achieved by embedding evaluation throughout so that design changes could be made to the project as learning emerged, and by involving colleagues at every stage. However, this will not be fully evidenced until that approach is more developed.
  • Timing of project: This project took place during a period of organisational transformation at CQC and shortly before the rollout of a new Single Assessment Framework. In many ways this is a helpful moment to undertake the work, as learning will help inform implementation of the Single Assessment Framework and our future approaches. However, it has also presented some challenges in ensuring colleagues have capacity to engage and input. In future, it could be beneficial for the conditions of such fixed-term projects to include more discretionary agility around timelines; something that innovators have told us they value of during this work. We will need to ensure ongoing sharing of the learning beyond the end of the project both internally and externally to maximise the impact of the findings.

Conclusions

There is more we can do to deliver innovation-supporting regulation that leads to more people receiving high-quality care, and our stakeholders strongly support us actively developing our innovation approach. We must do this while continuing to focus on the impact of innovation on the quality of care people receive. We will work with others to ensure we support rather than duplicate effort. Key findings from this project are:

1. Effective innovation relies on an improvement culture

Improvement cultures in providers, systems and regulators recognise the journey to improvement, including the potential for false starts and learning when things go wrong. Innovators need support and recognition throughout their journey. (Evidence from pilots A, B, C, D, G, J)

2. Innovators need more information and clarity on CQC’s regulatory approach to innovative practice, including safety and risk management implications

People told us that improving transparency will help providers feel safer to share and explore best practice. We can build on the positive collaboration in projects such as the AI and Digital Regulations Service. We can also ensure that language on innovation always focuses on impact and outcomes for people. People told us that our messaging on innovation can be inconsistent at different relationship levels. (Evidence from pilots A and B, and all roundtables)

3. Effective innovation support requires us to develop our approach to using our impact mechanisms, particularly stakeholder and relational impacts

Regulators impact the system in different ways. We often use case study examples, which can be helpful but are not sufficient on their own. Participants in our roundtables encouraged the use of stakeholder and relational impact mechanisms, exercising softer influence through relationships, and influencing to support innovation over direct regulatory action. Using different mechanisms will allow us to better tailor our approach to national, system and provider level conversations and provide a better picture of an area. Developing our approach to these impact mechanisms will support and inform our implementation of the single assessment framework, which has been developed to promote improvement cultures that support effective innovation.

4. People matter most

People and quality need to be at the heart of any innovation. Stakeholders told us that we should not be prescriptive in describing the best ways to innovate but use our strategic insight to pull out gaps. They expect an innovation-enabling regulator to:

  • focus on the impact on outcomes for people using services
  • actively look for innovation cultures
  • signpost and work with other relevant bodies
  • actively share stories and examples
  • support providers to become more innovative
  • embody these characteristics itself as an organisation with an improvement culture

5. Innovators are unsure where to find information about innovation requirements and support

Participants asked us to work closer with system partners so that providers and innovators can be more aware of risks, opportunities, and support. Speaking to the right organisation at the right time will help avoid duplication and burden. Better collaboration will strengthen clarity and efficiency. It will also add value and efficacy through shared knowledge and experience. Participants in this project explained that providers and systems rarely seek information on how to innovate and instead seek information about solving a particular problem. We should consider this in planning how to provide innovation support. (Evidence from pilots B, E, F)

6. There are unique opportunities now to improve our impact on innovation through our new powers and the single assessment framework

Our new assessment powers for local authorities and integrated care systems (ICSs) are viewed positively, especially the fact that the same quality statement relating to innovation will apply across all sectors. Anecdotal feedback indicated providers receive significantly varying levels of support. It is widely acknowledged that maturity across the system varied, but we should hold ICSs and LAs to account around how they are supporting innovation. The single assessment framework is intended to regulate in a way that promotes improvement cultures and to be more flexible and responsive to changing or innovative care provision, It should also help us collect evidence and knowledge in a way that makes it easier to share learning on good practice. (Evidence from pilots A, B, C)

Next steps

As a result of our early pilot activity, actions building on our learning in the short-term are already underway.

  1. We will publish our journey mapping product for pilot B and share this with our stakeholders for further development.
  2. A system to ‘tag’ CQC reports that are reviewed by the regional communications and engagement team and record where they include an example of innovative practice has been agreed.
  3. The key findings of the project and the proposed characteristics of an innovation-enabling regulator are informing development of our broader innovation approach.
  4. Case studies gathered in this project will be published, reflecting the learning and feedback about they can be most helpful, including:
    • Be innovation neutral, and highlight the problem being addressed.
    • Focus upon the impact upon people using services.
    • Take a proportionate approach to risk and consider how risks have already been considered and mitigated.
    • Provide the context, but also highlight why we felt it useful to publish and which service types it may be useful in.

Further considerations and recommendations:

  1. Our learning from this project should inform the way we adopt the single assessment framework, including the innovation quality statement. This includes ensuring we focus on people and outcomes, and support improvement cultures.
  2. We should be proactive in considering our full range of impact mechanisms available, particularly those highlighted in the roundtable feedback. This learning will feed into our broader approach to accelerating improvement; a theme of our strategy.
  3. We will be more intentional in considering our full range of impact mechanisms available, particularly those highlighted in the roundtable feedback.
  4. Building on the relationships established and strengthened in this work, we should continue working with stakeholders to understand key issues, develop a common language and support effective innovation.
  5. As we begin implementing new approaches to regulation and a new role in integrated care system and local authority assessment, we should build on this project’s engagement and ensure messaging about innovation is based on consistent principles at each level of the system.

Evaluation

While the full impact of this project cannot be evaluated until there has been sufficient time for next steps to be taken and change to be tracked, we have undertaken evaluation activity throughout. Much of this was formative evaluation, with a continual cycle of learning feeding into the project and leading to iterative improvements in project delivery. This section summarises our approach and learning so far.

Evaluation methodology

Our evaluation methodology had 3 parts:

  • Roundtable feedback
  • Feedback and testing built into pilot activity
  • Review delivery against plan and logic model

Evaluation findings

Roundtable feedback

Across four roundtables we had 49 external attendees (this includes people who attended multiple roundtables, so in total 42 different people). Attendees were asked to complete an evaluation form at each roundtable. Over the four roundtables, 39 evaluation forms were completed, a response rate of 80%.

The same 6 questions were asked in each roundtable feedback survey:

  1. Which sector is most relevant to your service?
  2. I felt that the logistics of the roundtable worked well (for example, timing, chairing, platform used etc)
  3. I felt that the information shared at the roundtable was clear and accessible
  4. I felt that the roundtable was facilitated effectively to ensure that all voices were heard
  5. I have a clear understanding of the actions/next steps that were agreed at the roundtable
  6. Do you have any other comments?

A final question was bespoke to each roundtable to reflect the aims and discussions of that roundtable. Evaluation forms were anonymous. Questions 2-5, and the final changeable question, had a five-point Likert scale response (Strongly agree – strongly disagree).

Results demonstrated that the roundtables were well received. 100% of respondents at all the four roundtables agreed or strongly agreed that:

  • The logistics of the roundtable worked well
  • Information shared at the roundtable was clear and accessible
  • The roundtable was facilitated effectively to ensure that all voices were heard

All respondents agreed or strongly agreed to the bespoke question at each roundtable, which were designed to reflect the roundtable aim (aims highlighted in bold):

  • The roundtable was effective in capturing views on what a good innovation-enabling regulator is (Roundtable 1)
  • The roundtable was effective in identifying and capturing view on differences needed in the innovation journey, and actions we can consider (Roundtable 2)
  • The roundtable was effective in setting out the findings and test and learn activity underway (Roundtable 3)
  • I agree that the recommendations and test and learn activities are the right priorities for us to take forward (Roundtable 4)

At the first roundtable, 6 respondents (55%) said they agreed or strongly agreed, and 4 respondents (36%) said they neither agreed or disagreed that they had a clear understanding of the actions/next steps. One respondent did not answer this question. In response to this feedback, we strengthened clarity on next steps for future roundtables. This change was effective, and in roundtable 2, 9 respondents (90%) agreed or strongly agreed, and in roundtables 3 and 4, all respondents agreed or strongly agreed that they were clear on actions and next steps.

Qualitative feedback across the roundtables reflected the positive scores and offered a series of constructive suggestions for future meetings. These were incorporated across the project. Example feedback included:

"I think this ran really well. If anything we could have had more time" (Roundtable 1)

"I know what the proposed next steps are for the project but less clear on what impact the project might have on CQC in the long term" (Roundtable 1)
Following this feedback, we made information on next steps clearer for future roundtables.

"Restate my thoughts that there could be greater diversity in the room and focus on skill/experience mix in the room. Facilitation was great. Could even be enhanced by external input in setting up the day.” (Roundtable 2)
Following this feedback we reached out to different stakeholders to encourage more diversity in attendees and worked with an external facilitator for Roundtable 3.

"These sessions really would benefit from end user representation… (one per table would be enough)" (Roundtable 3)
We had public representation at each roundtable, but not enough to ensure representation across all tables. This could be improved in future work.

"Really great sessions, you are all so approachable and willing to listen and be challenged" (Roundtable 4)

Feedback and testing built into pilot activity

We set out to ensure our pilot activity supported our developing understanding of how we use our impact mechanisms. We considered all mechanisms, with a focus on those that roundtable participants identified as having greatest potential impact (informational, relational, stakeholder and lateral). Most ideas were generated for the informational impact mechanism, for example, CQC sharing information for others to use, although there was a consensus that the relational and stakeholder mechanisms might have more power. This perhaps reflects the relative ease of generating informational activity (such as publishing information) and of tracking a measure of impact (like the number of downloads) in comparison to investing in softer impacts such as informal relationships. While the lateral impact mechanism came third in the roundtable exercise for long-term potential to support innovation, it did not feature heavily in the test and learn activity as the ideas generated often did not support this.

Our pilots proved to be a valid and helpful way of generating feedback, providing opportunities for roundtable participants and people in our wider network to share their views on early stage products and ideas. We had some engagement through provider bulletins though response rates were low in comparison to the many thousands of individuals on these mailing lists, perhaps reflecting provider pressures. We also heard that those working to provide care may respond better to communications framed around the specific problems they are innovating to solve, rather than innovation more generally. However, this is a challenging hypothesis to test within a short project.

Review delivery against plan and logic model

We used a logic model to ensure that we had clearly articulated the aims of the project, and the activities and deliverables that would lead to the outcomes and impacts we planned for.

Stated aims
1. Explore and articulate the role of an innovation-enabling regulator

This was a focus of Roundtable 1 and a theme throughout the project discussions. All Roundtable 1 attendees agreed or strongly agreed this was effective in capturing views on innovation-enabling regulation. This work, and subsequent refinement and engagement facilitated drafting of characteristics of innovation-enabling regulation featured in the key findings of the report.

2. Pilot innovation-supporting interactions between regulator and health and care system, and
3. Develop and pilot products to share learning and resources on innovating well with greater impact

Nine pilots have been undertaken offering rapid test and learn activity to inform our future innovation approach. These have piloted different interactions, products and activities using a range of our impact mechanisms.

4. Build learning from previous projects into tangible products and innovation in our regulation

In this project we have been able to take learning from other projects and messages we’d heard in other forums about our impact mechanisms, the relationship between innovation and regulation and how to deliver an impactful project in a short time frame. We were able to use these to strengthen the work.

Planned outcomes
Suite of products to promote better innovation and accelerate improvement in health and care

The pilots have generated a suite of products which have been shared with participants for feedback and improved. The latest versions are being developed for wider sharing, including journey mapping and case studies.

Products and activities are targeted at effective impact mechanisms, reducing wasted effort and maximising impact

We now have a better understanding of our stakeholders’ appetite and views on the impact mechanisms we should use to positively influence innovation in the short and long-term. Our roundtable activities told us there was most enthusiasm for our stakeholder impact mechanism (work with and through stakeholder groups or organisations to influence provider performance). We also heard that we need to work with other regulators to ensure alignment, and that we should approach this in a way that reflects the need for a differentiated approach at national and local level, based on consistent principles. There was also strong support for the value of the relational impact mechanism (creating change using softer skills, such as relationships between our colleagues and providers).

The stakeholder and relational impact mechanisms are more difficult to evidence direct impact from. We will therefore need to mature our approach to stakeholder and relational work on innovation, and our evaluation of that approach, to fully realise the benefits.

Collaboration between stakeholders on innovation is improved, fostering a more coordinated approach to innovation across regulatory system

The roundtable activity itself has strengthened our networks and skills in collaborating on innovation. This is reflected in positive roundtable feedback. This collaboration has also led to several follow up conversations and activities to take innovation support forward, based on opportunities identified either within roundtable discussions or in more informal interactions facilitated by the events. This includes conversations with AHSNs, for example.

CQC has an improved definition of its role in championing innovation and builds this into future work

We have identified characteristics of an innovation-supporting regulator, which are already being used to influence our broader approach and the roll out of our new regulatory approach and implementation of new role at system level.

Evaluation conclusions

This project has successfully delivered planned outputs to meet its aims. The overall impact is difficult to determine at this stage. It will become clearer over the medium-term, supporting stronger development of our innovation approach and therefore improved innovation-enabling regulation. We have strengthened our understanding of how we can encourage and enable innovation and the impact mechanisms we might use. Some of the strongest support and evidence is for using stakeholder and relational impact mechanisms, although these are among the most challenging to adapt to a rapid test-and-learn approach and to evidence direct impact from.

Appendices

Appendix 1: Stakeholders involved

  • Ashley Care
  • Australian National Disability Insurance Scheme
  • Bermuda Health Ministry
  • BUPA
  • Care England
  • Care Association Alliance, Ascot Residential Homes
  • Casson Consulting
  • Clinical Entrepreneurship Programme and Innovation Sites, Mid and South Essex NHS Foundation Trust
  • College of Paramedics
  • Department for Science, Innovation and Technology
  • Digital Social care
  • Dutch Health and Youth Care Inspectorate
  • Independent Ambulance Association
  • Felgrains
  • General Medical Council
  • Lemonaid Health
  • London School of Economics - Care Policy and Evaluation Centre (formerly PSSRU)
  • Mr Bren McInerney
  • MD Healthcare
  • NHS Confederation
  • NHS England
  • NHS Providers
  • North Tyneside ICS
  • Nursing and Midwifery Council
  • Orchard Care Homes
  • Oxehealth
  • Quiddity
  • Ramsay Healthcare UK
  • SDS Consultancy
  • Simply Health
  • Social Care Institute for Excellence
  • Think Local Act Personal
  • Wessex AHSN
  • Yorkshire and Humber AHSN

Appendix 2: Impact mechanisms

Research by University of Manchester and the King’s Fund identified 8 ways in which regulators could impact the organisations they regulate. We have used these within this project to consider how we make best use of these impact mechanisms to create a system and regulatory approach where innovation that improves care can flourish. The original research can be accessed through the King’s Fund and the University of Manchester websites.

The 8 impact mechanisms are:

  • Anticipatory: providers take actions in response to regulatory requirements before any interaction (for example, a provider prepares for an inspection)
  • Directive: CQC advises or instructs providers to take certain actions, often after a regulatory interaction (for example, CQC takes enforcement action)
  • Relational: CQC exercises softer influence through credibility and expertise of regulatory staff and through the relationships we develop (for example, CQC staff and providers discuss quality issues)
  • Organisational: regulatory approach changes organisational power dynamics and behaviours (for example, having been rated as requires improvement by CQC, a provider decides makes changes to its management)
  • Informational: CQC collates or produces and publishes information which others can then use (for example, a local Healthwatch group decides to focus a visit to a provider on issues identified in a CQC inspection report)
  • Stakeholder: CQC seeks to work with and through stakeholder groups or organisations to influence provider performance (for example, CQC participates with other stakeholders in Quality Oversight Committees to drive improvement in trusts placed in special measures)
  • Lateral: CQC encourages providers to interact/collaborate to learn from each other’s experiences (for example, a provider reads inspection reports of nearby providers rated as outstanding and visits them to learn about how they improved quality)
  • Systemic: CQC seeks to influence the whole system (including policymakers and wider interests) on common issues of concern requiring wider action (for example, CQC conducts a thematic review of the mental health system for children and young people)

Appendix 3: Pilot C - Questions used for innovation-supporting conversations with corporate providers

Questions for inspectors (shared ahead of the corporate meeting)

  1. Has the local service flagged any innovation or improvement with you recently? Or if not, have they faced any challenges recently that required creative thinking?
  2. If yes, what was it? And what were the outcomes for the people using services? This could include any examples identified in recent contact/inspection
  3. What support do they get/would they like from their local authorities/ICS’ to help them innovate?
  4. Is there any support you think CQC should offer to help you innovate/improve?

Question for the corporate providers (shared ahead of the meeting)

  1. Is there any recent innovation or improvement that they would like to share or flag in the meeting? Either at a corporate level or one of their locations. Or if not, have they faced any challenges recently that required creative thinking?
  2. If yes: What was it? What were the outcomes for the people using services?
  3. What support do they get/would they like from local authorities/ICS’ to help them innovate?
  4. Is there any support you think CQC should offer to help you innovate/improve?
  5. How would they like to discuss innovation with CQC within these meetings? (is more guidance needed for this one?) safe space? Us bring them examples?

Appendix 4: Pilot D - Questions used for the survey regarding innovation since the COVID-19 pandemic

  1. You kindly told us about a new way of working that you started using in response to/during the pandemic. Is this something that you are still doing?
  2. If yes, this came out of a very specific time, can you tell us why you decided to keep using this way of working?
  3. If no, what led you to stop using this way of working? Did it not seem necessary after the height of the pandemic or were there other factors?
  4. Is there anything from the way you were required to act during the pandemic which you are no longer doing, but would like to still be doing or able to do?
  5. Did you carry out any evaluation or collect any data about the impact of your new way of working? If so, could you tell us about what you found? Are there outcomes that you could share?
  6. How did your colleagues feel about the new way of working? Was there any impact on morale, wellbeing, retention etc?
  7. Have you been able to share your innovation with anyone other providers? If so, what was the result?
  8. Has your innovation had an impact on how you work with the local system?
  9. Has working with other parts of the system led to any outcomes you wouldn’t have been able to achieve without collaboration?
  10. Have you been able to share your innovation with other parts of the local system?
  11. Is there anything that you learnt from the innovation journey that informs the way you approach problems today?

Appendix 5: Pilot H - Questions used for the survey: Are innovators using existing data sources to inform innovation?

  1. Is the use of data embedded in your approach to innovation?
  2. Do you review existing data sources to information your work around innovation?
  3. What are the specific data sets that you use?
  4. Is there a particular reason that you do not use existing data sets?
  5. At what stage of the innovation process do you consider existing sources of data?
  6. Is there a particular stage of the innovation process where you think access to external data would be more valuable?
  7. Do you consider health inequalities when developing new ways of working or innovations? If so, how?
  8. Please share any further comments you have about the use of existing data sources in innovation.
  9. We're also keen to know how you might use external guidance from other bodies – have you used any particular guidance to shape inform, drive or shape your innovation?
  10. During this project, innovators and providers have described the innovation journey in the stages listed below. Using these descriptions, at what point in the journey were these bits of guidance most helpful?

Capturing innovation to accelerate improvement

Print this document