Evaluation of CQC's local authority pilot assessments

CQC’s Data and Insight team produced a dashboard for the assessment team to use to inform their assessments. It allows users to compare indicators associated with individual local authorities to get an overall understanding of how they meet their care duties. The report also provides contextual data to help users understand the broader context in which local authorities operate. Data for the dashboard was drawn from a variety of sources such as from the Adult Social Care Outcomes Framework (ASCOF) where it met a set of criteria for inclusion, such as alignment to the assessment framework.

The assessment team described reviewing the dashboard ahead of the fieldwork and reported finding the demographics and safeguarding data useful in providing context about the local authorities. Assessment leads also spoke of reviewing it as part of the report writing. External assessment team roles were not able to access the dashboard in full, but felt there could be value in having a summary ahead of the fieldwork.

The team felt the data was useful and reported that Data & Insight colleagues were very responsive to queries about it. However, the team could not be clear on the extent to which this information added value to the process. There were views that the dashboard would become more useful in time when there would be more comparative year-on-year data and between different local authorities.

Some wider points were raised about how the team might interpret and use some of the data. For example, one person pointed out that national data on safeguarding can contain wide variation. Team members themselves suggested that there may be a need to improve skills in how to use and interpret data and to better understand if positive or negative indicators are indeed as they appear. It was thought this would partly come in time as the team became more familiar with the data.

There are ongoing conversations about where some types of evidence are best placed in the new framework. As a result, the team found evidence in the dashboard was not always aligned to the published single assessment framework. This affected a small number of areas but did pose some challenges for the team when considering evidence in report writing.

Some local authorities reported being surprised they were not asked for more data about their performance as part of the information return. They understood this was because CQC was using nationally available data, but were interested in how this was feeding into the assessments. They also raised concerns that this data might not be as current as data they could supply directly.

Provider survey

To gather views from providers that the local authorities worked with, the assessment team and colleagues in CQC’s Data & Insight team devised a provider survey. This was developed during the pilots, which meant it was only ready to be used on the latter 3 assessments. Local authorities provided CQC with a list of their regulated providers and CQC sent the survey out through the Provider Engagement function. The intention of the survey was to provide the views of providers that worked with the local authorities, which may either inform the fieldwork and/or generally contribute to the evidence that the team was collecting.

Of the pilots that used this survey, there were mixed views about its value. We heard that it was distributed too late to inform the fieldwork and that the responses received were either so mixed in nature it was difficult to draw out any themes, or that the number of responses was too low to provide any additional insight. Despite this, the team felt the overall intention of the survey was good and that it could prove helpful. They felt it should be continued in the future approach providing it is distributed earlier to both increase response rates and allow time to analyse the information ahead of the fieldwork.

One local authority also commented that it led them to consider running a similar survey themselves. They also felt there was opportunity for CQC to share the responses collected with the local authorities for their own learning purposes, acknowledging these would need to be anonymised.