Challenges

Limitations

There are limitations with the data processing related to:

Geographies

Data are published at different geographies. While some geographies map coterminously with Integrated Care System geographies, some approximate methods of aggregation are used for the ones that don’t.

Time periods of published data

Data are published at different time periods. Data are generally published based on regulatory requirements. Metrics can change over time as well. Therefore, metrics may exist for a few years and then stop being published. Equally, metrics may only be introduced very recently.

Frequencies of published data

Metrics can be published by month, quarter, and year (both financial or calendar). The outputs of the project are intended to inform medium term strategic planning, so the shorter frequency metrics are aggregated up to an annual time period to allow for annual predictions.

Comparable metrics

Data are generally published as raw figures. Without context, it is difficult to evaluate whether a figure is high or low. For added context, the metric needs to be calculated with a denominator. This enables comparison with other areas. The choice of denominator determines the metric value and it can have a large influence on how the metric differs between areas. The description of the metric, along with the denominator and numerator description, can be found in the configuration-table.csv file.

Interpretation

There are challenges around interpretation of metrics. The value of a metric can be influenced by many factors. To ensure that the outputs are practical, these factors need to be considered at all stages to ensure the metrics are most useful to the end user.

COVID-19 pandemic

The pandemic period will heavily influence a lot of metrics. Each metric may have been influenced in different ways and not represent business-as-usual, therefore relationships between metrics may drastically differ during that period from the non-pandemic period. This will impact any predictive model.