Article

We wanted to measure adherence to the guideline for depression in
disability assessments. The research questions we addressed were: How can we
develop performance indicators (PIs) for adherence to the Dutch guideline for
disability assessment of patients with depression and how can we measure the
quality of the scores? What is the inter-rater reliability of these PIs? What is
the quality of the PI scores? METHODS: PIs, developed by the researchers, were
reviewed on various aspects, by a panel of seven experts in several consulting
rounds. After adjustments, senior insurance physicians (IPs) attended two
training sessions and scored the PIs on 10 different simulated case reports. Two
researchers developed proxy 'gold standard' scores for these 10 case reports. To
assess the inter-rater reliability and the quality of the scores, we calculated
the intra-class correlations (ICC) and 95% confidence intervals (CI) of the PI
scores and of the PI scores compared to the proxy 'gold standard', respectively.
RESULTS: Six specific and relevant PIs resulted from the consultation of the
panel of experts. The PI scores for the 10 case reports, rated by seven (of the
eight) senior IPs who completed both training sessions, showed that the PIs were
not reliable at individual level (ICC = 0.543; 95% CI 0.426-0.642). However, the
ICC became more reliable as an average of two raters was calculated (ICC =
0.704). The ICC of the PI scores with the proxy 'gold standard' was 0.538 (95% CI
0.419-0.640), but the quality was higher when calculated as an average of two
raters (ICC = 0.700). CONCLUSION: The PIs for adherence to the guideline were
sufficiently reliable, and the quality of their scores was adequate if at least
two well-trained raters were involved. The senior IPs evaluated the feasibility
of the PIs as good, with a prerequisite of sufficient training. This method may
be interesting for measuring guideline adherence and quality of disability
assessments in general.

Langue : ANGLAIS