<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=1717549828521399&amp;ev=PageView&amp;noscript=1">
Blog Feature

By: Kevin Asp on December 25th, 2015

Print/Save as PDF

Improving Inter-scorer Reliability

Sleep Technologist Advice

isrrecord1.png

What is the Inter-scorer Reliability program?

The AASM Inter-scorer Reliability program used patient record samples to test various technologists'  scoring ability. Each record featured 200 epochs from a single recording, to be scored individually for Sleep Stage (S), Respiratory Events (R), Limb Movements (L) and Arousals (A).The program's purpose was to provide a unique opportunity to compare a large number of scorers with varied levels of experience to determine sleep stage scoring agreement. 

Why this program is meaningful to sleep technologists

The objective of the program was to examine areas of disagreement to inform future revisions of the AASM Manual for the Scoring of Sleep and Associated Events

According to the Journal of Clinical Sleep Medicine, the results were the following:

Sleep stage agreement averaged 82.6%. Agreement was highest for stage R sleep with stages N2 and W approaching the same level. Scoring agreement for stage N3 sleep was 67.4% and was lowest for stage N1 at 63.0%. Scorers had particular difficulty with the last epoch of stage W before sleep onset, the first epoch of stage N2 after stage N1 and the first epoch of stage R after stage N2. Discrimination between stages N2 and N3 was particularly difficult for scorers."

What can sleep technologists takeaway from this program?

The findings of this program suggest that with current rules, inter-scorer agreement in a large group is approximately 83%, a level similar to that reported for agreement between expert scorers. Agreement in the scoring of stages N1 and N3 sleep was low. Modifications to the scoring rules to improve scoring during sleep stage transitions may result in improvement.

You can learn more about this program by reading the AAST's Journal Club article here.

New Call-to-action