HSR&D Citation Abstract
Search | Search by Center | Search by Source | Keywords in Title
What a catch! traits that define good annotators.
Forbush TB, Shen S, South BR, Duvalla SL. What a catch! traits that define good annotators. Studies in health technology and informatics. 2013 Jan 1; 192:1213.
Human annotation and chart review is an important process in biomedical informatics research, but which humans are best suited for the job? Domain expertise, such as medical or linguistic knowledge is desirable, but other factors may be equally important. The University of Utah has a group of 20+ reviewers with backgrounds in medicine and linguistics, and 10 key traits have surfaced in those best able to annotate quickly and with high quality. To identify reviewers with these key traits, we created a hiring process that includes interviewing candidates, testing their medical and linguistic knowledge, and having them complete an annotation exercise on realistic medical text. Each step is designed to assess the key traits and allow the investigator to choose the skill set required for each project.