Automated measures of speech content and speech organization in schizophrenia: Test-retest reliability and generalizability across demographic variables.

Minor KS, Lundin NB, Myers EJ, Fernández-Villardón A, Lysaker PH. Automated measures of speech content and speech organization in schizophrenia: Test-retest reliability and generalizability across demographic variables. Psychiatry Research. 2023 Feb 1; 320:115048.

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.    Search Dimensions for VA for this citation
* Don't have VA-internal network access or a VA email address? Try searching the free-to-the-public version of Dimensions

Search for Abstract from PubMed

---

Abstract:

Technological advances in artificial intelligence and natural language processing have increased efficiency of assessing speech content and speech organization in schizophrenia. Despite these developments, there has been little focus on the psychometrics of these approaches. Using two common assessments, the current study addressed this gap by: 1) measuring test-retest reliability; and 2) assessing whether speech content and/or speech organization generalize across demographics. To test these aims, we examined psychometric properties of the Linguistic Inquiry Word Count (LIWC), a speech content measure, and the Coh-Metrix, a speech organization measure. Across baseline to six month (n = 101) and baseline to one year (n = 47) narrative speech samples, we generally observed fair reliability for speech content measures and fair to good reliability for speech organization measures. Regarding demographics, multiple speech indices varied by race, income, and education. The lack of excellent reliability scores for speech indices holds important implications for examining speech variables in clinical trials and highlights the dynamic nature of speech. This work illustrates the importance of designing speech content and speech organization measures with external validity across demographic factors. Future studies examining speech in schizophrenia should account for potential biases against demographic groups introduced by linguistic analysis tools.