single case study francais

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Perspective
• Published: 22 November 2022

Single case studies are a powerful tool for developing, testing and extending theories

• Lyndsey Nickels   ORCID: orcid.org/0000-0002-0311-3524 1 , 2 ,
• Simon Fischer-Baum   ORCID: orcid.org/0000-0002-6067-0538 3 &
• Wendy Best   ORCID: orcid.org/0000-0001-8375-5916 4  

Nature Reviews Psychology volume  1 ,  pages 733–747 ( 2022 ) Cite this article

5 Citations

26 Altmetric

Metrics details

• Neurological disorders

Psychology embraces a diverse range of methodologies. However, most rely on averaging group data to draw conclusions. In this Perspective, we argue that single case methodology is a valuable tool for developing and extending psychological theories. We stress the importance of single case and case series research, drawing on classic and contemporary cases in which cognitive and perceptual deficits provide insights into typical cognitive processes in domains such as memory, delusions, reading and face perception. We unpack the key features of single case methodology, describe its strengths, its value in adjudicating between theories, and outline its benefits for a better understanding of deficits and hence more appropriate interventions. The unique insights that single case studies have provided illustrate the value of in-depth investigation within an individual. Single case methodology has an important place in the psychologist’s toolkit and it should be valued as a primary research tool.

This is a preview of subscription content, access via your institution

Access options

Subscribe to this journal

Receive 12 digital issues and online access to articles

55,14 € per year

only 4,60 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Comparing meta-analyses and preregistered multiple-laboratory replication projects

The fundamental importance of method to theory

A critical evaluation of the p-factor literature

Corkin, S. Permanent Present Tense: The Unforgettable Life Of The Amnesic Patient, H. M . Vol. XIX, 364 (Basic Books, 2013).

Lilienfeld, S. O. Psychology: From Inquiry To Understanding (Pearson, 2019).

Schacter, D. L., Gilbert, D. T., Nock, M. K. & Wegner, D. M. Psychology (Worth Publishers, 2019).

Eysenck, M. W. & Brysbaert, M. Fundamentals Of Cognition (Routledge, 2018).

Squire, L. R. Memory and brain systems: 1969–2009. J. Neurosci. 29 , 12711–12716 (2009).

Article   PubMed   PubMed Central   Google Scholar  

Corkin, S. What’s new with the amnesic patient H.M.? Nat. Rev. Neurosci. 3 , 153–160 (2002).

Article   PubMed   Google Scholar  

Schubert, T. M. et al. Lack of awareness despite complex visual processing: evidence from event-related potentials in a case of selective metamorphopsia. Proc. Natl Acad. Sci. USA 117 , 16055–16064 (2020).

Behrmann, M. & Plaut, D. C. Bilateral hemispheric processing of words and faces: evidence from word impairments in prosopagnosia and face impairments in pure alexia. Cereb. Cortex 24 , 1102–1118 (2014).

Plaut, D. C. & Behrmann, M. Complementary neural representations for faces and words: a computational exploration. Cogn. Neuropsychol. 28 , 251–275 (2011).

Haxby, J. V. et al. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293 , 2425–2430 (2001).

Hirshorn, E. A. et al. Decoding and disrupting left midfusiform gyrus activity during word reading. Proc. Natl Acad. Sci. USA 113 , 8162–8167 (2016).

Kosakowski, H. L. et al. Selective responses to faces, scenes, and bodies in the ventral visual pathway of infants. Curr. Biol. 32 , 265–274.e5 (2022).

Harlow, J. Passage of an iron rod through the head. Boston Med. Surgical J . https://doi.org/10.1176/jnp.11.2.281 (1848).

Broca, P. Remarks on the seat of the faculty of articulated language, following an observation of aphemia (loss of speech). Bull. Soc. Anat. 6 , 330–357 (1861).

Google Scholar  

Dejerine, J. Contribution A L’étude Anatomo-pathologique Et Clinique Des Différentes Variétés De Cécité Verbale: I. Cécité Verbale Avec Agraphie Ou Troubles Très Marqués De L’écriture; II. Cécité Verbale Pure Avec Intégrité De L’écriture Spontanée Et Sous Dictée (Société de Biologie, 1892).

Liepmann, H. Das Krankheitsbild der Apraxie (“motorischen Asymbolie”) auf Grund eines Falles von einseitiger Apraxie (Fortsetzung). Eur. Neurol. 8 , 102–116 (1900).

Article   Google Scholar  

Basso, A., Spinnler, H., Vallar, G. & Zanobio, M. E. Left hemisphere damage and selective impairment of auditory verbal short-term memory. A case study. Neuropsychologia 20 , 263–274 (1982).

Humphreys, G. W. & Riddoch, M. J. The fractionation of visual agnosia. In Visual Object Processing: A Cognitive Neuropsychological Approach 281–306 (Lawrence Erlbaum, 1987).

Whitworth, A., Webster, J. & Howard, D. A Cognitive Neuropsychological Approach To Assessment And Intervention In Aphasia (Psychology Press, 2014).

Caramazza, A. On drawing inferences about the structure of normal cognitive systems from the analysis of patterns of impaired performance: the case for single-patient studies. Brain Cogn. 5 , 41–66 (1986).

Caramazza, A. & McCloskey, M. The case for single-patient studies. Cogn. Neuropsychol. 5 , 517–527 (1988).

Shallice, T. Cognitive neuropsychology and its vicissitudes: the fate of Caramazza’s axioms. Cogn. Neuropsychol. 32 , 385–411 (2015).

Shallice, T. From Neuropsychology To Mental Structure (Cambridge Univ. Press, 1988).

Coltheart, M. Assumptions and methods in cognitive neuropscyhology. In The Handbook Of Cognitive Neuropsychology: What Deficits Reveal About The Human Mind (ed. Rapp, B.) 3–22 (Psychology Press, 2001).

McCloskey, M. & Chaisilprungraung, T. The value of cognitive neuropsychology: the case of vision research. Cogn. Neuropsychol. 34 , 412–419 (2017).

McCloskey, M. The future of cognitive neuropsychology. In The Handbook Of Cognitive Neuropsychology: What Deficits Reveal About The Human Mind (ed. Rapp, B.) 593–610 (Psychology Press, 2001).

Lashley, K. S. In search of the engram. In Physiological Mechanisms in Animal Behavior 454–482 (Academic Press, 1950).

Squire, L. R. & Wixted, J. T. The cognitive neuroscience of human memory since H.M. Annu. Rev. Neurosci. 34 , 259–288 (2011).

Stone, G. O., Vanhoy, M. & Orden, G. C. V. Perception is a two-way street: feedforward and feedback phonology in visual word recognition. J. Mem. Lang. 36 , 337–359 (1997).

Perfetti, C. A. The psycholinguistics of spelling and reading. In Learning To Spell: Research, Theory, And Practice Across Languages 21–38 (Lawrence Erlbaum, 1997).

Nickels, L. The autocue? self-generated phonemic cues in the treatment of a disorder of reading and naming. Cogn. Neuropsychol. 9 , 155–182 (1992).

Rapp, B., Benzing, L. & Caramazza, A. The autonomy of lexical orthography. Cogn. Neuropsychol. 14 , 71–104 (1997).

Bonin, P., Roux, S. & Barry, C. Translating nonverbal pictures into verbal word names. Understanding lexical access and retrieval. In Past, Present, And Future Contributions Of Cognitive Writing Research To Cognitive Psychology 315–522 (Psychology Press, 2011).

Bonin, P., Fayol, M. & Gombert, J.-E. Role of phonological and orthographic codes in picture naming and writing: an interference paradigm study. Cah. Psychol. Cogn./Current Psychol. Cogn. 16 , 299–324 (1997).

Bonin, P., Fayol, M. & Peereman, R. Masked form priming in writing words from pictures: evidence for direct retrieval of orthographic codes. Acta Psychol. 99 , 311–328 (1998).

Bentin, S., Allison, T., Puce, A., Perez, E. & McCarthy, G. Electrophysiological studies of face perception in humans. J. Cogn. Neurosci. 8 , 551–565 (1996).

Jeffreys, D. A. Evoked potential studies of face and object processing. Vis. Cogn. 3 , 1–38 (1996).

Laganaro, M., Morand, S., Michel, C. M., Spinelli, L. & Schnider, A. ERP correlates of word production before and after stroke in an aphasic patient. J. Cogn. Neurosci. 23 , 374–381 (2011).

Indefrey, P. & Levelt, W. J. M. The spatial and temporal signatures of word production components. Cognition 92 , 101–144 (2004).

Valente, A., Burki, A. & Laganaro, M. ERP correlates of word production predictors in picture naming: a trial by trial multiple regression analysis from stimulus onset to response. Front. Neurosci. 8 , 390 (2014).

Kittredge, A. K., Dell, G. S., Verkuilen, J. & Schwartz, M. F. Where is the effect of frequency in word production? Insights from aphasic picture-naming errors. Cogn. Neuropsychol. 25 , 463–492 (2008).

Domdei, N. et al. Ultra-high contrast retinal display system for single photoreceptor psychophysics. Biomed. Opt. Express 9 , 157 (2018).

Poldrack, R. A. et al. Long-term neural and physiological phenotyping of a single human. Nat. Commun. 6 , 8885 (2015).

Coltheart, M. The assumptions of cognitive neuropsychology: reflections on Caramazza (1984, 1986). Cogn. Neuropsychol. 34 , 397–402 (2017).

Badecker, W. & Caramazza, A. A final brief in the case against agrammatism: the role of theory in the selection of data. Cognition 24 , 277–282 (1986).

Fischer-Baum, S. Making sense of deviance: Identifying dissociating cases within the case series approach. Cogn. Neuropsychol. 30 , 597–617 (2013).

Nickels, L., Howard, D. & Best, W. On the use of different methodologies in cognitive neuropsychology: drink deep and from several sources. Cogn. Neuropsychol. 28 , 475–485 (2011).

Dell, G. S. & Schwartz, M. F. Who’s in and who’s out? Inclusion criteria, model evaluation, and the treatment of exceptions in case series. Cogn. Neuropsychol. 28 , 515–520 (2011).

Schwartz, M. F. & Dell, G. S. Case series investigations in cognitive neuropsychology. Cogn. Neuropsychol. 27 , 477–494 (2010).

Cohen, J. A power primer. Psychol. Bull. 112 , 155–159 (1992).

Martin, R. C. & Allen, C. Case studies in neuropsychology. In APA Handbook Of Research Methods In Psychology Vol. 2 Research Designs: Quantitative, Qualitative, Neuropsychological, And Biological (eds Cooper, H. et al.) 633–646 (American Psychological Association, 2012).

Leivada, E., Westergaard, M., Duñabeitia, J. A. & Rothman, J. On the phantom-like appearance of bilingualism effects on neurocognition: (how) should we proceed? Bilingualism 24 , 197–210 (2021).

Arnett, J. J. The neglected 95%: why American psychology needs to become less American. Am. Psychol. 63 , 602–614 (2008).

Stolz, J. A., Besner, D. & Carr, T. H. Implications of measures of reliability for theories of priming: activity in semantic memory is inherently noisy and uncoordinated. Vis. Cogn. 12 , 284–336 (2005).

Cipora, K. et al. A minority pulls the sample mean: on the individual prevalence of robust group-level cognitive phenomena — the instance of the SNARC effect. Preprint at psyArXiv https://doi.org/10.31234/osf.io/bwyr3 (2019).

Andrews, S., Lo, S. & Xia, V. Individual differences in automatic semantic priming. J. Exp. Psychol. Hum. Percept. Perform. 43 , 1025–1039 (2017).

Tan, L. C. & Yap, M. J. Are individual differences in masked repetition and semantic priming reliable? Vis. Cogn. 24 , 182–200 (2016).

Olsson-Collentine, A., Wicherts, J. M. & van Assen, M. A. L. M. Heterogeneity in direct replications in psychology and its association with effect size. Psychol. Bull. 146 , 922–940 (2020).

Gratton, C. & Braga, R. M. Editorial overview: deep imaging of the individual brain: past, practice, and promise. Curr. Opin. Behav. Sci. 40 , iii–vi (2021).

Fedorenko, E. The early origins and the growing popularity of the individual-subject analytic approach in human neuroscience. Curr. Opin. Behav. Sci. 40 , 105–112 (2021).

Xue, A. et al. The detailed organization of the human cerebellum estimated by intrinsic functional connectivity within the individual. J. Neurophysiol. 125 , 358–384 (2021).

Petit, S. et al. Toward an individualized neural assessment of receptive language in children. J. Speech Lang. Hear. Res. 63 , 2361–2385 (2020).

Jung, K.-H. et al. Heterogeneity of cerebral white matter lesions and clinical correlates in older adults. Stroke 52 , 620–630 (2021).

Falcon, M. I., Jirsa, V. & Solodkin, A. A new neuroinformatics approach to personalized medicine in neurology: the virtual brain. Curr. Opin. Neurol. 29 , 429–436 (2016).

Duncan, G. J., Engel, M., Claessens, A. & Dowsett, C. J. Replication and robustness in developmental research. Dev. Psychol. 50 , 2417–2425 (2014).

Open Science Collaboration. Estimating the reproducibility of psychological science. Science 349 , aac4716 (2015).

Tackett, J. L., Brandes, C. M., King, K. M. & Markon, K. E. Psychology’s replication crisis and clinical psychological science. Annu. Rev. Clin. Psychol. 15 , 579–604 (2019).

Munafò, M. R. et al. A manifesto for reproducible science. Nat. Hum. Behav. 1 , 0021 (2017).

Oldfield, R. C. & Wingfield, A. The time it takes to name an object. Nature 202 , 1031–1032 (1964).

Oldfield, R. C. & Wingfield, A. Response latencies in naming objects. Q. J. Exp. Psychol. 17 , 273–281 (1965).

Brysbaert, M. How many participants do we have to include in properly powered experiments? A tutorial of power analysis with reference tables. J. Cogn. 2 , 16 (2019).

Brysbaert, M. Power considerations in bilingualism research: time to step up our game. Bilingualism https://doi.org/10.1017/S1366728920000437 (2020).

Machery, E. What is a replication? Phil. Sci. 87 , 545–567 (2020).

Nosek, B. A. & Errington, T. M. What is replication? PLoS Biol. 18 , e3000691 (2020).

Li, X., Huang, L., Yao, P. & Hyönä, J. Universal and specific reading mechanisms across different writing systems. Nat. Rev. Psychol. 1 , 133–144 (2022).

Rapp, B. (Ed.) The Handbook Of Cognitive Neuropsychology: What Deficits Reveal About The Human Mind (Psychology Press, 2001).

Code, C. et al. Classic Cases In Neuropsychology (Psychology Press, 1996).

Patterson, K., Marshall, J. C. & Coltheart, M. Surface Dyslexia: Neuropsychological And Cognitive Studies Of Phonological Reading (Routledge, 2017).

Marshall, J. C. & Newcombe, F. Patterns of paralexia: a psycholinguistic approach. J. Psycholinguist. Res. 2 , 175–199 (1973).

Castles, A. & Coltheart, M. Varieties of developmental dyslexia. Cognition 47 , 149–180 (1993).

Khentov-Kraus, L. & Friedmann, N. Vowel letter dyslexia. Cogn. Neuropsychol. 35 , 223–270 (2018).

Winskel, H. Orthographic and phonological parafoveal processing of consonants, vowels, and tones when reading Thai. Appl. Psycholinguist. 32 , 739–759 (2011).

Hepner, C., McCloskey, M. & Rapp, B. Do reading and spelling share orthographic representations? Evidence from developmental dysgraphia. Cogn. Neuropsychol. 34 , 119–143 (2017).

Hanley, J. R. & Sotiropoulos, A. Developmental surface dysgraphia without surface dyslexia. Cogn. Neuropsychol. 35 , 333–341 (2018).

Zihl, J. & Heywood, C. A. The contribution of single case studies to the neuroscience of vision: single case studies in vision neuroscience. Psych. J. 5 , 5–17 (2016).

Bouvier, S. E. & Engel, S. A. Behavioral deficits and cortical damage loci in cerebral achromatopsia. Cereb. Cortex 16 , 183–191 (2006).

Zihl, J. & Heywood, C. A. The contribution of LM to the neuroscience of movement vision. Front. Integr. Neurosci. 9 , 6 (2015).

Dotan, D. & Friedmann, N. Separate mechanisms for number reading and word reading: evidence from selective impairments. Cortex 114 , 176–192 (2019).

McCloskey, M. & Schubert, T. Shared versus separate processes for letter and digit identification. Cogn. Neuropsychol. 31 , 437–460 (2014).

Fayol, M. & Seron, X. On numerical representations. Insights from experimental, neuropsychological, and developmental research. In Handbook of Mathematical Cognition (ed. Campbell, J.) 3–23 (Psychological Press, 2005).

Bornstein, B. & Kidron, D. P. Prosopagnosia. J. Neurol. Neurosurg. Psychiat. 22 , 124–131 (1959).

Kühn, C. D., Gerlach, C., Andersen, K. B., Poulsen, M. & Starrfelt, R. Face recognition in developmental dyslexia: evidence for dissociation between faces and words. Cogn. Neuropsychol. 38 , 107–115 (2021).

Barton, J. J. S., Albonico, A., Susilo, T., Duchaine, B. & Corrow, S. L. Object recognition in acquired and developmental prosopagnosia. Cogn. Neuropsychol. 36 , 54–84 (2019).

Renault, B., Signoret, J.-L., Debruille, B., Breton, F. & Bolgert, F. Brain potentials reveal covert facial recognition in prosopagnosia. Neuropsychologia 27 , 905–912 (1989).

Bauer, R. M. Autonomic recognition of names and faces in prosopagnosia: a neuropsychological application of the guilty knowledge test. Neuropsychologia 22 , 457–469 (1984).

Haan, E. H. F., de, Young, A. & Newcombe, F. Face recognition without awareness. Cogn. Neuropsychol. 4 , 385–415 (1987).

Ellis, H. D. & Lewis, M. B. Capgras delusion: a window on face recognition. Trends Cogn. Sci. 5 , 149–156 (2001).

Ellis, H. D., Young, A. W., Quayle, A. H. & De Pauw, K. W. Reduced autonomic responses to faces in Capgras delusion. Proc. R. Soc. Lond. B 264 , 1085–1092 (1997).

Collins, M. N., Hawthorne, M. E., Gribbin, N. & Jacobson, R. Capgras’ syndrome with organic disorders. Postgrad. Med. J. 66 , 1064–1067 (1990).

Enoch, D., Puri, B. K. & Ball, H. Uncommon Psychiatric Syndromes 5th edn (Routledge, 2020).

Tranel, D., Damasio, H. & Damasio, A. R. Double dissociation between overt and covert face recognition. J. Cogn. Neurosci. 7 , 425–432 (1995).

Brighetti, G., Bonifacci, P., Borlimi, R. & Ottaviani, C. “Far from the heart far from the eye”: evidence from the Capgras delusion. Cogn. Neuropsychiat. 12 , 189–197 (2007).

Coltheart, M., Langdon, R. & McKay, R. Delusional belief. Annu. Rev. Psychol. 62 , 271–298 (2011).

Coltheart, M. Cognitive neuropsychiatry and delusional belief. Q. J. Exp. Psychol. 60 , 1041–1062 (2007).

Coltheart, M. & Davies, M. How unexpected observations lead to new beliefs: a Peircean pathway. Conscious. Cogn. 87 , 103037 (2021).

Coltheart, M. & Davies, M. Failure of hypothesis evaluation as a factor in delusional belief. Cogn. Neuropsychiat. 26 , 213–230 (2021).

McCloskey, M. et al. A developmental deficit in localizing objects from vision. Psychol. Sci. 6 , 112–117 (1995).

McCloskey, M., Valtonen, J. & Cohen Sherman, J. Representing orientation: a coordinate-system hypothesis and evidence from developmental deficits. Cogn. Neuropsychol. 23 , 680–713 (2006).

McCloskey, M. Spatial representations and multiple-visual-systems hypotheses: evidence from a developmental deficit in visual location and orientation processing. Cortex 40 , 677–694 (2004).

Gregory, E. & McCloskey, M. Mirror-image confusions: implications for representation and processing of object orientation. Cognition 116 , 110–129 (2010).

Gregory, E., Landau, B. & McCloskey, M. Representation of object orientation in children: evidence from mirror-image confusions. Vis. Cogn. 19 , 1035–1062 (2011).

Laine, M. & Martin, N. Cognitive neuropsychology has been, is, and will be significant to aphasiology. Aphasiology 26 , 1362–1376 (2012).

Howard, D. & Patterson, K. The Pyramids And Palm Trees Test: A Test Of Semantic Access From Words And Pictures (Thames Valley Test Co., 1992).

Kay, J., Lesser, R. & Coltheart, M. PALPA: Psycholinguistic Assessments Of Language Processing In Aphasia. 2: Picture & Word Semantics, Sentence Comprehension (Erlbaum, 2001).

Franklin, S. Dissociations in auditory word comprehension; evidence from nine fluent aphasic patients. Aphasiology 3 , 189–207 (1989).

Howard, D., Swinburn, K. & Porter, G. Putting the CAT out: what the comprehensive aphasia test has to offer. Aphasiology 24 , 56–74 (2010).

Conti-Ramsden, G., Crutchley, A. & Botting, N. The extent to which psychometric tests differentiate subgroups of children with SLI. J. Speech Lang. Hear. Res. 40 , 765–777 (1997).

Bishop, D. V. M. & McArthur, G. M. Individual differences in auditory processing in specific language impairment: a follow-up study using event-related potentials and behavioural thresholds. Cortex 41 , 327–341 (2005).

Bishop, D. V. M., Snowling, M. J., Thompson, P. A. & Greenhalgh, T., and the CATALISE-2 consortium. Phase 2 of CATALISE: a multinational and multidisciplinary Delphi consensus study of problems with language development: terminology. J. Child. Psychol. Psychiat. 58 , 1068–1080 (2017).

Wilson, A. J. et al. Principles underlying the design of ‘the number race’, an adaptive computer game for remediation of dyscalculia. Behav. Brain Funct. 2 , 19 (2006).

Basso, A. & Marangolo, P. Cognitive neuropsychological rehabilitation: the emperor’s new clothes? Neuropsychol. Rehabil. 10 , 219–229 (2000).

Murad, M. H., Asi, N., Alsawas, M. & Alahdab, F. New evidence pyramid. Evidence-based Med. 21 , 125–127 (2016).

Greenhalgh, T., Howick, J. & Maskrey, N., for the Evidence Based Medicine Renaissance Group. Evidence based medicine: a movement in crisis? Br. Med. J. 348 , g3725–g3725 (2014).

Best, W., Ping Sze, W., Edmundson, A. & Nickels, L. What counts as evidence? Swimming against the tide: valuing both clinically informed experimentally controlled case series and randomized controlled trials in intervention research. Evidence-based Commun. Assess. Interv. 13 , 107–135 (2019).

Best, W. et al. Understanding differing outcomes from semantic and phonological interventions with children with word-finding difficulties: a group and case series study. Cortex 134 , 145–161 (2021).

OCEBM Levels of Evidence Working Group. The Oxford Levels of Evidence 2. CEBM https://www.cebm.ox.ac.uk/resources/levels-of-evidence/ocebm-levels-of-evidence (2011).

Holler, D. E., Behrmann, M. & Snow, J. C. Real-world size coding of solid objects, but not 2-D or 3-D images, in visual agnosia patients with bilateral ventral lesions. Cortex 119 , 555–568 (2019).

Duchaine, B. C., Yovel, G., Butterworth, E. J. & Nakayama, K. Prosopagnosia as an impairment to face-specific mechanisms: elimination of the alternative hypotheses in a developmental case. Cogn. Neuropsychol. 23 , 714–747 (2006).

Hartley, T. et al. The hippocampus is required for short-term topographical memory in humans. Hippocampus 17 , 34–48 (2007).

Pishnamazi, M. et al. Attentional bias towards and away from fearful faces is modulated by developmental amygdala damage. Cortex 81 , 24–34 (2016).

Rapp, B., Fischer-Baum, S. & Miozzo, M. Modality and morphology: what we write may not be what we say. Psychol. Sci. 26 , 892–902 (2015).

Yong, K. X. X., Warren, J. D., Warrington, E. K. & Crutch, S. J. Intact reading in patients with profound early visual dysfunction. Cortex 49 , 2294–2306 (2013).

Rockland, K. S. & Van Hoesen, G. W. Direct temporal–occipital feedback connections to striate cortex (V1) in the macaque monkey. Cereb. Cortex 4 , 300–313 (1994).

Haynes, J.-D., Driver, J. & Rees, G. Visibility reflects dynamic changes of effective connectivity between V1 and fusiform cortex. Neuron 46 , 811–821 (2005).

Tanaka, K. Mechanisms of visual object recognition: monkey and human studies. Curr. Opin. Neurobiol. 7 , 523–529 (1997).

Fischer-Baum, S., McCloskey, M. & Rapp, B. Representation of letter position in spelling: evidence from acquired dysgraphia. Cognition 115 , 466–490 (2010).

Houghton, G. The problem of serial order: a neural network model of sequence learning and recall. In Current Research In Natural Language Generation (eds Dale, R., Mellish, C. & Zock, M.) 287–319 (Academic Press, 1990).

Fieder, N., Nickels, L., Biedermann, B. & Best, W. From “some butter” to “a butter”: an investigation of mass and count representation and processing. Cogn. Neuropsychol. 31 , 313–349 (2014).

Fieder, N., Nickels, L., Biedermann, B. & Best, W. How ‘some garlic’ becomes ‘a garlic’ or ‘some onion’: mass and count processing in aphasia. Neuropsychologia 75 , 626–645 (2015).

Schröder, A., Burchert, F. & Stadie, N. Training-induced improvement of noncanonical sentence production does not generalize to comprehension: evidence for modality-specific processes. Cogn. Neuropsychol. 32 , 195–220 (2015).

Stadie, N. et al. Unambiguous generalization effects after treatment of non-canonical sentence production in German agrammatism. Brain Lang. 104 , 211–229 (2008).

Schapiro, A. C., Gregory, E., Landau, B., McCloskey, M. & Turk-Browne, N. B. The necessity of the medial temporal lobe for statistical learning. J. Cogn. Neurosci. 26 , 1736–1747 (2014).

Schapiro, A. C., Kustner, L. V. & Turk-Browne, N. B. Shaping of object representations in the human medial temporal lobe based on temporal regularities. Curr. Biol. 22 , 1622–1627 (2012).

Baddeley, A., Vargha-Khadem, F. & Mishkin, M. Preserved recognition in a case of developmental amnesia: implications for the acaquisition of semantic memory? J. Cogn. Neurosci. 13 , 357–369 (2001).

Snyder, J. J. & Chatterjee, A. Spatial-temporal anisometries following right parietal damage. Neuropsychologia 42 , 1703–1708 (2004).

Ashkenazi, S., Henik, A., Ifergane, G. & Shelef, I. Basic numerical processing in left intraparietal sulcus (IPS) acalculia. Cortex 44 , 439–448 (2008).

Lebrun, M.-A., Moreau, P., McNally-Gagnon, A., Mignault Goulet, G. & Peretz, I. Congenital amusia in childhood: a case study. Cortex 48 , 683–688 (2012).

Vannuscorps, G., Andres, M. & Pillon, A. When does action comprehension need motor involvement? Evidence from upper limb aplasia. Cogn. Neuropsychol. 30 , 253–283 (2013).

Jeannerod, M. Neural simulation of action: a unifying mechanism for motor cognition. NeuroImage 14 , S103–S109 (2001).

Blakemore, S.-J. & Decety, J. From the perception of action to the understanding of intention. Nat. Rev. Neurosci. 2 , 561–567 (2001).

Rizzolatti, G. & Craighero, L. The mirror-neuron system. Annu. Rev. Neurosci. 27 , 169–192 (2004).

Forde, E. M. E., Humphreys, G. W. & Remoundou, M. Disordered knowledge of action order in action disorganisation syndrome. Neurocase 10 , 19–28 (2004).

Mazzi, C. & Savazzi, S. The glamor of old-style single-case studies in the neuroimaging era: insights from a patient with hemianopia. Front. Psychol. 10 , 965 (2019).

Coltheart, M. What has functional neuroimaging told us about the mind (so far)? (Position Paper Presented to the European Cognitive Neuropsychology Workshop, Bressanone, 2005). Cortex 42 , 323–331 (2006).

Page, M. P. A. What can’t functional neuroimaging tell the cognitive psychologist? Cortex 42 , 428–443 (2006).

Blank, I. A., Kiran, S. & Fedorenko, E. Can neuroimaging help aphasia researchers? Addressing generalizability, variability, and interpretability. Cogn. Neuropsychol. 34 , 377–393 (2017).

Niv, Y. The primacy of behavioral research for understanding the brain. Behav. Neurosci. 135 , 601–609 (2021).

Crawford, J. R. & Howell, D. C. Comparing an individual’s test score against norms derived from small samples. Clin. Neuropsychol. 12 , 482–486 (1998).

Crawford, J. R., Garthwaite, P. H. & Ryan, K. Comparing a single case to a control sample: testing for neuropsychological deficits and dissociations in the presence of covariates. Cortex 47 , 1166–1178 (2011).

McIntosh, R. D. & Rittmo, J. Ö. Power calculations in single-case neuropsychology: a practical primer. Cortex 135 , 146–158 (2021).

Patterson, K. & Plaut, D. C. “Shallow draughts intoxicate the brain”: lessons from cognitive science for cognitive neuropsychology. Top. Cogn. Sci. 1 , 39–58 (2009).

Lambon Ralph, M. A., Patterson, K. & Plaut, D. C. Finite case series or infinite single-case studies? Comments on “Case series investigations in cognitive neuropsychology” by Schwartz and Dell (2010). Cogn. Neuropsychol. 28 , 466–474 (2011).

Horien, C., Shen, X., Scheinost, D. & Constable, R. T. The individual functional connectome is unique and stable over months to years. NeuroImage 189 , 676–687 (2019).

Epelbaum, S. et al. Pure alexia as a disconnection syndrome: new diffusion imaging evidence for an old concept. Cortex 44 , 962–974 (2008).

Fischer-Baum, S. & Campana, G. Neuroplasticity and the logic of cognitive neuropsychology. Cogn. Neuropsychol. 34 , 403–411 (2017).

Paul, S., Baca, E. & Fischer-Baum, S. Cerebellar contributions to orthographic working memory: a single case cognitive neuropsychological investigation. Neuropsychologia 171 , 108242 (2022).

Feinstein, J. S., Adolphs, R., Damasio, A. & Tranel, D. The human amygdala and the induction and experience of fear. Curr. Biol. 21 , 34–38 (2011).

Crawford, J., Garthwaite, P. & Gray, C. Wanted: fully operational definitions of dissociations in single-case studies. Cortex 39 , 357–370 (2003).

McIntosh, R. D. Simple dissociations for a higher-powered neuropsychology. Cortex 103 , 256–265 (2018).

McIntosh, R. D. & Brooks, J. L. Current tests and trends in single-case neuropsychology. Cortex 47 , 1151–1159 (2011).

Best, W., Schröder, A. & Herbert, R. An investigation of a relative impairment in naming non-living items: theoretical and methodological implications. J. Neurolinguistics 19 , 96–123 (2006).

Franklin, S., Howard, D. & Patterson, K. Abstract word anomia. Cogn. Neuropsychol. 12 , 549–566 (1995).

Coltheart, M., Patterson, K. E. & Marshall, J. C. Deep Dyslexia (Routledge, 1980).

Nickels, L., Kohnen, S. & Biedermann, B. An untapped resource: treatment as a tool for revealing the nature of cognitive processes. Cogn. Neuropsychol. 27 , 539–562 (2010).

Download references

Acknowledgements

The authors thank all of those pioneers of and advocates for single case study research who have mentored, inspired and encouraged us over the years, and the many other colleagues with whom we have discussed these issues.

Author information

Authors and affiliations.

School of Psychological Sciences & Macquarie University Centre for Reading, Macquarie University, Sydney, New South Wales, Australia

Lyndsey Nickels

NHMRC Centre of Research Excellence in Aphasia Recovery and Rehabilitation, Australia

Psychological Sciences, Rice University, Houston, TX, USA

Simon Fischer-Baum

Psychology and Language Sciences, University College London, London, UK

You can also search for this author in PubMed   Google Scholar

Contributions

L.N. led and was primarily responsible for the structuring and writing of the manuscript. All authors contributed to all aspects of the article.

Corresponding author

Correspondence to Lyndsey Nickels .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Reviews Psychology thanks Yanchao Bi, Rob McIntosh, and the other, anonymous, reviewer for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Nickels, L., Fischer-Baum, S. & Best, W. Single case studies are a powerful tool for developing, testing and extending theories. Nat Rev Psychol 1 , 733–747 (2022). https://doi.org/10.1038/s44159-022-00127-y

Download citation

Accepted : 13 October 2022

Published : 22 November 2022

Issue Date : December 2022

DOI : https://doi.org/10.1038/s44159-022-00127-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Case Study Research Method in Psychology

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Case studies are in-depth investigations of a person, group, event, or community. Typically, data is gathered from various sources using several methods (e.g., observations & interviews).

The case study research method originated in clinical medicine (the case history, i.e., the patient’s personal history). In psychology, case studies are often confined to the study of a particular individual.

The information is mainly biographical and relates to events in the individual’s past (i.e., retrospective), as well as to significant events that are currently occurring in his or her everyday life.

The case study is not a research method, but researchers select methods of data collection and analysis that will generate material suitable for case studies.

Freud (1909a, 1909b) conducted very detailed investigations into the private lives of his patients in an attempt to both understand and help them overcome their illnesses.

This makes it clear that the case study is a method that should only be used by a psychologist, therapist, or psychiatrist, i.e., someone with a professional qualification.

There is an ethical issue of competence. Only someone qualified to diagnose and treat a person can conduct a formal case study relating to atypical (i.e., abnormal) behavior or atypical development.

 Famous Case Studies

• Anna O – One of the most famous case studies, documenting psychoanalyst Josef Breuer’s treatment of “Anna O” (real name Bertha Pappenheim) for hysteria in the late 1800s using early psychoanalytic theory.
• Little Hans – A child psychoanalysis case study published by Sigmund Freud in 1909 analyzing his five-year-old patient Herbert Graf’s house phobia as related to the Oedipus complex.
• Bruce/Brenda – Gender identity case of the boy (Bruce) whose botched circumcision led psychologist John Money to advise gender reassignment and raise him as a girl (Brenda) in the 1960s.
• Genie Wiley – Linguistics/psychological development case of the victim of extreme isolation abuse who was studied in 1970s California for effects of early language deprivation on acquiring speech later in life.
• Phineas Gage – One of the most famous neuropsychology case studies analyzes personality changes in railroad worker Phineas Gage after an 1848 brain injury involving a tamping iron piercing his skull.

Clinical Case Studies

• Studying the effectiveness of psychotherapy approaches with an individual patient
• Assessing and treating mental illnesses like depression, anxiety disorders, PTSD
• Neuropsychological cases investigating brain injuries or disorders

Child Psychology Case Studies

• Studying psychological development from birth through adolescence
• Cases of learning disabilities, autism spectrum disorders, ADHD
• Effects of trauma, abuse, deprivation on development

Types of Case Studies

• Explanatory case studies : Used to explore causation in order to find underlying principles. Helpful for doing qualitative analysis to explain presumed causal links.
• Exploratory case studies : Used to explore situations where an intervention being evaluated has no clear set of outcomes. It helps define questions and hypotheses for future research.
• Descriptive case studies : Describe an intervention or phenomenon and the real-life context in which it occurred. It is helpful for illustrating certain topics within an evaluation.
• Multiple-case studies : Used to explore differences between cases and replicate findings across cases. Helpful for comparing and contrasting specific cases.
• Intrinsic : Used to gain a better understanding of a particular case. Helpful for capturing the complexity of a single case.
• Collective : Used to explore a general phenomenon using multiple case studies. Helpful for jointly studying a group of cases in order to inquire into the phenomenon.

Where Do You Find Data for a Case Study?

There are several places to find data for a case study. The key is to gather data from multiple sources to get a complete picture of the case and corroborate facts or findings through triangulation of evidence. Most of this information is likely qualitative (i.e., verbal description rather than measurement), but the psychologist might also collect numerical data.

1. Primary sources

• Interviews – Interviewing key people related to the case to get their perspectives and insights. The interview is an extremely effective procedure for obtaining information about an individual, and it may be used to collect comments from the person’s friends, parents, employer, workmates, and others who have a good knowledge of the person, as well as to obtain facts from the person him or herself.
• Observations – Observing behaviors, interactions, processes, etc., related to the case as they unfold in real-time.
• Documents & Records – Reviewing private documents, diaries, public records, correspondence, meeting minutes, etc., relevant to the case.

2. Secondary sources

• News/Media – News coverage of events related to the case study.
• Academic articles – Journal articles, dissertations etc. that discuss the case.
• Government reports – Official data and records related to the case context.
• Books/films – Books, documentaries or films discussing the case.

3. Archival records

Searching historical archives, museum collections and databases to find relevant documents, visual/audio records related to the case history and context.

Public archives like newspapers, organizational records, photographic collections could all include potentially relevant pieces of information to shed light on attitudes, cultural perspectives, common practices and historical contexts related to psychology.

4. Organizational records

Organizational records offer the advantage of often having large datasets collected over time that can reveal or confirm psychological insights.

Of course, privacy and ethical concerns regarding confidential data must be navigated carefully.

However, with proper protocols, organizational records can provide invaluable context and empirical depth to qualitative case studies exploring the intersection of psychology and organizations.

• Organizational/industrial psychology research : Organizational records like employee surveys, turnover/retention data, policies, incident reports etc. may provide insight into topics like job satisfaction, workplace culture and dynamics, leadership issues, employee behaviors etc.
• Clinical psychology : Therapists/hospitals may grant access to anonymized medical records to study aspects like assessments, diagnoses, treatment plans etc. This could shed light on clinical practices.
• School psychology : Studies could utilize anonymized student records like test scores, grades, disciplinary issues, and counseling referrals to study child development, learning barriers, effectiveness of support programs, and more.

How do I Write a Case Study in Psychology?

Follow specified case study guidelines provided by a journal or your psychology tutor. General components of clinical case studies include: background, symptoms, assessments, diagnosis, treatment, and outcomes. Interpreting the information means the researcher decides what to include or leave out. A good case study should always clarify which information is the factual description and which is an inference or the researcher’s opinion.

1. Introduction

• Provide background on the case context and why it is of interest, presenting background information like demographics, relevant history, and presenting problem.
• Compare briefly to similar published cases if applicable. Clearly state the focus/importance of the case.

2. Case Presentation

• Describe the presenting problem in detail, including symptoms, duration,and impact on daily life.
• Include client demographics like age and gender, information about social relationships, and mental health history.
• Describe all physical, emotional, and/or sensory symptoms reported by the client.
• Use patient quotes to describe the initial complaint verbatim. Follow with full-sentence summaries of relevant history details gathered, including key components that led to a working diagnosis.
• Summarize clinical exam results, namely orthopedic/neurological tests, imaging, lab tests, etc. Note actual results rather than subjective conclusions. Provide images if clearly reproducible/anonymized.
• Clearly state the working diagnosis or clinical impression before transitioning to management.

3. Management and Outcome

• Indicate the total duration of care and number of treatments given over what timeframe. Use specific names/descriptions for any therapies/interventions applied.
• Present the results of the intervention,including any quantitative or qualitative data collected.
• For outcomes, utilize visual analog scales for pain, medication usage logs, etc., if possible. Include patient self-reports of improvement/worsening of symptoms. Note the reason for discharge/end of care.

4. Discussion

• Analyze the case, exploring contributing factors, limitations of the study, and connections to existing research.
• Analyze the effectiveness of the intervention,considering factors like participant adherence, limitations of the study, and potential alternative explanations for the results.
• Identify any questions raised in the case analysis and relate insights to established theories and current research if applicable. Avoid definitive claims about physiological explanations.
• Offer clinical implications, and suggest future research directions.

5. Additional Items

• Thank specific assistants for writing support only. No patient acknowledgments.
• References should directly support any key claims or quotes included.
• Use tables/figures/images only if substantially informative. Include permissions and legends/explanatory notes.
• Provides detailed (rich qualitative) information.
• Provides insight for further research.
• Permitting investigation of otherwise impractical (or unethical) situations.

Case studies allow a researcher to investigate a topic in far more detail than might be possible if they were trying to deal with a large number of research participants (nomothetic approach) with the aim of ‘averaging’.

Because of their in-depth, multi-sided approach, case studies often shed light on aspects of human thinking and behavior that would be unethical or impractical to study in other ways.

Research that only looks into the measurable aspects of human behavior is not likely to give us insights into the subjective dimension of experience, which is important to psychoanalytic and humanistic psychologists.

Case studies are often used in exploratory research. They can help us generate new ideas (that might be tested by other methods). They are an important way of illustrating theories and can help show how different aspects of a person’s life are related to each other.

The method is, therefore, important for psychologists who adopt a holistic point of view (i.e., humanistic psychologists ).

Limitations

• Lacking scientific rigor and providing little basis for generalization of results to the wider population.
• Researchers’ own subjective feelings may influence the case study (researcher bias).
• Difficult to replicate.
• Time-consuming and expensive.
• The volume of data, together with the time restrictions in place, impacted the depth of analysis that was possible within the available resources.

Because a case study deals with only one person/event/group, we can never be sure if the case study investigated is representative of the wider body of “similar” instances. This means the conclusions drawn from a particular case may not be transferable to other settings.

Because case studies are based on the analysis of qualitative (i.e., descriptive) data , a lot depends on the psychologist’s interpretation of the information she has acquired.

This means that there is a lot of scope for Anna O , and it could be that the subjective opinions of the psychologist intrude in the assessment of what the data means.

For example, Freud has been criticized for producing case studies in which the information was sometimes distorted to fit particular behavioral theories (e.g., Little Hans ).

This is also true of Money’s interpretation of the Bruce/Brenda case study (Diamond, 1997) when he ignored evidence that went against his theory.

Breuer, J., & Freud, S. (1895).  Studies on hysteria . Standard Edition 2: London.

Curtiss, S. (1981). Genie: The case of a modern wild child .

Diamond, M., & Sigmundson, K. (1997). Sex Reassignment at Birth: Long-term Review and Clinical Implications. Archives of Pediatrics & Adolescent Medicine , 151(3), 298-304

Freud, S. (1909a). Analysis of a phobia of a five year old boy. In The Pelican Freud Library (1977), Vol 8, Case Histories 1, pages 169-306

Freud, S. (1909b). Bemerkungen über einen Fall von Zwangsneurose (Der “Rattenmann”). Jb. psychoanal. psychopathol. Forsch ., I, p. 357-421; GW, VII, p. 379-463; Notes upon a case of obsessional neurosis, SE , 10: 151-318.

Harlow J. M. (1848). Passage of an iron rod through the head.  Boston Medical and Surgical Journal, 39 , 389–393.

Harlow, J. M. (1868).  Recovery from the Passage of an Iron Bar through the Head .  Publications of the Massachusetts Medical Society. 2  (3), 327-347.

Money, J., & Ehrhardt, A. A. (1972).  Man & Woman, Boy & Girl : The Differentiation and Dimorphism of Gender Identity from Conception to Maturity. Baltimore, Maryland: Johns Hopkins University Press.

Money, J., & Tucker, P. (1975). Sexual signatures: On being a man or a woman.

Further Information

• Case Study Approach
• Case Study Method
• Enhancing the Quality of Case Studies in Health Services Research
• “We do things together” A case study of “couplehood” in dementia
• Using mixed methods for evaluating an integrative approach to cancer care: a case study

Related Articles

Research Methodology

Qualitative Data Coding

What Is a Focus Group?

Cross-Cultural Research Methodology In Psychology

What Is Internal Validity In Research?

Research Methodology , Statistics

What Is Face Validity In Research? Importance & How To Measure

Criterion Validity: Definition & Examples

The Advantages and Limitations of Single Case Study Analysis

As Andrew Bennett and Colin Elman have recently noted, qualitative research methods presently enjoy “an almost unprecedented popularity and vitality… in the international relations sub-field”, such that they are now “indisputably prominent, if not pre-eminent” (2010: 499). This is, they suggest, due in no small part to the considerable advantages that case study methods in particular have to offer in studying the “complex and relatively unstructured and infrequent phenomena that lie at the heart of the subfield” (Bennett and Elman, 2007: 171). Using selected examples from within the International Relations literature[1], this paper aims to provide a brief overview of the main principles and distinctive advantages and limitations of single case study analysis. Divided into three inter-related sections, the paper therefore begins by first identifying the underlying principles that serve to constitute the case study as a particular research strategy, noting the somewhat contested nature of the approach in ontological, epistemological, and methodological terms. The second part then looks to the principal single case study types and their associated advantages, including those from within the recent ‘third generation’ of qualitative International Relations (IR) research. The final section of the paper then discusses the most commonly articulated limitations of single case studies; while accepting their susceptibility to criticism, it is however suggested that such weaknesses are somewhat exaggerated. The paper concludes that single case study analysis has a great deal to offer as a means of both understanding and explaining contemporary international relations.

The term ‘case study’, John Gerring has suggested, is “a definitional morass… Evidently, researchers have many different things in mind when they talk about case study research” (2006a: 17). It is possible, however, to distil some of the more commonly-agreed principles. One of the most prominent advocates of case study research, Robert Yin (2009: 14) defines it as “an empirical enquiry that investigates a contemporary phenomenon in depth and within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident”. What this definition usefully captures is that case studies are intended – unlike more superficial and generalising methods – to provide a level of detail and understanding, similar to the ethnographer Clifford Geertz’s (1973) notion of ‘thick description’, that allows for the thorough analysis of the complex and particularistic nature of distinct phenomena. Another frequently cited proponent of the approach, Robert Stake, notes that as a form of research the case study “is defined by interest in an individual case, not by the methods of inquiry used”, and that “the object of study is a specific, unique, bounded system” (2008: 443, 445). As such, three key points can be derived from this – respectively concerning issues of ontology, epistemology, and methodology – that are central to the principles of single case study research.

First, the vital notion of ‘boundedness’ when it comes to the particular unit of analysis means that defining principles should incorporate both the synchronic (spatial) and diachronic (temporal) elements of any so-called ‘case’. As Gerring puts it, a case study should be “an intensive study of a single unit… a spatially bounded phenomenon – e.g. a nation-state, revolution, political party, election, or person – observed at a single point in time or over some delimited period of time” (2004: 342). It is important to note, however, that – whereas Gerring refers to a single unit of analysis – it may be that attention also necessarily be given to particular sub-units. This points to the important difference between what Yin refers to as an ‘holistic’ case design, with a single unit of analysis, and an ’embedded’ case design with multiple units of analysis (Yin, 2009: 50-52). The former, for example, would examine only the overall nature of an international organization, whereas the latter would also look to specific departments, programmes, or policies etc.

Secondly, as Tim May notes of the case study approach, “even the most fervent advocates acknowledge that the term has entered into understandings with little specification or discussion of purpose and process” (2011: 220). One of the principal reasons for this, he argues, is the relationship between the use of case studies in social research and the differing epistemological traditions – positivist, interpretivist, and others – within which it has been utilised. Philosophy of science concerns are obviously a complex issue, and beyond the scope of much of this paper. That said, the issue of how it is that we know what we know – of whether or not a single independent reality exists of which we as researchers can seek to provide explanation – does lead us to an important distinction to be made between so-called idiographic and nomothetic case studies (Gerring, 2006b). The former refers to those which purport to explain only a single case, are concerned with particularisation, and hence are typically (although not exclusively) associated with more interpretivist approaches. The latter are those focused studies that reflect upon a larger population and are more concerned with generalisation, as is often so with more positivist approaches[2]. The importance of this distinction, and its relation to the advantages and limitations of single case study analysis, is returned to below.

Thirdly, in methodological terms, given that the case study has often been seen as more of an interpretivist and idiographic tool, it has also been associated with a distinctly qualitative approach (Bryman, 2009: 67-68). However, as Yin notes, case studies can – like all forms of social science research – be exploratory, descriptive, and/or explanatory in nature. It is “a common misconception”, he notes, “that the various research methods should be arrayed hierarchically… many social scientists still deeply believe that case studies are only appropriate for the exploratory phase of an investigation” (Yin, 2009: 6). If case studies can reliably perform any or all three of these roles – and given that their in-depth approach may also require multiple sources of data and the within-case triangulation of methods – then it becomes readily apparent that they should not be limited to only one research paradigm. Exploratory and descriptive studies usually tend toward the qualitative and inductive, whereas explanatory studies are more often quantitative and deductive (David and Sutton, 2011: 165-166). As such, the association of case study analysis with a qualitative approach is a “methodological affinity, not a definitional requirement” (Gerring, 2006a: 36). It is perhaps better to think of case studies as transparadigmatic; it is mistaken to assume single case study analysis to adhere exclusively to a qualitative methodology (or an interpretivist epistemology) even if it – or rather, practitioners of it – may be so inclined. By extension, this also implies that single case study analysis therefore remains an option for a multitude of IR theories and issue areas; it is how this can be put to researchers’ advantage that is the subject of the next section.

Having elucidated the defining principles of the single case study approach, the paper now turns to an overview of its main benefits. As noted above, a lack of consensus still exists within the wider social science literature on the principles and purposes – and by extension the advantages and limitations – of case study research. Given that this paper is directed towards the particular sub-field of International Relations, it suggests Bennett and Elman’s (2010) more discipline-specific understanding of contemporary case study methods as an analytical framework. It begins however, by discussing Harry Eckstein’s seminal (1975) contribution to the potential advantages of the case study approach within the wider social sciences.

Eckstein proposed a taxonomy which usefully identified what he considered to be the five most relevant types of case study. Firstly were so-called configurative-idiographic studies, distinctly interpretivist in orientation and predicated on the assumption that “one cannot attain prediction and control in the natural science sense, but only understanding ( verstehen )… subjective values and modes of cognition are crucial” (1975: 132). Eckstein’s own sceptical view was that any interpreter ‘simply’ considers a body of observations that are not self-explanatory and “without hard rules of interpretation, may discern in them any number of patterns that are more or less equally plausible” (1975: 134). Those of a more post-modernist bent, of course – sharing an “incredulity towards meta-narratives”, in Lyotard’s (1994: xxiv) evocative phrase – would instead suggest that this more free-form approach actually be advantageous in delving into the subtleties and particularities of individual cases.

Eckstein’s four other types of case study, meanwhile, promote a more nomothetic (and positivist) usage. As described, disciplined-configurative studies were essentially about the use of pre-existing general theories, with a case acting “passively, in the main, as a receptacle for putting theories to work” (Eckstein, 1975: 136). As opposed to the opportunity this presented primarily for theory application, Eckstein identified heuristic case studies as explicit theoretical stimulants – thus having instead the intended advantage of theory-building. So-called p lausibility probes entailed preliminary attempts to determine whether initial hypotheses should be considered sound enough to warrant more rigorous and extensive testing. Finally, and perhaps most notably, Eckstein then outlined the idea of crucial case studies , within which he also included the idea of ‘most-likely’ and ‘least-likely’ cases; the essential characteristic of crucial cases being their specific theory-testing function.

Whilst Eckstein’s was an early contribution to refining the case study approach, Yin’s (2009: 47-52) more recent delineation of possible single case designs similarly assigns them roles in the applying, testing, or building of theory, as well as in the study of unique cases[3]. As a subset of the latter, however, Jack Levy (2008) notes that the advantages of idiographic cases are actually twofold. Firstly, as inductive/descriptive cases – akin to Eckstein’s configurative-idiographic cases – whereby they are highly descriptive, lacking in an explicit theoretical framework and therefore taking the form of “total history”. Secondly, they can operate as theory-guided case studies, but ones that seek only to explain or interpret a single historical episode rather than generalise beyond the case. Not only does this therefore incorporate ‘single-outcome’ studies concerned with establishing causal inference (Gerring, 2006b), it also provides room for the more postmodern approaches within IR theory, such as discourse analysis, that may have developed a distinct methodology but do not seek traditional social scientific forms of explanation.

Applying specifically to the state of the field in contemporary IR, Bennett and Elman identify a ‘third generation’ of mainstream qualitative scholars – rooted in a pragmatic scientific realist epistemology and advocating a pluralistic approach to methodology – that have, over the last fifteen years, “revised or added to essentially every aspect of traditional case study research methods” (2010: 502). They identify ‘process tracing’ as having emerged from this as a central method of within-case analysis. As Bennett and Checkel observe, this carries the advantage of offering a methodologically rigorous “analysis of evidence on processes, sequences, and conjunctures of events within a case, for the purposes of either developing or testing hypotheses about causal mechanisms that might causally explain the case” (2012: 10).

Harnessing various methods, process tracing may entail the inductive use of evidence from within a case to develop explanatory hypotheses, and deductive examination of the observable implications of hypothesised causal mechanisms to test their explanatory capability[4]. It involves providing not only a coherent explanation of the key sequential steps in a hypothesised process, but also sensitivity to alternative explanations as well as potential biases in the available evidence (Bennett and Elman 2010: 503-504). John Owen (1994), for example, demonstrates the advantages of process tracing in analysing whether the causal factors underpinning democratic peace theory are – as liberalism suggests – not epiphenomenal, but variously normative, institutional, or some given combination of the two or other unexplained mechanism inherent to liberal states. Within-case process tracing has also been identified as advantageous in addressing the complexity of path-dependent explanations and critical junctures – as for example with the development of political regime types – and their constituent elements of causal possibility, contingency, closure, and constraint (Bennett and Elman, 2006b).

Bennett and Elman (2010: 505-506) also identify the advantages of single case studies that are implicitly comparative: deviant, most-likely, least-likely, and crucial cases. Of these, so-called deviant cases are those whose outcome does not fit with prior theoretical expectations or wider empirical patterns – again, the use of inductive process tracing has the advantage of potentially generating new hypotheses from these, either particular to that individual case or potentially generalisable to a broader population. A classic example here is that of post-independence India as an outlier to the standard modernisation theory of democratisation, which holds that higher levels of socio-economic development are typically required for the transition to, and consolidation of, democratic rule (Lipset, 1959; Diamond, 1992). Absent these factors, MacMillan’s single case study analysis (2008) suggests the particularistic importance of the British colonial heritage, the ideology and leadership of the Indian National Congress, and the size and heterogeneity of the federal state.

Most-likely cases, as per Eckstein above, are those in which a theory is to be considered likely to provide a good explanation if it is to have any application at all, whereas least-likely cases are ‘tough test’ ones in which the posited theory is unlikely to provide good explanation (Bennett and Elman, 2010: 505). Levy (2008) neatly refers to the inferential logic of the least-likely case as the ‘Sinatra inference’ – if a theory can make it here, it can make it anywhere. Conversely, if a theory cannot pass a most-likely case, it is seriously impugned. Single case analysis can therefore be valuable for the testing of theoretical propositions, provided that predictions are relatively precise and measurement error is low (Levy, 2008: 12-13). As Gerring rightly observes of this potential for falsification:

“a positivist orientation toward the work of social science militates toward a greater appreciation of the case study format, not a denigration of that format, as is usually supposed” (Gerring, 2007: 247, emphasis added).

In summary, the various forms of single case study analysis can – through the application of multiple qualitative and/or quantitative research methods – provide a nuanced, empirically-rich, holistic account of specific phenomena. This may be particularly appropriate for those phenomena that are simply less amenable to more superficial measures and tests (or indeed any substantive form of quantification) as well as those for which our reasons for understanding and/or explaining them are irreducibly subjective – as, for example, with many of the normative and ethical issues associated with the practice of international relations. From various epistemological and analytical standpoints, single case study analysis can incorporate both idiographic sui generis cases and, where the potential for generalisation may exist, nomothetic case studies suitable for the testing and building of causal hypotheses. Finally, it should not be ignored that a signal advantage of the case study – with particular relevance to international relations – also exists at a more practical rather than theoretical level. This is, as Eckstein noted, “that it is economical for all resources: money, manpower, time, effort… especially important, of course, if studies are inherently costly, as they are if units are complex collective individuals ” (1975: 149-150, emphasis added).

Limitations

Single case study analysis has, however, been subject to a number of criticisms, the most common of which concern the inter-related issues of methodological rigour, researcher subjectivity, and external validity. With regard to the first point, the prototypical view here is that of Zeev Maoz (2002: 164-165), who suggests that “the use of the case study absolves the author from any kind of methodological considerations. Case studies have become in many cases a synonym for freeform research where anything goes”. The absence of systematic procedures for case study research is something that Yin (2009: 14-15) sees as traditionally the greatest concern due to a relative absence of methodological guidelines. As the previous section suggests, this critique seems somewhat unfair; many contemporary case study practitioners – and representing various strands of IR theory – have increasingly sought to clarify and develop their methodological techniques and epistemological grounding (Bennett and Elman, 2010: 499-500).

A second issue, again also incorporating issues of construct validity, concerns that of the reliability and replicability of various forms of single case study analysis. This is usually tied to a broader critique of qualitative research methods as a whole. However, whereas the latter obviously tend toward an explicitly-acknowledged interpretive basis for meanings, reasons, and understandings:

“quantitative measures appear objective, but only so long as we don’t ask questions about where and how the data were produced… pure objectivity is not a meaningful concept if the goal is to measure intangibles [as] these concepts only exist because we can interpret them” (Berg and Lune, 2010: 340).

The question of researcher subjectivity is a valid one, and it may be intended only as a methodological critique of what are obviously less formalised and researcher-independent methods (Verschuren, 2003). Owen (1994) and Layne’s (1994) contradictory process tracing results of interdemocratic war-avoidance during the Anglo-American crisis of 1861 to 1863 – from liberal and realist standpoints respectively – are a useful example. However, it does also rest on certain assumptions that can raise deeper and potentially irreconcilable ontological and epistemological issues. There are, regardless, plenty such as Bent Flyvbjerg (2006: 237) who suggest that the case study contains no greater bias toward verification than other methods of inquiry, and that “on the contrary, experience indicates that the case study contains a greater bias toward falsification of preconceived notions than toward verification”.

The third and arguably most prominent critique of single case study analysis is the issue of external validity or generalisability. How is it that one case can reliably offer anything beyond the particular? “We always do better (or, in the extreme, no worse) with more observation as the basis of our generalization”, as King et al write; “in all social science research and all prediction, it is important that we be as explicit as possible about the degree of uncertainty that accompanies out prediction” (1994: 212). This is an unavoidably valid criticism. It may be that theories which pass a single crucial case study test, for example, require rare antecedent conditions and therefore actually have little explanatory range. These conditions may emerge more clearly, as Van Evera (1997: 51-54) notes, from large-N studies in which cases that lack them present themselves as outliers exhibiting a theory’s cause but without its predicted outcome. As with the case of Indian democratisation above, it would logically be preferable to conduct large-N analysis beforehand to identify that state’s non-representative nature in relation to the broader population.

There are, however, three important qualifiers to the argument about generalisation that deserve particular mention here. The first is that with regard to an idiographic single-outcome case study, as Eckstein notes, the criticism is “mitigated by the fact that its capability to do so [is] never claimed by its exponents; in fact it is often explicitly repudiated” (1975: 134). Criticism of generalisability is of little relevance when the intention is one of particularisation. A second qualifier relates to the difference between statistical and analytical generalisation; single case studies are clearly less appropriate for the former but arguably retain significant utility for the latter – the difference also between explanatory and exploratory, or theory-testing and theory-building, as discussed above. As Gerring puts it, “theory confirmation/disconfirmation is not the case study’s strong suit” (2004: 350). A third qualification relates to the issue of case selection. As Seawright and Gerring (2008) note, the generalisability of case studies can be increased by the strategic selection of cases. Representative or random samples may not be the most appropriate, given that they may not provide the richest insight (or indeed, that a random and unknown deviant case may appear). Instead, and properly used , atypical or extreme cases “often reveal more information because they activate more actors… and more basic mechanisms in the situation studied” (Flyvbjerg, 2006). Of course, this also points to the very serious limitation, as hinted at with the case of India above, that poor case selection may alternatively lead to overgeneralisation and/or grievous misunderstandings of the relationship between variables or processes (Bennett and Elman, 2006a: 460-463).

As Tim May (2011: 226) notes, “the goal for many proponents of case studies […] is to overcome dichotomies between generalizing and particularizing, quantitative and qualitative, deductive and inductive techniques”. Research aims should drive methodological choices, rather than narrow and dogmatic preconceived approaches. As demonstrated above, there are various advantages to both idiographic and nomothetic single case study analyses – notably the empirically-rich, context-specific, holistic accounts that they have to offer, and their contribution to theory-building and, to a lesser extent, that of theory-testing. Furthermore, while they do possess clear limitations, any research method involves necessary trade-offs; the inherent weaknesses of any one method, however, can potentially be offset by situating them within a broader, pluralistic mixed-method research strategy. Whether or not single case studies are used in this fashion, they clearly have a great deal to offer.

References 

Bennett, A. and Checkel, J. T. (2012) ‘Process Tracing: From Philosophical Roots to Best Practice’, Simons Papers in Security and Development, No. 21/2012, School for International Studies, Simon Fraser University: Vancouver.

Bennett, A. and Elman, C. (2006a) ‘Qualitative Research: Recent Developments in Case Study Methods’, Annual Review of Political Science , 9, 455-476.

Bennett, A. and Elman, C. (2006b) ‘Complex Causal Relations and Case Study Methods: The Example of Path Dependence’, Political Analysis , 14, 3, 250-267.

Bennett, A. and Elman, C. (2007) ‘Case Study Methods in the International Relations Subfield’, Comparative Political Studies , 40, 2, 170-195.

Bennett, A. and Elman, C. (2010) Case Study Methods. In C. Reus-Smit and D. Snidal (eds) The Oxford Handbook of International Relations . Oxford University Press: Oxford. Ch. 29.

Berg, B. and Lune, H. (2012) Qualitative Research Methods for the Social Sciences . Pearson: London.

Bryman, A. (2012) Social Research Methods . Oxford University Press: Oxford.

David, M. and Sutton, C. D. (2011) Social Research: An Introduction . SAGE Publications Ltd: London.

Diamond, J. (1992) ‘Economic development and democracy reconsidered’, American Behavioral Scientist , 35, 4/5, 450-499.

Eckstein, H. (1975) Case Study and Theory in Political Science. In R. Gomm, M. Hammersley, and P. Foster (eds) Case Study Method . SAGE Publications Ltd: London.

Flyvbjerg, B. (2006) ‘Five Misunderstandings About Case-Study Research’, Qualitative Inquiry , 12, 2, 219-245.

Geertz, C. (1973) The Interpretation of Cultures: Selected Essays by Clifford Geertz . Basic Books Inc: New York.

Gerring, J. (2004) ‘What is a Case Study and What Is It Good for?’, American Political Science Review , 98, 2, 341-354.

Gerring, J. (2006a) Case Study Research: Principles and Practices . Cambridge University Press: Cambridge.

Gerring, J. (2006b) ‘Single-Outcome Studies: A Methodological Primer’, International Sociology , 21, 5, 707-734.

Gerring, J. (2007) ‘Is There a (Viable) Crucial-Case Method?’, Comparative Political Studies , 40, 3, 231-253.

King, G., Keohane, R. O. and Verba, S. (1994) Designing Social Inquiry: Scientific Inference in Qualitative Research . Princeton University Press: Chichester.

Layne, C. (1994) ‘Kant or Cant: The Myth of the Democratic Peace’, International Security , 19, 2, 5-49.

Levy, J. S. (2008) ‘Case Studies: Types, Designs, and Logics of Inference’, Conflict Management and Peace Science , 25, 1-18.

Lipset, S. M. (1959) ‘Some Social Requisites of Democracy: Economic Development and Political Legitimacy’, The American Political Science Review , 53, 1, 69-105.

Lyotard, J-F. (1984) The Postmodern Condition: A Report on Knowledge . University of Minnesota Press: Minneapolis.

MacMillan, A. (2008) ‘Deviant Democratization in India’, Democratization , 15, 4, 733-749.

Maoz, Z. (2002) Case study methodology in international studies: from storytelling to hypothesis testing. In F. P. Harvey and M. Brecher (eds) Evaluating Methodology in International Studies . University of Michigan Press: Ann Arbor.

May, T. (2011) Social Research: Issues, Methods and Process . Open University Press: Maidenhead.

Owen, J. M. (1994) ‘How Liberalism Produces Democratic Peace’, International Security , 19, 2, 87-125.

Seawright, J. and Gerring, J. (2008) ‘Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options’, Political Research Quarterly , 61, 2, 294-308.

Stake, R. E. (2008) Qualitative Case Studies. In N. K. Denzin and Y. S. Lincoln (eds) Strategies of Qualitative Inquiry . Sage Publications: Los Angeles. Ch. 17.

Van Evera, S. (1997) Guide to Methods for Students of Political Science . Cornell University Press: Ithaca.

Verschuren, P. J. M. (2003) ‘Case study as a research strategy: some ambiguities and opportunities’, International Journal of Social Research Methodology , 6, 2, 121-139.

Yin, R. K. (2009) Case Study Research: Design and Methods . SAGE Publications Ltd: London.

[1] The paper follows convention by differentiating between ‘International Relations’ as the academic discipline and ‘international relations’ as the subject of study.

[2] There is some similarity here with Stake’s (2008: 445-447) notion of intrinsic cases, those undertaken for a better understanding of the particular case, and instrumental ones that provide insight for the purposes of a wider external interest.

[3] These may be unique in the idiographic sense, or in nomothetic terms as an exception to the generalising suppositions of either probabilistic or deterministic theories (as per deviant cases, below).

[4] Although there are “philosophical hurdles to mount”, according to Bennett and Checkel, there exists no a priori reason as to why process tracing (as typically grounded in scientific realism) is fundamentally incompatible with various strands of positivism or interpretivism (2012: 18-19). By extension, it can therefore be incorporated by a range of contemporary mainstream IR theories.

— Written by: Ben Willis Written at: University of Plymouth Written for: David Brockington Date written: January 2013

Further Reading on E-International Relations

• Identity in International Conflicts: A Case Study of the Cuban Missile Crisis
• Imperialism’s Legacy in the Study of Contemporary Politics: The Case of Hegemonic Stability Theory
• Recreating a Nation’s Identity Through Symbolism: A Chinese Case Study
• Ontological Insecurity: A Case Study on Israeli-Palestinian Conflict in Jerusalem
• Terrorists or Freedom Fighters: A Case Study of ETA
• A Critical Assessment of Eco-Marxism: A Ghanaian Case Study

Please Consider Donating

Before you download your free e-book, please consider donating to support open access publishing.

E-IR is an independent non-profit publisher run by an all volunteer team. Your donations allow us to invest in new open access titles and pay our bandwidth bills to ensure we keep our existing titles free to view. Any amount, in any currency, is appreciated. Many thanks!

Donations are voluntary and not required to download the e-book - your link to download is below.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Perspect Behav Sci
• v.45(1); 2022 Mar

Advancing the Application and Use of Single-Case Research Designs: Reflections on Articles from the Special Issue

Robert h. horner.

1 University of Oregon, Eugene, OR 97401 USA

John Ferron

2 University of South Florida, Tampa, FL USA

This special issue of Perspective on Behavior Science is a productive contribution to current advances in the use and documentation of single-case research designs. We focus in this article on major themes emphasized by the articles in this issue and suggest directions for improving professional standards focused on the design, analysis, and dissemination of single-case research.

The application of single-case research methods is entering a new phase of scientific relevance. Researchers in an increasing array of disciplines are finding single-case methods useful for the questions they are asking and the clinical needs in their fields (Kratochwill et al., 2010 ; Maggin et al., 2017 ; Maggin & Odom, 2014 ; Riley-Tillman, Burns, & Kilgus, 2020 ). With this special issue the editors have challenged authors to articulate the advances in research design and data analysis that will be needed if single-case methods are to meet these emerging expectations. Each recruited article delves into a specific avenue of concern for advancing the use of single-case methods. The purpose of this discussion is to integrate themes identified by the authors and offer perspective for advancing the application and use of single-case methods. We provide initial context and then focus on the unifying messages the authors provide for both interpreting single-case research results and designing studies that will be of greatest benefit.

A special issue of Perspectives on Behavior Science focused on methodological advances needed for single-case research is a timely contribution to the field. There are growing efforts to both articulate professional standards for single-case methods (Kratochwill et al., 2010 ; Tate et al., 2016 ), and advance new procedures for analysis and interpretation of single-case studies (Manolov & Moeyaert, 2017 ; Pustejovsky et al., 2014 ; Riley-Tillman et al., 2020 ). Foremost among these trends is the goal of including single-case methods in the identification of empirically validated clinical practices (Slocum et al., 2014 ). Often labeled “evidence-based practices” the emerging message is that federal, state, and local agencies will join with professional associations in advancing investment in practices that have empirically documented effectiveness, efficiency, and safety. This movement depends on each discipline defining credible protocols for identifying empirically validate procedures, and in the present context, the use of single-case methods to achieve this goal.

This special issue comes to the field following recent publication of the What Works Clearinghouse 4.1 standards for single-case design (Institute of Education Sciences, 2020 ). At this time, the repeated demonstrations that single-case methods are useful, valid, and increasingly well-defined holds great promise. For single-case methods to achieve the impact they promise, however, there remains a need for (1) professional acceptance of research design standards, (2) agreement on data analysis standards (both interpreting individual studies, and for larger meta-analyses), and (3) incorporation of these standards in journal review protocols, grant review protocols, and university training programs targeting research design. This special issue offers a useful foundation for advancing the field in each of these areas.

A Role for Experimental Single-Case Designs

One important theme across the articles is recognition that the scientific community is unifying in acceptance that the core features of experimental single-case designs allow credible documentation of functional relations (experimental control). This is a large message, and one that needs to be more overtly noted across disciplines where single-case methods are less often used. Of special value is the distinction between rigorous single-case experimental designs and clinical case studies, or formal descriptive time-series analyses. The iterative collection of data across time with periodic experimenter-manipulation of treatments is useful both as a clinical tool and, when this approach is linked with designs that control for threats to internal validity, to the advancement of science.

Combining Visual Analysis and Statistical Analysis

Another major message from the recruited articles is that interpretation of single-case research designs will benefit from (even require) incorporation of statistical tools. Single-case researchers have used visual analysis as the initial step in examining evidence (Parsonson & Baer, 1978 ; Ledford & Gast, 2018 ; Kazdin, 2021 ; Riley-Tillman et al., 2020 ). Rigorous use of visual analysis involves (1) examining the data from each phase of the study to define within phase patterns, (2) comparing data patterns of adjacent phases, (3) comparing data patterns of similar phases, (4) examining the full set of data within a design to assess if the design has been effective at controlling threats to internal validity, and if there are at least three demonstrations of effect (each at a different point in time), and (5) determining if there are instances of noneffect or contra-indicated effect.

When assessing a single phase (or similar phases) of a study, the researcher considers (1) number of data points, (2) level (mean) score, (3) variability of scores, and (4) within phase trend(s). When comparing adjacent phases, the researcher examines if there is a change in the pattern of data following manipulation of the independent variable. Phase comparisons are done by simultaneously assessing (1) change in level, (2) change in variability, (3) change in trend, (4) immediacy of any change in pattern, (5) degree of overlap in data between the two phases, and (6) similarity in the patterns of data from similar phases (e.g., two baseline phases).

When assessing the overall design, the researcher looks at all the data to determine if an effect (e.g., change in the pattern of the dependent variable following manipulation of the independent variable) is observed at least three different times, each at a different point in time. The researcher also examines if there are manipulations of the independent variable where change in the dependent variable did not occur or occurred in the opposite direction expected by the hypothesis under consideration.

At present there is active discussion about the need for visual analysis as a component in the analysis protocol with single-case studies (Institute of Education Sciences, 2020 ). It is clear that the number of data points per phase, mean of these points, variability, and within phase trend are all easily calculatable. As the authors of articles in this issue note, there also are creative approaches to examining if there is change in the data patterns across adjacent phases. We view these approaches as major advances, and positive assets to the task of interpreting single-case evidence. We also recognize, however, that none of the proposed statistical options simultaneously examine the full set of variables traditionally used to guide visual analysis (level, trend, variability, immediacy, overlap, similarity of pattern across similar phases), nor do they include protocols for adjusting the weight given to each variable when assessing an effect (e.g., level is weighted differently in phases with stable data patterns than in phases with strong trends). Most important, visual analysis offers a more nuanced interpretation of data patterns. The role of outliers, within phase shifts in data patterns, and shifts in data patterns at similar times (within a multiple baseline design) are more apparent via visual analysis, and useful sources of information for assessing the stability and clinical relevance of effects. At this point we continue to see visual analysis as the appropriate first step in assessment of single-case studies, but strongly support the addition of statistical tools that yield valuable quantitative summaries of specific aspects of the analysis.

Align Data Analysis with Research Purpose

A theme that emerges in this special issue is the importance of aligning the aspects of the analysis that are quantified with the purposes of the study. We are fortunate to see in this special issue a variety of quantitative summaries that are tailored to meet a variety of purposes. There are methods helpful in estimating the size of the average treatment effect, and these vary depending on whether the focus is on quantifying in a standardized way the change in level (Cox et al., this issue ) or a change in slope or variability (Manolov et al., this issue-a ). In addition, there are methods to quantify the consistency of effects across replications (Manolov et al., this issue-b ) and other methods to summarize the degree to which the size of effects relates to characteristics of the participants (Moeyaert et al., this issue ). There are also estimates of the probability of the observed difference occurring in the absence of a treatment effect (Friedel et al., this issue ; Manolov et al., this issue-b ), methods that rely on a series of probability estimates to aid in the interpretation of FA (Kranak & Hall, this issue ), and summaries used to identify overselectivity (Mason et al., this issue ). In each case a strong rationale is available to support specific conditions where the proposed analysis would be useful. The important message is that no one analysis is applicable to all conditions and clarifying the purpose and structure of a specific study is critical when deciding which analysis to implement.

In addition to the need for aligning the statistical analysis with the study purpose, is the need for alignment of the logic and assumptions underlying the quantitative summary with the design and data from the single-case study. For example, the interpretation of a change in level as a measure of the size of the effect is made more meaningful when the experimental design controls for threats to internal validity and a visual analysis reveals an absence of trends, an absence of level shifts that are not coincident with intervention, and a problematic level of baseline responding. Probabilities based on randomization tests (Manolov et al., this issue-b ) are more meaningfully interpreted when the design incorporates randomization and the data permutations are restricted to the possible random assignments, whereas probabilities based on Monte Carlo resampling methods (Friedel et al., this issue ) are based on an assumption of exchangeability, and thus more meaningful when the time series are stable.

Single-case researchers will increasingly be expected to integrate statistical analyses in their reporting of results. The number of statistical options will continue to expand, and the analyses will become increasingly easy to implement through software applications. For the field to capitalize on these advancements, it will be important for single-case researchers to be flexible, selecting quantifications that are well matched to their purposes, study design, data, and visual analyses. Because single-case researchers cannot routinely rely on one specific quantification, efforts have begun to provide guidance in selecting among quantitative options (Fingerhut et al., 2020 ; Manolov & Moeyaert, 2017 ; Manolov et al., this issue-a ). These efforts will need to be extended so they include the techniques developed and illustrated by authors of this special issue, as well as methods that will be developed in the future to meet the varied needs of single-case researchers.

Computer Applications Supporting Analysis of Single-Case Designs

We also acknowledge the value of computer applications that can assist in analysis of data from single-case designs, and make use of statistical tools more accessible. The ExPRT application developed by Joel Levin and colleagues is one such program that provides rapid interpretation of single-case designs that have incorporated randomization criteria (Gafurov & Levin, 2021 ). The logic used by ExPRT is consistent with the approach to analysis of alternating treatment designs (ATDs) proposed by Manolov et al. ( this issue-b ) and is likely to prompt single-case researchers to consider incorporation of randomization options in the design of future experiments. The value of computer applications also is apparent in the Automated Nonparametric Statistical Analysis (ANSA) app offered by Kranak and Hall ( this issue ) as a tool for facilitating the interpretation of functional analysis data using alternating treatment designs. In addition, many of the effect sizes discussed by Manolov et al. ( this issue-a ) can be readily computed using computer applications, such as the Single-Case Effect Size Calculator (Pustejovsky & Swan, 2018 ) and SCDHLM (Pustejovsky et al., 2021 ). We anticipate an increasing number of computer applications for interpreting single-case data will become available as statistical strategies gain acceptance.

Falligant et al. ( this issue ) extend this theme by reviewing emerging statistical strategies for improving analysis of time series data. They summarize data analytic methods that will both benefit experimental studies and be especially useful in interpretation of clinical data (e.g., with designs that may not meet experimental requirements for control of threats to internal validity). Their message is joined by Cox et al. ( this issue ) in emphasizing the value of collecting rigorous time series data in clinical context even when experimental designs are contraindicated. The consistent message is that combining visual analysis with supplemental statistical assessment has value both for clinical decision making and advancing the science within a discipline. The improved array of statistical options, and the increasing ease with which they can be applied to time series data, make integration of visual and statistical analysis a likely standard for the future.

Implications for Designing Single-Case Research

The articles in this special issue emphasize innovative approaches to analysis of single-case research data. But the authors also offer important considerations for research designs. Two articles report procedures for identifying the role of intervention components (Cox et al., this issue ; Mason et al., this issue ). Too little emphasis has been given to the role of single-case designs to examine moderator variables, interaction effects, intervention components and sustained impact. Few interventions are effective across all population groups, all contexts, and all challenges. Effective research designs need to allow identification not only of the impact of an intervention in a specific context, but identification of conditions where the intervention is not effective. Likewise, a growing number of behavioral interventions include multiple procedures. Identifying the respective value of each procedural component and the most efficient combination of components is a worthy challenge for researchers and the creative application of single-case designs.

Single-case studies designed to examine component interactions or setting specificity may benefit from use of complex single-case designs that combine multiple baseline, alternating treatment and/or reversal elements (Kazdin, 2021 ). In other cases, analysis approaches may be helpful to both document effects and guide future studies. Cox et al. ( this issue ) offer examples for separating the independent and combined effects of behavioral interventions and medication on reduction of problem behavior for individuals with intellectual disabilities. Mason et al. ( this issue ) likewise document how statistical modeling can be used to isolate elements of stimulus control and document with greater precision the presence of stimulus overselectivity.

Research Protocols

Three articles in this issue focus on research protocols that will facilitate the inclusion of single-case research in larger meta-analyses documenting evidence-based practices. Aydin and Yassikaya ( this issue ) focus on the need for transforming graphic data into spreadsheets that can be used for statistical analysis. They report on the value of a PlotDigitizer application for extracting graphic data and provide documentation of the validity and reliability of this tool for delivering the data in a format needed for supplemental statistical analysis. Manolov et al. ( this issue-a , b ) propose procedures for both selecting and reporting the measures employed in any study to avoid measurement bias and misinterpretation, and Dowdy et al. ( this issue ) likewise encourage procedures to identify possible publication bias. These authors promote the value of research plan prepublication as a growing option that is both practical and valuable for maximizing rigorous and ethically implemented research protocols.

The major message from articles in this special issue is that single-case research designs are available and functional for advancing both our basic science and clinical technology. The efforts over the past 15 years to define professional design and analysis standards for single-case methods have been successful. But as the articles in this special issue show, single-case research methods are continuing to evolve. Innovative statistical procedures are improving the precision and credibility of single-case research analysis and posing important considerations for novel research design options. These innovations will continue to challenge prior assumptions, and open new opportunities. Each innovation will receive its own critical review, but collectively, the field is benefiting from the creative recommendations exemplified by the authors of this special issue.

Declarations

We have no known conflict of interest to disclose.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• Aydin, O., & Yassikaya, M. Y. (this issue). Validity and reliability analysis of the Plot Digitizer Software Program for data extraction from single-case graphs. Perspectives on Behavior Science . Advance online publication. 10.1007/s40614.021.00284-0
• Cox, A., Pritchard, D., Penney, H., Eiri, L., & Dyer, T. (this issue). Demonstrating an analyses of clinical data evaluating psychotropic medication reductions and the ACHIEVE! Program in adolescents with severe problem behavior. Perspectives on Behavior Science. Advance online publication. 10.1007/s40614-020-00279-3
• Dowdy, A., Hantula, D., Travers, J. C., & Tincani, M. (this issue). Meta-analytic methods to detect publication bias in behavior science research. Perspectives on Behavior Science . Advance online publication. 10.1007/s40614-021-00303-0
• Falligant, J., Kranak, M., & Hagopian, L. (this issue). Further analysis of advanced quantitative methods and supplemental interpretative aids with single-case experimental designs.
• Fingerhut, J., Marbou, K., & Moeyaert, M. (2020). Single-case metric ranking tool (Version 1.2) [Microsoft Excel tool]. 10.17605/OSF.IO/7USBJ
• Friedel, J., Cox A., Galizio, A., Swisher, M., Small, M., & Perez S. (this issue). Monte Carlo analyses for single-case experimental designs: An untapped resource for applied behavioral researchers and practitioners.
• Gafurov, B. S., & Levin, J. R. (2021, June). ExPRT ( Excel Package of Randomization Tests): Statistical analyses of single-case intervention data (Version 4.2.1) [Computer software]. https://ex-prt.weebly.com
• Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, & What Works Clearinghouse. (2020). What Works Clearinghouse Procedures Handbook (Vers. 4.1). https://ies.ed.gov/ncee/wwc/Docs/referenceresources/WWC-Procedures-Handboo v4-1508.pdf
• Kazdin AE. Single-case research designs: Methods for clinical and applied settings . 3. Oxford University Press; 2021. [ Google Scholar ]
• Kranak, M., & Hall, S. (this issue) Implementing automated nonparametric statistical analysis on functional analysis data: A guide for practitioners and researchers. Perspectives on Behavior Science . Advance online publication. 10.1007/s40614-021-00290-2
• Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single-case designs technical documentation . What Works Clearinghouse . https://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf
• Ledford J, Gast D. Single case research methodology: Applications in special education and behavioral sciences . 3. Taylor & Francis; 2018. [ Google Scholar ]
• Maggin D, Odom S. Evaluating single-case research data for systematic review: A commentary for the special issue. Journal of School Psychology. 2014; 52 (2):237–241. doi: 10.1016/j.jsp.2014.01.002. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Maggin DM, Pustejovsky JE, Johnson AH. A meta-analysis of school-based group contingency interventions for students with challenging behavior: An update. Remedial & Special Education. 2017; 38 (6):353–370. doi: 10.1177/0741932517716900. [ CrossRef ] [ Google Scholar ]
• Manolov R, Moeyaert M. Recommendations for choosing single-case data analytical techniques. Behavior Therapy. 2017; 48 (1):97–114. doi: 10.1016/j.beth.2016.04.008. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Manolov, R., Moeyaert, M., & Fingerhut, J. (this issue-a) A priori justification for effect measures in single-case experimental designs. Perspectives in Behavior Science . Advance online publication. 10.1007/s40614.021-00282-2
• Manolov, R., Tanious, R., & Onghena, P. (this issue-b). Quantitative techniques and graphical representations for interpreting results from alternating treatment design. Perspectives on Behavior Science . Advance online publication. 10.1007/s40614-021-00289-9
• Mason, L., Otero, M., & Andrews, A. (this issue). Cochran’s Q test of stimulus overselectivity within the verbal repertoire of children with autism. Perspectives on Behavior Science . Advance online publication. 10.1007/s40614-021-00315-2
• Moeyaert, M., Yang, P., & Xu, X. (this issue). The power to explain variability in intervention effectiveness in single-case research using hierarchical line modeling. Perspectives on Behavior Science . Advance online publication. 10.1007/s40614-021-00304-z
• Parsonson B, Baer D. The analysis and presentation of graphic data. In: Kratochwill TR, editor. Single subject research . Elsevier; 1978. pp. 101–166. [ Google Scholar ]
• Pustejovsky, J. E., & Swan, D. M. (2018). Single-case effect size calculator (Version 0.5.1) Web application. https://jepusto.shinyapps.io/SCD-effect-sizes/
• Pustejovsky, J. E., Hedges, L. V., & Shadish, W. R. (2014). Design-comparable effect sizes in multiple baseline designs: A general modeling framework. Journal of Educational & Behavioral Statistics, 39 (5), 368–393. 10.3102/1076998614547577
• Pustejovsky, J. E., Chen, M., & Hamilton, B. (2021). scdhlm: A web-based calculator for between-case standardized mean differences (Version 0.5.2) Web application. https://jepusto.shinyapps.io/scdhlm
• Riley-Tillman, T. C., Burns, M. K., & Kligus, S. (2020). Evaluating educational interventions: Single-case design for measuring response to intervention . Guilford Press.
• Slocum, T. A., Detrich, R., Wilczynski, S. M., Spencer, T. D., Lewis, T., & Wolfe, K. (2014). The evidence-based practice of applied behavior analysis. The Behavior Analyst, 37 , 41–56. 10.1007/s40614-014-0005-2 [ PMC free article ] [ PubMed ]
• Tate, R. L., Perdices, M., Rosenkoetter, U., Shadish, W., Vohra, S., Barlow, D. H., Horner, R., Kazdin, A., Kratochwill, T., McDonald, S., Sampson, M., Shamseer, L., Togher, L., Albin, R., Backman, C., Douglas, J., Evans, J. J., Gat, D., Manolov, R., et al. (2016). The single-case reporting guideline in Behavioural Interventions (SCRIBE) 2016 statement. Physical Therapy, 96 (7), e1–e10. 10.2522/ptj.2016.96.7.e1 [ PubMed ]

"More discoveries have arisen from intense observation than from statistics applied to large groups" - W. I. B. Beveridge

"Was ist das Allgemeine? Das einzelne Fall. Was ist das Besondere? Millionen Fälle." - Goethe

"Das Messen eines Dings ist eine grobe Handlung, die auf lebendige Körper nicht anders als höchst unvollkommen angewendet werden kann." - Goethe

"Each person is an idiom unto himself, an apparent violation of the syntax of the species." - G.W. Allport

"Given a thimbleful of [dramatic] facts we rush to make generalizations as large as a tub" - G.W. Allport

"Our typical contemporary mainstream psychology group study in which one single procedure is carried out with large numbers of cases and then treated statistically seems to fall short of any genuine psychological contribution" - Watson, 1934

Welcome to Single Case Archive

The Single Case Archive compiles clinical, systematic and experimental single case studies in the field of psychotherapy. Currently ISI published single case studies from all different psychotherapeutic orientations are being included in the database. These case studies were screened by an international group of researchers for basic information on type of study, patient, therapist and therapy. The objective of this online archive is to facilitate the study of case studies for research, clinical and teaching purposes. With an easy to use search engine, the archive allows the quick identification of relatively homogenous sets of cases in function of specific clinical or research questions. For more information on this archive, see ‘About’.

Start your search

Start browsing psychotherapy case studies in The Archive

Latest News

On Tuesday 19 March 2024, SCA board member Jochem Willemsen will give a keynote lecture at the Sigmund Freud U...

Case meta-synthesis

Metasynthesis is currently considered one of the most promising directions for qualitative...

On January 4th, 2024 SCA board member Jochem Willemsen presented a webinar on case study methodology...

Case study-prize

Three new prizes for evidence based case studies will be awarded by the Association for Psychoanalytic Ps...

Wim Trijsburg prize

The Dutch Association of Psychotherapy awarded their 2019 Wim Trijsburg prize to SCA board...

Another case study PhD

Another PhD study based on systematic case studies has been completed successfully! Ellen ...

Register to Single Case Archive

Register to SCA and get full access to The Archive and more!

Send us a case study

A quick and easy way to let us know about a case study you have seen.

Case of the Week

Case of the Week - N° 32: dou...

This week’s case points out the usefulness of the process of “double descripti...

Brought to you by

Thank you for registering to Single Case Archive. Your account needs to be activated before you can full access. You will be notified by email once your account has been activated.

Are you sure you want to remove this item?

This paper is in the following e-collection/theme issue:

Published on 28.5.2024 in Vol 26 (2024)

An Extensible Evaluation Framework Applied to Clinical Text Deidentification Natural Language Processing Tools: Multisystem and Multicorpus Study

Authors of this article:

Original Paper

• Paul M Heider 1 , PhD   ; 
• Stéphane M Meystre 2 , MD, PhD  

1 Biomedical Informatics Center, Medical University of South Carolina, Charleston, SC, United States

2 Institute of Digital Technologies for Personalised Healthcare (MeDiTech), University of Applied Sciences and Arts of Southern Switzerland, Lugano, Switzerland

Corresponding Author:

Paul M Heider, PhD

Biomedical Informatics Center

Medical University of South Carolina

22 WestEdge Street, Suite 200

Charleston, SC, 29403

United States

Phone: 1 843 792 3385

Email: [email protected]

Background: Clinical natural language processing (NLP) researchers need access to directly comparable evaluation results for applications such as text deidentification across a range of corpus types and the means to easily test new systems or corpora within the same framework. Current systems, reported metrics, and the personally identifiable information (PII) categories evaluated are not easily comparable.

Objective: This study presents an open-source and extensible end-to-end framework for comparing clinical NLP system performance across corpora even when the annotation categories do not align.

Methods: As a use case for this framework, we use 6 off-the-shelf text deidentification systems (ie, CliniDeID, deid from PhysioNet, MITRE Identity Scrubber Toolkit [MIST], NeuroNER, National Library of Medicine [NLM] Scrubber, and Philter) across 3 standard clinical text corpora for the task (2 of which are publicly available) and 1 private corpus (all in English), with annotation categories that are not directly analogous. The framework is built on shell scripts that can be extended to include new systems, corpora, and performance metrics. We present this open tool, multiple means for aligning PII categories during evaluation, and our initial timing and performance metric findings. Code for running this framework with all settings needed to run all pairs are available via Codeberg and GitHub.

Results: From this case study, we found large differences in processing speed between systems. The fastest system (ie, MIST) processed an average of 24.57 (SD 26.23) notes per second, while the slowest (ie, CliniDeID) processed an average of 1.00 notes per second. No system uniformly outperformed the others at identifying PII across corpora and categories. Instead, a rich tapestry of performance trade-offs emerged for PII categories. CliniDeID and Philter prioritize recall over precision (with an average recall 6.9 and 11.2 points higher, respectively, for partially matching spans of text matching any PII category), while the other 4 systems consistently have higher precision (with MIST’s precision scoring 20.2 points higher, NLM Scrubber scoring 4.4 points higher, NeuroNER scoring 7.2 points higher, and deid scoring 17.1 points higher). The macroaverage recall across corpora for identifying names, one of the more sensitive PII categories, included deid (48.8%) and MIST (66.9%) at the low end and NeuroNER (84.1%), NLM Scrubber (88.1%), and CliniDeID (95.9%) at the high end. A variety of metrics across categories and corpora are reported with a wider variety (eg, F 2 -score) available via the tool.

Conclusions: NLP systems in general and deidentification systems and corpora in our use case tend to be evaluated in stand-alone research articles that only include a limited set of comparators. We hold that a single evaluation pipeline across multiple systems and corpora allows for more nuanced comparisons. Our open pipeline should reduce barriers to evaluation and system advancement.

Introduction

An ironclad pillar of clinical data reuse is the proper protection of protected health information (PHI). Deidentification is the process of tagging and removing personally identifiable information (PII) to prevent incidental privacy breaches. Unfortunately, manual deidentification is an expensive and error-prone process [ 1 , 2 ], and automated deidentification remains an unsolved challenge [ 3 , 4 ]. Since the first published automated deidentification system [ 5 ], a variety of systems using a range of technologies have been released. In tandem, a series of competitions have been organized around shared corpora annotated with PII to further encourage the development of deidentification systems [ 6 - 9 ]. Researchers publishing about a new system tend to release comparative performance metrics against 1 or 2 publicly available systems using 1 or 2 corpora [ 3 , 10 - 12 ]. These stand-alone research articles that only include a limited set of comparator systems and corpora cannot always even be directly compared to create a single meta-analysis because of the differences in annotation categories evaluated and reported on or how exactly matches are aligned and scored. Replication of the process to confirm results and extension of the process to evaluate new systems or corpora have been stymied by a combination of lack of documented evaluation code, closed evaluation tools, and incompatible PII categories, among other reasons.

As such, a single evaluation pipeline across multiple systems and corpora allows for more nuanced comparisons between systems. A single pipeline allows researchers (and staff scientists responsible for evaluating systems before deployment) to consistently and reproducibly generate scores for a range of systems across a range of corpora using the exact same methods. Researchers with preferences to evaluate deidentification systems at the character level versus token level versus PII mention can run each evaluation in turn. Similarly, researchers can differentiate or collapse PII categories as desired across all systems and corpora at once to get a clear picture of how each system and corpus, respectively, treats different categories. Not all categories of PII are equally sensitive [ 3 ]. Patient names are more revealing than provider names, which are, in turn, more revealing than hospital names.

To that end, we evaluated PII extraction performance at multiple levels of granularity because these systems should not be judged on a single summary performance metric. For instance, this pipeline provides an easy means for surfacing the false negative rate or recall (also called sensitivity) for patient names as distinct from provider names (or other names). We can also easily compare performance when differences between PII categories are important to maintain, as opposed to when all PII categories are treated interchangeably.

As a case study, to help us understand the performance trade-offs of available deidentification systems that are critical to clinical data reuse and natural language processing (NLP) and to foster building larger repositories of directly comparable evaluation results, we developed a reusable and extensible pipeline for evaluating 6 off-the-shelf deidentification systems across 2 freely available corpora, 1 previously available corpus, and 1 private corpus. The systems and corpora all use English, although the pipeline is language agnostic. As none of the systems or corpora use exactly the same annotation schema, we provided mappings to allow approximately equitable performance metrics across all components. Any deidentification system programmatically runnable from the command line can be added to the set. Similarly, new corpora can be added to the evaluation process with minimal constraints on their format or annotation categories. Furthermore, the evaluation tool used in the pipeline allows for analysis at configurable levels of annotation category granularity and with multiple text annotation matching styles [ 13 , 14 ]. We focused on off-the-shelf systems for 3 primary reasons. First, off-the-shelf systems have the lowest barrier to entry. Not all potential users have the skills, capacity, or annotated corpora available to retrain a model. If a deidentification system developer considers their tool to be unusable without retraining, then that limitation should be made explicit, which brings us to the second reason. Undertaking the controlled experiments required to determine when a retrained system has met a reliable and safe performance threshold requires a test harness exactly like the one we propose here. Third, the potential variants for retraining any given system to optimize its performance for a local site constitutes its own large undertaking and falls outside the scope of this research. Furthermore, comparing the retrained variants to determine optimal performance is best organized using a test harness as described in this study.

Thus, we hold that a single evaluation pipeline across multiple systems and corpora allows for the nuanced comparisons required for safe deployment of NLP systems, in general, and deidentification systems, in particular. We document the use and extensibility of such a tool with deidentification as a use case. In summary, we found an order of magnitude difference between the fastest and slowest systems in terms of processing speed. We found that none of the 6 systems consistently outperformed the others across corpora and PII tag categories. A nuanced comparison of the top performers under slightly different conditions would be much more difficult without a cohesive framework like the one we describe.

Ethical Considerations

This study was assessed by the Medical University of South Carolina (MUSC) Institutional Review Board for Human Research (IRB) and officially considered Not Human Research. It was therefore not subject to oversight by the MUSC IRB, since it met the criteria set forth by the Code of Federal Regulations (45CFR46): (1) the data were not collected specifically for the currently proposed research project through an interaction or intervention with living individuals and (2) investigators including collaborators on the proposed research cannot readily ascertain the identity of the individuals to whom the coded private information of specimens pertains.

The other three corpora used in this study (ie, 2006 i2b2 shared task, 2014 i2b2 and University of Texas Health Science Center at Houston (i2b2/UTHealth), and 2016 Centers of Excellence in Genomic Science Neuropsychiatric Genome-Scale and RDOC Individualized Domains [CEGS N-GRID] shared task) were publicly available and already deidentified, and therefore not subject to IRB approval requirements.

PII Categories

In the United States, HIPAA (Health Insurance Portability and Accountability Act) is the primary legal mandate guiding and governing data privacy and security provisions within the health care domain [ 15 ]. Other regions of the world have enacted similar privacy laws such as the General Data Protection Regulation in the European Union [ 16 ], although they are not all specific to the health care industry, as evidenced by the General Data Protection Regulation. The HIPAA privacy rule specifies sensitive classes of PII that should be removed for the data to be considered deidentified. In this study, we focus on the subset of HIPAA’s 18 categories of PII relevant to unstructured clinical notes, as all our data sets were created in the United States under the jurisdiction of HIPAA. We have listed a curated and organized set of these categories in Figure 1 in the 4 left-most columns. The first column indicates the original HIPAA Safe Harbor category names. The following 3 columns are curated classes to help group or simplify the categories. We call these the tier 0 category (which represents the general class of PII), the tier 1 categories (which represent 7 high-level sets of categories grouped by semantic domain and common textual realizations), and tier 2 categories (which represent the finest-grained division of categories approximating the original HIPAA categories). The tier 2 categories include several practical and functional extensions of the strict HIPAA categories that have been treated as PII by deidentification researchers in the clinical domain. For instance, HIPAA considers ages >89 years to be PII but not the not younger ages, while some researchers consider any age to be PII. Additional columns in this figure represent categories annotated in deidentification corpora (columns 5-7) and categories flagged by deidentification systems (columns 8-13). Rows across the figure indicate rough equivalency of categories between tiers, corpora, or systems. Specific details of corpus and system categories are covered in the next two subsections: Deidentification Shared Tasks and Corpora and Deidentification Systems . Perfect category synchrony between corpora and annotations is impossible. We have included a sample sentence in Figure 2 , cross-annotated according to the specifics of each category schema to help highlight the local variation between corpora and deidentification systems.

Deidentification Shared Tasks and Corpora

Several shared tasks organized around the goal of evaluating deidentification systems have been organized within the clinical domain. The 2006 Informatics for Integrating Biology and the Bedside (i2b2) shared task was the first such task and focused on a small set of PII categories present in unstructured clinical notes written in English [ 6 ]. A similar shared task was organized as part of the 2012 NII Test Beds and Community for Information Access Research Medical Natural Language Processing task for fabricated but realistic medical reports written in Japanese [ 7 ]. Two more English language tasks were organized with an array of PII categories that were more representative of the full list of HIPAA categories as part of the 2014 i2b2 and UTHealth shared task [ 8 ] and 2016 CEGS N-GRID shared task [ 9 ].

We used each of the 3 English language deidentification shared task corpora for our study as they have been used as the standard reference corpora in the domain. The 2006 and 2014 corpora are publicly available with an appropriate data use agreement. The 2016 corpus was publicly available but has since been removed from circulation due to privacy concerns. The fourth corpus, also in English, is called the “Medical University of South Carolina (MUSC)” corpus in this study as it was developed at MUSC by the authors. It is not publicly available as its PII has only been annotated and not redacted or resynthesized to prevent the release of PHI.

We refer to the oldest corpus in our study as “2006” as it was used for the 2006 i2b2 shared task [ 6 ]. It consists of 889 discharge summaries from Partners HealthCare. This corpus has the most reduced set of annotation categories with 8 distinct categories: location, hospital, phone, doctor, patient, medical record, age (>89 years), and date, as shown in Figure 1 . Of note, the tier 1 category that we call “Address” maps to the 2006 category “Location.” “Medical Records” are the only recorded tier 1 “Identifiers.” “Age” is confined by the strict HIPAA notion to those >89 years, in contrast to the other 3 corpora that treat all ages as PII.

Internally, we aligned the annotation categories for the 2014 i2b2 and UTHealth [ 8 ] and the 2016 CEGS N-GRID shared tasks [ 9 ] on deidentification, which we refer to as “2014” and “2016,” respectively. In contrast with 2006, the tier 1 Address and Identifiers categories for 2014 and 2016 are split into fine-grained categories. Overall, there are 28 distinct categories, as shown in Figure 1 . The 2014 corpus consists of 1304 discharge summaries and correspondences between providers from Partners HealthCare. The 2016 corpus consists of 1000 psychiatric notes from Partners HealthCare.

The MUSC corpus consists of 728 notes split across 8 note types: consults, discharge summaries, history and physicals, nursing, pathology and cytology, patient instructions, plan of care, and progress notes. Similar to the 2014 and 2016 corpora, the MUSC corpus includes a wide range of tier 1 “Identifiers” but groups all of them except social security numbers into 1 class: “Other ID.” Due to a change in the annotation guidelines over the course of the corpus annotation, some “Street” annotations and “City” annotations have been merged into a single “StreetCity” annotation. Similarly, some “State” annotations and “Country” annotations have been merged into “StateCountry” annotations.

All 4 corpora have been divided into train and test splits. None of the systems have used the official test splits for training. Therefore, we report all performance metrics with respect to the official test split in the main body of this paper. Multimedia Appendix 1 contains results for both train and test splits and Multimedia Appendix 2 [ 2 , 11 , 17 - 32 ] contains de-identification system details. All reported timing results include both train and test splits.

Deidentification Systems

Our initial set of systems was constrained by an implementation science framing of the problem. As researchers invested in reproducible science, we need to understand the best tools for facilitating ethical data sharing, but our primary research area may not be related to deidentification. We need a resource for evaluating scalable, off-the-shelf systems that do not require additional model training or fine-tuning. Concretely, this framing restricts us to freely available systems that can be run programmatically on a local server without any requirements to annotate a local data set. A total of 6 systems meet these requirements: CliniDeID (version 1.6.1) [ 17 , 18 ], deid by Massachusetts Institute of Technology (version 1.1, now available via PhysioNet) [ 2 , 19 ], MITRE Identity Scrubber Toolkit (MIST; version 2.0.4) [ 20 - 22 ], NeuroNER (commit 3817fea on GitHub) [ 23 , 24 ], National Library of Medicine Scrubber (version 19.0403L Linux ×86 64) [ 25 - 27 ], and Philter (commit 780da99 on GitHub) [ 11 , 28 ]. All 6 of these systems run on English clinical notes, read in files from disc, write annotated files indicating identified PII mentions to disc, can be run from the command line on a Linux server, and have been run using the latest available version. System reporting will be in alphabetical order. Refer to Multimedia Appendix 2 for more details on each system’s configuration and use.

We evaluated systems in terms of both timing and binary classification performance metrics. For timing purposes (reported in the Timing Results subsection), each system processed 1 corpus at a time on the same Red Hat Linux server. This evaluation pipeline was the only program beyond routine background processes running on the machine. We used the real (ie, wall clock) time generated by the command line tool time. For performance metrics (ie, encompassing the 3 Performance Results subsections in the Results section), we used the Evaluation Tool for Unstructured Data and Extractions (ETUDE) to score each system by corpus pairing. ETUDE uses definitions from a configuration file for each of the reference and system outputs to determine how each “native” annotation category is represented in the corpus. These native categories are then mapped in the same configuration file to “scoring value” categories. Drawing from an example depicted in Figure 1 , the 2016 native categories of “Phone” and “Fax” are mapped in the 2016 configuration file to the “PhoneFax” scoring value category for tier 1 entries. The configuration files allow us to separate the logic for annotation extraction from annotation alignment matching.

ETUDE also provides several alignment matching algorithms for determining which set of annotations are considered a match between the reference and system outputs. We focused on 3 of these matching algorithms for this study: “exact,” “partial,” and “fully contained.” Exact matching requires that the character offsets of 2 annotations be the same to count as a match. Partial matching only requires that any part of the 2 annotations overlap to be considered a match. Finally, fully contained matching requires the system output annotation to at least cover the entirety of the reference annotation to count as a match. The system output annotation can include more text before or after the extent of the reference annotation but not less. The intuition relevant to deidentification is that if the system PII output annotation fully contains the reference PII annotation, then we know that no PII is leaking, as is potentially possible for a partial match of annotations.

The final evaluation feature we used in ETUDE was the ability to collapse all patterns into a single category for evaluation purposes. Thus, for some of our evaluations, we tracked both annotation and category. For other evaluations, we only tracked the annotations and ignored the differences in annotation category.

In the end, ETUDE generated counts for true positives, false positives, and false negatives. From these counts, we calculated precision, recall, and F 1 -score values (ie, the harmonic mean of precision and recall). Given the sensitive nature of deidentification, we give primacy to recall in our reporting in the main body of this paper but include all 3 values in Multimedia Appendix 1 . The F 2 -score, which gives more weight to recall than precision, can also be generated by ETUDE.

Processing Pipeline

Expanding on the work of our previous comparison of 3 deidentification systems [ 33 ], we developed a larger and more flexible pipeline for using a set of off-the-shelf deidentification systems to process a set of corpora and then scoring all system output at a range of PII category granularities, as shown in the block diagram in Figure 3 . The core of this pipeline rests on command line shell scripts with configurable custom functions for processing any given corpus with a given deidentification system. Specifically, the core shell script uses environment variables to set input and output folders for each corpus, running folders and Python environments for each deidentification system, and flags for which set of corpora and systems should run on any given instantiation. The processed output for a system and corpus combination is written to disc in a given folder, allowing for repeated evaluation loops by ETUDE [ 13 , 14 ], a freely available open-source tool developed by the first author. Each evaluation loop uses different configuration settings to highlight different tiers of categories, different alignments of categories between reference and system, and different annotation alignment algorithms. Adding a new deidentification system requires adding a few simple shell commands to preprocess files (as expected by the system), to run the system from the command line with all parameters fully specified, and to postprocess files (if they are not in a format already supported by ETUDE). A new corpus or deidentification system may require creating a new annotation schema mapping file, if the schema is not already covered by those shown in Figure 1 . As ETUDE, all shared task corpora, and all deidentification systems are already publicly available, we also released our shell scripts and R-based evaluation scripts on Codeberg and GitHub to allow for near-complete reproducibility of this study [ 34 , 35 ]. System-specific settings, in terms of explicit configuration files or command line flags and settings, are also included in these repositories.

Timing Results

The 8 timed trials for each system (ie, 4 corpora with 1 train and 1 test split) are summarized in Table 1 . We report both the seconds per note (for which a lower value is better) and notes per second (for which a higher value is better). MIST is the fastest with an average of 24.57 (SD 26.23) notes per second and is an order of magnitude faster than all systems except Scrubber, which averages 8.56 (SD 3.54) notes per second. CliniDeID is the slowest with an average of 1 (SD 0.38) note per second. Deid, NeuroNER, and Philter are all marginally faster at an average of 1.41 (SD 0.60), 1.28 (SD 0.48), and 1.36 (SD 0.64) notes per second, respectively.

a Each data point reflects a single corpus split (eg, the 2006 train split vs the 2006 test split).

Performance Results Across All Categories Based on Partial Matching

For our initial performance metric evaluation, we focused on each system’s overall ability to identify PII, regardless of the category. In the terminology of ETUDE, we collapsed evaluation across all categories. Figure 4 presents the recall, precision, and F 1 -scores for each system against each corpus using the partial match alignment (this is the most generous evaluation possible).

The u-shaped curves for CliniDeID and Philter indicate that both these systems prioritize recall over precision. In contrast, the other 4 systems consistently have higher precision scores than recall scores. CliniDeID and NeuroNER show the highest scores across all corpora. Deid and MIST have the largest variance between corpora.

Performance Results for Specific Tier 1 Categories Based on Partial Matching

As different categories of PII are differentially sensitive, we wanted to delve deeper into the specific performance of systems with respect to the 7 tier 1 categories shown in Figure 1 . For this analysis, we report the recall for partial annotation matching in Figure 5 . Precision and F 1 -score values are available in Multimedia Appendix 1 . However, the nature of the limited categories generated by deid and Philter means that precision values for these systems are not truly meaningful.

CliniDeID shows more between-corpus variance than between-category variance and overall performs most consistently well. NeuroNER performs the next best across all categories, although “Contact Information,” “Names,” and “Time” are clearly better identified than the other 4 categories. Philter shows the same facility with “Contact Information” and “Names.” Deid shows a large between-corpus variance but is generally consistent across categories. “Occupations” is not included as deid, MIST, and Scrubber do not extract that category. MIST identifies “Address” and “Contact Information” categories at a lower rate than other categories. Similarly, Scrubber identifies “Address” categories at a lower rate and does not extract any “Contact Information” annotations.

Performance Results for Names Based on Fully Contained Matching

As names are perhaps the most sensitive and unique PII category, we further analyzed the tier 1 “Names” category and the tier 2 categories of “Patient” and “Provider” for those systems that distinguish between these categories. For this analysis, we used the stricter fully contained annotation matching algorithm. Figure 6 summarizes the recall, precision, and F 1 -score values across systems and corpora.

No strong corpus-specific trends show up in these results. MIST and NeuroNER have worse recall for identifying patient names than provider names, while CliniDeID does not appear to treat them differently. MIST and NeuroNER also have worse recall than precision for patient names but, similar to CliniDeID, no obvious difference between the 2 metrics for provider names.

The gap between recall and precision is reduced for MIST and NeuroNER at the tier 1 “Names” level, which implies that some (but not all) of the tier 2 performance issues are due to patient names being incorrectly flagged as provider names. CliniDeID is consistently high for both recall and precision at tier 1. The deid system performs the worst for this evaluation, although its scores for “Names” are not so low for the partial matching evaluation in the Performance Results for Specific Tier 1 Categories subsection. This performance discrepancy indicates that deid does extract names reliably but does not extract large enough annotations to fully identify the relevant PII. Scrubber and Philter show much higher recall than precision for tier 1 “Names.” As noted previously, Philter’s precision for this evaluation is expected to be very low as the native system category relevant to “Names” includes all PII categories except dates. It is not obvious why Scrubber’s precision is low.

As expected, given the complexity of the deidentification domain and feature variability in deidentification tools, we found that that none of the 6 systems uniformly outperformed the others across corpora and PII tag categories. CliniDeID, NeuroNER, and Philter generally outperformed the older (and not actively maintained) systems, but they also each have their performance drawbacks or limitations. Our evaluation framework was helpful in identifying these strong and weak facets, which, in turn, helped us identify additional aspects to include in future releases of the framework, as discussed in the next subsection.

Principal Findings

The principal findings of this study fall into 3 major classes: the evaluation pipeline implementation, the category mapping for bridging analyses across corpora or deidentification systems, and the evaluation results for the 6 off-the-shelf systems. The evaluation pipeline implementation includes many reusable components, including the shell scripts for passing corpus notes through systems, the ETUDE configuration scripts for scoring system outputs against reference annotations, and R scripts for plotting results of these analyses. The code for replication of the workflow is available via Codeberg and GitHub public code repositories [ 34 , 35 ]. This code base can be used to increase reproducibility between studies on deidentification systems, lower the barrier to fine-tuning or retraining systems, and increase portability of evaluation across data sets and systems. The ETUDE configuration scripts provide a means for encoding and leveraging the PII category cross-mapping displayed in Figure 1 . The cross-mapping can also be used to help compare the coverage of different deidentification systems with an eye to identifying the validity of different deidentification systems for different use cases (eg, when some categories of PII are known a priori to be either absent from or very frequent in a data set). Finally, our analysis of the text deidentification systems themselves yielded mixed results. CliniDeID and Philter had the highest recall for select categories, while CliniDeID and NeuroNER had the highest recall across all categories. We also found large differences in performance when comparing partial matches for PII categories to fully contained matches. The tabular data used to generate all tables and figures in this study have been included in Multimedia Appendix 1 to allow for finer-grained evaluation of the results.

Limitations

As discussed in the Introduction section, we focused our analysis on off-the-shelf models. CliniDeID, MIST, and NeuroNER can be retrained, while National Library of Medicine Scrubber and Philter have dictionaries that can be customized to a local environment. Thus, while our framework provides a means to easily measure the average baseline performance (across multiple corpora), more work is required to ascertain the average potential performance for each system (after retraining). Fortunately, each retrained model would only require a limited pass through the evaluation framework but would not require reprocessing and reevaluating all systems against all corpora. To further facilitate this use case, future iterations of this tool will make it even easier to add new corpora, systems, and evaluation configurations. Specifically, a more robust implementation of the pipeline would leverage reproducible workflow tools such as the targets R package [ 36 ]. We have released a related implementation using the targets R package for evaluating algorithmic bias in deidentification systems [ 37 ]. Xiao et al [ 38 ] provide an alternate approach using 100 synthetic templates imitating realistic contexts for PII as it occurs in unstructured clinical notes. For researchers who do not have the infrastructure or privacy needs of data sets with PII, other reproducible pipelines and shared evaluation frameworks, such as NLP Sandbox, exist [ 39 ].

In addition, we have not included all the essential evaluations for deciding between deidentification systems and will continue to expand our result set. For instance, we did not account for system start-up time. A system that takes 1 second per note for 1 or 10 million notes has different usability than a system that takes 0.1 second per note after loading models for 10 minutes. Another evaluation complication is how to equitably compare systems with significantly different input or output features. For instance, CliniDeID resynthesizes annotations by replacing the original PII with similar surrogate values, which likely slows down the overall pipeline compared to a system such as deid that does not track PII category in a manner accessible to a standard user. Similarly, TiDE (Text DEidentification) [ 40 ] meets all the aforementioned system requirements but uses prefilled PII details to boost performance. That is, when running a note through TiDE, one can provide the system with the patient’s name, known providers, and other relevant PII that are likely to show up in a note. While this seems like a good plan at the system level, we need to develop a more complex evaluation pipeline to accommodate for the given information and to disaggregate results based on whether TiDE has correctly annotated known PII versus unknown PII (eg, a patient’s nickname or misspelled name and a relative’s name). TiDE was not evaluated and compared in this study for this reason, among others.

Finally, we relied on the mapping in Figure 1 to perform our evaluations, which, in turn, relies on a manual mapping built using experts’ knowledge. Kim et al [ 41 ] evaluated a successful automated category mapping algorithm on deidentification concepts, which could be used as an alternate means for cross-mapping categories between corpus annotations and system outputs.

Comparison With Prior Work

Previous studies have included many of the same individual components that we present as novel in this study but, to the best of our knowledge, none have included all the components together. For instance, a category mapping between the 2014 and 2016 data sets was released at the time of the canonical paper on the 2016 shared task [ 9 ]. That same year, the researchers who developed Philter [ 42 ] released a mapping between the HIPAA Safe Harbor categories, the i2b2 2014 corpus annotations, and the categories redacted in the public release of the Multiparameter Intelligent Monitoring in Intensive Care II corpus [ 43 , 44 ]. The mapping we present in Figure 1 includes more corpora and extends the mapping to the unique set of categories used by each deidentification system, which greatly extends the utility of this new mapping for the community as a whole. Furthermore, our evaluation tool (ie, ETUDE), along with the published configuration files, provides the means to easily evaluate different category groupings without reannotating the data sets.

Most prior studies do not explicitly mention details of the evaluation (eg, whether they use token counts and exact match character offsets) unless they are using the standard evaluation script released with the 2016 corpus [ 9 ]. While the widespread use and public availability of this script are a boon to reproducible science, the script’s limited options, hardcoded categories, and fixed file format are a bane. Most prior studies report the precision, recall, and F 1 -score, while a select few report the F 2 -score [ 11 , 45 ] or both [ 46 ]. We reported the F 1 -score, but ETUDE can easily be configured to report the F measure for any arbitrary β value. The relative processing speeds ( Table 1 ) for deidentification systems is also rarely reported [ 33 ]. A few prior studies have included specific annotation guidelines, including examples and edge cases, to help clarify the underlying ground truth intended by category labels [ 8 , 25 , 26 ]. Finally, our own work and many of the other researchers and developers cited in this study have relied on the same standard deidentification data sets released via the i2b2 or CEGS N-GRID shared tasks. For less common but still publicly available data sets, the developers of NeuroNER evaluated their system against the CoNLL 2003 shared task on named entity recognition [ 23 ], and Xiao et al [ 38 ] recently released a new smaller set of notes based on MIMIC-IV. We are not aware of any other study with a publicly released evaluation framework that includes all steps from initial corpus processing through plotting evaluation results.

Conclusions

We present a reusable and extensible evaluation framework applied to deidentification systems for clinical unstructured notes. We release the tool for other researchers to reduce the overhead in testing new systems and new corpora, a critical and common task within the clinical NLP community, especially for researchers who wish to ethically share data without leaking PHI.

From this initial case study, we found an order of magnitude difference in terms of processing speed between the fastest and slowest systems and that no single system out of the 6 uniformly outperformed the others across corpora and PII categories. Instead, a richer tapestry of recall and precision trade-offs emerged for different PII categories and groups of PII categories. We hold that a single evaluation pipeline across multiple systems and corpora allows for more nuanced comparisons between systems and serves as a boon to the clinical NLP community.

Acknowledgments

This publication was supported, in part, by the National Center for Advancing Translational Sciences of the National Institutes of Health (UL1 TR001450) and the SmartState Endowment for Translational Biomedical Informatics. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Data Availability

The 2006 and 2014 corpora analyzed during this study are available in the National NLP Clinical Challenges website repository [ 47 ]. The 2016 corpus analyzed during this study is not publicly available due to privacy concerns. The Medical University of South Carolina corpus is not publicly available due to privacy concerns, as the data set has not been fully redacted or resynthesized to remove personally identifiable information. Code for replication of the workflow is available via Codeberg and GitHub public code repositories [ 34 , 35 ].

Authors' Contributions

PMH and SMM made substantial contributions to the design and implementation of the research and to the analysis of the experimental results. All authors drafted the work or revised it critically. PMH drafted the initial manuscript. SMM performed critical revision of the manuscript. All authors provided final approval for the version to be published. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Conflicts of Interest

CliniDeID, one of the systems evaluated in this manuscript, was initially developed and marketed as a commercial solution but is not commercialized anymore and has been released as free and open-source software since 2023.

CSV data files used to generate figures.

Details on each of the 6 deidentification system’s release information (eg, version number), configuration, and use.

• Douglass MM, Cliffford GD, Reisner A, Long WJ, Moody GB, Mark RG. De-identification algorithm for free-text nursing notes. In: Proceedings of the Computers in Cardiology. 2005. Presented at: CIC 2005; September 25-28, 2005; Lyon, France. [ CrossRef ]
• Neamatullah I, Douglass MM, Lehman LW, Reisner A, Villarroel M, Long WJ, et al. Automated de-identification of free-text medical records. BMC Med Inform Decis Mak. Jul 24, 2008;8:32. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Steinkamp JM, Pomeranz T, Adleberg J, Kahn CEJ, Cook TS. Evaluation of automated public de-identification tools on a corpus of radiology reports. Radiol Artif Intell. Oct 14, 2020;2(6):e190137. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chen F, Bokhari SM, Cato K, Gürsoy G, Rossetti SC. Examining the generalizability of pretrained de-identification transformer models on narrative nursing notes. Appl Clin Inform. Mar 06, 2024. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sweeney L. Replacing personally-identifying information in medical records, the Scrub system. Proc AMIA Annu Fall Symp. 1996:333-337. [ FREE Full text ] [ Medline ]
• Uzuner O, Luo Y, Szolovits P. Evaluating the state-of-the-art in automatic de-identification. J Am Med Inform Assoc. 2007;14(5):550-563. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Morita M, Kano Y, Ohkuma T, Miyabe M, Aramaki E. Overview of the NTCIR-10 MedNLP task. In: Proceedings of the 10th NTCIR Conference. 2013. Presented at: NTCIR 2013; June 18-21, 2013; Tokyo, Japan.
• Stubbs A, Kotfila C, Uzuner Ö. Automated systems for the de-identification of longitudinal clinical narratives: overview of 2014 i2b2/UTHealth shared task Track 1. J Biomed Inform. Dec 2015;58 Suppl(Suppl):S11-S19. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Stubbs A, Filannino M, Uzuner Ö. De-identification of psychiatric intake records: overview of 2016 CEGS N-GRID shared tasks Track 1. J Biomed Inform. Nov 2017;75S:S4-18. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Murugadoss K, Rajasekharan A, Malin B, Agarwal V, Bade S, Anderson JR, et al. Building a best-in-class automated de-identification tool for electronic health records through ensemble learning. Patterns (N Y). May 12, 2021;2(6):100255. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Norgeot B, Muenzen K, Peterson TA, Fan X, Glicksberg BS, Schenk G, et al. Protected Health Information filter (Philter): accurately and securely de-identifying free-text clinical notes. NPJ Digit Med. Apr 14, 2020;3:57. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Liu L, Perez-Concha O, Nguyen A, Bennett V, Jorm L. De-identifying Australian hospital discharge summaries: an end-to-end framework using ensemble of deep learning models. J Biomed Inform. Nov 2022;135:104215. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Heider PM, Accetta JK, Meystre SM. ETUDE for easy and efficient NLP application evaluation. In: Proceedings of the AMIA NLP-WG Pre-Symposium. 2018. Presented at: AMIA NLP-WG Pre-Symposium 2018; November 3, 2018; San Francisco, CA.
• Heider PM, Accetta JK, Meystre SM. ETUDE: demonstrating multiple matching styles and offset calculators within a natural language processing evaluation tool. In: Proceedings of the 17th World Congress on Medical and Health Informatics. 2019. Presented at: MEDINFO 2019; August 25-30, 2019; Lyon, France.
• 45 CFR parts 160 and 164. U.S. Department of Health and Human Services. URL: https://www.hhs.gov/hipaa/for-professionals/privacy/guidance/introduction/index.html [accessed 2024-05-01]
• General data protection regulation compliance guidelines. EU General Data Protection Regulation. URL: https://gdpr.eu/ [accessed 2024-05-01]
• Meystre SM, Heider PM, Kim Y, Trice A, Underwood G. Clinical text automatic de-identification to support large scale data reuse and sharing: pilot results. In: Proceedings of the American Medical Informatics Association Annual Symposium. 2018. Presented at: AMIA 2018; November 3-7, 2018; San Francisco, CA.
• Clinacuity / CliniDeID. GitHub. URL: https://github.com/Clinacuity/CliniDeID [accessed 2024-04-08]
• De-identification software package. PhysioNet. Dec 18, 2007. URL: https://physionet.org/content/deid/1.1/ [accessed 2024-04-09]
• Aberdeen J, Bayer S, Yeniterzi R, Wellner B, Clark C, Hanauer D, et al. The MITRE identification scrubber toolkit: design, training, and assessment. Int J Med Inform. Dec 2010;79(12):849-859. [ CrossRef ] [ Medline ]
• MITRE identification scrubber toolkit files. SourceForge. URL: https://sourceforge.net/projects/mist-deid/files/ [accessed 2024-04-08]
• MIST: the MITRE identification scrubber toolkit (documentation). SourceForge. URL: https://mist-deid.sourceforge.net/docs_2_0_4/html/index.html [accessed 2024-04-08]
• Dernoncourt F, Lee JY, Szolovits P. NeuroNER: an easy-to-use program for named-entity recognition based on neural networks. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2017. Presented at: EMNLP 2017; September 9-11, 2017; Copenhagen, Denmark. [ CrossRef ]
• Franck-Dernoncourt / NeuroNER. GitHub. URL: https://github.com/Franck-Dernoncourt/NeuroNER [accessed 2024-04-08]
• Kayaalp M, Browne AC, Sagan P, McGee T, McDonald CJ. Challenges and insights in using HIPAA privacy rule for clinical text annotation. AMIA Annu Symp Proc. Nov 05, 2015;2015:707-716. [ FREE Full text ] [ Medline ]
• Kayaalp M, Sagan P, Browne AC, McDonald CJ. Guidelines for annotating personal identifiers in the clinical text repository of the National Institutes of Health (version 6/28/2016). Lister Hill National Center for Biomedical Communications, U.S. National Library of Medicine, National Institutes of Health. 2016. URL: https://lhncbc.nlm.nih.gov/scrubber/annotation.html [accessed 2022-12-23]
• NLM-scrubber downloads. National Institutes of Health National Library of Medicine. URL: https://lhncbc.nlm.nih.gov/scrubber/download.html [accessed 2024-04-08]
• BCHSI/philter-ucsf. GitHub. URL: https://github.com/BCHSI/philter-ucsf [accessed 2024-04-08]
• Carrell D, Malin B, Aberdeen J, Bayer S, Clark C, Wellner B, et al. Hiding in plain sight: use of realistic surrogates to reduce exposure of protected health information in clinical text. J Am Med Inform Assoc. 2013;20(2):342-348. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pennington J, Socher R, Manning C. GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014. Presented at: EMNLP 2014; October 25-29, 2014; Doha, Qatar. [ CrossRef ]
• Stenetorp P, Pyysalo S, Topić G, Ohta T, Ananiadou S, Tsujii J. brat: a web-based tool for NLP-assisted text annotation. In: Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics. 2012. Presented at: EACL 2012; April 23-27, 2012; Avignon, France.
• MUSC-TBIC/corpus-utils. GitHub. URL: https://github.com/MUSC-TBIC/corpus-utils [accessed 2020-07-27]
• Heider PM, Obeid JS, Meystre SM. A comparative analysis of speed and accuracy for three off-the-shelf de-identification tools. AMIA Jt Summits Transl Sci Proc. May 30, 2020;2020:241-250. [ FREE Full text ] [ Medline ]
• Heider PM. HeiderLab/article-addenda. Codeberg. URL: https://codeberg.org/HeiderLab/article-addenda [accessed 2023-10-04]
• MUSC-TBIC/article-addenda. GitHub. URL: https://github.com/MUSC-TBIC/article-addenda [accessed 2023-10-04]
• Landau WM. The targets R package: a dynamic Make-like function-oriented pipeline toolkit for reproducibility and high-performance computing. J Open Source Softw. Jan 15, 2021;6(57):2959. [ CrossRef ]
• Heider PM. Algorithmic bias in de-identification tools. In: Proceedings of the IEEE 11th International Conference on Healthcare Informatics. 2023. Presented at: ICHI 2023; June 26-29, 2023; Houston, TX. [ CrossRef ]
• Xiao Y, Lim S, Pollard TJ, Ghassemi M. In the name of fairness: assessing the bias in clinical record de-identification. In: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 2023. Presented at: FAccT '23; June 12-15, 2023; Chicago, IL. [ CrossRef ]
• NLP sandbox. GitHub. URL: https://github.com/nlpsandbox [accessed 2020-07-27]
• TiDE text deid. STAnford Medicine Research data Repository. URL: https://starr.stanford.edu/methods/tide-text-deid [accessed 2023-10-02]
• Kim Y, Heider PM, Meystre SM. Automated category alignment applied to different de-identification annotation schemata. In: Proceedings of the AMIA 2021 Virtual Informatics Summit. 2021. Presented at: AMIA 2021; March 22-25, 2021; Online.
• Dernoncourt F, Lee JY, Uzuner O, Szolovits P. De-identification of patient notes with recurrent neural networks. J Am Med Inform Assoc. May 01, 2017;24(3):596-606. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Goldberger AL, Amaral LA, Glass L, Hausdorff JM, Ivanov PC, Mark RG, et al. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation. Jun 13, 2000;101(23):E215-E220. [ CrossRef ] [ Medline ]
• Saeed M, Villarroel M, Reisner AT, Clifford G, Lehman LW, Moody G, et al. Multiparameter intelligent monitoring in intensive care II: a public-access intensive care unit database. Crit Care Med. May 2011;39(5):952-960. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ferrández O, South BR, Shen S, Friedlin FJ, Samore MH, Meystre SM. Evaluating current automatic de-identification methods with Veteran's health administration clinical documents. BMC Med Res Methodol. Jul 27, 2012;12:109. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ferrández O, South BR, Shen S, Friedlin FJ, Samore MH, Meystre SM. BoB, a best-of-breed automated text de-identification system for VHA clinical documents. J Am Med Inform Assoc. Jan 01, 2013;20(1):77-83. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Data sets. National NLP Clinical Challenges (n2c2). URL: https://n2c2.dbmi.hms.harvard.edu/data-sets [accessed 2022-03-08]

Abbreviations

Edited by G Tsafnat; submitted 20.12.23; peer-reviewed by C Taridzo, L Radhakrishnan; comments to author 07.03.24; revised version received 11.04.24; accepted 13.04.24; published 28.05.24.

©Paul M Heider, Stéphane M Meystre. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 28.05.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Customer Reviews

Why choose us

Don’t Drown In Assignments — Hire an Essay Writer to Help!

Does a pile of essay writing prevent you from sleeping at night? We know the feeling. But we also know how to help it. Whenever you have an assignment coming your way, shoot our 24/7 support a message or fill in the quick 10-minute request form on our site. Our essay help exists to make your life stress-free, while still having a 4.0 GPA. When you pay for an essay, you pay not only for high-quality work but for a smooth experience. Our bonuses are what keep our clients coming back for more. Receive a free originality report, have direct contact with your writer, have our 24/7 support team by your side, and have the privilege to receive as many revisions as required.

We have the ultimate collection of writers in our portfolio, so once you ask us to write my essay, we can find you the most fitting one according to your topic. The perks of having highly qualified writers don't end there. We are able to help each and every client coming our way as we have specialists to take on the easiest and the hardest tasks. Whatever essay writing you need help with, let it be astronomy or geography, we got you covered! If you have a hard time selecting your writer, contact our friendly 24/7 support team and they will find you the most suitable one. Once your writer begins the work, we strongly suggest you stay in touch with them through a personal encrypted chat to make any clarifications or edits on the go. Even if miscommunications do happen and you aren't satisfied with the initial work, we can make endless revisions and present you with more drafts ASAP. Payment-free of course. Another reason why working with us will benefit your academic growth is our extensive set of bonuses. We offer a free originality report, title, and reference page, along with the previously mentioned limitless revisions.

To describe something in great detail to the readers, the writers will do my essay to appeal to the senses of the readers and try their best to give them a live experience of the given subject.

Customer Reviews

Finished Papers

• On-schedule delivery
• Compliance with the provided brief
• Chat with your helper
• Ongoing 24/7 support
• Real-time alerts
• Free revisions
• Free quality check
• Free title page
• Free bibliography
• Any citation style

You are free to order a full plagiarism PDF report while placing the order or afterwards by contacting our Customer Support Team.

Verification link has been re- sent to your email. Click the link to activate your account.

Fill up the form and submit

On the order page of our write essay service website, you will be given a form that includes requirements. You will have to fill it up and submit.

Student Feedback on Our Paper Writers

We use cookies to make your user experience better. By staying on our website, you fully accept it. Learn more .

On the History of Single-Case Methodology: A Data-Based Analysis

• Original Paper
• Published: 29 April 2022
• Volume 33 , pages 163–183, ( 2024 )

Cite this article

• Collin Shepley   ORCID: orcid.org/0000-0003-1967-9714 1 ,
• Sally B. Shepley 1 &
• Amy D. Spriggs 1  

430 Accesses

Explore all metrics

To provide context surrounding the history of single-case research and to act as a benchmark for which future changes across the fields and disciplines that use single-case methods may be compared, we conducted this study to serve as an update and extension on the trends and prevalence of single-case research in the peer-reviewed literature. Our analytical sample was derived from 20 peer-reviewed journals over a 40-year timespan. Results indicate that since 1978, approximately one in six publications within our examined body of research employed a single-case research design. Exploratory moderator analyses revealed that a journal’s focus on behavior analytic research, and a journal’s impact factor moderated the prevalence of single-case publications within a journal. We discuss our findings in relation to prior studies examining the prevalence and trends of single-case research and future directions for single-case researchers to improve the quality, value, and understanding of single-case methodology ( https://osf.io/fns3m/ ).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach

What is qualitative in qualitative research.

Criteria for Good Qualitative Research: A Comprehensive Review

Barton, E. E., Ledford, J. R., Lane, J. D., Decker, J., Germansky, S. E., Hemmeter, M. L., & Kaiser, A. (2016). The iterative use of single case research designs to advance the science of EI/ECSE. Topics in Early Childhood Special Education, 36 (1), 4–14. https://doi.org/10.1177/0271121416630011

Article   Google Scholar  

Bishop, D. (2015, February 24). Editors behaving badly? . BishopBlog . http://deevybee.blogspot.com/2015/02/editors-behaving-badly.html.

Burns, M. K., Klingbeil, D. A., Ysseldyke, J. E., & Petersen-Brown, S. (2012). Trends in methodological rigor in intervention research published in school psychology journals. Psychology in the Schools, 49 (9), 843–851. https://doi.org/10.1002/pits.21637

Carter, E. W., Brock, M. E., Bottema-Beutel, K., Bartholomew, A., Boehm, T. L., & Cease-Cook, J. (2013). Methodological trends in secondary education and transition research: Looking backward and moving forward. Career Development and Transition for Exceptional Individuals, 36 (1), 15–24. https://doi.org/10.1177/2165143413475659

Chazin, K. T., Ledford, J. R., Barton, E. E., & Osborne, K. C. (2018). The effects of antecedent exercise on engagement during large group activities for young children. Remedial and Special Education, 39 (3), 158–170.

DiNoia, J., & Tripodi, T. (2008). Single-case design for clinical social workers (2nd ed.). NASW Press.

Google Scholar  

Gage, N. A., Lewis, T. J., & Adamson, R. M. (2010). An examination of 35 years of behavioral disorders: What, how, and who has been published. Behavioral Disorders, 35 (4), 280–293. https://doi.org/10.1177/019874291003500402

Geuke, G. G., Maric, M., Miočević, M., Wolters, L. H., & de Haan, E. (2019). Testing mediators of youth intervention outcomes using single-case experimental designs. New Directions for Child and Adolescent Development, 2019 (167), 39–64.

Article   PubMed   PubMed Central   Google Scholar  

Hammond, D., & Gast, D. L. (2010). Descriptive analysis of single subject research designs: 1983–2007. Education and Training in Autism and Developmental Disabilities, 45 (2), 187–202.

Jamshidi, L., Heyvaert, M., Declercq, L., Fernandez-Castilla, B., Ferron, J. M., Moeyaert, M., & Van den Noortgate, W. (2018). Methodological quality of meta-analyses of single-case experimental studies. Research in Developmental Disabilities, 79 (1), 97–115. https://doi.org/10.1016/j.ridd.2017.12.016

Article   PubMed   Google Scholar  

Kratochwill, T. R., & Williams, B. L. (1988). Perspectives on pitfalls and hassles in single-subject research. Journal of the Association for Persons with Severe Handicaps, 13 (3), 147–154. https://doi.org/10.1177/154079698801300304

Ledford, J. R., & Gast, D. L. (Eds.). (2018). Single case research methodology: Applications in special education and behavioral sciences (3rd ed.). Routledge.

Ledford, J. R., & Wolery, M. (2013). Procedural fidelity: An analysis of measurement and reporting practices. Journal of Early Intervention, 35 (2), 173–193. https://doi.org/10.1177/1053815113515908

Ledford, J. R., Wolery, M., Meeker, K. A., & Wehby, J. H. (2012). The effects of graphing a second observer’s data on judgments of functional relations in A–B–A–B graphs. Journal of Behavioral Education, 21 (4), 350–364.

Lortie-Forgues, H., & Inglis, M. (2019). Rigorous large-scale educational RCTs are often uninformative: Should we be concerned? Educational Researcher, 48 (3), 158–166.

Macgowan, M. J., & Wong, S. E. (2014). Single-case designs in group work: Past applications, future directions. Group Dynamics: Theory, Research, and Practice, 18 (2), 138.

Mastropieri, M. A., Berkeley, S., McDuffie, K. A., Graff, H., Marshak, L., Conners, N. A., & Scruggs, T. E. (2009). What is published in the field of special education? An analysis of 11 prominent journals. Exceptional Children, 76 (1), 95–109.

Natesan Batley, P., Contractor, A. A., & Caldas, S. V. (2020). Bayesian time-series models in single case experimental designs: A tutorial for trauma researchers. Journal of Traumatic Stress, 33 (6), 1144–1153.

Neuroskeptic. (2016, July 10). Can psychologists learn more by studying fewer people? . Discover. https://www.discovermagazine.com/mind/can-psychologists-learn-more-by-studying-fewer-people.

Normand, M. P. (2016). Less is more: Psychologists can learn more by studying fewer people. Frontiers in Psychology, 7 , 934. https://doi.org/10.3389/fpsyg.2016.00934

Plavnick, J. B., & Ferreri, S. J. (2013). Single-case experimental designs in educational research: A methodology for causal analyses in teaching and learning. Educational Psychology Review, 25 (4), 549–569. https://doi.org/10.1007/s10648-013-9230-6

Radley, K. C., Dart, E. H., Fischer, A. J., & Collins, T. A. (2020). Publication trends for single-case methodology in school psychology: A systematic review. Psychology in the Schools, 57 (5), 683–698.

Shepley, C., Ault, M. J., Ortiz, K., Vogler, J. C., & McGee, M. (2020). An exploratory analysis of quality indicators in adapted alternating treatments designs. Topics in Early Childhood Special Education, 39 (4), 226–237.

Shepley, C., Zimmerman, K. N., & Ayres, K. M. (2021). Estimating the impact of design standards on the rigor of a subset of single-case research. Journal of Disability Policy Studies, 32 (2), 108–118. https://doi.org/10.1177/1044207320934048

Tate, R. L., & Perdices, M. (2019). Single-case experimental designs for clinical research and neurorehabilitation settings . Routledge.

Book   Google Scholar  

Voeltz, L. M., & Evans, I. M. (1983). Educational validity: Procedures to evaluate outcomes in programs for severely handicapped learners. Journal of the Association for the Severely Handicapped, 8 (1), 3–15. https://doi.org/10.1177/154079698300800101

What Works Clearinghouse. (2020). What Works Clearinghouse Procedures and Standards Handbook, Version 4.0 .

Wolery, M. (2013). A commentary: Single-case design technical document of the what works clearinghouse. Remedial and Special Education, 34 (1), 39–43. https://doi.org/10.1177/0741932512468038

Wolfe, K., Barton, E. E., & Meadan, H. (2019). Systematic protocols for the visual analysis of single-case research data. Behavior Analysis in Practice, 12 (2), 491–502.

Download references

Acknowledgements

We would like to thank Maddie Calzi, Rachel Jacob, Hannah Keene, Katelyn Nicklow, Kaitlin O’Neil, Meredith Davis, Bryn Handley, and Hannah Dollinger for their assistance with this project.

No funding to report.

Author information

Authors and affiliations.

Department of Early Childhood, Special Education and Counselor Education, University of Kentucky, 229 Taylor Education Building, Lexington, KY, 40506, USA

Collin Shepley, Sally B. Shepley & Amy D. Spriggs

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Collin Shepley .

Ethics declarations

Conflict of interest.

The authors of this manuscript are primarily single-case researchers; therefore, interpretations of findings may reflect an implicit bias supporting single-case methodology.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The data analyzed in the manuscript come a larger in-progress study conducted by the authors; the pre-registration for that study may be found at https://osf.io/vkpz4 .

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 227 KB)

Rights and permissions.

Reprints and permissions

About this article

Shepley, C., Shepley, S.B. & Spriggs, A.D. On the History of Single-Case Methodology: A Data-Based Analysis. J Behav Educ 33 , 163–183 (2024). https://doi.org/10.1007/s10864-022-09477-2

Download citation

Accepted : 13 April 2022

Published : 29 April 2022

Issue Date : March 2024

DOI : https://doi.org/10.1007/s10864-022-09477-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Single-case design
• Publication trends
• Research methods
• Impact factors
• Find a journal
• Publish with us
• Track your research

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• For authors
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Online First
• The pEGASUS-HPC stent system for intracranial arterial stenosis: a single-center case series
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0003-2725-7685 Daniel Pielenz 1 ,
• Joachim Klisch 1 , 2 ,
• http://orcid.org/0000-0002-2677-8780 David Fiorella 3 ,
• Matthias Gawlitza 4 ,
• Andreas Steinbrecher 5 ,
• Elke Leinisch 5 ,
• Elmar Lobsien 5 ,
• Karl-Titus Hoffmann 4 ,
• http://orcid.org/0000-0002-3936-0981 Donald Lobsien 1
• 1 Department of Neuroradiology , Helios Hospital Erfurt , Erfurt , Thüringen , Germany
• 2 Department of Neuroradiology , Helios Vogtland Hospital Plauen , Plauen , Sachsen , Germany
• 3 Department of Neurosurgery , Stony Brook University , Stony Brook , New York , USA
• 4 University Hospital Leipzig Institute for Neuroradiology , Leipzig , Sachsen , Germany
• 5 Department of Neurology , Helios Hospital Erfurt , Erfurt , Thüringen , Germany
• Correspondence to Daniel Pielenz, Neuroradiology, Helios Hospital Erfurt, Erfurt, Thüringen, Germany; daniel.pielenz{at}helios-gesundheit.de

Background Intracranial arterial stenting is a technique for the treatment of symptomatic stenosis. In this single-center retrospective case series we evaluated a novel low profile laser-cut stent with an antithrombogenic hydrophilic polymer coating (pEGASUS-HPC, Phenox GmbH, Bochum, Germany) for the treatment of intracranial stenosis in the setting of acute ischemic stroke and elective cases.

Methods All patients treated with pEGASUS-HPC for one or more intracranial arterial stenoses at our institution were retrospectively included. Clinical, imaging and procedural parameters as well as clinical and imaging follow-up data were collected.

Results We performed 43 interventions in 41 patients with 42 stenoses in our neurovascular center between August 2021 and February 2024. Twenty-one patients (51.2%) were female and the mean±SD age was 71±10.8 years. Thirty-seven (86.1%) procedures were performed in the setting of endovascular acute ischemic stroke treatment. Technical or procedural complications occurred in seven patients (16.3%), six in the thrombectomy group and one in the elective group. One stent-related hemorrhagic complication (subarachnoid hemorrhage) occurred in emergency cases and symptomatic intracerebral hemorrhage occurred in one patient treated in an elective setting. Overall stenosis reduction following pEGASUS-HPC stent implantation was 53.0±18.0%. On follow-up imaging, which was available for 16 patients (37.2%) after an average of 32±58.6 days, 62.5% of the stents were patent.

Conclusion Our single-center case series demonstrates the feasibility of using the pEGASUS-HPC stent system, especially in emergency situations when thrombectomy fails.

• angiography
• atherosclerosis

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

https://doi.org/10.1136/jnis-2024-021737

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

Intracranial arterial stenting for the treatment of symptomatic intracranial stenoses is controversial but emerges as a valuable option in the setting of acute stroke thrombectomy. The pEGASUS-HPC is a novel laser-cut self-expandable stent with antithrombogenic polymer coating, potentially reducing the risk of local thrombotic complications.

WHAT THIS STUDY ADDS

In our study, the novel pEGASUS-HPC stent showed a high level of procedural effectiveness.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

Bail-out stenting using this stent in failed thrombectomy cases is a valuable treatment option as well as treatment of intracranial stenosis with recurrent symptoms under best medical treatment.

Introduction

Intracranial arterial stenosis poses a significant challenge in the treatment of acute ischemic stroke. While stenting is mostly used as a secondary therapy in elective cases after exhausting best medical treatment options, it remains a subject of ongoing debate as a bail-out therapy in acute stroke recanalization. Emerging data suggest potential benefits for ‘rescue’ or ‘bail-out stenting’ in certain acute cases. 1–4 The pEGASUS-HPC stent (Phenox GmbH, Bochum, Germany) is a novel low-profile, self-expanding laser-cut stent designed for intracranial aneurysm and stenosis treatment. It is available with the HPC coating that reduces thrombogenicity. 5

We present a retrospective single-center series documenting our initial experience with the pEGASUS-HPC stent system for the treatment of intracranial arterial stenosis in both emergency and elective settings.

Materials and methods

Study design.

The database of our neurovascular center was retrospectively screened and all patients and procedures fulfilling the following criteria were included in the study:

Patients (age ≥18 years) treated with the pEGASUS-HPC stent for intracranial arterial stenosis.

Conditions potentially leading to a luminal stenosis (ie, atherosclerotic, dissection, wall-adherent thrombus as bail-out in unsuccessful thrombectomy, unknown origin) of any location, grade or clinical presentation.

Patients undergoing an initial treatment or a retreatment of the index lesion.

Patients with or without acute demarcated ischemic infarction on pre-interventional imaging.

Balloon angioplasty prior to or after delivering the stent was optional.

All types of balloons for pre- or post-dilatation were allowed.

All types of access material (ie, guiding catheters, aspiration catheters, microcatheters and microguidewires).

All types of antiplatelet protocols before, during and after the procedure.

We defined the following exclusion criteria:

Acute intracranial hemorrhage.

Additional stenting of stenosis adjacent to treated aneurysm.

Treatment of cerebral vasospasms following aneurysmal subarachnoid hemorrhage.

The patients were categorized as elective cases if the stenosis was known and the treatment was planned and prepared several days or weeks before the treatment. These were all patients with recurrent neurological symptoms despite best medical treatment. Emergency cases were all patients with acute ischemic stroke with large vessel occlusion and a luminal narrowing discovered after mechanical thrombectomy.

Data collection

We documented demographic data for each patient, stenosis grade according to the WASID method, 6 location of the stenosis, and pre-, peri-, and post-interventional antiplatelet therapy. Additionally, we recorded peri-procedural, post-procedural, and delayed complications, as well as clinical and anatomical imaging outcomes.

The stenosis diameter in relation to the adjacent normal vessel diameter was assessed both before and after implantation of the pEGASUS-HPC stent, including any additional balloon dilatation if performed. Successful treatment was defined as a reduction of the initial stenosis by ≥50%.

A thorough evaluation of all procedures entailed documenting and analyzing all materials used, procedural complications and their management, as well as the type of anesthesia administered. Subjective impressions of the operators were also recorded, categorized as ‘pushing through the microcatheter’ (satisfactory/unsatisfactory) and ‘positioning of the stent’ (satisfactory/unsatisfactory).

Revascularization rates of the stenosis were assessed based on the percentage of residual stenosis and stenosis improvement at the conclusion of the procedure and during follow-up, if available. All available follow-up assessments, including digital subtraction angiography (DSA), magnetic resonance angiography (MRA) and CT angiography (CTA), were included. Early follow-ups were conducted based on individual clinical circumstances post-treatment.

Stent patency was graded using a 4-stage scale:

No stenosis.

≤50% stenosis.

>50% stenosis.

Occlusion, assessed at the conclusion of the procedure and during all available follow-ups.

Statistical analysis

All data were collected anonymously in an Excel spreadsheet. Image evaluations were conducted at our neurovascular center using our routine PACS (Picture Archiving and Communication System).

Demographic data, baseline and follow-up data as well as procedural characteristics were summarized and reported as mean±SD and range for continuous variables. Categorical data were summarized as numbers and percentages.

Basic demographics

From August 2021 to November 2023 we identified 41 patients in whom we treated 42 stenoses. Among them, 21 patients (51.2%) were female and the mean±SD age was 71±10.8 years.

Five patients underwent elective treatment, accounting for a total of six interventions (13.9%). The remaining 37 patients (86.1%) received treatment in the context of mechanical thrombectomy. During the same period we performed 826 thrombectomies resulting in intracranial bail-out stenting in emergency cases using the pEGASUS-HPC in 4.5% of all thrombectomies. This is in line with recent trends in endovascular therapies. 7 8

Before stent implantation the stenosis grade in relation to the normal adjacent vessel diameter averaged 87.8±9.9%. Following stent implantation, the residual stenosis measured 34.5±15.8% in emergency cases and 29.2±18.1% in elective cases ( table 1 ).

• View inline

Basic demographics, clinical and angiographic characteristics and outcome

The majority of stenoses were located in the anterior circulation (74.4%), with the posterior circulation accounting for the remaining 25.6%. Predominantly, stenoses were found in the middle cerebral artery (M1: 46.5%; M2: 9.3%), followed by the internal carotid artery (ICA, C2: 7.0%; C3 and C5 each 0%; C4 and C6 each 4.7%, C7: 16.3%, according to Bouthillier classification 9 ). Stenoses were also observed in the vertebral artery (14.0%) and basilar artery (9.3%). In six emergency cases (14.0%) dissection was diagnosed as the underlying cause, with an additional five patients (3 emergency, 2 elective) presenting with suspected dissection.

All patients underwent general anesthesia. Intra-arterial blood pressure measurements were performed primarily through femoral access sheaths during the intervention and via peripheral access (mostly radial artery) post-interventionally. The systolic arterial pressure threshold was set at 140 mmHg and efforts were made to avoid blood pressure spikes.

Endovascular treatment

For the treatment of 42 stenoses a total of 50 pEGASUS-HPC stents were used: 46 (92.0%) were implanted and four (8.0% of all stents) were discarded. To mitigate the risk of secondary dislocation and maximize the utilization of the stent’s radial force, pEGASUS-HPC stents were intentionally oversized, with a mean oversizing of 26.1±37.3% compared with the parent vessel diameter.

In three patients two stents were implanted during the same session: one instance involved a slight dislocation of the first stent necessitating the placement of an additional stent while, in another case, two stents were used to cover a long stenotic segment. Furthermore, one patient with bilateral high-grade V4 stenosis was successfully treated in two separate interventions.

In five cases (13.2%) a transition from a balloon-expandable stent to the pEGASUS-HPC stent occurred due to the inability to deliver the stent to the target lesion. Balloon angioplasty was performed in 41 interventions (95.3%), with the NeuroSpeed balloon (Acandis) being the most frequently used (55.8%). For stent deployment, the NeuroSpeed balloon catheter was employed in 51.2% of interventions followed by Excelsior SL10 (Stryker, Fremont, California, USA) in 34.9%. In all four cases where the pEGASUS-HPC stent was discarded, a NeuroSpeed balloon was used as the delivery microcatheter. This was due to dislocation caused by friction in three interventions, while insurmountable friction prevented the delivery of the stent in one patient. In these instances, the Excelsior SL10 microcatheter was used for the successful deployment of the second pEGASUS-HPC stent. In dilatation, we deliberately chose balloons that were undersized compared with the estimated vessel diameter in 97.5% of cases to mitigate the risk of high-grade dissection and vessel perforation.

For the first balloon the undersizing averaged 23.9±22.1%, with diameters ranging from 1.5 to 4.0 mm and lengths from 8 to 20 mm. If a second balloon was employed, the mean undersizing was 16.2±22.7%, using the same lengths. Pre-dilatation alone was used in 28 cases (58.1%), while pre- and post-dilatation were conducted in 13 cases (30.2%). Post-dilatation alone was performed in two cases (4.7%). In two cases (4.7%) angioplasty was not used as these were suspected dissection cases.

Figures 1 and 2 show examples of pre- and post-dilatation and pre-dilatation only, respectively. All angioplasty balloons used were uncoated and none were drug-eluting. Thrombectomy before stent implantation was carried out in 37 cases (86.1%), typically using the SAVE technique. 10 The mean number of thrombectomy maneuvers was 3.6±3.4, with a maximum of 16 thrombectomy maneuvers recorded in one intervention. The criteria for implementing bail-out stenting in cases of unsuccessful thrombectomy were the identification of high-grade stenosis in the previously occluded vessel, or the presence of a stenosis exhibiting low and/or turbulent flow, or a tendency toward early reocclusion. Regarding the infarct core, there was no specific quantitative threshold to avoid stenting. Systemic thrombolysis before intervention was administered in 12 patients, constituting 32.4% of patients with acute stroke, and the mean dose administered was 58.4±17.8 mg.

• Download figure
• Open in new tab
• Download powerpoint

Patient with acute ischemic stroke postoperatively (tumor nephrectomy). (A) Initial complete occlusion of left M1. Partial leptomeningeal collateralization from left anterior cerebral artery (ACA). (B) Proximal M1 stenosis after two thrombectomies using the SAVE technique. Possible wall-adherent plaque (note the fuzzy margin of the distal stenosis). (C) Pre-dilatation using a NeuroSpeed balloon catheter (2.0×8.0 mm) after 500 mg aspirin IV and 2500 IE heparin IV. (D) Minor residual stenosis following pEGASUS-HPC stent implantation (3.5×15 mm). (E) Post-dilatation using same NeuroSpeed balloon catheter (2.0×8.0 mm). (F) Final run (modified Treatment in Cerebral Ischemia grade 2c). ACA collaterals are diminished due to normal middle cerebral artery flow.

Patient with acute ischemic stroke. (A) Initial occlusion of distal right M1 despite Actilyse IV, aplasia of right A1. (B) High-grade M2 stenosis (superior trunk) after three thrombectomies using the SAVE technique. (C) Pre-dilatation using a Maverick balloon catheter (1.5×20 mm). (D) Positioning of pEGASUS-HPC stent (3.5×15 mm) from superior trunk of M2 to distal M1 following 500 mg aspirin IV. (E) Control run, no post-dilatation necessary. (F) Final run, complete revascularization (modified Treatment in Cerebral Ischemia grade 3).

Antiplatelet protocols

In all elective cases, patients received 100 mg aspirin orally and 75 mg clopidogrel orally before the procedure, along with 250 mg aspirin and 5000 IU heparin IV during the procedure. After the procedure, patients were prescribed 100 mg aspirin orally and 75 mg clopidogrel orally for 6–12 months. Antiplatelet function tests using the multiplate test (Roche Diagnostics, Mannheim, Germany) were conducted for all patients on the day before the procedure, with all patients having a satisfactory response to aspirin and clopidogrel.

In emergency interventions the antiplatelet protocol varied, largely due to pre-existing medications administered for other conditions. Details concerning the individual antiplatelet protocols used are summarized in table 2 .

Antiplatelet/anticoagulant protocols and thrombolytica

Technical outcome

In the final DSA run, all implanted pEGASUS-HPC stents were found to be patent. The mean±SD initial stenosis overall was 87.8±9.9% and, after stenting, the residual stenosis was 34.8±15.6% resulting in a mean stenosis improvement of 53.0±18.0%. In emergency interventions the stenosis improvement was 52.6±17.3%, while in elective interventions it was 58.1±21.7%.

At the first follow-up, which was available for 16 patients (37.2%) after a mean of 32.1 days, imaging modalities included 43.8% DSA, 31.2% CTA, and 25.0% MRA. Among these, no in-stent stenosis was observed in seven patients (43.7% of available follow-ups), moderate restenosis was found in three patients (18.7%), and severe stenosis in two patients (12.5%). Additionally, one stent was occluded (6.2%). In one case residual stenosis showed further improvement at the first follow-up and, in another case, a stent was impossible to evaluate by CTA due to beam hardening artifacts.

Clinical outcome

The modified Rankin Scale (mRS) was used to assess all patients at hospital discharge. Overall, the mRS score improved slightly from pre-intervention (mean 4.2) to post-intervention (mean 3.9), with a slightly more pronounced difference observed in emergency patients (pre-intervention mean 4.5, post-intervention mean 4.1). In elective patients, the mean mRS score of 2.7 showed no change. Improvement in the mRS score was recorded in 19 patients, with a good clinical outcome (mRS 0–2) achieved in nine patients overall (20.9%), including seven emergency interventions (19.9%).

Of the 37 emergency patients, 11 (25.6%) died with four developing sepsis, one experiencing massive abdominal bleeding and hemorrhagic shock, one suffering cardiogenic shock, and five encountering basilar thrombosis with vertebrobasilar infarction. No death was directly related to stent implantation. One death (due to hemorrhagic shock) was almost certainly caused by femoral access site bleeding after intra-/post-procedural tirofiban infusion.

Intra-procedural technical and clinical complications

Revascularization of the occluded vessel or improvement of stenosis was achieved in all cases with a reduction of the initial stenosis by ≥50%. No intra-procedural hemorrhagic complications or directly pEGASUS-related complications occurred in elective cases. However, in one patient (16.7%) dissection distal to the stenosis occurred after pre-dilatation. In response, a flow diverter (p64 MW-HPC 350-21, Phenox) was implanted and the remaining intervention proceeded without further incident.

In emergency interventions, seven cases (18.9%) experienced major intra-procedural complications. In two cases, emboli occurred distally to the stent. In the first case it was managed with microwire fragmentation and continued tirofiban infusion, achieving an overall mTICI 2c recanalization. Unfortunately, the patient’s neurological deficit did not improve post-intervention. Additionally, the patient suffered from severe urinary tract infection and died 4 days after the intervention due to combined severe acute ischemic infarction (National Institutes of Health Stroke Scale (NIHSS) score of 19 pre-intervention) and sepsis. In the second case an embolism distal to the stenosis occurred immediately after pre-dilatation and was treated with aspiration thrombectomy and tirofiban infusion, resulting in partial recanalization. However, the patient died the following day.

In a patient in their 80s with acute left-sided distal ICA occlusion, an acute in-stent thrombosis occurred in the M1 segment. Additionally, the patient suffered from a high-grade stenosis in the distal cervical segment of the ipsilateral ICA. Both conditions were addressed in the same intervention with a balloon-expandable stent (Multilink Vision 4×15, Boston Scientific) deployed to treat the high-grade stenosis. The in-stent thrombosis of the pEGASUS-HPC stent was managed with tirofiban and angioplasty, resulting in patency of the stent and a final mTICI 2c result. The patient’s condition slightly improved from a mRS score of 5 to 4 and a NIHSS score from 20 to 16.

Three (8.1%) iatrogenic dissections occurred, all in emergency cases. Two were caused by failed positioning of balloon-mounted stents and one was caused by pre-dilatation. These patients were successfully treated and experienced both angiographic and clinical improvement with mRS scores improving from 4 to 1, 4 to 3, and 5 to 4.

Additionally, one (2.7%) extracranial paravasation in nuchal tissue adjacent to the distal V3 segment of the vertebral artery occurred due to failed positioning of a balloon-expandable stent (PRO-Kinetic Energy 3×9, Biotronik SE & Co. KG, Berlin, Germany). The vessel was reconstructed with two flow diverters (2× p64 MW HPC 4.0×24 mm, Phenox).

Post-procedural complications

Post-procedural complications related to the pEGASUS-HPC stent system occurred in three (8.1%) patients undergoing an emergency intervention, consisting of one hemorrhagic and two ischemic events. In one emergency patient treated with two stents due to slight dislocation of the first stent, delayed post-procedural loading with clopidogrel resulted in subtotal occlusion of the stent within less than 24 hours despite successful initial revascularization. This led to aspiration thrombectomy and tirofiban bolus infusion, but the patient suffered partial MCA infarction and neurological deterioration. Another patient with M1 occlusion experienced post-procedural subarachnoid hemorrhage under continuous tirofiban infusion and died 6 days later due to additional complications. The third patient treated for an M1 occlusion due to an underlying high-grade stenosis developed high-grade in-stent stenosis, partial MCA infarction, three-valve endocarditis with sepsis, and non-occlusive mesenteric ischemia and died 3 days after the stent procedure.

In three (50.0%) of the elective interventions, post-procedural stent-related complications occurred. One patient treated for high-grade stenosis of the C7 segment of the left ICA with pEGASUS-HPC stentand with short-term postinterventional tirofiban perfusor experienced post-procedural symptomatic intracerebral hemorrhage (sICH) with motoric aphasia. Another patient with high-grade ICA stenosis underwent successful revascularization with the pEGASUS stent but required an additional flow diverter for the treatment of iatrogenic dissection distal to the stenosis. The third patient with bilateral high-grade distal V4 stenosis and elective stenting of the left-sided stenosis experienced rotating vertigo, nausea, and gait ataxia, possibly correlated with a small pontine infarction observed on MRI 7 days after stent implantation.

In this retrospective observational single-center study we present our experience with the pEGASUS-HPC stent system. Revascularization achieved success rates of 100%, defined as an improvement in the stenosis grade of 50% or more. Subsequent occlusion after initial treatment was observed in three of the 50 stents deployed. The majority of interventions consisted of futile thrombectomies (87.8%), with a minority comprising elective interventions for symptomatic stenosis (12.2%).

The findings of our study are noteworthy in light of the Stenting and Aggressive Medical Management for Preventing Recurrent Stroke in Intracranial Stenosis (SAMMPRIS) trial, which reported a higher incidence of stroke or death in the stenting+best medical treatment group (14.7%) compared with the best medical treatment group (5.8%). 11 Since 2009 there have been significant technological advancements in intracranial stent systems. For example, the Wingspan stent used in the SAMMPRIS trial is a relatively complex stent that requires larger delivery catheters than the pEGASUS-HPC (0.027 inches vs 0.0165 inches). This difference in catheter size reflects improvements in stent design and delivery systems, allowing for more minimally invasive procedures with smaller access vessels. Such advancements contribute to enhanced safety and efficacy profiles of newer generation stents like the pEGASUS-HPC. This stent system received CE (Conformité Européenne) mark clearance in the European Union in 2021 for the treatment of aneurysms and intracranial stenosis. It is coated with a glycan-based hydrophilic multilayer polymer coating (HPC) that is a mere 10 nm thick and covalently bound to the stent material, thus offering a less thrombogenic stent surface. 5 12–14 This surface alteration mimics the biological properties of the glycocalyx present on the endothelium, contributing to improved biocompatibility and reduced thrombogenicity of the stent. 12 The coating does not compromise the mechanical properties of the pEGASUS-HPC stent. In vivo studies have shown no inflammatory response or intima hyperplasia. 14 15 The coating has been shown to reduce thrombogeneity of the coated devices and promising results were reported of the use of flow diverters coated with HPC and mono antiplatelet therapy 16 .

One of the primary competitors to the pEGASUS-HPC stent in terms of construction similarities is the Neuroform Atlas Stent (Stryker, Portage, USA). Although the Atlas stent is not certified for the treatment of intracranial stenosis according to IFU, intracranial stenosis treatment with a good safety profile using the Atlas stent was recently published. 17 However, we chose the pEGASUS-HPC mainly because it is certified for stenosis treatment and also because it is available with antithrombotic coating. While there is no comprehensive quantitative comparison of the radial force of intracranial stents available in the published literature, it is estimated that the radial force of the pEGASUS-HPC stent lies above that of the Atlas stent but below that of the Wingspan stent.

Other reasons for stent selection may be anatomical reasons. Residual stenosis may be less pronounced with balloon-expandable stents, 18 but in elongated vessels it can be impossible to push them to the desired destination. In five (13.2%) patients a primarily selected balloon-expandable stent could not be placed at the site of stenosis. Recent studies have highlighted the benefits of modern stent and catheter materials. 19–21

NeuroSpeed double-lumen PTA balloons were used in 55.8% of cases, effectively eliminating the need for further probing or exchange maneuvers. This approach demonstrates the importance of selecting appropriate tools and techniques to optimize procedural outcomes when dealing with challenging stent designs. 22 However, in four cases (8.0%) the pEGASUS-HPC stent had to be discarded because of friction and dislocation of the NeuroSpeed catheter, a problem not encountered with single-lumen catheters. The reasons were not clear, since deployment of the stents worked well in most of the cases and might be due to anatomical reasons.

Our study included patients with considerable heterogeneity in baseline characteristics including medical comorbidities, premedication, blood pressure, body mass index, demographics, and history of smoking. All patients treated with the pEGASUS-HPC stent for intracranial arterial stenosis were included, which also comprised individuals with a high likelihood of poor outcomes. Many interventions were conducted in the setting of severe acute ischemic stroke, resulting in complex scenarios. Therefore, a certain degree of negative selection bias is likely, as evidenced by the mean number of thrombectomy procedures of 3.6±3.4. When excluding these patients from the analysis and focusing solely on those with a pre-stroke mRS score ranging from 0 to 2, it was observed that 50% experienced an improved mRS, a finding consistent with other recent studies demonstrating favorable outcomes in rescue stenting scenarios. 7 23–25

Moreover, 67.0% of all treated patients were transferred from smaller primary stroke centers or, in some cases, from peripheral hospitals lacking neurological care units. This situation often resulted in prolonged transfer times for these patients. 26 Additionally, higher NIHSS and mRS scores are strong predictors of poor functional outcome. 27 28

All elective interventions were conducted following a dual antiplatelet protocol. However, there was considerable heterogeneity in emergency stentings (see table 2 ), with 32.4% pretreated with recombinant tissue plasminogen activators. Intra-procedurally, aspirin was used in 62.2% of cases while tirofiban was used in 56.8% (with some overlap of the two medications). The rate of intracranial hemorrhage (5.4% in failed thrombectomy) was not higher than in current studies on bail-out stenting in failed thrombectomy 7 24 or in studies evaluating the risk of sICH for thrombectomy in patients on different antiplatelet or anticoagulation therapy. 29 sICH rates were also below the rates of the intervention group in SAMMPRIS. 11

Our study has limitations inherent to its retrospective observational design, including a relatively small sample size and a short follow-up period. Most post-procedural mRS values were assessed at the time of hospital discharge (with a mean±SD of 12.3±9.5 days) and the 90-day mRS scores could potentially be significantly lower.

This single-center case series demonstrates the safety and viability of the HPC-coated pEGASUS stent system, especially in emergency situations when thrombectomy fails. Further randomized controlled trials are needed to evaluate the suspected additional benefit of the biomimetic HPC coating in emergency and elective endovascular treatment of intracranial arterial stenosis and to appraise the best stent design for the diverse clinical scenarios.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

Ethics approval for the study was provided by the Ethics Committee of the Association of Medical Doctors of the state of Thuringia. Verbal confirmation of the Ethics Committee was given on March 6, 2024 (reference number: 22748/2024/6). Patient informed consent was waived due to completely anonymized data collection and analysis. This report adheres to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement guidelines.

• Ong NY , et al
• Doheim MF ,
• Al-Bayati AR ,
• Bhatt NR , et al
• Pan Y , et al
• Qureshi AI ,
• Ma X , et al
• Lenz-Habijan T ,
• Peters M , et al
• Samuels OB ,
• Joseph GJ ,
• Lynn MJ , et al
• Stracke CP ,
• Fiehler J , et al
• Alexander MJ ,
• Menaker SA , et al
• Bouthillier A ,
• van Loveren HR ,
• Kabbasch C , et al
• Derdeyn CP ,
• Chimowitz MI ,
• Aguilar Pérez M , et al
• Gawlitza M ,
• Kaiser DPO , et al
• Bannewitz C , et al
• Hellstern V ,
• Aguilar Pérez M ,
• Henkes E , et al
• Cao W , et al
• Kwak JK , et al
• Mohammaden MH ,
• Nogueira RG ,
• Tekle W , et al
• Ma N , et al
• Kyselyova AA ,
• Frölich AM ,
• Bester M , et al
• Li C , et al
• Pérez-García C ,
• Gómez-Escalonilla C ,
• Rosati S , et al
• Ifergan H ,
• Dargazanli C ,
• Ben Hassen W , et al
• Psychogios M-N ,
• Schlemm E , et al
• Wouters A ,
• Thijs V , et al
• Kitano T , et al
• Becerril-Gaitan A ,
• Hunt A , et al

Contributors DP: guarantor, data collection, study design, data analysis, writing of the manuscript and revisions. JK: data collection, careful revision of the manuscript. ELe and ELo: careful revision of the manuscript, data collection. DF, AS, K-TH: careful revision of the manuscript. DL: data collection, study design, careful revision of the manuscript.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests DP: Phenox – travel expenses. DF: Medtronic, Cerenovous, Microvention, Penumbra, Stryker, Balt USA, Seimens, Mentice, Neurogami, Rapid.AI, Rapid Medical, Q’apel Medical, Arsenal Medical, Phenox, Scientia, NVMed, Perfuze, Vesalio – consulting fees; Medtronic, Cerenovous, Microvention, Penumbra, Stryker, Balt USA, Q’apel Medical – speaker honoraria; Medtronic, Cerenovous, Microvention, Penumbra, Stryker, Balt USA, Seimens, Mentice, Neurogami, Rapid.AI, Rapid Medical, Q’apel Medical, Arsenal Medical, Phenox, Scientia, NVMed, Perfuze, Vesalio – travel expenses; Scientia, MENTICE, Neurogami, NVMed, Perfuze – leadership role; Scientia, Perfuze, NVMED, Mentice, Neurogami – stock options. JK: Phenox travel expenses, speaker honoraria; Phenox, Microvention – consulting fees (money paid to institution). MG: Phenox – speaker honoraria, consulting fees; Microvention – speaker honoraria, consulting fees; Balt – consulting fees; Simq GmbH – Scientific advisory board member. AS, EL, EL: none. KT-H: Bayer – speaking honoraria, advisory fees. DL: Phenox – travel expenses, speaker honoraria (money paid to institution).

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF: