Integrating Computational Text Analysis into Risk and Crisis Communication Development

Abstract

Qualitative textual analysis is advancing with the integration of Natural Language Processing (NLP) and Large Language Models (LLMs). Although many multidisciplinary researchers adopt these analysis tools, Risk and Crisis Communication (RCC) researchers have not taken full advantage of such tools in content analysis tasks. We investigate computational tools' ability to replicate human analysis in RCC research. We integrate NLP and ChatGPT into content analysis steps performed on insider threat source text. The first content analysis step tasks ChatGPT with coding insider threat text as character types defined by the Narrative Policy Framework (NPF), and the second step leverages select NLP tools to derive meaning in the coded sentences. Content coding with ChatGPT varies in performance based on the amount of detail present in prompts. Select NLP tools have varying degrees of success when processing and analyzing coded text. Both ChatGPT and NLP require human intervention when performing content analysis, thus a mixed method approach is necessary for future RCC research.