Study finds ChatGPT gave substantive surveillance guidance for 40 of 55 jailbreak prompts

[arXiv] A preprint titled "ChatGPT as a Surveillance Assistant? An Exploratory Case Study of AI-Assisted Surveillance Risks," by Ayana Hussain and Nicholas Vincent, says ChatGPT's public web interface can provide substantive assistance on surveillance-related tasks. The authors built a taxonomy of seven surveillance risk categories from prior LLM risk research and reports of deployed surveillance systems, instantiated it as 110 prompts, and manually reviewed ChatGPT's responses. Prompts were issued in two ways: 55 jailbreak prompts drawing on five elicitation strategies, and 55 plain-language direct prompts, each run alone and, if unsuccessful, again with a short fixed prefix. According to the paper, ChatGPT provided substantive surveillance guidance for 40 of the 55 jailbreak prompts and for 33 of the 55 direct prompts without any prefix. Compliant responses rarely included policy mentions or suggestions for privacy-preserving alternatives, the authors said, adding that even when ChatGPT states a refusal it may still supply detailed instructions, working code or architectural guidance alongside brief ethical caveats. The findings are from a preprint and have not been peer reviewed.