Our researcher Simon Münker published a new paper with the title “Fingerprinting LLMs through Survey Item Factor Correlation: A Case Study on Humor Style Questionnaire.” It is published in the Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing in Sozhou, China on November 5.
LLMs increasingly engage with psychological instruments, yet how they represent constructs internally remains poorly understood. In his paper, Simon Münker introduces a new approach to “fingerprinting” LLMs through their factor correlation patterns on standardized psychological assessments to deepen the understanding of LLMs constructs representation.
Using the Humor Style Questionnaire as a case study, he analyzes how six LLMs represent and correlate humor-related constructs to survey participants. His results show that they exhibit little similarity to human response patterns. In contrast, participants’ subsamples demonstrate remarkably high internal consistency. Exploratory graph analysis further confirms that no LLM successfully recovers the four constructs of the Humor Style Questionnaire.
His findings suggest that despite advances in natural language capabilities, current LLMs represent psychological constructs in fundamentally different ways than humans, questioning the validity of application as human simulacra.
Read the full paper here: https://aclanthology.org/2025.emnlp-main.13/
EMNLP 2025 is a premier international conference focused on empirical methods in natural language processing, showcasing cutting-edge research in NLP, large language models, linguistics, and multimodal systems. Held in Suzhou, it brings together researchers from academia and industry to present new findings, datasets, and analyses. The event emphasizes both methodological advances and real-world applications across the NLP ecosystem.
