LLMs as Human Simulacra: Opportunities and Limitations from a Linguistic Perspective

Basic information

• Workshop
• Tuesday, 24.2.2026, 1-6 pm (5 hours, including 3 x 30 min breaks w/ coffee)
• max. 45 participants
• registration opens mid-December

Abstract

The simulation of individual user behavior using Large Language Models (LLMs) has become increasingly popular in computational social science, offering novel opportunities to model human communication patterns at scale[1]. However, the validity of these simulations beyond superficial, believable human-like communication remains critically under-explored[2][3], particularly from a linguistic perspective that examines deeper structural and stylistic authenticity. Our workshop explores the evolving landscape of emulating individual users with a specific focus on TWins of Online Social Networks (TWONs)[4]. It presents contemporary approaches to aligning and evaluating task-specific behaviors — including writing posts and replying to content — while maintaining a critical lens on the general validity and limitations of these methodologies. We address fundamental questions about what constitutes authentic human simulation and how linguistic analysis can reveal both the capabilities and boundaries of current LLM-based approaches. Through presentations, discussions, and hands-on interactive sessions, participants will gain practical experience in training and evaluating LLMs using state-of-the-art reinforcement learning approaches[5]. The workshop emphasizes moving beyond traditional token-to-token comparison by implementing custom evaluation metrics. The workshop is designed for researchers working at the intersection of linguistics and social media analysis who seek to understand both the promise and pitfalls of using LLMs as human behavioral simulacra in digital environments.

Schedule

1. Introduction | Excerpt TWON: Simulating User Behavior on Online Social Networks
   (Presentation, 20 min, Simon Münker)
2. Background | Efficiently Aligning LLM on Individual Social Media Data
   (Presentation, 20 min, Simon Münker)
3. Break | 30 min ~ Technical Setup Support (all)
4. Tutorial | Contemporary Approaches for Training LLMs
   (Interactive Session, 60 min, Nils Schwager)
5. Break | 30 min
6. Background | Evaluating Artificial Generated Content across Linguistic Features
   (Presentation, 20 min, Kai Kugler)
7. Tutorial | Contemporary Approaches for Evaluating LLMs
   (Interactive Session, 60 min, Nils Schwager)
8. Break | 30 min
9. Closing | Outlook, Take Aways, Audience Proposals
   (Presentation and Q&A, 30 min, Kai Kugler

1. Wang, L., Zhang, J., Yang, H., Chen, Z. Y., Tang, J., Zhang, Z., ... & Wen, J. R. (2025). User behavior simulation with large language model-based agents. ACM Transactions on Information Systems, 43(2), 1-37. ↩
2. Münker, S., Schwager, N., & Rettinger, A. (2025). Don't Trust Generative Agents to Mimic Communication on Social Networks Unless You Benchmarked their Empirical Realism. arXiv preprint arXiv:2506.21974. ↩
3. Münker, S. (2025). Political Bias in LLMs: Unaligned Moral Values in Agent-centric Simulations. Journal for Language Technology and Computational Linguistics, 38(2), 125–138. doi.org/10.21248/jlcl.38.2025.289 ↩
4. TWON (project number 101095095) as a research project is fully funded by the European Union under the Horizon Europe framework (HORIZON-CL2-2022-DEMOCRACY-01-07). www.twon-project.eu ↩
5. von Werra, L., Belkada, Y., Tunstall, L., Beeching, E., Thrush, T., Lambert, N., Huang, S., ... & Rasul, K. (2020) TRL: Transformer Reinforcement Learning. GitHub repository. github.com/huggingface/trl ↩