Jing Yang

MAR 2.055,

Marchstrasse 23

Berlin, Germany

I am a post-doc researcher at the XplaiNLP group in TU Berlin led by Dr. Vera Schmitt, and supervised by Prof. Sebastian Möller. I am also a guest researcher at the German Research Center for Artificial Intelligence (DFKI). My current research project (FakeXplain) is a BIFOLD agility project related to generating natural language explanations for AI-based disinformation detection.

I completed my Bachelor’s degree in Information and Computing Science at the Hubei University of Technology in China, followed by a Master’s degree in Computer Science at Hunan University, China. My Master’s dissertation is related to identifying 3D printed objects and printers with Digital Forensics and Machine Learning. After obtaining my Master’s degree in 2019, I pursued a PhD at the RECOD.ai lab from the University of Campinas in Brazil, under the supervision of Prof. Anderson Rocha. During 2022-2023, I did a research internship at the Ubiquitous Knowledge Processing (UKP) lab in TU Darmstadt. My PhD thesis was related to improving fact-checking efficiency and explainability with few-shot learning and large language models.

I work on natural language processing with a focus on making large language models reliable, fair, and factually grounded. A consistent thread in my research is the question of whether automated systems can be trusted: I have studied this through explainable fact-checking, out-of-distribution evaluation of model rationales, large-scale analysis of evaluation practices, and behavioral studies of persona prompting and social bias. My current interest lies in how factual knowledge and the values and opinions learned from training data interact inside models, and in evaluation and interpretability methods that make model behavior transparent, controllable, and aware of the values and cultural perspectives it encodes.

My current research interests are:

Interplay of factual knowledge and learned values and opinions in LLMs
Evaluation of LLMs and natural language generation
Interpretability and explainability
Social bias, fairness, and cultural perspectives in LLMs

news

Oct 02, 2026	Really enjoyed teaching a lecture at the Fall School organized by UTN together with Veronika Solopova! Many thanks to the organizers, especially Michael Roth, for the invitation and the great event.
Sep 01, 2026	Our paper What Are We Measuring in NLG? A Meta-Analysis of Evaluation Trends 2020–2025 has been accepted at EMNLP 2026!
Mar 25, 2026	Very happy to be attending EACL 2026, and presenting our paper “Persona Prompting as a Lens on LLM Social Reasoning”.
Jan 12, 2026	Our work about NLG evaluation trend analysis based on LLM extracted structured data from NLP papers, is available on arXiv: Order in the Evaluation Court: A Critical Analysis of NLG Evaluation Trends.
Jan 06, 2026	Our paper “Persona Prompting as a Lens on LLM Social Reasoning” has been accepted to EACL 2026! Many thanks to our co-authors: Moritz Hechtbauer, Elisabeth Khalilov, Evelyn Luise Brinkmann, Vera Schmitt, Nils Feldhus. Looking forward to seeing you in Rabat, Morocco!

see all news →

selected publications

EMNLP

What Are We Measuring in NLG? A Meta-Analysis of Evaluation Trends 2020-2025

Jing Yang, Nils Feldhus, Salar Mohtaj, and 10 more authors

arXiv preprint arXiv:2601.07648, Oct 2026

EMNLP 2026, to appear

Bib HTML

@article{yang2026what,
  title = {What Are We Measuring in NLG? A Meta-Analysis of Evaluation Trends 2020-2025},
  author = {Yang, Jing and Feldhus, Nils and Mohtaj, Salar and Hennig, Leonhard and Wang, Qianli and Metheniti, Eleni and Hakimov, Sherzod and Jakob, Charlott and Solopova, Veronika and Rieck, Konrad and Schlangen, David and M{\"o}ller, Sebastian and Schmitt, Vera},
  journal = {arXiv preprint arXiv:2601.07648},
  year = {2026},
  month = oct,
  note = {EMNLP 2026, to appear},
}

EACL
Persona Prompting as a Lens on LLM Social Reasoning

Jing Yang, Moritz Hechtbauer, Elisabeth Khalilov, and 3 more authors

In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), Mar 2026

Abs DOI Bib HTML

For socially sensitive tasks like hate speech detection, the quality of explanations from Large Language Models (LLMs) is crucial for factors like user trust and model alignment. While Persona prompting (PP) is increasingly used as a way to steer model towards user-specific generation, its effect on model rationales remains underexplored. We investigate how LLM-generated rationales vary when conditioned on different simulated demographic personas. Using datasets annotated with word-level rationales, we measure agreement with human annotations from different demographic groups, and assess the impact of PP on model bias and human alignment. Our evaluation across three LLMs results reveals three key findings: (1) PP improving classification on the most subjective task (hate speech) but degrading rationale quality. (2) Simulated personas fail to align with their real-world demographic counterparts, and high inter-persona agreement shows models are resistant to significant steering. (3) Models exhibit consistent demographic biases and a strong tendency to over-flag content as harmful, regardless of PP. Our findings reveal a critical trade-off: while PP can improve classification in socially-sensitive tasks, it often comes at the cost of rationale quality and fails to mitigate underlying biases, urging caution in its application.
@inproceedings{yang-etal-2026-persona, title = {Persona Prompting as a Lens on {LLM} Social Reasoning}, author = {Yang, Jing and Hechtbauer, Moritz and Khalilov, Elisabeth and Brinkmann, Evelyn Luise and Schmitt, Vera and Feldhus, Nils}, editor = {Demberg, Vera and Inui, Kentaro and Marquez, Llu{\'i}s}, booktitle = {Proceedings of the 19th Conference of the {E}uropean Chapter of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)}, month = mar, year = {2026}, address = {Rabat, Morocco}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/2026.eacl-long.52/}, doi = {10.18653/v1/2026.eacl-long.52}, pages = {1152--1170}, }

TACL

Self-Rationalization in the Wild: A Large-scale Out-of-Distribution Evaluation on NLI-related tasks

Jing Yang, Max Glockner, Anderson Rocha, and 1 more author

Transactions of the Association for Computational Linguistics, Mar 2025

Bib Code Project Page

@article{yang2025self,
  title = {Self-Rationalization in the Wild: A Large-scale Out-of-Distribution Evaluation on NLI-related tasks},
  author = {Yang, Jing and Glockner, Max and Rocha, Anderson and Gurevych, Iryna},
  journal = {Transactions of the Association for Computational Linguistics},
  volume = {13},
  pages = {314--342},
  year = {2025},
}

ICASSP
Explainable Fact-checking through Question Answering

Jing Yang, Didier Vega-Oliveros, Taı́s Seibt, and 1 more author

In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Mar 2022

Abs Bib PDF Code

Misleading or false information has been creating chaos in some places around the world. To mitigate this issue, many researchers have proposed automated fact-checking methods to fight the spread of fake news. However, most methods cannot explain the reasoning behind their decisions, failing to build trust between machines and humans using such technology. Trust is essential for fact-checking to be applied in the real world. Here, we address fact-checking explainability through question answering. In particular, we propose generating questions and answers from claims and answering the same questions from evidence. We also propose an answer comparison model with an attention mechanism attached to each question. Leveraging question answering as a proxy, we break down automated fact-checking into several steps — this separation aids models’ explainability as it allows for more detailed analysis of their decision-making processes. Experimental results show that the proposed model can achieve state-of-the-art performance while providing reasonable explainable capabilities.
@inproceedings{yang2021explainable, title = {Explainable Fact-checking through Question Answering}, author = {Yang, Jing and Vega-Oliveros, Didier and Seibt, Ta{\'\i}s and Rocha, Anderson}, booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, year = {2022}, organization = {IEEE}, }