---
title: "Check LLM-based content scoring against real readers"
description: "If you use an LLM to rate tone, clarity or offensiveness in your content, check its scores against real readers before trusting them."
url: https://aeo.so/tactics/check-llm-based-content-scoring-against-real-readers
author: "Paul Andre de Vera"
updated: 2026-11-15
---

# Check LLM-based content scoring against real readers

If you use an LLM to rate tone, clarity or offensiveness in your content, check its scores against real readers before trusting them. A 2025 study of nine LLMs (arXiv 2311.09730) found their ratings sat closer to White annotators' than Black or Asian annotators', and adding the audience's identity to the prompt didn't reliably fix it.

- Area: [Experiments & Case Studies](https://aeo.so/categories/experiments)
- Credited to: [Paul Andre de Vera](https://aeo.so/experts/paul-andre-de-vera)
- Part of: [the aeo.so answer engine optimization checklist](https://aeo.so/categories)

The worked example, why this helps with AI search, and the original source are free for members: https://aeo.so/sign-in


Read the guide:

- [AEO tools: free and paid tools for AI search visibility](https://aeo.so/guides/aeo-tools)
- [What is generative engine optimization (GEO)?](https://aeo.so/guides/generative-engine-optimization)
- [Is SEO dead?](https://aeo.so/guides/is-seo-dead)

## More in Experiments & Case Studies

- [Create content specifically for AI training data](https://aeo.so/tactics/create-content-specifically-for-ai-training-data)
- [Experiment with prompt-optimized content](https://aeo.so/tactics/experiment-with-prompt-optimized-content)
- [Experiment with AI-generated video content](https://aeo.so/tactics/experiment-with-ai-generated-video-content)
- [Monitor AI-driven search results](https://aeo.so/tactics/monitor-ai-driven-search-results)
- [Understand and optimize for Retrieval Augmented Generation (RAG)](https://aeo.so/tactics/understand-and-optimize-for-retrieval-augmented-generation-rag)
