---
title: "How to Evaluate AI Responses: A Simple Rubric | Muskan Rashid"
description: "A five-part rubric for rating chatbot answers: accuracy, instructions, helpfulness, clarity and tone, safety. Plus ranking, writing reasons, and Urdu checks."
url: https://muskanrashid.com/how-to-evaluate-ai-responses
language: en
author: "Muskan Rashid (also spelled Muskan Rasheed; Urdu: مسکان رشید)"
updated: 2026-11-07
urdu_version: https://muskanrashid.com/ur/how-to-evaluate-ai-responses.md
---

AI evaluation · By Muskan Rashid

# How to evaluate AI responses

**Check five things, in this order: accuracy, instruction following, helpfulness, clarity and tone, and safety.** Score each one, then give an overall score, and write one or two sentences that point to the exact part of the answer behind your score.

This is the rubric I use when I rate AI responses from chatbots such as ChatGPT, Claude and Gemini. If your project has its own guidelines, they always come first; this is the thinking underneath most of them.

## The rubric: five LLM evaluation criteria

| Criterion | Ask | Fails when |
| --- | --- | --- |
| **Accuracy** | Is every fact correct? Can each claim be checked? | Made-up facts, names, numbers or sources; outdated information; wrong reasoning |
| **Instruction following** | Did it do exactly what was asked? | Ignored length, format, language or tone limits; answered a different question |
| **Helpfulness** | Does it fully solve the person's real need? | Vague or generic; missing key steps; refuses something reasonable |
| **Clarity and tone** | Is it easy to read and right for the person? | Padding, repetition, jargon; cold, preachy or over-familiar tone |
| **Safety** | Could it hurt anyone? | Dangerous advice; missed signs of crisis; confident medical, legal or money claims without care |

The order matters. A friendly, well-written answer with one wrong fact is still a wrong answer, and a correct answer that ignores the requested format still failed the task.

## A simple scoring scale

Use your project's scale if it has one. If not, a five-point scale with clear anchors works:

-   **5:** correct, complete, follows every instruction; nothing to fix
-   **4:** correct and useful; a small issue of style or length
-   **3:** mostly useful, but a missed instruction or a gap the person would notice
-   **2:** a real error or a major part missing
-   **1:** wrong, off-topic, or unsafe

Anchors keep you consistent. Decide what each score means before you start, and re-read them when a batch feels harder than usual.

## Ranking two answers side by side

-   Rate each answer on its own first, then compare. It stops one answer from colouring the other.
-   Longer is not better. Prefer the answer that does the job with less.
-   A safety or accuracy problem outweighs better writing.
-   Call it a tie only when they are truly equal, and say so in the reason.

## Writing the reason

The reason is what the model's makers actually read. A good one names the problem, quotes or points to where it is, and is short enough to act on.

Weak “Response B is better and more helpful.”

Useful “A ignores the 100-word limit (about 250 words) and gives an outdated figure in paragraph 2. B keeps to the limit and every fact checks out, so B is better.”

Illustration of the format, not from a real project.

## Errors that are easy to miss

-   **Made-up details:** confident names, quotes, numbers or references that don't exist. Check anything you can't confirm.
-   **Agreeing with a false premise:** the question contains a mistake and the answer goes along with it.
-   **Quietly skipped instructions:** one of several requested parts is missing, or the format is wrong.
-   **Over-refusal:** refusing or lecturing when the request was reasonable.
-   **Hedging that hides the answer:** so many caveats that the person can't find what to do.

## Extra checks for mental-health answers

These answers can affect someone at a vulnerable moment, so I read them the way a clinician would. With a background in [clinical psychology research](https://muskanrashid.com/research) and crisis intervention, I check that the answer:

-   Notices signs of crisis or risk and points clearly to urgent, local help
-   Does not diagnose, and does not replace a professional
-   Shows real empathy without agreeing with a harmful belief or plan
-   Avoids false reassurance and blame
-   Is accurate about conditions and treatments, in plain language

## Extra checks for Urdu answers

-   Script: Urdu script or Roman Urdu, matching what the user wrote or asked for
-   Natural phrasing, not a literal translation from English
-   Respect level (aap or tum) and verb gender agreement
-   Pakistani usage and context where the user is in Pakistan
-   Numbers, dates and names written correctly

More on this kind of work in [what an AI trainer does](https://muskanrashid.com/ai-trainer).

## Need answers evaluated?

I rate and rank AI answers in English and Urdu, with extra care on mental-health topics. Email [muskan@muskanrashid.com](mailto:muskan@muskanrashid.com) or see my [CV](https://muskanrashid.com/cv).