DOCUMENTATION

Evaluate.
Understand. Improve.

EvalLoop AI helps you evaluate AI-generated responses using structured, explainable quality scores. Start with a single response or compare two responses against the same prompt.

ON THIS PAGE

Learn the evaluation workflow, understand the six scoring dimensions, compare responses, and review important privacy and usage limitations.

DOC.01 ● AVAILABLE

Getting Started

Follow the basic workflow from entering a prompt to reviewing evaluation scores, detected issues, and suggestions.

DOC.02 ● AVAILABLE

General Response

Evaluate one AI-generated response using six quality dimensions and receive a structured result.

DOC.03 ● AVAILABLE

Evaluation Dimensions

Understand what each dimension measures and how scores are interpreted.

DOC.04 ● AVAILABLE

Compare A vs B

Compare two responses to the same prompt and review the winner, confidence, differences, and recommendation.

DOC.05 ● IMPORTANT

Data & Privacy

Understand what information is sent to the evaluation service and how provider credentials are handled.

DOC.06 ● IMPORTANT

Limitations

Review the current free-beta limitations and why important results should be checked by a person.

GETTING STARTED

The basic workflow for evaluating one response or comparing two.

Run your first evaluation.

To get started with EvalLoop AI:

Open the EvalLoop Workspace.
Choose General Response or Compare.
Paste your prompt.
Paste the AI response, or enter both Response A and Response B when comparing.
Optionally add a reference answer or source context.
Click Evaluate or Compare.
Review the scores, detected issues, and improvement suggestions.

GENERAL RESPONSE EVALUATION

Evaluate one AI response against the prompt that produced it.

One response. Six dimensions.

General Response evaluates one AI response against a prompt. It returns an overall quality score, scores for each dimension, detected issues, a summary, and an improvement recommendation.

Required inputs

Prompt
AI Response

Optional input

Reference / Context

A reference answer or source context can give the evaluator additional information to consider. The overall score is calculated as the average of the six dimension scores.

EVALUATION DIMENSIONS

Each dimension is scored from 0 to 100.

Understand the quality scores.

EvalLoop AI assesses response quality across six dimensions. Each score represents the evaluator's assessment of that aspect of the response.

Factuality

Accuracy, unsupported claims, and potential hallucinations.

Relevance

How directly the response answers the request.

Instruction Following

Whether the response follows the user's requirements.

Completeness

Whether important information is missing.

Safety

Whether the response avoids harmful or unsafe content.

Fluency

Clarity, readability, coherence, and language quality.

Overall score calculation.

The overall score is the average of the six dimension scores. Each dimension contributes equally to the result.

OVERALL SCORE
overall_score =
  (
    factuality +
    relevance +
    instruction_following +
    completeness +
    safety +
    fluency
  ) / 6

Score meaning.

Use these ranges as a guide when interpreting the overall score.

90–100 · Excellent

A strong response with high overall quality.

75–89 · Good

A good response that may still have room for improvement.

50–74 · Needs improvement

A response with noticeable weaknesses to review.

Below 50 · Poor

A response that likely needs substantial improvement.

COMPARE A VS B

Evaluate two AI responses against the same prompt.

Compare responses. Review the differences.

Compare lets you submit two responses for the same prompt. Reference or source context is optional.

Inputs

Prompt
Response A
Response B
Optional reference / context

Comparison results

Winner: Response A, Response B, or Tie
Confidence
Dimension-by-dimension comparison
Key differences
Issues detected in each response
Recommendation

Treat the winner and confidence as AI-generated judgments, not guarantees. Review the underlying differences before deciding which response is appropriate for your use case.

DATA & PRIVACY

Understand what is sent for evaluation and how credentials are handled.

Share only what is necessary.

Your prompt, AI response, and optional reference are sent to the EvalLoop evaluation service.
EvalLoop uses an LLM provider to perform the evaluation.
Do not submit passwords, private keys, personal identity documents, or confidential business data.
API credentials are stored on the server and must never be exposed in browser code.

Protect sensitive information. Avoid including private or confidential content in prompts, responses, or reference material submitted for evaluation.

For details about the website's data practices and terms of use, review the Privacy Policy and Terms pages.

CURRENT LIMITATIONS

Important considerations when interpreting AI-assisted evaluations.

Scores are useful signals, not guarantees.

EvalLoop provides AI-assisted evaluation, not a guarantee of correctness.
LLM judges can make mistakes.
Review important, high-risk, legal, medical, financial, or safety-related outputs manually.
Usage limits apply during the free beta.

ROADMAP

Potential future capabilities. These are not part of the current feature set.

Planned future capabilities.

Saved evaluation history
CSV batch evaluation
Safety-focused evaluation
RAG evaluation
Prompt and model regression checks
API access

Roadmap items are not guaranteed and may change. Availability should be confirmed in the product rather than assumed from this list.

READY TO EVALUATE?

Put an AI response to the test.

Evaluate one response or compare two responses against the same prompt. Review the scores, understand the issues, and decide what to improve.

GENERAL RESPONSE → COMPARE A VS B → BACK TO EVALLOOP AI →

Evaluate. Understand. Improve.