Quality Scoring

Overview

Quality scoring evaluates the reliability of cell type annotations by analyzing the reasoning and evidence behind each prediction. The scoring agent assigns confidence scores (0-100) to each annotation, helping identify clusters that may need further validation through the Annotation Boost or Compare agents.

Quick Start

runCASSIA_score_batch(
    input_file = "my_annotation_summary.csv",  # JSON auto-detected
    output_file = "my_annotation_scored.csv",
    model = "openai/gpt-5.4",
    provider = "openrouter"
)
R

Input

Quality scoring requires two files generated by runCASSIA_batch:

1. Summary CSV (*_summary.csv) - Contains cluster identifiers, predicted cell types, marker gene lists, and metadata
2. Conversations JSON (*_conversations.json) - Contains the complete conversation history with reasoning

Auto-detection: When you provide the summary CSV as input, CASSIA automatically finds the corresponding conversations JSON file in the same directory. No need to specify both files manually.

Keep the JSON beside the summary CSV with the matching base name (for example, my_annotation_summary.csv and my_annotation_conversations.json). The scored HTML report uses it to show the annotation, validation, and formatting analysis for each cluster, along with the new score reasoning. The summary CSV alone does not contain that analysis history.

Parameters

Required

Optional

Parameter Details

Model Selection

• Highly recommended: openai/gpt-5.4 or anthropic/claude-sonnet-4.6 for best accuracy
• Scoring requires strong reasoning capabilities to evaluate annotation quality

Output

Files Generated

Output Contents

The scored CSV file includes:

• Original annotation data
• Quality scores (0-100)
• Detailed reasoning for each score

Interpreting Scores