Quality Scoring

Overview

Quality scoring evaluates the reliability of cell type annotations by analyzing the reasoning and evidence behind each prediction. The scoring agent assigns confidence scores (0-100) to each annotation, helping identify clusters that may need further validation through the Annotation Boost or Compare agents.

Quick Start

runCASSIA_score_batch(
    input_file = "my_annotation_summary.csv",  # JSON auto-detected
    output_file = "my_annotation_scored.csv",
    model = "openai/gpt-5.1",
    provider = "openrouter"
)
R

Input

Quality scoring requires two files generated by runCASSIA_batch:

Summary CSV (*_summary.csv) - Contains cluster identifiers, predicted cell types, marker gene lists, and metadata
Conversations JSON (*_conversations.json) - Contains the complete conversation history with reasoning

Auto-detection: When you provide the summary CSV as input, CASSIA automatically finds the corresponding conversations JSON file in the same directory. No need to specify both files manually.

Parameters

Required

Parameter	Description
`input_file`	Path to summary CSV (from `runCASSIA_batch`). The conversations JSON is auto-detected.
`model`	LLM model ID for scoring
`provider`	API provider (`"openrouter"`, `"openai"`, `"anthropic"`) or custom base URL

Optional

Parameter	Default	Description
`output_file`	`{input}_scored.csv`	Output file name for scored results
`conversations_json_path`	Auto-detected	Path to conversations JSON. By default, auto-detected from input CSV name.
`max_workers`	4	Number of parallel scoring threads
`reasoning`	`NULL`	Reasoning depth for GPT-5 series via OpenRouter only (`"low"`, `"medium"`, `"high"`). See Reasoning Effort Parameter.

Parameter Details

Model Selection

Highly recommended: openai/gpt-5.1 or anthropic/claude-sonnet-4.5 for best accuracy
Scoring requires strong reasoning capabilities to evaluate annotation quality

Output

Files Generated

File	Description
`{output_file}`	Scored results CSV with quality scores and reasoning
`{output_file}_report.html`	Interactive HTML report with all CASSIA outputs

Output Contents

The scored CSV file includes:

Original annotation data
Quality scores (0-100)
Detailed reasoning for each score

Interpreting Scores

Score Range	Confidence	Recommended Action
90-100	High	Strong evidence, annotation is reliable
76-89	Good	Adequate evidence, generally trustworthy
<75	Low	Run through Annotation Boost Agent and Compare Agent