Skip to content

Scoring Model

The scoring model below is generated from the main repository README so the docs reflect the shipped scoring logic and naming.

Per-file: type-specific dimensions (0-100)

Every file type has its own scoring dimensions. Score starts at 0 and is earned through quality signals.

agent (CLAUDE.md, agent.md):

  • Actionability (22) - ratio of actionable instructions
  • Security (20) - no leaked secrets
  • Commands (15) - build/test/dev documented
  • Architecture (12) - project structure documented
  • Structure (10) - headers, no placeholders
  • Direction (8) - clear directives, no contradictions
  • Size Fitness (8) - within token budget
  • Token Efficiency (5) - no duplicates or verbose phrases

soul (soul.md):

  • Persona Clarity (25) - identity, tone, boundaries defined
  • Behavioral Rules (25) - MUST/NEVER/ALWAYS directives
  • Conciseness (20) - within token sweet spot
  • Structure (15) - organized sections
  • Security (15) - no leaked secrets

skill (SKILL.md):

  • Trigger Clarity (25) - when to activate/deactivate
  • Scope Boundaries (20) - what the skill does NOT do
  • Actionability (20) - dense, actionable instructions
  • Conciseness (15) - focused, not bloated
  • Structure (10) - organized sections
  • Security (10) - no leaked secrets

memory, hook, tool each have tailored dimensions.

Fleet-level: multiplicative factors

The project score combines file scores with fleet-level quality factors:

projectScore = weightedFileAverage x fleetMultiplier

Fleet dimensions (each produces a 0.0-1.0 multiplier):

  • Role Coverage - are required files present?
  • Role Separation - are files doing distinct jobs? (detects duplication)
  • Content Coherence - no cross-file contradictions
  • Codebase Awareness - blind spots documented
  • Reference Integrity - no dead file references
  • Token Budget - within context window limits

File type weights in project average: agent (3x), soul (2.5x), memory (1.5x), skill (1x), hook/tool (0.8x).

Auto-fix

opty fix safely applies these fixes (with backup):

  • Replace leaked API keys with env var placeholders
  • Replace connection strings with $DATABASE_URL
  • Replace chmod 777 with chmod 755
  • Remove duplicate paragraphs within files
  • Shorten verbose phrases ("in order to" -> "to")