STAR Method Evaluation Engine
This page documents the grading logic that checks candidate transcripts for structural markers matching the STAR methodology.
1. STAR Evaluation Framework
The STAR method is an industry standard for answering behavioral questions. Mockrithm parses candidate transcripts to confirm that they provide a structured response containing all four components:
| Stage | Focus Area | Dynamic Transcript Check | Weight |
|---|---|---|---|
| Situation | Context / Setting | Identifies background indicators (e.g., "At my previous company", "We were facing..."). | 15% |
| Task | Responsibility / Challenge | Identifies the core goal (e.g., "My task was to...", "We needed to resolve..."). | 15% |
| Action | Execution / Individual steps | Evaluates active verb usage and technical decisions (e.g., "I refactored...", "I led..."). | 45% |
| Result | Outcome / Telemetry | Flags quantitative achievements (e.g., "decreased latency by 20%", "improved score to..."). | 25% |
2. Text Classification Architecture
During active mock interviews, the system feeds candidate transcripts to the grading pipeline. It evaluates the response structure using system prompt heuristics:
graph TD
Trans[Transcript text fragment] --> S_Check{Checks for Context?}
Trans --> T_Check{Checks for Goal?}
Trans --> A_Check{Checks for Action verbs?}
Trans --> R_Check{Checks for Metrics/Results?}
S_Check -- Yes --> S_Done[Mark Situation complete]
T_Check -- Yes --> T_Done[Mark Task complete]
A_Check -- Yes --> A_Done[Mark Action complete]
R_Check -- Yes --> R_Done[Mark Result complete]
S_Done & T_Done & A_Done & R_Done --> Score[Calculate Total STAR score]3. Evaluation Schema & Prompts
Here is how the Server Action configures the prompt to calculate STAR scores:
import { generateObject } from "ai";
import { google } from "@ai-sdk/google";
import { z } from "zod";
const starGradingSchema = z.object({
situationChecked: z.boolean(),
situationEvidence: z.string().optional(),
taskChecked: z.boolean(),
taskEvidence: z.string().optional(),
actionChecked: z.boolean(),
actionEvidence: z.string().optional(),
resultChecked: z.boolean(),
resultEvidence: z.string().optional(),
starScore: z.number().min(0).max(100),
feedbackSummary: z.string(),
});
export async function scoreResponse(candidateAnswer: string, questionAsked: string) {
const prompt = `You are a professional mock interview evaluator. Grade the candidate's answer to this behavioral question.
Question: ${questionAsked}
Answer: ${candidateAnswer}`;
try {
const { object } = await generateObject({
model: google("gemini-2.0-flash-001"),
schema: starGradingSchema,
prompt: prompt,
temperature: 0.1,
});
return { success: true, analysis: object };
} catch (error) {
console.error("STAR evaluation pipeline failed:", error);
return { success: false };
}
}