Evaluating Data Analysts: Complete Framework
The data analyst title in Indian staffing is one of the most stretched: it covers everyone from "I run pivot tables in Excel for my manager" to "I run end-to-end measurement strategy for a product team." Most clients want the latter and most CVs claim it. This guide separates the two with a structured rubric and SQL/visualization tests that take 45 minutes to administer.
Key skills
Must-have
SQL fluency (intermediate+)
Window functions, CTEs, joins beyond INNER, basic query optimization. Most "data analyst" CVs claim SQL; few can write a window function under pressure.
A BI/visualization tool
Tableau, Power BI, Looker, or Metabase — at least one with multiple dashboards shipped to non-analyst consumers.
Statistical literacy
Understands when to use median vs mean, what a confidence interval is, when correlation does not imply causation. Not advanced stats — basic literacy.
Business framing
Can translate "we want to grow revenue" into "the metric to track is revenue per active user, segmented by acquisition channel" — not just "I will pull the data."
Nice-to-have
Python (pandas, basic plotting)
Useful for ad-hoc analysis beyond SQL. Not a hard requirement for most roles.
Experimentation experience
Has run or analyzed an A/B test. Bonus if they understand the statistical pitfalls.
Domain knowledge
Domain context (e-commerce, fintech, healthcare) reduces ramp time substantially.
dbt or modern data stack
Modern teams use dbt + Snowflake/BigQuery. Familiarity is a strong signal of recency.
Interview questions (7)
Walk me through an analysis you did that changed a business decision. What was the question, the data, and the outcome?
What to listen for
Real causal chain (analysis → recommendation → decision → outcome). Many analysts produce dashboards no one uses.
A stakeholder asks you "are signups going up?" Describe how you answer that question.
What to listen for
Should ask clarifying questions (which signup definition, what time period, what segment). Going straight to a query is a yellow flag.
Write a SQL query (live, on a whiteboard or notebook) that finds the top 10 users by revenue in the last 30 days.
What to listen for
Comfort with date filters, aggregation, ORDER BY/LIMIT. Bonus for handling the "what if a user has multiple records" case.
You built a dashboard and the team is not using it. What do you do?
What to listen for
Goes to talk to them, not "blame them." Investigates whether it answers their actual questions.
Describe a time your analysis was wrong. How did you find out, and what did you do?
What to listen for
Specific example. Honest acknowledgment. "I have never been wrong" is disqualifying.
When would you NOT trust an A/B test result?
What to listen for
Sample size, peeking, multiple comparisons, novelty effect. At least 2 of these in a real answer.
What is your favorite chart type and why?
What to listen for
Reveals visual literacy. "Pie charts" is usually a yellow flag for non-junior roles.
Evaluation rubric
Score each candidate against these weighted criteria. Total: 100%.
| Criterion | Weight | Signal |
|---|---|---|
| SQL/data fluency | 30% | Can write window functions and CTEs from memory. Comfortable with messy joins. |
| Business framing | 25% | Translates business questions into measurable analyses without hand-holding. |
| Communication | 20% | Can explain a chart to a non-analyst in 30 seconds. Writes clear summaries. |
| Statistical literacy | 15% | Knows when to be skeptical of results. Understands basic experimental design. |
| Tooling depth | 10% | Has shipped real artifacts (dashboards, reports) that were used by stakeholders. |
Red flags
CV says "SQL expert" but cannot write a JOIN under timer
Has never had an analysis influence a real decision
All experience is in Excel — has never used a real database
Claims "data scientist" experience but cannot explain a confidence interval
Dashboard portfolio looks visually noisy and inconsistent
Apply this rubric automatically with CVPRO
Upload Data Analyst CVs and let AI score every candidate against the same 42-point evidence rubric.
Try CVPRO Free