Catalog
8 branches of analytics, product and AI interview questions
Explore eight skill tracks, called branches, filterable by subtopic, company, tool and difficulty. SQL questions run against sample tables, choice and numeric answers are checked instantly, and written cases are scored against evaluation criteria.
Filters
157 questions · page 3 of 7
Sample size for a 1-point conversion lift
Baseline conversion is 10%.
Calculation · Design · 8 min · Medium
Experimentation & A/B Testing · Original Uplift question
Build a golden evaluation set for an agent
ProYou're standing up evaluation for a new AI coding/support agent.
Written case · Evaluation · 15 min · Medium
AI Agents & Automation · Company-context scenario
Type I vs Type II error trade-off
A medical screening test is being tuned.
Quick check · Intervals · 5 min · Easy
Statistics & Inference · Company-context scenario
Measuring groundedness in a RAG system
ProYour retrieval-augmented (RAG) assistant sometimes states facts not supported by the retrieved documents.
Quick check · Product metrics · 6 min · Medium
AI Agents & Automation · Company-context scenario
A skeptical marketing lead and a 'winning' test
ProA marketing lead ran an email A/B test, saw the variant get a higher open rate, and wants to declare it the winner and roll it out.
Written case · Communicating stats · 12 min · Medium
Analytics Case & Communication · Company-context scenario
Top 3 products per category by revenue
Return the top 3 products by total revenue within each category.
Live SQL · Windows · PostgreSQL · 12 min · Medium
SQL & Data Modeling · Original Uplift question
Spotting the vanity metric
A growth team celebrates that 'total registered users crossed 10 million.' Why might a PM push back on using this as the headline success metric?
Quick check · Metrics · 5 min · Easy
Product Management · Company-context scenario
Positive test, rare disease
ProA disease affects 1% of a population.
Calculation · Bayesian reasoning · 10 min · Hard
Statistics & Inference · Company-context scenario
Measure the success of Snap Map
How would you measure the success of Snap Map (the map showing friends' locations and activity)?
Written case · Measuring impact · 15 min · Medium
Product Analytics & Metrics · Reported interview question
Is the LTV:CAC healthy
A SaaS product has customer lifetime value (LTV) of $900 and customer acquisition cost (CAC) of $300.
Calculation · Unit economics · 5 min · Easy
Digital & Marketing Analytics · Company-context scenario
False-positive risk with four variants
ProYou run one control against four independent treatment variants, testing each at α = 0.05.
Calculation · Multiple testing · 6 min · Medium
Experimentation & A/B Testing · Reported interview question
Guardrails for a checkout change
ProYou're testing a one-click checkout that you expect to raise conversion.
Written case · Guardrails · 15 min · Medium
Experimentation & A/B Testing · Company-context scenario
You can only ship one of three features
ProYou have engineering capacity for exactly one of three proposed features this quarter.
Written case · Prioritization · 15 min · Medium
Product Management · Company-context scenario
The scoreboard for a customer-facing agent
ProYou launched an AI agent that can take actions (issue refunds, change orders) for customers.
Written case · Product metrics · 15 min · Medium
AI Agents & Automation · Company-context scenario
Longest consecutive login streak
ProEach row is one calendar day a user logged in (no duplicates).
Live SQL · Gaps islands · PostgreSQL · 18 min · Hard
SQL & Data Modeling · Original Uplift question
MMM vs multi-touch attribution
ProWith third-party cookies deprecating, a CMO asks why the team is investing in Marketing Mix Modeling (MMM) instead of relying on multi-touch...
Quick check · Attribution · 6 min · Medium
Digital & Marketing Analytics · Company-context scenario
What goes in the first line of a readout
You're writing a results email to executives about an experiment.
Quick check · Writeup · 5 min · Easy
Analytics Case & Communication · Company-context scenario
The returning-planes problem
In WWII, analysts studied bullet holes on planes that RETURNED from missions and proposed armoring the most-hit areas.
Quick check · Bias · 5 min · Easy
Statistics & Inference · Company-context scenario
A metric for 'sense of community' in Facebook Groups
ProWhat metrics would you track to determine a sense of community within Facebook Groups?
Written case · Metrics · 18 min · Medium
Product Analytics & Metrics · Reported interview question
Bias-variance trade-off
ProA model has near-perfect training accuracy but poor test accuracy.
Quick check · Modeling intuition · 6 min · Medium
Statistics & Inference · Company-context scenario
Wins in every segment, loses overall
ProYour treatment beats control in BOTH the mobile segment and the desktop segment, yet loses when you pool all users together.
Quick check · Readout · 6 min · Medium
Experimentation & A/B Testing · Company-context scenario
Which success metric reflects the user's experience
An agent solves 70% of tasks on the first attempt and 90% within three attempts.
Quick check · Product metrics · 6 min · Easy
AI Agents & Automation · Company-context scenario
Signups up, retention down — ship it?
ProA proposed change (aggressive signup prompts) boosts new signups by 20% in a test, but Day-30 retention of those users drops.
Written case · Product sense · 15 min · Medium
Product Management · Company-context scenario
Metrics after a user follows a new page
What metrics would you track after a user follows a new page on LinkedIn, and why?
Written case · Metrics for pms · 12 min · Medium
Product Analytics & Metrics · Reported interview question