Measuring what matters

A practical guide to impact metrics 

You’ve built a product with purpose. You’re starting to see signs that it’s working. Now comes the part many founders dread or avoid entirely: Measuring impact

But here’s the good news: it doesn’t have to take years, be overwhelming or expensive. When you’re clear about what matters and you start small, you can build a measurement practice that fuels growth, attracts funding, and helps your team make smarter decisions.

This article covers three essential pieces of the puzzle:

1. Picking the right impact metrics

2. Using Lean Impact Evaluation methods

3. Turning your data into decisions and direction

1️. How to pick the right impact metrics

The hardest part of measuring impact? Choosing what not to measure.

Companies can be flooded with data- clicks, signups, session time, survey responses. But most of that data isn’t impact. Remember, impact is about real-world change.

What makes a good impact metric?

A strong metric is:

• Aligned with your mission and theory of change

• Meaningful to your users and stakeholders

• Actionable - you can influence it through product or strategy

• Trackable with the resources you currently have

Brighteye mentor and co-founder at Newsela, Jenny Coogan, commented:

"Make sure to validate this with real users. Ask them if they understand your metric. How do they describe it in their own words? Once you achieve great impact, you're going to want to tell the world about it; make sure it's something that is easy to communicate at scale."

Leading vs lagging indicators

Leading indicators are early signals that things are on track (e.g. learner confidence, activity completion). Leading indicators are your outputs, the data you collect and can easily analyse.Lagging indicators reflect final effects (e.g. improved test scores, job placement). Lagging indicators are your outcomes, the data others can collect and analyse to determine your overall impact.

You need both, but early on, leading indicators are gold. They help you learn and adapt quickly.

Type	Example
Leading Indicator	% of users completing 3 lessons in their first week
Lagging Indicator	% of learners who improve reading level by 1 grade

Quantitative + qualitative

Not all valuable data comes from dashboards. A single powerful quote can bring your impact story to life. Quantitative indicators are numerical, while qualitative indicators are typically quotes. Both types provide valuable evidence and should be reported with equal rigor. Avoid relying solely on informal insights from friends or simply throwing in numbers without context. Use both to validate and explain what’s working (and what’s not).

A simple framework for startups

Pick 1–3 core metrics in each of these categories:

Category	Example
Engagement	% of users completing key onboarding steps
Experience	Avg. satisfaction or confidence rating
Outcomes	% of users achieving a specific goal

Keep it lightweight. What matters is starting now, not waiting for perfect. Decide on which metrics you can measure now with your current tool and which ones you need to measure later, with the help of an external research partner. Make a plan and allocate budget to it.

2️. Lean impact evaluation: MVP your proof

You don’t need a randomised controlled trial to validate your impact (you might one day, depending on your needs). 

Early-stage startups can run fast, lean experiments that produce credible signals, without big budgets or lengthy timelines.

Here’s how.

Use experiments 

You already test product ideas. Do the same for your impact hypotheses.

Example:Hypothesis: “If learners complete 5 practice sessions per week, their test scores will improve by 10%.”

Test it with:

• A small cohort (ideally a reasonable sample, in the region of 50 people)

• Pre/post survey or test

• Simple comparison to those who didn’t complete 5 sessions

You don’t always need statistical significance. You need an indication of impact.

Evaluation methods you can use:

• Pre/post assessmentsMeasure knowledge, skills, or confidence before and after using your product, using statistics for your conclusions

• Surveys with outcome questionsE.g. “Did this tool help you feel more prepared for interviews?”

• Case studiesIn-depth stories of real user journeys - powerful and credible when paired with data

• A/B tests with outcomes in mindDon’t just test what increases clicks - test what improves results and how the results shift (i.e. using statistics)

• Usage-outcome correlationLook for patterns (e.g. learners who use X feature are more likely to succeed), again, using statistics

• Third-party validation (later stage)Great when selling to large buyers, applying for funding, or scaling

Start small. Learn fast. Build the evidence base as you grow.

3️. Data-informed decision-making: making it real

So you’ve got some metrics. Now what?

The magic happens when data turns into decisions.

Build feedback loops

Set regular check-ups to review and act on your impact data:

• Monthly impact reviews with your team using central dashboards

• “What are we seeing? What’s working? What should we test next?”

• Tie data to operational decisions and product improvements 

Use impact to guide strategy

Examples of real decisions impact data can inform:

• Which feature to double down on

• Whether to pivot a product direction

• Which user segments to prioritise for success

• How to price or package your offer

• What to highlight in your investor or grant pitch

What if the data isn’t good?

That’s still useful! It helps you fail forward. Maybe:

• Your impact hypothesis needs refining

• A feature isn’t solving the problem you thought it was

• You’re reaching the wrong audience

This isn’t failure. The best impact-driven startups treat data as dialogue.

Founder mindset: make it a culture, not a chore

You don’t need to become a data scientist. But you do need to:

• Care about what you’re changing

• Get curious about whether it’s working

• Create space for your team to reflect, learn, and improve

If you make impact measurement a living, breathing part of your culture, it becomes a source of energy, not overhead.

You become a founder who has the coveted “evidence mindset”.

https://www.diplomaticourier.com/posts/the-power-of-the-evidence-mindset-in-education-technology