In today’s episode, Steli and Hiten discuss the KPIs (key performance indicators) a company should be tracking—from startup phase to the more established. Most emerging companies are clueless about which metrics to track while others are trying TOO many metrics measuring the numbers incorrectly. While it is common to make such mistakes, Steli and Hiten wish to SAVE YOU TIME and give you the rundown of where your focus needs to be. Tune-in as Steli and Hiten discuss everything metrics and emphasize the importance of goal-setting when it comes to improving your numbers.
Time Stamped Show Notes:
- 00:03 – Today’s Startup Chat is about what KPIs to track as a startup
- 00:55 – Most emerging startups are asking which key metrics to track and how to measure success
- 01:42 – As a startup grows, the struggle is there’s too much data to track or they’re tracking the wrong information
- 02:46 – Most startups don’t have many performance indicators in the early stages that are trackable
- 02:57 – For Hiten, there are several lenses that he flips on a business to determine the KPI
- 03:05 – First, is the stage of the company
- 03:24 – Second, is the type of business
- 04:08 – If a new company doesn’t know its initial product market fit and the product is working:
- 04:17 – The metrics will be around the product
- 04:25 – The key performance indicator will be on customer satisfaction
- 04:43 – As a business moves on to a later stage and the business model is more defined; the metrics will be determined by the business model
- 05:00 – Hiten’s Crazy Egg had only one feature called Heat Map for the last 12 years
- 05:14 – Recently, they’ve added 3 new features
- 05:32 – Their metrics have changed because of this
- 05:50 – The only metric that they’re focused on now is the MRR (monthly recurring revenue) for every new paying customer
- 06:02 – They want existing customers to become used to the new features
- 06:41 – Content specific KPIs are what Hiten is after more than anything else
- 06:51 – Steli shares the most impactful thing for them during their first week with Y Combinator
- 07:00 – They had a chat with Paul Grant and they described to him how their business was doing
- 07:14 – The conversation was around the measure of success and how people are getting value from the business
- 07:35 – Paul asks them if they can promise him a 10% weekly growth until the end of YC
- 07:58 – Steli was confident and he agreed
- 08:18 – Steli then asked Paul if he has his credit card with him so he can sign up on their website
- 08:29 – Paul wanted Steli to focus on one number
- 08:45 – Steli’s team were able to focus in on that one goal
- 09:15 – Prioritizing is important
- 09:50 – Challenge yourself to pick a number for your goal that is the most meaningful and closest to reality
- 10:58 – Tracking the wrong numbers won’t be helpful for your business
- 12:35 – Hiten shares the common mistakes he’s seen with tracking KPIs
- 12:42 – If you pick the wrong metric to improve, you’re going to improve the wrong metric
- 14:18 – One of the mistakes is not measuring the metric properly
- 15:32 – Compounding growth is what you’re going after
- 16:04 – MRR is naturally a compounding metric
- 16:36 – Third mistake is people having TOO many metrics
- 17:28 – A lagging indicator tells you where your goals should be
- 17:35 – A leading indicator helps you understand if you’re moving in the right direction or not
- 18:02 – You don’t want to focus your KPIs on a leading indicator
- 18:46 – KPIs are fluid—don’t change them unless you’re learning why you should change it
- 19:03 – If you keep changing your KPIs, you keep changing your goals which isn’t good
- 19:08 – Some would also change KPIs because they started wrong which isn’t good as well
- 19:15 – Others have too many advisors so they constantly change KPIs
- 20:12 – If you don’t have any measurement system or KPIs, you’re not going anywhere
- 20:45 – For any questions or feedback, shoot Steli and Hiten an email at firstname.lastname@example.org and email@example.com
- 20:53 – End of today’s episode
3 Key Points:
- Have a goal and stick with it so you’re not constantly changing your key metrics.
- Know which numbers to prioritize and focus on that number that is your achievable goal.
- Choose a number that is attainable and will improve your company’s growth.
Hey everybody, this is Steli Efti.
This is Hiten Shah.
In today’s episode we’re going to talk about what KPIs to track as a startup, how to do it right, and what mistakes to avoid when it comes to the key performance indicators that you’re tracking, the key metrics, the reports you’re generating as a startup, the data you’re looking into. You’re one of the most prolific farmers when it comes to tracking data, right? You’ve had multiple companies that are in the analytics space itself, and you’re a fairly analytical guy, Hiten, so I wanted to talk to you about this. Because I see almost every startup if not every startup that’s starting very early on has to ask themselves the question, what are the numbers that I need to keep track of? How do I measure success? What I see is that startups in all phases are struggling with this for one reason or another. In the early stages they might struggle because they’re either tracking too many things that don’t matter, or they haven’t found, or they track the wrong one metric. They pick the vanity metric, they’ll pick the “bad” metric versus a good one to have as their key performance indicator, the one number that they’re keeping the most track of. Also, as startups grow and in the later phases, there’s always this struggle of tracking too much data, not tracking enough, not having the right data internally, not having the right reports, having doubt about the accuracy of some numbers. It seems like there’s always a struggle. It’s very rare that I meet a startup or a company that has perfect control over their numbers and is very confident, and calm, and collected, and focused on it. Most startups, they track a lot but they’re kind of flustered, and they don’t know what to track and what not to track. I thought that talking about that would make a lot of sense. We’re probably going to focus most of the attention on the earlier side of things. You start a new project, and you’ve started a few new companies just recently. How do you decide what are the numbers, what are the KPIs we are going to be tracking? What’s the framework that you use to arrive at the right KPIs?
KPIs. This terminology came from back in the day in some MBA management books and stuff like that, right? It’s not exactly startup friendly, because honestly startups don’t really have many performance indicators in the earliest stages that are trackable and you could set goals on. The first thing I would say is to me, there’s a couple lenses I put on a business, especially a startup, and that would determine how you think about KPIs. The first lens is what’s the stage of the company? Is it super early stage, pre-product or early product, or is it kind of later stage? I’m just going to bucket everything later stage overall, where it’s post product with some amount of customers and/or users depending on the type of business. That leads me to my second lens, which is type of business. Every single type of business has a different set of KPIs, and by type of business I mean is it a subscription business? Is it a SaaS subscription business? Is it an ecommerce subscription business? Is it a B2B business? Are there people who are selling? Is there a sales team? Is it a consumer product? How does that consumer product make money? On advertising or through one time payments like the ecommerce shop or even like somebody who just sells ebooks and courses versus software? All these things, those two lenses are what’s really important to me when determining the KPIs in the business. For example, if you’re early stage, and you don’t know if you have product market fit, initial product market fit, and the product is working, then your metrics and your KPIs are all around, is the product working? For me, those metrics include, the key performance indicator there is mostly around customer satisfaction and/or a specific engagement metric such as retention or number of certain actions people are taking every day, every month, on average, per individual. That’s early stage where I like to focus. As a business gets later stage and the business model gets defined, that points to another point, which is, is the business model defined? Is it working or not? Because if it’s defined, then all your metrics go towards the business model, right? For example, right now I have a business called Crazy Egg, over the last until literally yesterday we’ve had one feature. It’s a heat map. Today, because yesterday this changed, we actually have more than one feature, we have three now.
Yeah, we 3X’ed the product, and as a result, and this is a business model that’s been going since 2005. This business has been one feature for 12 years.
Yeah, all of a sudden we’re like, “Holy crap!” Our metrics changed now. We were focused on different metrics before, like churn and things like that. Now we just have one metric, and that metric that we care about, our key performance indicator today, because we went from one feature to three and it’s still a business model thing we’re looking to improve. Because we already have a business model, we charge a subscription fee, is the average new monthly recurring revenue, MRR, for every customer that comes in that’s a new first time payment customer. The reason for that is it’s get older users, older customers used to new features. It’s much easier to introduce new customers that are just learning about your product and using it for the first time or the first few times, for the first trial or whatever, to start using all these features. We decided deliberately not to spend so much effort on a metric that’s based on existing users, because we know how that’s going to be. We’ve run tests for this, we know that’s going to be a little bit of a slog. Instead, all of our metrics are focused on these new people coming in, and is our MRR for these new people going up on average compared to what it’s been?
Contact specific KPIs is probably what I’m advocating for more than anything else.
I love it. I remember, and I’ve told this story before in other places, but I remember one of the most impactful things in our very first week going through Y Combinator and talking to Paul Graham, PG, was we went on our first office hour walk with him, and we described to him what our business, the startup that we were running back then, was doing. The entire conversation was revolving around, “What’s the measure of success here? How do we know if this is working? What’s the core thing we want people to be doing that shows us that they’re getting value and this is working?” Once we arrived at what that core number what, what that core thing was that people needed to do to show us that it’s really working, he turned around and he said, “Can you promise me 10% growth off that number every single week from now until the end of YC?” I think at the point where he asked, we had like 10. The number was literally 10. I was thinking, “Well, if I get one more person to do this next quick, that’s 10%, that seems pretty easy.” I’m like, “Sure, no problem!” He’s like, “All right.” Shake hands, and as I was shaking his hand I’m like, back then it was like the core number was fully enrolled members. Enrolled members back then meant people that signed up with a credit card to make donations that was based on a totally different business that we were running today. As I was shaking his hand, I’m like, “Do you have your credit card with you?” He started laughing and was like, “Yeah, I do.” I cried, “Let’s enroll you right now to our site.” He was like, “See? That’s why I wanted you guys to focus on one thing.” Besides just me that day walking through YC and getting everybody who had a credit card to sign up, what it did is it allowed us, because we had this laser focus on one number we wanted to improve on, it allowed us to make every decision based on answering the question, “Will this help us growing at least 10% next week?” That’s how we prioritized every single decision in the first three-four-five months of the company. It was a beautiful way. Now you can’t do that forever, and as the company grows, as the business grows it becomes a little bit more complex than that, but for the early days I thought that it was a beautiful way to prioritize, because you have so many things you could do, so many ideas. How do you prioritize? How do you make sure that you progress in a certain way? Also, and this has gotten old by now, but back then in 2011, I think we ended up having 25% week over week growth for the first three months. If you have like 10 or 20% week over week growth in a metric that truly matters, that’s not total BS, it makes a beautiful nice up and to the right curve. It creates the right kind of picture, the right kind of momentum. That was incredibly helpful to us, to really zero in on one thing and what matters. Also really challenging, and we’ve talked about this before, challenging yourself to not pick the number that looks the best or that’s the easiest to improve on, but to really take the number that is the most meaningful number, the closest to reality on what it means if what you are trying to accomplish, you’re truly accomplishing. I love this example with Crazy Egg, where it’s like right now the most important metric is new net MRR by new customers. We’ve tripled the product, we did it because we’re XYZ goals, and the best way for us to measure this is working is to focus on this one group of customers, which is new customers, and see how much MRR they are generating now when we offer them so much more product than the past customers. I love that. What are some common mistakes that startups make beyond the maybe not tracking anything? I don’t even know that that still happens today. I can’t remember a single case where I met a startup where they just don’t track numbers or don’t track any KPIs. Tracking the wrong ones is definitely something I see frequently. Just giving me bullshit numbers, and then when I challenge them on that, they just all fall apart. “Oh, we have over a million users on Facebook,” or, “Over a million users for our chat bot related mobile app.” Then I go, “Can you define what a user is? Is it somebody who downloaded your mobile app, and is logging in every day, and is doing XYZ access? How did you define a user?” Then they tell me, “No, user is somebody that has replied to a Facebook message or chat bot message that was initiated by our mobile app.” I’m like, “Okay, so if I get a chat bot thing on Facebook and I reply, and I’ve never downloaded your app, I know nothing about you, I don’t know what your app is called, I’m counted as a user because it’s some kind of a growth hack you’re trying to use?” They’re like, “Yeah.” I’m like, “Okay, how many people have downloaded your mobile app?” “4,000.” I’m like, “All right, mother fuckers, you can’t be walking around telling me you have a million users with your mobile app company when you really have just generated a million people to just reply to some chat bot.” There’s a lot of these BS metrics that people focus on, and I think that that’s a huge mistake people make. It’s kind of tricking themselves and others into them having more success than they truly have, but what other mistakes? You’ve seen so much on the KPI and tracking numbers side. What are the common mistakes you see again and again on like companies struggle with this or making mistakes with how they track KPIs?
Yeah, yeah. Number one, as much as I love Y Combinator, and Paul Graham’s advice to you, there’s one big word of caution I would have, which you already mentioned. If you pick the wrong metric to improve, you’re going to improve the wrong metric. The context in your business was like, look, these are people actively signing up with a credit card.
That’s generally a great metric to use regardless of what business you’re in, especially if your business is at a stage where it’s ready to accept a credit card. Right?
I’ll say something kind of interesting which is that methodology, that thinking is exactly why Facebook came out and said, “We have millions of advertisers using our advertising system.” They kept touting that metric, and the reason is because they had this attitude that, “Wait, our business doesn’t make sense unless our ad system is actually getting people to pay money and use it.” That’s not how Google thought about it, to be honest with you. They thought about it like, “We want the largest advertisers.” It’s not like Facebook doesn’t want to largest advertisers. They want to go out there and say to the market for their optics, even for their business, that “We have millions of people advertising, it doesn’t matter what size business they are.” I think that you have to pick the right metric. It’s really important, like that’s the number one thing. When you pick a goal like we’re going to increase 10% every week, make sure that it’s the right metric or you’re going to have a million chat bot conversations and 4,000 app downloads, and no one’s going to find your business, or you’re not going to be able to build a business. Number two, I’ll give a few. Number two is like they don’t measure it properly. When Paul Graham says 10% every week, let me just do math. If you have 10 right now, and you add 10%, you’re adding one, right?
Now you have 11, right?
Every week that number goes up, you have to add 1.1 in order for that to increase. Your 10% week over week goal isn’t week over week as much as it is compounded based on where you started.
Which is something people get so wrong, it really infuriates me, because what they think is, okay, let’s start with a little bit higher numbers. I have 100 sign ups, I want to grow 10% week over week, I got 10 next week, so now I’m at 110. I made it. It isn’t your goal to get like one more on top of the 10, even though in the beginning that sounds about right. It’s actually to get 10% of the total.
Right? It’s the 10% of the total. Not 10% of what you did last week. I can’t tell you how often this is overlooked and mistaken for the wrong thing, because compounding growth is what you’re going for. Honestly, I haven’t heard YC really explain this very well, and yet people tout it, and then they get confused. That’s my number two, and it’s not just YC, it’s a growth rate you’re going for. An increasing growth rate, just like your interest compounds every year, every month when you have money in the bank, and there’s an interest rate. That’s what you’re going for. That’s such an important thing that I’ve seen so many people miss, and the reason they miss it is if you have a subscription SaaS business, monthly recurring revenue is naturally a compounding metric. It’s hacked, like you just focus on that. That makes it really easy to calculate, and everybody knows what that means if they’re in a subscription business. If they don’t know, within a few minutes of launching their subscription business, they realize what that means, because the money compounds. I got 10 bucks, as long as I don’t lose those 10 bucks from that user, I get to keep having that every month. That’s compounding, that’s what I’m talking about. That’s number two. I think number three is like, I’m going to come back to this, and come back to something from earlier. I just see people having too many metrics. They have tons of KPIs, and I’m always saying I’m coming back to it because you already mentioned this, but I think it’s underscored in your story. Even at Crazy Egg, there’s a million metrics we could focus on. There’s so many metrics even that are sort of indicators of our average new MRR, such as people using these features during their trial, which will lead to an increase to average new MRR for us, obviously. We don’t count those as the main metric we’re trying to improve, and there’s a framework you can use called leading indicators and lagging indicators. A leading indicator is one that tells you you’re going in the right direction. I’m sorry, a lagging indicator is one that tells you on a high level that you’re actually, that’s where your goal should be. Usually it’s like revenue or churn or something like that. A leading indicator is an indicator, a metric, a KPI that helps you understand whether you’re moving in the right direction or not. For us, if people are using more than one feature in our product, that’s a leading indicator to them paying more money than our existing users, as an example. There’s this idea of leading and lagging indicators that gets really important as you get more advanced around understanding KPIs, because you don’t want to focus your main KPI on a leading indicator. You actually want to focus it, in an ideal world, on a lagging indicator initially. Then all these leading indicators are what you discover, and then you might flip the switch and say, “Oh, if we can get everyone using three features, average new MRR goes up, let’s just change our KPI that we focus on to getting feature adoption, and getting every single user.” Right now, let’s say on average, out of our whole user base, people are using 1.5 features. We want to increase that to three in the first seven days of using our product. That’s the kind of thinking you can use once you start learning more. I think, and this probably goes to my last point about this, which is these things are fluid, but don’t change them unless you’re actually learning why you need to change them and learning what the better KPI should be. It’s not like you should change them often. If I see someone changing their KPI every two-three weeks, that’s bad. It’s bad for two reasons. One, they keep changing it, they’re changing their goal, which is bad in a lot of cases, or two, they didn’t start in the right way by thinking of context when they determined it early, or number three actually, they’re listening to too many advisors. That’s common, unfortunately. Come talk to Steli or I, because we’re definitely better at cutting through crap like that, but in general you don’t want to change them, but if you learn something new about your business, you’d be stupid to be so rigid that you’re not going to change it. It’s likely even in our business, as we move and learn about this one metric, and what the leading indicators are that are going to increase our average new MRR, we might change it. We are not likely to because we’ve done a bunch of work to determine this one. That being said, I could imagine three or four months from now, once we get to a good place there or have a new learning, we might change it because that leading indicator is actually the key thing for us that we learn about. That’s kind of four or five tips that I have around metrics. These are really core. I did run an analytics company for a while, and still probably run one depending on how you look at it. This is not much more complicated than that, that’s why the KPI topic’s really interesting to me, because if you don’t know where you want to go or you don’t have a metric, you don’t have some measurement system and are focused on one metric ideally, then you’re not going to get anywhere. This is by team if you have a larger company, but usually a company has one metric they focus on, and the teams are focused on metrics that relate to that one.
Sweet. I think this was both powerful and beautifully put. All right, there’s not much more than that. If you need more help or feedback, you can always reach out to us, HNShah@gmail.com, Steli@Close.io. This is it for this set, so we’ll hear you very soon.
Yeah, bring us the math. See you.