In today’s episode, Steli and Hiten discuss the importance of arriving at accurate engineering estimates while doing product development. Typically, larger organizations that have a set process tend to be better at giving out estimates than smaller ones. Having experienced, engineering managers take care of this business function leaves the business owners free to handle what they need to—such as sales and/or marketing. Tune-in to learn how to give out accurate estimates and why knowing the value of your dollar will result in quicker, smarter and more cost effective product development.
Time Stamped Show Notes:
- 00:35 – Today’s Startup Chat is about accurately doing engineering estimates for product development
- 00:44 – Today’s focus is on software, as hardware is another ball game altogether
- 01:16 – Information on engineering estimates does not exist on the internet
- 01:26 – Hiten sent out an email to his entire mailing list quizzing them on how they go about doing engineering estimates
- 01:43 – There is no single way of doing engineering estimates, and some people do not get them at all
- 01:56 – Subscribe and be a part of the Product Habits mailing list
- 02:21 – Steli is constantly thinking of ways to better product development
- 02:37 – Larger organizations seem to be better at product development than smaller ones
- 03:15 – Analyzing all the responses, Hiten concludes that people who are doing product development the best have a process around it
- 03:41 – No generic process can be applied to all products; people figure out their own process as they develop their product
- 04:26 – For all of Hiten’s organizations, a simple process is in place for making engineering estimates
- 04:50 – It’s critical that engineers on the team understand the value of why estimates are run; cannot do prioritization of initiatives unless you understand how long the process will take
- 05:26 – Intent is not to hold engineers accountable for the estimate; main reason is to figure out what you can work on and how long it is going to take
- 06:17 – Waste of resources and rise in hourly costs if there is no estimate
- 06:53 – Go into the minute details to prepare an accurate estimate
- 07:28 – Companies like BaseCamp do not prefer to work on projects in excess of 6 weeks since larger projects tend to spiral out of control
- 07:50 – In early days, product development is generally on track if you are working with experienced engineers and using small, iterative work cycles
- 08:14 – As your customer base and your team grows, more variables kick in, and product development becomes more difficult
- 09:33 – Steli’s team is careful to not give out estimates if the project is really complex; break the project into smaller cycles if there are unanswered questions
- 10:21 – Steli’s team has gotten better at giving estimates, but they still have yet to perfect it
- 10:40 – Hiten has 3 engineering managers for each of his companies whose main responsibility is to make sure they ship on time—this allows Hiten to concentrate on the product, marketing and sales
- 11:47 – Sometimes you can be pleasantly surprised as development time will be really quick
- 12:30 – As companies scale up, there is a cultural shift and people try to better the process
- 13:00 – Sit down with the engineering team in order to remove objections about providing estimates
- 13:20 – Once you conduct some high level research and determine the probable future of the product, go to engineering and define a time which will let you determine how fast you can move
- 13:49 – Create smarter opportunities by bringing engineering into the product development prioritization process
- 14:58 – In Hiten’s organizations, any task that takes more than a day is torn apart in an attempt to make it quicker
- 15:09 – Quantifying a dollar value for the engineering cost that goes into product development will help you realize the importance of getting things done quicker
- 16:12 – Sign up on ProductHabits to learn more on this topic
- 16:38 – End of today’s episode
3 Key Points:
- The intent of engineering estimates is to NOT hold engineers accountable for the estimate; main reason is to figure out what you can work on, when you can do the work, and how long it is going to take.
- Create a culture of planning your engineering around time; as companies scale, there is a cultural shift and people try to better the process.
- QUANTIFYING a dollar value for the engineering cost that goes into product development will help you realize the importance of getting things done quicker.
Hey everybody, this is Steli Efti.
And this is Hiten Shah. Today as Steli does very often, since I’m actually putting a lot of content out there. I know you are to Steli, it’s usually about sales, mines usually about product. There’s this one thing that I’m really stuck on because I found it has one of the biggest challenges people have in product development, which is getting engineering estimates and really understanding how long things will take to build. And all of us that build software or even create hardware products have this problem and hardware is even more complicated, so we’re going to leave that for some other time maybe. Not many people I’m sure that are listening are creating hardware. If you are creating hardware please tweet at us and what not, we’d love to hear from you because that another fascinating challenge. So we wanted to do a discussion on this because I think Steli you might have some thoughts and I’d love to hear your thoughts. High level, I’m digging into this in my newsletter “Product Habits” and I’m learning a lot about how people do engineering estimates and what I find the most interesting is that this content doesn’t exist on the internet about how to do engineering estimates how to incorporate it into your product development processes and so I got very curious. I emailed my whole email list and asked them “How do you do it?” I’ve gotten dozens and dozens, almost a hundred responses so far and I just did this very recently, as per the record US podcast. So I’ve been getting a lot of responses and the short of it is there doesn’t seem to be one way that people get engineering estimates and some people don’t get them at all.
Yeah, I’m not surprised by that but I’m still fascinated by that. So, I saw the email, I’m a subscriber to products habits, if you’re not good go to producthabits.com and subscribe, you’re crazy if you’re not. And I saw that you put out that question, “Hey, how does your company do engineering estimates? Do you struggle with this I want to learn.” Anytime you do that type of thing, I know there’s going to be insights and things you’ll learn that I think are going to be valuable to our listeners and this is selfishly something I like. There hasn’t been a day in my life as an entrepreneur doing software products that I’ve not been wondering if I’m doing this right or if I’m not doing this right. It’s a struggle, I don’t know anybody who feels they have it perfectly down, it seems from a far that some of the large organizations have a pretty strong process in place that are really good on the product side of things but smaller teams, I don’t know man. Let me pull back before I ask you any questions, out of the hundred responses you said that it seems like there’s not one answer to this, which isn’t surprising, a lot of people struggle with it. Did anyone surprise you in terms of their process on how they do engineering estimates, anything new you heard or anything pattern that makes you go “oh, shit it seems like a lot of people are doing this?” Any kind of insight you could gain from these hundred responses?
Yeah, I mean the number one insight that I gained that was surprising to me and it might sound obvious but its really surprising is that the people that do it the best seem to have some kind of process around it. And what was even more surprising to your point about large companies having a process etc, the ones that are successful that are even really small, they all have a process. And the thing about that process is that its not a generic process. Like I can’t tell you “Hey the people that are successful with it always use this one process”. Its actually very, very much something they’ve developed on their own. That doesn’t man I think that there is no process everyone can use, it just means that today people are not doing anything with any structure they’re just making it up as they go and the best come up with a process that works really well for them.
So, let me ask you. You build multiple products, you work with different engineers or product teams, do you have a process in place and if so what is your process to do engineering estimates?
Our process is absurdly simple in terms of just how we think about it because I think that’s super relevant because at the different companies there are different stages, there are different engineering teams but the one thing that we spent a lot of time on a long time ago but even recently is literally making sure that the … there’s a couple of things … But really making sure that engineers on our teams understand the value of why we want to know estimates from them.
That’s the number one thing. The number one thing is the engineering team understanding that we cannot actually prioritization of our initiatives unless we understand how long they are going to take. So selling that idea, helping them understand that idea, pulling them into at least understanding your prioritization process on product development is the reasoning behind why we want estimates. It’s not that we want estimates to hold you the engineer accountable, although that is side effect and a core benefit but it isn’t the main reason. The main reason is how could we determine as a team what we’re going to work on and when we’re going to work on it without knowing how long it’s going to take. Its like saying I’m going on a road trip, Steli, I don’t know how long it’s going to take.
Well how do you decided where the stops are? How do you decide when you sleep? How do you decide anything if you’re going to go on a long road trip. If I say I’m going to go from here to New York, don’t I need to know how long it’s going to take? Don’t I need to know where and when I’m going to sleep? When I’m going to drive? That’s the most absurd thing to go on a road trip right … Unless you’re just out there and have enough free time where you can just do it, that’s totally different but that’s not how business works. Here we’re talking about product development, we’re talking about business, we’re talking about limited resources, hourly costs and all these things that waste time if we don’t get an estimate. I would hate for something and this happens every day in a company. You planned on shipping today but you didn’t, why didn’t you? It’s because someone didn’t plan ahead. Someone didn’t think about how long it would take and all these things came up once you started. Well imagine if all those things that came up once you started never came up. Well then you’d ship on time, okay, how do we prevent that from happening? Well, we get estimates, we do whatever it takes to get the estimate. And I’m not talking about estimates like, oh it’s going to take a week. I’m talking about detailed estimates, whatever that means for your organization. I’m going to share a lot more on my email list about this but like that’s the high level of it, that’s how we do it at our companies, which is we don’t actually build anything without having an idea of how long it going to take. Again, it sound absurd to a lot of people because they’re used to doing it not like that. There used to missing dates, they’re used to not having dates.
I remember even, I’m not sure if I remember this correctly so take it with a grain of salt but I remember even companies saying we’re not going to work on anything that would take longer than lets say something like six weeks or so, like three-two week cycles.
Because anything that’s a bigger project than that I so hard to manage and can easily grow out of control. Now obviously my experience has been that in the earlier stages if you work with really good engineers, really good experiences engineers, and you’re at the beginning of something building a product from scratch, I’ve hadn’t had a lot of issue with engineering estimates because people are experiencing good can fairly accurately say how long something is going to take especially if you take very small innovative development cycles in the early days. As things grow, as your product grows in complexity, as your customer base grows in complexity, as your product team grows; I have found then that it is harder to be very accurate because obviously there’s a lot more variables that are outside your control. AWS goes down or some other thing happens that unexpected, there’s more unexpected things that happen. There’s more technology you use, your stack is bigger, you’ve gotten more customers, which means more bug issues, more things, maybe some customer pounds your API in a way that was unexpected and now all of a sudden you have all these issues. A constant big list of things that need to be fixed and addressed with a complex large customer base and within a larger ecosystem versus when you start when you have nothing, no customers, no users, no nothing, you have two or three engineers and you want to build a simple version one of something. Estimating that is not as complex as estimating a big moving thing that has to respond to all kinds of outside factors as well. Not to say that at that time you have more resources you should be able to do that, still fairly well but at our company we do have a process of doing estimates but our engineering team is very careful on not giving estimates, not giving a timeline when they’re not very confident they can keep it and be very transparent and bring up hey this project we need to break it down in two cycles where we do some exploration because we have all these questions we need to figure out first before we can tell how long certain things ar going to take. So when they give an estimate they are pretty good at keeping it but a lot of time they drag their feet a little before they give an estimate right? They’ll want to buy more and more time until they figure out a lot of things to do that and we’ve gotten better at that. We’ve gotten better at breaking down bigger projects in multiple milestones so you have better ways to track progress there but I still feel like its something we’re tackling right now as a team and getting better at that. As we’ve grown the engineering team our company as grown a lot, we’ve definitely gone through phases where we have struggled with this and now we adding more and more to getting better at hitting our estimates but we’re still far from being perfect on this.
Yeah, that sound like most teams. In my case, today I have three engineering managers at three different businesses and some of them have more than one product and at all three the engineering manager understands how we do things and that has made the biggest difference for me being a non engineer, my co founders being non engineers and being able to actually speak with someone who understands why we want that because at the end of the day we’re the product people who are mostly having to figure out, doing the customer development, the interviewing, talking to customers, we’re also doing the sales and marketing between the co founders. Then the engineering manager his main responsibility, he is in three of those today. Their main responsibility it to make sure we ship on time and so then they work with the engineers and sometimes even the front end designers to figure out when we can ship and how long things will take. So we always have this one point of contact that we’re always bugging and asking about what’s the estimate? How long is it going to take? Sometimes it’s as quick as a conversation. We had recent one where it’s we dint know how long something was going to take and we knew we still don’t have any details on how we want to build it or whether we want to build it but in a quick conversation we ask the engineering manager “Hey is this hard? Is this easy? How long will it take?” He’s like “oh it’s easier than you think” because we thought it would be hard and he’s like “It’ll take less than a few hours to actually do because we’re already set up to do that based on how we’re doing x, y and z.” So sometimes we just have all these casual conversations just because we’re thinking about stuff and we want to be able to prioritize, even pre or having some kind of interface or some kind of conversation with a customer. We want to know beforehand how long it’s going to take so if you can create a culture on your product development, product engineering, engineering teams and design teams around this idea of time, things will just go much better. What I’ve noticed is that there’s a cultural shift that happens as a company scales beyond a few people of some kind of process and people wanting to make it better. In your case Steli, again in being the advisor here for a quick second, I might actually sit down with a team and be like “why don’t we do this? What are people objections about giving a concrete estimate before we start work? What are peoples objections about giving a concrete estimate before we even decide what to work on. A lot of things honestly aren’t about engineering, it’s about the product team knowing these are the 10 things we could do, or these are the high level ideas based on our research, the product landscape across all the products in the market; us talking to customers, our interviews, our sales process, wherever the future of this product goes, these are the 10 things we can do next. How do we determining, which ones we do and that’s the big question. So then going to engineering and making time and estimate a big component of that will actually dictate how fast we could move. It also creates much smarter opportunities that you might not be thinking of and what I mean by that is when you start talking to engineering this way you’re bringing them into the product development prioritization process, you’re bringing them into the process of creation in a different way than they’re used to and that helps a lot. Every I’ve had this conversation with my teams we always come up with a faster way to do something and that what we want. We also come up with an easier way because on product we’re like heres the dream, we need all this shit. Then engineers are like I can build it, it’ll take me two months but we’re like wait, wait, wait hold on fuck that how do we do it faster? Again the people I work with they understand what I mean when “how do I do it faster?” I dot mean they stay up 24 hours a day, I mean help us make the trade-offs, help us be smarter about what we build, when we build it and how small it is. I love the example you said about six weeks on base camp. One of the things that we had on our teams was, if something takes more than a day and someone scoping so that it’s a day, we’re literally tearing it apart how do we chunk it our more and it means we haven’t really thought about it because a day is a long time, a day is a solid four to eight hours of engineering time. That’s a lot, just think about the math. Let’s say on average an engineer is $50, which is low typically but just on average across the board I’d say lower but typically in the U.S. Lets say it’s $50 an hour right? Wow that 8×50, that’s $400 you wasted, if you’re saying that things costs $400. Another trick you can use, which is uncommon is can you quantify the dollar value of it? Dollar value I’m sorry no on the ROI but on the costs of it because then you can say “They’re giving me estimates that are $400 in time, I don’t know, that’s a lot of money.” I can buy a lot of things with $400.
I love that. Alright so there’s a lot more to unpack here and I’m sure there’s a lot of people that are listening that are working with engineering teams or outsourcing development and they’re like “Oh, I hate how my engineering team give me estimates, I want to learn more about it.” I know you’re going to be sharing more learnings, more processes, more stuff on this on producthabits.com; so for everybody who wants to know, more learn more sign up if you haven’t already and make sure to look out for that. I think for us for this episode this is it, we’re going to hear you very, very soon.
Yeah, as a quick side note. Out of the replies we’ve gotten on this survey there are few people who have written over 1,000 words so we’re going to analyze all that and actually share it on the list.
Yeah, that’s what’s up. It’s a very hot topic. Alright good luck .