How product leaders think about AI strategy, adoption, and integrating AI into their products. Strategic frameworks for the AI era from Lenny's Podcast.
“I think that where companies fail is that they're doing AI for AI's sake. They have a ton of projects that they're kicking off at the same time without a blueprint to understand how it actually worked and what their Stack looks like and they aren't treating it like a real investment, and so they don't have the measurement and the observability and the evals all set up.”
“I do have this theory of a lot of applications companies pursue because they can't measure the concrete outcome. And I feel like booking or a sales chatbot, it's very clear. There was a conversion rate right now with that chatbot with human operators and what could be conversion rate with a chatbot and certain, somehow I think it's very clear outcomes and companies are easier to buy into these solutions.”
“There's no question that fine-tuned models are the future. Models are going to be everywhere just like transistors are everywhere. AI is going to be just a part of the fabric of everything we do, but there are going to be a lot of fine-tuned models because why would you not want to more specifically customize a model against a particular use case?”
“There is something called the shiny object trap, and I'm always telling people, 'Hey, don't do AI for the sake of doing AI.' Make sure there is a problem there. Make sure there is a pain point that needs to be solved in a smart way. Once you have identified what that problem is, then reach out and try to figure out how to actually implement it.”
“AI hasn't really changed those foundational needs in many ways, and what we're finding is that AI is expansionary, and so there's actually just more and more questions being asked and curiosity that can be fulfilled now with AI. And so that's where you get the growth.”
“Information has a decay rate. Any new piece of data decays in its value to your decision-making very quickly. You've got this synthesis machine, which is this LLM thing, but if it hasn't got all that data to do synthesis on top of, it's got nothing. LLMs can only be as good as the data they are given and how recent that data is. They're ultimately like information shredders.”
“When I think about agents, I think about three things. One is an increasing level of autonomy and kind of independence that you can delegate higher and higher order tasks. Second thing I think of it is complexity. It's not a one-shot, it's build me this prototype. And then the third thing I would say is asynchronous. It works when you are not working.”
“All of the conditions, all of the ingredients for a new distribution platform to emerge are essentially happening. So I think we're at an inflection point where we're going to see this emerge really fast.”
Don't do AI for the sake of AI -- start with the problem and pain point first, then determine if a smart solution is the right approach (avoid the shiny object trap).
Marily NikaEmbody relentless improvement: great products come from a compounding effect of ruthlessly making things better every day until a tipping point is reached where users naturally adopt and love the product.
Robby SteinProduct management is fundamentally about leverage and finding differentiated value outside the building - PMs who spend 80% of their time thinking about customers, competitors, and markets dramatically outperform those dragged into internal politics and scrum execution.
Shaun ClowesNever build AI into your MVP -- fake the AI with a Figma prototype to validate the idea first, then invest in real model training once you have proven demand.
Marily NikaBe disgruntled on behalf of your users: the best product motivation comes from deeply feeling the pain your users experience and refusing to tolerate a broken or incomplete experience.
Robby SteinLLMs and AI products are only as good as the data fed into them - information has a decay rate, and 90% of the work in building great AI experiences is the data management problem of getting the right data to the model at the right time.
Shaun ClowesEvery PM will eventually become an AI PM because personalization, recommendations, and automation are becoming table stakes for every product category.
Marily NikaDeeply understand why people hire your product using the jobs-to-be-done framework, apply analytical rigor to diagnose root causes, and design for clarity over cleverness.
Robby Stein