Question 1

What are the key takeaways from Itamar Gilad's episode on Lenny's Podcast?

Accepted Answer

1. Opinion-based development is the root cause of massive product failures -- even at companies like Google with brilliant people, as demonstrated by Google+ consuming 1000+ engineers for years before being shut down. 2. The confidence meter helps teams honestly assess how strong their evidence is, ranging from 0.1 for gut feelings and thematic alignment to 10 for fully validated experiments -- most teams overestimate their confidence. 3. North Star metrics should measure value delivered to users (like WhatsApp's messages sent), not just business value captured -- and metric trees connecting these create alignment across the entire organization. 4. Teams should fake it before they build it -- Wizard of Oz tests, smoke tests, and fake door tests can validate ideas with zero code, as demonstrated by Gmail's tabbed inbox validation using manually sorted HTML facades. 5. Outcome roadmaps beat release roadmaps -- committing to launch a specific feature by a date kills learning, while committing to achieve an outcome preserves the team's ability to discover the right solution.

Question 2

What does Itamar Gilad say about evidence-guided and experimentation?

Accepted Answer

"Google+ was my first experience at Gmail. We went on and we launched a series of features in Gmail for a couple of years and none of it worked. It turned out people actually didn't need another social network. Eventually in Gmail we rolled back all the Google+ integration and Google+ itself was shut down in 2019." - Itamar Gilad "In hindsight I was asking and saying, why was this project so different? And I think the reason is that we didn't have that much confidence in our opinions. We had opinions, we had ideas but we didn't just go all in and just let's build it. We actually used an evidence guided system." - Itamar Gilad "Steve Jobs, he just brainstorm in his kitchen the iPhone and then just told the team to build it. That's the story Steve Jobs told but it's not the real story at all. The iPhone has actually a story of discovery, of trial and error, multiple projects, most of them failed. Steve Jobs was the architect. He was actually against it for a while but over time as he saw the evidence he was able to piece together something that was very useful." - Itamar Gilad

Question 3

What are Itamar Gilad's views on evidence-guided, experimentation, metrics?

Accepted Answer

Itamar Gilad: "In hindsight I was asking and saying, why was this project so different? And I think the reason is that we didn't have that much confidence in our opinions. We had opinions, we had ideas but we didn't just go all in and just let's build it. We actually used an evidence guided system." Itamar Gilad: "I think evidence is so powerful, if you come with data, if you run a secret experiment and you come back and you show them you usually get one of two results. Either they get extremely mad at you and you need to start polishing your resume. But the more common case is they're pleasantly surprised. Even Steve Jobs given evidence was willing to flip."

Itamar Gilad

Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft)

TRANSCRIPT

Itamar Gilad

Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft)

Itamar Gilad

Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft)

TRANSCRIPT

Itamar Gilad

Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft)