Building Confluent: From Open Source Side Project to Public Company | Jay Kreps (Co-Founder and CEO)

Why It Matters

Kreps’ story demonstrates how engineers can transform open‑source innovations into billion‑dollar businesses by mastering cross‑functional leadership and aligning go‑to‑market tactics with product realities, offering a roadmap for tech founders and investors alike.

Key Takeaways

• Transitioning from engineer to CEO demands mastering diverse business disciplines.
• Early customer wins came despite an unfinished product and limited features.
• A single blog post accelerated Confluent’s open‑source adoption dramatically.
• Betting on a cloud‑native offering faced internal resistance but proved pivotal.
• Go‑to‑market strategy must align with product nature, not copied playbooks.

Summary

In this interview, Confluent co‑founder and CEO Jay Kreps recounts how a side project at LinkedIn evolved into a publicly traded data‑streaming company built around Apache Kafka. He outlines the pivotal moments—from the open‑source release to landing the first enterprise customers while the product was still rough around the edges, and the bold decision to double‑down on a cloud‑native offering despite internal skepticism.\n\nKreps emphasizes the steep learning curve of moving from software engineer to chief executive, noting that a CEO must grasp roughly 80% of each functional area—marketing, finance, product—to make directionally correct decisions in a “fog of partial understanding.” He describes how a single blog post generated more traction than years of engineering effort, and how early sales were won and lost as the team wrestled with go‑to‑market tactics that didn’t fit the product’s technical nature.\n\nMemorable anecdotes include the internal politics at LinkedIn when the nascent data‑pipeline tool threatened existing owners, the reality that many early customers quit, and Kreps’ admission that “if we can’t express why this is exciting, it’s probably not going to be successful.” These stories illustrate the messy, iterative process of scaling an open‑source project into a commercial enterprise.\n\nFor founders and investors, Kreps’ journey underscores the importance of adaptable leadership, aligning market strategy with product architecture, and leveraging community‑driven momentum. The Confluent case shows that open‑source credibility, combined with a decisive cloud pivot, can create a defensible position in the rapidly expanding real‑time data infrastructure market.

Original Description

Jay Kreps is the co-founder and CEO of Confluent, the company built around Apache Kafka — the open-source data streaming platform he originally built while at LinkedIn. In this conversation, Jay shares his full journey: how Confluent grew from a scrappy group of engineers with no go-to-market experience into a publicly traded enterprise software company. He makes the case that the difference between what a company can do, and what it must do, is one of the most underrated building levers; illustrated through his years spent pushing Confluent towards a cloud product, in the face of widespread opposition.

In this episode, we discuss:

• Why moving from software engineer to CEO requires almost an entirely new skillset

• The product marketing pyramid Jay built to explain Kafka to the world

• How Confluent bludgeoned its way to a cloud-first business when the early product was “embarrassing”

• The critical difference between what a company can do and what it must do

• What keeps scaling companies from becoming "Chipotle”

References:

• Amazon Web Services: https://aws.amazon.com/

• Apache Kafka: https://kafka.apache.org/

• Benchmark: https://www.benchmark.com/

• Confluent: https://www.confluent.io/

• Jun Rao: https://www.linkedin.com/in/junrao

• LinkedIn: https://www.linkedin.com/

• McKinsey & Company: https://www.mckinsey.com/

• MySpace: https://www.myspace.com/

• Neha Narkhede: https://www.linkedin.com/in/nehanarkhede

• Oracle: https://www.oracle.com/

• Red Hat: https://www.redhat.com/

• Snowflake: https://www.snowflake.com/

Where to find Jay:

• LinkedIn: https://www.linkedin.com/in/jaykreps/

• Twitter/X: https://x.com/jaykreps

Where to find Brett:

• LinkedIn: https://www.linkedin.com/in/brett-berson-9986094/

• Twitter/X: https://twitter.com/brettberson

Where to find First Round Capital:

• Website: https://firstround.com/

• First Round Review: https://review.firstround.com/

• Twitter/X: https://twitter.com/firstround

• YouTube: https://www.youtube.com/@FirstRoundCapital

• This podcast on all platforms: https://review.firstround.com/podcast

Timestamps:

01:18 Making the leap from engineer to CEO

03:33 The 80% rule: what a CEO actually needs to know

04:54 Scaling different business disciplines

09:31 How Confluent’s story began in LinkedIn

12:13 The growing need for scalable data tech

13:37 What the early Kafka product looked like

16:38 Kafka’s underwhelming open-source launch

18:38 The blog post that accelerated Kafka’s adoption

20:16 Why so many marketing messages fail

28:08 The decision to build Confluent

34:24 Planning to fundraise before building the product

39:19 Confluent’s early years: Tough product decisions

47:07 The underrated growth lever question for companies

55:46 Why founder optimism is an overrated trait

1:00:29 What should founders give up as they scale?

1:02:47 Why people become trapped in a failure mindset

1:08:33 The Chipotle problem: Losing excellence at scale