I had outlined plans and aspirations at the previous year’s dinner. When I came to the podium at the end of 1991, I thought about everything the 126th board hadn’t accomplished — and all the political capital expended for mixed returns. Still: we published every day of the academic year despite two blackouts. We didn’t get sued. None of the corrections we had to run were catastrophic. We published at least a few good stories.

My successor called me to the stage. I received respectful applause — more due to the position than the man. I thanked each member of the board. The room exploded when I reached Adam’s name. The staff understood exactly the energy and creativity with which he had attacked the role of Opinions Editor. Despite our differences of, well, opinion, I remain grateful to Adam for his contributions to the paper, even decades later.

Adam and I ran into each other again at McKinsey. By then he had decided against litigation as a vocation and was already developing into one of the most rigorous thinkers I know on data and analytics. Senior roles at JPMorgan Chase, DirecTV, Zurich Insurance, and Point72 followed — teams as large as 170 people, enterprises with up to 60 million customer relationships. He now advises boards and senior leadership teams, teaches in Brown’s EMBA program, and runs an annual forecasting contest, which I fear to enter.

Everything we do — every business decision we make — implicitly rests on a forecast. Which is why I needed to get Adam’s view on what AI means for data, analytics, and the practice of forecasting itself.

Three things struck me as particularly worth your time.

First, the B2C-vs.-B2B gap in analytics is real — Adam puts it at twenty years — but the cause is not what most people assume. It is not that B2C data is cleaner. It is that B2C businesses have enough observations to find statistical regularities. A million customers generates surprises. Seventy-five institutional accounts does not. AI changes some of this at the margin, but domain expertise — knowing which variables to featurize, which hypotheses to form — remains the scarce resource.

Second, taking payoff curves into account as you thinking about your career and your job. Adam’s formative experience was as a litigator: limited upside, catastrophic downside, a profession structured for you to lose. He now teaches his kids to think about every career choice as a payoff curve before accepting it. Most technology managers are running the same analysis implicitly every time they commit to a timeline or decide whether to invest in redundancy. Making the curve explicit demystifies the decision.

Third, Adam’s argues that the manager who has actually formed a probability — even a rough one — about whether a project will come in on time is making a different kind of bet than the manager who just called it a plan. The first manager will be better calibrated over a portfolio of decisions. “It matters to be 53% right instead of 52% right” is not a hedge fund insight. It is a description of how operational advantage compounds.

Thanks for reading Prosaic Times — share it with a friend!

Introduction: from College Hill, back to College Hill

James Kaplan:** Welcome to another Prosaic Times video podcast. We have with us Adam Braff, who I’ve known since about 1990. I’ll let Adam introduce himself and talk a little bit about his journey, and then we’ll start to dive deep into forecasting and what AI means for forecasting.

Adam Braff: Thanks for having me, James. I guess if I could describe what I do now, it is not strictly about forecasting. It’s helping corporates and investors figure out what to do with data and analytics, and increasingly AI. So some of that is descriptive analytics and trying to figure out how many widgets are we selling and why, and some of it is predictive, and how many widgets are we gonna sell next year and how do we make that number go up?

So predictive does come into the work that I do

James: Yes, and we all want to hear about the Narcissist forecasting contest, which I am proud never to have taken part in — because I’m sure I would lose badly.

Adam: Nobody loses in the contest. You gain knowledge, you get smarter. And even if you’re gunning for first place, which is fair to do, it actually influences your strategy. It’s like when you’re entering a March Madness bracket, do you pick a lot of weird long shots, or do you try to be as accurate as possible?

I try to persuade people to just get more accurate year over year and have the least error overall in their forecasting. So nobody loses in this contest. They get a score, they can get coaching and feedback from me for free, and then they try to get better every year

James: Okay. So tell us a little bit about your journey

Adam: We graduated into a recession, if you may recall.

James: Was not fun

Adam: That was the reason why my wife and I — then girlfriend — decided to go to law school. It was a good way to hide out from the economic storms of the early ’90s, and that was kind of a mistake, I would say. It was a big detour in my career, but I ended up going to law school and then practicing as a litigator at a big firm in DC for three years, really not enjoying any of it, and trying to find a way out.

And as it happened, I had put myself into a bit of a corner because the world doesn’t really need litigators. There aren’t a lot of obvious ways for litigators to turn into some kind of beautiful butterfly. So I took advantage of the fact that it was the dot-com boom, the first dot-com boom happening at the end of the ’90s, and ended up getting a job at McKinsey, and it was a fresh start for me.

I was 29 years old, and I just said, “Let’s, let’s start again at the bottom, and this time pretend I had gone to business school instead of law school,” I got my mini MBA from McKinsey, which consists of three weeks of getting bombarded with strategy ops, marketing, and org stuff, and then ended up staying at the firm for a lot longer than I thought I would be there.

I ended up doing work that was really what we would now call data analytics AI. At the time, I think we called it big data. It was how do companies use larger data sets than would fit in Excel and creative new kinds of data and A/B testing on their customers and calculations of customer lifetime value?

How do consumer businesses figure out how to do more of that stuff in order to attract and retain the most valuable customers? And that’s something you can do in a lot of different industries, as it turns out

James: Did you go to Mini MBA in Kitzbühel or do you go someplace else?

Adam: I went to Kellogg. I had a bunch of trainings in Kitzbühel, but I was in Kellogg freezing my ass off for three and a half weeks in January of that year

James: I’m sure it was warmer than it was in Kitzbühel and easier to get to

Adam: Exactly. It was nice.

So I stayed at the firm for 10 years. I was a generalist partner in the DC office. None of my clients were really government or particularly local to DC, so I was traveling a lot. Had three kids in that span of time, and at some point I said, “All right. It’s time for me to get a real job and figure out what I want to do in the real world.”

So I left in 2009 and had a few different offers — some in New York, one in Vegas, which my wife vetoed — and ended up going to JPMorgan Chase to become their head of customer data and analytics. That was for the Chase consumer businesses: figuring out what to do with information about customers who might have a Freedom credit card and a Chase mortgage and a Chase checking account.

What do we do to put all that information together and help the bank make better risk decisions, better anti-fraud decisions, better marketing decisions? I did that for a few years and then got recruited over to DirecTV to run business analytics there. That was different because there was an existing team of really good analytics people doing the same work but for pay television — a much simpler business, with some interesting lifetime value drivers.

James: Interesting. And then you did an insurance company for a while, and you’ve also recently been teaching at Brown.

Adam: DirecTV got bought by AT&T, a bunch of us bailed out. I ended up going to Zurich Insurance and ran data and analytics there. I got a taste of the IT world by reporting into the chief operations and technology officer — which let me see what it’s like to actually build the stuff and not just give the requirements.

Then I spent three years at Steve Cohen’s hedge fund as head of data sourcing and strategy — data acquisition, essentially. I’ve been independent since 2019. Hung out a shingle that year and started teaching at the same time.

I began at NYU, created a class called Business Analytics and Data Visualization, taught different variations of it for a while. Then we moved to Providence, where I’m sitting now, and I’ve been teaching versions of that class at Brown.

Is B2C 20 years ahead of B2B in customer analytics?

* B2C is roughly 20 years ahead of B2B in customer analytics — not because the data is cleaner, but because there are simply more data points to work with.

* The analytics opportunity in B2B looks fundamentally different: customer targeting and Salesforce effectiveness rather than churn modeling and cross-sell.

* AI and agentic data ingestion could narrow the gap — converting unstructured B2B data into analyzable signals.

* Domain expertise still matters: knowing how to featurize variables (competitor footprint, not just state) is what separates the old hands from the clever prompters.

James: Let me ask this question. You’ve done a ton of stuff in what I’d describe as the B2C world, and it’s been my observation that B2C is light years ahead of B2B on analytics — just because the data is cleaner. And I hypothesize one of the interesting things about gen AI is you can now clean up the data, correlate it, convert unstructured text into structured data you can analyze.

So I was wondering if you could comment on how far ahead the B2C world is compared to B2B on analytics, and to what extent might gen AI or agentic data ingestion and cleansing help the B2B world catch up?

Adam: It’s like 20 years ahead[1], but probably for a different reason than the reason that you set out. It’s more that there are more data points to measure in the B2C world. And so if you’re gonna list all the use cases, if a private equity firm comes in and buys a consumer business or buys an industrial business, all the things you’re gonna go work on, there are gonna be analytical big data things to do over here that have to do with acquiring more customers and retaining your best customers and figuring out who those best customers are and cross-selling and stuff.

Over here, you’re like optimizing a factory or something, right? So to the extent that there’s data stuff, and very little of my career has been over here, right?

James: Right

Adam: You’re gonna be getting information from things — what was briefly called the Internet of Things — and getting instrumentation data from over there.

So big data definitely exists, and there are use cases for it, and I sit on councils with people who do this stuff. It just looks very different from the kinds of big data and analytical stuff that I did, and that my father did before me. He’s retired now, but he was doing data-based marketing back in the ‘70s and ‘80s.

Second-generation data and analytics — something that hadn’t existed before.

James: I was thinking about this in the context of customer analytics. If you have a million households in a B2C business versus seventy-five or a couple hundred corporate institutions — which have complicated internal dynamics — the N question becomes obvious. You don’t typically have many decision-makers when someone decides to buy cable services. Maybe one, maybe two. Whereas in B2B procurement, there could be twenty decision-makers in the process, and they all interact with each other.

Adam: The analysis you would do in B2B Salesforce effectiveness would have a lot more to do with customer targeting — going out and grabbing more information about your target, using some agentic tool to pull that and put it somewhere you can analyze it and make good decisions.

The reason the million-customer B2C business got ahead is that a lot of analytics is looking for patterns, for statistical regularities. If you can determine that customers in this region are stickier, or that customers who came through this channel spend more, there are just many more ways to do those analyses — with genuine comparative advantage over an executive doing everything by intuition.

There’s at least a fighting chance for the data to tell you a different story. You think the higher-priced product is better, but the customers don’t stick around as long once you control for everything — and therefore you shouldn’t be charging more or assume that’s the right answer.

You’ll come up with intuitions like that on the B2C side. You won’t on the B2B side because you don’t have enough N to surprise anyone with that kind of finding.

James: Tell me if this is right. The more independent variables you’d have to take into account, the larger the N you need in order to get a relevant statistical sample.

Adam: I think that’s right. And I don’t want to make too much of statistical significance here. A lot of what you’re doing in B2C consumer analytics is descriptive analytics — a BI dashboard, not a giant regression model with a lot of independent variables. You want to slice your data one or two dimensions at a time and just look at the graph. Which numbers are higher? Is it this product, this region, this group of call center agents? That’s what lets you drill in another level and understand what’s happening.

You can have lots of dimensions and measures that you think of as independent variables — you’re just going to be doing them a little at a time, flexibly drilling in to find the hotspots. The regression is just one part of that.

James: What you’re saying is there’s a lot of power in just being able to do Pareto analysis — being able to look at your churn by segment, by size of spend, by region, by product mix.

Adam: Yes, exactly.

Adam: Right, exactly. So we were at DirecTV, churn was the number one thing. It was the era of cord cutting and cord nevers. probably a lot of your viewers have never thought to buy a monthly pay TV subscription ’cause it’s all, they may get it through a streaming product, but not through, a cable

James: every now and again, I will try to explain the concept of channels to my kids, and they’re just confused by it.

Adam: Yes

James: I try to explain that in New York in the 1970s you had channel 2, 4, 5, 7, 9, 11, and 13, and that was it. They accuse me of being old.

Adam: The reason I bring up DirecTV is not to make you feel old. It’s that the main problem we were trying to solve was why churn was so bad and how to make it better. And if you think about the ways to slice churn, first you want the churn rate — within any given customer population, what percentage are canceling over a month or a year?

When you start slicing it, the intuitions you might have — is it by demographics, age and gender? — that’s not what matters. What matters is things like which customers are rolling off a $5-a-month promotion this month. Those turn out to be the vulnerable ones.

Or which customers are in the Fios footprint or the Comcast footprint — does that make a difference, and is it trending differently over time? You need the data to do that kind of work.

James: So what I hear you saying is: the availability of the data, obviously, but also the person who can develop a hypothesis about what might be a relevant driver — competitor footprint, for example.

We needed that hypothesis generation in a world where analytics were expensive — someone had to go do the cuts in a spreadsheet or via SQL. In a world where analytics are cheaper, how much of it is “let’s just run all three hundred cuts and see what it tells you,” versus using a model to discern which cuts might actually have real differences?

Because I think both you and I have done an analytical cut a million times, said “region has no impact,” “size of spend has no impact,” and then — “overlap with competitive footprint, that has the impact.” You have to do nineteen analyses before you land on the twentieth that’s meaningful.

Adam: Domain expertise and taste matter because you have to figure out how to featurize these variables. Region by itself, cut generically into US states or metropolitan areas, gives you a boring answer.

But if you join that data up with another data set — this is the Comcast footprint, this is the AT&T pre-merger footprint — suddenly you’re testing something real. Knowing how to featurize the data to test hypotheses efficiently is still very valuable.

And AI tools, if you coach them correctly and bring enough taste and judgment to it, can let a sufficiently senior person test a much larger number of hypotheses. They’ll also surface a lot of spurious answers along the way.

A brand new data science graduate would have a hard time hitting the ground running in one of those businesses. Even a very clever prompter isn’t going to get to the right answer the way an old hand would.

James: Creativity and context matter here. You make the point about competitor footprint. Who knows — state may be a weak signal, but population density may be a much stronger one. You get a weak signal from New Mexico and Wyoming being different from New Jersey and Rhode Island, but when you really dig in, it’s the Providence metro versus less densely populated parts of Rhode Island that’s driving it.

Adam: And you’re keeping track of the time dimension. Things change over time. Everything changed in COVID. You might decide to cut off your analysis at the post-COVID era, or account for two competitors merging. This happened constantly with banking analyses.

One of the analyses I used to do at the firm was on customer experience. We ran a big annual panel surveying people about their banks — and it was always difficult because the banks kept changing. The American Customer Satisfaction Index, which tracks these things longitudinally, has the names of ancient banks nobody’s heard of — Chemical Bank — that we now know as Chase.

There is genuine value in domain expertise that AI cannot yet replicate.

James: What I should do one of these days is a quiz — Chemical Bank, Manufacturers Hanover, First USA — and you have to guess which bank they’re part of now.

Adam: The world’s most exciting quiz. I agree.

The debate on virtual panels

* Synthetic data is valuable for prototyping and pedagogy — stuffing a dashboard mockup with plausible fake data is now standard practice.

* Where it breaks down: using synthetic panels as a substitute for real market research, especially for purchase-intent questions. There is no signal like that in the model.

* The motte-and-bailey risk: vendors pitch synthetic panels as research-grade but deliver something closer to a prototype illustration.

* James’s CIO/CTO panel is more useful for testing message clarity and resonance than for predicting buying behavior — a meaningful distinction.

James: Never said I was cool. What do you think about virtual panels? Particularly in the B2B space, I’ve started playing with using agents to simulate the decision-making of certain people. Valid? Less than valid? Intriguing?

Adam: I remain skeptical of the whole field of synthetic creation. On the one hand, I use it for pedagogical purposes where I’m creating a data set to illustrate a point to my students. It’s great for that, especially if you’re gonna sprinkle in personally identifiable information, which you don’t wanna do with actual humans.

I use it to make, prototype dashboards. This is something that has become very common now, not just me, but many people since the Claude Code stuff really got going in earnest back in January. I’m sure you’ve done this and you’ve seen it, but many people are mocking up prototypes very quickly, and stuffing them with synthetic data is a pretty good idea, right?

To give people a sense of the art of the possible there.

James: A couple of weeks ago I created the data for a change management program at a major financial institution — all dummy data, but realistic enough to demonstrate something. That would have taken weeks to do manually.

Adam: Incorporating randomness and signal to make it feel like real data is genuinely hard. Agreed.

Now, on panels: I’m doing a lot of consumer insights work these days, back on the B2C side. You want to survey 500 Americans about something — and it’s getting harder and harder to find real people who aren’t robots or speeders or cheaters. So people are pitching synthetic panels: 500 simulated respondents. Why not?

Here’s why not. The whole point of a real panel is the signal — what actual people in the world want, the clusters that form, the principal components that emerge. It’s not just big spenders versus quick deciders; there’s a third dimension that actually explains the variance.

Trying to derive all that from synthetic data seems crazy to me. I’ve had a couple of conversations with these vendors, and I worry that what they’re pitching sits somewhere between the prototype mockup work we were just celebrating and the actual market research my clients need to make real decisions.

It’s a motte-and-bailey: they pitch it as research-grade but it’s really only as good as a prototype illustration. I’d love someone to steel-man what synthetic data is genuinely good for beyond that.

James: For the blog, I created a panel of five hundred CIOs and CTOs. I gave each member of the panel a persona — this person is the CIO of a pharma company with five hundred million dollars in IT spend, this person is the CIO of an asset manager with a billion dollars in IT spend. Then I used the OCEAN framework to create psychological profiles.

Adam: Mm-hmm.

James: Each individual has a job, a title, a company, and a personality profile. I run each draft through them. I wouldn’t bet the farm on the scores, but the quotes I get back are really useful, and I’ll go through about seven iterations. Part of me thinks it may be less helpful for predicting buying intent and more helpful for testing marketing messages — because that’s where large language models are genuinely good.

It’s hard for them to predict whether somebody will buy, but they may be better at understanding whether a message is more or less understandable, more or less compelling. Does that make sense?[5]

Adam: A little bit. You used the words “testing marketing messages.” I think it’s more like — in this case you’re testing your blog. Generating responses that would go outside of what you and I would naturally brainstorm on, how people are thinking about things.

James: It’s pretty good for that. I don’t workshop my blog posts with people who might be more emotionally invested than I am.

Adam: We’ll have to look at your Ocean scores later. We’ll compare them offline.

I like the idea of computers generating things. It’s a large corpus of material — you sift through it, see what’s interesting, throw away what’s not. You and I have been around long enough to tell the difference. You generated it cheaply, so the discount rate on discarding it is low.

But that’s not really what market research is. Qualitative research is different: a focus group has real human beings who can say unexpected things. Small N, unpredictable. But once in a while someone says “I never knew you could use the product this way” — and maybe only 1% of people use it that way, but knowing that might spark a brand campaign.

So I’m in favor of the tools for idea generation and checking your work. What I’m skeptical of is using synthetic panels for commercial due diligence. You’re deciding whether to acquire a BI tool or a security product, and you’re going to rely on numbers saying 20% of synthetic CIOs might spend more than USD 10MM on it.

That doesn’t seem right to me. I don’t think there’s signal like that in there.

James: My take is it’s more a part of the chain. It doesn’t preclude or replace going and doing the interviews. But you can’t test and re-test and re-test with a live interview set. You can workshop messages or hypotheses and then go confirm with actual human beings.

Adam: This is a good segue into forecasting.

All business is forecasting

* Every business decision is implicitly a forecast — a bet that one management action will outperform another.

* The distinction is sharpest for investors: a binary buy/short decision forces the forecast into the open. Operational managers face the same bets but rarely frame them that way.

* Tetlock’s superforecasting research showed that structured generalists — triage tractable problems, decompose, work as a team, ensemble independently — beat CIA analysts with classified intel.

* The Brier score disciplines probability estimates: being 52% right vs. 53% right sounds trivial, but at sufficient scale and leverage it is everything.

James: Go there

Adam: Due diligence is a form of forecasting. You’re trying to predict what the financials of this business are going to look like at current course and speed — and what they’ll look like if we apply our private equity magic on top of it.

All business decision-making is implicitly a forecast: a bet that this management action will produce better results than that one. The point is sharpest when you’re an investor at the margin — do I buy because it’s going up, or short because it’s going down?

James: Very binary — buy or do not buy — as opposed to operational decisions, which are more continuous in nature.

Adam: Exactly. When you’re thinking about numbers and where they’re going in the future, it probably helps to be good at forecasting more generally. And this is what Tetlock demonstrated, decades ago with IARPA and these defense intelligence agencies.

For podcast viewers and listeners who aren’t familiar: Tetlock is a professor at Penn who has been studying forecasting for a long time, and came out with a very influential book called Superforecasting, where he packaged the lessons of groups of generalists who — the legend has it, and I think I’m giving a fairly accurate version —

were able to forecast world events — will this world leader be deposed or die — better than CIA operatives on the inside who had all the intel collected expensively by our intelligence apparatus.

And so these generalists —

James: Did the generalists forecast the Knicks were gonna win the championship this year?

Adam: Nobody predicted that except my sons, who believed in it. And so it happened.

Tetlock codified the rules of what it means to be a superforecaster. These people were really good at triaging problems by tractability. Some things are impossible to forecast — the Knicks. Some are too easy — my soccer team, Tottenham Hotspur, doing badly. The Jets are a better example of the tractable middle ground: will they make the playoffs, will a company beat earnings, will this entity win a major award?

Allocate your time to the tractable problems. Decompose them into constituent parts. Work as a team, but have people develop independent points of view first, then bring the theories together and fight about who has the better argument. Ensemble different independently-made forecasts where possible.

Those are the Ten Commandments of superforecasting that Tetlock codified in 2015.[2] I started my forecasting contest that same year.

I read the book and launched the first Narcissist forecasting contest in 2016 — The Narcissist being a long-running annual newsletter I’ve produced since 1992. Twenty-five binary propositions about the world, each either going to happen or not.

Everyone predicted the likelihood of each from zero to 100%, including, at the time, whether Trump would get elected — which everyone gave very low numbers to.

The scoring mechanism is a Brier score. If I predict 40% for Trump getting elected and he’s not elected, the truth is zero: 0.4 minus 0 is 0.4, squared to exaggerate the impact of being far off, giving me a 0.16 Brier. If Trump is elected, my error is 0.6, and the square is 0.36. A high number is bad. It’s like a golf score.

James: Here’s a business question. Can you predict a probability, or do you only state one? You can predict whether a candidate is elected or not — but we’ll never know the true likelihood of that election at any given moment. You can estimate a likelihood, or you can predict an outcome. But can you actually predict a likelihood?

Adam: It is a bit of a metaphysical question. In a sense, all these predictions are wrong — everything goes to zero or one. There are people who look at the whole enterprise and say it’s stupid. There’s no difference between .4 likely and .6 likely. Just make the call. I’d call those people extremists.

The other camp — generally called Bayesians — believes that we don’t know ex ante whether a thing will happen, and that there is genuine utility in knowing whether something is 60% or 40% likely. The value is in the repetition. You make predictions over and over again.

The clearest illustration is betting markets. In 2016, Trump was trading at under 20% on the eve of the election. Nate Silver had him at 28% — out of consensus to the high side. If you’d looked at the pot odds and said “I can buy Trump contracts at 20 cents because Nate says 28,” you would have done well. Even though you thought it was less than 50% likely, you thought it was more likely than the market priced it.

I’m a Bayesian. Being able to tell a 52% from a 48% likelihood is useful because we make a lot of bets. You’re making them every day when you’re running a business. Being slightly more right and less wrong compounds.

James: Everybody I’ve known who’s managed money for a hedge fund is a fanatical Bayesian. There’ll be traffic, but it depends on three things that might drive more or less of it — let me think through the prior and posterior. Which is interesting, though it gets a bit exhausting.

Adam: One of the most common approaches in a long-short public equity fund is calling the quarter — is this stock going to beat or miss earnings? More precisely: beat or miss the whisper number on whatever KPIs matter.

The hit rate is 52%, 53%. It’s much better to be right 53% of the time than 52% — that’s how they make their money. They’re repeated gamblers doing it at a very high level, trying to maintain that little bit of edge in a zero-sum world where everyone else is trying to do the same.

It’s hard. But it matters to be 53% right instead of 52% right.[4]

James: Because a hedge fund portfolio manager who’s right 53% of the time with sufficient leverage is enormously value accretive. Fortunes can be made with a fifty-three/forty-seven split, plus leverage. Operational managers who are right 53% of the time tend to get fired. And in the technology world there’s the expression “all day, every day” — you can’t be wrong in one direction.

Actually, one of the interesting things here is that in some cases the hedge fund payoffs are symmetric. On the other hand, in an operational world, the payoffs are often highly asymmetric. Could you reflect on that — applying the insights from superforecasting to an operational environment for people who have to make decisions every day with sometimes highly asymmetric payoffs?

Asymmetric payoffs and career risk

* Hedge fund payoffs are roughly symmetric: being 53% right vs. 52% right is the whole game. Operational payoffs often aren’t — a missed security filing, a waived privilege, a breached system can be terminal.

* Adam’s formative lesson came from a malpractice case: the concave payoff curve of law (bounded upside, catastrophic downside) drove him out of litigation and eventually into analytics.

* He vibe-coded a career payoff visualizer to make this intuition concrete — the same tool every technology manager implicitly needs when committing to timelines, availability zones, or budget targets.

* Agile is a structural hedge against the same asymmetry: shorten the cycle, reduce the blast radius of being wrong.

Adam: I would focus on cybersecurity — an area you’ve spent more time in than I have. Risk management in cyber is an area with that extreme concave payoff curve[8], where being wrong in a certain direction is extremely career-limiting.

One of the formative experiences in that professional journey I sketched out earlier was a malpractice case I tried as a lawyer. Very few cases go to trial — almost everything is writing briefs and pleadings. But this one did. My client was a lawyer being sued because he had failed to supervise a young associate who was supposed to file in a telecom case and simply didn’t get it done on time.

Once they realized the deadline was approaching, they scrambled, missed the filings, and the telecom project couldn’t go forward. The client sued. We did good work at trial and settled before it went to the jury.

What that case made me realize was that litigation — and being a lawyer generally — has that extremely concave payoff curve. If you accidentally produce a document with attorney-client privileged information, when you’re going through warehouses of material —

It used to be physical warehouses — for lawyers today it’s all on a computer, it’s a cyber problem. But if you accidentally produce a privileged document, you’ve waived privilege and you’re in serious trouble. Being a lawyer felt like limited upside — a nice house if you make partner versus getting fired and never finding work again.

So I thought more carefully about these payoff curves. The first conversation I had with my kids when they were old enough was: don’t choose a profession with that kind of payoff. Do you want a venture capital payoff — an extreme tournament where the upside is a million times the median, if you’re genuinely confident in your skills? Maybe. But you’re probably overconfident. Most people want something with reasonable upside and limited catastrophic downside.

I actually vibe-coded a tool to visualize this — it lets you look at payoff curves for different employers and say: if I get a top-percentile outcome, here’s how I shake out; if I get the bottom outcome, here’s how I shake out. You can vary economic conditions and other factors. I stuffed it with synthetic data grounded in actual information. It’s more of a toy model and theoretical exercise — an intuition pump to make visible the thing you were saying. Do I really want the job where 10% of the time it’s utterly fatal?

James: It’s directly relevant. People make decisions like this every day. Do I launch this project? Do I use high availability for this application? Do I run it across multiple availability zones? Do I commit to two months versus three? Those are technology examples, but every business manager is asking: what target will I commit to, in revenue or in cost? Thinking through your own payoff matrix — and the payoff matrices of the people around you — changes the analysis.

Payoff matrices differ enormously by corporate culture. In some organizations, blowing a budget a little is “try to do better next quarter.” In others, exceeding a budget in a single quarter is career-limiting.

Adam: That’s right. The types of IT projects I mostly see and lead are business intelligence projects — building out and deploying predictive models. These, especially the dashboards, are generally developed in an agile environment, where agile is kind of a hedge against the asymmetry you’re talking about.

I’m not building a whole stack waterfall-style with a perfect delivery date. I’m making it good enough to get adoption, then making it better — two-week sprints, continuous improvement.

So I would argue that agile is a way of coping with an uncertain world. Do you agree?

James: It’s more emergent strategy than deliberative strategy. There are people who believe that whoever wrote the Agile Manifesto had some connection to John Boyd — I haven’t been able to find as much documentation for that, but the idea is the same: increase the number of cycles, shorten the duration between action and feedback, and revise your strategy accordingly.

Adam: Exactly. Agile development and hypothesis-driven problem-solving are both hedges against disaster. One of the first things you learn at a consulting firm — I taught the intro to consulting class several times — is that you have a day-one hypothesis about the answer.

Part of what’s good about that: when someone stops you in the hall and asks what you think the answer is, you have something. “I’m going to test it and refine it and make it better.” That’s much better than a long list of ideas with no organizing principle.

The risk otherwise is getting caught out with no answer at all. The OODA loop, hypothesis-driven problem-solving, agile — these are all hedges against uncertainty and against the weird events that shock you.[7]

James: The interesting thing is that the nature of the bad thing varies by the nature of the system. In an analytics system, the bad thing is bad information — you make a bad business decision. In an operational system, like an order capture system, the bad thing is different: downtime, a cybersecurity event, a transaction failure.

We deal with those types of negative payoffs differently, using different mechanisms. It’s probably worth thinking about that more rigorously. Let me pivot just a little bit.

What do we tell early career professionals about AI?

* Get a job in a for-profit company and help it make money with data. That is still good advice. Those jobs are not disappearing as fast as the pessimists predict.

* The supply of problems is highly elastic: better, cheaper analytics generates more questions, not fewer analysts.

* Proof-of-work has decayed — anyone can generate a slop deck. What has not decayed: domain knowledge, taste, and the ability to have a real conversation about a business.

* Adam’s standing advice: pick the domain you care about, grab the public data, do the analysis. Show up to the interview having already thought about how to help the company make money.

James: In my teaching, I’ve seen a lot of anxiety from people early in their careers about what they’re going to do with their lives in the age of AI. A number of college seniors asked me, and I said they should learn data structures and data science — because no matter what happens, that will be valuable. Did I give them good advice or bad advice? And what are you taking away from your interactions with college students?

Adam: Getting some kind of job and getting out into the real world to develop domain knowledge — for all the reasons we talked about — is important. Getting that job is a separate question; that’s tactics, informational interviewing, picking your spots.

But as advice for how to spend the next one to three years: get into a for-profit company and help it make money by analyzing data against real problems. Those jobs are not going away as fast as the most pessimistic people are predicting.

The supply of problems is highly elastic.[3] Having people ask good questions — and the return to a business from asking more good questions and getting quality answers back quickly — is positive. I’ve bet my whole career on the idea that most companies would benefit from having one more data scientist, or in this case one incremental unit of data science capability.

Which is not the same as replacing all the humans with AI. I’m on the optimistic side of this.

James: There’s more data, available publicly. There’s more tools that you can use for free. And my inclination, for example, if someone were to ask me, “Gee, I want a job doing analytics,” I would say, “Well, pick the domain you’re interested in and start doing the analytics.” Right? Because what’s gonna be more compelling in a first-round interview? It’s “Oh, gee, I’m, I’m really interested in the telecom industry, and I grabbed all this from the FCC and I did this analysis, and here’s what I found.”

Adam: I’ve given a talk on this subject probably every year for the last five years, including the pre-generative-AI era. The advice holds: do actual work, show that you know something about the business, demonstrate that you care about it.

I will say that that coming in with a deck, the proof of work aspect of it has gone to kinda zero, right? ‘Cause anyone can generate a slop deck. People like you and me might be exquisitely sensitive to seeing a deck and understanding immediately that it was written by AI and the, and the tells in the stylistic, that come out of a Claude, PowerPoint deck.

The concept is still correct: get in there, do the work, iterate, think about it. You’ll be able to have a real conversation about the business because you’ve learned something along the way — “I think 80% of the value is in this part of the product, and here’s why.”

You’re not reading off a slide. You’re engaging a person. There’s a lot of value in that. And people still show up at interviews without having thought seriously about the business and how they would help it make money. There’s still a lot of alpha in that.[6]

James: Terrific. Anything else to add before we wrap up?

Adam: The forecasting contest runs every year. Free to enter. You get a bowl of pho if you win, and a book — this year it’s George Orwell’s essays. All the information is at braff.co/advice: the current contest, how to enter next year, and other thoughts about food and analytics.

James: Terrific. Two great topics to discuss. Adam, thank you so much

Adam: Thanks for having me.

Thanks for reading Prosaic Times — subscribe to get every issue!

Notes

[1] Obviously twenty years is an estimate -- that doesn’t make it wrong. The world is entropic, and deterministic systems are not talked about how organized complexity is much tougher to understand than disorganized complexity.

A B2C business with a million customers faces disorganized complexity. Idiosyncratic individual behavior averages out into stable statistical regularities — churn rates, cross-sell lift, tenure effects.

A B2B software company with sixty enterprise accounts faces organized complexity. One decision-maker changing jobs, or having a bad quarter, creates a measurable swing in the entire dataset. The N is too small to smooth anything, especially since many of the data points are connected to each other. That is why domain expertise is required in B2B analytics in a way it is not in mature B2C analytics — the data cannot naturally control for the variables that large samples would otherwise absorb.

[2] All IT commitments are forecasts based on implicit probability estimates. When a CIO says “We’ll deliver in two months” she estimates the probability of coming in at two months is high enough to avoid the political cost of articulating a more conservative estimate.

[3] Adam describes here the applicability of the Jevons Paradox to knowledge work. When the churn data got good enough to slice, the analytics team didn’t shrink. It grew. Each answer generated three new questions — why are customers in this region stickier? What’s driving the gap between acquisition channels?

[4] You’ll never know whether the 60% probability you assigned to a project coming in on budget was the right number. It either did or it didn’t. But across a portfolio of similar decisions, the manager who has actually formed a probability — rather than suppressed the uncertainty — will be better calibrated over time. That’s what “it matters to be 53% right instead of 52% right” means outside a hedge fund. You’re not making one bet. You’re making hundreds.

[5] There is no perfect data set for the organized complexity of B2B commercial due diligence. Direct customer interviews and focus groups are great, but you only can do so many of them. On-line surveys? Decision-makers responsible for large technology bets at Fortune 100 companies lack the time to respond to them.

Adam is correct. No virtual panel can, well, forecast purchase intent. But they should provide invaluable feedback on value propositions and messaging. And they are both patient and indulgent. You test many variations of a value proposition for example to see which one best resonates.

[6] We’ve heard people confidently predict that early tenure technology jobs will disappear. We’ve all fielded anxious questions from candidates who should probably believe less of what they read on social media. We have all tried to figure out what to say. As Adam points out learning is the most important thing at the start of your career. And it is easier to learn useful things than it has ever been before.

[7] Colonel John Boyd’s Observe-Orient-Decide-Act (OODA) loop is a model of how people and institutions make better decisions through recursive learning in competitive games. It provides a mechanism for balancing deliberate and emergent strategies, which emerge from the bottom up. It appears to have influenced the agile manifesto.

[8] A concave payoff curve: bounded upside, catastrophic downside. Law. Cybersecurity. Most operational IT. A convex payoff curve: bounded downside, extreme upside. Venture capital. Early-stage software equity. CIOs and CTOs often face concave payoff curves. Flawless uptime is invisible. A database corruption is not. When you put together a major incident response plan, you are trying to truncate the ugly end of a concave payoff curve.

Cloud infrastructure, SaaS applications, a revolution in cybersecurity and now AI have upended that unhappy equilibrium of oligopolies. Large enterprises less exclusively act as “takers” of offerings from giant technology providers. They integrate a wider array of technologies from a wider array of providers — some of whom are venture-backed attackers.

Acknowledging this, we did something new at May’s Technology Leadership Forum session. We convened a panel of venture capitalists to discuss and debate how they see the enterprise technology market. Ed Sim from Boldstart, Daniel Frankenstein from Joule Ventures and Will Summerlin from Autopilot engaged with the group on how enterprises can best work with VC-funded companies and how AI will change B2B software markets

Here’s what I wrote about it after the event:

Wow, there is some frustration out there! What do TLF members see from their incumbent software vendors? Slower innovation, degraded quality and more aggressive negotiation. They think some providers see their products as “falling knives” and have resolved to extract as much cash as they can from the portfolio before it declines into irrelevance. Others simply don’t get AI — they want extortionate rates for unimpressive capabilities that only reinforce silos between different parts of the environment.

TLF members found the discussion so valuable and VCs had so much fun that we decided to reconvene (after much scheduling gymnastics) for a Prosaic Times video discussion.

The conversation covers five things.

* Whether vibe coding and agentic software engineering are the same activity — they aren’t, and the distinction matters for how you staff and govern AI work

* Where the moat lives as the cost of producing code approaches zero — my answer involves latent information, and it’s more interesting than the conclusion.

* Which sectors are most exposed to disruption — legal and accounting are the easy answers; I have a less obvious one.

* How enterprise technology leaders should actually engage with the startup ecosystem — the VC-as-intermediary model turns out to be more consequential than most enterprises realize.

* And whether you should leave a corporate job to found a company. Ed has a checklist for that last one.

Thanks for reading Prosaic Times — share it with a friend!

Introductions

James Kaplan: Welcome to a special Prosaic Times podcast. Back in May, we did something different — we always get together about 50 or 60 enterprise CIOs, CTOs, and CISOs, and this time we invited a few of my favorite venture capitalists to share their perspective on the enterprise technology market.

All of our VCs had a blast and found the session both enjoyable and informative. All the members of the Technology Leadership Forum told us they appreciated the session and found perspectives they hadn’t heard before.

So I said, we have to get this group together again for a podcast — so let me start out with some brief introductions. Will, do you want to start?

Will Summerlin: My name is Will Summerlin. I’m a co-founder and general partner at Autopilot. We’re a growth stage venture capital firm, typically investing from Series B onward

Daniel Frankenstein: A co-founder and partner at a firm called Joule Ventures. Seed and pre-seed investor doing B2B software investments, coming out of the Israeli ecosystem.

Ed Sim: Boldstart Ventures. We partner at inception with technical founders when they’re in the idea maze — companies automating the autonomous enterprise, physical AI.

James: Daniel, when we were chatting before, you said that the interaction with the members of TLF was very eye-opening for you. Could speak to that a little bit

Daniel: One of the challenges we face in commercializing companies that aren’t from the United States is building that connection to corporate decision-makers and technology integrators. What we got from the TLF session was direct feedback on specific technologies — candid views on where enterprises are struggling to integrate AI solutions, and where their long-standing incumbents are trying to introduce AI but not yet meeting the mark.

It was a fascinating conversation, and it clarified where the real opportunities are for new technology businesses to carve out space.

James: We’ll say incumbent rather than legacy.

Ed, what was your takeaway? What did you learn from the day?

Ed: Everyone is truly, truly interested in deploying agents, and two, it’s really, really early, and I think that everything we think and know today may be irrelevant tomorrow.

People are talking about RAG and all these other things — like, hey, that was gone yesterday. And in fact, everything that we talked about almost since the last TLF, what was that? Six weeks ago? Is almost irrelevant again. The speed at which things are moving is staggering.

Will: We’re extraordinarily early on the adoption curve in terms of enterprise AI adoption. And I think it’s interesting in that there seems to be a disconnect from the myopic Silicon Valley view and the reality of enterprises. If you talk to a lot of founders or early stage investors or even growth stage investors in Silicon Valley, there’s this view that adoption’s happening very, very quickly, and we’re farther along on that adoption curve.

James: But what we heard from members of the forum is that we’re still very early. It’s still very much in the experimentation phase, moving to the broad-based adoption phase. And so I think that presents an incredible amount of opportunity ahead, but I also think it reflects this disconnect with the Silicon Valley view. You hear a lot in the ecosystem around token costs, and news stories about companies struggling with token costs. But the enterprises I know are spending trivially on tokens in the context of, say, a USD 1B or USD 2B or USD 500MM IT budget.

And that is a measure of quite how early we are from an adoption standpoint.

Ed: Some of the companies that were the more advanced ones were the ones beholden to one large language model provider.

The events of the last six weeks — particularly some where providers said, “Hey, we’re gonna interfere in some of these things” — have shown that not only do token costs matter, but also which model you’re beholden to matters.

So no one’s gonna want to be beholden to one model. Multi-model architectures have to be built, I think, because of that.

James: It reminds me of the era when some people believed you should standardize on a single provider.

Will: I think another interesting data point related to token costs is compute resources. We’re already in an environment where compute is radically supply constrained, and if you look at the growth rates of demand for compute versus supply for compute, demand is growing at a much faster rate than supply possibly could. [2]

Demand is growing because we see more broad-based adoption. It’s growing because agentic models use a lot more compute than legacy LLMs from a year or two ago. And on the supply side, we’re hitting all these constraints and bottlenecks related to supply chain in the semi space.

We’re entering a world where that gap is gonna continue to grow. And that’s all in this backdrop of what you said a minute ago, which is, the single digit millions of spend. And so I think we’re in this interesting environment where part of the world, is super early on that adoption curve, but at the same time we’re already hitting these constraints around compute.

If you want any semblance of an SLA from one of these model providers, you’re going to have to start committing to more meaningful spend. There’s this conflict and tension in betting your entire AI strategy on a single model provider.

James: The enterprise is maybe the shoe that hasn’t dropped yet in terms of token demand, and my God, what happens once the, spenders start using, scale?

The flip side is that you’ll probably see a lot more efficiency in token consumption. You don’t need to do all your non-deterministic processing at runtime. More of it shifts to build time — software engineering, data ingestion — with deterministic execution at runtime. And we may see a meaningful share of inference move to edge hardware entirely.

Ed: Think about models like OpenRouter. Companies now are about to sell a box — or a virtual box — that has a router for any frontier model on top, and then adds your own open source model on the box itself with H200s bundled in.

The idea is to make it very, very easy for people to build an agentic workflow, deploy it, and then actually maximize... I think we’re gonna move to a world of ROI per outcome. It’s no longer gonna be, are you automating me? But what is my ROI per outcome?

How does that look? And then there’s lots of other things involved in measuring success.

Daniel: Obviously folks have concerns about costs and limiting those costs. But right now, those costs are the actual real costs businesses face when they start using this at scale.

So the demand curve is absolutely outstripping the supply curve when it comes to compute. Enterprise adoption is going to be more of an increase on a dimmer light switch.

James: Enterprise adoption is always an increase on a dimmer — not a flip of a switch — which is why claims about AI impact on the bottom line feel like an attempt to be cute.

How could you expect otherwise? We’re what? Maybe a couple of years into the use of gen AI. How could you possibly expect that to have impacted enterprise operating income statements at this point?

The only impact on the income statement has been for some of those tech companies who are spending. I’m thinking about banks and pharma companies at the moment.

Ed: As far as the ROI on that spend — questionable.

Will: I do think we have small case studies where we’re starting to see it.

As one indicator: we have a portfolio company called Replit, which is an AI-native software creation platform. They allow anybody to create software using natural language. And what we’ve seen is in some cases the replacement of incumbent software providers where a non-technical user is able to create their own custom CRM system for a USD 100,000-a-year software license. The ROI on that is extraordinary. That’s like nothing you’ve seen from traditional software.

Ed: I still think the super complex stuff — Workdays — when people say, “Hey, we can vibe code that stuff away,” I think people have moved on from that really.

Whether or not those companies are gonna be massively successful, who knows? But I do think the whole idea of vibe coding things, the maintenance becomes absolutely incredible for some of these people as well.

Will: On one end, you have the bottom of the market that has simple use cases where they can vibe code their own CRM and displace incumbents. So I think that’s one end of the market.

The other end of the market in regards to enterprise adoption is just building more — where now you can use tools like Replit to build agents on top. And so it’s not that you’re gonna get rid of your incumbent provider, it’s just that you’re gonna build more agents on top of what you already have.

We see this primarily around internal tools, where to build a custom internal tool for your sales operations team or for your legal team, you had to either go work with a third-party software development agency to build that, and it would cost you hundreds of thousands or millions of dollars, or you had your internal software engineering resources, which would be expensive and you’re competing for internal resources.

Now, a non-technical person could go into a tool like Replit and build those agents or build those custom pieces of software on top of the existing architecture that you already have. And so I don’t think it’s a replacement for the incumbents in this case. I think rather it’s just gonna enable more software to be developed on top.

Daniel: When we got together initially, there was a real fear-of-replacement moment for incumbents.

We’re more in an all-of-the-above moment now, where a lot of incumbents have the customers, they have the data they own — which, by the way, is one of the key things to be competitive in this moment.

And that’s actually what makes this really exciting from a venture perspective: these incumbents are your customers, and also your acquirers.

You also have this entire new world to go after as well.

Ed: The most impressive thing was the idea that unicorn SaaS companies are all dead. This founder went all in on AI three months after ChatGPT came out, and today Salesforce announced it was buying it for — I don’t know — three and a half billion or something, which is above their ZIRP-era financing round. [1]

It takes very special companies, but maybe there’s a five percent chance that some of the very forward-thinking ones can create that kind of value. And combined with some of what Will said about Replit, there’ll be net new — more things banging on the system.

Vibe coding vs. agentic software engineering

* Vibe coding and agentic software engineering are distinct activities: one is a non-technical user building something simple; the other is a professional engineer with 2–10x leverage.

* The non-technical user persona is larger than the professional developer pool — and that’s where the near-term adoption surge is coming from.

* In enterprise, the term “vibe coding” doesn’t travel well; the equivalent is building skills and automations on top of existing platforms.

* The abstraction layer keeps rising — but semantics still matters. Syntax can be generated; data models and business logic still require human understanding.

James: What’s the difference between vibe coding and agentic software engineering? When I say vibe coding, I mean someone who’s not particularly sophisticated building something — maybe it’s a reporting tool or analysis. Versus a software engineer using a set of 2x, 5x, 10x tools.

Will: There’s a completely different user persona between the professional software developer that now has these agentic tools and the non-technical user who’s now empowered to create software. We’re seeing the adoption of the non-technical user — people in sales operations, people in legal, people in finance who have never written a single line of code, who don’t understand any JavaScript syntax — who are now empowered to create their own software in a way that is easy, simple, secure, compliant.

That’s where we’re really starting to see this adoption. The other important nuance here is that the pool of users who are non-technical is larger than the pool of users who are professional software developers. And so if you look at even a technology company, you might have fifty, sixty, seventy percent of the employees falling in a category that’s non-technical, and they are now empowered to go create software.

James: I wonder if we’re gonna see the rise of strats. In banking we have the strats who were either former traders who learned how to code, or former coders who learned about markets — very fuzzy. I was wondering if that might happen in more sectors and more domains.

Daniel: I agree that non-technical users are increasingly empowered to vibe code solutions. The problem is the influx of people who are vibe coding something, white-labeling it, and pitching it as a venture-fundable startup — and it’s not. Across sales enablement, marketing, the CFO stack, I can tell within five minutes of a pitch that there’s no defensible moat, nothing that takes particular expertise or meaningful compute to maintain. Buyer beware.

To use your strats analogy: to be a great trader takes years of domain expertise and longevity in the space. The same applies here. The people who can build something special are the ones with that depth. A vibe-coded sales stack with no moat is not that.

James: what do you think of non-technical founders?

Ed: I agree that more people building things is continuing to get more abstracted. I think the word “vibe code” and enterprise just does not really mix together — I don’t think you’re gonna walk into a CIO and say, “Let’s vibe code stuff.”

But it’s the same thing. It’s the same thing as what Will said — skills. The enterprises we talk to are using skills platforms, where you can type something into your AI of choice and say, “Hey, I want to build this, add this in, and then create a skill or automation.”

That is happening, and there’s gonna be hundreds of thousands of skills out there, and you’re gonna need to manage that and package it like software. You’re gonna need a version control system where there’s a skills repo for company XYZ, where these are the approved skills, and these are security vulnerabilities that can pop up, so let’s scan those skills.

What happens when Daniel changes something that Ed made? Is that better or worse? So how do you do an eval around that? So I, I think, I think we’re gonna be moving towards a world where skills become software,

James: If you remember the late ‘80s, we went from assembler to C — people said software engineers would go away. The engineering continued.

And I wonder if two things are true here. Abstraction matters, but you still need to understand the data model, even if you’re programming in a very abstracted way.

Ed: I think what you’re talking about is that you need specialized humans. Back to your earlier point — everyone wants to get automated, but they don’t know where to start.

Those specialists can help people understand what their workflows are so they can automate them. So you need specialized skill for that. And then two, you need specialized skill for evaluating whether a workflow is actually doing what it’s supposed to do. And then what happens, for example, when a model changes — is it actually gonna do better or worse?

People are gonna be needed, and even more specialized to understand those things. And I think people are finally understanding that right now.

James: Sometimes I draw a distinction between syntax and semantics, and in an AI software engineering world, we can worry a lot less about syntax, but semantics still matters. So Will, let me ask you, are we being overly negative on vibe coding? Give us the other end of the case here.

Will: First off, I know of several examples, including one where a company, built entirely on Replit is doing well over 100 million ARR, mostly enterprise revenue.

And so you, you can build real businesses, and I think we’ve gotten to a point in the technical maturity of the platforms like Replit where you have, implicit security, you can host on a private cloud instance. You hit all the enterprise check boxes, and a lot of the code that’s now hosted on tools like Replit specifically is as good as what you would get from a junior engineer. And so I disagree with the premise that you can’t build a venture-backable company on platforms like Replit.

Ed: I’m getting my popcorn. This is fun

Will: I agree that we as investors should operate with this premise that the cost to develop software is approaching zero over the next three, five, seven, ten years — just like the premium for having more software engineers is eroding. As that skill barrier is eliminated and the cost to build approaches zero.

Daniel: Of course there’s going to be vibe coded businesses built on companies like Replit that are gonna be absolutely venture fundable and extraordinarily successful.

The barrier to entry to put a product in the market has gone to zero, which means there’s so much stuff that isn’t defensible and isn’t venture-backable. It’s a huge increase in the denominator without a proportional increase in the numerator.

James: The percentage of effort associated with the core logic, relative to the effort required to fit into a broader enterprise environment, is decreasing. The pain is now in the integration: connecting to identity and access management, compliance, the whole stack.

So Ed, how do you think about this challenge of integration? Because in many places that will be the long pole in the tent.

Ed: The last mile in the enterprise is the longest, and it’s probably getting longer. It’s not easy for a large bank or a healthcare firm or a large CIO to bet their reputation on deploying your agentic technology.

There’s a lot of stuff you’ve gotta pass — all the SOC compliance and everything else. What you’re seeing is the idea of these forward-deployed engineers showing up and just getting it to work. Part of that is a response to how hard the last mile is.

The model is: build product, then finish the last mile with people. And the learnings you get from it are invaluable. The key question is whether the most important data to own is not just the data you have to run the AI on top of, but the private evaluation of whether the outcome looked really good — because that is really the knowledge of your business. That’s what separates a P&G from another marketing company.

Are these forward-deployed engineers going in there and helping you get those outcomes? And who owns the data? Are they giving it to you? Or are the forward-deployed engineers from the model providers themselves taking that data and using it to build vertical solutions?

That’s going to be the big battle. It’s what Satya calls hill climbing — reinforcement learning feedback. The question is: if you give that to a model provider and they’re helping you improve, is that the right trade-off to make?

I think we’re moving toward a world where people want to control their own evals — which means infrastructure on the edge, private models. We need a lot better open source coming down the line. It’s still not anywhere close to what the labs are providing, but that’s where we’re gonna move towards.

Where the moat lives when code is free

* If code approaches zero cost, the competitive advantage shifts to ontology, data models, and latent institutional knowledge — the things that can’t be scraped.

* The right model for the right task matters more than any single model: orchestration layer bets are more durable than frontier-model bets.

* Enterprises that hand their outcome data to model providers for convenience may be signing their own death certificate — private evals are the new proprietary asset.

* Architecture commoditization is coming but isn’t here yet; scaling laws are still working and a step-change in model capability is 12–18 months out.

James: sometimes I wonder if code becomes free or close to free, but the ontology and the data model becomes the source of competition.

Will: It’s very possible. This goes back to a point that we were talking about earlier, you probably don’t want to bet on a single model. It’s not just about cost, it’s also about, performance in different domains. And so I think that- that’s benefit of betting on something more at the orchestration layer, where, if you’re trying to do a task, we’re using software engineering as the example here, you might be writing, some more complex code, you might need a frontier model from one of the labs.

You’re writing simple front-end code, you could use an open source model. You’re doing a design task, maybe, Gemini or another model, from a, a different provider is better at that specific design task. So I think you’re seeing mixture of experts, to, solve these

As software code itself becomes a commodity, what matters is matching the right model to the right task — and everything else that isn’t captured by coding benchmarks: platform depth, integrations with Databricks or Snowflake, security scans, the full enterprise stack.

Daniel: think having specialized humans that really have domain expertise combined with owning your data and your moat, somewhat model agnostic and having that’s what the future it’s a mixture of really great humans, different models, and, owning your data. I, companies that choose to allow their outcomes and their data to be out there for the LLMs to scrape, because they’re, they’re prioritizing speed, ultimately are, are signing their own death certificate.

James: And does architecture get commoditized?

Ed: Not anytime soon. So the answer is not today, but I wouldn’t be surprised, like in 18 months if we kinda get there, because this stuff is changing so quickly.

Will: I don’t know, but if I were to place a bet, yes. Just look at the improvement in performance over the last three years. It doesn’t appear that we’re plateauing — in certain skills or certain domains it’s actually accelerating. It’s really hard to extrapolate out the next two, three, five years. But I think if the curve continues to look like it has the last three years, it’s very possible that architecture gets commoditized.

Ed: In other words, scaling laws are actually working, and if you have the compute, it’s improving. Everyone I’ve talked to on the models they’re training now says it’s still moving in the same direction. Hasn’t been proven false yet.

James: I’ll answer my own question and say my question was invalid. Any model requires context, so it depends on explicit information rather than latent information. Models will only get good at architecture to the extent that people using them figure out how to extract or capture the latent information they need.

Ed: How much will they be willing to hand over of how their system is architected and actually operates to a model provider? That’s the question.

James: Do they even understand enough to be able to write it down in such a way that it can be consumed? Taking latent information and turning it into explicit information is not a trivially easy task.

Ed: It’s the last place that hasn’t been automated yet. If you’re thinking about most of the automations — it’s on the edges. IT ops themselves have not really been automated. Partly because people are afraid of AI just shutting things down. But I do believe we’re gonna move toward a world where there’s more trust with AI around automation of IT.

James: Let me ask a related question, maybe one that’ll be controversial. So I, I, I got myself told I was up a tree because I offered the thought that we’re gonna see more strats in the future, more, convergence the line between, business optimization and technology execution.

But then I offered the hypothesis that it may be easier to teach software engineers about the business domain than it will be to teach people in the business domain enough about data models, for example, to, sufficiently engage. I don’t know whether you call it vibe coding or agentic software engineering.

Daniel: I spend almost all of my train technical people on the business side of things. so, so I probably over-allocate to frustration in that department. So, maybe that skews my answer, that maybe it’s a little easier to teach business people, the software side, but I could go either way.

James: Ed, what do you think?

Is it easier to teach a bus- a business analyst software engineering, or easier to teach software engineers about how to think about a business problem?

Ed: I think the underlying models can be so good that what matters is the expert who knows what to prompt and ask. If you actually look at how to use LLMs today, it’s all about how you prompt — two of us could sit next to the same thing and get vastly different outputs if we’re trying to achieve the same goal.

Will: I completely agree with Ed. There are so many nuances to different problem sets or workflows. The people who intimately understand them will have access to tools that can manage all the technical complexity.

One case study from us: we vibe coded our own custom LP portal, and my co-founder, who’s not technical but who’s been running all of our operations, built this in probably 12 hours. And it’s better than anything we could find from third parties. Daniel and Ed probably lament the challenges of fund admin more than anyone else.

James: Nothing’s more fun than fund administration

Will: it’s the great joy of life. there are so many little nuances to it that I think y- you just wouldn’t understand unless you’ve spent months or years in the trenches

Ed: by the way, that goes back to my other point, is that your person that built it and you are the experts, and , you’re the ones that are gonna actually evaluate it as well. You’re the only ones that can keep evaluating everything else, right? So that’s kind of your workflow, that’s your model.

And, and your same model could be deployed at Daniel’s place and my place, but how we evaluate it and what successes might be different, right? And that’s kind of what I mean by the institutional knowledge and workflow that, that I think those private evals is so valuable to capture that

Daniel: One of your great moats is domain expertise. domain expertise can be found on the business side, and it can also be found on the technical side. It depends on what problem you’re solving and who’s your constituency.

James: I had a discussion with a bunch of 22-year-olds a couple months ago, and they were, you know, having the typical 22-year-old, ”Oh my God, any job I might consider is going to be replaced by, by AI.“ They asked what I thought would be a good thing to learn, and I said data modeling.

I don’t know whether that’s business domain or technology domain expertise, you both need to understand all the nuances of the business process, but there’s that, process of abstraction, that I think becomes incredibly important, which people who are only deep in the business domain don’t always get.

Which sectors face the most disruption

* Legal and accounting are the obvious targets — high information volume, largely objective outputs, limited customization requirements.

* James’s less obvious answer: B2B interactions broadly. They haven’t changed in 15 years while B2C was transformed. Gen AI’s ability to ingest unstructured data makes this the next frontier.

* Ed invokes Jevons: freed capacity can be absorbed by new demand. Companies that cut headcount fastest are already adding back — different kinds of people, more senior engineers, specialists.

* Robotics is the adjacent disruption no one is talking about enough. Ed’s portfolio company has 500,000 hours of manipulation data and is deployed at four large manufacturers — the ChatGPT moment for physical AI is close.

Okay, Daniel, predictions about which domain, technology domains do you think will be disrupted the most?

Daniel: I’d say I’m still a big believer in places where there are largely objective answers to questions that require sifting through a lot of information. I think legal and accounting are two of the low-hanging fruits.

Ed: I think, James, that everything is up for absolutely massive disruption right now. But I’m also a believer in Jevons paradox — because if things are cheaper and you can use AI more, you’re gonna be able to do more. And I do believe in a world of more productivity. The tech companies I’ve seen that have automated the fastest may have cut headcount and used AI as an excuse, but now they’re adding. They’re just adding different kinds of headcount — more senior engineers, different kinds of people. Perhaps the specialists you’re talking about.

So I think it’s gonna disrupt everything short term. The next two or three years will take a while to absorb the hit. But I believe in the longer term that we will actually be more productive as a society and generate more output with less cost. And hopefully you see those GDP growth numbers increasing.

James: Will, what do you think?

Will: I completely agree. If you look at any technology cycle throughout history, for the most part it’s led to wage increases, job creation, new goods — and I think the same thing will happen here over a ten to fifty to a hundred year time horizon. What we see in the next five years: the most disruption today is in two categories. Where incumbents have low-quality software products and have not innovated, you can now build that version yourself — and often it’s better, cheaper, and higher quality.

That’s our case with this fund: we built it ourselves because the incumbent alternatives were not high quality, they were very expensive, and there was a lot of low-hanging fruit.

I think the other area where we’re starting to see more disruption is in domains where software is highly customized — back to the point both Ed and Daniel made, which is that it’s really important to have deep domain knowledge specific to your workflows.

The way we build an LP portal might be different than the way Ed wants to build one, because he has nuances in his business and portfolio that we don’t. Where you have a high level of customization required to make software work for your business, there’s a lot of opportunity for disruption — both disrupting third-party software and disrupting the incumbents who have mediocre products that work okay for everyone but not great for anyone.

James: it’s what I call the one size fits none challenge.

Ed, you said in the session that you looked to fund companies that were problem-obsessed rather than solution-obsessed because, you, you iterate through different solutions, but problems are eternal.

Messiest part of, the advanced economies is B2B interactions, and that’s not just CRM, but the broad scope of all the large, complicated businesses. , Over the past fifteen years, maybe twenty years, we’ve seen an utter transformation in B2C interactions.

But many B2B interactions look a hell of a lot like they did ten or fifteen years ago, and I think that, the, the ability of, gen AI to, to ingest complicated, unstructured data to transform those domains.

Ed: I, I love that. And by the way, can I add one thing

James: Please

Ed: we didn’t talk about software and robotics,

I have a company. We’ve built our own LLM for robotics, with our own data, and we’ve proved that scaling laws actually work in robotics as well.

So I feel like we’re almost at that ChatGPT moment in robotics where a generalized robotic software model will be able to do a lot. So in other words, once upon a time where you would have to have 100 different robots or 100 different kind of brains operating and, and doing 100 different tasks, you can now have one.

And I can tell you this, that the market is moving so fast in that space, so manufacturing, pick, pack, and ship, auto manufacturing, all of those kinds of things are getting transformed. In the next three to five years, you’re gonna see a massive, massive dislocation in those markets as well

James: Is that an LLM for robotics or is that a type of world model, right? I mean, are we moving from language to physics when we talk about that?

Ed: there’s three different kinds. So the one that we have is a basically a foundational model for robotics. Other people are trying the world model approach, and what my company is doing is they’re getting their own data, right? There’s no Reddit for data.

They , ship these 3D-printed devices that goes on people’s hands that are very, cheap. we take video of people manipulating these things as if they had these grippers, and we’ve got 500,000 hours of data.

And now we’re deployed at four very large companies doing very complex manufacturing tasks all within two years. This is massively moving so fast. [3]

It’s basically building off the shoulders of giants? Because it’s already been done before, and now you’re applying it to a different realm,

James: Daniel, how bullish are you breaking the, breaking the barriers between the physical worlds when we come to,

Daniel: How can you not be bullish on it, right?

James: What’s the pace, do you think?

Daniel: One of the common themes we’ve discussed throughout this conversation — which, again, was part of the reason why I so enjoyed our in-person discussion — is the pace of innovation. It’s at a lightning speed.

Literally every week things are changing. Every day things are changing. That’s moving at a breakneck pace. But the integration of those changes — the integration of products that encapsulate some of this innovation into large businesses — is very slow.

And so I, we’re, we’re, we’re it’s, it’s one of those where you’re like in an F1 race where innovation is just lapping, the jogger, which is the, the, the enterprise integrator.

And, and so the question is how do you lean out the window and throw something that the jogger can actually digest? That’s, where the rubber hits the road, not to take the, this too far. But, that’s, where the bottleneck is.

Will: Agree with, with, everything said here and that there’s an incredible amount of opportunity for AI in the physical world. you look at some domains in manufacturing where AI is now used to inspect batteries coming off an assembly line, and it can do it, ten times faster and, fifty times better than a human inspector can.

And that’s a job that humans don’t like doing. You wouldn’t want to sit under bright, bright fluorescent lights all day and look at batteries. So I think there are incredible amount of opportunities we’re already seeing implementation, the ROI is there. But I will say, I think that humanoids broadly is probably in bubble territory.

And I say that for two reasons. I think number one, we’re very early on the S-curve and that, the, the technology is not quite there. when you start to do things in the physical world and chain together different skills and make that work successfully in an enterprise environment, the, the, the success rate has to be very high in each skill.

You know, if your success rate is ninety-eight percent in one skill, ninety-eight percent in another, ninety-eight percent a third, you’re gonna fail a lot of the time because you multiply the probabilities together.

You really need to get to like ninety-nine point nine nine percent accuracy in each skill set or each domain in order for this to work reliably over time. The second problem I think we have in the humanoid space is people underestimate how hard it’s gonna be to scale up manufacturing.

Like you go talk to the team that, that scaled up manufacturing for the Model 3 at Tesla, like it’s a really hard thing to do to go from, you know, zero to scaled production of, of complex hardware products.

And I think the sort of view that we’re gonna be shipping millions of these in two years or three years is, is probably a bit shortsighted.

Ed: One comment though. I, I agree with you on humanoids, but my company’s actually going after, robotic arms, So when you control the variables with just this and a gripper, you don’t need five joints on it to do things. that actually allows you to deploy much faster.

On the humanoid side, there’s so many more issues. Battery life is another one. Safety, what happens when the battery runs out? Does it fall on somebody? there’s so many issues there to, to your point.

The, more you constrain all the, variables and focus the processing power on the task at hand, the better off, you are in terms of deploying these things with real accuracy.

How enterprises should engage the startup ecosystem

* Routing startups through innovation teams is a trap — those teams are typically disconnected from the business owners who actually have the problem.

* VCs are intermediaries in two directions: LP → founder, and founder → enterprise. The second role is underappreciated. They filter signal from noise, translate languages, and set expectations on both sides.

* Enterprises build reputations too. Kick the tires on three companies and do nothing and VCs stop bringing deals.

* The right frame for CIOs: decide where you want to sit on the adoption curve, then match the VC tier accordingly — seed investors for early bets, growth investors for proven businesses.

James: Let me pivot a little bit. I’d love to discuss a bit how enterprises, CIOs, CTOs, CSOs, others, should interact, with venture funds and venture-backed companies.

So Daniel, let me start with you. Advice would you have, for, technology executives or IT executives at big traditional enterprises as they engage with venture-backed companies and potentially as they engage

Daniel: Absolutely. First and foremost, large businesses are structurally challenged when it comes to engaging with younger, earlier stage businesses. Some organizations have tried to address that through innovation departments or groups built to test technology. We tell our portfolio companies to avoid those like the plague, because they are typically disconnected from the business owners themselves — the folks who actually need solutions to solve problems. We as a fund try to serve as a bit of a filter there, because it’s really hard as a startup to knock on the door of a Fortune 1000 business and get any attention. Our hope is that when we help knock on those early doors with our portfolio companies, that comes with a vetting process, and an understanding that we’re gonna ask for that time only when there’s good ROI going both directions. We tell our portfolio companies to go directly to the person who has the problem — try to engage directly with the business unit.

James: Will, what are the mistakes that large companies make when they interact with venture-backed, companies?

Will: Trying to push companies through an innovation team that doesn’t really own the problem is often ineffective and wastes everyone’s time.

As an enterprise customer, you have to decide where you want to sit on the adoption curve. If you’re willing to be an early adopter and really partner with a company, I think that can be incredibly fruitful early in their life cycle — you’re gonna get a lot more attention and probably more customization specific to you.

If you’re not willing to put in that effort, if you’re not willing to view this as a partnership, you’re probably better off waiting until, other early adopters have embraced the startup and it’s gotten to this inflection point where it’s now a more mature company, with a fully baked product.

And then third, you know, a lot of, a lot of larger companies now have venture programs where they’re investing in startups, especially investing in startups that they’re customers of.

If you’re gonna have a venture program, connect that directly to the business unit that’s actually buying the software and, and understand why you’re making venture investments. Don’t just do it because you think it’s cool

James: Okay, Ed, why is it so tough for, for large enterprises to engage with venture-backed companies?

Ed: Filtering. There are just thousands and thousands of startups, and our job is to filter through all those and invest in the ones that make sense. When you partner with venture firms, you should partner with all different kinds — because there are folks that go super early, at inception. There are folks that go a click later. There are folks that invest after the company has USD 30MM of revenue.

You need to understand where you are on the innovation curve and figure out which venture capitalists you’re willing to engage with to see what they have.

And we as VCs have to be very careful as well. We’re not gonna throw something at an enterprise knowing full well that it’s gonna be an 18-month sales process — because that could also kill the startup, but also we don’t want to waste your time.

It’s all about trust and reputation. On both sides, you have to figure out what you’re willing to show an enterprise to say that it’s ready for deployment. Maybe the company and team have actually built something new, and they’ve done it 30 times before — so therefore there’s trust.

And on the enterprise side too, they build reputations as well. Because if you kick the tires for 18 months and do nothing with three companies, VCs will say, “Don’t ever go to them again.” And therefore that enterprise might miss out on innovation.

It’s a two-way street where you actually have to trust each other.

James: How do you think about the filter? How do you think about who to speak to versus not to speak to?

Ed: Well, it just comes down to people though, right? It’s all about the people, right? You want people that move very fast that actually can solve a problem today, but also presents a kind of vision to solve your problem 18 months from now.

And that’s what we look for, and hopefully you come to us and we can present the founders that actually do the same

Will: The bar for speaking to somebody is very low. I’ll try to speak to as many people as possible. But for us, us to actually make an investment, we’re, we’re investing at a different stage, and so typically the companies that we’re investing in already have tens or even hundreds of millions in revenue. It’s a real business at that point, and the question is not will this succeed? Do they have product market fit?

The question is, what is the sort of revenue scale gonna look like five years from now? And what are the unit economics gonna look like five years from now?

We’re asking a fundamentally different question , than Ed or Daniel are when they’re making investments. We have a set of criteria that we filter to. We look for technologies that are inevitable, . There’s adoption, but we’re early on the S-curve. We look for markets that are default dominant, where an outcome is gonna be an oligopoly or a monopoly.

To Daniel’s point earlier, we don’t like markets that are commoditized, where you have hundreds of companies without any real moats. And then finally, we look for what we call dynastic DNA. It’s really about the people. Is this founder pursuing their life’s mission? Do they have the wherewithal and the fortitude to push through over the next ten, fifteen, twenty years to build a large, enduring business? Can they attract amazing people around them? Do they move at a high velocity?

James: So it’s your job, Will, to, speak to people, but if you’re a CIO or a CTO, it’s like, well, it’s my job to keep the business up and running, and I got, some number of hours per week to go speak to people, and an infinite amount of demand to be spoken to. So what’s your advice for someone who in an enterprise, role

Daniel: very nature of the venture business and the very nature of us having a funnel that ultimately results in a set of investments means when we are, making intros and trying to have conversations with corporate decision-makers,

if you ask for a busy person’s time and you do not use it effectively, you will not get it again. it is my, desire to be able to go back to the same folks time and time again when there’s an opportunity that’s relevant.

And so by the time I’m asking for the time of an enterprise CIO or an enterprise CISO, I’ve done my homework. I have done the work to know that even if it doesn’t result in an investment for us, it’s going to be a good use of both of their times.

Ed: It’s always not us pitching them. I find the most value is when I talk to CIOs or CSOs and say, ”Hey, what’s top of mind for you?“ Maybe once a quarter I have, like, a dozen folks I’ll catch up. ”What’s top of mind for you? What are the biggest problems you’re seeing? What are the existing, incumbent vendors not providing for you? What do you wish you had? What are you thinking about building?“ Right?

Because a lot of the things that startups end up creating are things that companies want to build themselves, right? So I think that’s part one. So once we understand that, we can say, ”Hey, I’ve got this one or that one.

We funded a company, um, you know, like a year and a half ago in the agent identity space. I, I walked into a very large bank and we just said, “Hey, look, I would love your feedback.” And, and they had eight people in the room. We spent an hour together. We, we mapped out what our architecture would look like, what problems we’d solve.

And then guess what? We went away, built stuff, raised rounds of funding, came back a year later — not only did we show that company that we had built everything, but we also showed them what the forward roadmap was.

And I remember as I was leaving the room, three of them were huddling in the corner and saying, “Holy s**t, did you see what they built in the last, you know, year?” We built it really fast, and then we had a future vision that kind of aligned with theirs. Another thing: we’re not expecting a sale. We want feedback. Why wouldn’t you provide feedback and spend the time? And then if we come back and make the mark, great.

James: You assume that VC funds serve as an intermediary between limited partners and founders. They channel investment to companies. But what I’m hearing here, and have heard in previous discussions, is you’re just as much an intermediary between startup companies and the enterprise — helping founders understand the enterprise, and helping the enterprise understand how to engage with startups.

Is that, is that a fair way of thinking about it?

Daniel: Absolutely. I have two track records. I have the track record to my LPs, and I have the track record to the entrepreneurs we back. And the track record that is for the entrepreneurs we back is: how many customers did I help you get?

How many versions of your product did I help you think about? how many key hires did I help you with? those are all critical things that are on the company support side of things. and it really involves being an intermediary with the enterprise, and understanding how to translate different languages.

A startup speaks a different language than a large enterprise. You gotta have somebody in the middle ideally making that translation

Ed: Look, I’ll toot our own horn here for the three of us — I’ve been doing this for 30 years, and not every venture firm does this. So if you’re a founder building in this space, come to guys like us who actually do this every single day, because we learn a lot during the process.

But all I’d like to say is our job is to help it, help you make it an unfair fight against your competitors so that, so that when you get funding from us, that we’ll , accelerate your path to your next round of funding and kind of product market fit or post-product market fit, depending on the stage.

One or two influential customers at the beginning of a company’s journey, like, of their first five customers, can actually reap tremendous dividends, create so much value, multipliers of value based on the contract size.

By the way, the contract size does not have to be massive. perhaps, James, that enterprise gets a massive discount in year one. The point is those referenceable customers are so massive for the success of the business because what you really want that CIO or CTO doing is talking to the next four or five large customers coming down the pike.

James: Will you invest in s- companies a little bit further on in the journey? How is your experience or your model similar or different?

Will: For a company that’s just starting out, having an Ed or a Daniel on your cap table is really important. It’s almost like they’re an extension of your founding team, to the extent that they can broker these relationships and help you navigate the enterprise. For us, the set of problems that a founder is facing is different.

They have often overwhelming market pull. The problem is not finding customers, the problem is scaling up the organization to meet the demands that you have. And so we try to help out a different way. It’s always great to bring customers. We try to do that where it makes sense. But I think often it’s more about talent.

It’s bringing in people who can scale, help you scale from 100 million in revenue to a billion in revenue. It’s more sort of strategic finance, and so how do you think about bringing in some of the larger crossover funds that will be shareholders, through an IPO and into the public markets?

It’s thinking about the next act. You have one product line that’s extraordinarily successful. You know, how do you start introducing a second product line in parallel, your second act? And so often we’re helping founders, navigate a different set of challenges than, than Daniel or Ed are.

Should you leave to found a company?

* The most compelling founding stories come from insiders who experienced a gap, built a workaround, and realized others had the same problem. Coralogix is the archetype.

* Ed’s checklist: Is this a market of one, or a market of many? Have you talked to peers at other companies? Who owns the technology? Who are you bringing with you?

* Domain expertise is necessary but not sufficient. The ability to abstract — to see that what solved your company’s problem could solve a category — is what separates a founder from someone who built a useful internal tool.

* Will’s criteria at the growth stage: problem obsession, grit, and “dynastic DNA” — the willingness to push past a billion-dollar exit toward a hundred-billion-dollar company.

James: So not infrequently, I have people working in enterprise technology organizations come to me and say, “Hey, I’ve done something really cool. I think maybe I should build a company out of this.” Maybe the grass is greener outside. Daniel, how do, how should someone think about that,

Daniel: Some of our most successful investments have come from that exact process where somebody is, at a business, they are experiencing a gap or a problem, and given their knowledge of the problem, their knowledge of the domain, decide, “Hey, I’m actually going to leave the company, and I’m going to build the solution I was looking for,.”

And, one of the, a, a really exciting company where we were involved from day one in the observability space called Coralogix is literally exactly that story, where , the founding team there was literally a product team at another business that built a solution they couldn’t find to buy. That’s a , very compelling, thesis. the challenge is a lot of times that thesis isn’t always right. The best thing to do is talk to guys like us, right? , Have a conversation while you’re still, ideating. Ed gets involved a little earlier than we do, we regularly meet with and advise founders that, aren’t yet founders.

They haven’t yet left their job. They don’t exactly know what they’re doing, but they’re really smart. They know their domain. They want to stay in the domain, and we want to stay close to them.so you gotta test the thesis, but you gotta test it with a toe in the water, not, jumping in the whole way, y-before that.

I, I, I always, when, when founders come to me and they say that they’re onto something big, I always say, “Show me your work. what validation did you do to, to achieve that conviction?” and to the degree we can be helpful in building that case with the founder that leads into an investment, great. but you gotta do the work to validate.

James: Ed, what do you think? And in particular, how should someone, what questions should someone ask themselves or him or herself,

Ed: There are a few lenses. You should have conversations with folks who do this. two is I think you should think about, is my solution a market of one, a market of a few, or a market of a many, ? Because I, I’ve talked to lots of folks building stuff, and they think they’re on their aha moment, but really their organizat- organization is so special that, I, I think it’s a snowflake, right?.

James: Abstraction is important

Ed: Do your homework, right? Also understand what’s in the market, ‘cause a lot of times also when you’re internal, you don’t understand what the other startup vendors are doing or what the other large incumbents are doing.

Like, why? Why is this built? Why’d you have to build this internally, doing a startup is really effing hard. You have to be absolutely stupid, insane, and crazy to go do a startup right now, and so I’m just warning you right now, it’s not all glory. It’s actually mostly 99% pain and 1% glory, right?

And, and so, so you better be pretty, pretty sure about that when you go out. And so let’s just have this conversation. A few things I’ll ask, will be understanding how’d you come to the conclusion that there’s just more than you. Have you talked to other people at other companies, your peers, for example, at other companies that might want to buy this solution? Two is who owns the technology? Are you gonna actually go leave and rebuild it yourself, or are you gonna spin this out of a larger org, and then perhaps maybe give them some equity, which also opens this a whole, a whole new ball of wax. I’ve tried that before, and that ends up becoming a very long legal nightmare.

I’m not advocating for that. You might be better off just going out and creating your own thing. If you’ve been locked in a corporate, corporate environment for, for 10, 15 years, man, I, I don’t know if you can... if you have even the stomach, that you can be a founder. so prove it to me, man. your, your salary’s gonna be 1/10th and 1/15th what you had before. It’s all gonna be about the equity. Who are you bringing with you? i-i-if you tell me, “I got five or six people I’m bringing with me” , I’m gonna know that you can recruit people, right?

I mean, those are the things you’re gonna have to think about once you get past even the does this make sense.

James: Will, what are, what are, what’s your observations about people who make for successful founders?

Will: I think first off, you have to be obsessed with a problem that you’re solving. I think that’s, that’s table stakes. And I think second, you’ve gotta have the grit and the fortitude to push through all the pain, all the moments of near death, all the suffering, and keep going and going. And for us at least, in the stage we’re investing at, we’re investing not because we think this could be a billion-dollar company.

Often it already is a billion-dollar company. It’s because we think it could be a $100 billion company. And any rational founder that got a billion-dollar offer , would sell their company and make tens or hundreds of millions of dollars and go retire.

It’s the people that, that have that fortitude, that grit, that drive to keep pushing farther and farther. And I think, it, it can happen. You look at one example is Daniel Dines, the founder of UiPath. he worked at a big corporate before, and, he decided to leave and, and start UiPath, and it took him, I think it took him, if I’m not mistaken, like six years to go from, zero to a million in ARR.

It was just this brutal slog. And so, you know, you look at that, and then obviously they went from, one to 100 million very quickly and, and now it’s a, it’s a large public company. And that’s the journey you have to expect going into this.

Daniel: Well, and also someone who can win, right? I, one, we talked about problem-obsessed. Absolutely. We talked about grit. Absolutely. we define it internally as like, do we think someone can win in their market? because it’s not enough to just have a good product.

It’s not enough just to, you know, have one of these pieces. You have to have all of the pieces, and you have to be resilient the entire way.

It’s actually one of the reasons why we focus on the Israeli ecosystem. , Really resilient founders, that have to persevere and go through a lot, that deal with real-world problems at a much younger age, that do their national service, that deal in real-world environments. and it creates a, a bit of a different type-built entrepreneur, someone that can, get up when they’ve been, you know, pushed over and, you know, when the market punches you in the face, you gotta get up. and that is , critical to being a successful founder.

James: All right. Let me ask for final words.

Will: I would say if you are a CIO or CTO trying to figure out where to spend your time, look inward. A lot of the most successful AI companies we’re seeing, at least at the application layer, are seeing incredible pull from the bottom up rather than being pushed top down. Go to your functional leaders, figure out where their problems are, figure out where they have pull, and focus your time and energy there rather than filtering all the people banging on your door trying to sell solutions top down.

Ed: Um, I would say that the time is now, right? Just don’t wait. Start small, start somewhere, just get rolling right now, and then don’t be afraid to throw things away. because I can promise you that by the time this podcast probably comes out, I know you’re fast, James, but the world may turn again one more time or two more times, right?

So whatever we say might be 90% irrelevant and 10% relevant right now. So I think the world’s just moving so fast. If I were a CEO or CIO, I would just build my own stuff on the weekend just to really understand the power of it.

That’s the only way you get to know these things

Daniel: We’re entering a really interesting moment where there’s not such a stigma anymore to try new things.

Companies need to look inward — to Will’s point — to really build the capabilities to integrate. And I think there’s openness now to try new stuff.

You’re not gonna get fired anymore for giving something a try. And I think that makes it a really exciting time to be an enterprise executive who can, to Ed’s earlier point, really put your foot on the gas on productivity.

James: Thank you all. This has been terrific.

Implications for CIOs and CTOs

Three things you probably expected going in: VCs are optimistic; enterprise AI adoption is early; legal and accounting are the obvious targets. One thing you probably didn’t: enterprises build reputations with venture funds too. Bring three companies through your procurement process, commit to nothing, and the introductions stop. The world is far enough from 2007 that waiting is no longer neutral. It has a cost now.

Thanks for reading Prosaic Times — subscribe to receive every issue!

Footnotes

[1] Ed is referring to Fin, the AI customer service company formerly part of Intercom. Salesforce announced the acquisition for USD 3.6B — above Fin’s ZIRP-era financing round, as Ed notes.

[2] The imbalance is structural, not cyclical, and the data supports Will’s claim. On the demand side, Epoch AI data shows frontier model training compute has been expanding at roughly 5× per year since 2020. On the supply side, the global stock of AI chips is growing at approximately 3.4× per year — aggressive, but mathematically insufficient to close the gap.

The constraint is no longer primarily silicon: transformer lead times have stretched from one year to five years (Bessemer Venture Partners), and RAND Corporation projections show single training runs scaling toward 1 GW of power per site by 2028 — approximately the output of a nuclear reactor.

Broadcom’s Q2 disclosures illustrate the production gap directly: bookings for AI accelerators surpassed USD 30B against USD 10.8B in actual shipments. The one credible counter is algorithmic efficiency improvement (~3× per year), which could flatten demand before infrastructure matures — but that remains a scenario, not a trend. For the physical infrastructure constraints in more detail, see my conversation with Harqs Singh of InfraPartners.

[3] Ed is referring to GeneralistAI, a foundation model company for robotics — Boldstart Ventures portfolio. Ed is an investor.

They responded. Then Senate Majority Leader Lyndon Johnson said Americans would not go to sleep by the light of a Communist Moon. The National Aeronautics and Space Act of 1958 created the National Aeronautics and Space Administration (NASA) with responsibility for the American space program. The National Defense Education Act of 1958 sought to dismantle John Dewey’s legacy in American education, pushing schools to replace “life adjustment skills” with set theory and symbolic logic. Less than 12 years after Sputnik, Neil Armstrong and Buzz Aldrin walked on the surface of the Moon.

Could Anthropic’s recent announcement of how Mythos can identify and exploit cybersecurity vulnerabilities create the Sputnik moment that will spur companies to use AI to change the way they operate enterprise technology?

The risks are real, and companies will need to move beyond buying tools and to build an agentic governance loop that uses a living graph of the environment to provide the context for spec-driven, immutable engineering, verified by adversarial automation rather than manual bottlenecks -- and then sustain and expand this change over time.

* Despite early indicators of transformative improvements, AI adoption in running enterprise technology has been shallow.

* Despite some fear-mongering, the impact of AI on the cybersecurity balance of power between attackers and defenders has been muted to date -- Mythos and subsequent models could change that.

* Mythos and subsequent models could dramatically improve companies’ cybersecurity posture in the medium term -- but they will need to use AI to accelerate their enterprise technology metabolism dramatically.

* Of course, the idea of a Sputnik moment is as much a warning as a call to action -- one-time programs are a lot easier than sustained cultural change.

AI adoption in running enterprise technology has been disappointingly shallow

Technology engineering and operations is one of the most exciting applications of AI for large companies. Large language models excel at interpreting and generating the structured content used in software engineering or technology configuration. AI can replace procedural programming with declarative programming [3], via spec-driven development. Agentic processes can better accommodate the edge cases and exceptions that have historically bedeviled efforts to automate technology operations. The early results have been exciting. My McKinsey colleagues have found that using AI to reinvent engineering processes can double team throughput. AWS has started to use agentic processes to reduce incident resolution time by three-quarters in some cases. Applied ruthlessly, AI could transform the economics of enterprise technology.

Yet adoption has been shallow. In last year’s DORA State of DevOps report, 90 percent of software practitioners said they use AI in some way, but most never used it in agent or autonomous mode and only 17 percent used it every day. The situation is no better with the cybersecurity team. According to a SANS report, Security Operations Centers use AI/ML tools, but don’t integrate them into their processes:

AI is present inside the SOC but not operationalized. Analysts use it informally, often with mixed reliability, while leadership has not yet established a consistent model for where AI belongs, how its output should be validated, or which workflows are mature enough to benefit from augmentation.

All this accords with my own observations: technology teams use AI as a tool to generate a code snippet or research an issue, rather than a lever to rip toil out of the way they do business. Why is this? The technology is still relatively new. Teams may be cautious or may not have the mental bandwidth required for change. Vendors have promised just installing a tool will solve their problems. And CIOs have not built the institutional support required to fund and prosecute the required change.

Mythos could change the cybersecurity balance of power between attackers and defenders

Since OpenAI released ChatGPT 4.0 in 2023 the great and the good have warned us about AI-enabled cyberattacks. The World Economic Forum said that specialized language models would allow hackers to get around endpoint security devices. The FBI said that AI would allow criminals to scale fraud schemes in a way that would swamp law enforcement. The UK’s National Cyber Security Centre said that GenAI lowers the barrier to entry for novice hackers allowing them to use vectors previously only available to experts. Some predictions approached fear-mongering -- sentient malware and HackerGPTs collapsing cybersecurity defenses. [4]

The worst...has not happened. I checked this morning, and the digital world continues to function. Only 16 percent of companies suffering breaches said they saw evidence of AI in prosecuting the attack. According to the Verizon Data Breach Investigations Report attackers have been just as dilatory as enterprises in using AI to reinvent their business processes:

It turns out the state-sponsored actors are just like legitimate organizations in their GenAI implementation life cycles. Attempts are being made, maybe some improvements are being found, but no one is revolutionizing anything yet.

At least as of 2024, GenAI tools could potentially assist attackers, but could not execute sophisticated attacks for them. One analysis found that GPT-4 only achieved a 7 percent success rate in exploiting vulnerabilities without clear human guidance.

Even before Mythos, the potential and the direction of travel have been worrisome. The structural factors that make LLMs effective in building and running systems also apply in compromising them.

* Intel matters in undertaking a cyberattack. LLMs have breadth of vulnerability knowledge no human analyst can read or retain -- LLM training data spans public CVEs, security research, disclosed exploits, and documented attack strategies.

* Success requires patience. Agents will cycle through potential vectors without boredom or fatigue.

* System compromise provides agents with a clear objective function they can optimize against.As a result, researchers have started to demonstrate that teams of LLM agents can cooperate to exploit zero-day vulnerabilities

Then came Mythos. Obviously we should be restrained in thinking about the implications of any software that isn’t generally available yet. And we’ve heard the too dangerous to release warning before. In its public statements, Anthropic said that Mythos had identified thousands of high-severity vulnerabilities across major operating systems and browsers—including legacy flaws like a 27-year-old bug in OpenBSD that evaded decades of manual audits. The model further demonstrated the ability to build complete, working exploits. Mythos can independently “chain” multiple vulnerabilities to gain a foothold, escalate privileges, and move laterally through a network, effectively allowing users with no formal security training to execute professional-grade, multi-stage cyberattacks at machine speed. Finding zero-days may get the headlines, but the ability to scale and operate autonomously may create the real risk.

This probably won’t realize the most dire predictions of 2024. [5] Several commentators have observed that a model’s ability to identify vulnerabilities and form plans doesn’t mean it will succeed in the face of sophisticated defenses (including the ones they have developed). But how many companies have sophisticated defenses like zero-trust in place comprehensively? And one compromise in the software supply chain could disable hundreds or thousands of institutions. Naturally, Anthropic and other frontier labs will seek to implement guardrails that limit attackers’ ability to exploit their models. The guardrails will not be perfect. And they will not apply to many of the open-weight models that will likely have Mythos-level capability within, maybe, a year.

Mythos and subsequent models could dramatically improve companies’ cybersecurity posture in the medium term. Could, not will.

After the Mythos announcement, American business and governmental elites acted. Anthropic delayed general availability of Mythos and launched Glasswing, giving early Mythos access to leading technology institutions so they could use it to identify vulnerabilities. Treasury Secretary and Federal Reserve Chair Jerome Powell called banking CEOs to Washington DC so they could urge them to take the risk seriously -- I expect they were pushing on an open door. Technology companies like AWS, MSFT, CRWD and CSCO reported that they were using Mythos to harden their products.

In the medium term, Mythos (and subsequent models) could provide a dramatic uplift in cybersecurity defenses. Companies spend fortunes each year scanning their code for vulnerabilities [6] -- Mythos-type capabilities will provide a level of transparency into vulnerabilities that we never could have imagined before. Most companies of any size do penetration testing, [7] but only the biggest tech spenders have dedicated red-team operations that figure out how a sophisticated attacker might compromise their environment. Mythos-type models should make this capability available to a much broader range of companies.

They may also revolutionize cybersecurity risk management and cyber insurance. Cyber-risk valuation frameworks like FAIR have foundered on the problem of likelihood assessment. Practitioners should be able to use a model like Mythos to simulate attack paths, determine the probability of success and make more fact-based remediation decisions. It could also revolutionize cyber-insurance, a segment historically held back by underwriting challenges.

And yet -- speed matters, and manual remediation is too slow. Mythos can help companies identify vulnerabilities. But identification protects nothing unless companies apply security patches from vendors and install fixes to code they have developed internally. That is the remediation gap the governance loop above is meant to close; in practice it breaks down into three concrete moves:

1. Create a living graph of your technology environment. You will very quickly face an overwhelming pipeline of vulnerabilities to remediate and vendor patches to apply. Not every one will be equally important, and the most critical nodes in your environment may not be immediately apparent given all the dependencies among business processes, systems, data and technology infrastructure.

Modeling your environment as a graph will allow you to identify the most critical nodes and prioritize what to remediate first. Ultimately every node in the graph should anchor in a non-human identity -- don’t connect IP addresses; connect non-human identities. Building the graph will also be an important step in moving to a zero-trust architecture.

2. Use spec-driven engineering to get to policy-driven systems. If you have bespoke software you will need to fix it. Autocomplete (or even asking models to write discrete code blocks) will not allow you to move quickly enough.

You need to retrain your engineering teams on how to use agents to diagnose root causes, build PRDs and execute on them autonomously. And you may need to do this on a timescale of months, not years.

As you develop strong capabilities in spec-driven development, you can accelerate efforts to retire technical debt, resulting in a more resilient environment. And you will want to define architecture, configuration and behavior in terms of policy-as-code so you can repave systems that demonstrate drift.

3. Move change control from human analysis to proof of safety. In many companies, the change approval board acts as a brake and a bottleneck on evolving the environment. It doesn’t have to be this way, and it cannot continue to be this way if companies seek to remediate the vulnerabilities Mythos identifies before attackers can exploit them. Heavyweight change approval processes are often ineffective. Teams of agents may collaborate to form an automated patch management pipeline.

Before you deploy a change, it must prove itself in a sandbox, both in terms of whether it breaks something and whether an adversary agent can compromise it, replacing the bottleneck of human analysis with the proof of safety. And you should deploy changes in stages, testing impact as you go. For years companies like Netflix have reconciled speed and safety by using canary analysis for staged change deployment.

None of these interventions are simple. [9] All will take attention, effort and time. But what is the alternative? Outsourcing might help, but it doesn’t remove the remediation burden at a stroke. Waiting for regulatory guidance (across dozens of jurisdictions and agencies) is uncertain and will likely take too much time. The age of security by obscurity is over. The cost of stasis may exceed the cost of change.

One-time programs are a lot easier than sustained cultural and organizational change

Less than a dozen years after Sputnik, Neil Armstrong and Buzz Aldrin explored the Sea of Tranquility. Ten more astronauts walked on the Moon in the next three years. Then, nothing. What poverty of the human spirit, what richness of bureaucratic incompetence caused us to tread on the moon and then retreat, without returning? Only this month has any human again transcended low Earth orbit?

Sputnik was a shock to the American educational system. By the 1980s, the National Science Foundation warned that Americans were in danger of scientific illiteracy. Shortly afterwards, the famous Nation at Risk report warned that post-Sputnik gains had wasted away. Not all the news is bad! American students have made large gains in fluid reasoning in recent decades -- and the dire standing of American students in global league tables may have more to do with compositional effects than school performance. But some of the news is really bad -- math scores have collapsed in the wake of Covid.

Just like a space program, the exploitation of AI in the enterprise is a generational project. Just like education reform, the ability of enterprise technology to use AI to build, run and protect systems is a foundational capability. Will your company move quickly enough to respond to the immediate challenge posed by AI-enabled cyberattacks? Will it sustain focus and attention over time to foster the capabilities to use AI not only to protect existing systems but to also make transformative leaps in business innovation, efficiency and resiliency?

Footnotes

[1] A great book. I read it every year in high school. The movie is pretty good too.

[2] Might I be evoking Arnold Toynbee’s theory of civilizational challenge and response here? Maybe.

[3] SQL is overwhelmingly the world’s most used declarative programming language. Perhaps its adoption provides the best historical parallel for spec-driven development. Replacing all the technical minutiae required for a query with a few SQL statements turned weeks’ worth of work into minutes.

[4] References not provided in order to protect the guilty.

[5] At first glance, you might ask: “How much does this matter for the enterprise? Once you get past the national security domain, how many attacks rely on zero-days?” More than you might think: a Mandiant analysis found that 70 percent of serious breaches they tracked involved a zero-day exploit. And of course a capable agentic attacker could assemble a sophisticated campaign out of a series of n-day exploits.

[6] The global security and vulnerability management market will shortly grow to the USD 20 billion.

[7] Itself a billion-dollar market.

[8] Special thanks to my colleagues Rich Isenberg and Charlie Lewis on these topics.

[9] These changes will require coordination across the technology organization. The infrastructure team will likely have to build the living graph of the environment, with input from each application team. Your core architecture or engineering team will lead the transition to spec-driven development, but much of the work will fall on application teams—and on infrastructure teams as they move from configuring systems to automating services. Transforming change control and patch management will require collaboration across the developer toolchain, infrastructure, and cybersecurity teams.

Thanks for reading Prosaic Times — share the entropy with a friend!

1. Every computer system we build is an imperfect representation of reality

“No, I didn’t go to Catholic school. Why do you ask?”

An eminent partner at a previous consultancy looked at me across an empty conference room and explained: “You have an instinct for mapping data onto a grid -- you have a Cartesian mindset, which many good consultants learn from a Jesuit education.”

Grids distort as much as they reveal. We’ve all seen (or built) grids for evaluating outsourcing vendors: 20 points for stability, 30 for service quality, 30 for cost, 10 for innovation. These never produce the right answer -- they ignore non-linearity. You won’t select a vendor below your financial stability threshold no matter how cheap; once a bidder clears it, you care about cost and service, not financial stability.

Every slide, every analysis, every system is an imperfect representation of reality. CRM systems are a simplified model of how your business interacts with customers. Operational systems are a simplified model of your factory. I learned this crawling around telecom carriers in the 1990s, where you had to ask the service techs which USOC “1FR” lines had bridge taps that had to be removed before upgrading to DSL.

2. The world is entropic

Do you know whether that deal will close next week and at what price? Whether your flight to Chicago will land on time? Or whether a new employee will turn out to be a hero or a goat? I do not.

As a recovering history major, I believe the past provides counsel on the future. On June 21, 1948, the Small-Scale Experimental Machine became the first computer to run a program stored in its own electronic memory -- 128 bytes of it. Weeks later, Weaver and Shannon published articles foreshadowing many of the problems we struggle with today: automating Byzantine business processes, capturing latent or ambiguous data.

Shannon defined entropy as average amount of uncertainty in a probability distribution. [1] Maybe that’s okay -- in a world without entropy no business could exceed the risk-free rate of return. The more entropic a domain, the more information required, and the more expensive the computer system to automate it.

Weaver laid out a relationship between complexity and uncertainty. Some problems are simple: hold all variables constant except the dependent and independent ones, and you get the steam engine, the automobile, the telephone. A payroll system is the modern analogue -- fixed variables, describable rules, few feedback loops.

Starting in the late nineteenth century, scientists attacked problems of disorganized complexity. Statistics could predict the frequency of calls in a telephone exchange or the claims paid by a life insurance company, even when individual causes remained opaque -- provided each data point was atomic. Fair Isaac developed the FICO score on this basis in the late 1950s.

Then Weaver addressed problems of organized complexity -- macroeconomic management, ecology -- where many inter-related factors multiply uncertainty because each may influence the others in non-obvious ways. Neither nineteenth-century analytics nor twentieth-century statistics could handle them. He hoped new computing devices might, and pointed to the operations research developed in the Battle of the Atlantic as a way forward.

Problems of organized complexity abound in modern business -- order-to-cash, pharmaceutical manufacturing. Given enough contracts, even payroll can turn into one. [2] How much time do knowledge workers spend massaging data into or out of enterprise systems because the system couldn’t capture every exception the domain implied?

We should forgive Weaver for not foreseeing the organized complexity required to address organized complexity -- ERP programs suffer their own, and enterprise technology functions suffer from it themselves.

Writing with Shannon, Weaver named another contributor: latent meaning. Ask an account executive the status of a potential sale and she might say “We’re in final price negotiations with the CHRO and he would like to get a deal signed within the month, but the general counsel is hacked off about a couple of legal terms we require and the CHRO heard he is going to call the CEO and ask that they reopen discussions with another bidder.” What is she supposed to select among the four CRM choices for “status” -- RFP open, RFP submitted, financial negotiations or closed?

Weaver and Shannon identified one more problem -- whether a system creates the desired behavior in the people who interact with it. Think about computer systems the way the military thinks about weapons systems: as inclusive of both the technology and the user. Design choices affect user behavior in unpredictable ways. Complicated interfaces cause users to resist entering data, or to prejudice what they enter to make themselves look good.

Calendaring is deceptively entropic. If Suzy, my assistant, asks me about a conflict at 10 am on a Tuesday, I might say: “I’ll go to the meeting with client 1, because I think that colleague A can cover me in the meeting at client 2. And don’t decline or mark me tentative for anything, because I don’t want anyone giving away the time slot because I might not be there.” I don’t verbalize dozens of assessments about people and situations that shape my decisions. You have entropic communication about organized complexity, latent information and adaptive behavior all in a couple of sentences!

3. Entropy creates cost and frustration when we build systems to represent it

How much time do we all spend in conference rooms debating how much granularity the data model should have -- and how we balance fidelity to the business process versus the cost of maintaining data? How frequently do you find the correct state for a transaction not in the fields designed to contain it, but in the free-text notes or the accompanying email? And how much user anger derives either from the endless fields (specified to capture the subtleties of state) or the disconnect between what the system says and the reality they observe?

Put another way: how much time do we devote to designing around entropy when we build systems? And how much frustration do we create because of the choices we make in doing so?

Operational systems work best when the world can be reduced to stable categories, deterministic workflows, and enumerable exceptions -- all these make a domain easy to formalize. Nobody complains much about most payroll systems because they apply a bounded number of explicit rules to available data.

Analytic systems work best with huge sample sizes and a few relevant independent variables -- making the entropy here tractable to statistical analysis. Machine learning systems for pricing consumer products work because massive amounts of structured data allow probabilistic inference where humans might have struggled to discern patterns.

Other domains and use cases have more intractable entropy. Systems to support treasury management for large enterprises? Modeling a thousand-page contract, involving hundreds of legal entities, dozens of jurisdictions and a thousand different exceptions you can have for a transaction -- all of which generate entropy. Which messiness must you model and which can you simplify away? CIOs have exceeded budget, blown deadlines and angered users in seeking to answer that question.

Yes, machine learning has blunted the impact of entropy in some cases by sniffing out the relationships among variables, but often the sample size is too small and the data too messy for traditional machine learning to be effective. Years ago, someone said collections managers could offer definitive recommendations about how to reduce losses -- why couldn’t CISOs do the same when talking about how to protect the business against cyberattack?

I tried to explain that each demographic segment included millions of households. You could experiment with a new script and quickly determine whether it increased or decreased promises-to-pay. Any large company’s technology environment is a snowflake -- even two companies of similar size and in the same sector may have radically different technology environments. Vulnerability depends not on individual decisions, but on how you connect all the pieces in your environment. And you might not know whether you had been breached for years, if ever. [3]

4. GenAI-based agentic systems process entropy

Deterministic systems cannot process entropy. They can rely on humans to pre-process it for them -- as happens when product managers and engineers sit in a conference room debating how to capture the data required to automate an ambiguous process. They push it on to users like account managers who must fill in endless fields about their pipeline -- and still wonder which option they should select to describe deal status. Or they can store it inertly, like the free-text notes that exist in some customer service platforms. All of these imply some combination of low efficiency and lower effectiveness.

When large language models process text, they convert tokens into vectors, points in a high-dimensional space. This encodes meaning through proximity -- similar meanings cluster together. Large language models can do things deterministic systems cannot do or do poorly:

* Allow inference across stored entropy: Written and spoken language are entropic. Vector embeddings allow LLM-based systems to query and analyze free-text notes, email threads and meeting transcripts.

* Identify implicit relationships: Vector geometry represents connections between concepts, such as the general counsel wanting to block the deal for group health insurance because he disliked the contract terms.

* Understand gradations rather than discrete states: A deterministic system requires the account executive to choose whether the deal has advanced to final negotiations or not. A vector representation of the same situation can sit between two states.

GenAI-based agentic [4] systems give us new ways to process entropy.

* At design time, by using software engineering agents to create business logic that reflects all the organized complexity that Weaver described.

* As part of the user interface, to mediate between the entropy of written language and a deterministic system, rather than forcing the user to do the work.

* Via data ingestion, to derive insights from the massive stores of entropic, unstructured data every enterprise has sitting on its servers.

* At run-time (either with or without a human in the loop), to make decisions and execute transactions non-deterministically.

We can also store data used and produced by agentic systems in different ways. Traditional relational databases excel at storing and retrieving massive amounts of transactional data at speed and with near-perfect reliability. They struggle with more ambiguous data, with dense relationships among all the elements. [5] We can choose to store this data either in vector databases or in knowledge graphs.

5. You have choices about how and where you use agents to process entropy

So much of the agentic discourse we see mirrors the flat Cartesian mindset captured in the grid used to evaluate bidders for outsourcing deals. Agents are risky! Or: everything will be an agent!

But entropy is, well, entropic. It varies in scale and shape from business domain to business domain. Sometimes it takes the form of a dense web of interconnections among idiosyncratic products, contracts, processes or customer relationships. Sometimes it takes the form of latent data that doesn’t fit into any data structure you might define. Not a few business domains suffer from multiple forms of entropy.

GenAI-based agents also have disadvantages compared to deterministic systems. GPU-based inferencing is slower and more expensive than CPU-based processing -- a single agentic transaction often costs cents and adds hundreds of milliseconds, where the equivalent deterministic transaction costs fractions of a cent and resolves in single-digit milliseconds. Non-deterministic systems are only as good as the context they receive. An agentic system can process the free-text notes in a CRM platform. It won’t know anything about the discussion two account executives had in the car on the drive back from the customer site. [6]

Abjure the false binaries -- you have real choices about how and where in the value chain you process entropy.

* Agentic software engineering: This will be relevant for almost any systems effort. It will allow you to automate all the business logic required to reflect organized complexity with more speed and reliability and less cost. And after agents help you develop the code, you can still apply all the quality assurance mechanisms you have developed over decades, just as you would code developed by hand.

* User interface: Retain traditional interfaces for situations where users have to enter small volumes of easily understood data. Develop agentic- or chat-based interfaces for order entry, CRM, or transaction processing systems where users complain about having to fill in screen after screen of data. You can always use a combination of deterministic rules and user validation to ensure the agent correctly captures user intent.

* Data ingestion: This is one of the most powerful and most underestimated capabilities. All companies receive and store vast amounts of valuable but entropic data -- eighty percent of corporate data is unstructured and even structured data can be fragmented and hard to correlate. Customer requests for quotation? Customer service notes? Legal contracts. Agentic capabilities can ingest all of this either to support operational processing or to generate new business insights, even if many companies have not focused here yet. [7]

* Run-time: This is complicated and has the highest stakes. Use agentic patterns for low-volume heterogeneous decisions. I am helping technology organizations use agents to compare project designs with technology standards. The agent will compare two artifacts (a project design and a standards document) that describe organized complexity in natural language (which is inherently entropic) and determine whether one conforms to the other. Hybrid patterns will probably become increasingly common, with deterministic cores and agents to handle edge cases.

* Data storage: Naturally, relational databases will continue to store structured and transactional information. Knowledge graphs excel in modeling product portfolios, customer relationships, process maps and other domains where relationship structure carries meaning. Vector search enables retrieval by meaning from notes, documents and transcripts.

6. Precision matters because entropy is, well, entropic

I have always hated the formulation “The technology is easy. The governance is hard. The organization is hard. The line at the cafeteria is hard.” Here, design choices will shape cost structure, risk posture and user experience for years.

With a capability to process entropy at scale, you have to think about where the entropy sits in your business system -- even a simple one.

Managing invites for the Technology Leadership Forum is lower-entropy than a hundred-million-dollar group insurance deal, but there is plenty of “Joe really wants to attend, but can’t, depending on when a personal conflict lands -- and wants to know if Sally can possibly attend in her place.” I put invitees and members into a knowledge graph and built Agent Serena to translate the entropy in my email into the organized complexity of that graph. [8]

Taking a more highly-scaled domain: entropy pervades enterprise change management in banking. Dozens of interconnected process steps and artifacts; latent information about why a milestone might be at risk of slipping and under what circumstances. Traditionally, banks managed it with managerial talent copying data among word processing, spreadsheet and presentation files. And many, many emails. The result: frustration, expense, and less insight into major programs than anyone would like.

Banks can use agentic software engineering to build a system whose agents ingest program documents, extract the relevant risks, issues, decisions and action items, and store them in a knowledge graph with the relationships among them. Which risks affect which business lines? Which work-stream lead owns which decision? Further agents interrogate emails and videoconference transcripts to enrich the graph. Vector descriptions let program managers search by meaning, not keywords. Still more agents read the graph to assess risks and surface opportunities to improve the program.

This combination -- agents, knowledge graph, vector database -- removes the entropy from a painful business process. It reduces cost and improves transparency. Those of us on team cyborg (rather than team android) will note that it also empowers managers: less toil, better information.

Agentic systems -- with knowledge graphs and vector databases -- can be transformative. When I wrote that global business must move beyond organized factories and chaotic offices, this is what I meant. The benefits are both economic and humanistic. Nobody grew up hoping to spend days pasting data from email to spreadsheets and back.

Precision matters here. Agents are not magic. Inferencing is expensive, latency is real, and new vulnerabilities arrive faster than we can grasp them. But the harder discipline is the one Weaver and Shannon left us: see your business as a system, find where the entropy sits, and choose where to process it. A flat, Cartesian grid won’t help.

Thanks for reading Prosaic Times — subscribe to receive every issue!

Footnotes

[1] This differs from standard deviation, which measures the magnitude of the spread around a mean. Shannon entropy measures the unpredictability of a distribution. Relevant to us: you can measure the Shannon entropy over a set of non-quantitative values just as you can over a set of quantitative ones. In addition, there are more and less expansive definitions of entropy. I am using a relatively expansive one here.

[2] CityTime is a great example of Kolmogorov complexity. The shortest possible description of the rules governing what every city employee gets paid is approximately as long as the rules themselves — decades of negotiated union contracts, grandfathered provisions, and exceptions to exceptions, each one load-bearing. You cannot compress it further without losing something that will eventually matter. Automation doesn’t reduce this complexity; it just moves it from humans who held it in their heads to engineers who must encode it in systems.

[3] My interlocutor said I was over-complicating the situation and CISOs were probably just incompetent. Sigh.

[4] Agents predate widespread adoption of large language models and genAI. By some definitions, the cruise control system in your car is an agent. You set a speed. It monitors the speed, accelerating when the car drops below that speed (because you are going up a hill) and easing off the throttle when you exceed it. Of course calling LLMs makes agents infinitely more capable than the cruise control in your car.

[5] The notion that relational databases might struggle with relationships among data elements might puzzle some. But only those who have never stared at a screen in the early morning trying to make sense of an outer join -- or who developed a better command of SQL than I ever had.

[6] I like the idea of decision traces and context graphs, but we should be realistic about which decision traces we can capture and which we cannot, unless we want to build a panopticon for ourselves.

[7] I stared in disbelief the first time Zubin Ghafari showed me how he used GenAI to integrate messy CMDB data with other telemetry information. I had assumed this would have taken him weeks.

[8] The natural language front end to Prosaic Graff was a life-saver, but the latency sometimes made me want to put my fist through the screen. So I created Python scripts I could run (instantly) from the Terminal that told me how many members planned to attend or the status of any individual member.

That same year, I told our teacher that the Cherry Hill Study Skills program was rudimentary that teaching it wasted my time and the taxpayers’ money. I left to go read a book in the library, and got me detention. A couple of years later, I asked my Spanish teacher to lower her voice in class as I had a hangover. Also detention. I butted heads less with professors in college, but I also skipped a lot of class to take care of the Brown Daily Herald.

Thanks for reading Prosaic Times — share it with a friend!

This guy. Me?

I have done a better job learning and sometimes teaching in my professional career than I ever did in high school or college. An email I just received reminded me of this.

Back in the day, Ed Hsu and I worked together fixing technology infrastructure organizations. He left McKinsey in 2010 and has since built a distinguished career at VMware, Rescale, and Mixpanel.

His note was brief:

hi James. This guy. You. :)

It included a link to his substack issue on The Personal Impact Chain, which read:

During my first weeks at McKinsey, a senior partner gave me the most concise career advice. Over dinner, I asked what I needed to get right first.

“Know the data,” he said.

Not “develop executive presence.” Not “build relationships.” Know the data. Everything else, the problem solving discipline, the story, the recommendation, was downstream.

Over the years, I built on it. The Personal Impact Chain is the result: a framework for staying grounded in facts, finding the insights that matter, building the case for change, earning the coalition to act, and delivering outcomes.

Accounts receivable optimization in Canada. In winter.

My first project out of business school (before even I worked at McKinsey) [1] sparked this insight. I know that the client wanted us to design customer programs to reduce Accounts Receivable, but the approach and the work-plan were mysteries.

Despite the confusion, I learned a ton on that study, including the importance of a healthy respect for the complexity and fragility of legacy systems.

The business unit’s IT department thought providing data for business analysis was not an important part of its job. Eventually I made friends with one of the DBAs, and we would log onto the mainframe together and download massive amounts of raw data that I imported into a SQL database I built in MS-Access. [For our younger readers, before you could use a cloud database…oh never mind.]

I could analyze consumer A/R patterns by BTN and enterprise A/R by major account. I could immerse myself in the data.

Like all the other consultants on the project I had a beat up metal desk on a big floor, and both other consultants and clients (some of whom had nothing to do with our project) would crowd around my desk asking me to query my Access database. People started calling me the counter-CIO. I liked being the counter-CIO.

I faced a limitation — I didn’t have any A/R or payment data on mid-market customers. After lots of investigation, I figured out the data I wanted lived in a system called CRIS (Customer Record Information System), but nobody knew how to get a query run against it.

So I sat at my beat up metal desk, and called everyone I could think of (using an actual landline telephone) to ask who could run a query against CRIS for me — then I would have all the data I needed. No luck.

Then one day my phone rang. The caller announced himself as Victor Rochambeau (not his real name), and asked if I knew who he was. Yes, I did. Victor was one of the most senior people in IT — not the type of person I was supposed to be talking to as a callow greenhorn.

Victor continued, “James, how old are you?”

I stuttered, “Excuse me?”

“How old are you?” he repeated.

“Twenty-six,” I replied.

“So you were born in 1970?” he asked. (This was in 1996.)

I assented.

Victor continued, “James, I have a new rule for you, so long as you are on this project. You are not allowed to request queries on systems older than you are.”

He went on to explain that they deployed CRIS in 1964 and ported it to minicomputers in the 1970s. And that ad-hoc queries crashed it. He was not crashing his system for my query.

Data is power

I never did get any mid-market A/R data, but familiarity with data increased my stature in small ways.

I thought it would be interesting to attend the first client progress review. So I asked my Manager, whom we called The Shadow), [2] suggesting I might be able to speak to the underlying data. The He scoffed: new associates had no place in progress reviews.

The meeting approach. After we printed out copies of the deck, [3] we watched the Shadow and the Senior Manager walk down a long hallway, wondering what progress reviews were like.

The Senior Manager and the Shadow stopped and talked for a minute. It looked like the Senior Manager was asking questions that the Shadow could not answer.

With a hangdog expression, the Shadow walked back toward us, pointed at me and indicated I should join them. So I excitedly trotted down the hall toward my first progress review. The project was a mess; the first progress review was a mess; and I learned a lot watching the mess.

At the end of that project I derived a theory of victory for my career in consulting. If you understand the data, you will be essential for the analysis. If you are in the middle of the analysis, people will want your point of view on the recommendations. If you understand the recommendations, you can help construct the narrative. Once you help craft the narrative you have an opening to participate in the client relationship. And then you’re getting somewhere.

That insight got me far. I resolved to understand more than anyone around me — about the telecom operations, then about technology infrastructure, cybersecurity, cloud computing and more recently GenAI in enterprise technology. I was a slacker in higher education, but I hit the books hard as an adult.

But I didn’t follow my theory of victory to the end. For too long, I focused too much on the data, analysis and recommendations -- and not enough on the narrative and how it motivates people to act. With much frustration, I haltingly learned to think as much about the second half of the chain as the first. Yes, the truth is the easiest thing to remember and you should ask every question. But also: to influence you must be open to influence and how you talk matters as much as what you say. [4]

Ed took advice I gate him and developed it further into something of his own — with a strong emphasis on the factors that I hadn’t focused on enough early in my career.

Pasted image 20260613225603.pngHe writes:

Product leadership is fundamentally about leading without formal authority. You set the direction, align stakeholders, and own outcomes; all without controlling the people who make it happen. That requires something more basic than product intuition: a personal operating system for getting from facts to impact, on any type of project, at any scale.

There’s a progression that underlies almost every professional initiative. It’s not in job descriptions, but it structures careers:

Data → Insight → Story → Action → Impact

Like spirals in nature, this progression runs in any scale. From a junior analyst delivering a report to an executive driving a company-wide transformation, it’s the same structural progression, just at a different scope and time horizon.

Action at the small scale is personal discipline. At the large scale it’s about diplomacy, dependency management and a dose of formal authority. At senior levels, this framework maps to how people are evaluated.

One of the pleasures of a long career is hearing how others might remember and use some of the things that you’ve learned. And you don’t get detention.

Thank you, Ed. You made my day.

Thanks for reading Prosaic Times — subscribe to get every issue!

Footnotes

[1] Yes, there was a time before I joined McKinsey. Dinosaurs roamed the Earth. We used fax machines.

[2] We had great nicknames on that study. The BA was Nermal. The other associate complained about everything except Texas, so we called him Lone Star.

I got everyone to call him Lone Star: our team, all the consultants on all the other work streams and all of the clients. Here’s how thoroughly he became Lone Star in our minds.

One evening I needed to ask Lone Star about something, but he had already gone back to the hotel. Almost nobody had a cell phone in that era (before AT&T rolled out the Universal One Rate plan), so I used the PBX handset on my desk to call the local Marriott switchboard.

“Get me Lone Star,” I requested.

“I don’t understand,” the operator replied.

Slightly frustrated, I said “Lone Star. Lone Star. I need to speak with Lone Star.”

Watching me, with no small amusement, Nermal told me: “James, I don’t think they know about Lone Star at the Marriott.”

[3] Back before everybody had giant LCD screens in every conference room...oh, never mind.

[4] Me at 30:

Scene: 5:30 pm, Friday afternoon, CIO’s office: she is packing up to leave for the weekend.

CIO: “Oh, yeah, we have time now. I’m trying to get out the door.”

Kaplan: “I should tell you about that cost savings plan we were supposed to validate.”

CIO: “Yes?”

Kaplan: “You told the CEO you had a plan to save $20 million.”

CIO: “Yes?”

Kaplan: “We dug into all the plans and the underlying data and we don’t think it’s $20 million.”

CIO: “How much is it?”

Kaplan: “Maybe $3 million.”

There is a time, a place and mechanism for delivering bad news. This wasn’t it.

I love Stephen Biddle’s book Military Power, which explains how armies have used the modern system of force employment (effectively tactical doctrine) to win battles. He explains that technology innovations since 1914 have requires armies to learn how to use:

* Dispersion

* Cover and concealment

* Suppression

* Small-unit maneuver

* Combined arms

* Defense in depth

* Decentralized initiative

Biddle points out that

* Technology provides nothing; the effective translation of technological change into coordinated tactics (via the modern system of force employment) provides everything, or least the ability to win battles and possibly wars

* Adopting the modern system of force employment requires a very different culture than more traditional forms of war-fighting, especially the ability to trust non-commissioned officers with a lot of responsibility

This tells us a lot about agentic software development

* Giving your engineers access to new tools buys you nothing; building a new method software engineering using some of those tools buys you 2x, or perhaps 10x, the productivity

* Adopting new methods of software engineering requires rethinking skills and mindsets

* The results could have strategic impact. For most companies, software development is the rate limiting factor for every new product, every new channel, every new market and most operational improvements. Enhancing software development productivity accelerates the metabolism of the enterprise.

I was excited to speak, with my colleague Matt Linderman, with Tessl CEO Guy Podjarny because of his ideas about how to create the scaffolding for agentic software engineering -- how to do it at scale. As always, Prosaic Times never endorses any product or offering, but we find great insight in hearing directly from builders bringing new capabilities to market.

1. The founder is a addict entrepreneur

In this section:

* Guy is x-Akamai CTO and Snyk founder, now founder/CEO of Tessl.

* He frames himself as an “official addict entrepreneur.”

* Matt is a McKinsey partner in New York leading the firm’s software practice — three to four years building an AI-first, now agentic, product and engineering practice. Grew up near an upstate-NY Air Force base that did AI research.

James Kaplan: Currently hanging out in southern Rhode Island with another Prosaic Times video podcast. Today we have my colleague, Matt Linderman from McKinsey, and Guy Padjarni from Tessl. Did I pronounce that correctly, Guy?

Guy Podjarny: Good.

James Kaplan: Guy, why don’t you tell us a little bit about yourself, and we’ll bring you to Tessl.

Guy Podjarny: Sure. I’m at this point an official addict entrepreneur. After being a developer and turning product and such through a few acquisitions, I founded my first company in the web performance space. I sold that to Akamai, where I became CTO, and did that for about three and a half years. After about three and a half years, I got the itch to found another company, and I went on to found Snyk, which I think had a good dent of impact on the application security world.

James Kaplan: Maybe just a little

Guy Podjarny: It had a couple of years of wandering the desert and near-death experiences that nobody remembers, because everybody remembers what happened post the sort of two years.

But then it grew nicely, formed the DevSecOps movement and the developer security space. So that grew nicely. And about two and a bit years ago, I accepted I’m an addict and fell in love with AI, and left Snyk, where I’m still chairman of the board, to found Tessl, focused on reinventing software development.

What is the new software development paradigm? That is what Tessl is focused on.

James Kaplan: You’re in London now?

Guy Podjarny: I am based in London. Born and raised in Israel, spent a decade in Canada, and I’ve been in London for the last 13 years.

James Kaplan: Matt, you wanna introduce yourself and tell us a little bit about your journey?

Matt Linderman: I’m a partner in McKinsey’s New York office, and in particular work in our software practice on engineering and product management topics. For the last three or four years, we’ve been building a practice around AI-first and now agentic product management and engineering practices, and lead that more broadly for McKinsey across software companies and across a number of companies outside of software — banks, telcos, et cetera — who are all going through this journey together.

And so Guy, very excited to chat with you about what we’re seeing, what you’re seeing, and mash this together as we go through the conversation here.

James Kaplan: And you grew up in upstate New York?

Matt Linderman: Grew up in upstate New York near an Air Force base that did AI research, actually. So I grew up with this very close to heart, and now live in the New York City area.

James Kaplan: Great. And we appreciate you classing up the joint here on the podcast by wearing a shirt with a collar, Matt. You’ve raised the bar for Guy.

Matt Linderman: I was forced to do it due to some bank conversations, but my T-shirt is available for later.

2. Code is disposable; context is the new code

In this section:

* Guy reframes “vibe coding” as the first leg of a journey toward agentic engineering — guiding the agent, not writing the code.

* The Tessl thesis, two years old now: code becomes disposable; context becomes the unit of work.

* Guy sketches the emerging stack — models → tools → context (skills, rules) → harnesses → factory lines → factories.

* Harnesses are deterministic guardrails on a probabilistic model: OpenAI’s no-commit-without-coverage rule; Intercom gating PRs on loaded guidelines.

James Kaplan: You’re allowed when it’s warm in London. Guy, give us a little bit of the big picture on the evolution of software development and the adoption of agentic — or spectroven — development.

I admit, I don’t love the term vibe coding. But tell us a little bit about where we are and where we’ve been and where you think the world is going.

Guy Podjarny: Vibe coding is interesting. I think about vibe coding to agentic engineering as a journey: there is vibe coding, where you just vibe away. But really what we want is agentic engineering. I’d say, thinking a little bit about Tessl’s journey, two and a half years ago — or just less than that, when we founded Tessl — we already had the conviction, which is more clear today, that software development will, at a high level, transform from revolving around code and implementation to revolving around intent and instructions.

It is more about guiding the LLM — at the time, and today the agent — to do what you believe to be right, versus writing the code yourself. At the time, we talked about how code will become disposable. It was a bit more heresy at the time. Today it’s well accepted that implementation will become something that is regenerated, and it’s less important.

We’re not fully there yet. We believe this will drive a new software development paradigm, and we didn’t really know what that is. What are the units? Today we’re starting to see the outline of what it is that you develop in this world. We’re still developing software, still creating something that will evolve, that we debug if it doesn’t work, that we observe in production, that we collaborate on.

So there’s still a thing we’re developing — what humans are developing — what I think of as the context development life cycle. I can describe a little bit of the stack that is shaping up, and I’m sure this will continue to morph and modify.

At the bottom of it, there’s the models. This is the new primitive we’re building on. Think of those like architecture. They’re like operating systems — whatever it is that you build, is it compatible with that layer?

A layer on top of that is a layer of tools that perform various actions. They might be your grep and FFmpeg. They might be custom built.

Above that is the layer of context. I think of context as the new code. We can dig into that a little bit more, but context is where most developers spend most of their time.

It’s conveying what they want the agent to do. We have skills, we have rules.

Further up the line, you start seeing harnesses. Harnesses are really about constraining the model — they’re harnessing the model. This is deterministic software that decides when to make decisions that are not delegated to the probabilistic model. Hooks, for instance — OpenAI saying, “You cannot commit, you cannot run Git commit until the test coverage is above a certain bar.”

Or Intercom saying, “You cannot open a GitHub pull request unless you loaded the relevant context around our PR guidelines.” These are examples where the harness — the configured harness — is telling the model, “You do not get to choose; this has to happen,” because it harnesses the model. And then in turn, harnesses compose up into factory lines.

These are more like pipelines. Harnesses are kind of like frameworks, if you will. Most organizations will choose and highly customize a framework, or they will build their own framework. They probably won’t have a gazillion of them, so it’s not the same as a dev. Up to the factory lines, they’re more like pipelines.

You want some consistent input. With a certain type of input, you have a successful output. You want to go all the way to factories, which are more like your development process. So this is a new software stack, and these analogies are helpful when we think about, okay, as we scale, what do we need?

What tools do we need? What practices do we need to build that out?

3. Stop spec-ing the product; start spec-ing the programmer

In this section:

* James offers three historical analogies for the shift — assembler to C, procedural to declarative, and entropy reduction down the traditional design chain.

* Guy takes the last two but insists this leap is larger than any prior transition, and is happening faster.

* The right comparison is to instructing humans: probabilistic, resilient, interpolating within boundaries — except agents can’t lose their jobs.

* Spec-driven engineering is a practice, not a product; specs are only one slice of the broader context-engineering problem.

* “Speccing the programmer” — encoding constraints, API choices, framework preferences, and billing limits alongside the spec itself.

James Kaplan: Which metaphor resonates most with you as we think about the movement to agentic development, or spectrum development? The first historical metaphor I sometimes think about is the transition from assembler to third-generation languages like C, which we went through in, say, the late ’80s, early ’90s.

And I’m old enough to remember a lot of rent garments and concern that, “Oh my God,” you know, third-generation languages would be slow and unwieldy compared to programs written in assembler. The second metaphor I sometimes think of is the transition, or the distinction, between procedural languages and declarative languages. And sometimes I think about when the world went from building database management systems by hand, or building databases by hand, to using SQL — which I think you can argue is a declarative language. And when you say spectrum development, that to me sounds like it’s declarative. And the third is entropy reduction — in the sense that in a traditional software engineering process, you went from, say, a senior business executive’s declaration of intent through to a business case, to a conceptual design, to a set of business requirements, to a set of technical requirements, to code.

You had humans at every step in the process, in effect moving entropy until you had deterministic code. Which of those metaphors resonates with you, or do they resonate in different ways?

Guy Podjarny: Yeah, I’d pick a spot if that was a continuum.

James Kaplan: Please.

Guy Podjarny: The last two. The analogy for software evolution — this is a bigger leap than any one of the changes that we’ve done. So even if it is directionally analogous, and that’s useful, ’cause we’re humans and we like analogies, and they help us reason about the world — but I think it is a bigger jump, so we have to acknowledge that, and it’s happening faster than the previous ones.

I’d say it’s somewhere between changing — taking that sort of entropy, or how do we instruct humans. As we cascade, in which we expect probabilistic behavior, we expect when we guide our reports — people that work for us — not to get perfect, but to get resilient. You want them to understand the spirit of what you’re saying and to have some judgment around when to bend the rules or expand the rules, to build it.

James Kaplan: Interpolate, often interpolate, right?

Guy Podjarny: Interpolate. And you want it within boundaries — which is always a tricky thing — and accountability. We should get back a little bit to accountability, ’cause the key distinction between that and the others is: a person who repeatedly makes bad decisions might lose their job, and an agent doesn’t really have that. But on the other side, where they’re meeting the other pull from software, is just the increased desire for abstraction. If you think about a bunch of these transitions you talked about — but even going into Java, and then in infrastructure going into the cloud — anything that became software-defined, suddenly you’re saying, “Hey, get me ten more servers.” You’re not dealing with anything. It’s just spin up ten more servers. Before, you said, “Give me ten more megabytes of memory.” You didn’t even say that. You said, “Give me an array of a thousand objects long.” And Java —

In each one of these things you’ve delegated decision-making to something downstream that you can configure, you can control, but is not entirely yours. The entropy piece is interesting because in all of those cases you still expect a lot more determinism, and a lot less resilience — a lot less adaptability — than you do with humans.

We meet somewhere between human instructions and a higher-level abstraction, and those are probably comparison tools as we think about what works and doesn’t. I’ll just refer one thing to the spec thing: at Tessl originally we talked about spec-centric software development. Spec-centric evolved into spec-driven development, which I believe to be a real practice.

I don’t think spec-driven development is a product. It is a practice.

James Kaplan: Of course. Yes.

Guy Podjarny: We should go from speccing the product, or speccing the program, to speccing the programmer. It’s important within the team — when you think about instructions, specs are just a part of the puzzle. What you expect from people in the team is not just to always update the information about the product and how it operates when they modify it.

That’s one of the things you want. But you generally expect them to make good decisions, and it includes that, but it also includes understanding your constraints and your preferences — how you chose what your API design is, how we chose to use this framework, what your billing constraints are — all of these other pieces. So I use spec-driven development, but I think it is a subset of agentic engineering and/or context engineering. Specs are just a piece of it.

4. Tech debt is becoming deflationary

In this section:

* Matt opens with the historical default — entropy increases in codebases over time, and tech debt is what we call it.

* Guy inverts the frame: tech debt is deflationary. If the modification will be cheaper in six months, accumulating debt can be the rational move.

* The carve-outs are one-way doors — architecture debt and data debt still warrant the cycles to keep fresh.

* James’s college-era friend said a program could be rewritten from scratch three or four times and made better; agents make that cheap enough to do routinely.

* At Tessl the team is pushing to eliminate interactive coding sessions entirely — one-shot prompts off a well-formed Linear issue, with tests as the enduring artifact.

James Kaplan: Matt, I’m sure you have questions. Let me not hog the mic here.

Matt Linderman: Well, just to add on to the entropy point, Guy — to your point, it’s a really interesting question, because in the past, at least, what we’ve often seen is that entropy increases in code bases over time. And you end up with tech debt, or different words to describe that. Now there’s a real interesting opportunity to think about how do we keep code bases evergreen and actually avoid the entropy degradation over time as you go forward.

Matt Pocock gave an interesting talk on this the other day about basically always doing your architectural reviews on a more frequent basis, keeping the code base more up-to-date and clean. There’s an interesting evolution there that I think historically was a one-way ship, and now we may actually be able to steer that in a slightly different direction, to maintain code bases far better than we have in the past.

So I think the entropy one is a really fascinating thing to look into, and we’ll see how it all evolves.

Guy Podjarny: I agree that you can maintain, because labor has become cheap. So you can do all these things that before were nonsensical financially. Now suddenly maybe they are. At the same time, there’s another view that is almost counter to that a little bit, and that is that tech debt is becoming deflationary. Whatever modification you’re gonna do in your code right now, in six months’ time it’ll be easier to make that modification.

The agents will be more able to help you resolve that. To an extent, this is an amazing time to accumulate debt because it’s deflationary. It’s gonna be cheaper. So if there’s a good ROI in terms of you not bothering with this, then you can do — heck, you’d be able to rewrite the whole thing in a path. There are types of debt that you need to be careful still of — maybe architecture debt, maybe data debt.

Maybe things that are one-way doors that are very hard to change — for those you might wanna invest the extra cycles, which are now cheaper and reasonable to do, to constantly keep it fresh. And then there’s the type of debt where, if you go even the step beyond, you might actually care less. It might be fine, because you would just be able to undo that. So it’s not worth the delay. It’s always about what’s on the other side of the equation — it’s not worth the delay or the opportunity cost to accumulate it.

Matt Linderman: Yeah, 100%.

James Kaplan: In college, I had a friend who’s a very good computer scientist, computer programmer, who liked to say that a software program could be rewritten from scratch three or four times and made better. I think what you’re articulating, Guy, is that now it’s a hell of a lot easier to do that.

We can imagine everything gets rewritten multiple times with what we’ve learned in the process.

Guy Podjarny: Yeah, absolutely. You have to work and adapt to get to that point. One barrier to that, for instance, is interactive coding sessions. At Tessl, when we develop software, we aim — we’re not fully there yet, but we aim to eliminate as much as we can interactive coding sessions.

Instead, you can say, “Fine, play around with Claude or Codex or whatever it is, build the thing that you want, help yourself shape the product to what you want.” ’Cause oftentimes as you build, you figure it out. Now translate all of those into the Linear issue that provides the right information, throw whatever it is — the code you’ve just created in the prototype that doesn’t go anywhere. That gets thrown away.

But the information, the learning out of that, gets done. And then it gets one shot. And if it fails one shot, you modify the information, you provide the relevant commentary, and you create that again. What that does is it puts you in a place in which the agent is, almost by definition, sufficiently informed to be able to build that.

Of course you want to then curate that context over time so that it doesn’t rot, so that it remains relevant. But yes, you’d be able to build it again and make it adaptable. Increasingly, code generation should become like compilation: “It’s okay, I don’t care if there’s a new version. I can compile this for this new version of Linux or whatever it is that I just have here.” You do come back to some sort of regular principles, which is you need to capture tests. You need to capture some definition of what good looks like, what correct behavior looks like. And you’re never gonna test everything — definitely not with agents — but you need enough test coverage, otherwise you cannot scale.

5. Taste is a preference you forgot to write down

In this section:

* In real enterprises the factory layer is mostly aspirational; for now the live action is in context.

* “Negligent skills” — those without safety instructions — force enterprises into governance: central registries, supply-chain controls, dedup, versioning.

* Skills rot like software; the carrot is auto-extraction from agent logs and PRs, the stick is the maintenance burden.

* “Taste is just a preference you didn’t bother writing down” — skills are how that preference becomes a rule, used both at development time and at code review.

* Three tiers of evals — regression, skill, project (unit / functional / end-to-end) — and LLM-as-Judge is good enough to make them tractable.

Matt Linderman: So Guy, you started to introduce the stack — models, tools, context, skills, harness, factory — which kind of grows in abstraction as you go down. I’d love to hear from you. There’s been a lot of talk on the upper half of that stack, all the way down to harness. The factories piece is really emerging.

I’d love to hear just what you’re seeing in practice in terms of the workflows people are building around the harness, and what impact you’re seeing day-to-day with the folks you’re working with.

Guy Podjarny: For sure. The reality is there’s the sort of AI-native tiny companies, full kind of greenfield world, and then you go all the way to the enterprises.

In enterprises, the reality is that there’s a massive chasm — both between the companies, and within the company as they grow. When you think about factories and factory lines, those are, in almost all companies of medium size and up, not the norm. They are a specific prototype, specific project, specific sections — they’re the forerunners as opposed to the majority. The majority of interest, or activity, that we’re seeing right now is around context.

Eventually the agent executes this stuff. It might be malicious. It might be vulnerable — vulnerable being things like it guides the user to put API keys in plain text, or things like that. Or it might be what I’ve come to call negligent skills. Negligent skills are skills that lack safety instructions — “add this to the database, update is needed, do not drop the table, delete the database,” or some sort of basic safety instructions. Once you get into risky skills, you naturally need the governance: who’s installing, what do I even have in my inventory, do people use it. To control, so they create a central registry. They control those, and all that. So that’s one pier. It’s the least sexy, but it is important for supply chain security, and it’s a blocker to roll things out.

The second thing that we see is challenges around standardization, reuse.

I heard a story that articulated this very well — a unicorn with about 1,000 developers — describing how everybody was creating skills. That’s wasteful because everybody’s creating the skill.

They’re wasting tokens. They’re wasting time. They’re creating a lot of the same thing.

So they put together a repo to be able to upload and share those skills, so everybody’s sharing those skills. Very quickly it becomes a mess — there’s a whole pile of duplicates: which one do I choose? They had compatibility issues — like compilation — one developer is using one agent, they publish the skill, and it doesn’t work well on another agent. To be able to collaborate, you need some basic software-like tools.

You need some quality barometer, and a means of knowing that it’s quality, some deduplication to be able to identify those, some versioning of the stuff that you roll out.

All basic stuff that we have for software.

Most organizations are not yet past that point.

There’s a carrot and stick over here. There’s the fact that skills rot just like software. They will get out of date. They live in a dynamic environment.

The software around them changes, the practices change, the learnings change. You have to maintain them, per the debt conversation we just had. And that’s the stick — you better maintain them, otherwise they’ll break.

And then the carrot is: can I look at agent logs? Can I look at the PRs? Can I auto-extract things that will improve that? That comes along. So that’s the exciting bit. What we’re seeing is that in organizations they do that on a nascent project. They do it on things where the blast radius, if the agent misbehaves, is relatively controlled.

James Kaplan: Fascinated by the ability to use context to enforce or encourage engineering standards. And I’ll give a very simplistic example. In the history of software development, nobody has been worse at naming variables consistently than myself. I am horrible about it.

I am the worst person in the world at it. And it was really interesting — as I started playing with Cursor and Claude Code, it’s, oh, I can set up some rules that determine how variables should be named. That’s an incredibly simple example, but it translates to a million things in terms of architectural standards and non-functional requirements. And we can be a lot more precise about how we engineer code and structure code, compared to a set of guidelines we would give to a new engineer or a relatively early-tenure engineer. And that to me is pretty exciting.

Guy Podjarny: It requires you to do something that many people don’t like, which is take the time to sit down and write down what good looks like.

I think with software, oftentimes we just don’t do the hard thing.

The word taste is thrown around a lot in the world of AI. And while it’s important, taste is just a preference that you didn’t bother writing down.

Skills are a very good way to enforce that. There are technical constraints right now — skills don’t always activate. So what we see in practical terms is there are three parts that you need to do.

One: you need to create the skill and write it down.

Two: you need to make sure that it is distributed. You need to make sure that it’s installed in various cases. On the Tessl side we help with that — both the tracking and the mandating of skills. But you need to know the skill was present when it was needed.

Third: you need to invest in verifying that it’s been acted on.

The beauty of skills is that you can use the same skill in two agent contexts. You can use the skill as part of the development process to say, “Hey, this is available to the agent to load,” and you’re trying to entice it to use it.

But then you can use literally the same skill in the code review process to say, “Hey agent, check if these practices have been applied,” because you’ve written it down once.

And that is actually a beautiful thing, because you can do it in both cases. And you can even further go on and say, “Look, historically, I learned that now this data pattern is not good.”

Not only from here on do you change that, but — Matt, to your comment on tech debt — go back in history and find all the cases in which that’s there, and set up a mini migration of something that you wouldn’t have bothered doing.

Matt Linderman: I’d love to maybe stick on this topic of context, ’cause obviously you’re a real expert in this space. One of the — I think there’s a two-part question. One is, when you look at the evolution, obviously the tools themselves have gotten a lot better at pulling in context, but what are you finding in addition to the standard, like grep and code-based awareness, that’s really critical to pull in?

I’d love to hear your thoughts on that. And then the second piece, which maybe we can go to after, is how do you then experiment and measure what context is more or less effective, and what should you be pulling in? We have a number of clients asking those questions and trying to wrestle through what is the information that they should connect via MCP, how should it be structured, et cetera.

And that comes down to some version of experimentation. Would love to hear your thoughts on how to think through that.

Guy Podjarny: I love the question.

The context window is the scarce resource that we optimize for. Everything is context. Your code is context. It’s important to separate between reusable context and real-time context.

In the real-time context you have things like prompts, tool outputs, and things like that. Those are more interactive context. It’s important to do prompt engineering and things like that — those are still competencies.

The models clearly need to make it easier, and they have interactive modes about knowing when to ask you questions.

There’s passive and active context. Passive context being the code itself. So you can do things like keep the code clean, add proper documentation, add some passive documentation inline — like MD files at the right spots.

There are advantages to continuing with a thing that the agent has regenerated. There’s a real advantage to having the agent rewrite the file, because when it rewrites the file it tends to follow a certain pattern that matches the training data that exists.

Its future decisions reasoning about that file are more likely to be successful. Clearly, it’s not practical to rewrite every file every time, but it’s still a useful guideline.

The third bit, though, that relates to all of those, is this reusable context — and how do you evaluate it, or how do you know that it’s good? This is really where the world is now evolving. Reusable context is mostly done with skills.

Rules are a more forceful set — your Claude MD, your Agents MD — information that is always shoved in.

But you have to be very careful about how much you put in there. If you put a lot in there, you basically make the agent dumber for everything else that it does.

Skills are more on demand — whether the user has invoked them or the agent has the hints to do them.

Define what good looks like for a skill.

Right now the most common barometer is the Anthropic best practices. Does it have progressive disclosure? Is it sufficiently concise?

But we see customers modifying it to their own barometer. So at least start by defining what a good skill is in your organization.

Does it have safety instructions?

Does it refer to data privacy?

The second quality measure is tests.

In skills, the equivalent is evals. In evals you define a scenario: here’s an environment, here’s the files that are involved.

Pull from this commit, modify this context file like this, install these different tools, and then you have a task.

In practice, running tests is just a lot harder to maintain, and LLM-as-Judge is pretty good. It’s like an expert reviewer.

So you define the criteria, you run the agent through it.

Test coverage is notorious: you can build amazing test coverage and have really poor quality controls with it, just because you’ve created useless tests.

State-of-the-art, which not many people are doing, is you have three tiers of evals.

You have things that are more regression tests — just a few samples to say this works.

Mostly they now serve as running something even in the CI, so when you’re modifying the skill it doesn’t break, and you can have some sanity checks on new models and things like that.

Skill evals — you’re evaluating the skill, like a unit test or a library test. You’re evaluating that unit of context on its own.

The next one up is more like project evals.

In this case you’re doing something that evaluates the entire project context. You now have 20 skills installed, and a bunch of rules, and a bunch of files, and maybe you’re giving a bigger task.

Those are heavier to run. You’re not gonna run them on every PR, probably, but you might run them on a weekly basis, on a monthly basis, to see that your context remains fresh.

And then the comprehensive test — the end-to-end test equivalent.

I think of the first one as unit tests. I think of the second one as functional tests. And I think of the third one as end-to-end tests or integration tests — which are comprehensive, so you can make strategic decisions based on them.

So, can I switch models over here? Can I run this on a different environment? Typically cost- or efficiency-related, but for things that matter.

6. The bottleneck is no longer coding; it is learning.

In this section:

* Guy’s diagnosis of what enterprises miss: how much they can actually steer agents — the binary “accept the risk or don’t” framing is the wrong question.

* At Tessl, 20% of engineering is dedicated to the factory itself; Guy doesn’t think that’s an overinvestment.

* Matt: the change-management piece — getting “I engineer the system” past the early-adopter teams to the rest of the org — is harder than the proof of concept.

* 10X productivity in five years is plausible, but the right metric isn’t coding speed — it’s iteration speed and time-to-market. The new bottlenecks are marketing throughput and user attention.

* Closing coda on the CS-degree question: don’t push the kid into the degree, push the kid to build something. Software matters more than ever; the university is the doubtful vehicle.

James Kaplan: So based on your experience, when you talk with enterprise CIOs and CTOs — banks, pharma companies, what have you, manufacturing companies — what do they not get about the future of agentic engineering? What do you think most people working in enterprise IT organizations need to understand about how agentic engineering will evolve over the next few years?

Guy Podjarny: A good question. People are confused between these two perspectives that feel binary.

I think people underestimate how much they can steer and guide the agents — both to success and to control — and that requires investment.

I’ve seen a mistake, and now I cannot deal with this creature.

They’ve seen the wonder around the vibe coding, so it’s: no, you just need to let the agent be and let it roam free.

They don’t appreciate just how much they can control it.

I think the companies that are at the cutting edge — they spend a lot of time on how they get the agents to build right, on enabling the agents.

And people underappreciate the importance, so they delay embracing the agent because they think of it as this absolute — all they need to decide is whether to accept the risk or not accept the risk.

James Kaplan: They can actually control a fair bit of that risk if they manage those agents.

Guy Podjarny: We have — at Tessl, 20% of my engineering team is primarily dedicated to improving the factory, to agent enablement. And I don’t think that’s an overinvestment.

James Kaplan: Matt, this accords quite a bit with what I’ve heard you say about operational change in software engineering — that it’s not just a set of tools, but a set of tools that fit into a broader system.

Matt Linderman: Yeah, to build on what you’re saying a little, Guy, we do a number of coaching initiatives with different folks, helping folks understand how do you move toward a more modern engineering stack. And one of the biggest things is actually more of a mindset shift: I’m not using a bunch of agents to just generate code, but I’m actually responsible for improving the factory, to use your word.

If I’m not getting exactly what I want, how do I go back into the skill, or the series of skills stitched together into some workflow, and re-engineer it in a way that gets me closer to that? And then that brings people, once you have that mindset, into all sorts of directions. How do we do the architecture, the context, better?

How do we feed InfoSec better into the different agents, so that they bring it in from the first time around? That simple mindset shift — from “I’m using the tools that I’m given” to “I am actually engineering the system that is then generating, and my job is to make that system better over time” — has been a massive shift.

You get a few teams usually that figure it out first. They actually are the ones building the agents, the workflows, et cetera. But what we found is if you can then get that mindset amongst the rest of the engineers, even if they’re already using what other folks have built as the baseline, they can then adapt it, they can shape it to work in their specific parts of the code base, and the types of work they’re doing.

But what’s quite interesting, at least from our point of view, is that a large share of that challenge is the change management piece that comes after defining and showing that it can work. Then it’s: how do you actually get everyone else working in that same way? And a lot of that’s the mindset shift that I was talking through.

Guy Podjarny: Yeah, no, I fully agree with that.

Look, it’s hard because — it is a change in the craft of what you’re operating. It’s a change in the types of mistakes that can happen.

With self-driving cars, when they make mistakes that are nonsensical, that a human would never make, people get actually mad. When there’s a leaf in the middle of the road and it thinks it’s a person and it would not continue, people are actually upset about it.

And I think we’re seeing things like that. There’s a new type of error that happens, and it’s hard for people to acknowledge.

You basically have this combo of: you see a problem, and you’re told, correctly, “Don’t fix the problem — don’t fix the symptom. Go upstream and get the agent to fix the problem.” Which loses a bunch of their craft. And is a new type of work that they might not want to do.

So yeah, a lot of this is culture change. I’m sufficiently a gray beard to have gone through the DevOps transformation. And DevSecOps, and the movement of security responsibilities.

These things are unsettling, and it doesn’t help that agents are happening maybe ten times faster.

It is a big change. So a lot of cultural — it always comes down to people.

James Kaplan: So for the companies that implement the model successfully — the operating model — is this a doubling of engineering throughput, a tripling, 50% improvement? Think five years down the road — what do you think the companies that enthusiastically embrace agentic engineering will be able to achieve?

I think it’s an order of magnitude. 10X.

Guy Podjarny: And potentially more. Not — again, maybe a bit of a common mistake — it’s not about the speed of coding.

James Kaplan: Of course.

Guy Podjarny: It’s the speed of iteration.

When you launch a product, you still should work iteratively. You still should build a minimal product so you get it out there and get people to validate it.

The amount of time it takes you to get a user, to get them to try the product, to give you the feedback and internalize it — those are all still relatively fixed.

You can only lightly optimize those.

But the type of product you can provide to them now can be a lot more comprehensive. You can bring them a product that is actually a lot more thought through.

Your ability to analyze and apply learnings from whatever it is that they did with the product is a lot faster.

And the number of people you need involved in each one of these iterations is a lot smaller, so you can learn more in parallel.

There are enormous opportunities to improve, and they compound. That sort of 2X improvement pace implies that if you’re at it and you’re progressing, you’re gonna be way ahead, ’cause of your pace of learning.

James Kaplan: You’re getting down the learning curve.

Guy Podjarny: It’s interesting to identify the new bottlenecks.

At Tessl we generally work in pairs most of the time.

That’s more because of organizational resilience. If someone’s on vacation or somehow cannot do it, then the other person can continue the work. They collaborate.

It’s mostly independently, and then they review each other’s work, so someone can step in a little bit more easily.

We’ve had a problem that we’re still working on fixing, which is product marketing is struggling to keep up with the pace of new capabilities that we have.

The answer is agents all the way down. You need to build more agentic analysis of what got built, move a few of the decision documents that happened earlier on, to be able to produce marketing material in parallel.

Once you have that, there’s still a scarce resource that we need to understand, which is the attention span of our users. You can’t email them ten times as many emails. You still have to send them a confined amount.

So it’s interesting to understand what the limiting factors are, what the new constraints are. And alignment is one of those.

But within each of those departments, more empowerment, more autonomy, less dependencies is a critical movement, because the cost of alignment relative to the cost of building is so much higher.

James Kaplan: Five years, 10X improvement in productivity via agentic engineering at best enterprises. You agree with that? More or less — what’s your view?

Matt Linderman: I think we need two things. One: it depends on the metric you look at, but I would say 10X order of magnitude makes a lot of sense. It could even be higher. If you look at your throughput metrics, the historical way of engineering — there are organizations getting five, 10X already, and they’re trying to now figure out how do you get that across the organization.

But I think that’s a bit misleading. To your point, Guy, really the metric to be solving for is more of a time-to-market view. If you look at the bottlenecks — it moves to your code review, it moves to your product management being able to build up the right requirements, it moves to product marketing, et cetera.

So really, I think there’s a measure of: one, if it’s a new product, how fast can you get to market? And then for existing products, how fast can you cycle through to get customer input, figure out what to build next, then go build it, go get more input, et cetera. I think that for sure can accelerate 10-plus X more.

But it requires real process change. You have to think, “Okay, how do I actually go get that customer input?” It used to be releasing it to actual customers, and then we went to alphas. Now can you even have customers on your team who can just test it and give you feedback every single day at 3:00 PM?

James Kaplan: Before I release something, before I write something, I have a panel of 500 virtual CIOs and CTOs who give me feedback. I often go through five or six rounds, and they help contain some of my literary excesses. Guy, final short question for both you and for Matt — would you advise a young person today to major in computer science if he or she were so interested?

Guy Podjarny: I would not advise someone to go do a computer science degree — not because — I think software development as a profession with some modifications will continue to live, and we will build software.

James Kaplan: And I think software will matter more than ever. I have a lot less faith that the universities would be the route — that the universities will be able to adapt. Your trepidation is about the degree, not about the computer science part.

Guy Podjarny: Yeah, exactly. Go build something. Go create a product. And I also feel like this is a world in which a breadth of perspective will come a long way.

Learning how to be a bit of a one-person army goes a long way — around touching product, touching marketing, touching your subject domain.

James Kaplan: Matt, computer science — 22-year-old or 18-year-old. Assuming that person will go to university, computer science or study something else?

Matt Linderman: Maybe building on what you mentioned, Guy — there’s never been a better time to build your own company. Going and doing that at some point will teach you far more of a breadth of experiences. Now, if you do go, I do think there’s really a skill around problem-solving, conceptual problem-solving, that is going to be applicable in any job that you have, in communications, et cetera.

That may come from engineering degrees, you could say math degrees, et cetera. But I would really be looking for something that pushes you in terms of how do you think, how do you structure problems, et cetera — that you can then bring into whatever type of work you’re doing moving forward.

And then go into the workforce and learn, as you said — entrepreneurship or not.

James Kaplan: Thank you so much. It was a great discussion.

Guy Podjarny: Thank you.

Matt Linderman: Thank you everyone.

7. What would Biddle say about agentic software engineering?

No single element of the modern system of force employment wins a battle—not combined arms, not suppression, not decentralized initiative. The doctrine wins.

The same is true of Podjarny’s stack: models, tools, context, harnesses, factories. None of them buy you anything in isolation. The factory is the doctrine.

And Biddle’s harder lesson—that adopting the modern system requires trusting non-commissioned officers with judgment they were not historically given—is the agentic problem in another voice. Just as you have to trust NCOs with fire teams, you have to trust engineers with agents.

The companies that learn it will not be 10X more productive in any narrow sense. They will be 10X faster at learning — which is perhaps the discriminant between victory and defeat in competitive markets.

Back when I did the research that turned into Beyond Cybersecurity, several people asked “does securing the software really matter if we can’t trust the chip?” or “wouldn’t it be great if the chip could be part of the security architecture?”

A highly capable and motivated attacker could compromise firmware supporting the most sensitive workloads, making security controls at higher levels of the stack ornamental. While the US military’s Orange Book (1983) first formalized the logical boundary of a Trusted Computing Base (TCB), its multi-user specifications assumed the underlying physical hardware was secure inside a guarded facility. Distributed architectures and modern threat models shattered that assumption, forcing the industry to move that TCB boundary down into the architecture of the chip itself.

The Trusted Computing Platform Alliance (TCPA) — which became the Trusted Computing Group (TCG) — developed the idea of a Root of Trust, with three pillars:

* Root of Trust for Measurement (RTM): The initial, immutable piece of code (usually embedded in the motherboard ROM or BIOS) that boots first and measures (hashes) the next component before handing over execution.

* Root of Trust for Storage (RTS): A secure memory zone within the chip that holds the cryptographic keys and the accumulated measurements. Software cannot overwrite or read these keys directly.

* Root of Trust for Reporting (RTR): The cryptographic engine inside the chip that signs those measurements using an endorsement key burned into the silicon at the factory, proving to an outside party exactly what state the machine booted into.

As of 2013, some devices had a Trusted Platform Module 1.2 (TPM), a distinct, low-cost microcontroller soldered onto the motherboard, completely isolated from the main CPU lines — but its separation from the main CPU both introduced bottlenecks and limited its security efficacy. Some data center managers declined to turn on this functionality because they feared the operational complexity it required could cause outages.

TPM 2.0 improved on 1.2. It embraced cryptographic agility and it moved the silicon root of trust closer to the CPU, often embedding it as an isolated firmware routine (fTPM) directly inside the processor chipset, eliminating the exposed external motherboard bus. To fix the datacenter uptime issues where a benign driver update would brick a server boot, it supported Policy-Based Authorizations.

But TPM 2.0 is an authentication and state-verification mechanism. It can verify that a system started up in a clean state, but it cannot protect data from being stolen while in memory.

To address these problems the Linux Foundation birthed the Confidential Computing Consortium, which published A Technical Analysis of Confidential Computing. This document established the non-negotiable baseline that unauthorized entities could not view or alter data or code running in a Trusted Execution Environment. This required chip manufacturers to redesign how processors handle memory management, CPU registers, and privilege rings. They couldn’t just build a “secure sandbox”—they had to modify the fundamental instruction set architecture (ISA) so the CPU could treat its own operating system or hypervisor as a potential threat.

Today, most enterprise-grade server and device chips have Confidential Compute capabilities, and companies are starting to use them. How do they use confidential compute to enhance the security of agentic systems without writing systems-level code to manage CPU instructions, page tables, and hardware register states?

Thanks for reading Prosaic Times — share it with a friend!

A discussion with Opaque Systems CEO Aaron Fulkerson

Opaque Systems aspires to bridge the gap between hardware primitives and the enterprise application layer, by providing software engineers with access to policy enforcement, orchestration and cryptographic proof.

CEO Aaron Fulkerson talked about what this all means as enterprises seek to implement AI-based systems. Prosaic Times remains studiously neutral on particular technologies, but hearing from builders in their own words is instructive, and we always respect people who are passionate about the products they develop.

Here are a few of the key takeaways:

Prompt engineering and code scanning are probabilistic mitigations of a problem that requires deterministic enforcement. Confidential AI uses the encryption key baked into modern CPUs and GPUs to do that enforcement cryptographically: what ran, where it ran, what rules applied, with a signed report any auditor can verify. The hardware capability has been sitting in silicon for a decade.

Regulators are starting to ask about runtime enforcement, not just configuration. The EU AI Act, emerging regulations in Asia and the Middle East, and likely HIPAA are converging on a requirement that isn’t “we configured this policy” but “we can cryptographically prove this policy was enforced at runtime.” That is precisely the proof confidential AI produces. The regulatory hook is probably what forces enterprise adoption ahead of any internal security posture. Design for sovereign cloud will only accelerate this.

The policy standards landscape is a fragmented patchwork waiting for consolidation. Anthropic ships mutually-attested TLS. Microsoft has its own flavor. Google, Meta, TikTok, ServiceNow each ship variants. None of it is portable across hosts. A consortium effort building on SPIFFE, EATS, and RATS is producing a superset specification where the policy travels with the data.

James Kaplan: This is James Kaplan with another Prosaic Times podcast. We have one of the most interesting people in the cybersecurity space, Aaron Fulkerson of Opaque. Aaron, welcome.

Aaron Fulkerson: Hi, James. It’s a pleasure to see you again.

James Kaplan: Why don’t we start out by talking a little bit about your journey? Let’s hear about your career and what brought you to the place where you’re at today.

Aaron Fulkerson: Sure. Opaque Systems spun out of UC Berkeley — the same lab that created a lot of the foundational AI infrastructure: Spark, Ray, vLLM. The team that runs the lab — folks like Ion Stoica and Raluca Ada Popa — are world-renowned, nerd-famous folks. I was approached by them while I was running a business unit at ServiceNow, and I was intrigued by their background building the essential foundational technologies that everybody uses.

Ion Stoica being the founder of Databricks certainly got my attention. When I spoke with them, what they helped me understand was that they had created a new technology called Confidential AI that allowed regulated enterprises, governments, or sovereign entities to bring AI to their most sensitive systems and data, with verifiable guarantees about what ran, where it ran, and what rules were enforced.

That’s now become a whole category that Gartner writes about and that all the silicon vendors, hyperscalers, and everybody else covers. It’s been an exciting ride over these last two years.

From confidential computing to confidential AI

James Kaplan: All right. Why don’t we step back? Tell us a little bit about the history of confidential computing — what it is, and why it’s important.

Aaron Fulkerson: Yeah. It’s been around maybe ten years, so it’s been around a long time. What it means is that the hardware — a CPU or GPU — has an encryption key baked into the silicon, and that enables two key capabilities. One is what’s called a trusted execution environment, or an encrypted runtime: you can run your workloads on data while encrypted. That’s what everybody thinks about when they think about confidential computing. There’s another capability of confidential computing that’s at least as important, if not more — the ability to do what’s called hardware attestation with that encryption key. What that means is that you can cryptographically sign software.

Now, confidential computing is a hardware layer that’s available on all hyperscalers. All modern CPUs and GPUs have these capabilities. It’s not specialized hardware. What this enables is what we call confidential AI — a software layer above the hardware, above Kubernetes — and that means you can have hardware enforcement of what ran, where it ran, and what rules were enforced at runtime.

I don’t need to tell you, James, or anybody who’s an AI builder, that you cannot enforce rules on AI agents at a software level. There’s a greater than 0% chance — at least 1%, typically higher — that these things are going to circumvent your software rules.

So you have to do this hardware enforcement, and the good news is this technology that’s been around for a decade has found a killer application: confidential AI.

James Kaplan: Tell us a little bit about that. Tell us why software enforcement doesn’t work for agents and why hardware enforcement does.

Aaron Fulkerson: Sure. AI systems are non-deterministic, so they behave in ways that are unpredictable. There are a couple of things here. There are adversarial attacks, which is what everybody immediately thinks about when you talk about an encrypted runtime — but adversarial attacks are far less of a threat than the non-adversarial ones.

So what’s a non-adversarial attack? These systems, designed the way they are with software enforcement, bleed sensitive data. Opaque, in collaboration with our partners at UC Berkeley and a bunch of ISVs and large systems integrators, has developed what we call an AI surface map for data bleeds. It catalogs 46 different ways these systems bleed data by design, simply because they haven’t been hardware-enforced. They will circumvent your software guardrails by design, and the only way to enforce the rules is cryptographically — hardware-enforced, as I described. It’s fundamentally different from legacy enterprise software.

We’ve shifted from a hardware compute layer to a software compute layer. As these agents build a context window, calling your most sensitive data, they pass all that context around to do software analysis — probability, right? That’s how language models work: an analysis that says, “Oh, here’s the highest-probability next one.”

That context lands in application performance monitoring packages like Splunk or Datadog. Your CloudOps or DevOps teams doing memory dumps may be able to see stock trades, maybe executive compensation — anything that passed through.

There are a whole lot of different ways this category of system, AI systems, bleed your data if you don’t upgrade the trust layer. The same way we had to upgrade the trust layer with the transition to the internet and encrypt data in transit, we have to upgrade the trust layer for these AI systems — or you’re just going to bleed your most sensitive information everywhere.

James Kaplan: Give us the use case or the example.

Aaron Fulkerson: I’ll give you some examples. We have customers like ServiceNow, Encore Capital, other financial services firms, and, in Thailand, SCBX. What they do is use confidential AI. They deploy what’s best thought of as an AI module, because they aren’t going to shift all their workloads onto confidential hardware in the cloud yet — they’ve already built a lot of their systems.

They’ll start with, “Hey, we’re doing some really sensitive data processing — stock trades, HR information, or a CFO tool for summarization and analysis.” They’ll deploy that, and using confidential AI, they’ll create policy enforcement at the infrastructure level. For example: it must run inside my virtual private cloud. It can only communicate over this subnet. All the way out to the agent graph, this agent can only speak with these tools. All of that gets rolled up into a strong, cryptographically enforced identity, so that before anybody sends data to a workload — I’m using “workload” generically here, to mean any AI workload or agent — they can inspect the signatures and the cryptographic enforcement.

Does it run inside my virtual private cloud? Does it have the data-ingress and data-egress policies I require? Does it use my identity? Whatever policies you want get hard-coded into the identity before you send the data, so you can verify the integrity of the system. When you send the data, it’s sent encrypted. The policies are verifiably enforced at the hardware level. And most importantly, James, you get proof. You get a report with cryptographic signatures that any internal or external auditor can check, proving exactly what ran, where it ran, and what rules were enforced.

Why agents require new types of guardrails

James Kaplan: Let me play this back to make sure I understand it. In the old days — say two years ago — if we were building an application and we had a set of rules about which tools that application could call, or where it could send data, we would scan code against a set of policies and see if it violated those rules.

And if it violated the set of rules, we’d send it back to the engineer and tell them to go fix things, right? Then we’d rescan, and we’d be fairly confident it wasn’t going to violate that set of rules because we’d scanned the code. So far, so good? Now, if we’re building an agent, the agent operates non-deterministically.

So we can’t know with certainty whether that agent will call one set of tools or not, and the guardrails we’ve placed around it… When we’ve tried to use prompting or other controls to limit the purview of that agent, sometimes that works, but you can’t say with certainty that it’s going to work.

However, if we build the enforcement of those rules into the hardware layer, then we can operate with certainty. Is that a fair summary?

Aaron Fulkerson: That’s exactly correct. You’re talking about the non-determinism of these systems — you’re exactly correct. Let’s say there’s a 1% probability that one of these agents bleeds data, which we know for a fact is at least 1%, probably greater. At 1%, maybe that’s okay if it’s a non-essential system or data set.

However, scale that to 100 agents and the math gets pretty frightening — you’re now at something like 40-something percent.

At 1,000 agents interoperating and communicating — which may seem like a lot, but I’m running thirty-something agents in the background on my desktop right now while I’m speaking to you — you’re at a 99.9% probability of bleeding data.

James Kaplan: And there’s plenty of workloads where one percent is unacceptable.

Aaron Fulkerson: Correct. Absolutely.

I was speaking with some industry analysts just last week, and one of them was asking, “Is it really that big of a deal? We’ve always had people who leak in elevators and hallways.” (A quick fact-check on my earlier math, by the way: at 100 agents, it’s 63% probability.) The analyst — from a big firm everybody knows — was asking, “Does it really matter? We have employees who talk in the elevator or the hallway, and they’re bleeding data at similar probabilities.” The response I gave, which their colleague already understood, was, “It does matter, because what an AI agent can achieve in a span of seconds or minutes vastly exceeds what a group of organized humans, even malicious ones, could achieve in a year.”

These things are operating at machine speed with human-like capability to reason, decide, and act. That’s quite different from an employee in the elevator accidentally leaking some competitive information. We’re talking about systems that can do damage within seconds or minutes that bad actors might not be able to achieve in a year. And furthermore, the leaked data — it used to be hard to sift through. Somebody leaks some data, is it really that bad? Who’s going to utilize that data? AI will.

James Kaplan: Yeah, there’s also an autonomous-vehicle analog here. You could probably make the case that an autonomous vehicle is safer than a vehicle operated by a human. But there’s just more cultural fear and nervousness around an accident caused by an autonomous vehicle than an accident caused by a human driver.

As we seek to advance agentic adoption, a problem involving an agent and exfiltration of data is going to be culturally more challenging than somebody talking in an elevator. So yes — I agree with everything you said, plus the bar is probably higher.

Aaron Fulkerson: Yeah, for sure. I think that’s true. To pull on that autonomous-vehicle analogy: hey, one bad driver can cause a lot of damage. But if you had a swarm of autonomous vehicles that could self-replicate on the fly, driving all over the freeway at a thousand times the speed — which is effectively what these agents are doing, compared to human drivers — don’t you think you should have some additional regulation and enforcement that’s verifiable in a deterministic way, rather than by probability or just rolling the dice?

James Kaplan: So let me ask a devil’s-advocate type question here — or maybe not a devil’s-advocate question, more an architectural-alternative type question. I saw a really interesting paper about how you enforce controls on agentic systems. The paper suggested that maybe the right way to do it is, instead of having the agentic systems operate directly, you have them generate procedural code in real time, which you can then scan against a set of policies.

Obviously that injects latency. But if you were an enterprise architect, how would you think about the option of using confidential AI versus the option of having an agent write procedural code that you’d then scan using traditional code-scanning techniques? How do you think about the alternatives there?

Aaron Fulkerson: I think it’s complementary — I don’t think it’s an either/or. Just to restate what you’re proposing —

James Kaplan: I’m not proposing — describing.

Aaron Fulkerson: You’re describing. Thank you. The scenario is that the agent generates deterministic procedural code, which then goes through a scan.

I think that’s complementary, right? You still have an agent making decisions about what procedural code to write, based on the sensitive systems it’s touching, the tools it’s calling, the data sources it’s pulling from. And I’d assert that even in that environment, you’ll want to have some rules — and you’ll have to have them, because it’s increasingly a requirement in most regions and nations to have proof of what ran, where it ran, and what rules were applied to that agent, even when it’s producing deterministic procedural code that goes through another scan. I suspect that whatever’s doing the scanning might itself be augmented by AI in some form, right? And there again, cryptographic enforcement, or hardware enforcement. I tend to use the two interchangeably because our customers — the people deploying confidential AI — are AI builders. They’re not the InfoSec or security team or the compliance team. So as soon as a lot of AI builders hear “cryptographic” or “encryption,” they go, “Yeah, that — I just wanna ship cool stuff.”

So I tend to use the two interchangeably because I want builders to know: no, this helps you ship cool stuff faster, because you don’t have to go through the protracted six- to eight-month surveys with your internal and external auditors that I’ve heard so many CTOs and AI builders describe as their current dilemma.

Hardware identity and the supply chain

James Kaplan: Trust me, I see it all the time. One thing that struck me hearing what you were describing is the ability to prove where something ran. Does this mean each chip has a unique identity, and therefore you’re able to confirm that it’s being run by who you think it’s being run by — in terms of an external service provider?

Aaron Fulkerson: That’s exactly correct. As I mentioned, with confidential computing there’s an — by the way, I’m oversimplifying. I’m not getting into too much technical detail —

James Kaplan: Every podcast everywhere oversimplifies everything, right? Otherwise they’d all be fourteen hours long.

Aaron Fulkerson: Exactly. I know somebody is going to say, “Well, the attestation service might be —” so I’m simplifying. But to your question: there’s an encryption key baked into the silicon, and that encryption key is what’s used to do the attestations. At the time you invoke a service, you’ll know exactly the profile. Where is it running? What is it running in — these Kubernetes nodes, that virtual private cloud? What are the network rules allowing it to connect to external services? Everything is encoded cryptographically and enforced by the hardware.

It’s physics. The significance here is, hey, can you trust AMD, Intel, NVIDIA at a hardware layer? And as long as you believe they know how to manufacture their chips, you don’t have to trust anything else. Nothing else in that stack do you trust.

James Kaplan: And do you know — or is there some way of knowing — that this individual chip was bought by this service provider versus that service provider versus somebody you may not want to be doing business with? Is there some registry somewhere? Apologies if that’s a naive question.

Aaron Fulkerson: That does exist. There’s actually a standard — a couple of them, in fact. SCITT is one, S-C-I-T-T. SALSA is another. They give you levels of verifiable provenance over the entire supply chain.

I don’t know if there’s a registry that exists independently. The technologies to do it exist, and I know all the silicon manufacturers and the hyperscalers have talked about having a registry so that they have complete supply-chain verifiability — but I’m not certain that registry exists yet. As an enterprise, though, you can bake that capability into confidential AI. That’s a feature of confidential AI, absolutely.

James Kaplan: Yeah. That’s an interesting idea. If you could say, “We know with certainty this is running on a set of chips that was bought by this service provider — there’s no chance it’s running someplace else,” or, “We know with certainty this is running on a set of chips located in this national jurisdiction versus that national jurisdiction.”

Aaron Fulkerson: Yeah. So I know there’s a podcast — AI Confidential — that I’ve had you on,

James Kaplan: Of course.

Aaron Fulkerson: A podcast that I host. I had a great conversation — I think on the very first AI Confidential — with Mark Russinovich, who’s the CTO of Azure,

James Kaplan: I’ve met him — he was on that panel I did at the conference you guys had.

Aaron Fulkerson: Exactly, yeah. And I’m looking forward to seeing you back at the Confidential Computing Summit at the end of June — the 23rd and 24th in San Francisco.

James Kaplan: I look forward to it. I’ll be there.

Aaron Fulkerson: Mark Russinovich, and Mark Papermaster — the CTO of AMD — in our very first AI Confidential were describing exactly what you’re talking about, James. And I was naive.

This was early in my tenure at Opaque. I was so focused on AI that I didn’t think about the entire hardware supply chain — securing it end to end. But to your point, you could extend this concept beyond provenance and supply chain to the actual manufacturer. You could do cryptographic enforcement of the design itself.

James Kaplan: I was going to say — that’s something incredibly important, I think, in the aerospace and defense community. They know exactly what the provenance of the hardware is, and exactly what’s running on what.

So let me ask a slightly different question. In some respect, what you’re describing reminds me of the early days of cloud security, when we started thinking about what became known as cloud security posture management — and we started to realize how important the business rules and policies were.

It took some time to sort out the model for defining those policies, for managing them, for figuring out what set of policies you wanted to have. Could you speak a little bit about that — about the process of policy management for confidential AI? How far along in the journey are people? What techniques do they use? What tends to work well versus less well?

Aaron Fulkerson: This is a really important topic you bring up. In the era of cloud, we did a trust-layer upgrade where we said, “Okay, SOC — we have to capture our policies at the time of deployment.” What’s different here, and what’s being demanded in a lot of different regions — the EU AI Act requires this, new regulations are coming online in Asia, same in the Middle East, and I believe it’s also true for HIPAA — is that you have to extend that to runtime.

It’s not just policies at the time of configuration; it’s policies at runtime, and you have to provide proof of runtime enforcement. The significance of confidential here is that it produces a report of all the policies that were enforced at runtime, and you can prove cryptographically that they were executed.

Because of the nature of agents behaving with human capabilities at machine speed, you have to have provable runtime policy enforcement. The good news is there’s been a proliferation of great frameworks and tools for policy-as-code. You don’t have to reinvent anything — you just take the policies-as-code, cryptographically enforce them, and measure at runtime that they were actually enforced.

James Kaplan: You’re building on all of these component parts. This is another one of those one-plus-one-equals-three situations you see in technology — the combinatorics of technology — where we already have this policy-as-code thing, and we already have this hardware key baked in. Right.

Aaron Fulkerson: Right. You add these two together and — oh my gosh — now we’ve actually got a way to adopt AI agents confidently and safely. Because we want to attach AI agents to all our systems and all our data sources — that’s how you get value. The more sensitive the data, the more value you can get out of it, but also the more risk there is in bleeding it. The key point is that this new security requirement in the agentic system, like all things in tech, is something we were doing previously, brought forward in a new way.

Sovereign AI and the standards landscape

James Kaplan: So how does this manifest itself in sovereign AI? Hark Singh, the CTO of InfraPartners, did a fireside chat at the Technology Leadership Forum last week, and was talking about how sovereign cloud and sovereign AI are an increasingly significant demand driver in the data center space.

How do confidential computing and confidential AI intersect with the desire for sovereign cloud and sovereign AI? I was wondering if you could speak a little bit about those dynamics.

Aaron Fulkerson: Absolutely. It’s an essential cornerstone capability. Look no further than Dr. Najwa Araj of the UAE’s ATRC and TII, who stated in a recent press release with Opaque that you can’t have sovereign without this kind of verifiability.

James Kaplan: And look at Jensen’s keynote at the April — was it April GTC?

Aaron Fulkerson: Was it April or March? I forget. I think —

James Kaplan: I think April, but I’m not sure.

Aaron Fulkerson: Jensen had it at the center of his keynote — the central topic was confidential and verifiability. Why? Because he understands what we’ve just been talking about. He said it in every session after the keynote: in order to safely adopt enterprise or personal agents, you have to have verifiability. He’s talking about confidential, hardware-enforced verifiability. That’s exactly what he means.

So why does this have to do with sovereign? If you look across all the big tech players, all the silicon vendors have oriented themselves around these confidential capabilities. Hyperscalers are demanding confidential. Frontier model labs are adopting confidential. What’s going on?

The hyperscalers demand it because they know they won’t get adoption in the rest of the world — outside the United States — if they can’t verify data privacy for sovereign cloud operators. So they’re demanding it from the silicon vendors, because otherwise the hyperscalers are going to get their lunch eaten by NeoClouds in the rest of the world. That’s happening right now.

For the silicon vendors upstream from the hyperscalers: if they don’t provide these verifiable guarantees, China is knocking on the door with great chips. They’re coming to eat the silicon vendors’ lunch, so the vendors have to differentiate. They’ve found that differentiating around confidential is a great way to do that — because the hyperscalers need it.

What about the frontier model labs? Why are they so insistent? They’re making incredible models — their intellectual property is their model weights. If they can’t protect their model weights in a sovereign cloud in the rest of the world, what’s their differentiation?

James Kaplan: Property is going to be stolen? And that’s what confidential does for the frontier model labs — it allows them to deploy in the rest of the world with verifiable proof that they protected their intellectual property, their model weights.

So I heard a couple of things. For the labs, you can use confidential AI to protect the model weights. For users and for AI or cloud providers in the rest of the world, you can demonstrate you’re compliant with local privacy regulations, which may mandate encryption, and compliant with data-localization regulations that demand you say, “Okay, this type of data or this type of workload can’t leave this national jurisdiction.”

Is that a fair way of thinking about it?

Aaron Fulkerson: That’s the simplest requirement, and it’s a good one to focus on.

The other thing we’re seeing in the rest of the world around sovereign goes back to what we already discussed — how do we reliably enforce policies on agents, not just data residency but every kind of policy. And another thing we’re seeing a lot is that many of these sovereign nations have their own frontier model labs, or they’re fine-tuning models with highly sensitive healthcare data, and they need to make sure that’s protected.

James Kaplan: So let me ask this question. You talked about what each participant in the ecosystem wants out of confidential AI. Where do you think things will shake out — in terms of what happens at the enterprise layer versus the application-vendor layer versus the cloud layer versus the frontier-lab / model-provider layer? How will those different pieces of the ecosystem interact with each other, and what functionality will they provide around confidential AI?

Aaron Fulkerson: Over the next two years, everybody will begin to offer this. We’re already seeing it. We saw it last year with Google’s offering — mutually attested components, or confidential Gemini. Everybody’s going to be offering endpoints to their language models and their applications.

That way, you can do mutual attestation and roll up verifiable guarantees. In fact, we’re working with a consortium of silicon, cloud, and frontier-model-lab partners to create an open standard that lets you create rules — not just on a single host, but portable rules that you can pass between hosts and that govern the policies of a particular workload. If you’re going to send your data, you can send it with, “Hey, here’s the specific set of rules I require enforced,” or it fails — and if it fails, I can prove that you couldn’t even access my data.

A lot of these standards already exist, but they’re very fragmented. To your point, you see frontier model labs enforcing their own flavors of confidential — Anthropic, a customer of Opaque, uses mutually-attested TLS, right? That’s a slice of policies being enforced. Microsoft does something different — their own flavor. TikTok, Meta, Google, ServiceNow — everybody’s got their own emerging patchwork quilt of cryptographic, hardware-enforced policies, but they’re not portable and they’re not standardized. What we’re going to see — and I think there’ll be an announcement at the Confidential Computing Summit this year on June 23rd and 24th in San Francisco — is that we’ll take all of these standards from the Linux Foundation and IETF, like SPIFFE, EATS, RATS, and say, “Here’s one standard that’s a superset, that you can enforce at your organization, and that will be portable as long as you write against this specific open specification.”

James Kaplan: Terrific. Anything we didn’t speak about? I’m sure there’s a zillion things. Thank you for joining us.

Aaron Fulkerson: Hey, thanks for having me, James. Look forward to seeing you.

Thanks for reading Prosaic Times— subscribe to get every issue!

A more honest answer involves Professor Litchfield’s class on The Industrial Revolution in Early Modern England.

Using spinning jennies and mechanical looms in factories to make textiles changed everything in England. It moved people from the countryside to the city. It allowed ordinary Englishmen to own a second suit of clothes. It demolished artisans’ livelihoods. It empowered a rising class of bourgeoisie, and undermined the aristocracy. It created the surplus that underwrote an empire. Software applications and computer hardware were the spinning jennies and mechanical looms of my own era. All especially relevant as we debate the social, political and economic implications of AI.

Professor Litchfield liked to say that “Political Science has the theories. In the History Department we are custodians of the facts.” I like to know what happened; only facts can tell you that. But the facts of history are imperfect. Much of the work in formulating a business or technology strategy depends on work that feels like history — trying to make sense of incomplete, contradictory, subjective, and sometimes unreliable information.

Bob Pasker co-founded WebLogic, where he led development of the first independent J2EE application server. BEA later bought WebLogic and Oracle bought BEA. [1] Now he’s pursuing a PhD in history at the CUNY Graduate Center. To support his research, he developed a machine-learning tool he calls Roscoe to discern new insights in American legal history.

Today’s interview excites me for two reasons. It’s always good to spend time with a kindred spirit passionate about both history and technology. More importantly, using AI to discover more facts intrigues me no end. In an academic environment it allows us to understand ourselves and our world better. In a corporate environment it allows us to make better decisions and pursue better strategies.

Here’s the takeaways:

* Wall Street, early internet experimentation, WebLogic and J2EE, post-acquisition chapters, then a PhD in history — technology innovation and academic aspirations can reinforce each other!

* Enterprise stacks still echo mainframe-era problems, but the internet forced looser transaction models, distributed-systems humility, and resilience design given shared infrastructure and unreliable networks.

* Mass digitization of historical documents means you can ask new questions at scale, but you have to cut through the “silence of abundance.”

* Roscoe is semantic retrieval across collections — embeddings, ETL, metadata, re-ranking — aimed at evidence that keyword search will not find.

* The hard problems ahead are precision, recall and cultural acceptance, richer analysis of hits, multimodal corpora, and partnerships with archives. Yes, the interpretive payoff is substantive, but attachment to existing methods bedevils the academy no less than the enterprise.

Thanks for reading Prosaic Times — share it with a friend!

1. From Wall Street and the VAX to WebLogic — and back to graduate school

Bob traces Wall Street transaction systems, the Java and Usenet milieu, co-founding WebLogic and shipping the first J2EE implementation, later ventures and CTO-for-hire work, and his return to history for a PhD at CUNY.

James Kaplan: James Kaplan here with the Prosaic Times podcast. I’m very pleased to have Bob Pasker with us. He played an important role in the development of modern software over the past couple of decades, and he’s now doing exciting work as a history PhD candidate. Bob, give us a little bit of your background as a tech leader and entrepreneur, and what drove you to pursue a PhD in history.

Bob Pasker: That’s correct. Thank you very much for having me on, James.

I started way back on Wall Street, building transaction processing systems on the good old VAX, if you remember that. I got interested in moving to San Francisco because I wanted to be part of the tech community in Silicon Valley. I moved to San Francisco and started working for one of the database companies back then. It wasn’t a great company and I didn’t stay very long, so after about ten years in the computer industry I decided to go back to college.

I went back as a history student, got an undergraduate degree, and decided I was going to be a college professor. I went out to graduate school as a history student. Graduate school didn’t really agree with me at the time, so I came back to San Francisco just about when Java was coming on the scene.

I decided to do some experimentation. I built an SNMP stack in Java and was able to browse public servers on the internet completely undetected, because nobody had Cloudflare or anything like that at the time. I was active on Usenet, if you remember that — before Friendster and MySpace. It’s how people communicated with each other over the internet. I met other people who were interested in enterprise Java, and together we co-founded a company called WebLogic.

Our first products were JDBC drivers for accessing Sybase and Oracle. My interest, though, was building a transaction processing system in Java. I spent eighteen months building the WebLogic application server, which is what it became known as. That kind of launched the whole enterprise Java thing. Sun adopted our model, if you will, of having all these different services available, and they called it J2EE — and we had the only working implementation out of the gate.

The WebLogic company was acquired by BEA in 1998, and BEA was acquired by Oracle in 2008. That’s how it became part of Oracle’s technology stack. After that I started another company to do what we called the real-time internet. This was around 2000. The idea was server push instead of request-response, which is something we now take for granted. It was fairly early technology, and we didn’t make much progress because we got caught up in the dot-com bubble.

The more things change, the more they stay the same. A lot of what we were doing back on the VAX a long time ago still gets done today in enterprise systems.”

Over the next ten years I spent time at various venture firms, including Accel Partners, and I was kind of a CTO / chief architect for hire at VC- and PE-backed companies. I also spent a year at Expedia rebuilding their enterprise architecture.

Getting into the COVID years, as my kids started to get older I had more free time, and I decided to go back for my PhD in history. That’s how I wound up at the CUNY Graduate Center as a PhD student in American history.

James Kaplan: As someone who’s passionate about both history and technology, I applaud your varied career.

Sun adopted our model, if you will, of having all these different services available, and they called it J2EE — and we had the only working implementation out of the gate.

Bob Pasker: Thank you. I’ve gotten to do a lot of different things. Sometimes I feel like I’ve had five or eight different careers instead of just one.

James Kaplan: It’s a hell of a lot easier to understand the monumental changes happening now if one has a bit of historical sensibility and a historical mindset.

Bob Pasker: For sure.

2. Enterprise architecture: what changed, what didn’t, and why resilience still wins

VAX-era patterns persist, but networks are more decoupled; ACID loosens over the public internet, fallacies of distributed computing still apply, and major outages remind us nobody is immune.

James Kaplan: Before we dive into some of the research you’re doing now, any reflections on the evolution of enterprise architectures over the past twenty years? Obviously the app server was a tremendous advance. We’ve moved on to containers and so forth — but any reflections on that arc?

Bob Pasker: I think the biggest thing is that the more things change, the more they stay the same. A lot of what we were doing back on the VAX a long time ago still gets done today in enterprise systems. But I think the biggest change has been that networks and computer systems are designed much more robustly and to be much more decoupled.

We always thought about ACID transactions and the problem of doing ACID-style transactions over long distances. It was very easy to do inside the data center, but banks and other companies required that either both sides of the transaction completed or neither did.

Now we have much more flexible ideas about how that happens, and a lot of it has to happen over the internet, with the unreliability that implies. So we’ve really taken a new look at ACID transaction ideas and relaxed them enough to make it all work over the internet.

James Kaplan: I’m old enough to remember when many people didn’t believe servers could be physically remote from one another — a combination of the fragility of the application architecture and the fragility of the network. If the servers weren’t in the same facility, they didn’t have confidence in the ability to transact in a robust way.

Bob Pasker: Absolutely. I used to send people the list of the fallacies of distributed computing. One of them is that the network is reliable — and it’s not. Another is that they’re all managed by the same person — and they’re not. Whoever wrote that list was very prescient about what we needed to do to build reliable systems on the internet.

James Kaplan: You’re making an important point that belongs under “the more things change, the more they stay the same”: it’s critical to design for resiliency. You can’t assume any given component will be perfect, so you have to design a system that’s robust in the face of degradation.

Bob Pasker: Absolutely — that’s basically where we are today. We still see it every day: status.x.com and all the other status pages keeping track of what’s working or not on the internet, and we’ve taken that to heart. The biggest catastrophes are when major pieces of infrastructure go down. We’ve seen it with Cloudflare and Amazon and all sorts of companies — nobody is immune.

James Kaplan: I remember spending a lot of time on geo-resilient architectures starting around 2010 — application and system architectures that would be resilient in the face of network failure or infrastructure downtime. Let’s pivot to the research you’re doing now.

Bob Pasker: Sure.

3. Legal history, massive digitized corpora, and the dissertation problem Roscoe was built to solve

Early spreadsheet-era legal history meets an eight-million-case digitized corpus — motivating Roscoe and a dissertation on nineteenth-century courts mediating ideals of liberty against laws of slavery, suffrage, and Native policy at scale.

James Kaplan: I asked you to join because I’m fascinated by digital humanities and the extent to which we can use AI and other digital techniques to enhance historical understanding. Could you tell us a little about the historical research you’re pursuing now and what got you interested in that topic?

Bob Pasker: When I was an undergraduate and in my first attempt as a graduate student, I became interested in legal history. For me it’s a unique field. At the time there were many untapped sources of legal documents that historians had never or rarely used. Personally, law had been a longstanding interest of mine, and I like to do things that are a little off the beaten path — that’s what I wound up doing.

I wrote papers on legal history: kinship and the way people left money to their children in the eighteenth century, based on published wills; and a paper on sex crimes in Providence, Rhode Island, in the late eighteenth and early nineteenth centuries. This was all done with word processors and spreadsheets — so I was doing digital history even then, taking the documents I was reviewing, putting them into spreadsheets, tabulating, and so on.

Using Roscoe I found ninety-three cases out of 226,000 — about four in ten thousand — in the appellate court records that would otherwise have been impossible to find … There was no keyword search in the world that was going to find those ninety-three.

Fast-forward to 2023: Harvard Law Library has transcribed the entire corpus of American appellate case law — about eight million cases — and digitized them, so you can download text files of all of those cases. I decided to combine my two fields, history and computers, build a conceptual search engine for that case law, and use that search engine for my dissertation research.

The system is called Roscoe. It’s named after the first legal historian, Roscoe Pound, who lived from 1870 to 1964 — he really started the field. My dissertation is a study of how, in the nineteenth century, American courts became the venue for working out conflicts between our constitutional ideals of freedom and liberty and the actual law that permitted slavery, denied women’s suffrage, and affected Native Americans.

A lot of historians have studied these topics; they’ve only used case law as evidence. Nobody has used case law to study the court system as an institution itself — on par with the other branches of government, religious institutions, and industries. The reason nobody could do that is scale: there are 226,000 cases up through 1860, and that’s the basis of my dissertation research.

4. How Roscoe works — embeddings, collections, ingestion — and how historians react

Semantic search over multiple public-domain collections via embeddings, vector indexes, relational metadata, and re-ranking — plus the human story of academic uptake, the ninety-three-case find, and “silences” created by bad retrieval, not missing archives.

James Kaplan: Tell us a bit about Roscoe. How does it work? Take us under the hood a little.

Bob Pasker: Roscoe is a semantic search engine. The idea is to replace arcane keyword and Boolean searches, which is how most archives still work. If you want to find case law on a particular topic, you have to know the exact words they used back in the nineteenth century — and the words they used in Georgia versus New Hampshire.

If you’re interested in a concept like canal building, you’d have to look up locks, canals, waterways, and so on to surface all the relevant documents. With Roscoe, you type something like “disputes over canals,” and it surfaces documents related to that concept.

The fundamental technology is embeddings and a vector database. An embedding takes a piece of text and turns it into a high-dimensional vector. That vector can be stored in a vector database; you embed the query, look it up in the database, find the k nearest neighbors, use those neighbors to look up the specific cases in Roscoe, and hopefully those cases are conceptually similar to your query.

I’ve organized Roscoe by collections — each collection has its own vector database. The first collection was those 226,000 cases. I’ve extended it to another collection called Chronicling America, which is millions of nineteenth-century newspaper articles. I have another collection with the papers of the founders, and also the Congressional Record. These are all public domain, and each collection is available inside Roscoe.

What makes Roscoe different is that you’re not searching one database at a time with arcane keywords — you’re searching across all of them at the same time. The key is the ingestion process: an ETL layer — extract, transform, load — that takes data as it comes from the archive, tests different chunking algorithms and embedding strategies, creates an index in a vector database, and cross-references that with a relational database that holds document metadata — names, dates, location — used for filtering and re-ranking. That’s basically how it works underneath.

Version one had a very simple user interface that produced a result table with metadata. Version two has multiple collections, searches across collections, and does unified re-ranking: it takes results from the different collections and re-ranks them against each other so the most relevant results rise to the top, regardless of which collection they came from. That’s basically how version two of Roscoe works right now.

James Kaplan: What’s been the reaction from people you interact with in academic history? I ask because some academics I know are incredibly excited about what AI can do for research, and others push back — anything involving quantification, or “that’s a science way of thinking, not a humanities way.” What’s the balance of enthusiasm versus skepticism?

Bob Pasker: It’s similar to the experience I had trying to get people to use WebLogic. There’s a whole lot of people who couldn’t care less, and a very few who are really interested and see the value. So there’s a huge evangelization process — different from a startup, but still a big thing.

I’ve had professors who, when I’m writing a paper using Roscoe, say: I don’t want anything in the paper about technology — I just want a history paper. I’ve had others who are extremely helpful and excited — but to be honest they don’t really understand it. They can conceptualize the benefit, but until it becomes a public utility they can try out, with enough collections for their own work, it’s mostly curiosity rather than adoption.

Right now I’m trying to write some papers using Roscoe. I’m working on a paper about how to explain Roscoe to the community of historians, which turns out to be fairly difficult — but I’m making progress, and I hope to publish it as an independent research paper. It’s not meant to be pure evangelism; it’s meant to ground Roscoe in historiography, the process of doing history, and archival science — what it means for both disciplines.

James Kaplan: It strikes me as historiographically important. A professor described to us how certain historians were paging through records to find birth and death dates to understand lifespans in early nineteenth-century England and how the industrial revolution changed mortality — whether it increased or decreased mortality in different places. Your approach is a way to vastly increase the datasets available to historians without sending grad students to page through bound volumes by hand.

Bob Pasker: Yes — and in a sense that’s a slightly different kind of digital history: it’s tabular. There’s been a lot of work since the late fifties on tabular analysis of data, the way an economist might do. I’ve been interested in that too; I did it in those earlier papers.

Roscoe is very different. It’s for finding documents that already exist in archives but are impossible to find. My paper last year was on whether Black people could testify in courts before the Civil War in the nineteenth century. The laws were basically against it, and we don’t have much conception that it was still a possibility. Using Roscoe I found ninety-three cases out of 226,000 — about four in ten thousand — in the appellate court records that would otherwise have been impossible to find. They span from the 1790s to the 1860s across eleven different territories and states. There was no keyword search in the world that was going to find those ninety-three.

Archivists have this idea of archival silences: what archivists admit into their archives. For the most part archives contain documents they consider important and leave out what they thought marginal or uninteresting — they have to curate; we can’t save everything.

I have a different kind of silence in mind: documents that are in existing archives, useful to historians’ research, but that they can’t find because they can’t come up with the right keyword search in the user interface. My paper argues that Roscoe makes it possible to find those — that there are interesting documents that have been, in a sense, silenced by arcane interfaces. That’s what I’m trying to create: a system that surfaces many more interesting documents than a historian would otherwise find.

5. What’s next: precision and recall, multimodal search, partnerships — then evidence, interpretation

Roadmap: recall versus precision, deeper per-hit explanation, map-level multimodal search — then partnerships and “index not copy” for archives, historiography of evidence, reading the ninety-three cases, and why he prefers “machine learning” to “AI.”

James Kaplan: To push one level further — and this is a little about where Roscoe might go — ninety-three cases you could read yourself, but you can imagine a search that surfaces a thousand or fifteen hundred cases. To what extent do you think the state of the art will advance so you can use analysis to identify trends in legal thinking? Could some of these documents go into a graph so you can see how legal thinking in one set of cases influenced another? What comes next after archival search? Does that make sense?

The hardest part isn’t really the technology. It’s twofold: one, making it useful to historians in a way that comports with historiography and archival science; two, building relationships and partnerships with libraries and archives so they’re interested in doing this without feeling they’re giving up their walled gardens around these materials.

Bob Pasker: Yeah, it does.

Version three of Roscoe, which I’m already working on, will address some of this. First, on returning too many results — that’s well known in information science: recall versus precision. You want enough cases in your result set that you see everything useful, but you don’t want false positives — things returned that aren’t useful. Search engines have dealt with that for a long time. You also want precision: the cases most relevant to you should rise to the top, and the useless ones should drop out. You don’t want to leave useful cases outside the result set, and you don’t want useless cases inside it. I work on that constantly: refining the system for better precision and recall.

Second, I want deeper analysis of how each case relates to the query. In version one, as results returned, the system re-ranked them and analyzed cases more deeply to identify exactly how each case related to the query. Once you have a large result set, you can go through it more deeply with machine learning to pull out the cases specific to what you’re looking for and leave out the rest.

Another direction I’ve experimented with is visual search — my experiments have been with maps. Old maps are crude line drawings with handwritten type. In the archive you get: “Here’s the Smith map of New York City from 1823” — and that’s all; it doesn’t tell you what’s on the map until you open it. I’ve used machine learning to read the maps, identify places written on them, get latitude and longitude, overlay them on modern mapping systems, and identify features — waterways, canals, mountains, farms. That information goes into a vector database so it can be searched semantically.

So when someone searches “disputes over canals,” you get not only case law, debates in the Congressional Record, and newspaper articles, but maps where those disputes actually took place — spatial context as well as temporal context from the dates. I think you can do that for other artifacts too: paintings, sculpture, textiles — so people doing research on material culture could search catalogs, say at the Museum of Natural History or the Museum of Modern Art, and find artifacts related to their topics.

James Kaplan: What’s the toughest thing technologically — where is the technology there, and where is it harder?

Bob Pasker: I guess I’m an optimist: I think I can build something really fantastic here. The hardest part isn’t really the technology. It’s twofold: one, making it useful to historians in a way that comports with historiography and archival science; two, building relationships and partnerships with libraries and archives so they’re interested in doing this without feeling they’re giving up their walled gardens around these materials.

The good thing about how Roscoe works is it doesn’t duplicate the archive — it creates an index, the way a card catalog is an index, not the contents of everything. Those are really human difficulties more than technological ones. We’ll keep wrestling with precision and recall and the right way to visualize and display what’s useful. At this point I don’t see anything I can’t get out of the technology.

James Kaplan: It’s potentially disruptive within the history profession in the sense that, over time, techniques like this could make history even more of an empirical than a theoretical discipline — ground it more tightly in the historical record by accessing a broader set of documents easily.

Bob Pasker: How historians use evidence is itself a historiographic topic — it goes back to ancient Greece and the way Thucydides used evidence to describe what happened. That had a transformation in the nineteenth century as people became more interested in an evidentiary basis for history rather than only the stories they had told. The rational, evidence-based side of history has been developing for a couple of hundred years.

I think this extends the same trajectory as building archives, electronic card catalogs, transcriptions, photocopies, seeing old documents as images on the web. I’m a novice historian; others could expound on this much more. But I think Roscoe is doing what needs to happen given the scale of digitization and transcription at the archive level — making huge corpuses visible. I don’t think it’s disruptive; I think it’s enabling.

James Kaplan: Hearing what you just said, it’s a continuation — the next evolution in a long series of transitions over the past couple of hundred years, increasing the dataset available to historians as they do history.

Bob Pasker: Yes — and that’s what my paper this semester is about: what this deluge of information means for historians and how Roscoe will help. I call it the silence of abundance: what’s hidden in this great abundance of historical records.

James Kaplan: As you looked at those ninety-three antebellum cases, was there anything especially insightful that wouldn’t have been available if you hadn’t found them?

Bob Pasker: The fact that those cases exist at all. For the most part we “know” that Black people were not allowed to testify in court — not as witnesses, they couldn’t give evidence. But now we see: wait, that’s not completely true, even given what the laws say.

I had hoped I’d find justices who really wanted to give people an opportunity to testify on their own behalf or on behalf of something they had seen — in a positive, rights-expanding sense. That’s not really why they were allowed to testify. They were allowed to testify because nineteenth-century justices had a very specific concept of justice. It wasn’t liberty and freedom in the abstract; justice was the process of adjudicating cases.

So you had very specific situations in these ninety-three cases: someone was injured, the only witness was a Black man, everyone knew the person was injured and the defendant was guilty — but there was no witness except this one Black man. The only way for the justice system to maintain its reputation as an institution that could adjudicate cases was to let that witness testify. Otherwise it would be as if nobody had seen it, the defendant would go free, and that would violate their notion of procedural justice. It was more about maintaining institutional coherence than about a grander sense of justice. That was my conclusion.

James Kaplan: Very helpful. Anything I neglected to ask — anything else you’d like to cover?

Bob Pasker: As I’ve talked to historians — classmates, people in my department — and looked at what historians’ associations have said about artificial intelligence: by the way, I don’t use the term “artificial intelligence” because I find it unhelpful. I use “machine learning,” the technology I use, without the generative piece.

The resistance to something like Roscoe — what’s often lumped as “AI” — comes down to three concerns. One is hallucinations, which we’ve discussed. Two is teaching — how this affects pedagogy. Three is the human aspect of writing history: history is a process conducted by humans, not machines, because history gives us a sense of who we are, where we came from, the story of our path — and humans should own that, not computers.

I’m hoping something like Roscoe, which uses the same underlying technology in a different way, will have a positive impact — people will understand it and find it useful in their research. It may take ten or twenty years, another generation of scholars, before that really bears fruit. I’m enjoying being at the forefront and I’m proud of what I’ve done so far.

James Kaplan: Congratulations. Thank you so much.

Bob Pasker: Thank you. I really appreciate it, James. This is something I’ve wanted to talk about for a long time.

Thanks for reading Prosaic Times — subscribe to get every issue!

Footnotes

[1] For our younger readers: Before containers, app servers provided transaction management, connection pooling and a runtime environment for J2EE applications.

This was great in that it demonstrated value, created demand and accelerated organizational learning. This was less great in that created risk and technical debt. Sometimes it foreclosed future options and turned out to be like trying to get to the moon by climbing the tallest tree -- the first fifty feet feel like great progress.

So too with agents. Enterprise technology functions must go quickly to meet business expectations, but also build the platforms that support agentic workloads with resiliency, security and efficiency.

Kong is a provider of API and AI gateways. I spoke with their CTO Marco Palladino to talk about the evolution from API to AI gateways and how AI gateways fit into a broader agentic strategy.

Thanks for reading Prosaic Times — share it with a friend!

Some of the most interesting comments from our discussion:

* “The dream has always been the same: we can build an assembly line of software so we can create new products faster. We can innovate faster by taking existing APIs and assembling them together — like the Ford assembly line, but for software.”

* “AI is useless if agents cannot use APIs. Agents consume two things: they consume LLMs for intelligence, and they consume APIs to do something with that intelligence.”

* “Not appreciating non-determinism in AI is like organizations in the early days adopting containers but deploying them one container per virtual machine. You’re using containers, but not really — you’re losing the whole point.”

* “We’re going to have agents telling agents to go build agents. It’s going to be agents all the way down.”

* “But when the agent becomes the buyer, all of a sudden the agent doesn’t care about the billboard. It doesn’t care about the YouTube video. How does an agent decide what product to use? It looks at documentation. It looks at examples, getting-started guides, what other agents have been doing.”

James Kaplan: This is James Kaplan with another ProsaicTimes podcast. I’m here with Marco Palladino from Kong to talk about AI gateways, API gateways, and security in the AI era. Marco, welcome.

Marco Palladino: Thanks for having me.

James Kaplan: Great pleasure. Give us a little bit of your background. Tell us about your journey to Kong.

Marco Palladino: I’m the CTO and co-founder of Kong, and I’ve been doing this API thing for fifteen years now — since the days when people were asking us what an API was. Now, obviously, everybody knows what an API is, and APIs are the backbone of pretty much the digital world, including AI.

So I started Mashape, which was an API marketplace that didn’t go anywhere — it was too early. We then open-sourced our core technology, and that became Kong. Kong started as an API management product, but over time it expanded to encompass all of connectivity: API connectivity, AI connectivity, microservices connectivity, mesh connectivity.

Today we work with approximately 1,000-plus enterprise customers around the world, and we’re very excited about what’s next.

James Kaplan: Some of us are old enough to remember when an API was something exposed from a Windows Dynamic Link Library, and you had to make sure you had the right version of the DLL in the right directory so your application could consume the right version of the API.

Marco Palladino: Everybody remembers DLL Hell.

James Kaplan: Tell us about how central API gateways were in the evolution of a modern artificial intelligence stack. What makes for a good API gateway? What makes for a good user of an API gateway?

Marco Palladino: The concept of an API has been there since forever. Even in the SOA world of the early 2000s, there was always this dream of web services we could use, consume, and integrate into new applications — but it didn’t go anywhere. It was too complex. Nobody wants to write SOAP or consume SOAP. And then new programming languages emerged — JavaScript, Ruby on Rails — where SOAP was very hard to consume. So it really didn’t get any traction whatsoever.

Then RESTful APIs, the modern ones as we know them, really became popular when Google created their own public APIs and Facebook had APIs that let people access the social graph. All of these apps were talking to monolithic backends through an API.

The iPhone really created a whole new need for APIs — not only to create ecosystems of integrations, but also to handle internal communication. We needed an API to connect our app to the backends, which were monolithic back then.

Then something happened in 2013: Docker was invented and released. In 2014, Kubernetes was created and released, and all of a sudden APIs — which used to be an afterthought bolted on top of monolithic applications to connect with a mobile app — were there since day one. We were building API-first because we were building microservices running in containerized Kubernetes environments across one or multiple clouds, and APIs as a means of internal communication became quite essential.

The dream has always been the same: we can build an assembly line of software so we can create new products faster. We can innovate faster by taking existing APIs and assembling them together — like the Ford assembly line, but for software. And I think what we’re seeing now is the latest iteration of APIs as the backbone of the digital world when it comes to AI.

There is no AI without APIs. We need an API to consume the models. We need an API to consume MCP tools, the data, and the systems and services that we want our agents to use. APIs are everywhere. If anything, APIs are going to keep increasing in numbers.

James Kaplan: I like to say every problem in computer science gets solved by abstraction, and APIs are a mechanism for abstraction. Let me ask you about adoption. We all know there’s pervasive API usage in the consumer internet and enterprise SaaS space, especially externally facing. I was wondering if you could comment on the traditional enterprise — banks, pharmaceutical companies. How far did they get in terms of creating APIs that allowed different applications, or different components of applications, to talk to each other? I think that has historically been a bigger lift.

Marco Palladino: You really only have two options: either you have APIs, or you have silos. APIs help us break silos. Many companies have not invested in APIs, or they’ve done it in a non-coordinated way — maybe they have APIs within one team or one product, but they don’t have a repository of APIs that the rest of the organization can look at and use when building new agents, new experiences, new products. Only a few organizations have made that kind of investment, and those are the ones that are going to move and innovate a lot faster.

Look at the best example — and I know it’s a bit of an old example, but it’s still one of the best. Look at Amazon. At one point, Jeff Bezos wrote a memo in the mid-2000s basically telling every team: you have to build APIs for every new product from day one, and if you’re not doing that, you’re fired. APIs were mandated. And because of that top-down push from the CEO and founder himself, Amazon was able to create AWS. Amazon was not the only e-commerce company in the world, and yet it was the only one that was able to generate a multi-hundred-billion-dollar business with AWS — thanks to their API culture and engineering methodology. When everything ships with an API, you can not only integrate it, you can productionize it and start selling it and creating new revenue streams. That’s how AWS was born.

So what’s the status of APIs in the enterprise? You see a few organizations making the investment because they understand APIs as a path to innovate and move faster. And then you see lots of laggards — organizations that are just now waking up to the fact that they cannot capture AI if they don’t invest in APIs. Better late than never, but this is an investment that should have been made ten years ago.

James Kaplan: When you said the alternative to APIs is silos — at many places, the method of integration is still moving flat files around. What drives the laggards? For the folks who have not moved to a more API-forward architecture, is it lack of skills? Lack of investment? A belief that an API-forward architecture is harder to manage? What have been the barriers, in your thinking?

Marco Palladino: Nobody makes the call. I’ve been working with hundreds of top Fortune 500 and Global 2000 companies around the world — we’re a global company across North America and Europe — and I see patterns everywhere. There is a belief that if you build an internal platform that supports APIs, the teams will eventually adopt it. I think that’s only partially true.

At some point, it can’t just be the carrot. You show the teams what needs to be done, you show them that adopting APIs will benefit them. But at some point, you also have to use the stick. Many organizations and their leadership think APIs are someone else’s job, so they never make that top-down call. And eventually you have to call it for what it is: teams, we have to use APIs, because if you don’t use APIs, we cannot reuse anything you’re building in any other product, any other market, or with any other partner.

There must be a top-down leadership call — but it can’t be mandate alone. You can’t tell teams to adopt APIs and then not give them a platform to use. So yes, there needs to be technology, but at some point there needs to be a top-down leadership call as well. Otherwise, you’re not part of this engineering culture. Many organizations fail because they don’t want to make that call — and that can be deadly.

James Kaplan: One of the most interesting examples I’ve seen of a company adopting APIs — they took a very developer-services point of view. They said: we’re going to do all the things we would do if we were a third-party software company selling a developer platform. We’re going to invest heavily in documentation. We’re going to invest in developer support. We’re going to hold conferences with developers. I thought that worked nicely — it was an interesting change management mechanism. They really treated the developers as customers rather than as people who could simply be mandated to use a platform.

Marco Palladino: I fully agree, and that’s the right way to do it. Developers are internal customers — and APIs are products. They have a lifecycle. Part of the problem is that many organizations don’t see APIs as products, but they are. Just as we version websites and mobile apps, adding features and removing them, APIs have a product lifecycle too: we version them, create new features, decommission them. That’s work that needs to be done.

Developers who are busy building features after features need to slow down at some point, look at their API portfolio, clean it up, and treat it as a product. An organization can’t have a great API ecosystem without allocating the right amount of time for developers and teams to curate those APIs. But it pays dividends — once those APIs are consumable, you can reuse them in countless places.

And especially for AI — I can’t stress this enough — AI is useless if agents cannot use APIs. Agents consume two things: they consume LLMs for intelligence, and they consume APIs to do something with that intelligence. Without an ecosystem of APIs to hook into, those agents may be smart but useless, because they can’t connect to anything meaningful for the business. APIs unlock AI.

James Kaplan: Which creates a lot of issues around control and security. So let’s talk about the transition to an AI gateway. Where does an AI gateway fit into the system? How do we think about how an AI gateway relates to things like MCP and A2A?

Marco Palladino: When we think about agents, we’re thinking about smart applications. An agent is coded, runs somewhere. What does it do? It talks to an LLM to determine what the next operation should be, and then it talks to data and services to get the right inputs to do the job it’s supposed to do. For example, if we’re building an agent that does loan origination for a bank, we need access to APIs about our customers — their social security numbers, their KYC information, and so on — and we can’t do that without investment in APIs.

Now, agents using AI will also have to deal with the non-deterministic nature of AI. When we build a traditional application, we know what’s going to happen. But when we use AI, we’re giving the LLM some degree of freedom to determine what needs to happen with an input and what output to generate. That output is non-deterministic compared to how we used to build applications, and in a non-deterministic world, we need guardrails and capabilities to control what AI does. This is the whole area of AI governance.

As more teams build agents, we don’t want them to reinvent the wheel — building their own guardrails for each specific use case. The platform team can’t monitor what all these calls are about, and the organization doesn’t have confidence that agents are doing the right things at the right time. What we need is a platform that manages all these AI interactions: AI governance, AI security, AI optimizations — including the ability to compress prompts to get more out of the tokens we’re spending. All of that can be abstracted away from individual teams by the platform team, which offers it as a service to internal developer customers.

That’s effectively what an AI gateway does: it centralizes those cross-cutting requirements, just as API management did for APIs — where we took authentication, security, rate limiting, traffic control, and encryption and pushed them into a centralized place.

James Kaplan: It’s at least evocative of what cloud security posture management meant as we all started building out cloud architectures — an important control point that governed how you configured things in the cloud.

Marco Palladino: Absolutely. I’m a big believer that whatever technology we put in place for developers — whether to build APIs, AI agents, or microservices — needs to fit very naturally into their workflow and best practices. The platform is there to help, not to get in the way of getting to an outcome.

James Kaplan: So the AI gateway sits between the agent and the large language model externally, and between the agent and various tools — controlling or limiting what goes in and out of the institution, what goes to the large language model and what doesn’t, and which tools can be called under which circumstances. Is that a fair description?

Marco Palladino: That’s exactly right. It sits between the agents and the models and the tools. What is an MCP tool? Think of it as a new API protocol. APIs don’t have to be REST — they can be SOAP, REST, GraphQL, gRPC. MCP is a new protocol that makes it easier for agents not only to consume data or services, but also to discover what data and services are available. MCP bundles key requirements into the protocol itself: bidirectional real-time communication, tool discovery. It packages all of that in a very consumable way for agents.

With that said, it doesn’t have to be MCP. Many agents still consume APIs with traditional function calling and use any protocol they want. MCP has emerged as almost a standard stack for agent development — if you want to build something without overthinking it, MCP gives you an ecosystem that supports you. But it can be MCP or not. And the AI gateway sits in between not only all LLM transactions, but also all MCP tool requests.

Now, how large that MCP ecosystem or API ecosystem is will determine how much agents can do. If an organization hasn’t invested in creating an ecosystem of MCP tools or APIs, developers building agents will be somewhat handicapped — there’s not much for their agent to hook into. MCP is very exciting, and there’s more than just MCP — there’s also A2A, a different protocol that governs agent-to-agent interactions.

James Kaplan: Those of us who struggled with the syntax of Windows API calls in the 1990s appreciate that MCP allows for less finicky syntax in calling APIs.

Marco Palladino: At least we don’t have agents consuming CORBA APIs. There’s that.

James Kaplan: Some things are best left in the past.

Marco Palladino: Yeah.

James Kaplan: One of the things I find especially interesting is the application of non-deterministic controls. Preventing a social security number or credit card number from being sent somewhere is easy — it’s a pattern. Figuring out whether sensitive pricing data should be allowed to go somewhere is more context-dependent. Deciding how many tokens you want an agent to consume can also be very context-dependent — you may not want a hard cap; you may want certain agents to consume more tokens in certain circumstances and fewer in others. Could we talk a bit about non-deterministic controls and how you think about them in the context of an AI gateway?

Marco Palladino: Non-determinism in agents and AI is both a curse and a blessing.

James Kaplan: Of course.

Marco Palladino: Organizations that are embracing AI and doing it right are also embracing the benefits of non-determinism. Think about it: if we’re using LLMs and AI but not embracing non-determinism, then what we have is just a workflow. Why are we using AI at all? We can build workflows the old-fashioned way. Not appreciating non-determinism in AI is like organizations in the early days adopting containers but deploying them one container per virtual machine. You’re using containers, but not really — you’re losing the whole point.

Using AI requires an appreciation that there is going to be non-determinism in the outcomes it produces, and that is the power of AI. But we also need to make sure the outcomes can’t just be all over the place. While outcomes may not be perfectly deterministic, we need a range of acceptable options.

The best way to think about non-deterministic AI is in terms of risk management. When generating a loan for a customer, there’s a certain degree of risk the organization is willing to tolerate — a range within which we’ll generate loans and beyond which we won’t. Thinking about AI-generated outcomes has to work the same way: determine what risk you as an organization are willing to tolerate for specific outcomes, deny outcomes that fall outside that range, and fully appreciate the non-determinism within the range you’ve defined.

Many organizations are struggling with this, and because of it they’re struggling to generate outcomes with AI. The biggest outcomes any organization will generate are the ones tied to the core business — for a bank, that’s customer financial data, loans, money; for a healthcare organization, it’s claims processing. Because the organization doesn’t feel comfortable putting AI in the core business, it will never generate outcomes that are truly impactful. It’s a chicken-and-egg problem. We need to appreciate AI and invest in the right platforms for managing it — so we feel comfortable enough to put AI in the core business processes that will generate outsized outcomes. Without that investment, the organization will never trust AI in the inner workings of the business, and the CFO will eventually ask: we spent $100 million on AI — did we generate $100 million in outcomes? The answer is always going to be no if AI never found its place in the core business. Only by being in the core business can AI generate that return on investment.

James Kaplan: Imagine you have a CIO or CTO who says: I’m convinced we need to bring AI to the core of the business. I’m convinced we need a platform. I’m convinced we need an AI gateway. What are the major design decisions? What are the major architectural choices he or she might face, and what are the reasons that might push you in one direction versus another?

Marco Palladino: They have to think about centralizing governance. We want to decentralize the execution of AI, but governance is very hard to run well in a decentralized way.

James Kaplan: Explain what you mean by centralized governance.

Marco Palladino: With decentralized governance, everyone is on their own — making their own judgment calls about what’s good and what’s not. Centralized governance reduces the risk of adopting AI across the organization, but at the same time you want to give teams a degree of freedom — a bounded degree — to experiment within the governance you’ve established. Within these guardrails, you can move, you can experiment. On one hand, we don’t want to slow down innovation, so we want teams to be able to try new models, new tools, build new agents. At the same time, we want that experimentation to be bounded by centralized governance in such a way that nobody can ever put the organization or customer data at risk.

James Kaplan: What are my red lines? What are the things I can’t compromise on?

Marco Palladino: Customer data is number one. Whatever we do with agents, we cannot put customer data at risk. Organizations will need a strategy to anonymize data going through a model — and to dynamically reinsert that data on the way back, so the model never sees it but end-user experience is unaffected. PII encryption. The organization may also want to determine which models can be consumed and which cannot. We may want developers to use models from trusted vendors, and not allow them to use an untrusted vendor that might learn from all the data and interactions — effectively copying IP and creating organizational risk.

What models are being used? What data is flowing to those models? What MCP tools can agents use? What identity are we giving agents, so we can identify them and determine what they can and can’t do with models, APIs, and MCP tools? There’s a whole agent identity problem: agents are using MCP tools, some of which use third parties — how do we identify the agent, and how do we identify the end user consuming the agent to act on their data? None of this can be reinvented every time a team wants to build an agent. That would be madness — a colossal risk for any organization.

As the industry matures from early experimentation to actually running agents in production, those agents need all of this underlying infrastructure. An AI gateway is a core part of that.

There was a lot of early experimentation in the last two or three years, and now organizations have identified hotspots where agents can help with specific business processes — moving faster, innovating faster. The question now is: how do we enable every team in the organization to become an agentic developer?

James Kaplan: Any other design decisions? We talked about centralized versus decentralized governance. What else is especially important when implementing an AI gateway?

Marco Palladino: There’s a whole area of making agents effective, which can also be centralized. For example, reducing token consumption — optimizing how we leverage AI, especially for organizations that have already found their use case. Their next problem is: how do we make this cheaper? Things like prompt compression, or semantic caching — the ability to understand the semantic meaning of prompts so you can build what is essentially a semantic CDN that doesn’t require hitting an LLM every time. If the meaning of a question has already been captured and cached by another agent, think of it like a CDN, but semantic.

For example, if I ask an LLM “What is the population of New York?” and then separately ask “How many people live in New York?” — I’m using different words but asking the same thing. That could be a cached response. It helps on two dimensions: cost control, which is increasingly sensitive and will become more so as LLMs stop subsidizing every token and start pricing to reflect actual costs — it’s like Uber in the early days, when you paid $3 for a ride that really cost much more. When real costs emerge, CFOs will ask whether we’re generating the right outcomes for the spend. Prompt compression can reduce token consumption while retaining the same semantic meaning: “Please tell me how many people live in New York” compressed to “What’s the population of New York?” — much smaller token count, same meaning.

And then there’s observability: not only measuring what models and MCP tools we’re using and what agents are consuming the most, but also what outcomes we’re generating. Our customers tell us their biggest challenge is understanding outcomes. They can build agents, consume LLMs and MCP tools — but are they actually generating any outcome? How do you quantify the economic impact an agent has generated if you’re not tracking those outcomes?

Even outcome tracking is something that can be centralized — so that when teams build agents, the entire observability stack, from low-level connections all the way up to business intelligence and outcomes, is captured centrally. Teams don’t have to rebuild it every time.

James Kaplan: I can see how an AI gateway might help you track usage. How does it help you track outcomes?

Marco Palladino: The AI gateway sits in between the agents and the models and the MCP tools and APIs they’re consuming. So everything the agent does, the AI gateway is aware of — because it is on the execution path of all of it. By doing so, the organization has a centralized control plane. Think of it as a control tower for AI, where you set up all the governance rules, security rules, data governance rules, optimizations, and observability rules you want. Then teams go build agents. Whenever an agent makes a request, it has to go through the AI gateway infrastructure, and all of that governance and observability gets captured centrally.

We could avoid doing this — but then we have a much bigger problem. Whether governance, security, and observability are established through an AI gateway or not, we still need all of it. We have to enable teams to succeed by removing those cross-cutting requirements.

James Kaplan: You can start to interrogate the traffic that goes to the models, and there’s a lot of insight about business value there. And at the same time, you have a lot of insight about cost — which is interesting, because for the first time in a long time, the marginal cost of compute may be relevant relative to business value, rather than being a small fraction of it. CPU has been cheap; GPU is expensive.

Marco Palladino: I would argue CPU will be cheaper only for now. I’m also a big believer in an autonomous agentic world where agents are going to be not only intermediaries but the actual buyers of software. In an agentic economy where agents are automating more and more business transactions, we’re going to hit a bottleneck on CPUs — because we’re effectively replacing humans with CPUs. We just haven’t seen that yet because we’re still building out the use cases. But at scale, when every organization runs with an army of agents handling its core business operations, there will absolutely be a CPU shortage. If agents truly become what I think they will, that day is coming.

James Kaplan: We may see a flipping of technology economics. For the past twenty years or so, infrastructure has been cheap and application development has been expensive. AI makes software engineering cheaper but creates massive compute requirements. Whether it’s more GPU than CPU, one way or another it makes infrastructure costs more relevant — and so the marginal cost of compute will matter in a way it hasn’t in several decades, relative to business value.

Marco Palladino: One use case that illustrates this: today, organizations are experimenting with agentic IDEs — Claude Code, Codex, Cursor. Can they help developers build faster, or build more, by leveraging AI? Today, a human developer is asking prompts in these tools to go build software. How far are we really from having another agent asking the prompts to go build software? Now you’ve removed the entire human component. And that agent will know what to build, or not build, thanks to inputs arriving via APIs or MCP tools. The agent will make the judgment calls a human developer used to make, but autonomously. We’re going to have agents telling agents to go build agents. It’s going to be agents all the way down.

James Kaplan: Anything I neglected to ask about? Anything else we should cover?

Marco Palladino: I think there are two things worth mentioning: a change in distribution, and a change in customer behavior. The distribution one is especially interesting.

Today, businesses invest enormous amounts of money and effort to reach human customers — a billboard on the highway, a TV commercial, a YouTube ad. The internet runs on those commercials. But when the agent becomes the buyer, all of a sudden the agent doesn’t care about the billboard. It doesn’t care about the YouTube video. How does an agent decide what product to use? It looks at documentation. It looks at examples, getting-started guides, what other agents have been doing.

There’s going to be a whole new distribution channel — potentially larger than today’s human distribution channel — where to attract customers, you’re not attracting humans anymore, you’re attracting agents. Everything is going to change when that happens. Every business that relies on digital advertising will find that channel works differently than it does today.

That is going to change the internet as we know it. I think it’s extremely exciting. What’s even more exciting is that those of us in this conversation are not only witnessing it — we have the opportunity to help build it. It’s a builder’s era.

James Kaplan: The distribution point is fascinating. It makes commercial markets more like equity trading, where there’s been no human in the loop for years. What you’re suggesting is that many commercial markets may become agent-based — algorithms, instantiated as agents, transacting with other algorithms instantiated as agents.

Marco Palladino: Exactly. They’re not transacting equities or securities the way a financial trading bot would, but they’re transacting outcomes. They can bid on taking an outcome and completing it. A whole new economy is going to be born from that.

And it sounds futuristic, but it isn’t. It has happened before. Thirty years ago, if you weren’t in the Yellow Pages, your business didn’t exist. Then the internet was born — all of a sudden you needed a .com website, or your business didn’t exist. Then the iPhone: customers moved from websites to mobile apps, and if you didn’t have a mobile app, your business didn’t exist. Well, the customer is moving again.

James Kaplan: What you’re saying applies to tokens as well. You may see secondary markets for tokens, with agents bidding against each other for them.

Marco Palladino: Anything that uses a limited resource will eventually find a way to create a secondary market for it — whether it’s a token or a thirty-year-old Mercedes-Benz, there’s always going to be a market.

James Kaplan: I’m still waiting for someone to set up the first trading desk for tokens.

Marco Palladino: It’s going to happen. And it’s quite exciting. Some people look at this and say AI is going to damage society. I think society will have to evolve, and there will be a transition period. I’m just making an observation — I don’t know exactly what’s going to happen. But with the last industrial revolution, things are much better now than they were then. There was a century of societal upheaval — capitalism versus communism, all kinds of political and economic realignment — because of that revolution. This is a new industrial revolution. Are we going to be better off a hundred years from now, when the hard work is done for us? I believe so. Is the transition going to be easy? I don’t know. But it’s progress, and progress encompasses moments of realignment in how we look at technology and how we adapt to it.

James Kaplan: Terrific. Thank you so much. This was great.

Marco Palladino: Thanks for the opportunity. I had a blast.

Thanks for reading Prosaic Times — subscribe to get every issue!

Thanks for reading Prosaic Times — share it with a friend!

How times have changed! Companies (mostly hyperscalers) spent USD 260 billion on data center construction last year before they installed a single server.

Historically, data center programs have been bespoke efforts, where much of the work happens on site. Companies had to build even “modular” data center architectures largely on site. Both providers and consumers are pushing to build more of the data center in a factory and less on site.

This applies a very old economic dyanamic to a very modern domain. At the birth of the industrial revolution, British industrialists transformed textile manufacturing productivity by replacing the “putting-out” model (in which workers spu thread and wove cloth in their cottages) with the factory system that brought the workers together at one site. Yes, this allowed for greater automation -- but knowledge sharing and management control as well.

InfraPartners is in the middle of the transition of data center construction from a bespoke project to an industrial process. CTO Harqs Singh and I talked about how that works and what that means. A few things that stood out to me

* Much of the productivity advantage from building data center modules in factories rather in on site derives from co-location rather than automation

* Sometimes customers push to tweak data center designs, even when it could add months to lead times

* Skilled labor from the trades is as much (or almost as much of) as bottleneck as chip availability

* Building data center capacity is as much of a moral as an economic project in that it contributes to abundance

James Kaplan: James Kaplan here for another session of the Prosaic Times podcast. I’m recording this on the Brown University campus because the Watson School is hosting a great session on governance in China, which I’ll have the privilege of attending today. But that’s not what we’re talking about here.

We’re going to talk about data centers, data center architecture, and data center economics. So I’ve asked one of the smartest hosts I know on that topic — Harqs Singh, who I’ve known for years — to join us.

Harqs, do you want to introduce yourself and talk a little bit about your journey from being an infrastructure leader to being a data center entrepreneur?

Harqs Singh: Absolutely. Thank you, James, for having me. Hi, everybody. My name’s Harqs Singh. I’m currently CTO and co-founder of a company called InfraPartners. We build prefabricated AI factories. I’ve been in this industry over 20 years now in technology — cyber, data — started at British Telecom, then Thomson Reuters, then BlackRock.

Then I decided to jump into this largest deployment of capital in human history, as I was told, in the AI space and the excitement around what this could enable for the world — and jumped in with both feet, riding the rollercoaster of delivering AI across the world.

James Kaplan: Fantastic. So you’ve seen multiple generations of what a data center is — from the bespoke Tier 2, Tier 3 data centers we built ages ago to the more modular enterprise data centers or Tier 4 data centers that got built in the 2010-ish timeframe to now — something that feels almost entirely different.

I was wondering if you could reflect on that evolution and talk about what, if anything, has stayed the same and what has changed.

Harqs Singh: Of course. It’s changed a lot in the last 20 years.

Enterprise to colocation to cloud, and now to AI — we’re seeing some really big technical changes. So we’re seeing direct-to-liquid chip cooling, which has really been driven by the intensity of the AI loads.

We’re looking at racks that are now 140 kilowatts per cabinet, going up to 200, 320 — which is something we didn’t imagine in the past. And so this brings along a lot of engineering challenges and thinking about things differently.

NVIDIA and the next generation of equipment are looking at using DC power — 800-volt DC power — for their data centers. And so I think you’re going to have two types of data centers: you’re going to have the CPU-focused data centers, and then you’re going to have the GPU-focused data centers, essentially.

The growth has been significant. If you look in the past, people talking about having a gigawatt portfolio was quite a large data center estate — and now everybody is looking to build at least a gigawatt, and you’re looking at some of the largest hyperscalers looking at 20 gigawatts. The projections are anywhere between 100 and 200 gigawatts of additional incremental data center capacity that’s needed by roughly that 2030 period.

James Kaplan: I remember when we used to talk about square feet rather than critical power, and we got really excited when we started talking about 75 watts per square foot. That was a reasonably sophisticated data center at some point.

Harqs Singh: Yeah, absolutely. The intensity has grown significantly. So these data centers fundamentally have changed and evolved. And I think to the point you talked about earlier: we need to change; we need to industrialize; we need to be able to deploy at a scale we haven’t deployed in the past.

We did some analysis on the supply chain to support the growth by 2030. There needs to be something like a 10× increase in supply chain. So we need to think about doing things differently. In the past, we used to build every data center differently. Every data center was almost like a snowflake.

But now we need to standardize and accelerate deployment by having really standard components and units that we can deploy very quickly wherever needed. The biggest issue has been — in the past, to your point — space was the biggest thing we optimized around. Now it’s power and available power. If you think about where the data centers are now going — whether it’s West Texas or the Nordics — a lot of the data centers are going where there’s spare capacity and spare power. We’re seeing a lot of issues with the grid. So a lot of data centers at the moment are looking at on-site generation using gas engines or other forms of technology to be able to essentially bring their own power behind the meter.

James Kaplan: It sounds like what you’re describing is much more of an industrialized process. It used to be the case that if you were a big bank, you might do a major data center program maybe once a decade — maybe once every 15 years — and you were sort of relearning it with each generation. What you’re describing is we’re adding — or the world is adding — tons of capacity every year, and therefore it’s much more of a continuous industrial process rather than an episodic project.

Harqs Singh: I think this wave that’s come with this sort of growth of AI and what it’s going to enable — capital isn’t a concern at the moment, which is a good thing — but it’s really things like supply chain.

You really need five things to be able to deliver AI capacity. First is land, power, and permits — that’s where we’re going where available power is. The second is supply chain — that’s where we’re really focused, because skilled labor shortages are a real problem. Being able to find skilled labor in West Texas or the Nordics is very difficult, and that’s why we look to build most of what we do in the factory. Being able to get chips from NVIDIA or from other chip manufacturers. The third thing you need is chips. Fourth is capital, and the fifth is an end user.

Only if you have all of those ingredients are you able to actually deliver AI capacity. And so it is complex. If you think about it, James — one gigawatt of data center capacity, including the chips, is about $50 billion. Which is obviously a significant investment. So being able to pull that together — although we hear a lot about these gigawatt announcements — think about that as $50 billion that’s been deployed.

James Kaplan: What is, to your thinking, the rate-limiting factor? Especially if you drill down one level — what type of labor is the biggest constraint? Is it electricians? Is it something else? And what type of equipment is it?

Harqs Singh: The biggest skills gap is electrical, mechanical.

There are shortages, especially in those more remote locations.

If you’re in demand as part of the skilled labor force, you may not choose to live in West Texas or the outer reaches of the Nordics, because you have choices. And so that’s the biggest issue on the skill side. If you look at the other side around supply chain, the longest-lead items are essentially generators — backup generators. They can be nearly two years now. And also transformers. We’re seeing some innovations in the design around looking at designs that don’t need generators — being able to design them out or just design them for a portion of the facility — 20% of the facility related to sort of the network and management pieces. But transformers are the other big one that we look to optimize around.

Either being able to find smaller transformers or reuse them if you can, if you’re not in the queue already to be able to buy them.

James Kaplan: Are demand signals starting to work on labor supply? Obviously this all happened relatively quickly, but you have a specialized skill set around the electrical and mechanical. Where did those people come from? What’s required to expand the flow of people into those jobs?

Harqs Singh: I think it’s really being able to encourage more people to come through the vocational side — the skilled labor side, apprenticeships, those kinds of things. It’s really interesting because AI is going to create an abundance of intelligence and agents and those kinds of things — sadly, it’s not going to create an abundance of electricians or plumbers or mechanical skills. So I think that’s something we’re going to have to focus on — getting people through colleges and universities and really trying to broaden that skilled labor pool. Jensen recently mentioned that if you look at the salaries now that some of these skilled trades for data center construction are pulling, it’s becoming really attractive.

I think the future of the distribution of skills we need on AI is going to be very different than where we are today.

James Kaplan: If you’re planning to deploy $50 billion, basic economics would tell you: if someone’s planning to deploy $50 billion of capital and the constraint is an electrician, they’ll pay real money for that electrician, right?

Harqs Singh: Absolutely. And that’s what you’re seeing, James. If you look at the labor costs of data centers that are being built in some of these remote locations, you’re looking at them paying 20, 25% more on the overall cost — even double, triple the salaries you would normally expect. And that’s really where our model is that we build 80% in the factory, 20% on site.

What that means is the skilled labor gets to still live where they want to live, in the cities they want to live in, and only for a short period of time are they in a remote location. So it solves for that skilled labor issue. But you’re right — there is a premium to pay to have somebody leave their family for a year or two and move to a remote location.

So there’s an increase in the running costs, but there are also large retention bonuses that are being paid to really help push people to stay out in those remote locations to get them done. But the growth is so much that they’re struggling to find, even at the premium rates, the skilled labor to go out and do the work.

James Kaplan: Who’s going to train these master electricians, as an example?

Harqs Singh: Yeah, absolutely. And I think, James, the challenge is going to be: can we get enough of them to market in the next — the conversations I’m having a lot right now with clients, they want data centers online next year.

There’s a demand-and-supply imbalance at the start of 2027, and so it’s not going to solve for the short term — but I think this growth curve will go out to 2030, so hopefully we can solve for some of that in the longer term.

James Kaplan: I find your investment thesis around productivity in the factory as opposed to productivity on the site incredibly compelling, and we’ve seen that in many different sectors of the economy. One of the reasons housing costs are high in the United States is that, for a bunch of reasons including regulatory ones, it’s been hard for the housing sector to do more in the way of manufactured housing as opposed to housing built on site.

How does the skill mix differ when you build something in one of your factories versus when someone builds something on site? Is it the same set of skills but you just get more out of them? Or is it a different mix of skills? How do you think about that?

Harqs Singh: It is the same set of skills, but we get more out of them. That’s the way to think about it — DFM principles that shave time off the schedule. I think the other thing the factory solves for is it standardizes, it gives you a level of QC, and it also accelerates timelines. We’ve found there’s less waste. If there’s something left over, we reuse it for the next set of modules and blocks that we build. So we’ve found that from a sustainability story there are some positives there as well.

James Kaplan: What’s your labor pipeline look like? What’s the ideal person working in an Infra Partners factory? Is this someone who spent 20 years at a big bank working on a data center? What’s the profile for that type of person?

Harqs Singh: It’s mostly engineers — mechanical engineers, ideally from different backgrounds. We have quite a standardized way that we deploy. We have a standard reference design, which we start with, and that saves time as well. We have a standard reference design that aligns with the NVIDIA DGX lean framework. What that means is we need specific electrical, mechanical, construction skills. We find that in Houston and in major cities we can find those skills quite easily. We’re not asking them to move somewhere where they’re going to be away from their families.

James Kaplan: If you think of that factory as existing somewhere on a continuum — and one end of the continuum is a big room with people in it, and at the other end of the continuum is a lights-out, fully automated factory — where are you guys now, and how do you see that evolving in terms of the construction of components?

Harqs Singh: Yeah — I think right now we are more manual than we’re automated.

Physical AI and more of the robotic stuff that we’ll see — more autonomous, essentially, in terms of being able to build the building blocks and prefabricated solutions we have. But at the moment, because in prefab — to your point, James — this is something that’s happened across other industries as well as industries have industrialized to be able to deliver product at scale — we’re not [fully there yet]. To your point about the prefabricated concept in general construction, it’s already been used in a lot of countries to be able to build commercial buildings and also homes — stadiums as well.

A lot of stadiums are prefabricated in the way they’re designed. So it’s taking that concept really to the data center space. If you look at the value of prefabrication, it is a 20% to 40% improvement in timelines, as you can centralize it. So it’s taking that concept, bringing it to a data center — well, bringing it to an environment which is changing quite significantly over the next few years as we drive these transitions: going from air cooling to liquid, from AC power to DC power, from racks that had a handful of servers in them to probably having a lot more GPUs in the future that are going to be much heavier.

Once those transitions play out over the next few years, aligned to NVIDIA’s roadmap and generally the industry’s roadmap around what’s happening with chips and the extreme co-design they’re doing, that will allow us to get to a more stabilized point in terms of technical design.

I think that will then increase — to your point about the spectrum — the amount of automation and robotics versus human input as well.

James Kaplan: I think what you’re suggesting is that some of the toughest problems in AI live at the intersection of the physical and virtual world.

Harqs Singh: Correct. And I think that’s where a huge opportunity lies. You’re seeing a lot of great companies that are focused on that. It’s exciting in terms of what’s going to happen in that space.

James Kaplan: If you think about the complexity of moving from a language model to a vision model to a world model — my God, how many orders of magnitude is that in terms of conceptual complexity? Moving from a language model, which interacts with text — as fascinating as that is — versus the world model, which conceptualizes objects interacting in three dimensions in accordance with the laws of physics.

Harqs Singh: Look — we’re going to have to train a lot of these models and these humanoids and whatever technology we end up using, because I think the best outcomes of AI are when you have trained it with reinforcement learning and human input to be able to get the outputs you want.

When people think “out of the box,” it’s going to make a huge difference — but that’s not enough on its own. You actually need to invest, train, and build the capabilities and the outcomes you want, just like you would train any employee.

James Kaplan: I was wondering if you could speak a little bit about the level of abstraction versus integration between what I might archaically think of as the system level versus the facility level. As context, I spent a lot of time working with banks on modular data centers in 2010, 2011, 2012 — and you always ran up against the constraint of, well, the systems don’t quite work that way. Is there more abstraction? Is there less system diversity? To what extent is the construction of the system, for want of a better word, a constraint on the degree to which you can modularize the physical infrastructure?

Harqs Singh: I think there has been some progress in that space. There’s been some maturity, especially on the IT side. If you look at a lot of the NVIDIA DGX design, it is based on a reference architecture that has become the default way to deploy.

James Kaplan: Mm-hmm.

Harqs Singh: What we’re starting to see — we just released our partnership with Emerald AI — what they’re looking at is working to make data centers grid-interactive, making it an asset where they take grid signals in. What they’re able to do is flex either the IT side or the facility side to be able to work with the grid when it’s under stress, essentially. So you’re seeing a lot more orchestration between the whole stack. One of the things that’s been driven over the last couple of years is: look at the whole thing from land, power, permits to data center to chips to models to data — how it all comes together as a full ecosystem across all of it.

In the past, James, to your point, there was this IT–facilities divide — but I think we’re starting to see this view, and this concept of extreme co-design that NVIDIA have driven — we need to do that across everything, especially when you’re trying to optimize something that’s going to be on the order of $50 billion for a gigawatt. Everybody’s throwing gigawatt deployments out every week. When you’re talking about that level of investment, there needs to be extreme co-design and it needs to work together as one unit. The revenue that’s on offer and the investment that’s taking place means we have to do it that way. There’s no other way to do it. You’re starting to see that in the industry with partnerships that are taking place and the companies that are working together.

James Kaplan: Back in the nineties, I think we used to see buttons and pins from Sun Microsystems that said, “The network is the computer.” It sounds like what you’re saying is: the data center is the computer.

Harqs Singh: Well, the GPU — the network is also very critical. If you look at the training cycle — computation, communication, updating weights and biases — the communication phase: you could have billions of dollars worth of assets sitting idle if you don’t optimize for network.

So that’s really critical as well, especially in the inference space where response rates to prompts are going to be important — obviously some of the more real-time use cases as well. We’re seeing data center designs being optimized and GPUs being positioned and optimized for east-west traffic but also north-south traffic and response times — because if you think about the investment that’s there, the more time it’s idled, you’re losing revenue essentially.

Tokens equal revenue in the future. And so it’s an —

James Kaplan: I am still waiting.

Harqs Singh: I would —

James Kaplan: Tell me it already exists.

Harqs Singh: I would not be surprised if in the next 12 to 18 months you see some token exchanges where tokens are being sold. But I’m a big fan — we’ve been in this industry for a while — efficiencies. I was having this debate with colleagues just last week: we’ve done some great stuff where we’ve created incremental efficiencies and then Jevons Paradox comes in and everyone consumes more.

So I sort of said: we really need to flip this whole thing on its head a little bit. What we need to do is create an abundance of everything — we want an abundance of food, an abundance of energy, an abundance of compute, an abundance of AI.

If we think about it in that way, hopefully some of these young, smart people will be able to take this abundance of everything and create something magical out there.

And that’s what I like about AI: the boundaries and rules that we historically have lived by don’t exist in this world. If you look at what some of the AI natives are doing and how they’re thinking about building companies differently and solving problems differently and getting data from different sources — there’s so much opportunity, and that’s what’s so exciting: the opportunity is untapped.

James Kaplan: I don’t think anybody in this industry is brave enough to say what the future looks like, because we just don’t know. That’s why it’s so exciting, right?

Harqs Singh: That’s why it’s so exciting, right? I’m an optimist in that sense — I think if we can unlock an abundance of everything, then some really smart people — probably young people around the world — will think about problems differently and create some magic.

I look forward to seeing what that looks like. But I’m very optimistic about what this can enable.

James Kaplan: This is a little bit of a tangent, so we shouldn’t pursue it too far — but I happen to agree that you’re touching on something very important: I think we in the technology world need to talk differently. We need to talk about abundance with more conviction, because I agree.

I think you’re saying that what we’re all jointly engaged in is a moral project that will contribute to human thriving and human flourishing — and we should be. I don’t think any of us should be apologetic about that. There are risks, there are guardrails, there are things we can do better versus worse — but I don’t think any of us should apologize for the larger project.

Harqs Singh: Agreed. And I think we all need to think at scales we’ve never thought about in the past. I think that’s the lesson from history. If you look at sizes of data centers — they’ve gone from megawatts to hundreds of megawatts to gigawatts, single digits to double-digit gigawatts — we should think at scales we haven’t thought about in the past to create that abundance.

Put that abundance in some very smart, intelligent people’s hands to see what we’re able to deliver and create for humanity. I think that should be the legacy of what we do in this industry: that we enabled that abundance of everything — and we are the backbone upon which digitization and technology and AI have permeated every industry.

In the past, you remember when we were a cost center — and now we’re fundamentally a revenue generator.

James Kaplan: Some people still think of it as a cost center, sadly — but we’re working on that.

Harqs Singh: Yeah — we haven’t won; we won’t declare victory there yet, James.

If you think about businesses that don’t have a core business that has a technology strategy and the revenue and products it’s making — or a data strategy or an AI strategy — even security: these things have gone from being something that happens at the back end of an organization to being key to the product you sell. Whatever product you sell in the future — whether it’s to humans that are probably going to be AI natives, or whether it’s to machines, because that’s who your consumer is going to be in the future — you’re going to need all those things around digitization, data, security, and AI. If you’re not thinking about putting that into your core business, you’re more likely to be left behind or disrupted.

James Kaplan: It’s fascinating what you guys are doing from a historical view. As someone who is fascinated by history, you could observe that there was a fundamental change when certain parts of the world in the late 18th century went from making cloth in cottages to making it in factories.

That sounds somewhat prosaic — but all of a sudden large numbers of people could afford two sets of clothes instead of one. I would say this project of moving things from a job site to a factory is incredibly important in terms of the way it increases productivity and reduces unit costs.

Harqs Singh: That’s exactly it, James. That’s a great analogy, and exactly what we think this should help drive for the industry.

James Kaplan: Just going back to this point about standardization a little bit — I presume, given who’s building data centers or using data centers, the large majority of your customers are either hyperscalers or people who are planning to lease capacity to hyperscalers. I assume that’s a correct assumption.

Harqs Singh: The client space has grown quite significantly. To your point, there’s the hyperscalers — U.S. or Chinese hyperscalers. Then there’s the Tier 2 colos.

This neo-cloud — and that’s been created: a lot of Bitcoin mining power has been migrated to building AI. Fourth segment is enterprises. We’re starting to see enterprises — certain sectors: financial services, healthcare, insurance — look to build small AI factories for their private data. And then the last segment is the sovereign AI space — where we’re going to see governments, for national security but also for public services for their citizens.

So what we’ve seen is it’s actually expanded significantly from the hyperscaler in the past to a few new segments in the market.

James Kaplan: To what extent do different segments or different customers within a segment require a slightly different product — or to what extent can they use a standard product? How much variation, how much tailoring do you have to do by segment or by customer?

Harqs Singh: Our starting position is always — our gold standard is the NVIDIA standard, the DGX standard. We start from there, which is a great starting point. Typically what happens is the modifications that take place are either for real requirements like security in the sovereign space, or some enterprise requirements — or it’s because the technical teams on the other side want to add their value, if we could put it that way. They tend to put in a few things that they would see as imparting their value to the firm — not always necessarily needed. We see that the hyperscalers — and the people that have these technical teams — tend to be slower than the others.

So it depends on whether there are real requirements like security or enterprise — or whether it’s technical teams adding their two cents.

James Kaplan: A head of infrastructure — one of the guys used to describe this as the blue cable problem. An application development team would say: build me a server that looks like this, and the cable should be blue. The cables cannot be red — I insist on blue cables, right?

Harqs Singh: James, your analogy around using color — we’ve seen requirements around using specific colors — but I think you have to go back to: does this fundamentally change the product? A lot of the time it doesn’t. It’s preference.

Some people want greater ceiling height; others want it to be within a shell of a building — all things that are doable. I think the most important things are speed and revenue. A lot of what we’ve done in technology comes back to revenue growth, helping the business expand to where it needs to be.

That’s the bit we should never lose: we’re building infrastructure, but it’s to help drive business value. That’s where you and your teams have been focused fantastically — what’s the intersection between technology, data, AI, and revenue, and how does it create real value? That’s the part I try to focus on in my day-to-day as CTO of the organization. We always need to come back to: how do we create value?

On that basis, one of the most common questions I get asked is: what is your standard reference design optimized for? People say — oh, is it the size of the generator? Is it the size of a panel? I say: look, if you look at the TCO of a data center, most of the cost is in the chips. Seventy-five percent of the cost is in the chips. So we’re optimized for that. We’re optimized to get the most out of the chips as possible, because it’s the most expensive resource.

You want to make sure it’s idling for the least amount of time — to the point about making sure your network connectivity is right — and that we deploy SuperPOD after SuperPOD, because that’s how you get the most efficiency out of the most expensive resource and the most value. We’re laser-focused on making sure that intersection between technology, physical, IT, data, and revenue — we’re always focused on optimizing for revenue and business value.

James Kaplan: When you have discussions with customers around customization — to what extent are you involved in dialogues like: if you want the off-the-rack version you can have it in these months, but if you want the blue cable — to use a metaphor — that’s these months? To what extent are you involved in dialogue around that tension between speed and bespoke configuration?

Harqs Singh: That’s exactly why we have a standard to start with.

The standard off-the-shelf is aligned with the latest chip technology and can be deployed much more quickly. It’ll save you three months by taking off-the-shelf. If you want the yellow cable or the red cable — whichever specification you have — that will add a number of months. If you want to do that, we can do that too. But it’s the tradeoff of time.

What we’ve found is: those that don’t have a large technical team will go off-the-shelf — it makes sense. Those with the technical team will say: no, we have standards here; we need to align with them — blue cable, please. And that creates that time.

A lot of the opportunity is the gap between those organizations — those that can run really quickly, especially when you think about there is this supply-demand imbalance in early 2027. Those are the people that will take advantage of that supply-demand imbalance.

James Kaplan: Let me ask one final question — I realize we’re coming up on time. At one point when we were chatting previously, you talked about, for want of a term, a generational waterfall for data centers — that some things for a year or two, or even less, might be on the hardest training problems and then would get repurposed to inferencing problems.

I’m sure I’m not describing this correctly — but I was wondering if you could talk a little bit about how you think about that.

Harqs Singh: Absolutely. If you think about the latest chips that are coming out, it’s really interesting to see how they’re doing in some of the benchmarks now. When we spoke last time, James, there was this piece where you put the n chip on training and then n minus one you look to reuse for other capabilities that are generating some revenue.

I can see right now, from some of the demand-supply imbalances in the whole market, even some of the older chips that have been in the market for four or five years are still quite valuable in terms of the GPU cloud prices they’re getting. So there is this sort of short-term skew right now because capacity is not coming online quick enough, essentially.

You need to think: are we building an upgradable design? If you deploy GPUs today and you get three, four, five years’ worth of value on them on a contract with an offtaker, then in five years’ time, how do you change the design such that you can deploy the latest chip from the industry — from NVIDIA, essentially?

What we’re seeing is: you need to update some of the power plant from AC power to DC power. You need to upgrade some of the densities they’re going to be operating at with liquid cooling. So we’ve designed that in from the start.

What that enables you to do is have flexibility — and also be able to create this into a long-term sustainable asset. Flexibility is really important because if you get out to five years, you’re then trying to guess for that chip you deployed five years ago: what’s the market value of that, and what’s the revenue potential of it?

I think at that point everyone is optimizing for flexibility because nobody knows what’s going to happen — as many options as possible. To your point, part of that option could be: you relocate the chip to inference. It could be: wait a minute — I heard a story just two weeks ago where a client signed up to a five-year deal; after five years, for the same chips, they signed up to a 20% price increase because the demand-supply imbalance was so bad.

James Kaplan: That’s a new thing — at least in my recollection.

Harqs Singh: Absolutely. What we’re trying to do is give you as many flexible options as possible — whether it’s relocating it; whether it’s saying that chip is no longer relevant because there is a demand-supply imbalance and you need to upgrade to the next chip; or whether there is real revenue and market value to that chip even if it’s five years old and fully depreciated. Even if it’s fully depreciated, James — obviously from an economic perspective, you don’t need to get the return you had before because it’s a fully depreciated asset, and that gives you some flexibility as well.

We’re fully focused on giving you full flexibility and options so you can maximize revenue at the time based on what the market is doing. And that’s the bit we don’t know: what the market is going to do.

James Kaplan: Fantastic. Thank you so much. I hope you found this conversation as enjoyable and as interesting as I have.

Harqs Singh: I have. Thank you for having me, James.

James Kaplan: Thank you so much. It was great.

Thanks for reading Prosaic Times — subscribe to get every issue!

This episode talks about a world where old boundaries seem to be disappearing:

* Between the intellectual and the practical worlds, as we need to apply theoretical knowledge almost as soon as it’s developed

* Between the sciences and the humanities — senior executives need to draw on both

* Between the business and the technology domains — as companies connect everything to a network

* Between strategy and execution — as companies set up closed loops between ambitious plans and granular improvement levers

* Between the world of commerce and the world of geopolitics, as non-state actors may attack commercial enterprises for strategic reasons

Thanks for reading Prosaic Times — share it with a friend!

From the 101st to McKinsey

James Kaplan: Welcome to the Prosaic Times video podcast. One of my very favorite colleagues, Charlie Lewis, is joining us today. Charlie, can you introduce yourself and tell us a little bit about your professional journey from the U.S. Army to McKinsey?

Charlie Lewis: James, you covered it all right there. But no, it’s truly an honor to be here. James. I think, you know, I’ve, loved getting a chance to work with you over the last eight years or so, but. You know, I am in the fortunate position now where I get to help our clients think about how they address their most critical security challenges, across a variety of industries, mainly banking, but also critical infrastructure.

You know, as you think about the utilities and energy, we think about life science and medical technology, and we think about broader healthcare. So really what frames, you know. Sort of the day-to-day life and what our requirements are within there. And I get to do this as, as sort of the leader of our service line in cyber in North America and Europe.

And I’ve been at the firm for about, you know, as I just said, eight years and before that was in the US Army. And while I was in the Army, I was at, I started as an artillery officer with the 101st Airborne Division with a Ranger tab. So, you know, a lot of esprit de corps within that organization.

and then went and had the, the fortunate opportunity to go to Harvard Kennedy School, where I learned from Eric Rosenbach and Richard Clark on, on cyber war and terrorism. And then I ended up. Teaching in the Department of Social Sciences at West Point. And at the time, the Department of Social Sciences at West Point, sort of three of the core leaders in that department had worked with General Alexander to help stand up cyber command.

It was Colonel Suzanne Nielsen, who was the deputy head, and then, just retired as the head of the department. now Colonel, Heidi, Brockman Deist, who is now the head of the department and Scott Handler, and you may not know Scott Handler, but he is close to us because of Stephanie Handler, his partner and wife, who is now a partner in a law firm, was one of our cyber legal folks.

and is actually, they’re very close to the Reserves as well because Maura and Stephanie Handler went through the basic school together in the Marines. but I was able to be with them and the army stood up, the cyber branch and as I thought about it at the time, it was, is artillery the right. Spot for me to be long term?

Or how do I think about the evolving nature of warfare? and where sort of the core one shaping of the battlefield, the intelligence around the battlefield. What do I need to defend? And then how do we think about influencing operations prior to actually getting in there and being able to conduct operations in a way that don’t put.

you know, humans and, and Americans and our allies in harm’s way. And so I switched to cyber when I was down at Fort Gordon, helping stand up their sort of leader course down there. I realized that there’s a lot more to do in cyber than just from a defensive standpoint, and that led me through the interview process and to McKinsey.

James Kaplan: Fantastic. I’m glad you brought up Stephanie Handler. I should drop her a note. She was one of the most — I hadn’t realized she went to basic school with Maura — but Stephanie I always thought was one of the most thoughtful people I’ve ever encountered on the intersection of legal issues in cybersecurity.

Charlie Lewis: I agree. And she’s catching a little bit of a stray here. Right. But I hope it’s a good positive stray on, on this one. But she’s been really good at it. Right. And, you know, and, It wasn’t easy. Right. You know, we, at the time when she was right, it was COVID. we were doing a lot of the more advanced technical testing at the time, which required her input and, and support and making sure that we were protecting our clients in the firm the most.

And so there were many weekend and late night calls with her as we were getting the work, going.

James Kaplan: And I’m also — you know, a chuckle, not chuckle — I’m glad you brought up Eric Rosenbach. When Tucker Bailey and I were doing the research that became the book Beyond Cybersecurity, Eric was very generous with his time.

Charlie Lewis: he, he’s been great. I remember, there were some individuals who had unfortunately ended up in the US that we had, You know that w we had arrested in in Iraq and I had to go meet with folks, and Eric had given me some advice and all of that guidance, and I think that Richard Clark would be a little upset if I did.

Also didn’t highlight that he was a faculty member who gave us his book for free — the Cyber War book — which still sits above my left shoulder up there. Actually, oddly enough, it’s — if you look, you have Cyber War right here,

and then you have beyond cybersecurity, right, right up there. So,

Oh I take the dust jacket off ‘cause it, it is a cleaner look with just the, the black. And that is, it’s, sometime, it’s sometimes a bit more about form than function for me. So.

Soldier-intellectual, technologist-intellectual

James Kaplan: So here’s one of the things I wanted to cover today, why I especially want to have you on the podcast. You know, many years ago I read the book The Fourth Star about the Department of Social Sciences in the U.S. Army. One of the reasons I find you especially intriguing, Charlie — you’re interesting — is you’ve gone from being what I would describe as a soldier-intellectual to being a technologist-intellectual. Right. I find that to be an especially interesting transition, and I just wonder if you could reflect a bit for us about what it means to be a soldier-intellectual.

What the transition to being a technologist intellectual means and what intellectual inquiry means in the Army and what it means to the private sector and how that differs a little bit. So I realize that’s a lot.

Charlie Lewis: That’s a, that’s a weighty question that we would, you know, teach to our cadets in a, in a lab. I, you know, for me, I think there’s one sort of through line on, on all of those, and it’s what it means to be a professional. Right? And so, you know, you could anchor in, you know.

you could anchor in Huntington’s sort of definition of it in the, the liberal arts education, the continued learning and what that requires, you know, you could anchor into even where I think a really good example of this is in the medical profession, and I grew up with. To parents in that my, my father, a pediatrician, my mother, a pediatric nurse practitioner, and their sort of continued need to learn and the requirements that they had to have to learn.

And then broadly, if you think about the profession of consulting and what that means and the academic requirements and the intellectual curiosity that I think we need to have. you know, I think being in, so, and, and it’s no longer there, but the halls of, you know, the, the hallways in Lincoln Hall, it, at, at West Point at the time were sort of, you know, a place where we could have large debates and some big thinking about what is the future of the force and how do we think about the future of the force and the freedom to study what.

We needed to study. I studied the impact of military voting on local election officials with Dr. Rachel Sondheimer. we, I wrote about social capital and the impact that it has on the military force in our ability to come together and work cohesively as a unit. But then there were some. Broader thinking that existed.

Right. Learning to Eat Soup with a Knife by Dr. John Nagl, right? Sort of the core foundation before the field manual on counterinsurgency was written by a social scientist, and there was just frequent debate, you know? And then how we actually think about the talent and the human capital management within the force.

Those ideas came out of a joint venture between the Army G-1 and Department of Social Sciences, and with OEMA, the Office of Economic and Manpower Analysis. Right. And like there’s just so much brain power that. Sat in there and, you know, we’d have like Thursday afternoon debates on the Oxford comma and I, I don’t think there’s, you know, many units in the army, but it was a bunch of folks who were super professional, who understood what they were doing there and sort of teaching, the, the, the future leaders of the Army who all had combat experience.

Right. Most of us had either commissioned right before nine 11 or right, you know, within the three years after nine 11. I think like the least amount of deployed time, in, in that group was like 27, 28 months. Right. of time overseas. And so it was just great to be able to have that. But the ability to go back and think and broaden and understand what your profession does, I think that’s.

Core to the military. You think about Army Futures Command, you think about sort of the broader strategist position, the ORSA positions, all of these sort of thinking and engagement with the broader community allows you to, you know, understand a bit of the broader impacts, right. And think more strategically about what could happen.

And that intellectual power is sort of a bit of what I’ve tried to take and what I’ve found a lot at the firm in terms of thinking about from a technologist standpoint. I get it — you know, I just got back from RSA, it’s nerve wracking to try to stay ahead of the threat and the changes that are there, right?

Because it’s no longer, you can, no longer, you have to, like I am, folks have to be super technical in identity and access management.

Right, but it — you used to get really deep on a specific technology and a specific component of that, but now we’ve gotta think about how that plays into the broader agentic landscape and what you need to do around identity and making sure that you meet the autonomy, the capability, and the controls that need to get put in place there.

But how do you learn? You have to read, you have to study, and you also have to do, right. I think that’s one. You know, a bit of a difference here on the like intellectual curiosity, right? You have to be able to play with the toys, understand what it is, and so you can explain some of those difficulties to to clients as they go forward.

Tinkering, agents, and evidence

James Kaplan: You know, one of the things to me is in the business world, sometimes it feels like technology is the most intellectually forward-looking of the business functions. Right. And then within technology, cybersecurity, you know, that feels like the most intellectually on the front foot domains, you know, within, within, within enterprise technology. And it’s a domain where people are constantly experimenting and also constantly trying to engage with, you know, the academic community in order to stay ahead of the threat.

Charlie Lewis: There’s, I know there are a few academic articles that are out there. Right. And I think this is an interesting place ‘cause the academics get to do some of the testing and it’s really about the impact of agentic AI on security. And then one about, you know, more broadly around.

You know, LLM security and, really thinking about what the various attack paths and the risks could be and how they get implemented, right? So we’re taking that at the exact same time. We’re looking at technology that’s being implemented, and it is a fascinating blend, right? And you have to have the researcher angle that goes in there.

I’ve seen more organizations start to stand up like an innovation and research arm, right? A group of sort of researchers or a leader who can pull in others. I also think there’s like a talent incentive there too, right? There’s an a sort of a, a constant tinkering where like I go back to like the videos like you and I have shared about tinkering or when my, you know, my Harley died and I was able to get that back up and running and how proud I was of myself, right?

There’s a bit of like a tinkering that goes on in here. And you have to have the patience. Right. The same way that, I’m a, I’m a geography major, so Right. But like I remember my roommate who was a CS minor, right? And like how much tinkering they would have to do, I would always go to bed early ‘cause I was a, you know, we called it dirt.

I was a people dirt, human, regional geography major. Right? But my roommate would, would, keep, would keep tinkering, right? And they’d have, and that’s sort of the mindset. And so I think there’s always this desire to tinker. And get better. And that fits a bit into the innovation side, right?

And I think that any CISO or security organization that is sort of resting on their laurels or previous success is going to find themselves behind a little bit. And I also think there has to be that communication, right? I think about the difference between a brand new technology environment and where I spend a lot of my time — legacy, overly complex environments, potentially IT, OT, multiple global systems, maybe four or five different identity programs in place.

A horrendous, IT asset management program with an incorrect and out of date CMDB, right? Like so sometimes it’s that education, but it’s also understanding what the foundational requirements are to be able to move forward too. And security folks are pretty good at understanding that. Right. Where we stink is communicating the value of that to the business and why they have to invest in that if they wanna continue to grow and scale.

And I think now security folks have to be able to do that. And then explain why. I’ve got, there’s an argument that I think we could make that the first use case for agentic deployment in any organization should be security. Right? And you say, why does it need to be security? And it’s because that’s where the hackers are going, right?

We’ve seen that with Truffle Security in their report on what they were able to do with a Claude agent. We’ve seen this on what has been built out in terms of the Microsoft sort of attack path with an agentic attack path; we’ve seen novel attack paths for legacy vulns. And so now I actually think like there’s the real value there in getting ahead of it.

‘cause if you don’t start now, you’re gonna lose. And the only way that you can keep the value you want going forward is to have that security on the backend to sort of protect the folks coming in as the business pushes out.

James Kaplan: There’s one thing you said a couple of minutes ago that I just wanted to emphasize about tinkering and the patience required for tinkering. And as someone who — you know, personally — maybe has the patience of a fruit fly, I’m so excited now because the patience required for tinkering has gone down, right? You know, the tools help you with the syntax — think about the underlying issues more — and therefore reduce the barrier to tinkering and, you know, sort of — I don’t wanna say democratize — it’s tinkering because it allows more senior people to tinker more, but allows maybe people who might have, like myself, who might have said, oh, God, I don’t have time for this, I gotta go write this memo — to get their fingers back on the keyboard in a way that’s, you know, very constructive along multiple dimensions.

Charlie Lewis: Mm-hmm. I, I’m, you, you’ve worked enough with me to know that my level of patience is, is slim to nil. right. Like, you know, I’m, I try to make my life as efficient as, possible. You know, and, and you know, to include even like, sitting on the same seat every flight on an airplane, if I’m, if I’m able to, right.

And, and that way I don’t have to to choose. and I, and I think that, that the tinkering allows us, there is no excuse now to not tinker. There is no excuse because it is just so, it is so easy. Right. And I think like. The first thing I will do no matter what, is just like we’ve built, right task and research agents, right?

Super easy, super basic to build them. It’s a 15 to 20 minute upfront investment. One of my buddies says, how’d you write a good prompt? I was like, I asked chat GPT to write me the prompt that I want, and then I. And then I go and I edit it, but they can write phenomenal prompts. And that take, that saves me a bunch of time.

And then I run that prompt and I, and then I start building out like the GPT and ChatGPT. And it’s like, and you know, now we’ve got, you know, Gemini and the rest of the stack — it’s super easy to build an agent within there. And so — it makes life just a lot easier and it’s fun just to see small changes in how you can get the right output that can come from it, and it can improve an output, it can improve everything you need.

And for me, you know, I think one of the best things that we’ve done is we had one of our colleagues, Jose, found an entire GitHub repository that had like every 2024 threat report, and that was out there, right? We downloaded it. I, was like, oh, this could be great for the practice to understand what the threats are by industry by.

By threat, actor by region, et cetera. We pulled that in, but we had to, like, I had to fiddle around with, learn the instructions on how to be able to do it and zip ‘em and you know, extract from the zip and do right. And it took me 30 minutes, 45 minutes to be able to do it on a Friday. But it has probably saved us hours of time and has given better answers to our clients when they’ve asked.

James Kaplan: And you were touching on one of the important themes to me, which is the power of using AI to convert unstructured text to structured data, which you can synthesize, analyze, and act on much more effectively.

Charlie Lewis: Correct. And you’ve gotta check it. Right? And I think one of the things I’ve learned from you and from, from Rich is how to build in the checks that go into there to make sure it’s right. You can’t just assume that all the data is right — you should never assume the data is right. Right? Like, look at a footnote.

Go to the, go to the, the source, and, read and read the source. Right? My students used to be like, well. Like, where do I find all these sources? And I was like, you can’t use, you can’t cite Wikipedia, but sometimes Wikipedia has pretty good sources, and all you need is one source from a Wikipedia page that then drives you into the original primary sources from there.

And then you’ve done a bit of your research and now you’re able to do that as well. And if I have to do deep type research on it, right, you are checking all of the sources. You’re making sure that the sources are, are correct and it, it is just about. Asking. It’s, learning how to ask better questions.

I use it with my daughter to teach her how to ask better questions.

James Kaplan: As I like to say, if your mother says she loves you, check it — as the old newspaper editors used to tell you.

Charlie Lewis: Yep,

James Kaplan: One of the phenomenal things about Wikipedia — it’s not that every sentence must be sourced, but sometimes it has sources. You know —

Charlie Lewis: That’s true. You can always check.

James Kaplan: And you and I have talked a little bit about how much I like the book Military Power: Explaining Victory and Defeat in Modern Battle by Stephen Biddle.

I love this concept of force employment. I think you are in some way describing force employment as applied to knowledge work. Everyone has access — Biddle said, at some point, many people may have access to the underlying technologies, but it takes discipline in an organization to apply them effectively in an integrated way. And you are describing, I think, how to apply a set of technologies that are available to pretty much everyone in a more integrated and disciplined way in the service of knowledge discovery, which I find really interesting.

Charlie Lewis: I think, it’s fascinating the number of times that. Someone sends me a job description and it’s like, must have a BS in information technology or computer science, and I just like cut that out. Right? And, and it’s like, I don’t care about that. Right? Because what you learned in CS 50, while it’s good from a foundational standpoint, when you’re moving out into the cyber world, what you learned in that.

You know, over those four years at this stage, what you learned four weeks ago may be out of date. And so you wanna be able to build and scale and have a team that’ll learn and doesn’t wanna just rest, right? Like the question, like, you know, what do you do on your free time? Right? Where are, where are you focused?

Are you sitting down? Just wasting and, and scrolling through something on a plane or like you right, you were running some work, a workflow on, on, on, you know, your flight from, from London to la right? And so what are you doing in the background and sort of the downtime to keep learning? And, and that’s what I tell clients to look for in people.

And, and the best example that I have is that the first class of cyber, Of, cyber basic officer leader course graduates. Now, these folks, they don’t, they, they take the core basic military, so they’re going out to the range. They’re doing land navigation. Right, but they’re also taking CISSP, right?

They’re taking CCNA. They’re taking a whole slew of SANS courses because they have to get this training up and running. The the top grad, the Honor grad, was not a computer science grad from. West Point was not a computer science grad from an ROTC program, but was an economics major from the University of Central Florida.

Right. And he was able to do it because he understood the application of what we were learning in the class. And so I think a lot of the learning allows you to then there, there’s not just learning, it’s the actual application within your day to day that then creates that broader scale that you need to have.

And then allows for something you’ve taught me is sort of like always question, right? Always see is there a slightly better way that we can do this? Is there an improved output that we can get because of it? Yes or no? Is it worth that investment? Most of the time it is, right? And then how do we get there and how do we help our clients get there?

Humanities, ‘be human,’ and geopolitics

James Kaplan: There’s an interesting tension here. On the one hand, I had a good discussion with Associate Provost Michael Littman from Brown about this — that the principles of computer science are now more important than ever. You know, the syntax of this language versus that language, who cares — but principles around things like data modeling and abstraction are more central than they ever were. On the other hand, I think you’ve talked a little bit about how essential curiosity is, how people who may not have majored in computer science can get up to speed. You are a geography major; I’m a history major. Right. And —

Charlie Lewis: Rich is an English major.

James Kaplan: — and Rich is an English major. Okay. We gotta talk about that with Rich too. I was wondering if you could comment on the relevance of the humanities and the social sciences in a business and a technology context. And let’s face it, Rich is also incredibly technical and someone who has done technical stuff. So I was wondering — how do you think about integrating the culture of the sciences and the culture of the humanities, and what’s the relevance of the humanities and the social sciences in a technology context, in a business context?

Charlie Lewis: I, and I wanna be like a bit, so like one? like. I was convinced to major in geography and people geography, right? Again, the people Dirt. And my, my EV 2 0 3, so my Principles of geography professor, right at West Point when I was there your sophomore year, everyone’s putting a cell on to get more majors.

More majors means more classes. More classes equals more, you know, faculty, more funding, more, more prestige, and You know, he stood up in the, in the class when we were picking our major and he was like, if you like to look out at the airplane window and wonder what people are doing down below, you might be a good, you know, human regional geography major.

And I was like, that’s it. Right? That’s what I want to do. Right? Like there wasn’t, for me, there wasn’t much thought I wanted to be a history major. Right. Unfortunately, I had a bad experience in my first year of history at. At West Point, no offense to anyone who was there. I just had a bad experience and it wasn’t where I wanted to go.

Same reason why. What, what pushed me into artillery, right? Like there were people I liked to went artillery, there were people I liked who went infantry, right? But the people I didn’t like also were infantry. Not the people, but the officers, right. Who had had a bit of a problem with Right. And I tend to like flow with where I want it to go.

But I think, you know, as you think more broadly about the role of, of sort of the humanities, within that space and sort of more of the, the soft sciences, right? It’s the same reason why they’ve scaled and grown those majors at an engineering school like. West Point is the ability to critically think and analyze, right?

There is a structure that goes within there and we can apply the same sort of structured thinking, the hypothesis tending testing in a, In a political science way, in a historical way, and even looking at broadly at some of say the logic arguments in philosophy and apply those specifically to life in general and then understanding there.

And I think. You know, there is a ton of value in the foundational learning, right? The, the firm just, you know, you know, hired someone who was a physics and philosophy major at West Point. I asked him, when I taught him at West Point, why are you doing those two? And he said, because if I understand the foundations and I can do better at understanding everything else.

And I think there’s a clear, you know, fundamental thinking behind that. But the humanities to me. Provide that broader thinking and that broader understanding. And you know, I’ve, I’ve been telling people frequently in a world where everything is becoming a robot, be human right. And I think that a lot of times being a human requires the humanities and, and a bit of an understanding within there and the application and the impact that something could have on the world.

And, and, and that’s where I think like when you get to. Thinking about where we are in terms of technology now, it’s like, what is the impact it can have on a business? Is this good or bad? Right? What is the impact that this can have on the society that the business serves? Is this good or bad? Right? And for me, we, I spend my time trying to stop the bad that’s impacting society by improving, you know, the defenses that a business may have.

And like, you know. we’re fighting, like businesses are fighting nation states right now. Right. And businesses are fighting, you know, international criminal syndicates that you would never think would wanna go after a paper company, right. Or, or a drink company or anything like that. But they are because, or a small town, they are because they can make money off of it.

James Kaplan: Well, okay, let’s, let’s lean into the history a little bit. You made an interesting point the, we’ve seen a, you know, compared to, say, the Cold War era, there’s much less of a — or maybe even the post–Cold War — Much less of a hard line between the world of geopolitics and the world of business, or as you point out that we have non-state actors with geopolitical aspirations engaging in conflict with pri, you know, private sector institutions.

Right. I, you know, know, this was not what we’re used to, but it’s not historically. Unprecedented. Right? If you think about the history of the East India company, you think about the history of privateering. You know, you, you go back a couple of hundred years and there was, you know, at that point a much blurry line between the world of geopolitics and the world of commerce.

Charlie Lewis: Like I, I think, I, you know, like I enjoy reading, even though I live in coastal Connecticut and look goofy in cowboy boots in a cowboy hat, right? Like, I like reading about Western expansion and, and one of ‘em is, you know, read a book recently on fur trading, right? And the impact that, and the success that American Fur Traders had.

Keeping like the Canadians in really Hudson Bay out of sort of the northwest of the US and, and, and sort of that broader view when you think about just these, these men, these like they were men at the time, but like rough men who went out and basically. Didn’t think they were conquering territory, but they were conquering territory for, for a business end and sort of to own that business globally when it came to the broader fur trading.

Right now, they, they basically worked themselves out. There was not a conservationist thing, it was just make as much money as I possibly could. Exceptionally violent, all of that, right? But they were, they were. Private citizens that were going out and conducting business on their own and getting into conflict on their own.

With, with, with, you know, the Native Americans at the time, same exact thing with some of the, the privateering from ships, right? And like what you find on like the Treasure Coast and in, in Florida and the potential of gold still rolling up there in some of those requirements. and so I think that.

The, the difference now though is this is one, it’s not nearly as visible. Right. Like it is very hard to, to see and understand what this means. I think about the various, you know, volt and, and salt typhoon and what that means for our society, right? And really going and targeting core telcos and core utilities within the United States.

So it’s one, it’s not visible, and two, the scale is just massive, right? Like previously, those fur traders had to go out to California, in Oregon, in Washington, right now, someone could sit in Eastern Europe or Asia or anywhere that they wanna sit and still be able to conduct operations. Where, wherever they’re able to, and then now when you build in the ability to build agents to go out and do that one, it’s faster.

And two, you can build sort of robotic, you know, robotic armies that are operating in, in, in terms of ones and zeros through. Through what we use for, to communicate all the time, and it that becomes just an avenue of, of, of approach and an avenue for them to get into the environment in a scale that, like I, you know, you’ve had Phil Venables on here, he talked about it in his recent blog coming out of, o out, out of RSA, right?

Like it is just. I think there’s just a, a nervousness within the security community about like what is gonna happen in the next six months. Right? And look, in February we’re talking about what’s gonna happen over the next year. Right. You know, and now we’re a month later and it’s like, it’s here. It’s not the next year.

And so now everyone’s like trying to tighten the timeline down a bit. But not to sound too spooky, but it really could be tomorrow. Right. And I think that’s like, that’s the, the scariness and like how do you, you not get ahead, but get up, get up to speed.

When binaries break

James Kaplan: You know, if I were to take a step back and I were to synthesize the discussion we’re having, the theme is one of blurring lines, right? We talked about blurring of lines between the academy and the intellectual world and the business world. We’ve talked at some level, I don’t know whether it’s the blurring of lines or the overlap. between the sciences, the social sciences, and the humanities, how important, the humanities are for applying social, you know, computer science, for example, effectively. now we’ve talked about the blurring of lines between the geopolitical world and the commercial world, right? At least in the, in the current tier. I was wondering, you know, and yes, I admit this is a bit of a weighty question. If you just take a step back and. How should a CISO A CIO think about this? Like we have this world with many fewer binaries, and what does that mean for a leader who is trying to translate all this uncertainty into practical action?

Charlie Lewis: You know, that that is the, the, the million or — in many companies’ cases — the billion-dollar question. And I think it really comes down to — I know, I know — and I think there’s sort of a structured flow that I try to think about when having these conversations and like it’s imperfect, right?

The conversations have to happen at the C-suite level, right? And we try to get into those as much as possible and you have to be able to. I actually get really nervous when I talk to a client and they’re trying to hire a CISO. Like I want a really technical person who’s worried about the inside, doesn’t have a lot of broad engagement, right?

And I’m like, you need that technical person, plus you need. Someone who can communicate to the business. Plus, if you sell something, you need someone that customers can talk to and feel confident that you’re securing their product. Right? And so I think broadly that the leadership of a business has to look at not just technology now, but security of the business.

As a core sort of business enabler is a cheap way, but as part, as a core component of the business, right? You wouldn’t run a business without investing in hr. You wouldn’t run a business without investing in your finance arm, right? You’re, you, you need to do the exact same thing from a security standpoint and make sure that it is treated as that level of importance and it is a critical component.

But the way you have to do that from a C-Suite or from a CIO or a CSO talking right, is one, it’s critical to outline what are the requirements to run the business, right? Phil will talk about the minimum viable organization, right? And, and so. What is critical to run that broad business, you have to have the engagement with business leaders and say, for you to conduct your operations, what are the three applications, the four applications you need?

What is that process? All right, great. I’ve now gone through there. What this actually means for you is I have to maintain and prioritize these following pieces of infrastructure, right? Like down to like the load balancer level. So you understand what that is, a business person, and then you say, here’s where we’re performing against what our expectations are.

So you have to work with the business to say, what is your risk appetite? This is what you and I spent a lot of time on before, right? What amount of risk are you willing to take on? How long are you willing to be down? It Right. How. You know, how, how, much data are you willing to lose, right? What are you thinking about from a reputational and trust issue?

Now, from an integrity standpoint, we used to not talk much about integrity. Now, the data poisoning, data leakage broadly throughout it. Like that’s a massive risk, right? And then you have to frame that and what that investment is in, what it means to the business, right? So it’s not, oh, we, we have risk of, of, of ransomware, or we have risk of, you know.

X number of apps don’t have MFA. That doesn’t mean anything, right? What really means is like, unless we implement this, I actually think we’re more, we’re not within risk. We’re not within our risk appetite for disruption for a standard operation. Look, there are. On the extremes, you’re not gonna solve for those, right?

Like if you’re not, you’re not gonna solve for those. But if you’re able to bring all of that together and to have that conversation across the business one-on-one, show them what they actually use from a technology standpoint to get their work done, show them what that requires, all the way down and then, right, that gives you a good opportunity to get your CMDB up and running.

Right to make sure that your IT asset management program is good, that you can run through business continuity plans, right? And, and if, if, if, God forbid, something happens and you are disrupted from a tech resilience standpoint, which we’ve seen in some of the largest banks to ransomware, which we’re all much more familiar with, I think you understand what the recovery process is and what the order is to get everything up and running.

James Kaplan: I hear you, Charlie. If I play this back, I’m proposing the destruction of another binary between the strategic and the tactical. If I hear what you’re saying, you need both — to connect all the dots, right? — to engage as a systems thinker who can connect the business to the architecture, to the controls, to the security operations. And at the same time, you need to translate those implications down into a set of actions that will change the nature of the environment. You know, okay, we’ve thought about all these things and this means we have to do this about the web application firewall or this about which data we encrypt.

Charlie Lewis: Or. Exactly, or, or even where you are running or storing your data, if you have that level of fidelity on a server in a, in a, in a data center in one country or one region versus another, making sure you have that. You know, and there’s little things too, right? Everyone wants to offshore every their, their work right now and.

Great. Tons of cost savings. Right? There is good opportunity, a ridiculous amount of talent where everyone wants to offshore in India, right? But what they’re doing is they’re offshoring the, what’s that?

James Kaplan: Vietnam, Latin America,

Charlie Lewis: Eastern Europe. Right. Vietnam. The Philippines, Uruguay. Right. Mexico City, San Jose. Right.

There’s a ton there. Right. But most organizations are just thinking about it in one spot, opposite side of the globe. But they also don’t think about like, what does that mean if the people go down, right? If something happens in their systems are down.

We worry about hurricanes in the southeast of the us right?

You’ve gotta worry about. You know, large range, you have potentially have to worry about flooding and power issues. Right. And, and so how are you thinking like structurally, like you said, operations, strategically we’re making this decision like technically and tactically this means this, and then operationally, like you said, there’s that little bit of that piece in there and how do I connect those two to make sure that we’re prioritizing and the other things like.

Tech teams and CISOs need to be aware of, right? Is is there’s a lot of technology. It’s not shadow it, right? But there’s a lot of technology that the business owns. That does not get managed as well or in the same exact way as what the technology team and the security team own, and you have to do more now than just the annual manual attestation of where that, what it is, what it’s used for.

Does it have the controls and where does it sit? You have to have those conversations. If anything moves. Or priorities change, and that has to be more continuous or at least more frequently than, than, than, you know, once a year.

James Kaplan: That’s another binary that’s being blown up between what is it and what is not. It,

Charlie Lewis: Correct?

James Kaplan: more things get connected to the network and, and operating theater is part of your technology environment. A factory line is part of your technology

Charlie Lewis: Yep.

James Kaplan: A bench in a lab, in a pharmaceutical company is part of your, technology environment.

The vending machines in a break room are part of your technology environment.

Charlie Lewis: The conference rooms. Right. It’s, it’s, it’s all there. And it, you know, and, and in fact you could say, you know, if you think broadly on the security side, your ID badge. Right. And where can my ID badge get me into? that’s part of the technology. ‘cause as soon as someone can get in, they’re able to figure they can get around and they can, they can see a lot more.

I can learn a ton about what’s going on just by getting into a business. Right. And we look for the badge, not the face. Right.

James Kaplan: Okay. Charlie, anything else? I, we’ve covered a ton. Anything, any, and, there’s a ton we haven’t covered. we only

Charlie Lewis: I know you didn’t mention, you didn’t mention constructivism, but that’s fine because that’s for another conversation.

I think that’s been, that’s been blown away anyway, so

James Kaplan: we, we drove by and waved at Samuel Huntington, but didn’t stop, to talk about him.

But anything else?

That you want to, cover in this discussion? Any, any last thoughts you wanna leave, people with?

Charlie Lewis: I am obviously biased ‘cause I love what I do, right? I love security, right? And I think that, being a good security professional means I need to understand the business, I have to understand the technology, and then I have to be able to protect both of them, and find ways to also help them achieve what they they need to achieve.

The push towards Agentic AI is making for security operators in one, understanding how they secure their own internal environment and what is being built so the business is able to achieve what they want to and that. Slow them down. And there’s tons of good opportunity there, specifically around the development life cycle, right?

The, the, the, various security checks will start being run by agents and you just have sort of a, a QA towards the tail end, and that’s awesome, right? You can run threat modeling much faster, more effectively, right? But it also means that the threat actor is coming and this is changing. The vulnerability landscape and requiring an entire review of what used to be a low VUL or a medium vul, and is it now potentially a critical vul?

And there has to be thinking about what does this require and security teams should start investing now and anticipating it because if you, it happens before you’re ready, right? There’s very little chance that you can catch up effectively without a ton of spend or additional risk and, of, threats being realized.

James Kaplan: So let’s close with one final question that’s even more serious than vulnerability management. When I spoke with Rich, he said The Hartford Whalers would beat the Buffalo Bills and shuffleboard. Let’s say the game was Scrabble. Okay? The Buffalo Bills versus the Hartford Whalers and Scrabble, who wins?

Charlie Lewis: Bills first off, the Hartford Whalers don’t exist anymore.

James Kaplan: Well, the.

Charlie Lewis: I’m just gonna be very objective on this one. Right. The NFL has rules on how far you have to go in college before you can play professionally, whereas the NHL does not.

So I’m going to make, I’m gonna go back against what I said before, ‘cause I also believe security folks don’t need a degree. . But in a world where you’re comparing those two, right, I think that having to get into college and go through college Will, will roll, will roll with, with, with that one.

Plus, Josh Allen, you know, brilliant, brilliant human being. and so he’s able to go there. But that, I, I also just wanna say that the first ever professional hockey game I went to was the Buffalo Saber versus the Hartford Whalers. and again, that’s why like above your book over there, I have a Pat LaFontaine, so to be frank, I don’t remember who won, but I remember standing on the boards and some Hartford Whaler took a slap shot right at the board and the puck like would’ve.

Hit me right in the face. And so I’ve always had a bit of a, even though I live in Connecticut, a bit of a negative view, on, on that one.

James Kaplan: The first professional sports evet I attended was the Yankees versus the Oakland A. In 1978, the Yankees won and it was less violent. Nobody, there were no slap stakes. They slap shots aimed at the at the

Charlie Lewis: No, hockey, like, I love that sport is an incredible sport. The movement, it’s, it is, it is a beautiful sport, I think. but I also can never let Rich win, anything. So that’s,

that’s the core.

James Kaplan: We’ll have to have the two of you on together. You can debate

Charlie Lewis: I could, I knew I couldn’t compete with his microphone the last time, so that’s why I couldn’t join.

James Kaplan: that is exactly true.

Thank you

Charlie Lewis: thank you, James.

Thanks for reading Prosaic Times — subscribe to get every issue!

This is the wrong way to at it. We see the great advance in human learning since Gutenberg. Ideally without it leading to a modern-day Defenestration of Prague. Michael Littman is wrestling with the big issues at Brown University.

Here’s a few of the things we talked about:

* Implementation to Quality Control: The human role shifts from writing the “how” of code to performing high-level verification and contextual judgment of machine-generated output.

* Higher Agentic Cognitive Load: Orchestrating multiple AI agents increases mental strain, requiring a “bond trader” mindset to manage complex context and flow.

* Closing the Abstraction Gap: Gen AI finally allows computing to tackle qualitative “messiness” and fuzzy objectives that traditional quantitative abstractions couldn’t reach.

* Motivation over Information: While AI excels at personalized instruction, the “rock star” lecture survives because humans require interpersonal inspiration to engage with difficult material.

* A Paradigm Shift in Humanities: LLMs enable a “Kuhn revolution” by allowing historians to systematically interrogate massive datasets that were previously locked in dusty, qualitative archives.

Thanks for reading Prosaic Times — share it with a friend!

James Kaplan: Hi there. This is James Kaplan, and welcome to our most recent Prosaic Times podcast. We have a slightly different guest today—one I’m incredibly excited about—from the world of academia. Michael, do you want to introduce yourself?

Michael: I’d be happy to. Hi everybody. I’m Michael Littman. I’m a professor of computer science and artificial intelligence at Brown University, and I’m also serving as the university’s first associate provost for artificial intelligence. I’ve been having a wonderful time interacting with James on a variety of topics.

James: So, do you want to tell us a little bit about your journey? How did you become a computer science professor? And tell us a little about your area of academic research.

Michael: Sure. Thanks so much. When I was a teenager in the seventies, I was wandering through a shopping mall and looked through a window and saw a computer in a Radio Shack. I didn’t know what it was, but it looked very interesting. I walked up to it and started typing things at it—and it knew the answer to all the arithmetic problems I could pose.

I tried to be really tricky. One plus one? Two. Okay, sure, everybody knows that. But what about one plus one plus one? What if I put parentheses in? No matter what I did, it figured all of them out. That was the kind of thing I thought only people could do. I guess I didn’t have a very good calculator back then, but it just blew my mind. I found it fascinating. I kept talking to my parents: I think I want this. They said, well, it’s very expensive. Have your bar mitzvah. I said, I don’t want a bar mitzvah, I just want the computer. So they said okay. I got the computer. I spent the next couple of years rewiring my brain to become a computer scientist, and I’ve never really looked back. To me it’s just so deeply interesting.

What draws me to it in the first place is how much it feels like a machine that’s thinking. The area of computer science that studies that—what we can think of as a kind of thinking—is artificial intelligence. Once I got to college and learned that word, it was: oh yeah, that’s my path. I’m going to be a computer science professor who studies artificial intelligence. I’ve tried to stay pretty focused on that and it’s worked out really well.

Except now I’m a university administrator who studies artificial intelligence, and that was not on my radar as a teenager.

James: You woke up one day and found yourself in the middle of the complexities of university administration—something artificial intelligence had never quite prepared you for.

Michael: You can think of it as a kind of complex computational system, because it really is. It’s very organic; it’s got people in it, and people are so complicated and weird and noisy. It has a lot of aspects that feel very different to me from computing.

But what I’m really loving about this position right now is that I’ve always been interested in AI. I always thought it was super interesting and nobody believed me—until ChatGPT came out, and then suddenly everybody was really interested in talking about artificial intelligence. I feel like: this is great. I finally have the conversational partners I’ve been wanting all these years.

James: I’ll admit I thought machine learning was really boring, and here’s why. The first time someone showed me machine learning, I thought: yeah, okay, that’s sort of interesting. Clearly they’re doing massive numbers of correlations and using some sort of champion–challenger model to find their way to the best optimization algorithm. From what I understand that’s not precisely how it works, but it’s kind of how it works. And I said: that’s great if you have a really clear objective function and at least reasonably good quantitative data. But the problems that interest me and the problems I have to deal with often don’t have really clear objective functions and never have really good quantitative data. What came to interest me about Gen AI is that we could work with fuzzier objective functions—and even more, we could attack problems by hook or by crook that involve qualitative data.

Michael: I think that’s really insightful. The gap you’re mentioning—between what computers are really good at and what people really have to deal with in the real world—is very interesting. I think prior to Gen AI becoming something people are aware of, those two camps really didn’t speak to each other very clearly.

The folks who understood the complexity of the real world said: well, it just can’t be done; we can’t put it in a computer; we can’t have this conversation. Whereas computer scientists are trained from almost day one to create an abstraction and then really embrace that abstraction.

When you see people in the news, public figures saying “oh, this is just an optimization problem”—first of all, they’re wrong; second of all, that’s because of their training. They’re actually taught to think that way. The boundaries between the real world and the abstraction start to vanish for them. They see the abstraction and they think: if I solve this abstraction, I’ve solved the real world. And that’s also wrong. The notion that “nothing the computer can do can help” is wrong. But the notion that “the thing I’m solving in the computer is the real world” is also wrong. In this Gen AI moment, those camps still exist and they’re still not talking to each other—but they’re closer together than they’ve ever been. We can have messy machine learning and we can have a digital real world to some degree.

James: Gen AI has challenged my computing model. Thirty years ago a colleague who knew nothing about technology asked me: what can you do with a computer versus not do with a computer? This guy loved yellow legal pads; he would work out every problem on a yellow legal pad. I said: computers are really, really fast yellow legal pads. Anything you can do on a yellow legal pad, you can do on a computer more quickly. Anything you cannot do on a yellow legal pad you will never do on a computer. I’m not so sure that’s as relevant in 2026.

Michael: That’s really interesting. But you could draw a portrait of someone on a yellow legal pad. Was that part of the way you thought about it?

James: That’s fair. You may be testing the assumptions of my model. This person was not inclined to draw on yellow legal pads. He was inclined to sketch flow charts and rough out financial models.

Michael: The yellow legal pad was that person’s abstraction. That’s fascinating. It’s very true to some degree—or maybe to a lesser degree, but not to no degree. I do think people are continuing to forget that. When I hear some tech leaders saying “oh, this is going to replace everybody’s jobs,” I think: you don’t understand jobs. There’s so much more to it than the abstraction you’re thinking of. Oh, a job is just: you take this input, you produce this output; I can train a computer to do that.

James: I would argue there are many jobs like that. Large parts of the white-collar workforce can be thought of as engaging in copy-paste operations—moving data from email to spreadsheet to word processor. We’d like to think we can reduce that toil and allow those individuals to do more interesting and rewarding things than copy-paste.

Michael: Yeah, though sometimes I also think part of the job is to go to meetings and hear how things are changing. People adapt the way they’re doing this over time because they understand at least a piece of the context—the little sphere they live in, or the little sphere we all live in. We all live in our own little spheres, but those spheres are way bigger than the yellow legal pad. There’s all this interpersonal stuff and an understanding of the organization as a whole that I think you lose if you just abstract it to: oh, it’s just an input–output relationship.

James: I’m talking about part of their job. Someone may go to the meeting, take something away from it, and then say: okay, now I need to do 14 hours of copy-paste operations and go back to the next meeting.

Michael: That’s fair.

James: It’s nice to say: okay, your job is now going to be to interact with other humans, and a machine will do the copy-paste operations, hand you back the result, and you can go interact with more humans. That feels like a more rewarding job than 14 hours of copy-paste operations.

Michael: Another thing computer scientists are trained to do is to hate copy-pasting. Anything we do that feels mechanical—we’re taught: automate that, automate that. So you’ll find some computer scientists spending way more time than necessary solving a problem because they just refuse to do the mundane version. They develop a whole software system that’s completely unnecessary to solve something they could have just cut and pasted.

James Kaplan: Franklin Covey taught us: always take five hours to save five minutes.

Michael: There you go.

James: I’ve gone through my own experiments with Cursor and Claude Code—a lot of taking five hours to save five minutes. I spent probably two hours getting Cursor to write an email for me using Gmail for a non-work thing. I had to download and install the new Gmail command-line interface, authenticate it, figure out how to use it. That went a lot more quickly because I didn’t have to figure out the syntax myself. But at the end of it I had spent two hours to send an email that I probably could have drafted in 15 minutes—because I needed to assemble data from various places. But now I can send that email much more quickly. I’ve achieved operating leverage. I can send the next email much more quickly too.

Michael: Right. That makes a ton of sense.

James: This is incredibly interesting for universities. I think there is both a real risk of universities being disintermediated along multiple dimensions and a tremendous opportunity. A lot will depend on how they choose to act over the next decade. I was wondering if you could reflect on both the opportunities and the risks.

Michael: That’s a great question. There are two things I spend a lot of time thinking about—how universities are kind of special in this moment. One is the notion that AI is this force that’s been splashed down upon us, and we’re trying to figure out how to make use of it in a way that’s productive and consistent with missions that civil society can make positive use of. I think it’s hard. I think it involves a lot of change on both sides: the technology needs to change to some degree, and the structures of society need to change a little bit as well. I don’t think companies are necessarily in the right stance to work on that.

James: Companies have all sorts of problems figuring out AI.

Michael: Fair enough. It’s hard. I think we all need help figuring this out. But what I’m getting at is: on campus we have people who have regular civil-society problems they’re trying to solve. A bookstore; we have a sanitation department, a security force. We have the things you’d have if you were running a city. But we also have AI experts. We have sociologists. We have humanities people who’ve thought about the sweep of history and how human beings adapt to certain kinds of change. We can actually construct this future and live in it and be a model for the rest of civil society if we do this well. So I feel we have both an opportunity and a responsibility to try to do that.

The other side of it is that these chatbots are essentially homework machines. They take as input homework questions and they produce answers. The API that’s been established with these chatbots perfectly subverts what we’ve been doing for decades in terms of how we run our educational process. That’s extremely disruptive. We can’t just ask the same kind of essay questions we used to ask. We can’t just give the same kind of problem sets, because the temptation—almost the demand—is for students to feed it to a chatbot and write down the answer. So we’re in a situation where we’re being completely subverted. Even if we’re not disintermediated—even if others don’t step in to do what we do—the way we do what we do has to change. That’s extremely unsettling for me and my colleagues.

James: I think that’s an easier problem than everybody says. Here’s why. Your professor who teaches—what, grades 13 through 17?

Michael: Roughly. I actually have PhD students, so it goes up a little further.

James: Okay, 17 plus. I teach grades 18 through 30, in a sense. When they’re done with you, some of them come to me. In a professional-services world we give different homework assignments—assignments that face the brutal grading of the market. I have not seen any circumstance where raw LLM output is good enough. There’s part of me that thinks college professors are saying: hey, folks, what used to be an A++ is now a C−. That is what it is. Because as you move into law, business, finance, government—testifying before Congress, writing for a publication—it will neither be acceptable to avoid AI tools nor acceptable to just hand in raw output from a large language model. Does that make sense?

Michael: Yes, I think that’s exactly right. There are two aspects that make this hard for us, but I think that is exactly the path we’re going down. What we think of as education is changing to reflect this reality: we have this entity we can delegate some of the details to, but we’re responsible at the end of the day for the final product. That’s a shift. We spend a lot of time teaching people about implementation, and now we have to spend a lot more time teaching people about quality control.

James: You teach not only application but underlying theory, where there’s a little more of a textbook solution potentially. You may need to collapse the teaching of theory and application—you learn the theory by applying it in real time, which is what you do in the professional world. We know the textbook solution you can get from the machine. But the application of that solution to a particular context you’re much less likely to get from the machine. So that’s what we’re going to challenge you to do from the day you step onto campus.

Michael: That leads perfectly into the second point I wanted to make. We do have to change how we’re teaching and what we’re teaching. But the thing we’re scared about is we don’t really understand the role of asking these kinds of questions in people’s cognitive development. When you see people at 18 to 30—grades 18 to 30—the hope is they’ve got a really solid cognitive base on which all this other stuff is built. Building that base is an art. Good educators somehow manage to unlock this in people and help them get the right concepts and the right motivation to work together in a beautiful way. If we were to shift everything tomorrow to the model you’re describing—theory and practice really come together, they’re only useful when they’re supporting each other, so we should just do it that way—I don’t think we know what’s going to happen. I think that’s a giant experiment.

James: If anyone knows what’s going to happen, please let me know. I’d be curious to hear about it.

Michael: It’s interesting because it is a bit of a black art. I think there’s a lot we do know, but the questions haven’t really been asked quite in this context before. There’s a lot of work we have to do to feel comfortable. For example, last night the director of undergraduate studies in my department sent an email to me and the chair saying: hey, we’re thinking of basically replacing all our intro computer science stuff with Claude Code. I was like, wait, wait. It turned out that’s not what she actually meant—but making sure the Claude Code stuff is integrated into what people are doing. They’re using it anyway; we might as well make sure they’re using it intelligently and not just foolishly.

James: The principles of computer science become more important as you use what I would call tools for spec-driven development. I would suggest you could write pretty okay code for certain things without understanding much computer science. The chance you’re going to be able to orchestrate a dozen agents writing code without understanding the constructs of computer science, to me, feels minimal.

Michael: I think that’s exactly right. It’s easy for people to fall into the trap of: oh my gosh, you produced a product with so little effort, this is great; a little more effort and it’ll be a great product. It’s like: no, a lot more effort. It’s going to be a great product, and knowing how to apply that effort is challenging.

James: I would argue the early returns I’m seeing from multiple places is that the cognitive load for agentic development is higher, not lower. We’ve all written code—me probably more in the past than recently—but there’s some code that’s just mindless, really copy-paste and what have you. What we’re hearing from some engineers is the agents are getting rid of all the mindless stuff, and you’re like a bond trader—

Michael: Right.

James: —orchestrating these agents, trying to figure out what they’re doing and trying to keep them productive. It takes a tremendous amount of attention to figure out what these agents are doing and where they might be getting off track.

Michael: I think that’s exactly right. One of the things I’ve heard that supports the argument we’re making is that now, if you get interrupted in the middle of working, it’s so much more painful than it was before. Interrupting is always hard, but now once you’ve got three screens open and you’re orchestrating all these agents—they’re producing different parts of it—and then someone comes in and says “I just have a quick question,” you’ve lost so much context.

James: You’ve been thrown out of flow state is the way I would describe it. Question for you. You were talking about how we don’t really understand how people learn things—which I think is very much a true statement. Are you familiar with the idea of the zone of proximal development?

Michael: Yeah, very much so.

James: I think AI will do a much better job of landing instruction in the zone of proximal development, because it can be tailored.

Michael: That’s exactly right. To the extent these systems can get a feel for where your thinking is, they could potentially design instructional material or explanations that just stretch you a little bit. For people who don’t know about the ZPD—that’s the way the concept is used today, though I don’t think it’s what the original author meant; I’ve read a paper about that specifically—the idea is you want to always be teaching just a little bit outside of what you’ve mastered so far. That’s what keeps things moving forward. It’s hard in a 300-person classroom: everybody’s in a slightly different place. As a lecturer what I’m trying to do is go a little bit beyond the median, try to figure out where the lump is. If I go too far ahead I’ve lost the stragglers; if I go too slowly I’ve bored the advanced people. What AI could do potentially is basically provide a personalized lecturer for each person, which could be amazing.

James: So let me ask maybe a provocative question. Why in God’s name do universities still have lectures? You can watch them on YouTube; they can record your lectures and put them on YouTube. Why still do that?

Michael: It’s a really interesting question. Let me take you back—maybe ten years. In this office where I’m talking to you from right now, I had this giant desk set up with all this recording equipment because MOOCs were going to be the next thing.

James: MOOCs failed. I get it.

Michael: MOOCs failed—well, they did and they didn’t. A massive open online course: the idea is we don’t need a million dynamic, exciting lecturers. We can have rock stars—the same way we don’t need a million people playing bad guitar. We have the ability to take a couple exceptionally talented people and have everybody listen to them. It scales. I like to say the history of EdTech is littered with the bodies of really good ideas, because every time, what seems to be the case is that human beings learn best when they’re motivated by other human beings. It’s not—as much as we want to abstract it away and say it’s basically information transfer, there are all these facts in the world and we have to get them into the person’s head—that’s not a natural thing for people. The only way we know to get people to swallow all that information is to be inspired by somebody.

One of the models people are talking about these days—I think it’s called Alpha School or something like that—is that each student spends some amount of time interacting with personalized AI fact-givers, and then they spend the rest of the day working with peers and mentors and talking to human beings. The balance is crazy: something like 20 minutes or two hours—a shockingly small percentage of the day—getting new material through the computer, and a very large percentage of the day interacting with other people. Is that the perfect blend? I don’t know, but they’re showing some really tantalizing early results. So why lectures? I think lectures are to get in people’s faces—to show a live human being who could actually care about them, presenting the information they need to know.

James: I wasn’t advocating for books—I agree with you. Someone passively sitting back watching a video is not helpful. If I were teaching a class, part of me would say: okay, I would tell the students all the lectures are in this video library, go watch them. I’m going to assign you the lecture the way I would assign a book; make sure to watch it. Then I would say: instead of having lectures, I’m going to break the class up into seminar sessions. Each session will be led by a TA, but I as the professor will rotate through. Or maybe I’ll do tutorials with a smaller number of students, depending on the size of the class. That sounds a little bit like what you’re describing. Because to me a lecture is not getting in someone’s face—you have a lot of students sleeping in the back of the class. I often was the student sleeping.

Michael: I think that’s right. We’re going to continue to experiment with different kinds of models. The agentic programming class we’re teaching this semester we’re calling Agentic Studio, because the structure is modeled after studio classes in art school. We’re teaching computer science the way artists are taught how to create—partly because we’re teaching people how to create and they need that constant feedback from people who have more experience. Lectures are not that. Lectures are kind of a cheap substitute. But this kind of detailed mentoring doesn’t scale very well. If you want face time with a world expert in some topic—French literature or the impacts of city-building on the environment—lectures are the only thing we’ve come up with that gives people a chance to be in the presence of the people having those thoughts and ultimately feel connected to them. I agree with you though. I don’t think it’s perfect. I just know there are plenty of things that sound plausible that actually fail.

James: The one sort of definitional learning experience for me at Brown happened outside the classroom when I ran the Brown Daily Herald. It was like: hey, kids, go figure out how to run a newspaper. The seniors who had just graduated trained you a little bit—though they only knew so much. We had an alumni board of directors who worked for places like the New York Times and the Washington Post, and every so often they’d come in and tell you how stupid you were. They were nicer about it than that, but they were telling us all the things we’d screwed up. It was: go figure it out. Figure out how to structure a staff, how to assign a story and what have you. I wonder if there will need to be even more of that in education—which I think is what you’re saying about the studios going forward.

Michael: I think that makes a ton of sense. I got a question like that this past week in New York City, talking to some alums and business leaders about AI. They said: shouldn’t it be the case that students are given a chance to do projects? That’s what we do at the university. I don’t know what you think we do, but extracurriculars are extracurricular—they’re not part of the curriculum, but they’re absolutely part of the college experience. The people who get the most out of college are the people who really avail themselves of those opportunities. It’s not like we’re not doing it or not offering students those chances.

James: The thing that made the Herald—or other extracurriculars like WBRU—so vivid was that if we screwed up, we heard about it the next morning. Oh boy, did we hear about it. Direct feedback loop. It was a rapid-fire teacher.

Michael: I think that’s a terrific example. My daughter also went to Brown, and her experience like that was directing a musical. There the feedback is partly slower because you do months and months of work and then there are the performances—you don’t really get to see how things land until then. But you’re getting constant feedback from the people you’re working with, because if they’re unhappy you can tell right away.

James: One of the best classes I took at Brown was educational software with Andy van Dam and Ted Sizer—two legends teaching one class. Every group of students had to build a piece of educational software and let students use it; we worked with high school students. It was both terrifying and rewarding to watch high school students use the thing you built.

Michael: Right. You discover that your wonderful pet idea—nobody cares about it—and there’s some stupid throwaway thing that actually draws people in and gets them engaged. There’s no substitute for that. I’ve spent some time trying to become a standup comedian, and there’s this notion that you have to try the joke out. It can be the funniest thing in the world in your head, but it has to land for the audience. Making that part of your process—teachers and instructors do this all the time. Good ones, anyway. There are some who are just pontificating into the void. But the rest of us really do feel the students’ reactions and we’re trying to figure out how to get through and get that positive feedback.

James: And here I was thinking I was the only person trying to be funny about AI.

Michael: I did one routine once that had AI in it. Mostly I talked about being an academic or being a dad. This was pre-pandemic, so AI was not on people’s radar. When I talked about AI, people didn’t think it was funny at all.

James: AI may or may not be funny. Corporate America is endlessly funny.

Let me switch topics to something I’m especially passionate about: digital humanities. I’m a technologist but was an undergraduate history major, and it occurs to me that we can interrogate data in the humanities in a way that would have been unimaginable even two years ago. For example, I took Professor Litchfield’s class on the industrial revolution in early modern England. He talked about how in the sixties people did history from the bottom up—grad students tramping around in dusty archives to record birth and death information. If that’s digitized now, we can interrogate and analyze it at scale. To me that’s an astonishing leap in human understanding of history or literature.

Michael: I think that’s right. History, probably not surprisingly, is going to be a little slow to respond because they have all this context about how things are done and how they used to be done. But I had a meeting last week with the chair of a committee in the history department—what role can AI and information technology play in supporting the discipline and helping people do better history work? He was really excited about it, quite passionate. The image I had from our emails was that he’d be someone freshly graduated, the kind of person asking how to put an interest in video games together with an interest in history. But no—he’s a legit, classic historian. I said to him: what are some of the barriers to switching to this model? He said: I don’t think there are a lot, but when I’m in the library walking through the stacks, alone for six hours with these books—that was the person I was born to be. So the act of doing history, getting into those materials—for some of them that’s what drew them to the field. But even he is recognizing we can do things better and differently. It might not be that every single historian will be digitally enhanced, but the field is definitely taking this on board and trying to find ways to make use of it.

James: How optimistic are you about the willingness of the social sciences and the humanities to embrace this? I’ve spoken to academics at multiple institutions who’ve heard the kind of dialogue we’re having and said: that’s thinking like a scientist—prioritizing quantification over other forms of knowing. That’s not the humanities.

Michael: Right. I think that’s a reaction people have for sure. But it’s a pretty diverse community. Some people will have that reaction; some will have others. Between us, at the end of the day, I feel the most likely outcome is they’ll all be using these tools to great effect and won’t understand why we thought they had a problem with it early on—they’ll say: we were always doing this; this is exactly what we’ve always been doing; I don’t understand what you’re talking about. But it’s going to take a little while for them to wrap their heads around how the use of these tools doesn’t have to subvert what they see as the core intellectual contribution of their work. It can enhance it. It doesn’t have to undermine it.

James: I don’t know whether cynical is the right word, but in a corporate environment, once you can interrogate records from sales calls, it changes the channel a lot because you’re being a lot more systematic about what works and what doesn’t. You can interrogate electronic health records to look at the efficacy of treatment protocols—you’ve changed what being a doctor means quite a lot. I would hypothesize that intellectual history is probably overweighted in the output of departments because it’s logistically easy; you’re not trying to get at data sets that don’t exist. You could see some pretty radical shifts, or some pretty tough fights about what history is—depending on whether we study this set of topics because the data has historically been available, or this other set where we have newly available data.

Michael: The picture you’re painting is really compelling. It could be one of these—what do they call them—Kuhn revolutions. It could change the paradigm: the way they ask questions, what they consider a valid result, what understanding even means in that discipline. It could actually change the discipline. It happened in the sciences—chemistry was a different kind of creature before. The humanities may be a little late to the party, but they may ultimately be impacted in very similar ways. It’s super interesting. That won’t go down easy—people will fight it kicking and screaming—but ultimately it’s them as a community who have to work through it. The hope is there’s at least some objective measure by which the field can decide: this is actually better. I don’t know exactly what the metric is, but if we’re actually doing better history than we were, we should do that. I think that’s what the fight is ultimately going to be about: what is our measure of quality, and are we improving in that dimension? Otherwise it’ll just branch off as a different field. There could be a different field with different objectives. But if it’s going to stay one field, that whole field is going to have to feel its way to a new way of thinking.

James: We’re going to have to do a follow-up—we have about three minutes left and there are whole questions of epistemology, institutional imperatives, the availability of new forms of data. What do knowledge graphs mean for the capture and management of data? So let’s declare this the end of part one and turn it back to you. What would you say to a young computer science graduate thinking about their career over the next 10 years?

Michael: Oh wow. One of the things we’re starting to speculate about is that what it means to be a successful computer scientist may actually change. It could be that people who are not well suited to this discipline are actually who we need right now, and the people who’ve been traditionally good at it—maybe that skill just isn’t needed anymore. That’s a hard thing for someone like me to think, who’s dedicated his entire life to this one subject area. Maybe it’s not about me anymore. Maybe my successors are going to be very different from me.

James: Maybe we even think about the idea of a discipline very differently.

Michael: Yeah. In many ways I feel Brown is an interesting place to be having those discussions, because we’re a little less bound by disciplinary boundaries than a lot of places. Those boundaries exist and help us organize our thinking, but we’ll walk over them if necessary. There’s a wonderful course being taught right now on the history of artificial intelligence—the historian knows a ton about the technology and what it’s done and what it’s meant. That’s a high-level thought from my perspective. What would I say to a young graduate? I spend more time talking to my fellow faculty. What I try to encourage people to do is embrace this moment. It’s forcing us to revisit some very long-held and unquestioned assumptions. It’s painful, but we should do it. We get to be the people at the revolutionary boundary. Play your role. Do your thing. Help us figure out where we’re headed. It’s exciting, but it’s a big lift—a cognitive load for all of us. Some of us just want to keep doing the thing we’ve been doing for 30 years, and it’s just not okay anymore. Let’s be excited by that instead of depressed by it.

James: You could argue the biggest distinction among people in probably every field—academia, business, or anything else—will be between those who are intrigued by revisiting old assumptions and those who are scared or resistant to it. This was a blast. Thank you so much for doing this. We’re going to have to do a follow-up because there’s a whole bunch we didn’t get to. I hope you enjoyed this as much as I have.

Michael: I always have a great time speaking with you, and this forum is a great way to do it. Thank you so much for thinking of me. Thanks.

Thanks for reading Prosaic Times — subscribe to get every issue!

< 1 > Despite what every says, I sort of like Canberra, even I got some sort of bug there. Being under the weather seventeen time zones from home is not wise.

< 2 > Many corporate executives see two mechanisms from using large language models: bottom up innovation with front-line employees using chat interfaces to enhance their work and building agentic workflows (or workflows using agentic software engineering)

< 3 > I worry about the perspective in LLM chat as bottom up innovation.

* We know that context is everything, so you don’t want to have to depend on the user remembering to provide all the right context in every prompt

* Naturally, copy-paste is a poor mechanism for agentic learning

* So chat-based LLM usage quickly devolves to a combination of fancy web search, summarize this document (which probably didn’t need to exist in the first place) and write me a email (which possibly didn’t need to exist either). Thin gruel.

< 4 > All the exciting things people are doing with Cursor, Claude Code, Obsidian and other tools? That’s the real innovation -- and it reminds me of the ferment we saw in the PC revolution in the 1980s and early 1990s.

< 5 > Of we need agentic software engineering will be important. Very few people, though, could answer this question when I posed it: if you could develop software functionality at half the cost you could today, what would you build. This will become an existential question for many companies.

I’m so excited about this interview with neo4j CTO Philip Rathle because context is everything in the age of AI, making the architectural synergies between AI and knowledge graphs very important.

* At first, I thought: wow, you could use GenAI to populate a knowledge graph

* Then my colleague Zubin Ghafari showed me how you could use GenAI to accelerate ontology definition, dramatically reducing time-to-value for knowledge graphs

* And finally: a graph is the ultimate context repository because of its ability to depict a dense network of relationships

Thanks for reading Prosaic Times — share it with a friend!

Everything is a graph — a discussion with Neo4j CTO Philip Rathle

What you need to know: Knowledge graphs treat relationships as insight, not overhead. They complement vector search: vectors get you to relevant chunks; graphs give you deterministic, explainable answers and deep context.

What you need to do: Identify where your data is inherently a graph—multi-level hierarchies, supply chains, B2B CRM, digital twins, CMDB-style infrastructure — and start small: let the model evolve as your needs evolve.

What you need to decide: Which problem is the right first use case for a minimum viable graph; and how much to formalize your ontology upfront versus letting it emerge with human-in-the-loop curation.

James Kaplan: Hi there. This is James Kaplan from McKinsey, and this is another episode of the Prosaic Times podcast. Greetings from New York City, where it’s cold and miserable and the frozen snow mixed with truck exhaust is still on the sidewalks. Fortunately I’ll be in Australia next week. We’re expecting warmer. I’m thrilled that Philip Rathle, the CTO of Neo4j, the knowledge-graph database company, is with us. Philip, thank you for joining us.

Philip Rathle: Pleasure to be here, James.

A background in databases

James Kaplan: Alright, so why don’t we start out with a little bit of background. Do you mind just telling us a bit about your journey? I recall from our email exchanges there may have been a Commodore 64 somewhere deep in the background. We’d love to hear about that as well.

Philip Rathle: I certainly started dabbling with computers probably the same time you did—mid-eighties or so.

James Kaplan: We’ve got the New Wave soundtrack playing in the background.

Philip Rathle: Yeah, something like that. And then if I fast forward past college—after lots of computer dabbling—I ended up studying chemical engineering, ’cause I was really into physics, math, chemistry, and just hard challenges and quantitative stuff.

James Kaplan: That’s closer to what we’re doing today than being a history major as I was.

Philip Rathle: Okay. Exactly. So from there I fell straight into software—working at Accenture, which was then Andersen Consulting—and really pretty quickly got into databases. I kind of fell in love with the notion that I can represent a business with data and then manipulate business outcomes with data. Of course we do that in a much more sophisticated way today than we did in the mid-nineties when I actually started doing this.

James Kaplan: Yeah. When you got started on SQL databases in the mid-1990s, we should remember how relatively novel that was. I think the first commercially available SQL database was, what, late seventies—about 1980 or so. So you’re talking about a world that was 15 years old.

Philip Rathle: So organizing data was really interesting to me. It’s a set of rules, much like a geometric proof.

I got into data modeling pretty early on, then got into a bunch of programming around data—data-warehouse kind of work. And then I had this eye-opening experience where I had built an application that functioned perfectly on paper, and when you started running tens of millions of call detail records through it for a telephony application—

James Kaplan: Call detail records. Kids today don’t know CDRs. I suppose wireless telecom companies still have CDRs.

Philip Rathle: I realized that it was possible to build programs that looked beautiful and functional and actually worked and passed unit and system test—and then you get into the physics of the next level of data: how do I treat it and manipulate it? Anyhow, I’ll just fast forward past 20 years of that. Lots of work with high-performance relational database systems. Then I encountered the graph way of doing things.

Emil Eifrem, Neo4j’s founder and CEO, and I met in late 2011. He had this really interesting idea that was very counter to the trends in databases at the time—at the time it was “let’s simplify the data model and remove things like pesky relationships, which get in the way when you’re trying to store and retrieve at scale to support e-commerce.” He was saying no, actually those things are pretty valuable; I’m building an entire database company around it. So there was an interesting combination of something I really believed in—the model is something I always saw as really valuable. Relationships, data context, causality are things that emerge once you understand connections—and the opportunity to help pioneer and grow a new category in databases.

Knowledge graphs treat relationships as insights, not overhead

James Kaplan: I was wondering if you could just give us a brief definition of a knowledge graph.

Philip Rathle: It’s a graph-based structure—nodes, relationships—that represents and reflects some part of the world that I care about, be it real or digital. So think of it as a digital twin; that term is often used to describe these kinds of things, and you may have heard it more in a manufacturing context—physical objects could be a digital twin of part of your enterprise. It’s taking a real-world system or a digital-world system that shows up naturally as a network—computers, biology, ecology, payments, et cetera—or as some kind of hierarchy: reporting structures, HR ownership structures, financial services, et cetera. And just representing those in their natural way.

So I look at a knowledge graph as a logical description, and then I have a choice: Do I put my knowledge graph into a database that is a graph database, naturally designed to take that data and represent it, query it, evolve it as a graph? But I can also take a knowledge graph and put it anywhere—in a spreadsheet, in tables. The important question is, where do you put your graphs? Arguably some of them already live in relational databases, but you probably have a lot of limitations in how you can use them, because the distance between the representation as a graph and the representation in tabular form is quite great. If I’m trying to understand context several levels out, or causality multiple levels out, the ability of that system in tabular form ends up being limited.

James Kaplan: When people ask me why I think knowledge graphs are interesting, the reasons I tend to give are: one, it almost mirrors what somebody would draw on a whiteboard—even a business user. It’s intuitive for many types of data. Second, the relationship is a first-class citizen of the database, which makes it much easier to model domains where there are many types of relationships in many directions. And finally, the fact that the relationship is a first-class citizen means you can model causality over multiple steps in the chain. It’s much easier to model, for example, counterparty risk via a graph than in a traditional database.

Philip Rathle: I love all of those, and I a hundred percent agree. We even used the term “whiteboard friendly” early on to describe the fact that the way a business person would model their domain is using circles and lines—which is a graph.

I have a colleague, Ajay Singh, who recently came in after years of working at Databricks. He made an observation about the category: many people assume they know something about graphs—that they’re niche, they’re hard to learn, and they don’t scale. What he’s discovered, first of all, is they’re everywhere. We’re all graph-database end users. Number two, they’re much easier to learn than a relational database. If you haven’t yet learned to think in terms of a relational database, graphs are how we think. If you’ve learned to think as a relational database, it’s pretty daunting at first—it was for me too. You kind of have to unlearn and come back to your original way of thinking. He said that for a business person, you show them a graph visualization and they’ll leap up and go grab their colleagues and say, “Wow, for the first time I can really see my domain.” And then they can take the next steps and ask more complex questions, like you said—multi-level causality, deeper context and influence. So it’s easier to learn. Of course there’s a little activation energy to unlearn whenever you learn a new skill. But my experience is that once a technologist starts working with graphs, it just becomes so natural, and your projects tend to go faster because you don’t have to deal with all the ORM and the impedance mismatch.

James Kaplan: I wonder if you could comment a little bit on the history of graph databases. I may be one of the few people in the world who’s interested in databases and interested in history, but, you had this sort of boomlet, in the two thousands and a lot of interest in Owl and then it didn’t quite work out.

I was wondering if you had a view on what the evolution of the interest in graph databases was?

Philip Rathle: So relational obviously became an ISO standard pretty early—SQL 86. Prior to that you actually had hierarchical databases, IMS, and—

James Kaplan: Which was a good thing to do if your processors basically ran on hamsters or something. A hierarchical database is a little less processor-intensive than a relational database.

Philip Rathle: But something interesting about them is they represent hierarchies in a more natural way. Of course the way it was done is very different from what’s done today with graph databases.

James Kaplan: As someone who’s tried to represent knowledge hierarchies and relational databases, yes, I hear what you’re saying.

Two original stories for graph databases

Philip Rathle: That’s right. Graph databases came out, and there are kind of two origin stories. One started with the worldwide web and Tim Berners-Lee’s idea: let’s annotate the worldwide web and have more than just HTML; let’s add—I’m gonna use the word “context” to mean many different things in this conversation—

James Kaplan: We need context on context. We need meta context.

Philip Rathle: That’s right.

James Kaplan: Oh, come on. That was funny.

Philip Rathle: That actually emerged as W3C.

James Kaplan: And that’s the whole semantic web—W3C.

Philip Rathle: That was the semantic web.

James Kaplan: So a lot of that collapsed under its own weight. I just don’t think the world was ready for it at that point. Is that an unfair statement?

Philip Rathle: There’s still some degree of tagging—schema.org is heavily used. But you’re right, it’s not used in context that much. It’s become a fairly specialized technology. I’ll come back to where it’s shown up. There’s been a thread on the database-management side.

The other origin story came from Neo4j founder Emil Eifrem. He had a literal napkin moment on a flight to Mumbai from Sweden, working through a content management system and dealing with the confluence of many different hierarchies intertwining.

So one, it was a hierarchy of content. This was an application that had, digitized and brought online a number of photo CDs that this company licensed. So you have the hierarchy of different, photos and how they’re structured into CDs and collections. You have another hierarchy of like this photo, the ontology, like the hierarchy of meaning on top of it.

And then you had users and groups and permissions and then, relationships across these things. And he’d been using Postgres in the background and realized 90% of his time was actually spent doing effectively, like object to relational mapping or graph to relational mapping.

And he said, the famous words, how hard can it be?

Let me just invent my own database that actually treats the data. In a way that looks more like my domain, so I don’t have this giant impedance mismatch.

There the property graph was born. We added labels a few years later in 2013, for reasons we can or can’t get into. But the idea is you have nodes—nodes can have any number of key-value pairs, which are properties. You have relationships that have a type and a direction, but relationships can also have properties.

James Kaplan: Which I absolutely love. Not only that we know each other, but how well do we know each other, right?

Philip Rathle: And how did the relationship start? You typically have fewer properties on relationships than on nodes, but these can be incredibly powerful. In lots of domains like Customer 360, there’s a whole set of data where you suspect things but don’t know them a hundred percent. But there are certain kinds of operations where you want to use that information. Having that waiting in the relationships, and having different types to denote it, can be very powerful. So that ended up being the core design—let’s solve enterprise database kinds of problems. What happened in parallel in the RDF sphere—sorry, I don’t think I’d used the term RDF yet—the semantic web—

James Kaplan: Well, before we come back to RDF versus LPG, which I very much want to get into—where does the famous Google blog post “Things, not strings” fit into all this?

Philip Rathle: So Google, things not strings was 2012.

Trillion-dollar graph companies already exist.

Philip Rathle: There’s been, some talk and blogging and you had a guest recently talking about context graphs as the next trillion dollar opportunity.

There is already a a trillion dollar graph company and that’s Google.

James Kaplan: There’s multiple trillion dollar graph companies.

Philip Rathle: I cannot disagree with that. Facebook is certainly one. LinkedIn is certainly

James Kaplan: And then—people sometimes ask, are graphs real? Can they scale? When people have asked me to explain a knowledge graph I say: you are a knowledge graph user. If you use LinkedIn, Facebook, Google, Wikipedia, you’re a knowledge graph user.

Philip Rathle: A hundred percent. So Google’s first, foray into web search, was in, I think 2 22 thou, 2000 or 2000, 2001, 2002, somewhere around

they, th they not only had the world’s largest index of the world wide web, that that wasn’t particularly differentiated. You had 30 other companies doing this.

James Kaplan: Alta Vista. Remember Alt Vista?

Philip Rathle: I Sure do. And of course, Yahoo and, web crawler and Lycos and so on and so

James Kaplan: Oh, oh my God. It’s old Home Week.

Philip Rathle: We’re just dating ourselves.

James Kaplan: Exactly. I, I was on a panel last night and, someone on the panel described having an iPad as a child. I was 40 right when the iPad came out, I think, or maybe a maybe or, or almost 40.

Philip Rathle: What initially made Google so successful was one graph algorithm—PageRank—a centrality algorithm. They solved the problem that you can easily gamify websites: if your ranking is based on the number of times a word appears on the page, you just repeat that word a gazillion times, white on white background, one-point font. That’s what people were doing. Google cut through that noise by saying, let’s rank webpages based on the number of inbound links, and let’s do recursive algorithms so you can’t gamify it—weight the inbound links based on the number of inbound links from the referring page. That took them about a decade.

After about a decade they acquired a company called Metaweb in 2010—which had built a knowledge graph. What does “knowledge graph” mean in the Google context? Take the worldwide web and the things that show up; put them into a graph so you have entities, and for each entity something that describes its meaning. So you search for “Rio.” If you don’t know the intent, you could bring back Rio the city, Rio the hotel casino—

James Kaplan: Duran Duran in the background—we had Commodore 64s.

Philip Rathle: There you go. So today when you use Google you’ll oftentimes get this pane on the right. They introduced that in 2012 with “Things, not strings”—a brilliant blog post that says it’s one thing to return text, another to return meaning. That knowledge-graph pane is actually a graph visualization of that node, where each blue link is a link to an adjacent node through a relationship—like “capital of” Albany. That catapulted them for another decade; they disrupted themselves, no one could keep up. They did it again in 2024.

They seem to do this in 12-year cycles—they described using LLMs but pulling data back from the knowledge graph. There’s a term for that: graph RAG—retrieval-augmented generation by pulling in information from the graph and using it in one of multiple ways. There are various flavors of graph RAG. So Google is a trillion-dollar graph company. “Things, not strings” I think is particularly relevant now, where people are trying vector search.

With vectors there’s a relative meaning—if I have word embeddings I can test the distance between these two and these two and see which is closer. But nowhere in that is any information about what the thing actually means in any human-understandable way. So it’s powerful, but you very quickly hit limitations.

Vector search and knowledge graphs as complements

James Kaplan: I was wondering if you could speak about the trade-offs and the synergies between vector search and graph databases. Vector doesn’t necessarily have meaning at very high scales; it can sort of collapse on itself. I especially like the scenario where you use vector search to populate a graph and then do combined deterministic and stochastic search on the graph. Could you comment on when to use vector search versus when to search a graph?

Philip Rathle: The first thing I’ll say is I don’t view these as mutually exclusive.

James Kaplan: It’s always a continuum. There’s always complementarity.

Philip Rathle: Vector started out as a specialized kind of database—really one index operation on one data type, one particular transformation. Now it’s in all databases; you don’t need a special database for it, including graph databases and the one I work on. Where vectors become powerful is when I have unstructured text and I’m trying to use it to inform an AI operation—specifically to inform an LLM. I ask a question and want to find a similar question that’s been asked in the past. You can find a similar chunk, map to that answer, bring back the text around it to the model and get a better answer. What you don’t get: it’s not normally used for structured data—it’s really an unstructured technique. You don’t really have fine-grained access controls, which are important for discernment and preventing certain data from being misused. Maybe you have it at a super coarse-grained level—

James Kaplan: I love the idea of having access control per node.

Philip Rathle: And per relationship.

James Kaplan: You may or may not want people to know that you and I know each other.

Philip Rathle: Or you may want people to know there’s a connection but not the nature of it. There’s actually a special permission called Traverse for that. So there’s real fine granularity once you have your knowledge graph in a graph database supporting these permissions. Other things: you can’t look at a vector and make sense of it—it’s just numbers. You could say it’s understandable by a machine; I’d argue it’s not, because what the machine “understands” is distance between vectors. There’s no meaning there; it’s just a discrete thing. So if I’m trying to understand supply chain, it’ll bring back the one thing that refers to supply chain, but I can’t store my supply chain in that format. I can’t store my employee hierarchy. These things I just brought up are where graphs come in. What we’ve found: when we have both vectors and graph—same database or different databases referencing each other—vector indexing is a handy trick for landing into a chunk. Then you can do entity extraction from the chunk to see what parts of the domain it refers to, and traverse from one world to the other. “Here’s a chunk of text—now I’m gonna go into the domain, pull back the supply chain, use that for ranking the vectors.” You might run PageRank, use it for filtering what’s brought back, use it for bringing back several levels of context around this particular thing. Another way to use the knowledge graph: have the AI system delegate certain questions to the knowledge graph—run a deterministic operation. A supply-chain question probably has an exact answer. Ultimate beneficial owner in finance—exact answer. And that answer might be 20 levels deep. You’re not gonna get there with an LLM or with vectors. But it’s easy to have the LLM translate the question into a graph query, delegate to the graph. It’s almost like the right hemisphere of the brain delegating to the left: go figure this one out. Pass it back, and you get an exact, explainable answer. So you get determinism in your AI system, which is pretty cool.

James Kaplan: I mean, you bring up a great example, which is, sort of beneficial owners, legal structures or legal entity structures and financial services, which is all sorts of fun to deal with. If we to, if we were to zoom out a bit, what do you think are like the four or five most exciting business use cases for graph databases over the next few years?

What are the things you say? This is where you see people getting a lot of value

Philip Rathle: I see a lot. This is really a horizontal phenomenon. One—let me start with an AI use case that hasn’t been so much in the headlines but I see a lot—is democratizing certain kinds of complex data operations that were accessible only to data scientists who knew how to query and pull the data together. I’ve seen this at companies like Uber, Comcast Business, Walmart. They’re taking complex questions that normally a business expert who understands a database query language has to go and ask—in that case Cypher slash GQL, the industry standard for graph database querying. You end up with one or two or maybe five top people in that department who understand that query language and have business domain expertise and can iteratively ask complex questions. When you say, let’s democratize it through an LLM, with a SQL database on the backend, the questions very quickly get intricate—next thing you know you’ve got 200-line SQL queries with 15 joins and 10 levels of recursion. I see far worse every day. Those queries might take half a day on million-dollar hardware. You put that into a graph database on modest hardware and they’ll run in subsecond.

Knowledge graphs for dynamic modeling

James Kaplan: What’s the intersection between graph databases and dynamic models,. if you think about many supply chains, many commercial markets, they’re graphs, right? You have customer, you have vendors, you have customers, you have assets, you have products, and there’s relationships between these things.

Are you seeing people doing dynamic modeling or doing complex modeling using graphs at all?

Philip Rathle: Yes, in two ways. It has to do with separating the storage layer and the reasoning layer. What are you modeling? The real world or the digital world. What makes it graph-worthy—where a graph database will add value? If it’s a simple model, you’re probably fine with Excel or basic SQL. Where it becomes valuable to put it into a graph is when you have data that shows up in the real world as a multi-level hierarchy or more. And of course some of these aren’t just hierarchies. HR: you have your direct reporting structure, but also communities of practice, mentors, dotted lines to projects—all of that changes over time. Each person has skills you can tag into the graph. People are in teams; you can include team composition relative to project. So anytime I have something like that—and likewise a digital twin of a car, of an aircraft, parts and parts of parts—

James Kaplan: and it would seem to me, doing a digital twin in a relational database seems like a painful thing to do, although I’m sure people do it.

Philip Rathle: We had an aircraft manufacturer going back over a decade that had an application that they would give to their clients to, essentially spec out their airplane. each time you choose one particular thing, like I want a screen in the seat back that would. all of these dependencies. And the dependency calculation would take like a day to run.

Each time you just added one thing and then you put that in a graph database and all of a sudden it’s running like that. So there’s lots of that. Essentially any data that shows up in the real world as a network or a hierarchy where you want to do a path or a journey—

Customer journey, patient journey. That kind of model ends up being perfect for storing in the graph. So now I can store my data structure in the graph, and I have the choice of where to reason about it.

James Kaplan: And what does reasoning about the data structure look like? Give me an example.

Philip Rathle: A simple one: I change a light bulb. I want to use this new kind of light bulb in a plane because I’ve deprecated the old kind. Maybe that changes the threading, which changes the housing, which changes the weights and balances. You have all these cascading dependencies and the stakes are incredibly high—you can’t get any of this wrong. Another example with automobiles, a real one from Volvo: they have the door system and the key fob. They want to do research and say, what other systems are impacted by the key fob? What a new person may not realize is if you hard-press the key fob, the windows roll up—so the window system is tied to the key fob, but you wouldn’t know that without a digital twin. They use a graph for the dependencies and relationships between this subsystem and that subsystem and this functionality—and to the person working on it—so I can go talk to them and make sure the work I’m doing doesn’t negatively impact their system. And inside computer and telephony networks it’s things like, what are all the attack vectors? Red-teaming,

James Kaplan: cybersecurity, at least in the enterprise. Cybersecurity was an early adopter of graphs,

Philip Rathle: And there are a lot of cybersecurity companies that embed graph databases inside of their, inside of their offerings. There are also a lot of cybersecurity activities that aren’t yet covered by off the shelf software. And particularly like banks go off and build their own, adjunct cybersecurity solutions to close those gaps using graphs.

Tech environments, corporations, file systems -- all graphs

James Kaplan: One of the reasons a CMDB—a configuration management database—is often so painful is that trying to model a modern technology environment in a relational database borders on insanity. Any modern environment where you have applications that support business processes, applications that run on infrastructure, applications that consume data, infrastructure that consumes other infrastructure—it’s just naturally a graph.

Philip Rathle: This is a great example. We have a public case study with British Telecom. BT has a CMDB, an off-the-shelf one they’ve been using for years. A bunch of stuff runs in it; it does certain things well. It doesn’t do dependency analysis or network planning well—and those are pretty key. So they have a mirror image where the graph feeds on that; it’s kept up to date and enables a lot more capability. Back to your modeling and reasoning question. There are a few kinds of reasoning. One is simply connect the dots—sounds basic, but connect the dots across 20 levels. You can do things that seem like magic. Almost every pharma company we’re involved with does drug discovery by connecting the dots between a hundred thousand pharmaceutical compound candidates at one end and, at the other end, everything that’s known about some kind of ailment and how it interacts with the body—just connect the dots across the body in between. So you could look at this as connect-the-dots graph reasoning; no one’s really come up with the term, so I’m doing it myself. What you could also do: if I want an LLM to be a good sparring partner or an agent to come up with its own hypotheses, dip into the graph, pull back something four, five, ten levels out, hand that back to the model as sentences. Node–relationship–node can easily be translated into English; feed that as a giant piece of context or a giant prompt. Now I’ve got an agent that can riff because it understands more about that particular domain at that point in time—stripped of what’s appropriate for that use. Say I have a firewall between investment banking and corporate banking.

James Kaplan: Another couple of examples: it’s a natural fit for modeling an organization, particularly when we don’t have entirely rigid hierarchies. Once you move to Joe reports to Bob who reports to Sally who reports to Sue, and you have project teams, people working across organizations, matrix reports—that begins to feel much more like a graph than the kind of simple hierarchy you can easily model in a relational database.

Philip Rathle: This is very true. You have companies like Daimler that when they spin up a new team actually refer to a graph to include all the things you described—where people are physically located, time zones, team members who are long-term experts in a particular domain, some people from outside the company to bring new perspectives, some who have worked together and some who haven’t. You do that with a graph. If listeners want to tune in, Walmart has a graph-based application available to all employees for career-journey analysis for exactly this reason. So graphs are showing up—on any given day you’re using graph technology in the background in more ways than meet the eye.

James Kaplan: And then B2B CRM. B2C CRM is a bit simpler—more of a hub-and-spoke model. I know a cable provider that has a relationship with a large number of atomistic households. But once you get to B2B sales—capital markets, group health insurance—you have multiple entities in the customer, multiple people in the selling organization, relationships that evolve over time, multiple products. That feels like a natural graph.

Philip Rathle: So funny enough, I don’t think I’ve talked to you about this, James. I co-founded a startup in the early 2000s after being involved in building United Airlines’ first Customer 360 system—focused on passengers. We went out and talked to other airlines and found the top problem they wanted to solve was what you said: not passenger, corporate—the mapping of their sales rep to some level in some giant corporation, some big hierarchy. We had a standard out-of-the-box relational model; we did our best and ended up punting because every company has a different number of hierarchies and we couldn’t generalize. It was super hard. Fast forward to when I started at Neo4j—we already had four or five customers. The very first was Cisco, and their use case was exactly that. They had a system, still running, called the Hierarchy Management Platform, initially to deal with the fact that Cisco acquires a new company every month. Each company has salespeople that attach into a sales hierarchy, products into a product hierarchy, and territory mappings. You need this super complex, multi-level calculation across multiple hierarchies to figure out the commission trail, ownership, and exception assignment—this person has this territory but this one other account in some other territory because they have a personal relationship. You can’t even model some of that in a relational database. So that’s a pitch-perfect use case. And then you overlay AI on top—agents with access to these data structures. They interact with them pretty well, because they’ve been trained on a decade plus of graph modeling and graph language. You end up being able to solve these complex problems much more naturally than with a relational database.

James Kaplan: A file system is in many respects a graph. We all struggle with finding the right document at the right time—maybe I’m describing knowledge management. I’ve talked about the Obsidian-Cursor stack I built to manage my own professional life. You could argue the way we interact with documents is in many respects a graph.

Philip Rathle: And the way we interact with ideas. Mind maps have been around for a while—that’s a form of graph. There’s a note-taking company called Roam Research that’s been out for a while and recognizes that we think in networks; it’s a network-based note-taking tool that takes mind maps to the next level. I keep waiting for Drive or some equivalent to incorporate graph structures. When you have data in a graph structure you can run centrality—what’s the most relevant document—filtered by recency, by people in my function. It lends itself to much more powerful search. In this world of AI we have a lot of powerful search capability; a lot of off-the-shelf products are primarily vector-based. Some use the term “knowledge graph” to describe going maybe one level, generously, with the graph, but they’re not fundamentally structured to allow the rich exploration you’re describing. So companies are building their own. No doubt we’ll see off-the-shelf graph-native options to solve the search problem we’ve had for years.

James Kaplan: Where do you, let’s just pivot a little bit. What does ontology mean to you and how important is ontology and are you one of the people if there’s a continuum between those people who think that you should have the ontology nailed down upfront, and those people who think that the an ontology should be emergent, where do you land on that continuum?

Philip Rathle: It’s funny. For the longest time, it was a piece of jargon that I tried to get our teams to avoid using because we like to use friendly terms like node in relationship instead and Edge,

Like nobody needs more terms and more complexity. Having said that, like, I credit to Alex Karp for having popularized that term. , It’s a super important for companies to have a handle on inside of their data systems.

You’ve got three kinds of graphs.

One kind is metadata: how all these different systems hook together—provenance, lineage, definitions. There’s no actual data in that. Then you have your actual data, maybe two levels down. Your ontology is a definitional structure of the things available in your domain—what’s the structure. It’s super abstract. Let me give you an example. Say I’m IKEA. IKEA has a product hierarchy and a product catalog, and there’s a hierarchy of meaning associated with that: furniture—underneath that outdoor and indoor, home versus business—tables and chairs that fit in one category or the other, and a hierarchy of brands, colors, collections, and so on. So your ontology describes the unique set, grouped as a hierarchy, of what the things in your domain are. Oftentimes it’s multiple overlapping hierarchies. Usually the first thing you do with graph data is bring in an ontology from somewhere. These things are usually already lying around—it sounds like a mysterious new term, but you probably already have a database, probably relational, reflected in many places: a product master, your product catalog. Bring that in so you can use it when making a product recommendation—“I’ve got three things in my shopping cart, what are they trying to do?” To understand that you need to understand what kinds of things they are and how those things go together. That’s a collaboration between your ontology and your graph of what’s been bought and what’s in the cart. There are some big ontologies out there—eBay has had a billion products in an ontology in a graph with us for many years. In the rare cases where you don’t have ontologies in structured form, you can often infer them from unstructured documents using unstructured-to-graph construction—for example, when you’re trying to understand a competitor’s ontology.

James Kaplan: Ontology creation process can be very strategic. It’s actually quite a good thing for a management team to have an explicit discussion about what they mean by a product or what they mean by a customer, or what they mean by a service offering and how those things relate to one another.

Philip Rathle: That’s right. I think there’s an observed reluctance to hire people or have people spend their cycles managing ontologies. I’m of two minds. One: there’s a lot that can be automated and a lot of work already done—so there’s probably a lot you can do with what you already have. The other piece of good news is you can do a lot with agents riffing on each other to improve it. The third is there’s a lot of cool gamification. I had lunch with my brother-in-law, who’s a doctor; he said he often gets garbage recommendations from the AI system but then has an option to annotate—what was wrong with this? That’s feedback, a way to crowdsource ontology creation, kind of like Wikipedia but more in-stream. Having said all of that, these things are so high-value—as you point out—that it’s worth having some small number of people spend time making sure they’re correct, especially when your AI comes back and says “I’m 80% sure but not totally sure.” In the same way that foundation-model companies, whose entire business depends on high-quality data, use the likes of Scale AI and lots of people to do RLHF—reinforcement learning with human feedback—and other annotation. There is some value in doing some of that in-house, as much as I know that’s not a popular thing to hear, and in addressing the long tail of the ontology. But you don’t necessarily need to start there.

Is context graph a form of knowledge graph?

James Kaplan: So what, what is a context graph? How is that similar or different from a knowledge graph?

Philip Rathle: That word’s being used in a lot of different ways. You had the Foundation Capital blog post—I think that’s one legit and powerful definition: your decision traces and the graph that comes out of that. What’s a decision trace? Information about the actor and what’s being acted upon, and all the people in the approval process for that decision—so that if you pull the thread you have information about why a decision was made. I have my actor and the thing being acted upon. Then, if we’re talking ontology, I need to know what that thing is. So now I have my ontology. “Semantic context” is often used to describe the metadata graph—these terms get recycled a lot.

I might pull data back at a metadata layer, then some purchase history around the person who carried out that action—now I’m pulling back a graph of what’s been bought, how it relates, what’s been returned, complained about. Something challenging about that definition is you end up pulling in everything. What’s reflective of the world I’ve been living in, having worked with graphs for close to 14 years, is that this is the nature of the graph: you pull the thread and suddenly you have the entire world. We say “digital twin of a—” What’s good about it, or what I recommend: don’t get caught up or scared off by the idea that you need to boil the ocean and have a graph of everything before you get any value.

That’s very much not the case. You can start small, get value in a very short time by deciding what particular problem you’re trying to solve and what’s the minimum viable graph around that. Oftentimes it’s fairly small; you can start at a departmental level. What’s great with a graph database—and we should disambiguate: there’s context graph, knowledge graph, graph database (the technology you put your knowledge graphs and context graphs in, purpose-built for that)—you have schema-flexible implementation.

You can add more data without a schema migration; your model can evolve; you can add to your graph and evolve your use case and have an asset you build on and reuse rather than a one-off you have to recreate from scratch.

James Kaplan: Maybe this is overly simplistic, but I see a context graph as another instantiation of a knowledge graph—a form of knowledge graph that focuses on decisions.

Philip Rathle: That’s right. It’s either that or synonymous with knowledge graph—I could argue both. At the end of the day, what we call them is maybe less important than understanding how this stuff can be useful.

James Kaplan: Thank you so much. This has been wonderful.

Philip Rathle: It’s been a pleasure, James.

More discussions with deep thinkers on enterprise tech on the way — subscribe to Prosaic Times!

A couple of years ago I attended an event where a former anti-trust policymaker spoke at Brown University. In making the case for breaking up big tech, he said markets had never disrupted a technology monopoly and ascribed American technology dominance to antitrust actions in the 1970s.

While I think the later 1970s and early 1980s are an especially fascinating era in modern American economic history, I challenged the point. I got up and asked, “Is it possible that the structure of American capital markets contributed to the success of the American technology sector?”

The speaker looked at me confusedly.

I continued: “For example, we have a much more robust venture capital ecosystem here in the United States. Is it possible that the ability of technology disruptors to get funding helped the American technology sector succeed?”

I didn’t really get an answer, so I sat down.

Fast-forward to the current day. I sat on a panel with several young start-up founders. And by young I mean 20, not 35. Very inspiring. I saw next to Teddy Solomon. Teddy is 23, and co-founded social media app Fizz when was 19. Fizz has received more than USD 40MM in venture funding.

The venture sector’s willingness to fund young entrepreneurs, outsiders and disruptors contributes mightily to technology innovation and to American economic success. Boldstart ventures, which Ed Sim leads, invests in the domains I am most passionate about -- enterprise infrastructure and cybersecurity.

Three takeaways for me from the discussion

< 1 > Venture capitalists don’t just allocate capital -- they are mentors and coaches who help entrepreneurs with disruptive ideas build companies. (Though Ed cautions us that not every VC engages with portfolio companies in a helpful way.)

< 2 > Procurement and other corporate governance function are not preventing enterprise tech organizations from doing business with disruptive firms. Some of the larger firms are explicitly investing in technology innovation teams to engage with the startup community.

< 3 > Everything is going to start moving more quickly, but platforms will be critical. Large companies are getting serious about AI-enabled development, but they will need the platforms that provide guardrails and enable resiliency, security and performance at scale.

Thanks for reading Prosaic Times — share it with a friend!

James Kaplan: Alright, can you hear me, Ed?

Ed Sim: I can, James,

James Kaplan: Excellent. Alright. Welcome, Ed Sim of Boldstart, to the Prosaic Times podcast.

Ed Sim: I’ve been loving your newsletter.

James Kaplan: I’m glad. Why don’t you introduce yourself — tell us a little bit about yourself, and particularly your journey to becoming a venture capitalist.

Ed Sim: My name is Ed Sim. I’m the founder and general partner of Boldstart Ventures. we started that, or I started that back in 2010. So I’m on year 16

James Kaplan: Congratulations.

Ed Sim: Yeah, thank you. I mean, year 30 being an enterprise venture capitalist, believe it or not. So I started back in in the mid-nineties, and as I like to say, being the partner to founders from the very beginning, from the inception, and what that is has changed over time.

But that’s what I’ve done and I’ve seen a lot of trends and seen a lot of different things.

James Kaplan: I was on on a panel yesterday with a couple of founders who were between 18 and 23, so I’m just real stoked to be talking to somebody who remembers the mid 1990s.

Ed Sim: I got into it because I actually learned about venture capital when I was in college. I played lacrosse at Harvard, and someone on the team — their brother was a venture capitalist who came in and spoke one day.

I was enabled with the idea of, of like, you could either be a founder or you could actually back multiple founders, having your eggs in one basket or in a few, but then also having the ability to be a player coach. And really the idea was helping founders accelerate the process of, of realizing their dreams.

James Kaplan: If you could speak a little bit about that — I think most people have the perception that venture funds mostly allocate capital and don’t realize how essential they are in helping companies become successful. So I was wondering if you could speak a little bit about that.

Ed Sim: You know what’s funny? I won’t say that we help them become successful because my running joke is that I think more VCs have been responsible for damaging companies than helping them succeed, to be honest. But I think the idea of partnering at the beginning of the journey — being an early true believer, the first investor, writing that check to help them get started and back the business so they can focus on building a product and getting to customers — to me is the most exciting part of the journey. Because you have no idea, things will change a hundred times over. And having the ability to believe in a founder and their vision, and then also having the patience to live through the thick and thin — that to me is the best part of the journey.

It is the most romantic part, is the hardest part and is the most fun part.

James Kaplan: You mentioned — and you joked, not without justification — that some VCs help companies and some VCs may not be as helpful with the companies they fund. What’s the distinction between a helpful VC versus a less than helpful VC?

Ed Sim: For me, I tell the founders when we partner with them that my job is to help them accelerate their path to product-market fit. I don’t actually tell them what to build in the product, but I’ll make some introductions to Fortune 500 executives, for example, who might be buyers of the technology, so they can provide feedback. It might be introducing them to a key hire. It might be challenging them on their strategy. What I like to say is that you want to feel like you’re an adjunct to the team. The best founders are the ones that pull from us versus us pushing on them. If you’re constantly pushing on a founder, then you’re really annoying them. So the best founders, in my opinion, actually pull from you, and they know exactly what they’re getting from you and how you can help them realize their goals faster.

James Kaplan: I recall Marc Andreessen said that in founders, he looked for a certain degree of disagreeableness — the willingness to get the door slammed in your face 50 times and then say, no, no, no, this 51st time we’re going to get that deal, we’re going to get that logo. What do you think makes for a good founder?

Ed Sim: Yeah, I have boiled it down. I like frameworks. I’m a framework thinker, and I think you are too, James, in certain ways.

It’s the five Ps. When I look at companies and founders, the first thing I look at is what is the product. The founder has to have the ability to zoom in and understand the day-to-day life of a user, which could be different from the buyer. And if they cannot understand that product perspective with a deep, unique insight — in terms of why they’re starting the company — that’s a problem. Two is they have to be passionate about it. They’re usually obsessed about this. They wake up in the morning thinking about it. They’re in the middle of the night thinking about it.

They’re in the shower thinking about it, and that passion carries them through hard times where they get 50 nos and get the 51st yes, because they believe so deeply that this should exist in someone’s hands — it’s really going to help that person achieve better goals or outcomes. Three is I think about the art of the possible. In the game of venture, to be clear, our job is to return capital to our investors. And you have to ask the question very simply: how big can this be? When you’re looking at emerging technologies like AI security — when we invested in that a few years ago, there’s no market size TAM, there’s no investment bank or Gartner research on it.

It is just like, hey, will there be more ML and AI in the world? Yeah. And are they going to drive better decisions? Yeah. And do we need more security? Yeah. Okay. That TAM could be massive. I just don’t know when. But that art of the possible is super, super important. And there are a few other Ps — two other Ps that I have — but one is the people. The other thing I look at, James, is: where did you come from, and who else is coming with you? If you’re not a pied piper of people, that means perhaps in your prior roles — and look, there could be some situations where you didn’t get along with someone — but in your history of working, you probably have people that want to join you, that will follow you. And that people aspect is absolutely massively important.

James Kaplan: where do you stand on the technical versus non-technical founder debate? There’s some people who say they only want technical founders. They some, some people who say can be technical or non-technical. What’s, what’s your view there?

Ed Sim: We only invest in technical founders. If you look at our tagline, it’s partnering with technical founders from inception, building the autonomous enterprise. And that to me is very important. But let me say one caveat, Technical founders. I’m not talking about the person just sitting behind a desk all day not talking to people.

Part of being a great technical founder and a founder, itself is being a great storyteller. I like to say that this person, whether it’s the founder or the co-founder of both of them together, have to have an ability to tell two stories. One is zoom in what I talked about earlier and tell the life of a product user and why they’re going to be 10 to a hundred times better using your product. And then two is they have to zoom out and tell a bigger story of why. I should buy your product. Why I should join your company. So that has to be a bigger story. Like this is how we’re going to go on this mission to build something bigger, to change the world, to, to do whatever. And you have to have both, because if you’re just on the product piece itself, then no one may want to join you because they’re like wondering how big can this be? You have to be able to do both.

James Kaplan: Back in the 1990s, somebody told me the most effective people could operate at lawnmower view and jetliner view and switch back and forth between the two fairly quickly.

Ed Sim: I love that that is exactly what I’m talking about. And that context switching is rare in, in one person. but those are the best kinds of founders. and you know them, James, you’ve met with many founders before, and there’s this trusted relationship, particularly in the early days you built with them.

Like, I think this founder’s just going to figure it out. I trust that person to figure it out and they have enough tech technical depth to make it happen.

James Kaplan: What makes a great storyteller and what makes a great story to your thinking? What are you looking when someone says, okay, gee, I want to go do this, or I want to go say this to customers.

Ed Sim: yeah. first of all, first and foremost is clarity. Clarity of vision. And the way I break down a story is what’s the problem succinctly? Is it a hair on fire problem? what is your product that actually solves that problem? And then three is, how big can this be if we all go down that path?

Three, three very simple sentences around that. So clarity. And for me, a founder usually doesn’t have it all the time, right at the very beginning, that. That, I’ll call it the blurb, like that statement that you put in an email or the introduction three sentence statement that evolves over time.

And part of being a great founder means how fast do you learn? How fast? What is your learning velocity? How fast do you take input? How fast do you figure out signal from noise? How fast do you adapt that blurb, and change it? And how open are you to changing it over time? Because you can’t be rigid, you can’t take all the input period, right?

You have to be disagreeable in certain aspects and you’ve, and then you come up with your story. And I think that is, that is part of the importance around it.

James Kaplan: Tactically flexible and strategically disagreeable, perhaps, or tac, strategically fixed and tactically flexible.

Ed Sim: It is you have to have some stake. There has to be meat in the bones, but there also has to be some sizzle.

James Kaplan: Yeah. I mean my, my observation is the best storytellers, even though those three lines can, imbue it with some drama, it’s not that you have a problem and fix it, but you make people care about the stakes and the problem.

Ed Sim: I think what you’re saying is that passion jumps off the screen, person you feel the energy, like that energy transfer when they walk into a room and tell you a story or when you share, there’s a dynamic give and take that is I think, going to be even more important in the world of ai.

There’s being a lot of slop out there, but that is why you choose one versus another.

James Kaplan: And how much of a coach are you to the founders? Because I imagine there’s some folks who have that vision but haven’t quite figured out how to light up the room yet. How much time do you guys spend trying to teach people how to do some of that? Do you, do you see what I’m saying? To what extent are you looking for people for habit, and to what extent are you looking to help them get it?

Ed Sim: I think it’s a combination too, right? When you’re paying for a second or third time founder and you’re paying a higher valuation from the very beginning, even in inception, you’re going to expect them to have all of that. And then there might be some new technical founder creating some brand new market that they only raise a million and a half and they’re young and inexperienced, but they’re also stupid, crazy, right?

They have that list where they’re trying and you’re like, I’m willing to work with this person as well to help them along. And I will also introduce other founders. Who’ve been there before, who can help them along as well. And so I think it’s both, and I enjoy doing both.

It’s actually a lot of fun. And I’ll give you one last perspective. I think the best investor as a, I’m on boards, like we decide to take board seats early, but I call it the three Cs. You have to know when when to cheer. Usually when you cheer is when things are actually going bad.

Like if a founder came to you and said, I just lost that million dollar deal, and they’re about to jump off the roof, you want to pull them back and say, that’s just one. We’ve got three others. Let’s just keep going. need to know when to challenge them, and that’s usually when they feel invincible. Hey, let’s go spend three times as much on this.

Because things are going so great and we’re never going to lose, and my product is so good. You have to beat them up a little bit to make sure that they, they don’t get hubris. And sometimes James, you just have to know when to chill. You just have to know when to leave them alone and let them do their thing and not get in their way.

And I think those three different opposing viewpoints, I think is, is very important to bring to the, to the game.

James Kaplan: One of the best pieces of advice I ever got in my life was from my dad when I was, it was seventh grade or something. I was in, I was in the little league and it was , two outs bottom of the seventh inning. I come up and I was just having a great year and I, um, you know, and we, my team was behind and I asked.

Absolutely knock the daylights out of the ball. Hard line drive right at the third baseman’s chest, six inches to the left. It’s a standup double six inches to the right. It’s a standup double right where it was. I lined out and ended the game and you know my, I was really dejected, right? My dad’s like.

Nothing you could have done. All you can do is hit the ball, just keep hitting the ball with authority. And most of the time it’s going to land. And that’s, to me, that’s your, you know, sometimes you, you do everything right and someone else gets the deal and you have to not let it get inside your head, I’m sure.

Ed Sim: I love that. Yeah, that’s so true. That’s what the journey is like, and people think it’s just glorified. It’s always up into the right, and it’s not like that. A lot of times it’s two steps forward, four steps back, five steps forward. I mean, it, it, it’s a journey and you, you can get beaten up a little bit.

James Kaplan: Tell us a little bit where Boldstart sits in the VC ecosystem. What domains where in the life cycle, where do you guys tend to focus and why?

Ed Sim: I like to say that in today’s world of venture capital, there are only two ways to win in my opinion. One is to specialize and the other is to go big. It’s like the traditional adage in any business: you go big, you go niche, or you go home.

We decided to specialize, and we specialize by stage and by outcome or product that we look at. So stage — I call it inception. This is not seed. It’s not pre-seed. It’s two people, one person with an idea, and they’re in the idea maze, the idea jungle. And we want to help them accelerate the process to saying, let’s start a company. And we’re there at incorporation and leading the round.

And that check size could be anywhere from, 200 or 300 K up to 15 million right out the gate, depending on who you are and what your background is and what the opportunity is. And so we

James Kaplan: is that an alternative to Angel, or is that a compliment to, how does that relate to, the angel round for want of a better term?

Ed Sim: is usually we had, we bring some angels on, we usually leave some allocation for high profile angels, former CEOs, former executives, but like getting an angel round done and you’re raising a million dollars, versus when you need five at once, right?

You want one place to go when it’s at the idea stage. And the problem with preed and seed in all these other categories is that a lot of these. Investors want to actually have more proof of traction. And it’s just a, a, a weird dynamic. And so, the other aspect would be the multi-stage firms do everything.

They write checks at our stage, they write checks, in series H at, 50 billion valuations, right? Or 500 valuations. So, I think our specialty is working with these founders at that stage, knowing the ups and downs and actually getting them to a place where we can bring the other people into the. Cap table very early, and types of companies, it’s, this is why we’re.

We’ve met each other through all the things that we do in IT infrastructure and cloud and security. I call it the autonomous enterprise. That’s when we launched our fund back in June — our Fund Seven, our 10th fund. And the idea is that we back technical founders building the autonomous enterprise. My running joke is that the goal is that one day we want that one person, thousand-billion-dollar company.

James Kaplan: the, the solo unicorn, right?

Ed Sim: But the reality of it is, is that there’s a lot of plumbing that has to change.

There’s a lot of last mile technology that needs to, to be delivered. There’s a lot of security. And so the aspiration of that is we want to fund everything underneath that, all the plumbing, the security, even to the agents being built on top. and I think that’s a, a great aspiration.

James Kaplan: So when I talk about the tyranny of the use case — just looking at vertical use cases only gets you so far. The companies you fund are the solution to that problem. They build the underlying infrastructure that allows — I also find myself saying technology is easy.

Technology at scale with security, resiliency, compliance, and maintainability and efficiency is hard. You do all that second bit.

Ed Sim: Yeah. And it’s really interesting because there’s been a lot of debate, as you saw in the last couple weeks, in the stock market about — oh, Claude released a plugin, right? So, and my running joke is, if you’re a startup can be reduced to a plugin or skill, you’re not long for this world and probably not a good company to start and bank your life on. the underlying infrastructure, you see, the Mongo dbs, the Datadog and all those, and Databricks, right? They’re all still growing very fast. You still need a lot of that. I’m not saying that we’re going to back RAG, vector databases or things like that because I think that’s a already subsumed into what the model providers are providing.

But you nailed it, James. I mean, we talked a lot to lots of Fortune 500 CIOs and CISOs and you do as well. And man, that last mile problem, there’s so still so much more to do and build and everything else like that and I can, walk you through things that we’re looking at. But I would love to hear your thoughts on that.

James Kaplan: Before we go into, the evolution of the space, I was wondering if you could speak a little bit about the purchasing dynamics for Fortune 500, which is, it’s a different world than it was 20 years ago, right? We have, it’s more systematized. It’s more consolidated. The procurement department is a lot, stronger than it used to be.

the security function is a lot, fortunately is a lot stronger than it used to be. We have approved vendor lists each. To my perception. Tell me if this, a few companies have official innovation programs, but that’s relatively small. My perception is it’s harder to get a foothold than it would’ve been 20 years ago.

Is that a correct or an incorrect perception? And how do you know, how, how do you, and how should companies manage some of those challenges in selling to behemoths with increasingly complicated purchasing cycles?

Ed Sim: I would say it’s yes and no because I work with a lot of them themselves. And I do work with like a lot of the, what call it the emerging tech programs, innovation, lots of great ones out there. As you know, there’s JP Morgan, Morgan Stanley, bank of New York, has a new thing

James Kaplan: So that, that’s tends to be certain companies with a billion dollars in spend it up. If you’re like

Ed Sim: Exactly.

James Kaplan: to a billion in IT spend, you’re less likely to have that.

Ed Sim: Yeah. Yeah. And so these larger companies, like here, let’s, let’s just take a step back.

James Kaplan: Yeah.

Ed Sim: I’ve found certain companies that I invest in reaching out to, not only them, but even people without formal programs being as interested in moving so quickly, at least to take first or second meetings.

James Kaplan: Interesting.

Ed Sim: And the meetings, having a check from a vetted investor who’s done this a number of times. We’ve had lots of exits and companies have grown to multi-billion market cap, I think helps get the awareness. So when we do send something, I mean, for example, James, I send you something, I only send you a few things a year, but I want to make sure that it’s going to be interesting stuff and hopefully you’ll have the chance to take a look at it.

But I think it’s easier to get a meeting in certain ways. but I do think that there’s also. So much noise out there, it may be harder to close deals, right? So not only you get those meanings, you’re going to have to separate. You’re going to have to be that person that we talked about, the dynamic individual that sells a story, builds trust. And then you also have to have incredible time to value. you have to be able to pitch incredible time to value so that they can get, a quick ROI from it. And then B is tell a bigger story about why they should standardize you versus 12 other people

James Kaplan: Right.

Ed Sim: the same product. Because AI has just really reduced the moat for many people.

James Kaplan: The, well that’s, that’s encouraging because I, will, on reflection, compared to say 2007 or 2010, we have less of a technology of legal opin. If you go back to 2010, three enterprise. Application vendors. one of two storage vendors. one of three server vendors, one of two network switch vendors, what have you.

And we just have a more distributed world than we used to, and I think we have more CIOs and CTSA, gee, I can’t rely on. My two or three, two or three behemoths to do my technology strategy for me. So that, I suppose it’s the positive thing. And then the flip side of that is the, you know, just managing the, you know, the procurement process, which can be quite comp and getting on a approved vendor list, which can be kind quite complicated.

Ed Sim: [Enterprises] have no idea what they’re getting at, because they still don’t know that on-prem may need, may be important, hybrid may be important. And so when they hit that like, oh man, I thought everything’s cloud-based. Well, no, when you have data and privacy and all those things that you worry about and you’re feeding ai, they’re, you don’t want to pass your crown jewels over to a model provider, sight unseen.

James Kaplan: Yeah, one of my, one of my favorite enterprise IT stories is, some from a, a cloud. related business. Went to meet with the head of infrastructure for one of the major, major banks, and they come in and they do the whole pitch, automation and auto scaling and this, the whole thing, right?

And this, head of infrastructure who is a bit of a dry sense of humor says this is, this sounds fantastic. I get just one question. I get data that can’t enter Switzerland. I get data that can’t leave the United States. I get data that can’t enter Singapore. How do you guys, how do you guys, how do you guys deal with that?

And he swears, the folks from this sales team look back in and say, why would you ever want to run your business that way? To which he replied, you seem like nice boys. Come back when you have this figured out. Which to me was the, the disconnect between, the, the startup environment and the enterprise right there.

Ed Sim: Yeah. But, but to that point I do

James Kaplan: They did, they did figure out. I will note,

Ed Sim: because of how fast the technology’s changing, I do see larger orgs being more willing to

James Kaplan: absolutely.

Ed Sim: of next generation things.

James Kaplan: So sometimes I will admit, I do see tension between the C-I-O-C-T-O organization. And the procurement organization.

Ed Sim: Yeah.

James Kaplan: A horizontal there is any to be horizontal platforms as opposed to vertical use cases. So are they see less of the tension between the center and the application development teams than perhaps some others do?

Ed Sim: absolutely that that is the case. But look, if you’re selling automation technology, there’s always that though, whether it be automating cybersecurity operations, automating it operations, there’s always a tension of, once it goes below the CIO. Are you automating my job away? Right? And so there’s always that healthy tension and the answer of course is no.

We allow you to do more with less. We allow you to grow faster with less headcount. We allow you to do the more interesting things. But there is that tension. Let’s, let’s not, kid ourselves on that.

James Kaplan: So now, now let’s talk to how the market’s evolving. What do you see the themes and the major, what areas are you excited about over the next two or three years, and how has that evolved compared to the past three or two or three years? Give us a little bit of the, of the movie as it were.

Ed Sim: Yeah, so that I think about is that when I laid out our new fund strategy back in June, we talked about the autonomous enterprise. And in my thinking, we are 10 to 15 years plus away from that happening. And that we should be prepared for this long cycle, for enterprises to move from pilot to production.

So that was our thinking. However, I think there’s a sea change in the world in mid-December when people went back for the holidays and started playing around with Claude Code. If you think about it, I think about what developers do first as a tipping point before what regular humans do.

James Kaplan: I’ve used both Cursor’s and Claude Code’s agents, and I’ll say — without getting into details — that my coding agent experience has been transformative.

Ed Sim: Engineers are saying, “I don’t need to look at the code. Let me just have the agent crank away and fix itself and keep fixing itself.”

So that was a major step change. And then two, because the developers were using it, and they’re using it for other things like personal productivity — they launched Claude for Work.

James Kaplan: Yeah.

Ed Sim: I’ve played around with all that stuff. And fast forward, you have Goldman Sachs announcing just last week that they had been working with Anthropic and that they want to automate the back office for KYC and accounting. And then basically what happened was that Anthropic had been there for six months working with the folks at Goldman and they made this statement. And so my point is saying, wow, when have you ever seen something come out so fast? A bank, a regular bank, coming out and saying, “I’m going to do this.”

That doesn’t mean everyone’s going to do it, but I can tell you this, I do really feel like, based on everything I’m seeing, is that this agentic technology, or the autonomous enterprise as I call it. It is moving faster than ever because the technology is getting better. people, perhaps are trusting it more, but that also means more opportunity to secure all that stuff and everything else.

And we can talk about all that later.

James Kaplan: Can I challenge one of your assumptions or one of your framings?

Ed Sim: please.

James Kaplan: Where you said developers aren’t looking at code — let me suggest something to you. I suggest we call Cursor or Claude Code a compiler. Which is to say, 30 years ago we had this debate: how come people aren’t looking at assembly language?

Right. And I would suggest to, engineers are still writing code, they’re just writing. Declarative code via spec-driven development, right, rather than writing procedural code and declarative code is an infinitely more efficient way to communicate a set of instructions to a computer. Does that resonate at all?

Ed Sim: Well, it does in the sense that English, English is the language of coding right now. Right. In

James Kaplan: Right.

Ed Sim: explain things and that they’re writing things. So Yes, it does. But my point really is that when

James Kaplan: I’m not disagreeing with you. I’m agreeing with you, quite frankly. I’m just —

Ed Sim: Yeah. No, I agree in the sense that engineers are still writing — they have to use the English language to actually write something, a framework, and how people think about it. But man, when you have founders you’re sitting with that can’t wait to get back from dinner, saying “I have 30 agents waiting for my instructions” —

we live in a different world,

James Kaplan: The value in AI enabled development is less that we get to use English language. It’s more that we can be declarative rather than procedural, because we don’t have to do all the error checking ourselves. We can just say, these errors must be checked.

Ed Sim: Yeah. In other words, how about this? In other words, building things to abstract it away is, is more fun again, for a lot of these engineers. Right. And they can

James Kaplan: Exactly, so. Exactly, so.

Ed Sim: I think that that is removing the grunt work. And, but I think that applies to, so basically you see what happens in devs and that will apply to a lot of knowledge work as well.

We’re just going to be managing a fleet of agents. All of us. We’ll be managing a fleet of agents doing work for us, and then we get to spend more time doing podcasts and going to steak dinners and things like that where you build relationships.

James Kaplan: I just think we may wind up spending more time managing agents. When you have a 500-person team working for you, that’s a lot of work.

Ed Sim: This is why some of my founder friends were like, “Oh man, I’ve got to get back from dinner.” They were getting anxiety because they felt like the agents weren’t working —

James Kaplan: Exactly. Or they’re going off in the wrong direction. I think you touched on something else very important there, which is I suspect the line between software development and non-software development will get very blurry very quickly.

Ed Sim: Completely agree.

James Kaplan: I sometimes compare it to what happened on the trading floor in the 1990s, where people started building algorithms in a spreadsheet, and then started building algorithms in code, and it just wasn’t clear when somebody who had been a trader became a developer.

Does that make any sense?

Ed Sim: Yeah. Yeah, absolutely.

James Kaplan: It’s just — “Oh gee, I came here to be a trader” — or vice versa, when someone who was an engineer wound up helping define trading strategies.

Ed Sim: A lot of these are going to end up as a skill. Things will end up as a .md or markdown file. There’ll be a registry of these skills and people will be able to tap into them. It could be for knowledge work, for algorithms, for anything you want. You can run it on your own machine, you can run it in the cloud, but it’s going to be really interesting as these become packaged as skills — that’s going to be the standard.

James Kaplan: And then there will be a skill — a human skill — in building skills.

Ed Sim: Yeah. And then, and then there’s going to be a skill package manager and a bunch of other things, and I won’t name companies, but I have one of those that just launched today who you’ve talked to.

James Kaplan: Right. Yes. One thing I’ve been thinking about a lot is the rules and skills we place around agentic development and the version control we place around those rules or skills.

Ed Sim: You nailed it. And so that’s why I think you need a registry, you need a package manager, you need to vet it. So you need evals and all the skills, and every time there’s a change, it needs to update automatically because these are living, breathing documents that actually can execute.

That’s the difference between having a spec in the past where people usually never even updated it. Right. And then

James Kaplan: Right. Yeah. Now we have living tech.

As, I mean, we used to joke about, not joke about, we used to talk about security as code a few years ago, which eventually became cloud security posture management. Now we’re going to have spec as code, right?

Ed Sim: And now they’re just, I think they’re just blow down to a skill.

James Kaplan: Yeah, and part of those, some of those specs will be functional, go deliver this functionality. And some of those specs will be non-functional, here, I don’t know, perform this, scan the code for this type of vulnerability before it’s allowed to go into, go into production. Or even something like, we’re going to use this set of semantics to name variables and not that set of semantics.

Ed Sim: And some of those skills may be abstracted away so that they’re actually end user skills, for example, prioritize my buckets and do things like that. Right. So that’s what makes skills so interesting. I think basically we’re all going to be living in one large markdown file.

Thanks for reading Prosaic Times! This post is public so feel free to share it.

James Kaplan: You happen to be right. I, the, the future is marked down.

Ed Sim: Isn’t it crazy? The oldest tech — .md. GitHub — you write your website in MDs.

James Kaplan: Everything was a text file. Right.

Ed Sim: Yeah.

James Kaplan: Which is great because, text is, text is the ultimate universal spanning layer,

Ed Sim: I am spending more time in text edit right now in my machine than I ever did.

James Kaplan: It’s fascinating because you can edit the same file with two different tools. Everything’s readable if you need to. It feels like everything old is new again.

Ed Sim: And then the question is: it’s all cool for a startup, it’s all cool for individuals, but then how do you deploy enterprise-grade? How do you point that to 500 developers, to a thousand, to 10,000 developers? How do you actually do that? How do you build the security guardrails in the version control system, as you said? How do you have an enterprise-approved registry where all these skills are secure and have been tested and they actually work?

So, and what happens when there’s an upgrade, right? you don’t automatically use the latest skill because it has to be tested. It has to be so these are all the things that people forget will have to be built in the enterprise.

James Kaplan: There is one thing that gives me a bit of hope, which is people are really bad at interpreting rules. But AI isn’t. And you can start to encode a set of rules around: we name variables this way, we break up code blocks that way, we scan for this type of vulnerability. And you can imagine some of that being foundational and built into the underlying tools and some of that being idiosyncratic to an individual technology shop.

And something that the architects or the technology strata, strategist for that institution defined.

Ed Sim: I agree. And then also the key will be then it will learn continuously on each individual person. And over time it will be dynamic, because nothing’s ever static in life. Any organism’s not, static. And so that will also adapt over time with how your organization, does things.

James Kaplan: Well, and that I think gets to the, some of the real complexity about doing this in scale because when do you want to let it adapt and when do you not? Right? Because there’s certain things like, okay, let me give you a simplistic example. social security number or credit card number should never go out outside the four walls of the organization.

You don’t want to adapt that.

Ed Sim: No, that should not.

James Kaplan: You don’t want to say, oh gee, the context is different, so it’s okay to send this. So

Ed Sim: we’ll call that an anomaly, if that happens, right?

That there’s enough, action items that I can learn from.

James Kaplan: We’ll need to figure out, okay, what are the enterprise class rules? What are the enterprise class rules that. that never change and can change over time. What are the either business domain or sort of segmented rules and what are the idiosyncratic rules to a particular project or particular developer and managing that hierarchy, I imagine will be both interesting and challenging and a tremendous source of opportunity for a range of companies,

Ed Sim: I agree. And the question is, is how can AI help you write those rules, right. And keep them updated,

James Kaplan: right? And manage right, maintain.

Ed Sim: because that’ll be interesting, right? because there’s a big debate right now about, systems record and large companies like a ServiceNow or Salesforce, just crud, right? And there’s lots of rules built into it, right?

Lots of money spent and creating triggers and rules and how things work. Hundreds or thousands of them are encoded into the

process in these systems. So they’re going to be hard to rip and replace

over time, the question is, will agents get better to

those over one at a time, two at a time,

James Kaplan: and I think the, the SaaS space is, the business application spaces. I think it’s going to be choppier than the infrastructure space over the next few years because I think there’s a lot more, one size fits none at the business application layer than there is at the infrastructure and security layer.

Ed Sim: Agreed. For sure.

James Kaplan: All right, so we’re coming up at, coming up at 45 minutes or so. What, what have I not asked about? What should, what should viewers know about

Ed Sim: I guess where do I see holes out there, or

James Kaplan: please?

Ed Sim: some of

James Kaplan: sounds great. Yeah, that’s, yeah, that’s a fairly obvious question. That’s one I should have asked. Thank you.

Ed Sim: Look, at the end of the day, we need more security. There just needs to be more security around agents. For example, when you have Claude Code running around and doing stuff on people’s machines — let’s say you have an organization with people using Claude Code.

How do you make sure that the machine is locked down and not giving up credentials and other important things? I think that’s going to be very important. The other thing I keep thinking about is that in a world of agents doing work, we’re going to need fine-grained authorization and authentication.

Authentication is: is the agent who the agent says it is. Authorization is: what fine-grained control should they have? What should the agent be able to do, and what should it not be able to do? And I think agents have to have their own credentials because you have to have an audit trail.

Particularly with banks and regulated industries. Then on top of that is what happens if there’s a hijacking of an agent during a tense. Step workflow. How do you make sure that cryptographically, that agent, is what that agent is and continues to be throughout the process. So there’s a lot of plumbing that people don’t talk about that’ll need to get fixed.

Another thing to think about is the OpenClaw movement. We didn’t talk about that.

But the idea is that I’m definitely playing with putting it on a virtual machine on Digital Ocean — its own Gmail account, I created its own Telegram bot account, I have it training on my newsletter “What’s Hot in VC” to understand how I think.

And then I forward emails to it that are interesting and important, but I’ve gated it. I think that architecture as a gateway — this gateway that can do a lot of things — is really cool and should be very enterprise-ready. At the same time, who’s going to lock it down? Because we can’t get the full value. You said earlier, “I need a personal assistant” — that could be your personal assistant, but you’re not going to give it access to everything.

James Kaplan: I’ve been experimenting quite a lot with agents for myself. The administrative stuff works much better than the content-based stuff.

Ed Sim: Correct.

James Kaplan: Yeah. Let me ask this question. Rich Isenberg and I talked about this the other day — non-deterministic security. Here’s what I mean by that.

We currently have what I would describe as a very deterministic security model. You have this control. It’s either implemented or it’s not. Either it applies to this or it doesn’t. Social security numbers? Never, ever, ever. The DLP tool will never allow the social security number to exit the environment under any circumstances.

But if it’s a strategy presentation. No, most of the time you don’t want it, but if you’re doing, some sort of funding, funding round you, maybe you want to send it to your investment banker. Right? So is there, will we see more contextual or more cybersecurity controls to take context into account?

Ed Sim: Oh, context is key.

I won’t tell you which one, but I have a portfolio company that literally says, “Hey, you’re going to build agent technology to automate some of the work you do in security.”

But it’s very hard to do it without context. Do you want to do a one-off? Or do you want to have a platform that literally connects to every one of your security systems, connects to your identity systems, connects to your cloud, connects to who James is and what he does — and then that’s the context. Then we’ll watch people do work and say, “Hey, these are some alerts, but I think you can automate these processes.” I think context is absolutely king. And I’ll give you one other interesting aspect — we just invested in a network security company, name to be determined, that’s reinventing network security. But it has to be reinvented.

Why? in a world of agents

Agents are going to do 99% of the traffic. You have to understand the business context of what the agents are doing in order to block traffic or open up traffic.

It can be deterministic in some ways and non-deterministic in other ways.

because it has to be flexible enough to understand, hey, if I read this. Business context and I read what the rule is. I think it’s important that we let it through. Because based on everything I know about the business —

Or wait, it seems like an anomaly, it seems like someone’s trying to hack and do something different.

So that’s where I think you hit on in the non-deterministic deterministic framework. The context is everything moving forward.

James Kaplan: Perfect. Anything else?

Ed Sim: The only thing I can leave with is: you and I have been doing this for a couple of years.

James Kaplan: Yep.

Ed Sim: This is probably the most exciting and the scariest time there’s ever been in doing this.

James Kaplan: We can do things that were unimaginable 18 months ago.

Ed Sim: And I wake up even like two months ago. I wake up every day wondering like, what am I missing?

There’s this constant fear in a way. I felt some of it during the internet boom in the mid-nineties, but nothing like this because back then we had dial-up. I mean, the speed

at which everything disseminates and happens is absolutely insane. And by the way, the winners that we have today may not be winners tomorrow.

So that’s what is, is thrilling and it’s scary at the same time.

James Kaplan: Exactly right. I could not have said it better myself. Thank you so much. This was fantastic.

Ed Sim: Thanks. I enjoyed it.

James Kaplan: Right.

Thanks for reading Prosaic Times — subscribe to get every issue!

Click the video above or listen to the audio-only podcast.

Here’s what I heard:

* You have to scale AI governance. Rich points out many companies moving too quickly (taking on risk they don’t understand) and too slowly (missing market opportunities). “Autonomy without governance is not innovation; it’s negligence,” he says -- but also that workflows scale, while committees do not.

* Securing AI will require experimentation. Rich and I talked about emerging techniques for security: using GenAI to accelerate security reviews, having agents create deterministic code at run-time that you can scan. Neither of these is a panacea. GenAI-enabled adherence review requires well-formulated standards. Scanning code at run-time can introduce latency in some workflows. There are no schoolbook solutions yet.

* The idea CISO advises, so business leaders can decide. CISOs aren’t there to eliminate risk, or even necessarily to reduce it. Just as there are no financial returns without market risk, there is no technology value without technology risk. Great CISOs provide business leaders with the facts (and advice) that allows them to take intelligent risks. Rich argues that when companies adopt this mindset, they won’t need large security teams. The CISO will focus on building and applying coherent risk management processes. The rest of the organization will build and consume the required platforms.

Thanks for reading Prosaic Times — share it with a friend!

Early thoughts

I love seeing different organizations and technology environments every week -- how they work (or not), where they might go, how they evolve. I’ve included a few reflections below on what I’m seeing that I haven’t fleshed out yet into an article. Think of this as the assignment deck for myself.

Governance will be a killer application for GenAI

We all devote more time and effort to governance than we did a couple of decades ago. Some of this results from increased corporate scale and complexity. Some from a better appreciation of the risks and a more risk-averse culture. Some from more regulatory mandates. Sarbanes-Oxley, know-your-customer, anti-money laundering, tech architecture, cybersecurity, procurement and privacy governance are all essential, but all require massive efforts from the governors and governed.

GenAI is bad at many things, but wonderful at making comparisons. Automating the comparison of artifacts with standards (possibly encoded in a knowledge graph) reduces the governance tax, rips wait time out of important processes -- all while improving policy adherence.

Don’t always communicate top down

In communicating everyone (I hope) tells you “BLUF: bottom line up front” and “Show, don’t tell.” This is good advice.

The Pyramid Principle by Barbara Minto reconciles these imperatives. You tell the reader or listener the bottom line and then you show the evidence for it -- this is a broadly applicable and stunningly effective communications patterns.

Yet, in some circumstances up front isn’t help. Bottom-line conclusions -- “it works,” “we’re on track,” “we’re done” -- are subjective syntheses of underlying facts. So, in some cases, you and the listener may disagree of synthesis more than the underlying facts.

In which case, you may consider a different communication pattern. Instead of saying “we’re on track,” start with “here’s where we are” -- and invite the listener to problem solve you on whether that means “on track” or not.

Rich Isenberg and I talk about what makes a great CISO below. CISOs must enlist facts to create alignment on synthesis. Even if everyone agrees on which data is encrypted, what does that mean in terms of aggregate level of risk? Is some variance to standard an unavoidable reality or an emergency that the leadership team must address.

Great CISOs, for example, evaluate the clarity of the facts, the political correlation of forces and their personal credibility to determine when to say “we must fix this now” and when to lead with “here are the facts -- let’s problem solve on what to do about it.”

The Interview: Cybersecurity leader Rich Isenberg

On the go? You can also listen to this in the car or on the elliptical via Spotify

James Kaplan: Welcome to another Prosaic Times podcast. I’m thrilled to have my colleague, Rich Isenberg, joining us this evening. I’ve wanted to do a podcast with Rich since I first got the idea that we should do podcasts for prosaic times. Rich, could you introduce yourself and tell us a little bit about your background and your journey from Connecticut to Atlanta to Capital One and SunTrust.

From computer camp to cloud security

Rich Isenberg: Wow, you’re making me feel old having to recount all

James Kaplan: Rich, just to be clear, we are old.

Rich Isenberg: That’s a fair point.

James Kaplan: Do you remember five-and-a-quarter-inch floppy disks?

Rich Isenberg: I do

James Kaplan: You are old my friend. Old.

Rich Isenberg: I had an Apple IIe and then an Apple IIc, and then a Mac. Always Apple, though.

James Kaplan: My very first computer was a VIC-20 with a tape drive, with 5K of RAM. And I think I got a C on a paper in seventh grade because I got an A on content, F for handwriting.

So, which averaged out to a C, so I wrote a word processor in Commodore BASIC so I wouldn’t have to write my English papers more clearly.

Rich Isenberg: Going to computer camp as a kid and learning how to, I think I grew up in Connecticut in the Hartford area,

James Kaplan: yeah, New Jersey. We didn’t have computer camp that was purely a Connecticut thing.

Rich Isenberg: Commodore 64s and oddly enough as you’re saying it, I still remember the programming. I learned BASIC programming to make my name scroll across the screen diagonally.

James Kaplan: I had a VIC-20, which I had inherited from my uncle, and then he replaced the Commodore 64 with an Apple, so I inherited the Commodore 64 and I was very excited because it came with a 1541 disk drive, which was the size of a shoebox.

It made terrible noises. But the neat thing about Commodore BASIC is the memory map was on page three of the manual. So at the age of 12, I was oh gee, if I want to do anything cool, I’m gonna have to write, learn how to read and write directly to video memory, using PEEK and POKE commands, the things that kids don’t learn today.

We should actually talk about cybersecurity or something that.

Rich Isenberg: Do you remember the first video game you played on the first computers you owned? What was the first game that somebody gave you?

James Kaplan: I remember writing two, the first of which was, a little thing where you, wandered around the dungeon and, tried to avoid monsters.

And I remember watching my brother and a friend of mine play it, and that was very satisfying. In seventh grade I wrote a program to keep track of baseball statistics for the science fair. And I realized that was sort of boring.

So I wrote a, program that you had to navigate some entity around the screen, without bumping into other things. Everyone came and played.

Tell us a little bit about your story then. We should actually talk about something relevant to what we’re supposed to be talking about.

Rich Isenberg: Alright, so, Rich Isenberg. If I were introducing myself to clients, I’d say I’m a partner with McKinsey in the Atlanta office. Been here for six years, work on interesting things and tinker with stuff. I grew up in Connecticut in the Hartford area. Went to college in...

James Kaplan: Was a big Hartford Whalers fan.

Rich Isenberg: I was a big Hartford Whalers fan. As you can see with Sherman the Otter wearing his

James Kaplan: Yep.

Rich Isenberg: jersey. Always on my desk.

Ended up working for Panasonic, leading the network division of their research and development in the United States, when the first SQL Slammer worm came out.

There really were no information security professionals outside of government. The internet was fairly new — mostly advertising and stuff like that. We weren’t really transacting. But as the network manager, I had to figure out how to play cat and mouse with the bad guys and shut this stuff down.

When I left Panasonic, I met the guy who invented online bill pay. He had a company called CheckFree in Atlanta, Georgia. I spent the next decade there figuring out things like remote deposit capture — things people take for granted today. How do you securely transact financial transactions over the open internet instead of private lines?

I continued to learn. Eventually, I joined Capital One, running cybersecurity operations and figuring out how to take a regulated bank to public cloud.

I came back home from DC to Atlanta and took over the chief information security officer role at SunTrust, my hometown bank, which was really fun. They merged with BB&T to become Truist. And I joined the firm as a direct-hire partner the day before we closed it down for COVID.

What makes a great CISO?

James Kaplan: Alright, I remember that. So you get the opportunity to interact with a bunch of different technology organizations, a bunch of different cybersecurity organizations. You get to interact with a whole bunch of different CISOs. What in your thinking makes for a good CISO versus a not so good CISO, and what makes for a great CISO.

Rich Isenberg: That’s an interesting question. I think if you go back 20 years, CISOs came from a non-technical, audit-heavy background and always got a reputation as the draconian Dr. No — those are not good CISOs.

After that it became clear that CISOs who understood the business started to make great CISOs. If you don’t understand what the business is trying to accomplish, you can’t possibly understand how to enable it securely and how to advise business leaders on risk trade-offs. I still think this is true today.

Oftentimes, I’ll quiz CISOs: tell me about your products. Tell me about your customers. How do they interact with your systems? Surprisingly, a lot of them don’t know the answers, nor do their teams.

I tell CISOs when I counsel them as they take new roles that at least 30% of your time should be spent up and out with the business, understanding the business, talking to business leaders, understanding the revenue streams. That’s the way you’re gonna be able to help prioritize and really help the business build value.

James Kaplan: I push that even a little bit further. Certainly in, in B2B markets, CISOs spend time with customers. And some of the best CISOs I know, have to fight off demands from the channel and from the sales force to go meet with this customer and that customer and the other customers.

What, what’s your thoughts on that, or what’s your reflection?

Rich Isenberg: I think that’s valuable. I also find that sometimes it can be distracting because customers want you to provide comfort, security, and safety as a feeling. And sometimes that distracts from the work of actually creating safety and comfort within the products.

It’s like any other CISO role — you have to balance the stresses on your time. I’ve even seen some B2B product companies create a separate role for customer-facing CISO versus CISOs accountable for responding to incidents. I think that’s a healthy move.

James Kaplan: Another term you could use for feeling of security is trust. And you could argue a lot of, a lot of what. Some CSOs do is spend, is spending time building trust with markets, which we could describe as a function of being secure and people perceiving, the security as existing.

And so it has to be both things.

Rich Isenberg: I agree. And when markets are concerned, oftentimes perception is reality

James Kaplan: The distinction between B2B and B2C markets for cybersecurity is fascinating to me. Many consumers are a little bit cheap and cheerful in terms of security, excluding high net worth and mass affluent in banking.

But oh my god, B2B customers care about it. If you’re doing group health insurance or prime services, there’s real pressure from the customer base around security.

Rich Isenberg: Absolutely. And the pressure’s building. A lot of businesses are getting smarter in their contracting demands around security in the B2B market. That’s put a ton of pressure on being able to even submit a bid. You have to prove the security features and that you can meet their standards.

And I think that’s fairly new — and increasingly specific.

James Kaplan: And it goes to your point about understanding the business. Because if you are being asked as a ciso, can we underwrite this risk or can we warranty this degree of protection — that is a fascinating business technology problem. You need to understand the business, and quite frankly, to your point, earlier point, you need to understand the technology,

Rich Isenberg: A hundred percent, and I would even push it a little further and say that the ideal CISO doesn’t make these decisions. They advise, and the business makes the risk decision, but the CISO has to present them in business language, to your point, to be able to actually determine what are the chances we could underwrite this risk if X thing happens.

James Kaplan: Jim Routh, who is the CSO of a, a number of companies, I probably won’t even list them, was, fond of saying that, He wasn’t in the business of eliminating risk. He wasn’t even in the business of reducing risk. He was in the business of allowing the business to make intelligent, take intelligent risks, that enabled them to do certain things from a business perspective or market perspective.

Rich Isenberg: 100% agree with that. The only way to eliminate all the risk is to unplug the computers and send everybody home.

Nobody gets paid, and that’s ridiculous, right? So you do have to think in that manner. I would also push it to say that if this were perfect and you finally had the perfect organization, right, the CISO wouldn’t even have a team.

It would be an individual contributor because I think these large CISO teams have become a crutch. If you asked the business who’s responsible and accountable for security, they have a 500-person team with a billion dollar budget to point at.

I think the more mature model for a CISO to push is that security is everybody’s job and everybody’s business and everybody is accountable. They are more in charge of ensuring the right things get prioritized and the right things get done. But if you really build that culture, you don’t need a team of 500 people because it’s baked into the business processes already.

What should the cybersecurity organization look like?

James Kaplan: Well, this is a really interesting question and which opens the door to a bunch of other. Interesting questions. You could say at some level, the CSO role, at least in many places, is a bundle of two functions. There’s a risk management function, right, which is Jim Routh’s point, helping the business, business leaders to understand which risks, right?

And there’s a service delivery function. There’s a number of things. soc, incident response, DLP, often identity often, but not always. Identity and access management, which are typically put underneath the ciso, right? Should the CISO organization be purely a risk management organization, or should it have all these technology delivery responsibilities?

I tend to favor putting them together because it makes the CISO’s role more real, but others have a different point of view. Where do you, where do you land?

Rich Isenberg: I think it goes back to reality versus a hypothesized ideal future. Having done the role and been in the C-suite at very large organizations, the ability to execute delivery and get things done is important because it helps you carry the mandate.

But if you could truly have an ideal security culture driven from the top down, there is this hope that you could turn it into more of a risk management function. Maybe a lot of the agentic AI capabilities are part of what may unlock this. But I don’t think anyone’s there right now.

I am a fan of combining it right now. What I always advocate is that every CISO needs solid engineering — not infrastructure engineering, but full-stack developers integrating with APIs and actually building security as a service.

The most effective CISO teams have a large part of that function today.

James Kaplan: Even if the CISO has a lot of delivery, a lot of what you need to be secure happens outside the CISO’s office. The security team doesn’t patch the servers. It doesn’t write secure code. Is that something you agree with?

Rich Isenberg: I do. When people ask me about vulnerability management, which generally sits under the CISO team, I point out that the CISO generally doesn’t own asset management. They don’t patch the servers, they don’t write the code.

But these vulnerability management programs and testing programs are simply grading the effectiveness of the IT management processes. A vulnerability management scan is showing you how effective your policies and processes on patch management and software lifecycle management are.

The CISO doesn’t own what creates this, but they’re using these programs to get the information to determine how effective they are.

James Kaplan: Where do you stand on CSO versus Chief Technology Risk Officer? If it was up to you, how would you structure the role?

Rich Isenberg: I mean, titles are titles.

James Kaplan: Not, not, not the title. I mean, whether there’s one person who’s in charge of cybersecurity and then other people who are responsible for other forms of technology risk. I’ll be transparent. I favor combining all the technology risk together because you would reduce the chance that you have two different teams going to talk to application development folk about patch management, right?

But curious what you think.

Rich Isenberg: I agree because there’s a lot of overlap in controls around different areas of technology risk management. Having different teams with different methods and different opinions, often assessing the same controls, is not a model that works very well.

James Kaplan: A couple years ago, there was a bit of a move to take CISO out of it. I was wondering what your take is. I think you see less of that now because I, my perception hasn’t been terribly effective. What’s your take?

Rich Isenberg: I agree. There was a push many years ago because of this theory that if CISOs line-reported to the Chief Information Officer, they wouldn’t have an unbiased view and voice to stand up for safety over speed.

Oftentimes speed would trump safety because of the way technology orgs were historically incentivized — on speed and code. They didn’t consider a security defect a real defect, and nobody’s bonus depended on it.

I do think the only way to really bake security and safety into products and technology organizations is to be embedded within it. Otherwise, you end up bolting it on afterwards.

James Kaplan: If you think your CIO is gonna prevent your CISO from providing transparency to the management team and the board, you need a different CIO and a different CISO

Rich Isenberg: Right. But I do think one of the norms that’s enabled healthy behaviors here has been that while a lot of CISOs line-report to a CIO who then reports to the CEO, they also have an independent relationship with the CEO and the board. And I think that’s important.

When I was the CISO at SunTrust, even though I reported to the CIO, the CEO, Bill Rogers, would directly say: “Nobody gets to edit Rich’s board slides.” You could see them, you could talk about them, but you don’t get to change them because we need transparent talk of risk. Debates are healthy. There’s a healthy tension by design in that system.

James Kaplan: The one place where I’ve seen the the CISO not report to the CIO is, in technology organizations where you have technology spread across the entirety of the business. And sometimes then I see the CISO reporting to the CEO.

Rich Isenberg: I’ve seen that too. The giant technology organizations are very engineering-driven. The security teams there don’t have the same mandates that a large bank security team does. They don’t have the big regulatory stick.

Nobody’s gonna yank their ability to do banking if they’re disappointed in their security model. So I do find they become very innovative and often do things better.

These large tech companies benefit from ruthless standardization of their technology stack and not having to deal with 50 years of tech debt and mainframes. They have a database, which is sometimes a lot easier to deal with.

What makes a good CIO on security?

James Kaplan: I think there’s an interesting dynamic for the cloud service providers because they face so much pressure from the banks. So we’ve talked a bit about CISOs. What makes a good CIO in terms of information security or cybersecurity and technology risk? You and I have both heard many CSOs expressed frustrations with the CIO.

What are the great CIOs or what makes for a great CIO or what great CIOs have you observed or what are the characteristics of great CIO on security?

Rich Isenberg: Number one, they’re really good listeners. Some of the best that I’ve observed or worked with, even though they may actually be the smartest person in the room, they’re always the last to talk. They’re always asking for everybody’s perspective.

When you start to have leaders like that, it actually develops healthier conversation and makes it easier to gain influence and do the right thing. What makes a good CIO is a lot of what makes a good leader. You have to build an emotionally connected team — the emotional connections and the trust within the team.

Now, I will say they also have to care about technology and understand it. There are a lot of long-tenured CIOs that operate from a seat of fear because they don’t really understand the technology. Those don’t really get inquisitive and don’t always get to the right answer.

James Kaplan: And it’s a challenging thing, right? Because if you’re a CIO, it’s a long time since you’ve been fingers on keyboard. the proliferation of technologies in a large institution is almost terrifying, right? And very hard to keep up with. it’s funny, the head of infrastructure at, one of the big banks was a good friend of mine and they were having problems with — I forget whether it was,

Tandem or Unisys or something — some decades-old system in some part of the world, and he’s this is great. This is the thing I was first programming on in the 1970s. I’m getting on a plane, I’m going there myself, I’m gonna show him, show him how it’s done. But I do have a degree of sympathy.

For people who’ve been around a long time and you’re managing a huge organization and my God is a variegated stack, and how the heck do you stay on top of all this?

Rich Isenberg: Yeah. Great CIOs are great talent multipliers — that’s how they stay on top. They invest in people, modern skills, and partnerships. They know when to build, buy, or partner rather than doing everything themselves.

Secondly, they operate with discipline and clarity. Who has decision rights for what? Get rid of the friction in teams. The best CIOs make it easier for CISOs to do the right thing rather than attack with brute force.

They’re also great architects of trust. They care about balancing speed with safety, innovation with control. They wanna make sure their org can move fast without creating invisible risk.

I think they’re really good at translating technology into business outcomes. I don’t hear the best CIOs talk about systems and platforms. They talk about growth, efficiency, resilience, and customer trust.

James Kaplan: I would, I would say I would take maybe a slightly different view, which is the best. CIOs will talk about platforms and technologies depending on the audience. Right. They can talk about growth and margins to bus, fellow members of the executive team. But when they’re talking to their town hall, in many cases, they can show, they can understand the platforms and the technologies.

And the other thing I’ll add is many of the best CIOs I’ve observed are obsessive readers, right?

Rich Isenberg: Yes. And that’s another translation too — the best leaders I know are voracious readers. But the mandate for a CIO has changed. It used to be that CIOs needed to deliver technology.

Now, when you look at the amount of technology conversation in stock rating calls and analyst questions, it’s clear that the great CIOs are delivering confidence that the organization can actually innovate, scale, and adapt without losing control.

James Kaplan: Well, I’d also say if you go back about 15 years, we had much more in the way of technology oligopoly. A couple of storage vendors, three compute vendors, a major enterprise application vendor in each domain, three or four DBMS providers.

You had a simpler stack and you could rely on the vendors more. Fifteen years ago, a CIO was, in many respects, a vendor manager rather than an organizational and environmental architect.

Rich Isenberg: Yep. Back in the day, nobody got fired for buying the 800-pound gorillas in the market, whether networking, storage, or compute. There was safety in that. And I agree that vendor management was what was really going on.

Securing AI

James Kaplan: All right, so let’s talk about AI and security or AI and technology risk.

There’s two mistakes you can make in security. You can go too slow or you can go too fast, right? Or you can take on too much risk or too little. Which direction do you think most companies are?

Erring in, or is it a mix or both? So, the modal company, do you think is, is moving too fast on security and incurring too much risk or moving too slow and incurring market risk rather than technology risk?

Rich Isenberg: I think it’s definitely both, and oftentimes within the same company. Many companies are being pressured to move very fast on AI experimentation and early deployment without fully recognizing how autonomy has changed the risk profile. They assume existing cloud or application security controls will be sufficient, even though agentic systems introduce new failure modes, chained actions, and hard-to-trace decisions.

At the same time, there’s this reaction driven by fear and uncertainty. Some orgs respond by applying a one-size-fits-all control, treating every AI use case as the highest risk. That slows everything down and pushes the business toward shadow AI.

The orgs that get it right aren’t the fastest or the most cautious — they’re the most deliberate. They tier things by risk. They’ve invested in monitoring and containment early. And they allow people to move very quickly within those boundaries.

Speed, when done right, can be achieved not from ignoring risk, but from having confidence that it’s managed.

James Kaplan: I thought, and I think you thought that cloud was a tremendous disruption. From a security standpoint that, we had to give up the perimeter, everything was much more automated, things were moving faster.

We needed a, a whole new security model. And I would say, generally speaking, most technology organizations, rose to the challenge. Over time, after, some deceleration or some slowness. But yeah, it feels a much bigger disruption to security models than, than cloud ever was..

Rich Isenberg: I agree. And part of it is happening faster. When I talk to clients, I use this analogy: it’s very similar to cloud, but on steroids.

When public cloud started being experimented with in large organizations, security teams would dedicate 5% of their multi-talented architects’ time to try to figure it out while the business and the rest of the tech team hired a thousand people. Then they realized security was the blocker because it needed full-time focus. Suddenly we saw the creation of cloud security teams, and then you started to get business enablement.

I think AI is different enough that the same thing is happening. Those that are gonna get it right realize that you need some dedicated horsepower directly involved with the tech and business teams to figure out strong agent and machine identity and to build that deep observability.

But I do think there’s a challenge: there’s nobody in the job market who’s been doing this for 15 or 20 years. When I think back to my Capital One days, part of the reason Capital One was able to accomplish so much so early is there was no competition for talent.

So I do think there’s this great need that security teams have to figure out how to upskill their teams to step up to the challenge.

James Kaplan: It’s organizationally similar, but conceptually different. And the way it’s organizationally similar is death by use case, which is a million little, not little. A million vice presidents, directors want to commission their project using cloud now using ai, and they’re gonna go solve their use case around.

I don’t know. customer retention in middle market. Something, something, something, and the rest of the organization can go hang and they’re not gonna use a platform and they’re not gonna use standards. And maybe they’ll get some vendor that will do whatever they want, want to do. And, it’s, it’s sort of in terms of transformation, it’s as I to say, it’s trying to get to the moon by climbing the tallest tree.

The first 60 feet feel great. But you’re very quickly creating a lot of technical debt, and I think you see some of the same things with ai. Now, conceptually it’s different because cloud as complicated. It was, at least it was deterministic.

Rich Isenberg: Yeah, I agree with that a hundred percent. There are lessons learned from people dealing with enormous inventories of tech debt because of what you just described, and it can easily be recreated in the agentic world.

If you allow every business to choose their own agent platform, build their own agents, and create their own MCP servers and AI gateways, you’re eventually gonna end up with 87,000 outdated versions of Linux residing in most data centers today that somebody will have to go deal with.

James Kaplan: Or not even using AI Gateway, just go directly to the, go directly to the provider.

Rich Isenberg: Yep.

James Kaplan: We’ve talked about some of the governance and organizational changes. What do you think are the three or four biggest architectural changes or technology changes required from a security standpoint in the age of AI or the agentic age?

Rich Isenberg: If I think of three, the biggest shift is moving from a thought process of securing individual systems to securing end-to-end workflows. Agents and AI really break the old paradigm that there’s a single user, a single action, or a single boundary. You really have to make that mind shift.

Second, companies need to think about strong agent and machine identity. All these agents need verifiable identities, scoped permissions, and least privileged access, just like humans. I find that’s one of the hardest things to get right, and not getting it right from day one will make it very difficult to scale.

James Kaplan: Is agent identity more complicated than machine identity? Because we’ve been dealing machine identity for a while now, is agent identity thornier?

Rich Isenberg: Yeah, I would argue most companies have not really solved how to manage machine identity.

James Kaplan: Okay. But a few have, right?

Rich Isenberg: They have, and it is thornier. Some of it is because there are new protocols like OAuth and rich access requests to deal with.

If you believe part of the goal is to build an inventory of reusable agents that you can stitch together in different workflows, then these agents have different roles every time they execute a task. You have to ensure they have an identity fit for purpose for that role on that task — which may only last 15 minutes. And they need a different identity with different permissions 15 minutes later to execute their task on another workflow.

This leads into another architectural challenge: policy enforcement now has to happen at execution time, not just at design time. You need controls that can evaluate, in runtime, what an agent is trying to do in context before the action actually occurs.

Third, you need really deep observability from an architectural design perspective across planning, tool use, and code generation. That has to include containment and rollback. You need the ability to isolate agents, revoke credentials, and reverse actions.

If I had to sum it up, the big change is security has become dynamic, identity-driven, and behavior-aware — not perimeter-focused.

James Kaplan: Yeah, I mean, I’ve had Jaya Gupta from Foundation Capital on a week or so ago, and she was talking about decision traces. But it occurs to me that decision, the sort of, the deliberative exhaust data about deliberations that agents can throw off, right? That you see on the screen when you use cursor or what have you.

It may be the case that decision traces will be, or agentic decision traces will be an important part of the security telemetry going forward. We need to capture that to understand what the risks are and how to manage them.

Rich Isenberg: Yeah, a hundred percent agree.

James Kaplan: You made the point that we need to. we can’t do things at design time. We need to deal with them at runtime. One of the most interesting papers I saw recently, asked the question how in God’s name do you manage what agent is allowed to do what or access what tool in running a non-deterministic process, right?

And it, the paper offered an interesting suggestion. That instead of the agent accessing a tool directly, the agent should generate code to access the tool and then we can, we can analyze the code to see if it’s accessing something it shouldn’t. And then the agent should, and then, you can have tooling that says, yep, this code looks okay, and then the agent can run the code.

I was wondering how practical you think that is.

Rich Isenberg: Very interesting idea. I actually think that creates a pretty strong pattern, especially if you’re talking about higher-risk actions.

It reframes the agent from an autonomous actor into something more like a developer. A just-in-time developer, right?

James Kaplan: A just-in-time developer, but we know how to scan code.

Rich Isenberg: While you’re right, the code can be scanned and validated using security tooling. You could do policy checks, static analysis, compliance rules, pre-execution. But I don’t think it’s something you’d use everywhere — it would add latency and complexity.

But for actions involving data or infrastructure change? I think it could be extremely effective at turning opaque agent behavior into something that’s inspectable, which is kind of what security teams want.

James Kaplan: The paper proposed this in terms of calling tools that would send data outside the environment, which is not everything, but is higher risk than other things.

Rich Isenberg: Yeah, so I think case by case, it’s a pretty strong pattern to have in the quiver, but I don’t think it’s a de facto single silver bullet.

AI for cybersecurity

James Kaplan: So let me turn this around and talk about some of the opportunities associated with cybersecurity in the agent era. We talked a lot or we talked a bunch before about how the role of cybersecurity is to help businesses make decisions about which risks to take.

Risk management is inherently non-deterministic. It requires judgment. But a lot of our thinking about security is often very reductionist. This data must be encrypted at rest. Those systems need two-factor authentication.

That works great if we’re talking about preventing social security numbers from being sent outside the organization. It works less great if we’re talking about whether we can share this strategy document with a vendor — is it baked enough?

And I was wondering, is there a way to build reasoning into cybersecurity processes to help them scale?

Rich Isenberg: What an interesting idea. The strongest point is that you’re correct in saying a lot of controls today rely on a very static yes-or-no rule.

A non-deterministic control would evaluate the intent and the context. Rather than saying social security numbers can never leave the environment, it would ask: What data is it? Who’s requesting it? What’s the purpose? What’s the downstream exposure? And it would make a risk decision: should I allow it? Should I allow it with constraints? Does it require human approval? Or does it get blocked?

If you took that principle and combined it with policy engines and feedback loops that can reason and learn over time — tightening or loosening based on observed behavior and human grading — it could reduce friction while still maintaining security. It could be key to how orgs can scale without drowning in approvals.

One of the benefits of agentic AI is removing that human delay. But what orgs are struggling with right now is there’s an enormous amount of human delay in building and approving all the things that go into agent AI.

James Kaplan: Yeah, I mean, it’s a, it’s a, it’s an interesting interconnected challenge because we need, at least I’m observing as rapid a pace of technology consideration. As I’ve seen in decades, right? There’s all sorts of things, individual technologies you need if you’re gonna do AI in a scalable, responsible way, all of which need to be evaluated and risk assessed.

And many organizations I know of are, sort of bottleneck in that, in that way.

Rich Isenberg: Yeah, I’d say I agree completely.

James Kaplan: Yeah. And the and what’s interesting to me, particularly about this issue of agent to control is if you think about what you and I have done many times over the years or constantly over the years as we think about security strategy, is we help people think through the intermediation between a necessarily,

Non-deterministic set of risk analysis, right? How much risk is, how much property risk is too much, too much intellectual property risk. There’s no equation for that, but that has to drive a set of deterministic controls based on human judgment, about how much IP risk is too much IP risk.

We either do or don’t have multi-factor authentication on this system.

Rich Isenberg: Yeah. I mean, I think you’re right. I think the biggest risk in AI isn’t what the technology can do. It’s deploying it without knowing how to stop it.

James Kaplan: Exactly.

Rich Isenberg: If you can’t slow it down or shut it off, if you can’t explain it, you don’t actually control it.

And now we’re — I think it’s about autonomy, and autonomy without governance is not innovation. It’s negligence.

James Kaplan: Right. And then the one other thing I’ll note is there, there are more prosaic things that I think companies need to look at from a security standpoint, which is, you know, can you get some of the. Can you automate some of the toil of just comparing this architecture to that set of architectural standards or security standards?

I was wondering if you could talk about that. How, how, how real is that now and how big a deal is that the ability to compare different things to even deterministic technology standards.

Rich Isenberg: I think it’s here and it’s a big deal. Companies with all their technology are very concerned about resilience. One thing we’ve learned with pattern recognition is that a way to measure resilience is how compliant everything you build is with the actual standards in place around building it. Most folks can’t answer that question.

To be able to really do reasoning, you feed it your knowledge base and start to do the comparisons. You figure out where you have non-compliance to architectural or security standards that are creating invisible risk, so you can actually make decisions.

We have clients doing this. A common bottleneck — not just for AI but for development organizations writ large — is the architecture review and security review. It becomes very opinionated and committee-focused. One thing we’ve learned is that committees don’t scale; workflow scales.

James Kaplan: The second thing I wanted to ask you about, which may be a little bit closer in than non-deterministic controls, a number of folks have observed that there’s a security opportunity and the ability to integrate and interrogate the massive amount of exhaust data that any technology environment of any size throws off.

How real is that and how close in is that?

Rich Isenberg: That said, I think all these things are possible. Where people are getting hung up is that a lot of chief executive officers and boards are demanding: where is our agentic transformation in the business? They’re not really focused on the opportunities to fundamentally change the IT operating model and cost structure through some of these opportunities you’re talking about.

To truly succeed in this transformation, they have to do both. The opportunity to take in the exhaust and get decisions and insights, as well as accelerate the pace of technology done safely, is gonna be market-changing.

For the folks that get it right, they’re going to outpace their competitors and do it more efficiently.

James Kaplan: Yeah, I mean, I’d say that first off, For most companies or many companies, the biggest single rate limiting factor for their strategy is technology. They just can’t deploy capability quickly enough or efficiently enough. And the second thing is, or the clear implication of that, is you need to apply AI in a disciplined and systematic way to the technology function.

No less than any other. Business function, quite frankly, you won’t succeed in applying it to any other business function unless you apply it to the technology function.

Rich Isenberg: I agree. And this goes back to what CISOs and CIOs should be thinking about right now. AI is not gonna replace technology leaders, but technology leaders who can’t govern AI are gonna replace themselves.

James Kaplan: Or be replaced.

It’s that cartoon I love showing a horse in the 1890s: you’re not gonna be replaced by a car, you’re gonna be replaced by a horse driving a car.

Rich Isenberg: I tell my kids, right, AI is not taking your job, but someone who knows how to use it better than you will.

James Kaplan: Anything we, anything we neglected to talk about? Anything else you would want to say?

Will the NHL ever return to Hartford?

Rich Isenberg: I don’t think we spent enough time talking about the tragedy of Peter Karmanos moving the Hartford Whalers out of Hartford.

James Kaplan: Well, I was gonna get to that — that was the next thing. Alright. I mean, and that’s an even bigger tragedy than the Dodgers moving from Brooklyn to Los Angeles. So here’s the question I was gonna ask. What is the likelihood that there will be an NHL expansion team in Hartford in the next decade if you had to bet on it?

Rich Isenberg: Zero.

James Kaplan: There is no non-zero price you would buy that security at?

Rich Isenberg: No, and I’m tainted by history, so there is bias in here.

But if you remember — most people don’t care about this or are too young to remember — the reasoning is that Hartford is in between the television markets of Boston and New York. A lot of sports profitability is driven by broadcasting deals, and in the NHL it’s team by team — there’s no NHL-wide broadcasting deal. So the Hartford market is just not competitive.

People forget that the reason the Patriots got Massachusetts to help fund Foxborough was they threatened to move the team to Hartford.

Even though there are die-hard fans, one of the most popular and best logos, and the Brass Bonanza goal-scoring song, I don’t think the NHL will do it. The community would embrace an NHL team, but I don’t see it happening.

James Kaplan: Okay, follow up question, as I’ve noted too many times. Whalers really refers to either a type of ship or a person on the ship. Hartford is a long way from the seashore when, if it were up to you, when they moved the team from Boston. Boston Whalers, that makes sense to Hartford, should they have renamed the team as the Hartford actuaries.

Rich Isenberg: Probably, but then it would’ve left the city even earlier the rest of the insurance companies.

James Kaplan: I mean, Hartford Whalers makes about as much sense as the Utah Jazz.

Rich Isenberg: Yeah, that’s a true story. Yeah.

James Kaplan: All right. Well, Rich, thank you very much.

Rich Isenberg: Oh, this was exciting. I’m excited to listen when it comes out.

James Kaplan: All right, terrific.

Rich Isenberg: Thanks.

Thanks for reading Prosaic Times! Subscribe to receive new posts.

* Traditional enterprise applications (and systems of record) only capture decisions, not the rationale for decisions

* Agentic applications will benefit from decision traces -- “the exceptions, overrides, precedents, and cross-system context that currently live in Slack threads, deal desk conversations, escalation calls, and people’s heads” and drive decisions

* Giving agents access to decision traces, stored in a context graph, allows them to to make better decisions on a broader set of topics by drawing implicit knowledge locked up in your organization

* Agents can build the context graph dynamically, without a pre-defined ontology

Thanks for reading Prosaic Times — share this interview with a friend!

Kellblog predicted that context graphs would storm the market in 2026. STAC Research said that 2026 would be the year of the context graph. Flexrule countered that context graphs confuse memory with meaning.

Jaya and I sat down for a discussion about the article. Here were my takeaways:

* The concept of a context graph (and especially the decision trace) is an important contribution to the idea graphs provide more flexibility than other mechanisms in modeling a business domain

* There may be more impact in B2B markets than B2C markets -- there’s more tacit knowledge and less rule-based decision-making

* Ideally you’d ingest data from email and slack to capture decision traces, but that could make employees comfortable; privacy will be essential here

* Instrumenting agents to capture their deliberations may be an important source of insight

* Ontology creation will be a dialectic. Experts will provide a starting point. Capturing data in real time will evolve the ontology so that it can accommodate new data -- and experts will tune it further

* Traditional enterprise software may not be dead yet -- some of them have large customer success and implementation teams that have insight into decision traces themselves

James Kaplan: Hi there. This is James Kaplan. Welcome to our second video podcast for Prosaic Times. We are here with Jaya Gupta. Did I pronounce that okay?

Jaya Gupta: Close.

James Kaplan: Jaya Gupta, excuse me, from Foundation Capital, and we’re going to talk about context graphs and decision traces and what that means for the future of the software business and the future of the enterprise application business.

How does that sound?

Jaya Gupta: I’m excited. Let’s do it.

James Kaplan: Do you want to just briefly introduce yourself?

Jaya Gupta: Yeah so my name’s Jaya I’m a partner at Foundation Capital and a little bit on Foundation Capital. We partner with technical founders from Day Zero to build enduring companies and we believe that context graphs are a trillion dollar opportunity in AI and the layer that captures how decisions get made and turns out reasoning into competitive advantage Prior to foundation I spent some time at McKinsey and super excited to be doing this podcast.

James Kaplan: And you’re talking to us from the Bay Area today.

Jaya Gupta: Talking from the Bay Area I’m based in San Francisco.

James Kaplan: Okay. No snow on the ground,

Jaya Gupta: No snow on the ground.

James Kaplan: no snow on the ground, no ice. Everyone’s able to get to the office.

Jaya Gupta: Exactly.

James Kaplan: Exciting. Very different experience than we’ve had here on the East Coast the past couple of days. So. Congratulations. I, I, I think you won a significant part of the internet in December with your article about context graphs and decision traces.

Agents can read data and take action but they don’t know why decisions get made.

I certainly found it super interesting, always being interested in graphs, but I was wondering if you could just describe a little bit of that article for us. How’s that sound?

Jaya Gupta: Absolutely. So I think if you take a step back you know 2025 was supposed to be the year of AI agents and I think the models did actually get a lot better.

But in enterprises, agents still don’t act and you’d expect them to. And I think the reason’s actually pretty simple.

Agents can read data and take action but they don’t know why decisions get made. And I think that reasoning is what we call decision traces. They are scattered across tools buried in Slack or other communication platforms.

And basically sometimes it’s never really even recorded. And so we think that the winners will be the companies that capture those traces and turn them into context graphs.

James Kaplan: Give me an example. What is a decision trace?

A decision trace is really an example of how a specific decision was made and what inputs were considered what rules were applied and sort of what exceptions were granted: who approved and what that outcome was.

Jaya Gupta: It’s a great question. Player Zero — they do you know incident debugging and solve technical support tickets. A decision trace is really an example of how a specific decision was made and what inputs were considered what rules were applied and sort of what exceptions were granted: who approved and what that outcome was. And so in Player Zero’s case they’re creating a decision trace by you know. Taking examples of you know what are the incidents that we’re receiving from PagerDuty. What are the open escalations in Zendesk. What are you know where are examples of maybe prior renewals and stitching that data across to understand what is it that you know a person or employee decided with different pieces of data why

James Kaplan: So in, in this case, we’re, we’re talking about sort of a, you know, to use an old fashioned term, it service management. We’re going to use that in, you know, incident management. You know, you have all sorts of decisions there. Why is something a severity one versus a severity two, why do you take down a particular system?

Why do you roll back a change? What’s some of the data that you would look at in making a change and then what are some of the decision traces to your thinking that lead to decisions like those?

Jaya Gupta: Yeah, and so that’s a good question I think in every single application it’s going to look a little bit different.

James Kaplan: Let’s, let’s use this example. Let’s use the incident management example, right?

Jaya Gupta: Let’s do that. And so on on Player Zero for an incident ticket it would be sort of what did the human decide when there’s an incident kind of what is the next actions that they took.

Or based on prior incidents what is it that the human decided to make a change on. Or what you know what they slacked someone and you know what they kind of communicated to someone. They might have communicated an exception for a certain incident And so it’s kind of replaying that past history.

James Kaplan: Right. So it could be, the decision could be where an incident gets routed it. A decision could be whether to reboot a server, it could be one of probably hundreds of different things. And if, I’m assuming, if I’m understanding the logic here, you would have an agent look at a combination of contextual data, which would be environmental data about the system, about the application.

Logs, what have you, as well as, you know, some of the Slack information because very well, you know, a particular person handling an incident may have slacked the expert on this particular technology or this particular domain saying, Hey, gee, I’m thinking about whether to reboot this server. Do you think that’s a good idea or not a good idea?

Is that, is that the gist of it?

Jaya Gupta: Yes, exactly. And I think it’s it’s a little bit of figuring out what was actually true at that time. Exactly as you mentioned. What was the evidence that was consulted? Was it ticketed episodes? Was it dashboards was it different links what were sort of the constraints evoked and so this might be run books in this example? It might be as you said in Slack the rationale whether it’s could be in the future in agent-generated justification or it’s a human note and the action taken as well as the approver chain.

What was the evidence that was consulted? Was it ticketed episodes? Was it dashboards was it different links what were sort of the constraints evoked and so this might be run books in this example?

So I think there’s even a lot to learn from who you communicated to and in certain incidents you know you talk to certain different people that have that tribal knowledge.

James Kaplan: You said something very interesting. It could be the deliberations of the agent. So I think one of the things I read you saying is this might not just be the slack messages between humans. This might also be the deliberations of various agents or the communications among various agents as they’ve made decisions in the past.

Is that, is that correct?

Jaya Gupta: Yes that’s correct I think that’s going to be the future state I think enterprises are probably far away from that world but I do think that that will become reality eventually over time.

James Kaplan: Well, I’ve been building something in Cursor and, you know, you watch the deliberations of the of the agent in the in the chat window there, and there’s, there’s a lot of information that could potentially be captured and used to determine the course of future events.

Is that a fair way of thinking about it?

Jaya Gupta: Exactly. Have you been using Cursor?

James Kaplan: Yes, I have. I’ve gotten addicted. Gotten addicted to cursor.

Jaya Gupta: I love that thing.

James Kaplan: Yes. Exactly. It’s very odd for those of us who you know, haven’t been doing frontline software development in many, many years. But the world has indeed changed.

Jaya Gupta: Exactly We’re all software

James Kaplan: There were two things I thought that were really interesting. A number of people have talked about making decisions based on the environmental information. The idea of the decision trace is — it’s one I certainly had not seen before.

And there’s two parts of it I suppose. One is, you know, your point about all the slack messages and email messages and instant messages, right? Do you think, and then the second is the agent deliberations, how much of a, how much of a tension do you think there will be between questions of privacy and the desire to really get at how people make decisions.

Jaya Gupta: That’s a great question I think that’s actually going to be one of the biggest roadblocks that because you know decision traces can leak judgment patterns And so the example I like to use is you know let’s just say I work at a you know Let’s just take some random law firm and I’m

James Kaplan: right?

Jaya Gupta: of other clients and you know let’s just say my client was some big manufacturer of Some beverage. And I want to you know I’m a first year associate at this law firm and I want a query and I’m on a different case or different you know client and I want to figure out what did that client do

But you know what did another associate or what did another lawyer do on another project. And so is there a way you know and I think you’re going to have to people are going to have to solve.

At inference time as well as at retrieval time permissions. And so I think that’s going to be one of the hardest technical challenge and and why you know if context graphs.

I think won’t come alive until we see solutions that can solve for the security issue And I think it’s going to be a really hard one and I think that the industries that are kind of like legal and consulting even and and anything with client services it’s it’s going to be tough to to implement.

James Kaplan: Yeah. Well there actually, it’s a great point. There’s a couple of different aspects of this. There’s a for want of a better term, a data segregation issue, right? Or a conflict issue, which isn’t just in professional services. Contract manufacturing has tremendous issues around that.

I was thinking a little bit in terms of employee privacy or employee trepidations. Okay. Let me give you an example. Let’s, let’s go back to incident management. Say someone sends, say, I send a slack message to Joe. Saying that you know, Frank suggested I reboot the server, I do X, Y, or z, and Joe sends a slack based back to me saying, don’t listen to Joe. Joe’s an idiot on these issues. Listen to Sally.

That’s actually valuable information or that may indeed be valuable information. Because there may be people on the team who are very smart in domains A, B, and C, but tend to get over their skis in domain D. That’s part of working in an organization to know which people are good at what.

But that can be very uncomfortable if we’re capturing that in any sort of systematic way.

Jaya Gupta: Absolutely And. I and I think you’re spot on and I think this is probably one of the big reasons why decision traces are going to take a while for them to work.

Because as as you’re saying your best employees want to be the best employees.

And and I think that’s also why it really really hurts when your best employee leaves and can’t find a replacement And I and nor can anyone else fill their shoes because they have all this rationale.

James Kaplan: Tacit knowledge, right?

Jaya Gupta: Exactly spot on. But I think the one of the key differences here is that with context graphs over time they’re kind of a byproduct of how the agents work.

And so when the AI agent solves the problem it’s naturally traversing the organizational state pulling the data from the CRM checking the incident history and PA and PagerDuty and looking at the support thread and Zendesk or whatever people use these days and so that traversal of what is it touching you know in what order.

And to solve what problem is sort of kind of how a sample of how an organization actually works.

And so I think that they emerge from these agent trajectories over time and that’s sort of the difference maybe that I think you’ll still have to model the entities upfront to some degree. But I think there will be applications that emerge where you’ll have to do less of that and maybe a behavioral ontology.

James Kaplan: Well, it seemed to me that the abstraction here becomes incredibly important, which is you don’t want, you know, you don’t want to be telling people Joe said to reboot the server, don’t listen to Joe. He doesn’t know what he’s talking about on this issue. What you want is to say, you don’t reboot the server in these circumstances.

Right. That’s not going to work. And that’s, you know, that’s at least analytically a solvable problem if having a certain degree of complexity doing that doing that at scale. Right?

Jaya Gupta: Yeah.

James Kaplan: And this does this work just as well outside of technology domains? Does this work for things like pricing and discounting and so forth?

Jaya Gupta: It’s a good question and and one that we are actually thinking through as well.

So I think that there’s two things that I think it’ll depend on and you’re kind of also hinting at my next post of what are the what are the companies that and and what are the features of the problems.

But I think that some of it is going to be a data maturity question of how ready are organizations there’s many enterprises that reached out that I talked to that you know are far behind this because they don’t even have something like Slack potentially.

James Kaplan: Right. Yeah.

Jaya Gupta: Most of their decisions are made at steak dinners.

James Kaplan: Mm-hmm.

Jaya Gupta: When you have some of those industries I think that it will will look a little bit different.

And and I think the second piece is in in pricing which is a great question.

I think it depends on pricing and what industry but in in software at least I think that it it can be done.

And the reason being is let’s just say you use your CRM and it has a deal that closed at a 20 percent discount and that’s the state.

But you know why that discount got approved. You can track those actions someone pulled maybe three recent service outages from an incident log they found a reference a similar exception that the VP made last quarter and so I think you can actually track back the reasoning chain.

But you know why that discount got approved. You can track those actions someone pulled maybe three recent service outages from an incident log they found a reference a similar exception that the VP made last quarter and so I think you can actually track back the reasoning chain.

James Kaplan: And you want to tease apart the circumstances where you give the discount because y the sales guy plays golf with the appropriate person at the at the customer versus you gave the discount because there were a few service outages in the last quarter, and you need to you need to buy back some goodwill.

One hypothesis I have is this will be more relevant in B2B markets than in B2C markets, right? Because in B2C markets, it’s simpler. Traditional machine learning analytics have progressed further. More of the data is structured. There’s fewer complicated decision traces around who gets a discount for consumer auto insurance.

Right. If consumer auto insurance, they, they look at me and say I’m a 55-year-old man who lives in New York City who drives a station wagon. Boom. Here’s what the appropriate price is, right?

Jaya Gupta: Yeah and what you’re getting at at to here is actually super interesting because I think decision traces in B2C have always existed if you think about Netflix.

James Kaplan: Yeah.

Jaya Gupta: Amazon, TikTok, Google know when we click anything or what what made us pause.

James Kaplan: Yeah, but the decision traces are all structured information.

Jaya Gupta: Exactly, exactly. And and I think the highest value judgment in B2B actually lives in what we like to call dark surfaces.

James Kaplan: Yep.

Jaya Gupta: Inboxes, side debates, unwritten precedent. And so there wasn’t anything to compound. But when you have agents you sort of have decision surfaces I guess that you can now instrument.

James Kaplan: The way I put it is: now for the first time, we can convert unstructured data to structured data at scale and economically at scale and therefore analyze it.

Jaya Gupta: Exactly, exactly. And I think that more of enterprise work now also lives on maybe what you would call instrumentable surfaces too.

James Kaplan: You assume there is a incredibly high end of B2B where there’s fewer decision traces because as you pointed out, it happens over dinner, right.

Where there’s, you know, and maybe there’s notes that are shared after the dinner, maybe there aren’t, as opposed to, you know, for buying software, a lot of that is going to happen over Slack and email and text and other communications and via RFP responses, other communications that you can potentially, potentially interrogate.

Jaya Gupta: Exactly, exactly and and I think it’ll also be another reason for I think some of the examples that I’ve heard are you know on the factory floor sometimes people need to write some rationale into something but they never do it because there’s no use of it.

James Kaplan: Right.

Jaya Gupta: There will be still some organizational change required especially and so I think there will be a there still will be some human change required

James Kaplan: So what is a context graph and in particular. Is it different from a knowledge graph? Is it a form of a knowledge graph? What does a context graph look like when you implement it?

Jaya Gupta: Yeah that’s a good question I would say it’s it’s definitely different than a knowledge graph.

And I would say the key difference is because you’re not building them intentionally but they’re more of a byproduct of how the agents actually work.

And I would say the key difference is because you’re not building them intentionally but they’re more of a byproduct of how the agents actually work.

And so you’re you’re delivering some form of value at the application layer.

And then that application is generating thousands of agent trajectories.

And so Cursor is a great example. They’re they’re generating thousands of agent trajectories every single time we use it. Every single time we accept or reject one of their suggestions. We’re learning something that we couldn’t have learned upfront —what are the entities that matter, what are the relationships real and what are the patterns that reoccur in successful decisions.

And so those trajectories are are going to be the kind of accumulation of those trajectories become the context graph

James Kaplan: And what, when you say trajectory, what is a trajectory to your thinking?

Jaya Gupta: Yeah so an agent trajectory is really what are the you know what touch points is the agent making.

And I would say it sort of has you know five dimensions.

And one is what is the timeline of that agen. What are the events that the agent is going through?

And then you also are baking in semantics into the agent of what certain things mean in the organization because you know what the term risk could mean at one company could be very different than another company.

James Kaplan: Mm-hmm.

Jaya Gupta: And then you’re also the I think one of the most important thing is you know you’re trying to figure out outcomes in this trajectory and in cursor the the it’s super fast because you’re you’re generating something every single time but when you’re looking at things like pricing decisions and you know sales workflows where you don’t know it is very hard to figure out why a deal closed or what actions.

James Kaplan: Of course. Yeah.

Jaya Gupta: ...to close a deal.

James Kaplan: So many people are working on that question at the moment, I’m sure.

Jaya Gupta: Thousands.

James Kaplan: Yeah. So how much does this relate to? I mean, there’s a lot of debate going on and there was a lot of debate going on after your article about RDF triples versus labeled property graphs and about whether you have a formal ontology or do not have a formal ontology with a graph.

Thanks for reading Prosaic Times! Subscribe to receive new posts.

It sounds like you are inclined towards the side of the debate, which says you have an emergent ontology rather than a predefined ontology. Is that a correct way to hear what you’re saying?

Jaya Gupta: Yes And I think this debate will continue to to go on but I I would say that’s the key distinction and why this isn’t just another metadata layer or you know metadata 3.0.

It’s more that you don’t need to hand specify the ontology of the enterprise upfront. You can learn it through it agent behavior.

It’s more that you don’t need to hand specify the ontology of the enterprise upfront. You can learn it through it agent behavior.

Now I think that is some in some cases if you’re doing an migration or of any software I think that to some degree you are going to have to specify a part of the enterprise upfront.

Now I think that you can do instead of doing the entire ontology upfront you can do a smaller portion. And so I think it will actually end up being a combination still.

James Kaplan: Please continue.

Jaya Gupta: I was going to say the more physical the world gets the more you will have to model upfront.

James Kaplan: Yeah, I tend to have a for want of better term, I call the dialectical view. On ontology development, which is, you know, I would suggest you start with a structure created by an expert. You gather data, the structure created by the expert is wrong. You start to create. You evolve the ontology in an emergent way.

You re then you reshape the ontology based on expert input that it will be, you know, in, in effect, a dialectic between the individual and the and the artificial intelligence. It sounds like you may not be in a terribly different place.

Jaya Gupta: Yes I I would agree I think it’ll be a combination of both. And I think the thing that is interesting about agents is that it lets you for the first time in for some of the behavioral ontology.

James Kaplan: I think, and I think that’s incredibly powerful because you know, you could argue that. Developing software and what, by whatever mechanism is always a form of abstraction. You have to model a business process. And there is necessarily some form of, some degree of abstraction required to model anything or else it would be fiscally impossible.

And the question is how much, you know, how much abstraction can you, can you engage in How much and what’s the trade off between the level of granularity you try to capture versus the cost and complexity you create? And I think what you’re suggesting is the use of agents to capture emergent structures pushes out that frontier in terms of what level of granularity and precision you can capture in your, in the model you build and manage.

Is that, is that fair?

Jaya Gupta: Yeah that’s spot on

James Kaplan: Okay, sure. Now, I, I think a lot of much of the reason or not much of one of the reasons your article drew so much attention is it, it has some probably pretty profound implications for the enterprise software space. Do you mind telling us about that a little bit.

Jaya Gupta: Yeah of course so I think that you know we we like to call incumbents I think that you know one of the reasons that It it’ll be a struggle for them to build con context craphs is they usually capture the what and most systems are record today you know they’re structure to capture the what not kind of what led to that and why And so I think of the examples of certain categories of software could be you know CRMs you know ERPs.

James Kaplan: Supply chain management. You can imagine the list, right?

Jaya Gupta: Yeah.

James Kaplan: Product lifecycle management, what have you, right?

Jaya Gupta: So I think that the way I the example I to to use is that there’s all these functions that have created and they have the word ops after all of them Funny enough RevOps, SecOps. DevOps.

James Kaplan: Now DocOps. DocCps is my new favorite.

Jaya Gupta: Say that again.

James Kaplan: DocOps. The process of creating of using a document as code mindset to create documents in a structured way.

Jaya Gupta: I like that one. And I think that you know these really exist because none of these systems or records really own the sort of cross-functional workflow. And a lot of this unstructured data that you’re synthesizing exists in what we used to call systems of engagement layers.

Most reasoning or most human reasoning I think actually happens over voice or it happens over video. It’s multimodal in nature versus happening on text.

And so what I like to say is most reasoning or most human reasoning I think actually happens over voice or it happens over video. It’s multimodal in nature versus happening on text. It’s the way we make eye contact. It’s the way the tone changes. And so I think those are some of the subtle you know differences as well in capturing those decision traces.

James Kaplan: Yeah, it’s interesting, I, I think many cases, decision traces aren’t in systems of engagement. They’re in spreadsheets and word processing documents and emails and what have you. And, for any, any number of enterprise applications, there’s a lot of work that gets done in spreadsheets before the data gets entered into whichever system of record.

Now let me maybe just play devil’s advocate. Let me turn this around in terms of the enterprise application providers. You could argue at least some of them have a ton of tacit knowledge about decision traces. A lot of that goes into how their platforms have been implemented in individual companies.

And they all have, many of them have significant professional services or customer success or implementation arms. And is there a case you could make that if these companies were sufficiently motivated or sufficiently determined, they could combine the information they capture about the end result with the tacit knowledge that may exist already in their organizations about how people tend to make the decisions that lead to the what.

Does that make sense at all?

Jaya Gupta: It does it does and so I actually do agree with you. I think there will be some incumbents that that will be strategic. Some of it will be is the professional services arm.

James Kaplan: How much of that that knowledge do they have within the four walls of the institution?

Jaya Gupta: Exactly, exactly. And what and it is also maybe a function of what sorts of data that they’re sitting on and it’s also do they own any system of engagement sort of companies too.

James Kaplan: Yeah. The other way I think about it, and this may be, I’m not sure if it’s orthogonal to what you’re saying or not, so I’m curious about your view is.

The amount of business domain content in the platform will matter a lot. There’s certain, you know, there’s certain types of software where they have domain content about tax laws and tax treatments or inventory rules in various parts of the world. And it would seem to me that will be much harder to replace than something which is just a database and some workflow and a UI.

Jaya Gupta: Yeah so I think this one it depends on the the how I guess private that data is to the enterprise.

I think for I think the reason you don’t have that many companies today that are you know automating migrations for example is because a lot of that information is private to the enterprise.

James Kaplan: Yep.

Jaya Gupta: And so you can’t really just build an agent wrapper to to do that quickly. You’re going to need a lot of you know fine tuning and a lot of specific to the company.

James Kaplan: Well, it’s inherently complicated information, right?

Jaya Gupta: That’s right.

James Kaplan: There’s a lot of nuance there. There’s a lot of implicit stuff that’s not written down.

Jaya Gupta: Exactly and I think where you you know maybe retrieving information on different tax laws in country in different countries I think to the extent that it’s public and on the internet I think that will actually get easier and easier.

James Kaplan: As it becomes easier to ingest data there’ll be less of a dominant advantage in capturing that type of information. That more people will be able to access that type of information because it will now be easier to ingest that information given GenAI.

Jaya Gupta: Exactly.

James Kaplan: Alright, so where are you taking this next? It sounds like this has become a major theme in your research and your thinking. Where, where do you go from here? What are the open questions and what are you, what are you thinking about going forward?

Jaya Gupta: That’s a so I think a great question. I think some of it is where’s the opportunity for startups.

And we have a a point of view on that and we’re already starting to get I feel thousands of pitches per hour and so our our view is is forming also on the fly as well.

I think some of it is who’s going to own it. Is it going is there going to be a universal context graph or are context graphs going to be verticalized some of it is

James Kaplan: And horizontalized. Right, because you can imagine one for it service management and one for marketing ops and what have you, right?

Jaya Gupta: Exactly, exactly. So either by function or by industry and so you know will the opportunity go to the app layer. What new info will have to be built I think coming back to our security question? Where will we find who will build kind of the layer that of security which governs inference and not just retrieval?

James Kaplan: Right. Yeah.

Jaya Gupta: And so sort of who who will capture the opportunity and as well you know I think the thesis for us here is you know decision traces are are sort of the the trillion dollar layer that maybe B2B never had and consumer B2C has also always had. And so how do you how how do you figure out creative ways to capture them? And I think we’re seeing a very different approach in in many different sorts of applications across industries.

I think the thesis for us here is you know decision traces are are sort of the the trillion dollar layer that maybe B2B never had and consumer B2C has also always had.

James Kaplan: I mean, sometimes I to talk about the semantic layer, but that may be, and I may be coming at a similar idea from a different direction. Especially in for businesses that have incredibly complicated data with lots of tacit data there’s been, a hell of a lot of challenge that’s limited digitization and limited automation over the past couple of decades.

And we may now now have the opportunity to go address some of that in a way that would’ve been impossible even two or three years ago.

Jaya Gupta: Exactly, exactly. And I think you know the semantic layers of the world there was you know months and years of workshops and and stakeholder alignment and and I think this actually makes a lot of that much faster too.

James Kaplan: Exactly, because you know, getting people to agree on semantics I will entirely acknowledge is incredibly tough.

Jaya Gupta: Yeah.

James Kaplan: Any final thoughts?

I think I mean you know I’m biased but it’s going to be a great time for for startups.

Jaya Gupta: I think I mean you know I’m biased but it’s going to be a great time for for startups.

And I think you know it’s it’s also a you know I think many enterprises they were some some of them are have been a little bit slow to adopt AI.

I think that I’ve seen from early reactions is that it’s gotten many many enterprises early in their adoption cycle to to move a lot faster

And so I think it’s going to be super exciting on you know one the all the large companies and enterprises in the world of you know maybe moving up adoption cycles And then two for for startups to capture some of the opportunity as well.

James Kaplan: Thank you so much.

Jaya Gupta: Thank you.

James Kaplan: That was terrific.

Jaya Gupta: Awesome.

* Attackers’ Resource Constraints: Industry’s “dirty secret?” Most vulnerabilities go unexploited because attackers lack the time or monetization methods. AI may soon allow them to industrialize these efforts.

* Structural Advantage for Defenders: Long-term superiority belongs to defenders because they possess the organizational data and context required to design for defensibility and resolve vulnerabilities more effectively than outside actors. But it will take a while for defenders to adapt for AI security.

* The Power of Defaults: Implementing “secure by default” configurations beats “secure by design” alone — nearly all users will maintain high-security settings, like MFA, if they are the starting requirement.

* “Shifting Down” Security: True enablement occurs when security, compliance, and privacy are bundled directly into the underlying technology platforms so that the secure path becomes the easiest and most cost-effective route for engineers.

* CISO Role Bifurcation: The position is evolving into two distinct paths: a Chief Digital Risk Officer focused on enterprise-wide risk management and a technical leader who modernizes infrastructure to built security into the IT fabric.

Thanks for reading Prosaic Times — share Phil’s wisdom with a friend!

The Transcript

James Kaplan: Okay. Good afternoon everyone, and welcome to the first podcast for Prosaic Times. And, we’re very lucky to have the first video interview with, I guess, cybersecurity eminence—cybersecurity legend. Which nomenclature do you prefer, Phil?

Phil Venables: Phil is just fine.

James Kaplan: All right, that is fine. Do you want to, briefly, introduce yourself and then we’ll get into it?

Phil Venables: Yeah, so hi everybody, Phil Venables. I’ve been a CISO at a number of organizations for many different years, as well as a chief risk officer in various other roles. I’m now a partner at Ballistic Ventures. We invest in seed and Series A level cybersecurity and some cyber-adjacent companies. So, it’s good to be here.

James Kaplan: All right. So, if a CEO were to stop you in an elevator and say, ‘How much is changing about information risk and about technology risk, and how should I be thinking about this differently than I might have even two years ago?’ what might you say to this person?

Phil Venables: Well, I think that the first thing is the ongoing trend that I’m sure everybody listening and watching is aware of. It is the ongoing—I’m not a big fan of the phrase, but it’s probably apt—digital transformation. This includes companies, the public sector, and the economy writ large, covering everything from automation and digitization of supply chains to moving to the cloud and adopting AI technology.

However, we’re also seeing widespread and much more sophisticated adoption of AI by attackers.

It’s just more and more of the same, but at an accelerating pace and with an accelerating expectation of the degree of value that’s obtained from that. A big part of that, which security has been a key part of for many years, is making sure that—I know it’s kind of clichéd but true—security is an enabler. It shouldn’t be a blocker, but at the same time, it shouldn’t be a passive observer if such transformation is going to create immense risk. The role of the CISO increasingly is to be in the middle of that business strategy, not just playing catch-up on adding security to every project they can find. It’s being much more in the heart of it.

We are currently in the middle of using AI to reimagine how business is done. In the past couple of years, much of what we’ve been doing on AI was a fairly naive application for routine automation and pilots. We’re now starting to see a reimagining of business and security processes using AI. We’re just starting to see the first wave of that, and that brings with it lots of opportunity.

However, we’re also seeing widespread and much more sophisticated adoption of AI by attackers.

James Kaplan: Attackers are reinventing their business processes.

Phil Venables: No, absolutely. I think the thing that a lot of organizations potentially underestimate is just how much resource-constrained attackers have actually been. The industry’s ‘dirty secret’ is that there are way more vulnerabilities that go unexploited than those that are exploited. That is a signal that either attackers don’t know how to monetize it or they just don’t have the time. Attackers using AI to industrialize their capabilities is worrying; it means more things must be defended more timely.

Against that backdrop, there’s a tidal wave of vulnerabilities coming at us because there’s more code generated by AI or engineers augmented with AI. Vulnerability management processes are going to have to be revolutionized. This isn’t just an AI versus AI story. High universal baselines of control—like multifactor authentication, segmentation, or high-velocity patching—become table stakes for everyone, not just the advanced. That complex dynamic is way more than a one-minute elevator conversation, but there’s a lot of stuff going on.

James Kaplan: I think history is important in thinking about technology strategy. In World War I, the combination of the railroad and the machine gun gave the advantage to the defensive. In World War II, airplanes and tanks gave the advantage to the offensive. Do you think the advent of GenAI advantages the attackers or the defenders?

I sound pessimistic in the short term because, in every wave of change, attackers are usually ahead because they have fewer constraints. They don’t have to fill out regulatory compliance forms or ensure the trust, safety, and security of their AI deployments.

Phil Venables: In the long term, I think AI advantages the defenders quite significantly, but defenders have to do the work to take advantage of it. I sound pessimistic in the short term because, in every wave of change, attackers are usually ahead because they have fewer constraints. They don’t have to fill out regulatory compliance forms or ensure the trust, safety, and security of their AI deployments.

Long-term, I’m more positive that this represents a structural advantage for defenders because they have the data and their own organizational context. They can design for defensibility and use the same technology to find and resolve vulnerabilities. Ultimately, the data, context, and environmental advantage of defenders outpaces attackers. But CISOs, CIOs, and CEOs must partner ever closer together to build technology environments where security is built in, not bolted on.

James Kaplan: You could draw a comparison to the early days of cloud. Initially, attackers had an advantage with misconfigured S3 buckets, but now we have a far more robust security model.

Hyperscale cloud providers have gotten better at being ‘secure by design’ and ‘secure by default’. They ship products with full safeties on.

Phil Venables: That’s right. Most organizations get a significant security and resilience upgrade by moving to the cloud. Hyperscale cloud providers have gotten better at being ‘secure by design’ and ‘secure by default’. They ship products with full safeties on. In prior roles, we actually implemented ‘loosening guides’ rather than hardening guides—products shipped with full safeties, and you only loosened them if it was too stringent. If a product is secure by design but ships with configuration settings wide open, it’s still exploitable.

James Kaplan: It’s like companies that require employees to opt out of a 401(k) rather than opt in.

Phil Venables: Exactly. We once pushed out a mandatory upgrade for strong multifactor authentication and then let people switch it off if they wanted. Turns out 99.9% of people kept it on and were just fine with it. If you introduce the default, most people take it anyway.

James Kaplan: Economists tell us that stated preferences and revealed preferences are sometimes different. I’m excited by the idea of using GenAI to interrogate exhaust data and modeling technology environments as a graph. Relational databases are tough for modeling these relationships, but there is a lot of ferment around building graphs to detect anomalies or vulnerabilities.

Phil Venables: It speaks to choosing appropriate data and AI models for the task. Many successful companies recognize that relationships and technology are graph-based problems. Large language models (LLMs) have seen successes, but also disappointments when they are applied to problems they aren’t suited for. Some tasks are better for traditional deep learning or even graph neural networks. The art is deploying multiple models and techniques, using agents to orchestrate outcomes rather than just naively deploying a chatbot to point at some data.

James Kaplan: That’s why I’m not sure machines will replace us yet; it takes intelligence to know which machine to use. You mentioned security being an enabler rather than the source of ‘no’. How do you make security an enabler of innovation?

The best security teams find a way to make the secure path the easiest path. They partner with engineering and AI teams to deliver platforms with built-in safety.

Phil Venables: There are two extremes of bad security: the team that says ‘no’ to everything and the team that says ‘yes’ to everything. The best security teams find a way to make the secure path the easiest path. They partner with engineering and AI teams to deliver platforms with built-in safety. It’s important to ‘shift left’ but also to ‘shift down’—bundling control, compliance, and privacy into the platforms people use. If you make the secure path the easiest and cheapest, you must also make the insecure path expensive and difficult. Sometimes a business unit might need to take a risk to avoid being late to market, but that path should be arduous and involve executive approval. A CISO who can mediate that in commercial partnership with business executives is highly regarded.

James Kaplan: I like to say ‘APIs, not binders’. Teams hate it when you show up with a three-inch-thick binder of standards.

Phil Venables: Right—it’s solutions, not policies.

James Kaplan: AI-enabled automation might help here. Historically, it’s been hard to get investment for underlying services over business functionality. But we may see a secular improvement in software engineering productivity that frees up capacity to automate security controls.

I’m skeptical of the view that software engineering jobs are at risk because companies have a massive backlog of ideas they’ve never implemented due to resource constraints. They are welcoming increased productivity to go after that backlog.

Phil Venables: I agree. I’m skeptical of the view that software engineering jobs are at risk because companies have a massive backlog of ideas they’ve never implemented due to resource constraints. They are welcoming increased productivity to go after that backlog. We should use thought experiments: if you had infinite people and automation, how would you reimagine what you’re doing? We could commoditize high-end capabilities like red teaming with AI and make them available to more companies.

James Kaplan: Let’s shift to the CISO role. I’ve tended to believe in an integrated Chief Technology Risk Officer (CTRO) role to reduce fragmentation. What is your view?

Phil Venables: My bias is that these risks should be brought together. My past CISO roles have included resilience, continuity, privacy, and compliance. Integrating these functions allows for common controls; for example, data annotation delivers lineage, privacy, and security. Without integration, you get conflicting parallel controls.

Outside of highly regulated banks, I see the CISO role separating into two: a leadership role to embed security into the IT fabric (perhaps a Chief Security Engineer) and a broader enterprise risk management role. CISOs are increasingly expected to be Chief Digital Risk Officers. You need both independent risk management and embedded engineering.

James Kaplan: I’ve long believed that in a traditional enterprise, it’s easier for that team to report into the CIO to influence the technology organization. If you think your CIO will prevent the CISO from being honest with the board, you should get rid of both.

The debate about reporting lines is often just a symptom of root causes like a lack of accountability or a wrong strategic approach.

Phil Venables: I agree. The debate about reporting lines is often just a symptom of root causes like a lack of accountability or a wrong strategic approach.

James Kaplan: What advice would you give to a first-time CISO?

Phil Venables: Don’t feel pressured to make a massive impact in your first 90 days. You should be visible, but that time should be a listening tour—working with peers, stakeholders, and the board, and walking the floor of your industry. It’s dangerous to try to change things without understanding the context.

When hiring a CISO, CEOs should know if they want a firefighter for the next 12 months or a long-term strategist. These are often two different types of people.

James Kaplan: In B2B sectors, how much time should a CISO spend with customers?

Phil Venables: I used to spend 20% to 25% of my time with customers. It’s useful to hear their perspective and what we should be building. I often used customers as ‘the wind at my back’ to convince colleagues that certain security features were necessary.

James Kaplan: Top three things to look for in a new CISO?

Third, you have to be technical. You don’t necessarily need to code, but you must understand technology so you don’t get misled about what can and cannot happen.

Phil Venables: First is the ability to collaborate and build relationships. Second is curiosity—you need to see around corners and predict what’s coming. Third, you have to be technical. You don’t necessarily need to code, but you must understand technology so you don’t get misled about what can and cannot happen.

James Kaplan: Anything else, Phil?

Phil Venables: I’m curious to see the trend of CISOs becoming CTOs and running infrastructure. If a CISO is concerned about lack of modernization, being given the task of modernizing for defensibility is almost perfect.

James Kaplan: It’s consistent with moving security down—security just becomes part of the platform. Thank you, this was fantastic.

Phil Venables: Always a pleasure.

Thanks for reading Prosaic Times! Subscribe to receive new posts.