WTF is a Software Factory?

And if you go ask the people in the labs, you're like, hey. Do you know how to do this? They don't. They don't know. They're not better at it than we are because we're the ones who actually know how our own systems work, and they don't. How do product people feel about that? The answer is bad. They feel bad about it because their their job is to tell engineers what to do and to be ahead of them. And you can't stay ahead of them because now they can produce outcomes at such a fast rate that you couldn't possibly keep up. That personalization layer is the sort of interesting newest frontier Wes it's like, oh, what if software got smaller, not bigger?

Welcome to Syntax. Today, we're gonna be talking with Adam Jacob, the CEO of Swamp Club, all about software factories.

Software factories, it's a Scott button topic, things that people either get a lot of stuff wrong with and or, maybe that just feels like a giant mystery to them. Like, what... How how could a software factory even exist? I've used AI models, and, well, I just prompt my way into getting something. Well, we're gonna dispel some of those myths. We're gonna dive in, and we're gonna understand all about what software factories are and how to use them.

Adam, welcome to Syntax. Hey. Thanks. Stoked to be here. So, Adam, you are the CEO of Swamp Club.

Would you mind giving us a little quick run through about what Swamp Club is, but then also,

what your background is? Yeah. For sure. Yeah. Swamp Club is basically a way to build AI agents with, determinism and and some structure that sort of allows you to sit underneath those agents and describe, hey. Here's what it is that I wanna do. And then Swamp Club knows how to take what you're asking it to do and then break it down into modular automation. So it knows how to basically turn those things into components that have typed inputs and outputs.

It knows how to then, like, wire those together into workflows. It knows how to refactor them. And when you ask it to do wacky things ESLint integrate with arbitrary APIs or internal stuff, it'll just figure out how to, like, hit that API and do the right thing. And it has all the things that you need in order to build these sort of systems quickly. So, you know, it has, like, shared memory that gets stored in a big data lake, and you can query it and all of those sorts of things. So people use it to do all kinds of useful stuff. So people use it to, like, run their production infrastructure, or they use it to do, like, wet lab science and control robot arms, or they, like... I use it to read my email.

Like, people do all kinds of stuff with it because what it's designed to do is basically take whatever it is that you need AI to do and turn that into deterministic processes that work

all the time. Yeah. Man, I've been doing a lot with this specifically.

So even just give the audience some more examples too about what I've been doing personally because I I use swamp. I have automations that, like, keep my pie and its extensions up to date. I have automations that check my folder structure for my sites folder. I don't know about US, but my sites folder turns into a graveyard of just, like, making projects in different places. And this will, using Jev, it classifies things and puts things in the right location.

That's cool.

Yeah. Well, cleans up work trees for me, cleans up old Rust builds that are just taking up space.

My... I I got so many of these things that I've been... Oh, I have one that I've I've mentioned before on the show where it's pulling in, Sentry bugs, cataloging them into, issues and tasks, completing them, and then, creating PRs for me. So that's some of the stuff I'm using it for.

I'm... I don't get it just yet. I know Scott's been talking about it, and I know that it's for automations. Yeah. But, like, maybe you can just describe... And and, like, people are probably listening right Node, like, being like, should I turn this off? I'm not quite sure if I understand. Like, where does this fit alongside somebody who is already running

Cloud Code or codex or any of these things? Yeah. Totally. So if you think about, like, the experience you have when you use Cloud Code. Right? So let's say you... Let's use let's use Scott's, like, cleaning up folders example. Right? Yeah. So if you just fire up Claude and you're like, hey, Claude. Clean up my site's folder. Here's the instructions.

It will do something.

Right? And maybe it'll do what you wanted it to do, and maybe it won't.

And then Scott will write a skill. He'll be like, I know. Here's the way I want you to always clean up my folder. And so he writes a skill, and he's like, hey. Here's the list of instructions about how to always, like, clean up my sites folder. And what's happening is he's basically spending Frontier intelligence tokens. He's, like, burning tokens so that Claude can, from scratch, reinvent that for loop over and over and over again. Right? And so what Swamp does is take that same prompt. Hey. I wanna clean up my Vercel folder. Here's the list of my criteria.

Maybe use JEV to classify it. And what it then does is take that intent, take that prompt, and turn it in to modular automation on your behalf. So it knows how to, like, then take that and be like, Okay. Great. I'm gonna maybe start by building a model that has inputs. Then the input might be what's the sites folder I'm cleaning up. And then maybe it takes an array of the criteria that he wants to measure it by. Right? And then it's gonna figure out how to actually go look at the sites folder. It's gonna write all the code that actually does that. It's gonna wire it together so you can call it, and it's gonna store a bunch of data about what it actually did. It's gonna be like, hey. What was the classification result I got back from Jeff? Right? And it stores that as information, which then is queryable for other steps in the workflow. So it knows how to take that intent and turn it into working code. The result is he's not burning Frontier intelligence tokens every time he runs it, and it runs the same way every time. Right? Which is the most annoying thing about AI, actually, that, like, it was really Scott, and it did the thing I wanted. And now, you know, I warp it to do it more often. And so the way that I make it do it more often is I just... I go to the newest model, and I pay more money. You know? I'm like, ah, if I run that skill with Fable, you know, it works most of the time. You know? It moved from working 80% of the time to working 90% of the time. And if you use swamp, it'll just work a 100% of the time. Okay. So you're taking...

And this is something I see all the time is that, like, instead of... I do it myself. Instead of, like, going and having a script written Yes. For for doing this task that I do every single day, I will simply just ask the model to do it for me, and and burn tokens on it. So you are...

We can still describe what we want it done in natural language, but then it it then turns that into a...

Something that you can run with just, like, a regular script. Yeah. You're gonna take that thing, and you're gonna tack the word with swamp on the end of it. And then what swamp is gonna do is break that down into modular automation for you. So, like, if you were to write that script yourself, right, you can totally just be like, write a Python script. You know? Yeah. And it'll write you one.

And then you have new interesting questions. How does that get refactored? How does it grow over time if it needs to? Right? How does that... Maybe he goes from having his site folder need to figure out how, how to classify itself to the automation that deploys his site's folders when they change. Okay. And all of those things can be connected models, and it's sort of building you this web of automation that's already modularized, that knows how to factor itself, that knows how to work in this context, and provides all the things you'll need as it grows. So you start out building these tiny little agents just for yourself, but over time, you wind up building bigger and bigger.

And if you want to see all of the errors in your application, you'll want to check out Sentry at century.io/syntax.

You don't want a production application out there that, well, you have no visibility into in case something is blowing up, and you might not even know it. So head on over to century.io/syntax.

Again, we've

been using this tool for a long time, and it totally rules. Alright. The other thing about, like, using an AI model to run these types of stuff is that you can just feed it stuff, like, data that is messy or, like, not not perfect or not implemented in a specific way. You know? Like, I might I might have done something with a one API, and I wanna do the exact same thing with a totally different API that that looks different and authenticates different and all of that type of stuff. And that's what that's what I love about AI because I could just say, hey. Run it with this one, and we'll figure out how to do that. Can it JS it, like... Can it heal itself or, like Yeah. Yeah.

Go out to a model? Yeah. Yeah. Yeah. And it's roughly the same loop. Right? So if you think about it, like, what Warp is doing is building you, is building an internal component that understands what that thing is inside the context of the job. So if you think about it as I have a step, let's let's use Jev as an example because OpenAI announced the other day that they're launching a a decisions API that will be roughly JEV. Right? Yeah. Or Cloudflare already launched one. Right? So let's say that Scott wants to shift to try one of those models. Because he's using warp, all he has to do is say, hey. Make that a configurable option of that of that component.

And then what it will do is go refactor that one component to support multiple APIs probably with an input option that's like use dev.

Right? And so now, like, that step or use Cloudflare's version of the decisions API. Right? And so what's happening is you Scott of reverse the loop where instead of going from the... From letting the LLM be the thing that's running the outer loop, you say, hey. The LLM is now calling in to a deterministic loop, and then that loop is calling out to intelligence when it needs it. Right? And it's getting structured inputs and outputs. So because we've factored it so that the outputs are standardized, right, he knows what the decisions he needs to make are about that site's directory.

Now he can plug and swap them, and it spends less tokens and less effort in order to do it. Does that make sense? Yeah. Yeah. So

Swamp Club is...

Like, you... We... It's a paid thing. Right? We should we should say that. You sign up for it, and you pay per user per month. Yep. And then do you provide the inference, or does it work with our own inference?

Yeah. You bring your own inference to the party. So, like, it sits underneath whatever agent you're running, whatever harness you wanna run, whatever models you wanna use. Like, it plugs into those things and provides this behavior. Beautiful. Yeah. And you don't need to pay to use Warp, Wes.

You do in the end. But not Bos, like, just for experimenting with it, for building your automations. Yeah. Totally.

Cool. So, let's actually talk about software factories as a concept because people often talk about software factories.

And I see a lot of the dialogue is always kind of all over the place on this. Like, you still need to babysit these things. The software factory is just producing slop or whatever. Can you, dive into what a software factory is, Adam? And and just to, like, clear the air for people who might not have a good understanding.

Yeah. For sure. I mean, if you don't have a good understanding of it, like, let's just lead with some empathy. I think that's, like, broadly shared across industry. You know? So, like, I think if you asked... If we had five people on this podcast, they would give you five different answers about, like, what they think software factories are, which is part of why it's such a frustrating topic.

My answer and sort of the growing answer of our community is that, you know, if you think about your traditional SDLC, so if you think about everything you do to figure out, like, what goes into the product, how do you break that work down, how do you then use that work as a team sort of and flow through that process all the way through to production? When we talk about a software factory, what we're saying is that as much of that loop as possible, in the end, probably all of that loop other than inputs and outputs and maybe some spots in the middle, are things that agents handle for you. So when you think about, like, how am I thinking about putting work into the system and then how does the system keep itself stable? How do we produce good code? How do we produce code that makes sense for the system? How do we make sure it's it's aligned to the architecture? How do we make sure, that it doesn't regress, and it continues to work the way it's supposed to? All of that stuff is the stuff of software factories. Right? And so you can think of it as as a replacement if you think of your SDLC. You're taking as much of that as possible, and you're turning that into a behavior that an agent is running on your behalf.

And there's lots of different styles of doing that work because it depends pretty tightly on what the application you're building is. Right? So, like, what it is that you're trying to make the factory produce is is a very important piece of the shape and the details of the factory. Right? So no two factories are the same, which is part of why it's confusing.

And then the the other piece of it is that sometimes the inputs look different depending on the team. So some people are starting those factories, like, all the way at the very beginning where they're just, you know, putting in a bunch of tickets, and they're like, run these tickets to completion, you know, like a queue that's just getting... Having a robot sort of minted out. Other people like us, we're a little more, careful about what the inputs are. Right? We're... We keep a much smaller backlog or Node, and what we're doing is is standardizing the process by which the software gets produced.

But that's... Those are what software factories are. Yeah. I I think a lost on people in this conversation is often that a software factory, like you mentioned, is is not just like a a singular way of building a software factory for everyone. Right? Like, you you... The way you see it is software factories are

individual to the project, the team, and how how they work. Right? Yeah. Absolutely. Because if you think about software development, just think about engineering. Like, the reason that any given product is good, the reason that any given piece of software is good is because of the decisions that are being made about how that software works, about how it comes together, about its internal architecture, about how it looks and feels to the end user. Like, all of those choices, that is what makes your product good.

That's what makes any piece of software good. And it's... And our designs are unique. Right? Like, the way that we think about those products and the way we think about what they do, they are, by nature, unique to the people who are building it. And so when we start building software factories, what we're doing is we're trying to encode as much of that behavior and as much of that knowledge as as possible into the system so that we can go faster from intent to outcome. Right? And that the and that the factory is the thing that's responsible for the output in the middle. Right? And that's why the factory is bigger than just, like, here's how I prompt Claude. You know what I mean? That's why it's not just, here's my, you know, here's my bag of skills, and I do my best. You know? Because that structure is what's helping to inform the full shape of of that factory to make it safe to say, hey. Here's what I want the product to do and be, and then trust that the system will align the internals to the right outcome. And so it's always unique to the product you're building because the internals are unique. Does that make sense?

Yeah. Yeah. Totally. In in in in that same regard, there's a post from potato on Twitter. I don't know. Do you see, she did a video and a post about shipping 25, 100 PRs last month for Grok.

And she talked a lot about trust and automations being a part of it. Like, how do you trust the output of your agents, and and then how do you verify that trust, etcetera? So, I mean, that sounds to me a lot like a software factory. Is that... Do you think that that fits into that same realm? Yeah. It feels like it to me. Right? So, like, in our own experience,

the... Like, one of the things we have started doing in... With Warp that I stopped doing, I don't know, 1999, 2000 era, was building, like, really big comprehensive black box acceptance testing suites.

And, you know, we stopped doing that in the continuous delivery kinda era because it slows everything down. Like, if you have a big acceptance testing suite that had to be written by hand by every engineer who was ever putting in a feature to the product, right, you were just, like, tripling the amount of effort that every engineer had to do. Everything took longer for what felt like marginal gain. Once you have a factory that's that's responsible for aligning the software to... From intent to outcome, like, what happens if it breaks the product? You know? What happens if that contract gets broken, and you're not looking? And so the answer to that is black box testing. Right? I need to go from the outside in to understand, like, hey. Does the product still work? You know? Is it still doing everything you said it was supposed to do? Because no human was involved in this middle process. And so... And even when we were involved in the middle process, it's not like we avoided regressions. Right? Like, we should regressions all the time. But in order to get to the level of trust that you need for the factory to actually accelerate the team, you know, like, you gotta build those things. Yeah. It sounds like a factory to me because you're talking about those those those elements of, like, of trust, of repeatable automation. Right? Like, how do I get the system to behave the same way every time as we go through this process, regardless of who the engineer or the product person is that's putting in the pnpm. Right? Yeah. It's one of those things with software factories. I I, like, wonder if it's just like a straight up perception and marketing issue around the the term because I think if you ask most developers and you were to say, let's add some more deterministic

systems for validating that trust, or let's say, we want to standardize the process for something that happens over and over again. I think most people would agree with that being something that they needed. But in the same regard, a lot of people might look at a a software factory as just being like a slot factory. Right?

Yeah. Well, because if you're not using these techniques, then you're... And you haven't changed your SDLC, it's awful.

Like, if the way your team is working right now is, like, I've got 10 software developers, and we're doing, you know, Scott, and we have the same set of meetings we had last year or the year before, and we're doing the exact same work. Only Node, we're firing thousand line PRs that were written with Claude that no one understood that are poorly aligned to the outcome, and it's our job to, like, deal with it. And all we've done is just cranked up the velocity because we can wrap out the code faster. Like, that's like being stuck in the worst part of the ocean when you're surfing. You know? It's like big wave surfing, and it's like you just... You can never get past the breakers. Like, it's just gonna constantly murder you. And so, like, the way out of that process, paradoxically, is more AI, not less. It's more more automation, more structure, more... Having having more of the system get designed. And that's why it becomes a systems design problem. Right? Like, I I didn't come up with the software factory word, but it is what people are calling it. You Node? Like, I helped come up with, like, DevOps, for example, and I have regrets about how that work went out into the world. And, like, I'm sure the software factory people probably have some regrets Yeah. About how that metaphor has taken hold. But, like, you know, ultimately, that process is the critical piece. And so, of course, it feels like a Scott factory because your lived experience for the most part is that it is a slop factory. And so if somebody comes and says to you, hey. I have this technique that's gonna make the slop factory go even faster. Are you in? You're like, absolutely not. You know? Like, the thing that's killing me, I do not want more of it. You know? I have a question about,

like, specifically companies that have lots of engineers and are sort of, like, looking at, like, how do we tackle this? Right? You... Like, you just gave an example of, like, 10 engineers. But I I think about, like... I remember hearing a stat that, like, LinkedIn has, like, 300 engineers committing CSS every single day. You know? Like, just the scale of some of these things is... Like, what would you recommend to a company that has hundreds, thousands of engineers

of... Like like, how do you rearchitect? How do you even start to change how you work with this type of stuff? Yeah, man. That's a good question. And I I sort of believe that there's two emerging frontiers happening in AI. So, like, one frontier is the labs and all the things that they're doing. The other frontier is is the answers to these questions. You Node? Like, how do we JS as regular mortals, you know, like, receiving the LLMs, figure out how we're gonna build working systems that can still work, you know, first in the small and then in the large. I think the the true answer here is that there's very, very few people, maybe none, but very few, who I think have figured out how to do that, organizationally.

The reason is that it's less about technology.

I'll get to that in a second about how how I think I would run a project to try to make that transition. Mhmm. But the hardest part is that that transition sort of breaks everything about your existing work process, about the points of control. Right? If you're a product person, and you have a bunch of engineers who have a trustworthy software factory, they're... They could implement whatever they want to implement. They could refactor whatever they want to mind without having to ask you questions and and without having to need your permission. And so, like, how do product people feel about that? The answer is bad. They feel bad about it because their job is to tell engineers what to do and to be ahead of them. And you can't stay ahead of them because now they can produce outcomes at such a fast rate that you couldn't possibly keep up. And so now your job is to look at it exclusively from the end.

Your job is to, like, review the outcomes of your engineering teams, try to align them to the theme, and then and then because it's... The cost of rework is so low, if you don't like it, you can change it. That is such a weird organizational pivot that very few people have figured out. How do I restructure my organization in order to get these gains? One thing we learned from the last of these movements, which was sort of digital transformation, the cloud, all of those things, is that the first move is always the same, which is you need to have a small team of people who experience what life is like on the other side. So instead of saying, how do I transform, you know, my 60,000 software engineers to use software factories and thinking of it in the large, which is an intractable hard problem.

What you need instead is to find five and say, okay, you five. Take this slice of a real problem that exists inside our environment, and I give you permission and only you to just break all of the rules about how we do everything.

Like, you're allowed to break all the rules about how the org structure works and about how we plan work and all of that stuff. And your only job is to figure out what it feels like to build a software factory end to end that you trust and that ships at this rate of speed and that produces reliable output.

And once you have one of those things, what happens is those five people now understand both what it feels like to work in your current environment and what it feels like to work in the new. Right? The lean manufacturing talks about those moments as Kaikaku. Right? It's a it's a moment where some technology has come in, and it's forever changes the nature of the factory.

And, therefore, what we must do is redesign the entire factory around that new technology because it's it's now somehow dramatically altered the bottlenecks in the system.

So what you do is take a small set of people. You get those people to know what it feels like. Then those people's job is to then figure out how to translate that into the org ever larger. Now the org problem with that, where that usually goes to die, is you have, like, a VP or a director Yeah. Who doesn't like the answer.

And so, and so what happens then is they start to water it down, and we wind up back in the tough surf zone, you know, of like, hey. Now now we're all sort of drowning.

The advantage you have in this moment if you're in large organizations is the vast majority of them have a mandate now to alter in this way. So what that team needs to do once they get assembled is strike while the iron is hot and show working proof in code. The more you get mired into the philosophical argument about, like, well, is it... Is this good or bad for the org? Is it good or bad for the shape? The Wes things will be. But the more what you do is show working software and get working engineers to try it. You Node, like, one of the great outcomes of working this way is that everybody gets to be more creative.

And so people tend to love it actually when they get to the other side even though in the middle, they're like, oh, I don't know how I feel about it because, you know, because I really loved being a programmer the way I was a programmer two years ago. You know? Have you seen any large companies that have, like, at least started to take this on or or switched how? I... I've seen people who have started. I have seen no one who has finished. Okay.

And, that's... You Node? And I think that's true even amongst my peers. You Node? Like, there was an era... It's slowing down Node. But I would say six months ago, I was averaging three calls a week from friends who, you know, my company was failing at that moment, six or eight months ago. So I was in the I was in the position of having to wrestle with this, but not having customers or, you know, not having any of those things sitting on top of me. And all my friends who did have real orgs and real customers. Like, we were just calling each other and talking about the existential crisis we were all feeling. And I would say that, like, you know, largely, we're still in the phase where where everybody is trying to figure this Scott, and we're now starting to communicate with each other about what works and what doesn't, which is eventually gonna lead us down a path of, like, being able to explain to each other how this thing works and getting clear about what the right techniques are. But we're in the discovering right techniques phase. We're not in the this is a solved problem and people know how to do it and you're just not paying attention hard enough or whatever. Like, we're not in the we're not in the phase where we all know what this is or even that that we've had enough experimentation to know what works. Does that make sense?

Yeah. And I and I I I feel like you have such a good, like...

Your thumb on this type of stuff because I'm I'm just... I keep thinking about big companies like LinkedIn. And Yeah. Once they get this... Something like this in place, what do they do with... I just looked it up. They have 10,500 engineers. And and this might not be a a question people like to ask, but what do they do with 10,500 engineers? Do they start... Do... Can they possibly make something that... Now that they're... Everybody's running 10 times faster, can we possibly make something that is...

That needs this many people, or or do we just have less engineers?

I I don't I don't know. Nobody knows. Yeah. And here's my guess. My belief is that, is that the answer is Wes. They can.

What we don't know is what that is yet.

Because because what happens when you get this increase, like, what happens when a system changes that dramatically, when that order of magnitude jump occurs, everything breaks.

And so the question of, like, sitting from where we are now saying, well, what could those 10,000 engineers at LinkedIn possibly do to LinkedIn? Yeah. And my answer is a lot. There's a lot I don't like about LinkedIn. And there's a lot I Bos, I guess. Yeah. There's a lot they don't like about LinkedIn. You know? Yeah. And and I bet that once they fix those things, like, we'll we'll find yet more things that we can imagine doing with LinkedIn, and we will want them to do it, and they will want to do it. And so, like, you know, the history of automation broadly in in, I don't know, the last thousand years, regardless of whether it was was digital or not, has always been that it opens up the capacity. Everybody has this conversation that says, well, that'll be the end of whatever that job is. You know? And the result is actually just that that job changes, and we do more of it because it has to be capped by demand otherwise.

And so far, we have not found what the demand is for software, what the demand cap is.

And given that software is infinite, right, that given enough compute and power, we get to continue to make it, and it's Scott, like, a resource that, that that is finite. It's not like by LinkedIn shipping 10,000 more times, there's less LinkedIn to go around, you know, or, like, you get to ship less often because you took all the trees.

Like, I don't know what the cap is on human software creativity. It seems like it's just getting weirder and faster and that it will just continue to get weirder and continue to get faster, and and more and more of the system will have to adapt to the fact that that's true. So, like, you know, I don't know where it leads LinkedIn, but it leads LinkedIn probably to some wild places because everything that you couldn't do because you didn't have capacity or because you didn't have the ability, all of that is now swept away. I have no idea what the limitations are practically of what you could build at this point. Maybe they're infinite. You know? Like, you could decide to build a new operating system to run LinkedIn tomorrow, and I bet you'd be... Have a prototype done in a week. Yeah. I bet you could probably replatform it in two.

Mhmm. That's insane.

It's insane. Yeah. We're just sitting here right now, and we're looking at this, like, Vibe Node Adobe Scott that came out, and it's it's better than GIMP, which has been around for twenty eight years. It's way better than GIMP. Way better. We're just sitting there being like, I I had this feeling, like, a week ago, and Wes I have a feeling again. Like, when is this gonna stop?

The answer the answer is never. What's interesting about it is what nobody has done yet JS made me a better one. Yeah. Like, why... Like, is that still the right shape? What they did with it is be like, hey. Let's go rebuild the Adobe Creative Suite. That's cool because the Adobe Creative Suite was cool and because I didn't have access to it and because, you know, whatever, that was complicated.

But what's gonna be cooler is when what happens is you start thinking about, like, well, is that still the way I want it to work? Is that still the way I wanna think about this problem? I don't think it is. Though rarely ever is a better, faster version of the exact same thing the answer to Yes. To the... What the next version of something is. Yeah. Exactly. And so if you think about this move and the software factories as a creative outlet, that's the thing that's exciting. What's exciting is, hey. When you get to the other side and we stop being like, hey. Isn't it great that AI can make pictures, or isn't it great that it could rebuild the Adobe Creative Suite in a week by throwing tokens at it? And you're like, yeah. Yeah. Yeah. All of that's great.

Now tell me what the new... What's the new thing? Like, what's the... What is it you'll build that never... Nobody ever got to do? So, like, a good example inside Warp is I have a partner who... We worked together for more than a decade now, and we have this spreadsheet we've been carting around. That's how we do all of the, like, mundane business reconciliation stuff, you know, like bookings and revenue projections and that kind of stuff. Right? That you just, it's just stuff you gotta do in startup land.

And, you know, I built him an application with a little software factory that does the projections and that knows how to extend itself. And now what he does is use Claude as not a software developer to to extend that thing across all these different systems that we use. Right? It's like talking to NetSuite, and it's hitting the CRM. And he's, like, pulling in all of the, like, bookings to actuals and building the UI exactly the way he wants it. And then we're gonna use that to report to my board.

And that seems small, but that piece of software is the best thing he's ever touched in his life. He couldn't be more stoked. Now I don't know that he could hand it to y'all and be like, hey. Do you wanna use my software projection thing? Because it's, like, tied to Scott Club. You know? Like, it works. It's tied to our choices.

And I think that that personalization layer is the sort of interesting newest frontier Wes it's like, oh, what if software got smaller, not bigger?

What if it was... You know? I've been building so much personal software in the past, I mean, the past, six months, but in specifically the past, like, two months. The amount of little things that it's like, oh, this fits my needs so specifically, and I can just... And it is. It's it's perfect for me. I I did I did a a whole, Agentic to do system that I have with a hardware device, and people are asking me to, you know, open source it or make it available. And it's like, yeah. I could, but this is so specific to MindTap, and it's the perfect thing for me. Yeah.

It... It's it's just weird. It's wild. Yeah. And not to loop it back to warp, but I will for a hot second. Yeah. That's why what we build are primitives.

That's... Like, when we describe it, often we describe it as, like, a platform of primitives that you need in order to build this sort of stuff faster.

And, like, what is the platform of primitives that we need to build those personal applications? What would the Adobe Creative Suite look like if it was built on on some kind of flexible foundation that made those things make sense? And software factories are an example of how we align software to that shape. Right? Swamp, for example, ships a software factory kind of in one version of the world inside of it that knows how to break down tasks into automation problems for you so you don't have to think about it, and you just think about the intent.

What's that look like for Adobe? What's it look like for Riverside? You know? Like, I don't know, but I wanna see it, and it's gonna be weird.

And I'm I'm I'm into it.

Mhmm. Yeah. Yeah. It's so funny. I... As you're talking, I just think about all the stuff I'm currently using these automations for and just, like, I have a... An automation for editing my wife's podcast.

It now uses Cloud MCP. It uses Python.

It determines who's talking via the frequency of their voice and then, transcripts pulls out the um's and i's. Repeatable automation. I just run it all the time. Right? And that's, like, something that I'd be doing by hand in a linear video editor for eternity for me. I've been doing that for so long. And the fact that, like, that shape now has changed for me JS been... In in a repeatable pattern has been so good. One thing I'm really curious on your thoughts on specifically is that conversation around, like, considering your background and everything, testing in general. Right? We hear all the time about AI is just gonna write the worst Wes. They're just gonna slap out some tests. The test passing now becomes, unnecessary or anything like that. Like, what are your thoughts on, like, AI writing tests, test suites, and in general in this this current age of things?

Yeah. I think black box testing, which is basically taking your product as it would be deployed or as it would be used. So no internal knowledge of the system at all.

That just how do I confirm that the features of the product work the way they are supposed to work.

Those tests, when they're outside the context of the agent that does the software development. Right? So they're outside the context of an agent that's trying to change the software in some way, super valuable. Right? Because...

Especially when you get to the refactoring part of the loop, which is still here in this world. Right? It's not like it's not like AI makes it so you don't have to refactor. What changes is that you learn to refactor because you experience bugs over and over again in the same part of the product, and therefore, you learn that you need to refactor instead of sort of coming up to that knowledge through the code base, you know, by by trying to edit it and realizing it's brittle.

So those black box tests, I think, incredibly valuable in this era in a way that they were not necessarily viable in the previous. I talked about that a minute ago.

I think then you start to shift backwards and be like, well, how valuable are integration tests? And it's like, well, probably still relatively valuable. Right? Because knowing that all the different parts work together can maybe catch those bugs sooner.

You know, historically, integration tests also co commitment with the source Node, so AI is gonna be able to change the integration test in order to prove that the internals work.

That's probably still useful, right, because it's gonna catch regressions, earlier.

And then you get down to, like, unit ESLint.

And I've probably skipped some steps. Like, there's somebody who's really great at software testing that's, like, either my vocabulary or there's more zones of of testing that I'm missing. But, you know, for most of us, you back down to unit tests.

I think there's an open question about whether unit tests provide much value at this point.

Mhmm. And I think the open question there is that we're not often seeing AI write code that is functionally incorrect.

And if we're not looking at the internals, then the unit testing isn't really helping me align myself to the two higher code quality standards. Right? Whereas that loop of thinking through the Wes, like, if you ever did test driven development and you wrote the unit test before you actually wrote the code, there was a real interesting loop that happened there between you thinking about the outcome of that one method and then you implementing the method.

That loop, I think AI does not need. There was a good article that got posted on, on the Thoughtworks blog, about basically them going back and doing some research to see if they asked agents to do test driven development if it improves or hurt the outcomes, and it hurt the outcomes in the end. It was just... It was worse than, than not doing it at all.

So, like, is that test suite useful or not? I think it's an open question. My guess is it's more useful in some languages than others. For example, TypeScript, still very useful because that fast feedback loop of the types being caught, Rust is the same. Right? Like, Rust JS a glorious language for refactoring because the type system is so good that it catches a lot of those issues. That's a lot of the issues we used to catch with unit tests in languages that don't have strong types.

So, you know, maybe I still want unit tests if I'm writing in in languages that don't have strong type systems, but I don't care in languages that do.

And this is one of those places where I think we just have to do more experimentation together.

We're not quite at the part where we're ready to delete all of our unit tests in in Warp, but we're not far.

So I would say that, like, it wouldn't surprise me if if Paul Stack, who is the product owner and and engineering leader at Swamp Club, just deleted all the unit tests as an experiment in the next couple of weeks, that wouldn't...

It wouldn't freak me out, which is a strange thing to say. You know? If you ask me as a CTO for the last fifteen years that one of my engineers wanted to delete all the unit tests, I'd be like, yeah. No. We're not doing that. But now I'm like, I don't know. Could be fine.

We know worst case scenario, you'll ask an agent to rewrite all the unit tests, and it'll have them again. Yeah. Right. Get them out. Get them back in. Look at them again. Yeah. Who cares? Yeah. You know, we can literally run a test to measure and see what the performance is, and see if we get more or less regressions. And I think I think as an industry, that's precisely what we're gonna do. Black box Wes, for sure, super valuable. If you don't have them and you're heading towards software factories, that's probably the single most important thing you can start to do first is just build a black box testing suite so that it's safe to refactor.

And then that means it's also safe to build the factory, and you can sort of work backwards from there. Yeah. Yeah. It all makes so much sense to me given where things

are at right now in my personal experience.

I ESLint general, like, how how do you... In a factory, how do you validate that trust in a more deterministic way? I know it's different for every language pnpm every system. But, like, what are the, like, high level concepts that people should be thinking about when they're thinking about, like Yeah. Like, making sure that their output is any good? Alright. So black box testing is one. So I won't harp on it anymore, but that's that's that's a big one.

Another is, for most of my career, we've made fun of software architects.

So if you think about people who had that job title and they worked in, like, a bank, most engineers were like, you know, like architecture, astronautting, you know, all of that stuff.

The thing about software architecture is that it gives you a vocabulary for a fixed set of patterns that if you use those patterns, results in code that is supple to requirements.

Supple to requirements just means it's easy to change as we learn more things about how the... About the domain we're working in. You know? So as I learn more about the problem, I learn more about how I want the software to work. Is it easy for me to change the software so it works the way that I warp it to now that I've learned more? Right? That software architecture historically has been a thing that emerges for teams. So when we talk about the architecture of the software, usually what we mean is engineers wrote the software, and they put some architecture in. You know? Like, they built... They they they decided the shape of the house as they built the house.

And then backwards, they describe it to each other as if they had architected it. You know? As if I had, from on high, put all the Bos in place.

Now what happens is you actually are building it from up on high. Yeah. Yeah? Yeah. Like, you actually are saying, hey. I want a house.

And then the question of how should the house be constructed, what is its software architecture, is a question the agent must fill in. Yeah? And left to its own devices, let's just say it's bad at it.

And the reason is that what it tends to do is pick whatever it feels like in the moment is the right thing.

And so part of why you see this pattern of, like, I built a big thing and I just vibed it and then it fell over JS because there is no consistent architecture. Right? The software it generated was not subtle to requirements. It breaks. It's brittle. And that's why every time you add new features, it gets worse. So in software factory land, the other thing you have to do is commit to a software architecture.

And so if you don't have one, I would recommend domain driven design because what it gives you is a really rich vocabulary, which an agent already understands. Like, your agent's already read the book about domain driven design. So if you say, I feel like this program needs an Sanity corruption layer between these two components, the agent knows precisely what you mean.

And if that architecture is also expressed in a skill that says, here is an example of an anti corruption layer in my code Bos, in my language.

It will mimic that pattern. It will look for other examples of anti corruption layers. If you are listening and you're like, what is an anti corruption layer? It's just if you have two APIs and you want the right hand side to be able to change, to not have to worry about the left hand side changing, you build an anti corruption layer in the middle that says, hey. I will translate between the left hand side and the right hand side so that the right hand side doesn't need to care if the left hand side changes. Right? The fact that I can say that sentence and every domain driven design person listening to this podcast Node their head, knew exactly what I meant, that's the difference between software architecture and not software architecture. Right? And so that, it turns out, is critical to being able to trust the system because the the... You have to be able to express that architecture from on high.

So write a software architecture skill and then have all the code that gets generated be reviewed against that skill. Right? So whenever code comes out, you say, hey. Is this thing architecturally aligned to my principles? You know? And those principles will be unique to the application. They'll be unique to the language. They'll be unique to your domain.

Right? All of that stuff.

But that process, we've seen really consistently return good results.

Then adversarial review for other nonarchitecture requirements.

So for example, let's say you ship a SaaS service and you want to be WCAG double or triple a compliant for, accessibility.

What you'll should do is write a skill or find one that already exists that describes those standards.

And those skills can get really lean because, like, if you just ask an agent right now, like, what are the standards, it does a pretty reasonable job of telling you the standard. But, like, if you have specifics you wanna make sure are always true. For example, never allow two colors to be too close together in hue on a web page. Right? Which, you know, no engineer ever thought about.

You encode that now, and now you make it so any change that goes through the system always gets evaluated against accessibility requirements. And if it fails, then go back and redo the work. Right? And then every time you find a new seam in the product where it fails to meet the outcomes that you're looking for, you go back as an engineer and you tweak the factory.

Right? You don't say... You don't go back and be like, ah, the AI robots are dumb. You go, how do I change this factory so that this whole class of error gets caught more often? Yeah. It won't be caught all of the time. Right? It'll be caught, you know, some larger percentage than it was before, And then you tweak. So... And then and then the technology will change, and you'll change again. So for example, in that accessibility requirement, if you didn't change any forward facing code, I shouldn't need to run the accessibility tests or the accessibility adversary. And so I can use a model like JEV to be like, hey. Here's the change I made.

Here's the list of criteria by which I judge quality.

Which are the things that I need to use based on the nature of the change at hand? And suddenly, you've got this really adaptive system that you couldn't think of before. And the reason I say it is because what I wanna put in your brain is that that's engineering.

Like, what you're doing is engineering the system whose job is to build the program you want on the other side. Yeah. And, you know, that move is weird because you've stopped building the thing you wanted, and you've started building the machine that builds the thing you wanted.

And... But you're still engineering. And so back to Wes question earlier about, like, hey. What do all those engineers do? Man, there's a lot of engineering that can happen. And, like, how do I build the machine that builds the machine? It's just moved to this other level.

And so that loop of iteration, I think those are the core things that we... That I think everybody could do, that they should just do right now, which is, like, software architecture skills, especially in the greenfield, but also in the brown. Like, you probably do have a software architecture that you've defined. And so if you can write that software architecture and its vocabulary down, then suddenly you'll see that you can align the system to that architecture and to that behavior in a way you couldn't before if you do it consistently.

And then work on the machine every time you get you find classifications of errors that you wanna have be better. The answer is always go back, fix the machine,

make the machine better. Yeah. I could hear... I'm glad you you talked to just... Even just said, you know, make a skill for it in that regard. Because I could imagine people in the comments might be saying, like, why not just do skills for everything here? And I know you touched on it before, at the very beginning. But it... This isn't, like, replacing skills. This isn't repla... It's augmenting. Right? It's additive. Node. Yeah. I think the the way I think about that evolution is

at the beginning of this era, we all looked at what was happening with LLMs, and we were just like, they're gonna get smarter and smarter. And so we're just gonna keep throwing more and more stuff at it, and that the Node machine will sort it out.

And what we've learned is that that is, a, it doesn't sort it out all the time in ways that are reliable or good. B, it's also very expensive. Right? Like, it just costs a lot of money to just throw it at it. And so what we need to do is have skills that describe the things where what we're doing is applying judgment or what we're doing is applying taste or what we're doing is applying reasoning to the problem. And, of course, what we need is to apply the right context to the right time to get the right outcome of that piece of the process. But we have to think of it as a piece of a process, as a piece of the system. And once we start to think about that intelligence as a piece of the system, now we can start to build a factory that's reliable. Right? Because as an engineer, I can ask, well, what are the inputs to that piece of judgment I need? You know? What are the in... What are the outputs of that judgment? What are the outputs of that reasoning? And then how do I measure whether those outputs are right or wrong? And so a good example there is do they meet the standards for accessibility? That's essentially writing a test suite that describes accessibility.

It happens to be a test suite that isn't always right.

It applies, you know, it applies judgment based on my skill, but it's also an order of magnitude better than anything you've ever had before. Right? Swamp's the first thing I've ever shipped that has a 100% lighthouse score. Mhmm. Whoever got a 100% lighthouse score? You know? But Swamp does, and it's because I have an adversarial skill that says says don't ever let it drop, and so it doesn't. Yeah. You know? Totally. Yeah.

Man, it all makes so much sense. And I I think if people could just look at it like this, they would, they would really just understand that the, yeah, the the software factory, again, it doesn't just exist to throw out as much code as possible into the world. It's it's there as guidelines to output good code or code that works.

Good products. You wanna build great things. Like, it can't just be volume. Volume does not necessarily equate to good outcomes. And, man, volume can build some sick outcomes.

Yeah. You know? Yeah. Like like, once you have Photoshop, you'll also be sick.

And I I just... I feel like it... It's really important that, like, where we are as an industry right now is that this is all new.

And for most of us, we have requirements. You know? Like, I am blessed to have... Like, I run my own company, and I start my own startups. And I get to... Like, that means I get to go swim in these waters, and I get to focus on these problems, and I get to hang out with you, and I get to talk to all these people. And so, like, that part of my job is to gather that information and figure out how to distill it for other people and then try to build products that are good. That's great.

But that works because what happens is we get together and have this conversation.

Like, it doesn't work because I'm smart. It works because we are Scott, because you went and built some stuff. Because Wes was like, hey, man. I'd... What what happens when I build this thing in Python and it gets a little better? And I'm like, yeah. What if it was like this? And he's like, oh, I like that. But what if it was also like this? And, like, that conversation in the large, that's how we discover all of these patterns. And so if you're, like, you're new in the industry, you don't know that that's true because everything we... That came about that way, you didn't experience it already. Like, you don't remember that the Internet wasn't a thing until we made it a thing, and we made all these decisions that you live with now as if they were inevitable, but they weren't. They were just us talking to each other, figuring it out, and that's exactly the phase that we're in now. And because the labs are so big and because the technology is so strange and they've framed it in this weird way, we sort of we sort of freaked ourselves out from just hanging out as engineers and being like, yeah. Yeah. But what if we built it differently? Or what if we did this? Or what if we experimented that way? And, like, we're entering this phase of sharing that information with each other. That is... That's what's driving this forward. Like, everything I've said about software factories, I know that because I've been sitting in a Discord with dozens of other people building software factories, and we're telling each other what works and what doesn't. You know? We're like, oh, I tried that, and it worked. Oh, I tried this, and it didn't. You know? We did this, and it was awful. And, like, the more we do that, the more we figure this out and the faster we get out of that not... Nasty surf zone. You know?

Yeah. Man, Adam, you explained things so well.

I don't I don't have any more questions on the list, Wes. Do you have anything that you didn't get to? No, man.

It's a excellent interview. I appreciate you coming on. It's my pleasure. Anytime.

And like I said, I'm I'm... It it really is true that, like like, we... What you're doing is a great Vercel. Like, not because you had me on the podcast or whatever, but because, like Well, yes. It is. We are awesome. Yes. You are awesome. But but, yeah, because you are serving your community. And because, like, this conversation of, like, how do we do this and how do we learn from it and how does that move? Like, that is the place we need to get to. Like, there's a bit of power seeding that happened where JS engineers, we were like, oh, I don't know. I guess I'm a slave to Claude. Mhmm. You know? I guess I guess I'm just I guess I just push prompts now. And it's like, no. Absolutely not. And and if you go ask the people in the labs, you're like, hey. Do you know how to do this? They don't.

They don't know. They're not better at it than we are, because we're the ones who actually know how our own systems work, and they don't. And no amount of having been the people that came up with AI makes it so you know how LinkedIn works Yeah.

Or how JPMorgan Chase works or how Swamp Club works. And, like, you Node,

like, we're the ones who Node, so we gotta go do that shit. You look at these labs, and they're just throwing products at the wall because they have unlimited tokens and limited engineering and limited money, and they still don't know what the the product is for a lot of these things. You know? They certainly hit it with coding, but you can see them with the the personal agents right now. They're just trying to figure it out.

And it's not the software engineering that's the

the hard part right now. It's just trying to figure out what it is. Yeah. And that's a... That is a gift to software engineers everywhere. Like, that has always been the truth. You know? Like, if you raise venture capital money, you always will get asked, like, why isn't Google just doing it, or why doesn't Facebook just do it? And it's like, well, I don't know, man, because they can do anything they wanted. So if what they wanna do is destroy me, they will. You know? Like, if they turn their... You know, if they turn the evil eye of Sauron on Swamp Club and they're like, that Adam guy's got it. What I warp is... I want what he's got. Like, they'll just take it because what do I got? Yeah. But what I do have is this. Like, I can do this. I can come and be with you, and I can figure out what that practical outcome JS. And doing that is the way you find great products.

And it's and it's also the way that good engineers engineer.

And, like, I think that is actually the golden age of what's possible for us in this whole thing. And that's why the software factory thing is interesting because it helps move us more toward how do I engineer in this era because it's so fun to be an engineer. It was always fun, and it's still super fun to be an engineer in this era when what you're doing is adjusting to the constraints of the era. Do you know what I mean? When Yeah. When you're not stuck. Yeah. Man, you explained things so well. Thanks so much, Adam.

The next part of the show is the sick picks and shameless plugs. A sick pick is just anything that you're enjoying in life right now, something that gives you joy. Could be literally anything, media or physical.

Is... Do you have anything that is bringing you joy in life right now?

Oh, yeah. What's bringing me joy in life right now? Yeah. Okay. I have two things. Yeah.

So I switched to a Dell XPS 14, one of the newer ones with the Panther Lake in it running Tolinski.

Yeah. And everything just works, and that's bringing me, like, real great joy. Really? Is it is it this one? Yeah. Yeah. It's exactly that one. Are you... What do you... What Linux distro are you running on it? I'm running Amache on it. Alright. And, which was very close to the Arch Linux I was already running. I was running Arch and Hyperland and all those things anyway.

And it's like that has become incredibly more polished, in a way that is a delight, and also points the finger at this sort of new future of, like, you know, popping up a clod and telling it to fix my laptop. That's a delight. Yeah.

I think the, the other is I love heavy metal, and, like, you know, abrasive annoying heavy metal. And I would say that, like, I've I've once again been enjoying Lorna Shore.

So there's something about, Lorna Shore is awesome. Sort of slept on the latest Lorna Shore record, and,

had a transcendent moment on the airplane the other day where I was like, oh, no. This record was great, and I just wasn't in the right headspace to hear it. Okay. I don't know Lorna Shore. I gotta get on this, so I will... I'll put this on later. Cool. And and for shameless plugs, is there anything you want people to check out of your work?

Yeah. I mean, you should come check out Swamp Club, and you should for sure join the Discord.

Like, you know, if you've heard this conversation and what sounds good is being in a room filled with like minded people who are building stuff, like, that Swamp Discord is filled with people who are trying to figure out how to build stuff in this way, and they're just constantly sharing with each other what's going on, and it's super fun. So those are the two things I would plug. Like, obviously, you should come and and check out Swamp Club, but you should also come hang out in the swamp Discord because hiding inside there is actually the magic of Yeah. Whether you wind up using Swamp or you wind up not using Warp. Like, what you need is those people around you

helping you figure out how to navigate this world in a way that is, is moving forward. I gotta say, I've been a part of a lot of Discord channels. I probably am in, like, a 100 right now, and I should clean this up. And the warp Discord is the only one I really hang out in. Some Discords are like... They have all these weird, internal systems of how conversation needs to be had or there's too many, messages flying in or people are just coming in for here or there or whatever.

The swamp Discord is the one I get the consistently the most value out of and feel like the conversations there are... Like, if I if I have a question about...

Like, for instance, I just posted on there, like, how are people here using dev right now? That's, like, the one place I feel like I'm gonna get the most value out of my answers. So I I hang out there all the time. If you're like... Yeah. If you don't want another Discord channel or whatever, I get it. But this one is is really a great place to hang out on this type stuff. So a lot of really smart people with, good opinions. Cool. Well, that's all I Scott, Wes. You got anything else? That's it. Thanks so much for coming on, and thank you everybody for tuning in.