From DevX to BotX
How LLMs are re-shaping DevX into something different
DevX, Developer eXperience, is the field of software engineering that is concerns with how developers interact, think, update and generally work with code.
It includes tools, architectures and patterns that software developer use and interact with every day.
As our industry evolves, developers will craft fewer lines of code, but they will control more Bots, or Agents, or plain LLMs.
The tools and strategies that have helped human Developers works with code have been useful for Bots as well. But cracks are showing. And we need to adapt to a different way to develop code.
The following are the strategies that I saw working on codebases managed primarily by Bots.
Frequent testing and quality assurance
Bots are extremely fast at accumulating changes. If those changes are not continuously tested the risk is to build a castle of cards that falls over as soon as we hit CI or actually we run the tests.
What I saw working best, is to run tests after every change.
Bots are smart enough to figure out if the file they just updated made the tests fails, and they can easily backtrack (revert the changes) or rollforward (update the tests).
Bots have a strong advantage over Developers, they don’t care how long a test or lint run takes.
Nevertheless, it is not feasible to run tests and lint if they take more than a mere seconds. And this bring me to the second point.
Fast inner loop
A fast loop of: making changes, then see the results, then run the rest, then makes some more changes, helps developer.
The faster the loop is, the higher is the quality of the software and the faster new changes are produced.
For Bots it is even more critical. Developers have an hard-earned experience and intuition about what changes will be more dangerous. Bots lack this experience. Running the whole loop frequently make up for the lack of experience.
However, if the tests and the lint are too slow, this strategies are not feasible.
Lints tend to be fast already, Developer have optimize for years for a fast inner loop.
However tests, often time, tend to be slow.
Tests are usually split into integration tests and unittest.
Pragmatically, it is generally fine to run integrations tests seldomly - just before sign-off work.
While unittests should be run continuously while working.
A fast inner loop is made of unittests, integration tests are not included.
However, unittests are often slow.
Slow unitttests tend to fall on a mainly two categories:
Fake integration tests
Tests that wait on a timer
You can spot a fake integration test, because they usually require some other process to run.
If your unittests need a database, or redis, or kafka, or calling some API - then they are not unittests. They are fake integration tests.
If a test need to sleep and wait on some timer, then it will be slow. And flaky.
Removing these classes of tests and convert them in unittests is paramount for BotX.
As we move further into BotX, much more code will be generated. Unfortunately, Bots, unless specifically directed, are not great at keeping an elegant and consistent architecture.
Architecture that is needed to keep code maintainable and valuable.
Lint your architecture
The next step of BotX is to lint and test the architecture of your application.
Which file can import which other file?
Without putting constraints on the architectures Bots tend to find the easiest way to reach their goal. Unfortunately after few iteration everything depends on everything else. Files are huge. The codebase is hard to navigate. And it become difficult to maintain for both Developers and Bots.
The paradigm that I found working best is to loosely follow the hexagonal architecture.
Wrap IO (database, network, caches, queue) in abstractions and plug them into the business logic. Never communicate directly to IO if not using an abstraction.
In Django I tend to have the views (presentation layer) communicate only with the services (business logic) and the service communicate only with other services OR their own models (database/IO)
`foo.service` can import `foo.model` but it cannot import `bar.model`
`foo.service` can import `bar.model`
`foo.view` cannot import any `*.model` but can import any `*.service`
Similarly to tests, the lint of the architecture should happen at every update. This allow the Bots to catch immediately what is wrong.
Once you reach this point, you can let the Bots run wild in your codebase. The constraints will limit the technical debt that can be produced, dependency will not sprawl to wide and code will be reasonably tested.
Ensure coverage
The last guardrail I like to put in place is to ensure test coverage.
I don’t believe that a codebase should be 100% cover. But I believe that a codebase that could be easily covered by tests is better has a simple and reasonable architecture.
The goal is not strictly elegance and cleverness of the code. It is more making sure that the code is simple and obvious to follow.
Forcing high coverage forces the codebase to be modular. Each piece can easily be swapped out and tested both in isolation and together with other. Exactly what Bots needs to quickly produce more maintainable code.
I found that using dependency injection, together with the hexagonal architecture above, works really well and produce code that is extremely simple. Maybe verbose. But Simple.
Fin
As DevX evolves in BotX, I believe we will find ourselves design the policies that a swarm of Bots will need to adhere to. Without those policies, Bots will soon take cover most codebases that will quickly grow into a chaotic spaghetti mess.
While DevX is about providing tooling and infrastructure to other developers, BotX will be about creating the policies not drown and thrive in a code ambundant world.