Background AI Agents - The Spark

When I read this Ramp blog about how they built their background agents, it felt like somebody had looked into my head and created exactly what I had been imagining over the last couple of months. In the blog, they mention using Modal as a sandbox service, which reminded me of a blog by Restate on the same subject.

The Appeal

Why is the idea of background AI agents so appealing?

If I look at my current workflow, I mainly use the Augment code plugin in IntelliJ, which only allows you to do one thing at a time. I work around that by having multiple clones of the same repository, but that is not very scalable.

Augment does have remote agents, but it currently only supports GitHub. Since we use GitLab, that is not an option for me. Furthermore, since I'm using the agents inside my editor, they are not available when I'm away from my desk. I can't unblock or prompt them on the go through Slack, for example.

I would love to be able to start an agent on a random idea through Slack right when I have the idea. Otherwise, the clarity of the idea might be gone by the time I get back to my desk.

Having the agent work in the background in its own sandbox environment with its own copy of a repository also removes local file system restrictions—and therefore the "one task at a time" constraint—without the risk of running into changes made by another agent.

When an agent is remote, it also removes the 1-to-1 interaction limit (just me with an agent in a private chat). It is perfectly possible to have multiple people in a session contributing to the context and moving the task forward.

I'm all out of excuses; I just need to build this to optimize my workflow. Let's get started!

Building

I've been mainly working with AWS Serverless tech, but I'm looking to expand my experience with different technologies. Here are some initial ideas:

• CloudFlare workers for compute.
• Turso for main databases, a separate database for each organization/sessions/...
• Clerk for authentication, users, and organizations management.
• Modal for sandboxes and volume snapshotting (which will be important for session start performance)
• Restate for connecting everything in a durability way, retries & workflows.
• SpiceDB for authorization - I've been looking for an excuse to get some experience with it. This will also require a Postgres database; luckily for me, PlanetScale has launched a $5/month Postgres DB.
• OpenCode in server mode inside the Modal sandboxes to work with LLMs.

After some initial experimentation I found that the Modal SDKs are using gRPC, so I'll need to use a CloudFlare container wrapped with a worker to manage the volumes and sandboxes.

Let's see where this leads!

References

Why We Built Our Own Background Agent

The craft of engineering is rapidly changing. We built our own coding agent to accelerate faster.

Ramp BuildersZach Bruggeman, Jason Quense, Rahul Sengottuvelu <>

A Durable Coding Agent — with Modal and Restate | Restate

Resilience, suspendability, observability, human-in-the-loop, and multi-agent coordination, for any agent and SDK.

AIIgal Shilman