Assessing internal quality while coding with an agent

Erik Doernenburg is the maintainer of CCMenu: a Mac
application that shows the status of CI/CD builds in the Mac menu bar. He
assesses how using a coding agent affects internal code quality by adding
a feature using the agent, and seeing what happens to the code.

more…

Assessing internal quality while coding with an agent

Erik Doernenburg

Erik is an experienced technologist and software engineer. He's always interested in emerging technologies and software excellence.

This article is part of “Exploring Gen
AI”. A series capturing Thoughtworks technologists' explorations of using gen ai technology for
software development.

27 January 2026

There’s no shortage of reports on how AI coding assistants, agents, and fleets of agents have written vast amounts of code in a short time, code that reportedly implements the features desired. It’s rare that people talk about non-functional requirements like performance or security in that context, maybe because that’s not a concern in many of the use cases the authors have. And it’s even rarer that people assess the quality of the code generated by the agent. I’d argue, though, that internal quality is crucial for development to continue at a sustainable pace over years, rather than collapse under its own weight.

So, let’s take a closer look at how the AI tooling performs when it comes to internal code quality. We’ll add a feature to an existing application with the help of an agent and look at what’s happening along the way. Of course, this makes it “just” an anecdote. This memo is by no means a study. At the same time, much of what we’ll see falls into patterns and can be extrapolated, at least in my experience.

The feature we’re implementing

We’ll be working with the codebase for CCMenu, a Mac application that shows the status of CI/CD builds in the Mac menu bar. This adds a degree of difficulty to the task because Mac applications are written in Swift, which is a common language, but not quite as common as JavaScript or Python. It’s also a modern programming language with a complex syntax and type system that requires more precision than, again, JavaScript or Python.

[Screenshot of CCMenu showing a list of pipelines and their status]

CCMenu periodically retrieves the status from the build servers with calls to their APIs. It currently supports servers using a legacy protocol implemented by the likes of Jenkins, and it supports GitHub Actions workflows. The most requested server that’s not currently supported is GitLab. So, that’s our feature: we’ll implement support for GitLab in CCMenu.

The API wrapper

GitHub provides the GitHub Actions API, which is stable and well documented. GitLab has the GitLab API, which is also well documented. Given the nature of the problem space, they are semantically quite similar. They’re not the same, though, and we’ll see how that affects the task later.

Internally, CCMenu has three GitHub-specific files to retrieve the build status from the API: a feed reader, a response parser, and a file that contains Swift functions that wrap the GitHub API, including functions like the following:

func requestForAllPublicRepositories(user: String, token: String?) -> URLRequest
func requestForAllPrivateRepositories(token: String) -> URLRequest
func requestForWorkflows(owner: String, repository: String, token: String?) -> URLRequest

The functions return URLRequest objects, which are part of the Swift SDK and are used to make the actual network request. Because these functions are structurally quite similar they delegate the construction of the URLRequest object to one shared, internal function:

func makeRequest(method: String = "GET", baseUrl: URL, path: String,
params: Dictionary<String, String> = [:], token: String? = nil) -> URLRequest

Don’t worry if you’re not familiar with Swift, as long as you recognise the arguments and their types you’re fine.

Optional tokens

Next, we should look at the token argument in a little more detail. Requests to the API’s can be authenticated. They don’t have to be authenticated but they can be authenticated. This allows applications like CCMenu to access information that’s restricted to certain users. For most API’s, GitHub and GitLab included, the token is simply a long string that needs to be passed in an HTTP header.

In its implementation CCMenu uses an optional string for the token, which in Swift is denoted by a question mark following the type, String? in this case. This is idiomatic use, and Swift forces recipients of such optional values to deal with the optionality in a safe way, avoiding the classic null pointer problems. There are also special language features to make this easier.

Some functions are nonsensical in an unauthenticated context, like requestForAllPrivateRepositories. These declare the token as non-optional, signalling to the caller that a token must be provided.

Let’s go

I’ve tried this experiment a couple of times, during the summer using Windsurf and Sonnet 3.5, and now, recently, with Claude Code and Sonnet 4.5. The approach remained similar: break down the task into smaller chunks. For each of the chunks I asked Windsurf to come up with a plan first before asking for an implementation. With Claude Code I went straight for the implementation, relying on its internal planning; and on Git when something ended up going in the wrong direction.

As a first step I asked the agent, more or less verbatim: “Based on the GitHub files for API, feed reader, and response parser, implement the same functionality for GitLab. Only write the equivalent for these three files. Do not make changes to the UI.”

This sounded like a reasonable request, and by and large it was. Even Windsurf, with the less capable model, picked up on key differences and handled them, e.g. it recognised that what GitHub calls a repository is a project in GitLab; it saw the difference in the JSON response, where GitLab returns the array of runs at the top level while GitHub has this array as a property in a top-level object.

I hadn’t looked at the GitLab API docs myself at this stage and just from a cursory scan of the generated code everything looked pretty okay, the code compiled and even the complex function types were generated correctly, or were they?

First surprise

In the next step, I asked the agent to implement the UI to add new pipelines/workflows. I deliberately asked it not to worry about authentication yet, to just implement the flow for publicly accessible information. The discussion of that step is maybe for another memo, but the new code somehow needs to acknowledge that a token might be present in the future

var apiToken: String? = nil

and then it can use the variable in the call the wrapper function

let req = GitLabAPI.requestForGroupProjects(group: name, token: apiToken)
var projects = await fetchProjects(request: req)

The apiToken variable is correctly declared as an optional String, initialised to nil for now. Later, some code could retrieve the token from another place depending on whether the user has decided to sign in. This code led to the first compiler error:

*[Screenshot of Xcode panel showing a compiler error in GitLabProjectList.swift. The error reads "Value of optional type 'String?' must be unwrapped to a value of type 'String'".]*

What’s going on here? Well, it turns out that the code for the API wrapper in the first step had a bit of a subtle problem: it declared the tokens as non-optional in all the wrapper functions, e.g.

func requestForGroupProjects(group: String, token: String) -> URLRequest

The underlying makeRequest function, for one reason or another, was created correctly, with the token declared as optional.

[...]

Original source