The impact of AI coding tools on software development

It has been suggested that using Generative AI coding tools like Github Co-pilot or TabNine could lead to a productivity improvement of 20% to 50% of a software developer's time. These numbers are likely exaggerated given the complex nature of software development, but even if they were correct for an individual developer; productivity improvements in most organisations will only be marginal without wider organisational changes. However, these tools can help you perform more complex work if combined with other effective working methods.

Measuring developer productivity

Measuring a software developer's productivity is tricky, and there's a lot of debate about how best to do it and compare it across individuals, teams, and companies. In the past, this was measured by the amount of code a developer wrote, which is often compared to someone working on a production line. Today, we often see story points being used as a measure of productivity, even though they are not suited for that.

Story Points were introduced to abstractly measure progress without time-based estimates. Using time estimates can lead to misunderstandings, pressures, and unrealistic expectations. Initially, Story Points aimed to size tasks relatively, but some converted them to real-time estimates, fostering unhealthy competition. For example, if Team A had 10 average Story Points and Team B had 15, it's incorrect to assume Team B is more productive. This can distort the purpose of Story Points. True productivity is delivering working software to users, not just having code on a computer.

Measuring developer effectiveness is more important

A better measure is developer effectiveness, which is measured by the developer's ability to deliver maximum value to the customer. Modern methods such as DORA and SPACE are increasingly used to track the frequency of releases and the number of defects in order to more accurately measure the effectiveness of the development environment.

How could AI impact developer productivity

The value of AI tools varies depending on the complexity of the task. AI tools are not equally good at all development tasks, and the benefits are not evenly distributed. While these tools can be useful for basic tasks and scaffolding, there is some evidence that they can reduce productivity on more complex tasks, due to the time it takes to understand and review the generated code.

Individual productivity versus team productivity

Team productivity is more important than individual productivity. It is less ideal to have one very productive individual on a team while the rest of the team struggles, especially under pressure. It can be beneficial for an individual to lower their personal productivity to help others on the team, as this can increase the team's overall productivity, increase confidence, increase knowledge, and increase resilience. Generative AI tools are only beneficial if they contribute to the team's overall productivity. Using the tools to automate tasks frees up time for more collaborative work on more complex tasks.

Effective AI generated code requires talented people with inherently human skills

The work that software engineers do is complex. It consists of problem solving in a messy, non-linear environment where there are no right or wrong solutions and where compromises must be made all the time. Attempts to neatly compartmentalise this work by adopting project management frameworks or tools such as low-code/no-code usually end in frustration because the work does not fit the models created or the tools do not cover all use cases.

If AI is used to write code, but the result is that it's harder to understand and maintain in the long run, the developer has lowered the overall productivity of anyone who has to read or maintain that code. Writing bad code faster is never a good outcome.

We cannot look at developer productivity in isolation. They work in collaboration with other people and processes. They are a link in a process of delivering value to users from ideation to release and measuring customer experience. It's rare that developers' ability to write more code, faster, is key for a company to deliver this value.

Improving the quality of AI generated code

AI tools don't know if the code they generate is "correct". They simply give the most likely response to the given command. This can lead to errors. Some form of test automation will be needed, but there is a potential for poorly generated code to negatively impact productivity.

AI will shift the focus for software developers

I am convinced that the growing use of generative AI tools in programming will not necessarily lead to a reduced demand for software engineers. On the contrary, I foresee that software engineers will be able to focus on more complex tasks.

This in turn will result in a significant increase in value for both companies and end users. However, the degree of improvement that can be achieved depends on several factors. These should be considered and properly implemented:

• There must be efficient software delivery processes that ensure close cross-functional collaboration.
• Fast and secure code releases should be possible through efficient deployment pipelines to get feedback as quickly as possible.
• There must be an ability to work together as individuals and teams to solve complex problems.
• The skills must be there to address problems in an empirical and incremental manner.
• High levels of team autonomy and self-organisation should be promoted.
• There must be easy access to high-quality company data.

Without these improvements, integrating AI tools into the process will see - at most - marginal gains in the amount of value delivered. But when these factors are taken into account, the use of AI tools can significantly increase the value that software engineering teams can deliver. Therefore, the impact will be that we need more software engineers capable of displaying curiosity, empathy, and problem-solving skills.

We're experts in building high-performing teams as well as remarkable AI solutions for our clients. Learn more about how we help organisations like yours to reshape operations, products and services like never before through the use of AI.