AI Code Assistance: Are we talking about the same thing?

With AI Code Assistance, the entry to building software has never been lower. Building software is now so easy that the vast majority of people who are building software have been doing so for less than a few years. This majority is reflected in software and AI communities online and in feedback shared by users of AI tools. Feedback on the helpfulness of AI-powered code-assist is extremely varied and often at polar opposites. Where some will profess amazing productivity gains, others will share that they are more productive without AI assistance. Unfortunately a lot context is missing from these discussions which makes taking the feedback at face value unhelpful to the reader.

Building software is (very) roughly split into two phases though these are not necessarily sequential:

1. Prove that a thing can be built and attract users as quickly as possible
2. Build a thing that is maintainable and can be scaled across users and developers

There is different terminology used to describe each phase though some of it is unhelpful - both phases are very important and can deliver tremendous value. AI assistance can significantly benefit either phase, though the approach and application of assistance is different.

In the first phase, AI assistance significantly reduces the barrier to entry for anyone (experienced and non-experienced developers alike) to reach the goal. The pattern to success is in Phase 1 is to:

1. Define the system you're trying to automate. This might be via a prompt, a diagram, or a list of requirements.
2. Determine whether there are any parts of the system that are already solved by existing software libraries. Prompting can do this automatically, but your experience could help here too.
3. Have AI write the integration code to connect the existing libraries & systems together. Having it work is probably a good enough.
4. Have AI write the new or non-library logic. Add any of the novel parts of the system that are not already solved by existing libraries.

The secret sauce of software development is all of the libraries that someone else already wrote. Before AI, the barrier to entry required your knowledge of (or desire to learn) the programming language the library was written in. AI drastically reduces this and now the inexperienced developer can leverage the amazing power of existing libraries. Developers of all levels of experience have been, and will continue to, operate inside the first phase. All that's required is an idea and the determination to automate it.

At some point a software system might reach a level of value that it requires the second phase. This is however the minority of software systems. The second phase is costly, effort intensive, time consuming, and often extremely annoying or painful. As such, it's advisable to avoid the second phase unless necessary. There's a bunch of commonly used language that describes the second phase as more important or superior to the first phase. This is nonsense. The second phase cannot exist without the first. The first phase requires much less effort and, depending on who you speak to, is potentially more fun.

That said, if you have calculated that the immense cost of the second phase is worth it, then AI assistance can again help, however the barrier to entry is higher and the application of assistance is different. The pattern to success in Phase 2 involves the same steps as Phase 1, but with a few more steps:

1. Determine an architecture that suits. AI might help you, but you'll need to understand why you've chosen it. Existing patterns and experience are key.
2. Improve the efficiency of how you use code-assist. These steps are helpful in phase 1 but are mandatory in phase 2.
   1. Don't ask the LLM to do repetitive tasks (or things other libraries already do). Ask it to write a command or script that performs the repetitive task.
   2. Be clearer on context selection. Point the LLM to relevant files and code both in where to make the change and where to see examples of existing code.
   3. When undesired output is generated, revert the change and edit rules or system prompts to avoid the same result. Then re-prompt. Over time you want to guide the LLM towards producing what you except from a prompt, rather than having to constantly correct it. This might be similar to a code style guide. Consider context7
   4. Have the LLM write tests. Guide it (via system prompt) to write few assertions per test.
   5. Review every line of code that the LLM generates. Don't trust it, own the code. Don't spam AI slop and expect your colleagues to review it.
3. Know when not to prompt. Over time you'll learn the limits to the types of changes that the LLM can make faster than you. Some changes are still faster to do manually, or, some changes take longer to prompt than to do manually.

In the second phase, techniques that contribute to delivering quality software before AI, like continuous integration, automated testing and making small changes can also help to guide the LLM towards producing better output. Considering teams with access to senior developer talent still struggle with this, it's not surprising to see LLMs struggle with it too. Ultimately in Phase 2 we need a collaborative approach the enforces quality and as yet it appears that AI is not able to do this without expert human assistance.

Understanding these two distinct phases — and matching your AI approach accordingly — is essential for making sense of the wildly different experiences developers report with AI coding tools. While AI dramatically lowers barriers in Phase 1, the complexity of Phase 2 work means all types of software are not yet ready for AI to take the lead, and its easier to apply AI assistance to some types of software than others.