phpMick's avatar

How to review AI-generated code

So I'm now creating more and more code with AI. Using multiple agent threads at the same time and doing both front end and backend with front end frameworks that I'm not familiar with.

How is everybody else reviewing this code? I'm finding that I have huge amounts of code that I need to review and feel like I'm missing things that I shouldn't. And sometimes I don't have a very good understanding of the code that I've created.

Any tips or workflows to help with reviewing this code?

1 like
2 replies
martinbean's avatar

@phpmick You should be in control of any and all code making its way into your codebase, whether that’s written by you, a colleague, or an LLM. Here are some tips and guidelines I follow when doing AI-assisted development:

  • Give agents very discreet tasks to complete.
  • When you’re prompting the agent, ask it to ask you about anything it’s unsure of instead of guessing. You’ll find you’ll get something far more in line with what you had in mind and were expecting, instead of giving an agent a loose description, and it making assumptions and making something that’s maybe 60% of what you wanted.
  • Give agents way to verify the work they’re producing. There should be a goal, as well as instructions on how to run any tool such as linting and testing tools. If linting/tests fail, the agent should go back and fix what’s broken before asking for your attention.
  • Agents should also be given guardrails to avoid getting stuck in a loop and burning tokens.
  • You should only be merging code you actually understand. If you don’t, review the agent output log. If you’re still unsure, then ask the agent to explain what it’s produced. As with human-produced code, less code is easier to grok than lots of code. Don’t have your agent spew out 50,000 lines of code and then review.
Tray2's avatar

While AI is a good tool to use in many tasks, writing code that you don't understand or know exactly what is does, is not only a risk for bugs, it might also be a security risk.

You can probably get away with not understanding all of the code it writes for a while, but sooner or later it will bite you in the ass.

The smarter the AI tool becomes, the better of a programmer you need to be to spot the mistakes.

A few years back it was quite easy to spot an AI generated image, it could have six fingers or something similarly visible, but now it is really hard to spot one, the same goes for code, it used to be easy to spot the mistakes made by AI, now it isn't as easy.

Please or to participate in this conversation.