My experience developing with AI

Building scalable distributed systems in Python comes with its own challenges, especially when AI is part of the workflow. Here are some lessons I’ve learned.

Be precise with AI prompts

Don’t overextend the usage of AI. Give it vague descriptions only when you’re experimenting or when you don’t have a clear picture of what you want. If you know a specific best practice, mention it exactly. Stating what not to do is just as important as stating what to do.

Know how to use AI agents effectively

AI agents are very capable, but you have to use them properly. If you know a best practice that applies, spell it out explicitly. The don’ts matter as much as the dos.

Keep the codebase modular

The codebase grows extremely large, making it hard to maintain past a certain point. Don’t give the AI a single monolithic file — the more you split code into files and folders, the better it is for both the AI and for testing.

Always add tests during development. Ask the AI to verify its steps through the test suite to make sure nothing is missed. Making code testable often means more boilerplate (though not always), and the codebase grows larger as a result. That trade-off is worth it.

Organize by feature

The way you apply design patterns (Gang of Four, etc.) and organize folders and files becomes far more important when working with AI. Well-structured code makes it easier for the AI to consume only the relevant files, saving tokens. It also helps you navigate and make changes.

The most optimal approach for my use case is to organize files and folders by feature. For example, if you’re working on an image processing module, put it inside a folder called ImageProcessing rather than a generic folder like ExternalAPIs. The names of folders and files should give you an immediate sense of what they contain.

Use feature flags

Instead of piling feature upon feature, put new features behind feature flags so they can be enabled or disabled at any time. This makes it easy to roll back and test different feature combinations.

Build a personal coding benchmark

Create a coding benchmark specifically designed to test AI agents on the kind of work you do. This makes it easy to evaluate new models as they come out and see which ones are most useful for your specific use cases. Make sure your requirements are deterministic and not vague.

Use Git religiously

Track your changes with Git so you can roll back when something goes wrong. Mission creep is real — you keep adding features and the scope keeps expanding. Git gives you a safety net.

---

That’s it for now. I’ll update this post as I learn more about developing with AI. I hope this is useful, and if you have any suggestions or feedback, please let me know.

Note: In fact, I used AI to convert my voice notes into text for this post, and I also used it to structure the post and make it more readable. So, it’s a great tool to use for writing as well, but you have to be careful about the accuracy of the content it generates. Always verify the information and make sure it aligns with your own knowledge and understanding.