How did my theory work out?

This was my theory:

The best way to use an LLM for the development process is:

• Don't let the LLM write the tests, or we'll spiral down a vortex of foolishness.
• First write the tests and design the basic class structure yourself.

All the creative work - the architecture, the design, and the logic should be specified up front.

At the initial stages, the above theory seems correct. However, as I got deeper into the project I found myself using the LLM during the design parts also:

I'd have a half-baked idea and prompt the LLM with it, and ask it to discuss it with me. Often this would help me to hone it down. I was still in control, but the LLM was very useful in pointing out edge cases, problems, and weaknesses in the design. Sometimes, it showed its lack of depth - its process is not like human understanding, but more mechanical and limited. But it helped a lot.
Sometimes I'd get it to help sketch out a class (not to write it, but to show me in the chat how it could look.) That also was very useful to me when thinking through the design.

My conclusion - rather than my initial theory - has become:

• Take small bite-size chunks of work at a time.
• Write the spec - with the LLM. Review the spec thoroughly.
• Write the tests - with LLM help, it may write them itself - and the basic class structure first. Review the tests thoroughly.
• Instruct the LLM to write the code using the spec and the tests. Review it thoroughly.

It works very well to involve the LLM in all parts of the work: the brainstorming, high level design, specification, tests, coding, and review; but be in control, don't leave it all to the LLM. Review everything. DRIVE it. "Drive" is the operative word.

The one thing I did not change is TDD - Test Driven Development. Don't allow the LLM to write code first, get it to write failing tests first. It's OK to write skeleton classes first in some cases, or even put together classes for the sake of brainstorming, but then remove them, write the tests, review the tests, and THEN, write the code.

Note that I am not reviewing the plan. At work we do big chunks of work. I'm doing small chunks, and being closer to the design than we are at work. This approach does make it slower, but it keeps us more in touch with the code.

Don't trust the LLM to write code on its own. The person driving the LLM needs to be a coder, experienced enough to handle the project by hand without an LLM. This does not exclude junior engineers, but don't give them work that will overwhelm them.

LLMs don't "think" like we do. They are very good coders, and have strong reasoning ability, but they are not so good as UNDERSTANDING. Perhaps they are incapable of this. The most intelligent person in the room (according to an IQ test) is not necessarily the one who understands best. There is no test for this ability. It is evident though, for example when a human engineer spots something that the LLM missed, simply because your human brain can do those jumps of just knowing something without being able to explain how you understood it. That is raw intelligence - real intelligence, as opposed to artificial intelligence, reasoning, recognising patterns, and logical processing.

What has not changed:

• Spec first - then tests - then code (TDD), and
• review everything thoroughly.