← Previous Next →

Are we being replaced?

AI is changing things

I think a better question is, "What and how much will AI change?"
Well, "a lot" would not be an incorrect answer.

Technological progress is rapid. In software development, developers have to keep up with the new. We are always learning. One of Amazon's Leadership Principles is "Learn and Be Curious". This is good advice, not just for software developers, but for everyone, because there will be a lot of the new to keep up with.

AI is performing some tasks better than we can do ourselves, similarly to how a backhoe digs a trench better that I can with a spade, or how a handheld calculator is faster and more accurate at evaluating mathematical expressions than a man or woman. A backhoe only digs. It is powerfully effective for that, but useless at everything else. This is why it needs a driver. The driver must know how to use it, what to use it for, and what not to use it for.

An AI can perform an extraordinarily sophisticated software-engineering task and then fail miserably at something we would consider trivial. It is also a tool that needs someone steering it.

Grounding, not raw intelligence

In this age, we probably have put a higher value on intellectual ability and not valued some other qualities as we should. An IQ test only grades a set of language and logical skills such as pattern matching. Some people are very good at these things, others not so good. One man could be a great empath and intuitively helpful to other people. Another man could be a mathematical genius, but useless in social situations. Whose brain works the best? The man with the high IQ, or the man who understands people? We still (most certainly) have the edge over machines. The ability to perform logical tasks (IQ) is not the issue. "Grounding" is one of the main issues. "Creativity" is another.

A calculator does not know the significance of a number. It is fast, and accurate, but it has no idea what it is doing. We do. We have bodies with limbs and five senses, through which we experience the world. This gives us an understanding of how the world works. Machines learn from records of real experiences - text, images, and logs. They don't experience anything first-hand. They can't, they are machines without a soul.

We infer cause and effect by intervening in the world. I drop something, it falls, I learn something about gravity. Machines can only learn about this if someone wrote it down.

They are useful within what they have been trained for and a waste of time outside of it. They fail because they have no model of the world to generalise from - only a description of the world. Put them into something genuinely novel and they will confidently produce something that looks like a good answer, but it probably will be wrong.

An LLM is optimised for coherent text, not guaranteed correctness. As in the 60s counterculture, "If it feels good, do it", but for an AI it's: "If it sounds good, write it."

Practically, this is why they cannot be trusted to work unassisted. They need to be directed by us.

Judgement and discernment are unique human abilities, and we need to exercise them well as we learn how to navigate in what is becoming a whole new world, and the LLMs can't help us with that. Only God can.

The story

Which brings us to the money and the 'superintelligence' narrative. All it takes is a story of a potential super-power, and enough people with the financial incentive to keep embellishing it, for a company's perceived value to become wildly inflated.

Consider the following about Anthropic. Although the IPO prospectus was filed confidentially in June 2026, Reuters has since reported its contents. (This is admittedly second-hand information).

A company with about $4.6B of revenue is talking up a $2T valuation. Do they mean to imply that superintelligence is imminent enough to make that valuation plausible? (When researchers are making such big forward strides behind closed doors that it frightens them, it sounds like what they are creating is pretty amazing.)

But I go back to when Altman stated that when he looked at the capabilities of GPT-5 he became frightened (Yahoo). From what he was saying, I expected GPT-5 to be an epic step forwards from GPT-4, but it was a huge disappointment. GPT-3 to GPT-4 was a much bigger improvement. At about the same time he was also discussing needing trillions of dollars in infrastructure and data centre investments to scale future models (Bloomberg).

In contrast, AI "godfather" Yann LeCun says that effective altruism (the philosophical movement obsessed with the risks AI poses to humanity) is "super toxic" and a "complete disaster". He also said, "Apparently people are having mental issues." (Fortune).

Asking for a slowdown

This really does not sit comfortably with me.

On 12 September 2026, Amodei published an essay, We Must Pace the Frontier, arguing that the industry needs to slow down and that models need independent monitoring (his essay). At the same time he was preparing an IPO at a valuation of $2T. The essay is not a call to stop, incidentally. Amodei argues explicitly that not building the technology would hand it to authoritarian powers, and what he proposes is pacing, third-party evaluation and regulation. Sam Altman and Elon Musk both publicly agreed with him (BBC).

In my opinion, calling for a slowdown while raising unprecedented sums creates an obvious tension. I could be wrong; it may be entirely in good faith, but others have read it even more harshly than I have:

"Dario makes the case to stop open source and concentrate enormous technological and economic power with Anthropic," wrote Chamath Palihapitiya, investor and co-host of the All-In podcast  BBC news

What I take from all this

I do not mean to say that further fast advancement in this technology is not dangerous. I just don't want to overreact. Amodei made the point that self-governance of AI is not enough (by a company like Anthropic, an open-source collective, or someone else), but I'd more likely trust the open-source community with the task of developing and governing AI.

Getting back to the question: will it replace us? AI is developing fast, and the Titanic comes to mind, but the danger is probably mainly to the companies themselves. They could be uncontrollably spending within the delusion of their own inflated valuations. They could be blindsided by a near manic form of FOMO.
Whether the alarmists are correct or not, we do need to be responsible. I mean, don't equip a humanoid robot that already knows how to do kickboxing with the ability to self-improve.

Replace us though? I think not, but I do think it will change what talents we value most. I think skills like computer coding, writing, summarisation, translation, predictive analysis (in stock), fraud detection, even medical diagnostics won't be valued so highly anymore, because a lot of it can be done by AI. However, empathy, relationship building, nuanced negotiations, creativity (original thought), high-stakes strategy, politics, abstract logic and "common sense", critical thinking, judgement and discernment will be highly valued intellectual skills. AIs either don't excel or fail miserably in these areas.

In other words, I expect there to be a lot of change. Whether that change will ultimately be for the good or not, who knows? Hopefully... but with change comes instability, which will make the near future a bit rough.

Getting close to home, as a Software Developer... we'll still need people who know how to code, but rather than coding, they'll be driving the AI, and putting their creative abilities to work much more than they ever have in the past.

Being fatalistic about AI won't help us. Having our eyes open will. Learn to use it, understand it. Adapt, and be the person who is capable, even in a small way, of helping to govern the development of AI, rather than be fitted for your tin-foil cap.

← Previous: Drive the machine Next: My workflow →