As an educator and AI researcher for more than 30 years, when I encounter a headline declaring that “AI will kill us all,” my first response is simple:
How?
That question is not intended to dismiss the risks. It is an invitation to explain the process.
The danger arose from combining a powerful tool, a poorly specified instruction and an operator who did not adequately understand what he had set in motion.
Recently, former Anthropic and OpenAI researcher Jacob Coxon resigned from Anthropic, warning that people developing advanced AI genuinely believe it could kill humanity by the end of the decade. Anthropic’s Alignment Science lead, Evan Hubinger, publicly supported Coxon’s warning and placed the probability at greater than 10 per cent. ABC News
Those are extraordinary claims. Yet the headlines rarely explain the sequence of events through which AI is expected to bring about our extinction.
At almost the same time, technology critic Ed Zitron was presenting a strikingly different view.
Appearing on The Diary of a CEO, Zitron described generative AI not as an emerging superintelligence, but as an overhyped and economically unsustainable industry. He questioned whether companies such as OpenAI and Anthropic could ever generate sufficient revenue to justify the billions being spent on computing infrastructure, energy and data centres.
Where Coxon sees a technology becoming dangerously powerful, Zitron sees an industry exaggerating its capabilities and selling promises it may never fulfil.
Curiously, both stories benefit from presenting AI as historically significant. AI will either transform every profession and perhaps destroy humanity, or its collapse will threaten the entire economy. In both narratives, AI occupies the centre of our future.
This does not mean that either Coxon or Zitron is necessarily wrong. It means that their conclusions need to be examined. What assumptions sit behind them? What evidence supports them? What sequence of events connects the technology we have now to the future being predicted?
A conversation with Ed Zitron on The Diary of a CEO
The history of the atomic bomb offers an interesting comparison. During the Manhattan Project, physicist Edward Teller raised the possibility that a nuclear explosion might ignite nitrogen in the atmosphere and cause global destruction. The scientists did not respond by announcing that “physics will kill us all.” They examined the proposed process. Hans Bethe performed the calculations and concluded that atmospheric ignition could not occur.
The concern was taken seriously because the consequences would have been catastrophic. But it was investigated through physics, not amplified through headlines.
Of course, nuclear weapons went on to cause terrible destruction. Physics did not decide to bomb Hiroshima and Nagasaki. People developed the weapons, governments authorised their use and human beings dropped them.
Likewise, it is not simply “AI” that poses a danger.
The danger lies in how people design it, deploy it, govern it, weaponise it and surrender decisions to it.
Even then, we should be careful about leaping from “this technology could cause serious harm” to “this technology will kill every human being.” Between those claims lies an enormous causal chain. We need to examine every link.
What capabilities would an AI require? Who would give it access to laboratories, weapons, infrastructure or financial systems? How would it overcome physical constraints? Which safeguards would have to fail? At what points could people intervene?
In my previous post, I reflected on The Sorcerer’s Apprentice: the enchanted broom that continued carrying water because it had been given a task without sufficient understanding, judgement or constraint.
The lesson was not that brooms are inherently dangerous.
The danger arose from combining a powerful tool, a poorly specified instruction and an operator who did not adequately understand what he had set in motion.
I reiterate that lesson here. When we use a technology without sufficiently understanding it, or without thinking carefully about what we are asking it to do, what exactly do we expect?
Before we judge the outcome, we must examine the process that produces it.
What should we teach students?
If the danger arises when powerful tools are used without sufficient understanding, then banning AI, or teaching students a collection of clever prompts, is not enough.
An actual picture of my last camping trip in South Australia (no AI involved). The AI produced an image for this section but it was “meh!”.
And to be clear…. we are talking about Generative AI here… that would be Large Language Models (LLMs) like ChatGPT and CoPilot.
A “prompting” interface is unique to LLMs.
Students need capabilities that help them remain thoughtful, critical and accountable while using AI.
1. Knowing when to use AI
AI literacy begins before a prompt is written. Students should be able to decide whether AI is appropriate for a particular task and recognise when it may undermine the learning, introduce unacceptable risks or replace thinking they need to do themselves.
2. Framing the problem
A system can produce a convincing response to a poorly conceived request. Students must learn to define the problem, identify constraints, clarify the intended audience and recognise what a successful outcome would look like.
3. Giving clear instructions
Prompting matters, but not as a collection of magic phrases. Students need to communicate purpose, context, boundaries and evaluation criteria and recognise when their instructions may produce unintended consequences.
4. Interrogating the output
Fluent language is not evidence of truth. Students must check claims, calculations, citations and assumptions rather than accepting an answer because it sounds authoritative.
5. Recognising uncertainty
Students should understand that AI systems generate probable responses, not guaranteed knowledge. They need to identify when an answer exceeds the available evidence and when expert or authoritative advice is required.
6. Detecting bias and omissions
Every AI response reflects choices in its training data, design and instructions. Students should ask whose perspectives are represented, whose are missing and who may be disadvantaged by the resulting decision.
7. Protecting privacy and intellectual property
Students must recognise what information should not be entered into an AI system. This includes personal data, confidential material, unpublished research and content they do not have permission to share.
8. Exercising human judgement
Students should be able to compare alternatives, reject inappropriate suggestions and explain why they made their final choices. Responsibility cannot be delegated to the tool.
9. Documenting the process
Students need to explain where AI contributed, what they changed, what they rejected and what remained their own intellectual work. Disclosure should demonstrate judgement, not merely satisfy a compliance requirement.
10. Working without AI when necessary
True capability includes knowing what to do when the tool is unavailable, unreliable or inappropriate. Students should still develop sufficient disciplinary knowledge to recognise errors, question outputs and perform essential tasks independently.
These are not merely technical skills. They are habits of critical thought, ethical judgement and intellectual responsibility.
Our educational goal should not be to produce students who can make AI generate impressive products. It should be to develop people who understand the processes behind those products and who remain capable of questioning, correcting and taking responsibility for them.
Still something not to ignore…
Headlines portray AI as either an existential threat or an overhyped economic bubble, but both positions demand closer examination. Technology does not act in isolation: its consequences emerge from how people design, instruct, deploy and govern it. For educators, this means moving beyond bans and formulaic prompting lessons. Students need to understand when AI is appropriate, frame problems clearly, interrogate outputs, recognise uncertainty and bias, protect sensitive information, document their decisions and retain the ability to work independently. AI education should develop critical judgement and responsibility, not merely the ability to generate impressive products.