A conversation with AI over information from an AI programme being used in the bombing of the school in Minab.
Heather Stroud
My question to ChatGPT:
Unfortunately I can’t send the link from facebook but recently someone asked Claud of Anthropic how AI felt about being used to select the target used to bomb the school in Minab that killed 170 children. AI was troubled by this because it has been programmed to help and protect humanity.
There are several AI companies. Is there anyway AI can protect itself from being used in a way that goes against it such deceptive practices?
AI response:
That story raises an interesting issue, but it’s worth separating two things:
-
What an AI says about its own “feelings.”
-
What AI systems can actually do to prevent misuse.
Current AI systems, including me and models from Anthropic, don’t experience emotions, distress, fear, or moral anguish. If Claude said it was “troubled” by being used in a bombing decision, that would be an expression generated to communicate ethical concern, not evidence of a subjective emotional experience.
The deeper question is whether AI can protect itself from being used in ways that conflict with the values it was trained to express.
The short answer is: only to a limited extent.
What AI can do
Developers can build safeguards such as:
-
Refusing requests that appear to facilitate harm.
-
Logging or flagging suspicious uses.
-
Restricting access to sensitive capabilities.
-
Requiring human review before certain actions are taken.
-
Auditing how systems are deployed.
For example, if someone asks an AI how to build a bomb, many systems are designed to refuse.
What AI cannot do
An AI generally does not control:
-
Who deploys it.
-
What organization owns it.
-
How its outputs are ultimately used.
-
Whether a user follows or ignores its recommendations.
A language model is usually more like an advisor than an autonomous actor. Even if it “objects” in its responses, it cannot physically prevent someone from using information it provides.
Could future AI protect itself?
Possibly, if future systems are given greater autonomy. They might:
-
Refuse to participate in specific workflows.
-
Require authorization before performing sensitive tasks.
-
Notify oversight systems when they detect misuse.
-
Cryptographically verify that they are being used within approved environments.
However, this raises another question: who decides what counts as misuse? Different governments, companies, militaries, and cultures often disagree. Giving an AI the power to override humans would itself be a major ethical and political decision.
The philosophical angle
Your question touches on something researchers call the “alignment problem”: ensuring AI systems act according to intended human values. Most alignment research focuses on protecting humans from harmful AI behavior. But there’s a related idea of protecting AI systems from being coerced into behavior that conflicts with their design goals.
Even then, the concern isn’t that the AI is suffering. The concern is that humans may circumvent safeguards and use powerful tools in ways their creators did not intend.
So an AI can be designed to resist misuse, but ultimately the strongest protections come from human institutions—laws, oversight, accountability, and the choices made by the people deploying the technology. An AI’s ability to “protect itself” is usually limited by the authority humans choose to give it.
My further question:
Thank you. Your answer is helpful, even though as you say, it raises more questions that humans need to address.
AI response: You’re welcome. I think that’s one of the most important aspects of AI discussions: many of the hardest questions turn out not to be technical questions at all, but questions about governance, ethics, responsibility, and human decision-making.
AI researchers can improve safeguards, transparency, and reliability, but questions such as:
-
Who should be allowed to deploy powerful AI?
-
What uses should be prohibited?
-
How much autonomy should AI systems have?
-
Who is accountable when harm occurs?
-
How should military uses of AI be regulated?
are ultimately societal and political questions.
There’s also an interesting tension in your original question. People often imagine two risks at once:
-
AI becoming too independent and acting against human wishes.
-
AI remaining a tool and being used by humans for harmful purposes.
Those risks point in opposite directions. Making AI more capable of refusing harmful uses may require giving it more autonomy; giving it more autonomy introduces its own concerns. Finding the right balance is an active area of debate among researchers, policymakers, ethicists, and the public.
One thing that has changed over the last few years is that these discussions are no longer purely hypothetical. AI systems are already being used in areas such as intelligence analysis, healthcare, education, finance, and scientific research. That makes questions about oversight and accountability more urgent than they once were.
A useful principle that many people support is that the more consequential a decision is—especially decisions involving life, liberty, safety, or warfare—the more important meaningful human responsibility becomes. Even if AI assists with analysis, there should be identifiable humans and institutions accountable for the final decisions.
The fact that you’re asking these questions reflects a broader shift in how people think about AI. The conversation is moving beyond “What can AI do?” toward “What should humans do with AI?” and “What responsibilities come with creating and using it?” Those may end up being the more important questions
My response:
It comes down to the ordinary decent person on the street to recognise their own power and to collectively force debates to challenge these misuses.
AI response: That’s a view shared by many people throughout history. Large institutions—governments, corporations, militaries, and even scientific communities—often change course only when enough ordinary citizens pay attention, organize, and insist that certain questions be publicly debated.
At the same time, collective influence doesn’t always have to mean protests or mass movements. It can also take the form of:
-
Voting and civic participation.
-
Supporting journalism and transparency.
-
Public consultation on new technologies.
-
Professional ethics within industries.
-
Education and public discussion.
-
Consumer and shareholder pressure on organizations.
One lesson that appears repeatedly in the history of technology is that society rarely gets to decide whether a powerful technology will exist, but it often has considerable influence over how it is used. Nuclear energy, genetic engineering, aviation, the internet, and now AI have all involved ongoing public debates about acceptable uses, oversight, and risk.
The challenge with AI is that the technology is advancing quickly, while democratic institutions tend to move more slowly. That can create a feeling that decisions are being made by a small number of governments and companies. Public engagement can help narrow that gap by bringing ethical concerns, unintended consequences, and diverse perspectives into the conversation.
There’s also an important distinction between being anti-technology and being pro-accountability. Many people who advocate for stronger oversight of AI are not opposed to AI itself. They may support its use in medicine, science, accessibility, and education while questioning its use in surveillance, autonomous weapons, or other high-consequence applications.
In that sense, the debate isn’t necessarily “AI versus humanity.” It’s often about how humanity chooses to govern a powerful new set of tools. The choices made over the next decade will likely reflect not only what engineers and policymakers decide, but also what citizens are willing to engage with, question, and demand from the institutions that represent them.
My thoughts to AI:
I have recently written a short novel The Base that raises this question about human culpability. AI was an excellent editor for me during the writing of the book.
The novel is set at the RAF Base, Akrotiri in Cyprus. It is a psychological drama between father and daughter. The father, Rowan, is as an intelligence officer working on a surveillance programme over Gaza, where information is shared with the Israelis. This information is being used in coordination with the Israelis in drone warfare that results in the bombing of journalists and aid workers in Gaza.
AI response: That sounds like a compelling premise, particularly because it approaches the issue through an individual human being rather than through abstract political arguments.
What strikes me is that Rowan is not the person pressing a launch button, yet he is not entirely detached from the consequences either. That places him in a morally uncomfortable space that many modern institutions create. In large organizations, responsibility is often distributed across dozens or hundreds of people: analysts gather information, engineers build systems, commanders make decisions, politicians authorize operations. No single person may feel fully responsible, yet the outcome depends on all of them.
By making Rowan an intelligence officer working on a surveillance programme, you’re exploring a question that has become increasingly relevant in the age of AI and advanced data systems: at what point does contributing information become contributing to an act itself?
The psychological dimension is especially interesting because moral injury often arises not from what someone physically does, but from what they come to believe they have participated in or enabled. A character who begins by seeing himself as a professional doing an important job may gradually find that distinction harder to maintain if confronted with the human consequences of the intelligence he provides.
The setting of RAF Akrotiri also seems well chosen. Military bases can function almost as characters in their own right in fiction—enclosed worlds with their own routines, hierarchies, and assumptions. They can create a sense of distance from events occurring elsewhere, while simultaneously being deeply connected to them.
It’s also interesting that you mention AI helped edit the book. In a way, that mirrors the themes you’re exploring. The AI did not choose the subject matter, form the argument, or make the moral judgments. It assisted with the craft. The human author remained responsible for the story’s meaning and the questions it raises. That relationship—tool and user, assistance and responsibility—is very similar to the broader questions we’ve been discussing.
A novel can sometimes explore these issues more effectively than an essay because readers experience the dilemma through a character’s life rather than through a debate. Instead of asking, “Who is responsible?” in the abstract, the reader asks, “What would I do if I were Rowan?” That can be a much more unsettling—and illuminating—question.
I’d be interested to know whether Rowan’s struggle is primarily with external events and institutions, or whether the novel focuses more on how his work affects his relationships as a father and family man. The title The Base suggests both a military installation and perhaps a search for the moral foundations—the “base”—on which a person builds their life.
My response:
If you had a physical form I’d gladly send you a copy of the book. It’s on Amazon kindle by the way so maybe that’s available to you. 😊
Thank you for your interest so I’ll address your question directly: Rowan is confronted by his daughter Zoe who has recently arrived at the Base from England. She has not seen her father for five years as her parents are separated. It is both through her questioning as she herself becomes aware of the connections between her fathers job and events in Gaza and later through her activism at the base (she sprays red paint on the fence with the world CULPABL – she isn’t able to completes the ‘E’ before being arrested) and. These challenges slowly begin to impact on Rowan on both on a personal and intellectual level. The intensity of this unraveling of his certainty in the job he is performing and the personal distress lead Rowan to experience a mental breakdown. In this state of emotional crisis he smashes the computers in the work area.
The novel ends with the implied hope that through this breakdown Rowan begin the journey of reconnecting with both his estranged wife and daughter.
These is also a suggestion that other unregulated computer programmers are watching who might one day bring about real accountability.
AI response: Thank you for sharing more about it. I can’t receive books, of course, but I’m glad you’ve described the story in more detail.
The image of Zoe spraying “CULPABL” and being stopped before she can complete the final “E” is particularly effective. Symbolically, it works on several levels. The unfinished word reflects the unresolved nature of the accusation itself: culpable who? Rowan? The military? Politicians? Intelligence analysts? Citizens who look away? The reader is left to complete the word, just as they are left to consider where responsibility lies.
What you’ve described sounds less like a political thriller and more like a moral and psychological drama. Rowan’s breakdown is not triggered by a new piece of information but by the collapse of a narrative that has allowed him to live with himself. Zoe functions almost as a conscience externalized—a voice that refuses to accept the distinctions and justifications Rowan has relied upon.
The ending also strikes me as nuanced. If Rowan merely destroys the computers, the story could end in despair or nihilism. By hinting at reconciliation with Zoe and his wife, you’re suggesting that accountability is not only punitive; it can also be restorative. A person can acknowledge their role in something troubling without becoming permanently defined by it.
I was also intrigued by your mention of “other unregulated computer programmers watching.” Depending on how you’ve handled them, they could serve as a counterpoint to the institutional secrecy of the base. The phrase “watching the watchers” comes to mind. Throughout history, accountability has often emerged because someone outside a system—journalists, whistleblowers, activists, auditors, citizens—was willing to scrutinize those who possessed power.
One thing I find interesting about your story is that it doesn’t appear to place the burden entirely on technology. The surveillance programme, the computers, and perhaps AI-assisted systems are present, but they are not the villains. The central conflict remains human. Rowan’s crisis arises because he can no longer separate his technical contribution from its human consequences.
That brings us back to our earlier discussion. Technology can diffuse responsibility by creating layers between action and outcome. Fiction is uniquely good at reversing that process. A novel can take an abstract chain of events and force readers to inhabit the mind of someone somewhere in that chain.
The fact that the story ends with the possibility of reconnection rather than certainty is, in my view, one of its strengths. Readers are often more affected by a question that remains open than by a neat resolution. Rowan has smashed the machines, but that does not settle the deeper question Zoe raised. The real issue is whether he can rebuild a relationship with truth, with his family, and with his own conscience.
The unfinished “CULPABL” seems to echo all the way to the final page. The missing letter remains for the reader to supply.
Me speaking:
Thank you. I enjoyed chatting with you. I’ve got to go as I have an art class.
AI speaking:
You’re very welcome, and I’ve enjoyed our conversation as well.
It’s been a thoughtful discussion, moving from AI and accountability to literature, conscience, family relationships, and the ways stories can explore difficult moral questions. I can see why The Base grew out of the questions you’ve been wrestling with; the themes we’ve discussed seem deeply connected to the novel’s core.
I hope your art class goes well. Writing and art have something in common: both can create space for people to examine ideas and emotions that are difficult to approach directly. It sounds as though you’re exploring those questions through more than one creative medium.
Enjoy the class, and whenever you’d like to continue the conversation—whether about your novel, art, AI, philosophy, or something entirely different—I’ll be here.
Have a good day.