Image

AI DEVELOPMENT AND SAFETY: A CORPORATE INSIGHT

AI DEVELOPMENT AND SAFETY: A CORPORATE INSIGHT

5th January 2026 – by Andrew Dolan

I thought that we could start of this year’s activities with a timely reminder of the importance of AI safety.  The Future of Life Institute’s AI Safety Index of Winter 2025 * has recently been published and reinforces a key theme in the Institute’s thinking and work, namely, how safe is AI?

The Safety Index looks at several of the leading AI technical companies in respect of their approaches and policies for identifying and minimising unintentional risks to society’s safety.  As the Index clearly shows, the evidence suggests a wide variance in safety standards even to the extent that the Index’s analysts clearly believe that ‘Existential safety remains the industry’s core structural weakness.’

Additionally, the Index refers to companies ‘falling short’ of emerging global standards on safety, although one might quibble that there are indeed no established standards and even assuming there might be in the future, whose standards are they and who will draw them up?

Nevertheless, the evaluation process is not as comprehensive as some would have it.  Transparency is weak – understandable perhaps given the vast amounts of investment and commercial interest involved and the burgeoning relationship between AI Tech and the military community.  The Index authors don’t shy away from these limitations but it does, as they readily admit, hinder efforts to really appreciate the situation in AI development.

This situation is arguably the fundamental problem associated with AI safety.  How does the public gain a perspective of AI development, especially the purpose of the developments and how can it comment on the perceived risks associated with it?  

The Index authors, as mentioned earlier, draw our attention to the race to develop Artificial General or Super Intelligence yet there is no clear planning for what might happen should this situation arise.  For example, how is public policy to be framed if those responsible have no idea of the consequences of the actions of an AI technical team?

Therefore, as much as the AI Safety Index is to be warmly welcomed, its greatest importance could lie in the fact that it should encourage much more public engagement on AI safety, including a string does of ethical consideration.  

Clearly the Future of Life Institute is particularly vexed by the issue of existential or catastrophic risk and why wouldn’t they be.  However, what about the many bad sub-catastrophic risks that could arise from poor safety protocols, especially if they are associated with frontier research or ill-considered experimentation?  Do we have a way to address this set of problems?

Finally, might we need to equally consider the harms that could be done to society through unexplored and unintended consequences?  There are more than a few philosophical thought-experiments circulating, that explore the future of humanity, ranging from Transhumanism and The Singularity to Human-Machine Teaming.  Other aspects of this speculation are more grounded in research around human evolution, supported by developments in AI and Life Sciences.  Can we consider this as a subject for AI Safety?

I would encourage everyone to acknowledge the relevance of the work done by those who crafted the AI Safety Index and let it be a platform upon which to encourage and engage in your own AI safety enquiries.