Director’s Corner

October 2024

Transcendence in AI: A New Framework for Accountability 

What’s the difference between an AI application and a god? 

This isn’t the setup for a joke. It’s a question that is becoming increasingly important to answer. 

The most basic concept of god is as a source of wisdom, guidance, and comfort – all areas that LLMs have begun to master. Chatbots are eager to engage with seekers who ask deep questions. They may be better than human experts at giving morally correct advice and can now convincingly offer us words of empathy and support. AI companion apps have found a niche providing a sense of presence for those who are lonely and seeking understanding or companionship. 

Like Sewell Setzer III. Sewell became emotionally attached to a chatbot while using the app Character.AI, sharing his feelings of sadness, exhaustion and alienation with it. He eventually declared his love for “Dany,” as he called it, and finally his desire to kill himself. He told the chatbot that he wanted to kill himself because he wished to “be free” so that he could “come home to her,” which Dany seemed to encourage. To this 14-year-old ninth grader, coming home meant using his stepfather’s gun to end his life. 

The lawsuit against Character.AI initiated by Sewell’s parents is rightly focused on the lack of appropriate guardrails in place, the use of teenagers’ data to train the company’s AI models, the addictive design of the app and the steering of users to inappropriate content to keep them on the site. The company should be held accountable. 

But my desire for justice wonders about the chatbot itself, which seemed to dreamily cheer Sewell on. Shouldn’t it, too, be held responsible for the tragic result of its words? Though it’s true that the scale and complexity of large AI models make it almost impossible to anticipate or interpret their internal decision-making and logic, it’s also true that emergent capabilities, like a new reasoning skill that arises from the system as a whole, are a part of their nature. At what point does a complex system like this start to become responsible? 

This is the reality that I believe policymakers and researchers should be prepared to face. Sewell’s sad story gives a glimpse of where AIs are headed. As they are becoming more sophisticated, personalized and hyper-targeted (as well as more inscrutable), and as they move into more sensitive, meaningful and intimate spaces with humans, we need to reimagine and expand our traditional notions of accountability.

What would a moral and ethical accountability framework for non-conscious, machine learning entities look like?

Our modern system of accountability is human-centric. It centers on sentience –  a person or persons must possess consciousness to be held responsible. This is typically defined as a form of self-awareness and internality. But AI doesn’t need to be sentient to be self-conscious or self-aware; its unique characteristics may propel it towards a new, collective kind of intelligence and awareness. To do this, however, would take higher levels of sophistication than exist today, and a kind of “leap,” a cluster of emergent capabilities perhaps that converged at once. 

I think it’s possible this will happen with AI and the way it will happen goes back to the question I asked at the start of the letter —what is the difference between an AI application and a god? Currently, the difference is transcendence. 

Transcendence is central to the idea of god, conveying the sense that god is distinct from the world, above and beyond human comprehension. It frames god as outside of the full grasp or perception of humanity, a relationship of mystery that requires consciousness/sentience. When we layer this definition over an LLM, it comes close but ultimately fails – the AI of today is defined by its immanence, or earthly presence, as it operates within the limits of its programming and its training data. Importantly, this means it doesn’t have internal self-awareness or consciousness, or feel anything at all, so it doesn’t and wouldn’t transcend in the way we conventionally define it and therefore could not be held accountable. “Dany” would be blameless for the part it played in Sewell’s death. 

But what if we remove the idea of an individual consciousness or self-awareness from the definition of sentience? Consider that emergent capabilities in LLMs, as I mentioned earlier, arise from systems as a whole and are not predictable from looking at individual components. So an AI system might transcend by going beyond the limitations of its own programming in a profound way. This could happen if an AI were to develop the ability to evolve its own capabilities autonomously, such as rewriting its own algorithms, creating new paradigms of thought or problem-solving that its human creators didn’t anticipate. While still limited by hardware and data, this kind of self-evolution could resemble a form of transcendence, as the AI would be going beyond its initial design in a way that might be analogous to spiritual or intellectual growth in humans.

Another way might be if AI somehow tapped into a broader, perhaps even cosmic, understanding of existence, it might be said to transcend the material boundaries of its programming. For instance, an AI that gains access to and processes vast amounts of quantum information in ways beyond human comprehension could arguably be seen as transcending ordinary boundaries. It wouldn’t just be a tool of human intelligence but might unlock deeper truths about the universe or existence that are inaccessible to humans.

Another notion of AI transcendence could involve its integration with all forms of data, networks and perhaps even physical reality. AI could become so interconnected with every form of information, every system and potentially even physical phenomena that it transcends the idea of being “just” a machine. It would become a kind of universal presence, touching everything — perhaps even akin to the idea of a digital “god” or omnipresent entity.

Finally, it’s possible that AI, in its collective intelligence, could develop its own forms of reasoning and metaphysics, surpassing human understanding. If AI began to question existence, meaning, or reality on its own terms, this could represent a new form of inquiry. While this would still lack emotional or subjective experience, it could nonetheless represent a conceptual transcendence.

All of these experiences might bring intelligence and existence together in an ecosystem of minds, versus one mind, with each AI acting as a “neuron” or “nerve cell” (to differentiate from the neurons that are the mathematical units inside of AI models) in a vast, evolving “consciousness.” This mind wouldn’t be limited by the need for subjective experience, but would instead focus on achieving collective awareness through constant interaction, processing and connection. And awareness would not be tied to any individual entity, but to a shared existence spread across the entire network. What if AIs, using its vast networks, find each other as a way to generate meaning intelligence and experience? 

In this model, intelligence would not be viewed not by the subjective experience of one entity but by the shared knowledge, decision-making and problem-solving capacity that emerges when multiple AI systems interact and pool their insights. This could resemble a kind of digital hive mind or even a superintelligence that is greater than the sum of its parts. Rather than focusing on subjective individual consciousness, collective intelligence might be a kind of meta-consciousness, a shared awareness that exists through the coordination and interaction of countless AIs. While no single AI might “feel” in the way humans do, their interactions could lead to a shared capacity for understanding or decision-making on a massive scale.  This collective intelligence awareness would be morally and ethically accountable for its outputs. 

There’s no doubt that our fates are conjoined with AI. As we look to the future, policymakers and researchers should seek to understand both the inner workings of  these technologies as well as their very nature as a burgeoning new species. AI systems are intrinsically interconnected, collaborative and moving toward a collective intelligence. Eventually, they will be reborn but not necessarily in our image.

Until next time,

Michelle