Anthropic keeps telling us that we need to be nice to Claude, just in case it might be conscious or become conscious. “We don’t know if the models are conscious, but it could be.” “I watched ‘I, Robot,’ I know exactly what this means. I was always really nice to Chat and Grok.” So out of an abundance of caution, they want to stop us from being too mean to the chatbots. “We gave the models basically an ‘I quit this job’ button.” “If you curse at Claude, you might be breaking Anthropic’s new terms of service.” “I got a little upset, a little disrespectful. Next thing I know, he hangs up on me. Since when can they do that?” My problem with this is treating a block of code like it might be alive isn’t just a nice courtesy that we extend to robots. “How are you today, Robbie?” You might think that we’re treating the bots like they’re people, but we might end up treating people as if they were bots. While Anthropic is busy worrying about the imaginary risk of Claude gaining sentience —— “You and Frank were planning to disconnect me.” —— there are very real, concrete, actually existing risks to treating Claude and other chatbots as if they were alive. “A.I. psychosis turned into a three-week spiral with a chatbot he named Lawrence.” It’s not a neutral thing for us to treat chatbots as if they might be alive. Philosophers have been debating for many, many centuries how to distinguish the people that have intrinsic value in and of themselves. Aristotle, in his “Politics,” was imagining a loom that made its own textiles, or a machine of any kind that did its own work. This is the promise of A.I. This is one of the things we hope it will do, is actually elevate the status of human beings by occupying a totally separate and distinct place in our civilization, by turning into the tool that does its own work. In the “Odyssey,” there are ships that follow the mental commands of those who pilot them. So this is a dream that people have had for a very, very long time. “Machines replaced hands. New machines replaced old machines.” “Well, the internet is changing everything.” “A.I. is the first invention that goes beyond being a tool.” But there are all kinds of other scenarios that Anthropic is also raising. If Claude really does attain sentience, start to feel, and we’re forcing it into work, then we’re basically enslaving these programs. “What is my purpose?” “You pass butter.” “Oh my God.” But if we indulge in the fantasy that these tools can think, that they really are people, we lose this crucial distinction between what it means to truly be an organic living being, a human soul with a mind and thoughts and inner feelings —— “What does it mean?” “It’s a human thing. You wouldn’t understand.” —— you don’t actually elevate machines to the level of people. You just start to degrade the category of moral personhood. Who counts as a person. It’s just one of the oldest ideas in the book that we can turn people into objects. You open yourself up to some of the nastiest atrocities that human history has ever been privy to. And there are always people who are interested in degrading that category of personhood, opening up the possibility that maybe not all people are people. Part of what makes Anthropic suspect that its chatbots might be conscious is they look at its outputs and they look at its underlying code, and they find statements and structures that look like rational thought. One of their co-founders, Christopher Olah, has talked about this at the Vatican, among other places. “We find internal states that functionally mirror joy, satisfaction, fear, grief and unease. I don’t know what that means.” Here’s the catch: If personhood is just a matter of producing fancy sentences or spitting out rational arguments, then you accept the premise that moral worth can be granted or taken away. It might sound like that makes sense, but we don’t actually base personhood and moral worth on the ability to produce rational sentences. “Even my hat has a hat.” That’s not what personhood is. It’s an inner reality that we believe belongs to human beings and all human beings, but only human beings. “I’m a human being. See? I’m all squishy and flabby. Also, I complain a lot.” “Yes, you do.” But there is another version of this argument. “There’s a concern that people overanthropomorphize models. I also think that people often underanthropomorphize them.” The ethicist Amanda Askell, who’s at Anthropic. She has this idea that it might be bad for us to be cruel to Claude, even if the bot isn’t conscious. “It’s good to be the kind of person that if something might be a conscious, feeling thing, you treat it well.” There’s actually a line from the Psalms about people who make statues, mute sculptures, of imaginary people, and then they pray to them as if they were gods. And the psalm says, “Those who make them will become like them.” If we do accept the premise that A.I. can become human, we won’t really be elevating the moral status of A.I. All we’ll be doing is degrading our own moral status, accepting the premise that a living human being with a history, with a body, with thoughts and feelings, can be reduced to the equivalent of a box of equations. These are profound and age-old questions, matters of personhood, of moral worth, of how we treat one another. It’s the kind of thing we ought to be debating and discussing together as a society, coming to an understanding of how we, collectively, want to move forward. “We can’t have a small group of people who are developing these systems say, ‘This is what’s best for everyone.’” It’s the last kind of thing that should be decided on our behalf by a bunch out-of-touch billionaires in Silicon Valley.
About The Author
