I’m an AI investor: here’s why I just deleted Claude
Anthropic’s messianic pronouncements are no longer like a marketing strategy, but a religious playbook, says Lewis Liu
I canceled my personal Claude subscription this week and deleted years of work. I don’t object to the technology (I’m an AI investor currently working at the sharp end of its capabilities) but I believe that tone and intention matter as much as speed and ability. I’ll explain my decision shortly but first, let’s recap on what an insane few weeks it’s been in the world of AI.
A few weeks ago, NYU math professor Tristan Buckmaster and Anthropic researcher Levent Alpöge announced that, working with a mix of AI models, they were very close to solving the Navier-Stokes equation, a major problem involving how fluids flow, and one of the great unsolved problems in mathematics. Within days, OpenAI announced it had independently achieved a solution after spending over $20m on compute. It turns out, however, that this solution is extremely similar, if not identical, to Buckmaster’s. When he accused OpenAI of stealing his work, OpenAI announced that it “cannot rule this out”.
There are several interesting implications here. First: human mathematicians were already extremely close to the solution. What AI did, after spending tens of millions on compute, was relentlessly, and somewhat mindlessly, test a vast number of potential solutions. It is a bit like solving a maze by testing every route, instead of having some deeper intuition about how to solve it. But this begs an ironic question: that amount of compute could fund several math departments for years. If humans are already so close, what has a better ROI?
The second, far deeper implication is that nothing you put into a frontier lab is ever truly yours, regardless of what the terms and conditions say. No wonder that two weeks ago, the second-largest American law firm, Latham & Watkins, announced it is building its own AI infrastructure on Nvidia chips instead of relying on Anthropic or OpenAI. Several leaders of regulated American businesses have told me they now feel the risk of an open-source (Chinese) model, contained or air-gapped, is safer than giving away their most precious business knowledge to the frontier labs. My own start-up, Twin1 AI, has been actively exploring open-source models (for those clients comfortable with them) and moved off Anthropic entirely months ago, following concerns from our large enterprise clients about Anthropic siphoning off data and know-how.
My view: if you are using a frontier lab as a vendor, chances are your data is being siphoned into their training regardless of your terms and conditions. What are you going to do, sue a trillion-dollar company?
Pacing the frontier?
Around the same time, a junior-to-mid-level Anthropic engineer, Jacob Coxon, announced he was resigning because he could no longer work for a company that might bring about the end of the human race. To make things interesting, Evan Hubinger, who leads alignment science at Anthropic, responded that he reckons the chances are about 10 per cent that Anthropic’s AI will wipe out human civilization. This prompted Dario Amodei, Anthropic’s CEO, to ask for an antitrust waiver so the AI labs could “pace the frontier” – in other words, slow down.
The public pushback has been tremendous. Here are some of the more salient arguments.
One: the whole thing is potentially just marketing theatre, including the “whistleblower.” The Wall Street Journal reportedly ran the story before Coxon’s resignation tweet even went out. Both Anthropic and OpenAI, though especially Anthropic these days, are known for using scare tactics to show off how powerful their models are. Commentators note it was Amodei himself who said in 2019 that GPT-2 was “too unsafe” to release when he was at OpenAI. By scaring the world with how “civilisation-destroying” Anthropic’s models are, it only makes them seem the best, and thus only Anthropic is good enough to “stop it and save humanity”, thereby boosting the upcoming IPO valuation. As of this writing, OpenAI’s Astra and even some Chinese open-source models are outperforming Anthropic’s Fable on many if not most benchmarks.
Two: this is a push for regulatory capture when open-source models are almost if not just as good as the US frontier labs. By asking for an antitrust waiver and effectively writing the regulation, the US frontier labs hope the government will ban open-source models under the guise of “these models are too dangerous”, thereby securing regulatory capture. My favourite response was from Hugging Face co-founder Julien Chaumond (Hugging Face is being acquired by Nvidia for $13bn), who tweeted: “open source won’t pace”.
Three: in a time of extraordinary competition from open source and depleting trust, by arguing for “pacing the frontier”, the frontier labs are protecting their IPO. This leaves the public able to imagine them as “the good guys”: that they can flip a switch any day to implement Godlike AI, but are choosing not to. Effectively distracting investors from the real commercial challenge of the burn and compute resources required to keep these labs going. Effectively saying: “we’re slowing down because we’re good people with an all-powerful AI, not because we can’t generate solid free cash flow”.
Why I cancelled my Claude account
Back to why I deleted years of work. I loved Claude for my personal life and built a huge context layer around me over two years. I was one of the first users of Claude when everyone was still crazy about ChatGPT. I was even willing to look the other way on the “pacing the frontier” shenanigans, or the fact that Anthropic would automatically change my writing if it was critical of the company.
But I can’t ignore the growing sense that Anthropic, more so than other frontier companies, appears to adopt a more religious undertone. I am not the only one to notice this, but it seems that Anthropic’s playbook is not that of a classical marketing strategy, it is one of a religious playbook. Step 1: create a super-powerful crisis and evil threat (the AI God is going to destroy humanity). Step 2: proclaim that you are the only one powerful and moral enough to stop this evil. Step 3: invite people to join you, as you are the only salvation of the human race. And you know what? This co-opting of religious narratives is simply not for me.
AI has the potential to do so many great things for humanity, but we cannot let a small number of men determine what is moral and right. Nor can we let companies whose fundamental technology is based on theft at an untold scale transfer wealth (through a highly controversial IPO) from everyday Americans to a few thousand people. I did my small part by cancelling my account. We can all do our part to ensure AI works for all of us, not just the few.
Oh, and as for my data: I doubt it’s actually deleted. Happy training data to you, Anthropic!