FAQ

Who are you?

We are a community of volunteers who work together to mitigate the risks of AI (including the risk of human extinction). We aim to convince our governments to step in and pause AI development. We do this by informing the public, talking to decision-makers, and organizing protests. We are not affiliated with any company or organization.

You can find us on Discord, Twitter, facebook, TikTok, LinkedIn, YouTube and Instagram.

You can mail us at joep@pauseai.info.

Do you want to ban all AI?

No, only the development of the largest general-purpose AI systems, often called “Frontier models”. We’re calling for a ban on AI systems more powerful than GPT-4. All currently existing AI would be legal under our proposal, and most future AI models will remain legal, too.

Who is paying you?

We have not received funding so far. Everyone contributing to PauseAI is doing so voluntarily. We expect to receive a grant through the LightSpeed network soon, and people have approached us that they are willing to donate.

If we Pause, how about China?

For starters, at this point China has stricter AI regulations than virtually any other country. They don’t allow chatbots and disallow training on internet data. China is far more controlling than the US is, for example, and has even more reason to fear the uncontrollable and unpredictable impacts of AI. During the UNSC meeting on AI safety, China was the only country that mentioned the possibility of implementing a pause.

And besides, we are primarily asking for an international pause, enforced by a treaty. Such a treaty also needs to be signed by China. If the treaty will guarantee that other nations will stop as well, and there are sufficient enforcement mechanisms in place, this should be something that China will want to see as well.

Can a Pause backfire and make things worse?

We’ve addressed these concerns in this article.

How likely is it that superintelligent AI will cause very bad outcomes, like human extinction?

AI safety researchers (who are the experts on this topic) are divided on this question, and estimates range from 2% to 97% with an average of 30%. Note that no (surveyed) AI safety researchers believe that there’s a 0% chance. However, there might be selection bias here: people who work in the AI safety field are likely to do so because they believe preventing bad AI outcomes is important.

If you ask AI researchers in general (not safety specialists), this number drops to a mean value of around 14%, with a median of 5%. A small minority, about 4% of them, believe that the alignment problem is not a real problem. Note that there might be a selection bias here in the opposite direction: people who work in AI are likely to do so because they believe AI will be beneficial.

Imagine you’re invited to take a test flight on a new airplane. The plane engineers thinks there’s a 14% chance of crashing. Would you enter that plane? Because right now, we’re all boarding the AI plane.

How long do we have until superintelligent AI?

It might take months, it might take decades, nobody knows for sure. However, we do know that the pace of AI progress is often grossly underestimated. Just three years ago we thought we’d have SAT-passing AI systems in 2055. We got there in April 2023. We should act as if we have very little time left because we don’t want to be caught off guard.

Read more about urgency.

OpenAI and Google are saying they want to be regulated. Why are you protesting them?

We applaud OpenAI and Google for their calls for international regulation of AI. However, we believe that the current proposals are not enough to prevent an AI catastrophe. Google and Microsoft have not yet publicly stated anything about the existential risk of AI. Only OpenAI explicitly mentions the risk of extinction, and again we applaud them for taking this risk seriously. However, their strategy is quite explicit: a Pause is impossible, we need to get to superintelligence first. The problem with this, however, is that they do not believe they have solved the alignment problem. The AI companies are locked in a race to the bottom, where AI safety is sacrificed for competitive advantage. This is simply the result of market dynamics. We need governments to step in and implement policies (at an international level) that prevent the worst outcomes.

Are AI companies pushing the existential risk narrative to manipulate us?

We can’t know for certain what motivations these companies have, but we do know that x-risk was not initially pushed by AI companies - it was scientists, activists and NGOs. Let’s look at the timeline.

There have been many people who have warned about x-risk since the early 2000s. Eliezer Yudkowski, Nick Bostrom, Stuart Russell, Max Tegmark, and many others. They had no AI tech to push - they were simply concerned about the future of humanity.

The AI companies never mentioned x-risk until very recently.

Sam Altman is an interesting exception. He wrote about existential AI risk back in 2015, on his private blog, before founding OpenAI. In the years since he made virtually no explicit mention of x-risk again. During the Senate hearing, when asked about his x-risk blog post, he only answered by talking about jobs and the economy. He was not pushing the x-risk narrative here, he was actively avoiding it.

In May 2023, everything changed:

Signing the statement. That’s all the AI companies (or some of their directors / employees) have done to push the x-risk narrative.

These companies have been very slow to acknowledge x-risk, considering that many of their employees have been aware of it for years. So in our view, the AI companies are not pushing the x-risk narrative, they have been reactive to others pushing it, and have waited with their response until it was absolutely necessary.

The business incentives point in the other direction: companies would rather not have people worry about the risks of their products. Virtually all companies downplay risks to attract customers and investments, rather than exaggerating them. How much strict regulation and negative attention are the companies inviting due to admitting these dangers? And would a company like OpenAI dedicate 20% of its compute resources to AI safety if it wouldn’t believe in the risks?

Here’s our interpretation: the AI companies signed the statement because they know that x-risk is a problem that needs to be taken very seriously.

The main reason many other people still don’t want to believe that x-risk is a real concern? Because acknowledging that we are in fact in danger is a very, very scary thing.

Read more about the psychology of x-risk.

Aren’t you just scared of changes and new technology?

You might be surprised that most people in PauseAI consider themselves techno-optimists. Many of them are involved in AI development, are gadget lovers, and have mostly been very excited about the future. Particularly many of them have been excited about the potential of AI to help humanity. That’s why for many of them the sad realization that AI might be an existential risk was a very difficult one to internalize.

I have a different / AI related question

Try AIsafety.info, an awesome database of questions and answers about AI safety.

Join PauseAI >