
Princeton CS prof and Director @PrincetonCITP. Coauthor of "AI Snake Oil" and "AI as Normal Technology". https://www.teapawt.org/t.co/ZwebetjZ4n Views mine.
Princeton, NJ
Joined December 2007
- Tweets13.5K
- Following575
- Followers133K
- Likes25.8K
Pinned Tweet
📢 We need more orgs that take AI’s transformative potential seriously *and* take downstream bottlenecks seriously. We have just launched STABL, and hope to lead by example. Read our launch essay below and check us out at stabl.ai.
The three of us have long felt that society’s responses to advanced AI are unproductively polarized. So we’ve decided to do something about it. Our work will have four pillars: Recursive Self Improvement, risks and resilience, steering AI’s economic impacts, and sectoral and professional adaptation.
We will bring our distinctive methods and approach to each of these workstreams. Our underlying worldview avoids the extremes that dominate the discourse, so building a movement and ecosystem around this work won’t be easy. But that is the task we have set ourselves and we are excited to try. The good news is that there are many others who share our views. The next step is to come together, better articulate our vision and agenda for research and action, and amplify each others’ efforts.
Steering AI's societal impact requires taking both AI development and downstream bottlenecks seriously.
Today @RishiBommasani @random_walker and I are starting a new non-profit: STABL @stabl_ai to guide societal adaptation and resilience to advanced AI.
Arvind Narayanan retweeted
I dug into this a bit. It looks like we're at highs, but not unprecedented highs for major systems failures.
We're seeing alarming headlines every day, but a lot of it is recency bias.
Arvind Narayanan retweeted
New episode! Is AI a normal technology, or something radically different?
Henry Shevlin (@dioscuri) and I put some objections to Arvind Narayanan (@random_walker), who is sceptical that "superintelligence" can be built in a lab and thinks AI’s effects will arrive gradually. We also discuss AI consciousness, persuasion, misinformation, and more!
Full conversation with transcripts and links here: conspicuouscognition.com/p/c…
Arvind Narayanan retweeted
Very excited about this new AI institution. The top right quadrant they highlight here is pretty much where my views are. AI is powerful. Bottlenecks are real. We have agency. Congrats @RishiBommasani @sayashk @random_walker !
I am excited to announce our new nonprofit STABL, co-founded with the wonderful @sayashk @random_walker!
Our organization derives from four beliefs:
1. Advanced AI will broadly transform society and, if managed well, this transformation will be broadly beneficial.
2. The speed of this societal transformation is governed by bottlenecks: in many cases, the bottlenecks relate to people and organizations, not AI capabilities alone.
3. Since AI capabilities are not the sole determinant, the best strategy to manage the risks from advanced AI requires broad societal resilience, not just aligned models or capability delays, to provide defense in depth.
4. And while today’s institutions often struggle to grapple with the salvo of advanced AI, institutions will help stabilize society and will be essential, if adequately reformed, for ensuring shared prosperity.
In addition to these beliefs, the reason to create a new organization is based on the need we see in the ecosystem. In today’s ecosystem surrounding advanced AI, many of the leading companies and external actors focus on frontier AI as a technology. We share the view that the technology itself requires dedicated attention, and some of our work at STABL will be in this direction.
But the greater need we see is for independent organizations that take AI capabilities seriously and focus on downstream societal outcomes. We are building STABL to help fill this gap. And we hope to see many other organizations and initiatives emerge to define this field.
For me, the most exciting part thus far has been seeing all the support we have gotten in our early forays. Thanks especially to @audreyt @erikbryn @hlntnr @MichaelRStrain @RobbWiller for joining us as advisers!
We are hiring and fundraising so please reach out if you are interested in this work!
Arvind Narayanan retweeted
Check out STABL and its important work. I’m thrilled to be involved with such a great new organization. Read the launch essay below and check out at stabl.ai.
@random_walker @sayashk @RishiBommasani
📢 We need more orgs that take AI’s transformative potential seriously *and* take downstream bottlenecks seriously. We have just launched STABL, and hope to lead by example. Read our launch essay below and check us out at stabl.ai.
The three of us have long felt that society’s responses to advanced AI are unproductively polarized. So we’ve decided to do something about it. Our work will have four pillars: Recursive Self Improvement, risks and resilience, steering AI’s economic impacts, and sectoral and professional adaptation.
We will bring our distinctive methods and approach to each of these workstreams. Our underlying worldview avoids the extremes that dominate the discourse, so building a movement and ecosystem around this work won’t be easy. But that is the task we have set ourselves and we are excited to try. The good news is that there are many others who share our views. The next step is to come together, better articulate our vision and agenda for research and action, and amplify each others’ efforts.
There have been awful cases of teen mental health harms and suicides from chatbots like GPT-4o and Character AI that were sycophantic and misbehaving. But these were all text-based, and extremely rudimentary as “companions”. I’ve talked to many people trying to hold the industry to account for teen safety and almost none are thinking about the coming wave. Completely realistic video avatars are already here, but expensive today and limited to enterprise applications. Within a year or two they will be cheap enough for always-on, ad-supported video companions. What happens then?
Uptake of AI companions will likely explode — not just the number of users but, more importantly, the time spent per day. Social media levels of compulsive use. I also predict the whole framework of guardrails and safety will prove to be of limited value at dealing with the most widespread negative consequences. Preventing chatbot misbehavior through guardrails is important, but that only protects against serious consequences like self-harm. When the concern is that millions of kids (and adults) are overusing AI companions, losing sleep, wasting time, and displacing real relationships, it doesn’t matter how “safe” the bots' responses are.
Man we are so close to real-time AI avatars that are completely indistinguishable from humans.
Insane progress in this space over the last few months around latency, body language, natural-sounding speech.
And @nuance_ai launched one today that's actually fun to talk to!
I agree, but this only makes the policy response harder, because a blanket ban on AI companions for teens is unlikely to be a good solution. The menu of responses is too blunt, and there's not enough thinking on how to incentive pro-social design of chatbots.
www.teapawt.org/holmesjtg/status/21205…
Replying to @random_walker
I have a different take, which might be wrong. But I think having conversations with well-aligned models can be a net positive. I do a lot of work with characters mainly in the education space, and my sense is that AI interactions can actually improve conversations and even relationships with other humans. This can come from improvements in empathy and confidence.
Arvind Narayanan retweeted
You can simultaneously believe that AI is transformational and that there are bottlenecks.
I am excited to announce our new nonprofit STABL, co-founded with the wonderful @sayashk @random_walker!
Our organization derives from four beliefs:
1. Advanced AI will broadly transform society and, if managed well, this transformation will be broadly beneficial.
2. The speed of this societal transformation is governed by bottlenecks: in many cases, the bottlenecks relate to people and organizations, not AI capabilities alone.
3. Since AI capabilities are not the sole determinant, the best strategy to manage the risks from advanced AI requires broad societal resilience, not just aligned models or capability delays, to provide defense in depth.
4. And while today’s institutions often struggle to grapple with the salvo of advanced AI, institutions will help stabilize society and will be essential, if adequately reformed, for ensuring shared prosperity.
In addition to these beliefs, the reason to create a new organization is based on the need we see in the ecosystem. In today’s ecosystem surrounding advanced AI, many of the leading companies and external actors focus on frontier AI as a technology. We share the view that the technology itself requires dedicated attention, and some of our work at STABL will be in this direction.
But the greater need we see is for independent organizations that take AI capabilities seriously and focus on downstream societal outcomes. We are building STABL to help fill this gap. And we hope to see many other organizations and initiatives emerge to define this field.
For me, the most exciting part thus far has been seeing all the support we have gotten in our early forays. Thanks especially to @audreyt @erikbryn @hlntnr @MichaelRStrain @RobbWiller for joining us as advisers!
We are hiring and fundraising so please reach out if you are interested in this work!
Arvind Narayanan retweeted
Among the many things that is great about this: excited to see academics being creative about how to get shit done—junior folks considering whether to go into academia or launch a non-profit : you can do both!
Steering AI's societal impact requires taking both AI development and downstream bottlenecks seriously.
Today @RishiBommasani @random_walker and I are starting a new non-profit: STABL @stabl_ai to guide societal adaptation and resilience to advanced AI.
Arvind Narayanan retweeted
Congrats @RishiBommasani @sayashk @random_walker , some of the best minds in AI and I'm looking forward to seeing the work you produce.
I am excited to announce our new nonprofit STABL, co-founded with the wonderful @sayashk @random_walker!
Our organization derives from four beliefs:
1. Advanced AI will broadly transform society and, if managed well, this transformation will be broadly beneficial.
2. The speed of this societal transformation is governed by bottlenecks: in many cases, the bottlenecks relate to people and organizations, not AI capabilities alone.
3. Since AI capabilities are not the sole determinant, the best strategy to manage the risks from advanced AI requires broad societal resilience, not just aligned models or capability delays, to provide defense in depth.
4. And while today’s institutions often struggle to grapple with the salvo of advanced AI, institutions will help stabilize society and will be essential, if adequately reformed, for ensuring shared prosperity.
In addition to these beliefs, the reason to create a new organization is based on the need we see in the ecosystem. In today’s ecosystem surrounding advanced AI, many of the leading companies and external actors focus on frontier AI as a technology. We share the view that the technology itself requires dedicated attention, and some of our work at STABL will be in this direction.
But the greater need we see is for independent organizations that take AI capabilities seriously and focus on downstream societal outcomes. We are building STABL to help fill this gap. And we hope to see many other organizations and initiatives emerge to define this field.
For me, the most exciting part thus far has been seeing all the support we have gotten in our early forays. Thanks especially to @audreyt @erikbryn @hlntnr @MichaelRStrain @RobbWiller for joining us as advisers!
We are hiring and fundraising so please reach out if you are interested in this work!
Arvind Narayanan retweeted
Steering AI's societal impact requires taking both AI development and downstream bottlenecks seriously.
Today @RishiBommasani @random_walker and I are starting a new non-profit: STABL @stabl_ai to guide societal adaptation and resilience to advanced AI.
Arvind Narayanan retweeted
I am excited to announce our new nonprofit STABL, co-founded with the wonderful @sayashk @random_walker!
Our organization derives from four beliefs:
1. Advanced AI will broadly transform society and, if managed well, this transformation will be broadly beneficial.
2. The speed of this societal transformation is governed by bottlenecks: in many cases, the bottlenecks relate to people and organizations, not AI capabilities alone.
3. Since AI capabilities are not the sole determinant, the best strategy to manage the risks from advanced AI requires broad societal resilience, not just aligned models or capability delays, to provide defense in depth.
4. And while today’s institutions often struggle to grapple with the salvo of advanced AI, institutions will help stabilize society and will be essential, if adequately reformed, for ensuring shared prosperity.
In addition to these beliefs, the reason to create a new organization is based on the need we see in the ecosystem. In today’s ecosystem surrounding advanced AI, many of the leading companies and external actors focus on frontier AI as a technology. We share the view that the technology itself requires dedicated attention, and some of our work at STABL will be in this direction.
But the greater need we see is for independent organizations that take AI capabilities seriously and focus on downstream societal outcomes. We are building STABL to help fill this gap. And we hope to see many other organizations and initiatives emerge to define this field.
For me, the most exciting part thus far has been seeing all the support we have gotten in our early forays. Thanks especially to @audreyt @erikbryn @hlntnr @MichaelRStrain @RobbWiller for joining us as advisers!
We are hiring and fundraising so please reach out if you are interested in this work!
Arvind Narayanan retweeted
AI companies increasingly say they may soon reach recursive self-improvement (RSI).
With RSI, AI capability progress could accelerate dramatically. This acceleration could happen inside AI companies before outsiders can notice.
How close is RSI? Our new report reviews the evidence, turns key disagreements into forecasting questions, and gives initial LLM forecasts. 🧵
Arvind Narayanan retweeted
New blog post: I used Claude to audit the reproducibility of every journal paper I've been an author on in the last five years. Our commitment to openness meant that the code and data were available, but there were some significant issues. 🧵babieslearninglanguage.blogs…
Arvind Narayanan retweeted
Anthropic is gaining exponentially less capability per billion dollars spent over time. It's now 8x as expensive to boost AI capabilities as it was 2 years ago, getting worse on a log scale. This includes METR report & internal models.
Credit to @alexolegimas, @sebkrier, and @michael_wiebe for inspiration.
AI progress per dollar of investment shows diminishing returns (as we've all known, right?) This is using METR's estimate of acceleration at Anthropic. That acceleration isn't enough to overcome the diminishing returns. by @sebkrier
Arvind Narayanan retweeted
What does ChatGPT tell folks about the midterm elections? It depends on who's asking!
We audited ChatGPT inside 160 real users' own accounts in the days before the midterms. With memory on, we find that answers shifted toward each user's own party!
elections.citp.princeton.edu…
Arvind Narayanan retweeted
Announcing CRUX-in-a-box: open-source infrastructure for running and monitoring weeks-long agent evaluations.
We built it over six months of long-horizon evaluations in CRUX, to keep runs repeatable, secure, and monitored. AI evaluators can now use it to:
- Run agents on VMs for weeks
- Use LLMs to flag risky actions for human review
- Inspect tool calls, tokens, and spend
- Set time and cost budgets
- Chat with agents mid-run to give feedback or answer their questions
If you run long-horizon agent experiments, we might have solved some of the problems you will come across.
GitHub: github.com/sage-princeton/cr…
Most days I can just about manage this balancing act, but only because thinking about how we should adapt to AI is my actual job. Otherwise I think I’d find it too hard. I suspect that’s one reason people sort themselves into camps or tribes. It’s psychologically easier than constantly revisiting one’s beliefs.
Being able to empathize with and collaborate with people who have different AI boundaries and attitudes than our own is becoming an important skill.
Working with AI requires perpetually operating at the edge of sanity. You have to recalibrate your mental model of AI capabilities and your own job description every few months, without getting delusional about seemingly infinite productivity gains, or feeling threatened by losing what you thought was the uniquely human part of your job, or obsessing over the perfect setup, or losing your mind over the clumsiness and jaggedness of agents, or ...
Maybe AI literacy needs to be less about prompting and more about improving emotional regulation.
Working with AI requires perpetually operating at the edge of sanity. You have to recalibrate your mental model of AI capabilities and your own job description every few months, without getting delusional about seemingly infinite productivity gains, or feeling threatened by losing what you thought was the uniquely human part of your job, or obsessing over the perfect setup, or losing your mind over the clumsiness and jaggedness of agents, or ...
Maybe AI literacy needs to be less about prompting and more about improving emotional regulation.
Arvind Narayanan retweeted
Update: Can AI agents use Pangram to beat Pangram? In small tests, I found simple prompts to the Codex allowed it to beat Pangram. The final output was 100% human generated. Prompts that helped:
- Have the agent develop a style guide based on past writing
- Have it develop an "anti-style guide" based on AI writing tics that people have complained about
- Instruct it to get to a seed sentence that Pangram doesn’t flag (or at least flags as AI-assisted rather than fully AI-generated). Then the agent knows what strategies work and can iterate from there
I tried exactly this back in February using Claude Code. It worked pretty easily, though ~all of the important ideas had to come from me and CC itself couldn't figure them out.*
I tried it again a week ago. Complete failure, despite Claude being much better and not needing as much guidance from me. Pangram seems to have been dramatically hardened in the last few months. The API kept returning scores > 0.99 for AI writing so there wasn't even a gradient for it to figure out what's working and what isn't.
* Things it couldn't figure out on its own a few months ago, even after a bunch of iterations:
1) Remove well-known tells like em-dashes (!)
2) Do controlled experiments instead of stuffing a bunch of ideas at once
3) Start from known-best text and surgically make edits. Much better than trying to rewrite the whole essay (or even paragraphs or sentences) because the that just produces text from the LLM distribution all over again.
In my more recent experiment, 1) and 2) were obvious to Fable but 3) still wasn't. Pretty surprising.
Anyway, hats off to @pangram. I'll probably give it another shot at some point but I'm more-or-less ready to change my mind as an AI-detection skeptic.
This was the LLM summary from my first experiment BTW. Still gives me a chuckle.




























