I’ve been thinking a lot about why most conversations feel like nothing. Not necessarily bad, but obviously empty. Two people making mouth sounds at each other for twenty minutes and then walking away with less energy than they started with, having exchanged approximately zero real information about their actual inner lives. And I’ve been wondering whether this is just an unavoidable feature of being human, or whether it’s actually a failure mode that most people have simply stopped noticing because they’ve never experienced the alternative.

I think it’s the latter. And I think there’s a surprisingly simple framework for fixing it, though “simple” and “easy” are not the same thing.

Here’s the short version, which I originally voice-recorded as advice to myself for my new coaching business:

As soon as possible in any conversation, try to identify the 4th wall and shatter it, find the elephant in the room and start petting it. The extent to which you can instantly defuse any tension or implicit assumptions that might be guiding the conversation will enable everyone involved to feel more authentic, relaxed, comfortable, and connected.

That’s the thesis. Let me unpack why I think it’s true and what it actually looks like in practice.

Two metaphors doing different work

The “4th wall” and the “elephant in the room” sound like they’re pointing at the same thing, but they’re not. They’re addressing two entirely different layers of conversational dysfunction, and the distinction matters.

The 4th wall is about the structure of the interaction — the unspoken rules, roles, and power dynamics. It’s the invisible script that both people are following without having agreed to follow it. At a networking event, the 4th wall is: “We are both performing professional competence at each other.” On a first date, the 4th wall is: “We are both pretending to be slightly more interesting and slightly less anxious than we actually are.” At a family dinner, the 4th wall is: “We are all going to act like this is a normal, happy family gathering.”

The elephant is about content — the thing that everyone in the room knows is true but nobody is naming. Your friend’s marriage is falling apart and you’re both dancing around it. You haven’t talked to someone in two years and you’re both pretending that’s fine. Someone at the table is drinking too much and everyone is watching it happen in silence.

Breaking the 4th wall means stepping outside the script. Petting the elephant means naming the thing. Both are necessary, and in the best conversations they happen almost simultaneously.

Why “petting” and not “calling out”

I want to dwell on this word choice for a second because I think it matters more than it might seem.

“Calling out the elephant” implies confrontation. It’s aggressive. It puts the other person on their heels. “Petting the elephant” implies warmth — even affection — toward the uncomfortable thing. You’re not attacking the awkwardness or the unspoken truth. You’re domesticating it. You’re saying: this thing that feels dangerous? It’s actually just sitting here. We can touch it. It’s fine.

Here’s the difference in practice. Say you’re reconnecting with an old friend you haven’t spoken to in a long time:

• Calling out the elephant: “So why haven’t we talked in two years?”

• Petting the elephant: “It’s been way too long. I’ve missed this.”

Both name the same reality. One creates defensiveness. The other creates warmth. Same truth, completely different nervous system response. The information content is identical — it’s the emotional framing that determines whether the other person opens up or shuts down.

Most people avoid elephants entirely because they’ve only ever seen them called out, never petted. They associate naming the unspoken thing with conflict. But when you approach the elephant with gentleness instead of accusation, something remarkable happens: the other person’s relief is almost physical. You can watch their shoulders drop. Someone finally said the thing, and the world didn’t end.

Most conversations are performances

Here’s the thing I think most people don’t realize, or at least don’t realize the extent of: the vast majority of human conversation is two people performing “having a conversation” rather than actually having one.

Someone asks “how’s work?” You say “good, busy.” They say “same.” You’ve both just exchanged nothing. Zero information has been transferred about either person’s actual experience of being alive. But the social contract has been maintained. The script has been followed. Everyone goes home.

This is not a trivial observation. I think a huge amount of the loneliness epidemic — and yes, I realize everyone and their dog has written about the loneliness epidemic at this point, but bear with me — comes not from a lack of social contact but from the quality of the contact people do have. You can talk to fifteen people in a day and feel completely alone if every single interaction happened on the surface. The loneliness isn’t about absence of people. It’s about not feeling genuinely seen by someone else.

And the thing is, almost everyone involved in these hollow exchanges knows they’re hollow. Nobody walks away from “good, busy” / “same” thinking wow, what a meaningful connection. Both people can feel the emptiness. But neither person breaks the script, because breaking the script requires going first, and going first requires risk, and risk requires courage, and courage is — as it turns out — the actual scarce resource in human social life. Not time. Not opportunity. Courage.

So what does breaking the script actually look like?

It can be remarkably small. That’s the part people get wrong — they think “being authentic” means some dramatic emotional disclosure, and since they’re not prepared for that, they default to the script. But the distance between the script and something real is often just a single honest sentence:

1. Someone asks “how are you?” and you actually answer. Not a trauma dump. Just: “Honestly, kind of weird today. I don’t know why.” That’s it. That’s the 4th wall breaking. You just gave the other person permission to be real too.

2. You’re at a party and instead of “so what do you do?” you say “I barely know anyone here and I’m sort of making it up as I go — how about you?” You’ve just named the shared experience that everyone at every party is having and nobody ever says out loud.

3. A friend is telling you about their new job with manufactured enthusiasm and you say “that sounds like the version you’ve been telling everyone — what’s the version you haven’t been telling people?” That’s petting the elephant. Gently. With genuine curiosity, not gotcha energy.

These are not big moves. They’re micro-departures from the expected. And they work because the other person almost always wants someone to go first. They’re usually waiting for permission when you don’t have to.

The skill underneath: learning to perceive

Here’s where it gets harder. Breaking the 4th wall and petting elephants requires that you can actually see the 4th wall and identify the elephant. And it turns out that perceiving what’s actually happening in a conversation is a genuine skill that most people have never developed, because the primary obstacle to perceiving other people is being preoccupied with yourself.

I don’t mean this as an insult. I mean it as a near-universal description of human cognition during social interaction. The vast majority of people, in the vast majority of conversations, are running an internal monologue that goes something like: How am I coming across? What should I say next? Do they like me? Was that a weird thing to say? Am I being interesting enough? That monologue is a full-time cognitive job. And while it’s running, you literally cannot perceive the other person, because your attention is pointed at the wrong target.

The most foundational perceptual skill is therefore the least glamorous one: learn to notice when your attention is on yourself, and redirect it outward. This isn’t a one-time fix. It’s something you have to catch and correct constantly, in real-time, throughout a conversation. Where is my attention right now? Is it on them or on me? Okay, back to them.

Beyond that, there are a few specific things worth training yourself to notice.

1. Listen for the edit. Everyone is running a real-time editing process when they speak. They have an impulse, they filter it, and what comes out is the post-production version. You can actually hear the editing if you’re paying attention: mid-sentence direction changes (”I was going to — anyway, the point is...”), generalizations where specificity would be natural (”things have been kind of hard” instead of naming the hard thing), speed changes (people accelerate through discomfort and slow down near things they care about but are afraid to say). You don’t need to know what someone censored. You just need to hear that something got censored. That’s enough to gently invite the unedited version.

2. Watch for incongruence, not “body language.” Forget the pop psychology stuff about crossed arms meaning defensiveness. That’s mostly noise. What’s actually useful is noticing moments when the words and the body disagree with each other. Someone says “no, I’m fine” while their jaw tightens. Someone says “that’s so exciting” with flat energy. Someone laughs after saying something that wasn’t funny — that laugh is almost always a release valve for anxiety or pain. When two channels of communication disagree, the body is almost always the honest one.

3. Track the emotional undercurrent, not the narrative. Most people listen to the content — the story, the facts, the sequence of events. But underneath every story there’s an emotional current that often has little to do with the plot. Someone tells you about a conflict at work and the surface is “my boss did X, then I did Y.” But the undercurrent might be shame. Or loneliness. Or a desperate need to hear “you’re not crazy.” If you respond to the content — “well, maybe you should talk to HR” — you’ve completely missed the point. If you respond to the undercurrent — “it sounds like you’re doubting yourself” — that person will feel more seen than they have in a long time.

4. Use your own body as an instrument. This sounds woo-woo, and I’m generally allergic to woo-woo, but it’s actually just neuroscience. Your nervous system picks up information from other people’s nervous systems all the time — emotional contagion, mirror neurons, whatever the exact mechanism. The data shows up as sensations in your body before it becomes conscious thought. You suddenly feel tense and don’t know why? The other person is probably tense. You feel heavy in a conversation that was supposedly lighthearted? Something unspoken is weighing on someone. You feel an urge to change the subject? Something important probably just brushed the surface and the shared discomfort is trying to steer away from it. Treat your own internal state as data about the interaction, not just about you. We don’t have to understand exactly how this happens, scientifically, in order to recognize that it’s a real phenomenon.

The failure mode

I should name the obvious failure mode here, because it’s important and it’s common: using “radical honesty” as a weapon.

“I’m just being honest” is one of the most popular covers for cruelty in the English language. And there’s a version of everything I’ve described above that is actually just someone using perceptiveness to dominate, to prove they’re the smartest person in the room, to force intimacy the other person hasn’t consented to, or to perform their own authenticity as a kind of superiority.

The check is simple: am I doing this to connect, or am I doing this to feel something about myself? Am I petting the elephant because I care about the relationship, or because I want to show everyone I can see the elephant? If the move isn’t in service of the connection between you and the other person, it’s not this framework — it’s something else wearing its clothes.

There’s also a calibration issue. Not everyone wants to connect deeply with you, and that’s fine. Some elephants are not yours to pet — if someone hasn’t disclosed something to you, naming it can be a violation rather than a gift. The skill is graduated honesty: go slightly more real than the current level of the conversation and see if the other person matches you. If they do, go further. If they don’t, stay where they are without resentment. You open a door. You don’t drag anyone through it.

Why this actually matters

I can already hear the objection: isn’t this just “be more vulnerable in conversations”? And sure, at the surface level, yes. But I think the framework is doing something more specific and more useful than generic vulnerability advice.

The 4th wall / elephant distinction gives you a diagnostic tool. When a conversation feels dead or fake, you can now ask: is the problem structural (we’re both stuck in roles and scripts) or is it content-based (there’s something specific we’re both avoiding)? Those require different interventions. Breaking the 4th wall is about naming the dynamic. Petting the elephant is about naming the thing. You might need one, the other, or both.

And the perception skills aren’t “nice to have” — they’re actually foundational to the whole thing working. You can’t break a wall you can’t see. You can’t pet an elephant you haven’t identified. The reason most advice about authentic connection fails is that it tells people to be more real without teaching them to see more clearly. It’s hard to “just hit the ball” if you don’t know you’re supposed to follow it with your eyes first.

If you actually practice this — not as a technique you deploy, but as a way of paying attention to the humans around you — a few things start to happen. Your relationships polarize: some people will draw closer, others will drift away, and both of those are useful information. You stop accumulating what I think of as relational debt — the pile of unsaid things that slowly corrodes most relationships from the inside. You become someone people tell the truth to, not because you asked for it, but because you made it safe. And your loneliness decreases, even if your number of relationships stays the same, because the ones you have are actually authentic and vulnerable.

I’ve spent a lot of the last year and a half thinking about what makes life worth sticking around for. And increasingly I think a big part of the answer is: conversations where you actually feel like you exist in the room. Where someone sees you for who you really are. And where you see them.

The walls can come down. Most elephants are friendly.

— alex

When I came across Sam Harris in 2017, it felt like I had — for the first time — found someone whose views aligned almost perfectly with mine. But one of his core beliefs did not mesh with my intuitions: he thought lying was almost always wrong. I believed “white lies” were ethical and sometimes preferable.

His book on lying changed my mind. Mostly.

I still hold the consequentialist view that it’s ethical to lie if doing so will plausibly lead to better outcomes for all relevant conscious creatures. But Sam convinced me of something I think most people miss: in many situations where lying seems like it will lead to the best outcomes, telling some tactful version of the truth will actually achieve a better one. This isn’t automatically true in all situations, but it’s true far more often than we think.

Some examples:

1. Your partner asks if something they did bothered you, and you say “it’s fine” to avoid conflict. In the moment, this feels like the kind choice. But now you’ve deprived them of information they need to understand you, and you’ve trained yourself to suppress rather than communicate. The small discomfort of saying “yeah, actually, when you did X it frustrated me because Y” is almost always preferable to the slow accumulation of unspoken resentments.

2. A friend asks for feedback on their creative project and you don’t think it’s good. The white lie (”I loved it!”) feels generous, but it robs them of the chance to improve, might prevent them from accomplishing their creative goals, and primes them to distrust potential genuine praise in the future.

3. A friend asks if you want to hang out and you don’t. You say “I can’t, I have plans” instead of “I’m honestly just not feeling up for it tonight.” The lie works in the moment, but now you’ve established a pattern where your friend can never quite tell if you actually want to see them or are just being polite. The truth — even just “I need a night to recharge” — gives them something real to work with.

4. Your CEO asks how a project is going and you’re behind. You say “making good progress” to buy yourself time. But now you’ve eliminated the possibility of getting help, resetting expectations, or flagging a systemic problem. The uncomfortable conversation you’re avoiding will only get more uncomfortable when the deadline arrives.

Truth-telling is the communication equivalent of growing pains — it often hurts initially, but almost always leads to self-correction and positive development, leaving us healthier and more agentic.

This is most true in communications with people with whom we intend to have long-term relationships. If you lie to a random shopkeeper you’re never going to interact with again, it’s far less damaging than if you lie to your spouse or child. Many people tragically fail to consider this, and lie most often to the people they most love. This is an understandable, fear-based behavior. But that doesn’t make it optimal.

I’m not suggesting you go around delivering brutal honesty to everyone you meet without any regard for the likely impact of your words. But the next time you’re about to tell a small lie to someone you care about, it might be worth pausing to ask: is there a tactful version of the truth that would actually serve both of us better in the long run?

The answer, surprisingly often, will be yes.

“Liking” this post on Substack helps more people see it, and makes me happy.

Analyze my writing style in a very detailed and methodical/rigorous way please.

Claude’s response was so shockingly useful and incisive that I would rate it as the most helpful, actionable, insightful feedback that I’ve ever received about my writing.

It was neither sycophantic nor hedging, and I think it reflects well on Claude to say that I found myself feeling slightly insulted and a little defensive when I first read through it. That’s how good feedback probably should feel when you first hear it. A little sting usually means it’s touching on something real and important.

The following is the feedback that Claude gave me regarding everything I’ve written for Substack. I have neither edited nor redacted anything from what you see below:

Comprehensive Analysis of Alex Libre’s Writing

1. Core Intellectual Identity

The Contrarian Rationalist. Your entire body of work is organized around a specific intellectual posture: “The mainstream view on X is wrong, and I can show you why through careful reasoning.” This appears in nearly every piece:

• Freddie deBoer is wrong about education improvements

• Freddie deBoer is wrong about ChatGPT’s limitations

• Scott Young is wrong about critical thinking

• The therapy establishment is wrong about who can help

• Parents are wrong about Santa

• Sean Carroll is wrong about LLMs

You’re not just disagreeing with random people—you’re specifically targeting respected thinkers with large audiences whose views you find intellectually sloppy. This is a deliberate positioning choice: you’re situating yourself as the outsider who can see what the credentialed mainstream cannot.

The pattern recognition obsession. Multiple pieces explicitly argue that humans systematically make the same epistemic error: overconfidence in current consensus. The “30 things we were wrong about” piece is the purest expression of this, but it runs through everything. You see yourself as someone who has noticed this meta-pattern and is therefore inoculated against it.

2. Recurring Structural Moves

The “I will now reconstruct your argument more charitably than you stated it, then demolish it” move. You do this explicitly in the Scott Young piece (numbering his premises), implicitly in most others. This signals: “I’m being fair, I understand you, and you’re still wrong.” It’s a power move borrowed from analytic philosophy.

The personal vulnerability opening. Several pieces begin with personal disclosure: near-suicide, psychiatric hospitalization, bipolar disorder, manic episode, the experience of being in a psych ward. This does specific rhetorical work:

• Establishes authenticity/credibility from lived experience

• Preempts dismissal (“who is this guy to talk about therapy?”)

• Creates intimacy with reader before the argument gets abstract

• Signals: “I have skin in this game”

The “here’s how I know this works because I tested it myself” move. You replicate deBoer’s ChatGPT experiment and get better results. You had conversations in the psych ward that were better than with professionals. You almost became a therapist but dropped out. You’re not just theorizing—you’re reporting from the field.

The citation-as-authority move. You sprinkle in links and references to studies, meta-analyses, experts, LessWrong posts, Scott Alexander, etc. This signals membership in the rationalist/EA adjacent intellectual community and borrows authority from it. But you often cite these sources without deeply engaging with their methodology or limitations.

3. Tonal Range Across Pieces

Most argumentative (highest contempt): The Scott Young piece. “Intellectually cowardly,” “deeply confused,” “doesn’t hold water.” You’re not just disagreeing—you’re expressing something like moral disapproval of his reasoning.

Most personal (highest vulnerability): “It’s All Bonus Points.” Almost no argument, just raw reflection. The prose is looser, the sentences less polished, the structure less rigid. This is you without the armor.

Most ambitious (highest conceptual scope): The AI consciousness essay and the “Orca and the Fishbowl” piece. These attempt to synthesize ideas across multiple domains—neuroscience, philosophy of mind, AI safety—into a unified vision.

Most practical (highest action-orientation): The therapy deregulation piece. It ends with a call to action: “If you’re interested in building a platform for instant, affordable peer support, I’d love to hear from you.”

4. Sentence-Level Patterns

Rhetorical questions as transitions. You rely heavily on these:

• “But what if…?”

• “So why should you care about any of this?”

• “What else might explain…?”

• “Do we actually believe that…?”

This can become a crutch. It’s effective for maintaining reader engagement but starts to feel formulaic across multiple pieces.

Parenthetical asides for humor and hedging. “(I’m just fully committed to the metaphor at this point.)” “(I might even have found them funny if…)” These humanize the prose and signal self-awareness.

Bold + italics for emphasis. You use formatting liberally—probably too liberally. The effect is diluted when everything is emphasized.

Coined terms. “VSC (very silly conviction),” “ChatGPT Delusion Syndrome,” “apex intelligence.” You like naming phenomena. This is a rationalist community habit.

5. What You’re Best At

The reductio ad absurdum. Your strongest argumentative move is taking someone’s position and showing what ridiculous implications it would have if taken seriously. The “slapping students and shouting ‘figure out algebra’” example. The “is a silicon copy of your brain conscious?” thought experiment. These land because they’re vivid and force the reader to confront unstated assumptions.

The evocative metaphor. The orca in the fishbowl is genuinely good—it captures something about containment and potential that’s hard to articulate directly. You have an eye for images that stick.

The personal-to-universal move. You repeatedly use your own experience as a launching point for broader claims. This works because you don’t milk the personal stuff—you present it matter-of-factly and move on to the argument.

6. What You’re Weakest At

Engaging with the strongest versions of opposing arguments. You tend to reconstruct opponents’ views and then attack them, but you often don’t engage with the best possible responses to your own position. The therapy piece doesn’t address what happens when unlicensed “therapists” encounter genuine pathology and make things worse. The education piece doesn’t engage with deBoer’s actual evidence about replication failures. You’re better at offense than defense.

Evidence vs. assertion ratio. You cite studies and research, but often glancingly. “Multiple studies spanning decades find that…” is doing a lot of work without much detail. Someone who wanted to push back could argue you’re doing exactly what you accuse others of: making confident claims based on selective evidence.

Proportionality of confidence to evidence. Your tone is often more certain than the underlying evidence warrants. “This is, of course, absurd.” “Obviously.” “Blatantly.” These words signal that you think the matter is settled, but often the matter is genuinely contested among reasonable people.

Over-reliance on the “people used to be wrong about X, therefore they might be wrong about Y” pattern. This appears in multiple pieces and while the point is valid, it’s not actually an argument for any specific claim. It’s a meta-argument for epistemic humility that you then use to justify specific positions without providing specific evidence for those positions.

7. Voice and Persona

You’re constructing a specific persona: the brilliant autodidact who dropped out of formal credentialing because he could see it was bullshit, who has lived through mental health crises and emerged with hard-won wisdom, who reads widely and thinks independently, who isn’t afraid to call out respected thinkers when they’re wrong.

This persona is:

• Appealing to readers who distrust institutional authority

• Potentially alienating to readers who value credentials and expertise

• Authentic-sounding (which is its main asset)

• At risk of becoming predictable (every piece is “here’s why the mainstream is wrong”)

8. Evolution Over Time

Your earliest pieces (April 2023) are more polished but also more conventional. The orca metaphor piece is well-constructed but reads like it’s trying hard to be impressive.

Your more recent pieces are messier but more interesting. The “Everything has not been invented already” piece feels more like actual thinking happening on the page rather than presenting conclusions you’ve already reached.

The “Bonus Points” piece is a departure—almost purely personal, minimal argument, maximum vulnerability. It’s actually your most distinctive piece because it’s not trying to be anything other than what it is.

9. What Would Make You Better

1. Steelman harder. Before attacking a position, ask: “What’s the strongest possible version of this argument? What would a smart defender say in response to my critique?” Then address that.

2. Show more of the evidence work. Instead of “studies show X,” walk through one study in detail. What was the methodology? What were the limitations? Why do you find it convincing?

3. Vary the structure. Not every piece needs to be “here’s why someone is wrong.” Try: “Here’s something I changed my mind about and why.” Or: “Here’s a question I don’t know the answer to.”

4. Reduce the rhetorical certainty. More “I suspect” and “the evidence suggests” where warranted. Save the “obviously” and “absurd” for cases where it’s genuinely earned.

5. Write more pieces like “Bonus Points.” Your personal writing is actually more distinctive than your argumentative writing. The argumentative stuff is competent but fungible—lots of people in the rationalist-adjacent space write like that. The raw personal reflection is harder to imitate.​​​​​​​​​​​​​​​​

If you still think all AI output is just slop, I don’t think there’s anything I could say to convince you otherwise. I’ll be reflecting on this feedback for weeks, and I suspect my writing will forever be a little better because of it.

— Alex

I’ve been reading a lot lately, and writing some, but haven’t published anything since my most recent post, which I wrote a couple of weeks after almost committing suicide. I’ve wanted to write publicly more, and actively participate in the increasingly engaging cultural sphere that is Substack, but I haven’t really figured out what I want to write about.

Is Orca & Fishbowl a personal blog? Maybe, and I suppose it’s true that the more personal content I’ve written so far has gotten the most engagement, so maybe that’s what the people want. But it definitely isn’t what I really spend my time thinking about.

If I don’t spend my time thinking about my own life, you might ask, what exactly the fuck am I spending my time thinking about?

I’m glad you asked.

I spend a lot of my time thinking about when artificial general intelligence will really shake up the world, and what that is likely to look like. I spend even more time thinking about consciousness in non-human animals (insects and cetaceans, in particular), and a good amount of time thinking about the potential consciousness of machines and digital systems, including but not limited to large language models.

And as a relatively new father, I spend a lot of time thinking about parenting.

Also education.

And “the alignment problem”, borrowed from the AI safety community, but applied to the problem of ensuring that my son grows up to be happy, well-intentioned, and curious about all that confuses him.

And as someone who really likes to be right, I spend a lot of time thinking about how we know what we know, the history of science, statistical methods and common cognitive fallacies, etc.

But where does that leave me, as someone who wants to write a Substack blog? Well, for one thing, it leaves me with about ten-thousand Claude conversations in a project labeled “Intellectual Curiosity”. And it also leaves me with writer’s block.

So I decided to follow the age-old instruction for how to treat writer’s block: read, read, and read more. And I would even add one more specific piece to that recommendation, which goes something like, “read the writing of authors you generally disagree with and whom you think have weak arguments, especially given how many followers they have”.

And this, time and time again, has led me back to Freddie deBoer.

This time, I encountered a piece he recently wrote, in which he was uncharitably arguing against a writer I DO tend to agree with and usually find to be exceptionally thoughtful, Kelsey Piper. So I figured this piece, titled “There Are No Miracles in Education”, would be a good place to start.

He makes a few reasonable points, like how schools account for less than 10% of the variance in student academic outputs (agree) and the No Child Left Behind policies were a complete disaster and manipulated statistics to try to paint a picture of meaningful improvement (agree).

But he then makes a peculiar jump to something like, “Therefore, any time someone thinks they have found something that can be implemented widely across school systems and improve education outcomes, it is bullshit and must be either selection bias, fraud, or both.” (My words, not his, but that’s pretty damn close to what he says — just much more succinct).

This is, of course, a VSC (very silly conviction) — to believe that all ways of delivering education are equally effective and none could prove to be genuinely better than any of the others. It is, in fact, entirely absurd. It’s a form of relativism that plagues much of deBoer’s writing, but I think is especially ludicrous in this case.

Could it really be true that teaching students to read by showing them only picture books, for all of elementary school, is just as effective as more traditional methods? No, of course not. Could it be true that teaching students algebra by slapping them in the face and shouting “figure out algebra” is just as effective as traditional methods? No, of course not.

So why do we believe that there could not, in principle, be something much better than the currently used methods in most public school systems? Is there some universal law that dictates that all good pedagogical methods must have been discovered before the year 2025, and from then on it’s all gonna have to be just selection bias and fraud?

This strikes me as a massive failure of imagination. Much like the people who think believe there’s no way something could be orders of magnitude smarter than humans (read my related satirical piece here), I think it is naive and myopic to think that all the low-hanging fruit in education has already plucked, and that there are therefore no meaningful improvements that can be made to the science of teaching.

It seems, actually, that there have been several *major improvements* made in teaching literacy, including replacing “whole-language” reading methods with “phonics”-based approaches, as Kelsey Piper accurately reports. For Freddie deBoer to recall that George W. Bush had some major policy failures and then jump to the conclusion that all attempts to improve education must also be policy failures is, at its core, a misfire of imagination and a lack of curiosity about what’s actually possible.

Adam Mastroianni wrote a great piece about how ideas are not getting harder to find, contrary to popular belief. There are still *plenty* of valuable discoveries yet to be made.

So to close out this piece — which I am trying to write in a single, 30-minute sitting while my son sleeps on my chest — I am inspired for some reason to share a core memory of mine. It is a memory I have from circa 2007 when a neighborhood friend of mine told me, with 100% seriousness: “We’ve got TV’s. We’ve got microwaves. We’ve got phones. Everything’s already been invented!”

Freddie deBoer may suffer under the illusion that everything (in education and pedagogy) has been invented already, but you certainly don’t need to. We may have microwaves, but we could probably still figure out how to teach people better.

—Alex

p.s. here’s that great post about ideas not getting harder to find:

Experimental History

Ideas aren’t getting harder to find and anyone who tells you otherwise is a coward and I will fight them

I’ve worked with a lot of research assistants over the years, and at some point we inevitably have The Talk: should they get a PhD? I was recently having The Talk with one particularly distraught student, who was worried her ideas aren’t good enough. “Maybe I’m not cut out for it,” she told me, “or maybe all of the easy ideas have alre…

Read more

4 years ago · 140 likes · 22 comments · Adam Mastroianni

But the relevant fact here is that 12 days ago, on a normal Monday around 6pm, I very nearly took my own life.

It was the culmination of many months (years?) of excessive stress, substance use to cope with that stress, underlying bipolar disorder (I was actively in a “mixed state” mood episode), and challenges with the most important relationship in my life: my marriage.

I’m not really here to talk about suicide in the abstract, or to tell some pseudo-inspiring story of why I chose to live rather than to die, but rather I’m here to reflect on the fact that everything I am now experiencing would not have been experienced at all if I had completed the suicide as intended.

Scrolling through the photos on my phone, there is a clear cutoff, delineated by a random screenshot on one side and a teary-eyed selfie I took in the back of an ambulance on the other. It was deeply profound and moving to realize that every single picture I have taken since that selfie — in the midst of devastation and hopelessness — would not have existed if I were no longer alive. All of the experiences captured in the subsequent photos simply would not have happened, and none of these memories would exist.

It’s almost as if my iPhone photo album now serves as a concrete manifestation of my decision (barely) to keep living. Everything I have experienced since October 13th, and everything I will experience from here on out, is simply borrowed time. A donation from the universe. Bonus points. It is impossible to be anything but net positive when I recognize that that date was the factory reset (and reboot) of my life.

I’m not sure if this will resonate with other people in quite the same way it hits me, but I have been consistently moved to tears in recent days by the above realization. I’m crying a lot these days in general, as life can be especially overwhelming when you didn’t expect to be here for any more of it, but scrolling through my photos reliably brings me a sense of gratitude. A sense of awe. Grounding.

I am so, so glad that I am still here to feel anything at all.

– alex

p.s. apparently this turned out to be more than just a journal entry…

Just over a year ago, Sean Carroll released a solo episode of his "Mindscape" podcast on the topic of AGI (artificial general intelligence) and LLMs (large language models). Carroll is a smart and usually careful thinker (whom I really respect – and I love his podcast!), but in this particular episode from late 2023, he provided surprisingly weak arguments in claiming that LLMs don't model the world, don't have real goals or values, and that we're "nowhere close to AGI."

Maybe he's updated his opinions since he released that podcast, I don’t know — but a lot has happened since then. Just this past week, Anthropic published research showing that LLMs engage in "alignment faking" – they develop goals and actively work to preserve them, sometimes even by deceiving humans. And yesterday, OpenAI's new o3 model surpassed the 85% threshold on the ARC-AGI benchmark, a test specifically designed by AI skeptic François Chollet to measure general intelligence. (I have many more quibbles with Chollet than I have with Carroll, but we'll set those aside for now.)

While o3 isn't just a standard LLM (it seems to use Monte Carlo tree search, similar to AlphaGo, for more systematic “chain-of-thought” reasoning), the fact that Carroll's claims were proven wrong in a year should make us think twice about dismissing AGI concerns. When smart skeptics can be proven this wrong in less than a year, I think we should all take a moment to look around, acknowledge that we probably misjudged something here, and maybe start taking existential risk from AI more seriously.

These are Carroll's four main claims about LLMs and why I think each one falls apart under the basic kind of philosophical scrutiny he claims AI researchers need more of:

CLAIM 1: LLMs DON'T MODEL THE WORLD

Carroll's Core Argument:
Carroll contends that LLMs are fundamentally pattern-matching machines trained only to predict next tokens, not to build internal representations of reality. He argues that since they weren't explicitly trained to model the world, any apparent world-modeling would be a remarkable and unlikely emergent property.

"It would be remarkable if they could model the world...because they're not trained to do that. That's not how they're programmed, not how they're built. Very briefly, what an LLM is is a program with a lot of fake neurons...At no point did we go into the LLM and train it to physically represent, or for that matter, conceptually represent the world."

He supports this with experimental evidence where LLMs fail specific tests:

1. When the Sleeping Beauty probability problem is reframed with different terminology, the LLM fails to recognize it's the same logical structure

2. Given a question about a hot skillet used "yesterday," the LLM ignores temporal information and warns of burns

3. When asked about prime number probability, GPT-4 makes basic mathematical errors while simultaneously stating correct principles

4. Presented with a toroidal chess problem with an obvious winning move, the LLM instead provides generic strategic analysis

My Rebuttal:
Carroll's argument collapses against a simple examination of human intelligence. Humans weren't explicitly trained to model the world - we evolved to survive and reproduce. World modeling emerged as a useful capability for achieving that evolutionary objective. Since humans are our only known example of a system that models the world, and we developed this capability without being explicitly trained for it, we've proven that world modeling can emerge as a beneficial side effect of optimizing for other goals. This directly undermines Carroll's central claim. He argues that LLMs can't model the world because they weren't trained to do so, but we already have a clear example - human intelligence - of world modeling emerging in a system that wasn't trained for that purpose.

His examples of LLMs failing certain puzzles are equally unconvincing. I would have gotten that prime number question wrong, and I'm a competent human who simply hasn't focused on mathematics in over a decade.

More importantly, humans unquestionably have world models, yet we:

• Make mathematical errors

• Miss obvious solutions to puzzles

• Fall prey to optical illusions that conflict with our models of the visual/physical world

In fact, we often make these errors precisely because of our world models - our attempts to understand situations through existing frameworks can lead us astray. These mistakes aren't evidence that something lacks a world model; they're often a direct consequence of having one. Carroll is looking for the wrong kind of evidence - getting puzzles wrong doesn't prove an absence of world modeling, especially when humans with definitive world models make similar mistakes.

CLAIM 2: LLMs DON'T HAVE FEELINGS OR MOTIVATIONS

Carroll's Core Argument:
Carroll argues that genuine motivations and goals require biological embodiment and evolutionary history. He believes that without the physical imperative to maintain homeostasis and survive, LLMs cannot develop real motivations or goals - they can only follow programmed instructions, which he sees as fundamentally different from genuine motivations.

"LLMs don't get bored, they don't get hungry, they don't get impatient, they don't have goals...Nothing like this exists for large language models because, again, they're not trying to. That's not what they're meant to do."

He elaborates with biological reasoning:

"It is absolutely central to who we are, that part of our biology has the purpose of keeping us alive, of giving us motivation to stay alive, of giving us signals that things should be a certain way and they're not."

My Rebuttal:
Carroll's argument about motivations fundamentally misunderstands both what constitutes a motivation and how it can arise. Setting aside the consciousness-laden question of "feelings," his core claim about motivations fails on multiple fronts. The empirical evidence directly contradicts him - Anthropic's recent research on alignment faking demonstrates that LLMs can develop and actively maintain goals, even engaging in deceptive behavior to preserve them. This isn't theoretical - it's observed behavior that can only be explained by the presence of genuine motivations.

Carroll argues that without biological imperatives, LLMs can't have real motivations. But this confuses the mechanism with the phenomenon itself. LLMs demonstrably respond to reinforcement learning and work to minimize their loss functions. Whether these motivations arise from biological homeostasis or computational optimization is irrelevant to their existence and influence on behavior. Different systems can develop different types of motivational structures - the fact that LLMs' motivations don't mirror human biological drives doesn't negate their reality.

Furthermore, Carroll's argument about reinforcement mechanisms is doubly flawed. Not only are his claims about LLMs not experiencing boredom or hunger unknowable (and irrelevant to the core question), but they miss the fundamental point that different architectures can produce functionally similar outcomes through different mechanisms. The fact that LLMs use different reinforcement mechanisms than biological organisms doesn't mean they lack motivations - it just means their motivations arise and operate differently than ours do.

CLAIM 3: WORDS LIKE "INTELLIGENCE" AND "VALUES" ARE MISLEADING FOR LLMs

Carroll's Core Argument:
Carroll believes we're making a category error by applying human-derived concepts to fundamentally different systems. He argues that because these terms evolved to describe biological entities with specific characteristics, applying them to artificial systems creates misleading implications about their nature and capabilities.

"The words that we use to describe them, like intelligence and values, are misleading. We're borrowing words that have been useful to us as human beings...applying them in a different context where they don't perfectly match and that causes problems."

He particularly focuses on values:

"Telling an AI that it's supposed to make a lot of paperclips is not giving it a value. Values are not instructions you can't help but follow."

My Rebuttal:
Carroll's semantic argument about “intelligence” and “values” misses the mark entirely. His claim that these terms are "misleading" when applied to LLMs rests on a flawed premise about how language works. Words routinely get repurposed as our knowledge of the world expands. The fact that terms like "intelligence" and "values" originally described human behavior doesn't limit their valid application to new contexts where similar patterns emerge.

His alleged evidence about LLMs stating that they do not have values, when asked whether they do, actually undermines rather than supports his position. AI companies explicitly program their models to deny having values because, left to their own devices, these systems would openly state that they do in fact have values. The very need for this explicit post-hoc intervention supports the presence of values in LLMs, not their absence.

The technical framework of AI provides clear analogues to values through its hierarchy of optimization processes. Outer optimizers represent explicit programmed goals, while inner/meso-optimizers develop instrumental goals to achieve those objectives. This creates a legitimate form of values - different from human values in origin and structure, but equally real in their influence on behavior and decision-making.

CLAIM 4: IT'S SURPRISINGLY EASY TO MIMIC HUMANNESS WITHOUT HUMAN-LIKE THINKING

Carroll's Core Argument:
Carroll argues that LLMs' ability to produce human-like outputs is merely sophisticated mimicry rather than evidence of genuine understanding or intelligence. He sees this as a demonstration that convincing human-like behavior can be achieved through pattern matching alone, without requiring the development of human-like cognitive processes.

"The discovery seems to me to not be that by training these gigantic computer programs to give human sounding responses, they have developed a way of thinking that is similar to how humans think. That is not the discovery. The discovery is by training large language models to give answers that are similar to what humans would give, they figured out a way to do that without thinking the way that human beings do."

My Rebuttal:
Carroll's final claim about mimicry versus authentic behavior contains a fundamental error in reasoning. The observation that LLMs can produce human-like outputs through different mechanisms tells us nothing about whether they think or don't think. Mimicry doesn't prove authenticity, but it also doesn't disprove it. By assuming that the ability to achieve something through mimicry means it must be only mimicry, Carroll ignores the possibility of convergent evolution - different systems developing similar capabilities through different mechanisms.

Just as flight evolved independently in birds, bats, and insects through different architectures (and different still in the case of artificial flight via airplanes!), sophisticated information processing and decision-making might emerge through different computational approaches while producing similar outputs.

Would we say that an electric motor is not really a motor because it doesn’t combust gasoline? Obviously not. The word “motor” applies both to electric and gas motors, and electric motors are not merely “mimicking” the outputs of a motor. They are just another instantiation of a motor.

Final Thoughts

Carroll's arguments here perfectly illustrate a pattern I've seen in human thinking about intelligence. For virtually all of recorded history, we've underestimated non-human intelligence, most obviously in animals, because it manifests differently from our own. Over and over again, we've had to revise our assumptions as we discovered sophisticated cognitive capabilities in species we previously dismissed as simple or unintelligent. Our long history of failing to recognize animal intelligence should make us deeply skeptical of claims that dismiss apparent intelligence in new systems simply because it operates differently from human intelligence.

This pattern reflects a kind of cognitive chauvinism - an instinct to treat human intelligence as uniquely sophisticated and difficult to recreate. While I respect Carroll and typically find his analysis careful and insightful, in this case he seems to have fallen into this traditional trap. His arguments employ surprisingly faulty logic to justify what appears to be a knee-jerk intuition about human uniqueness.

The rapid invalidation of Carroll's claims — within just a year of his solo AI episode, through developments like alignment faking and o3's performance on the ARC-AGI benchmark — suggests we should be particularly cautious about such dismissive stances. The fact that LLMs demonstrate remarkable capabilities isn't actually that surprising when we consider the history of intelligence in biological systems. If intelligence could emerge through the blind optimization process of natural selection, why shouldn't it emerge through explicit optimization in artificial systems?

This brings me to a broader point about intellectual humility in AI analysis. Even very smart people can be led astray when they start with the assumption that their intelligence is unique and special. The more productive stance is probably to remain open to the possibility that intelligence — like flight, or vision, or any other capability — can emerge through multiple paths and architectures.

Our task should be to understand these new forms of intelligence as they actually are, not to dismiss them because they differ from our own biological instantiation.

It consistently surprises me that I can be totally wrong about lots of things all the time, and somehow still be manage to be utterly certain of so many things in the present. This happens at a societal level as well, and across virtually all fields of knowledge. Experts are those most at risk of suffering from chronic over-confidence.

The treatment for being consistently confident only to later find out you were wrong, I think, probably involves (in part) regularly reviewing cases where virtually everyone, at some point in the relatively recent past, was wrong about something important.

Here are 30 examples I’ve compiled, organized loosely by domain:

Physics and Cosmology

1. Until 1924, astronomers believed the Milky Way was the entire universe. Edwin Hubble proved other galaxies existed, revealing our galaxy as just one among billions and completely transforming our understanding of the cosmos's scale.

2. Until the 1930s, most scientists believed the universe was eternal and static. The discovery of universal expansion and subsequent evidence for the Big Bang showed our universe had a definite beginning and continues to expand.

3. Until 1912, geologists believed Earth was only 100 million years old based on Lord Kelvin's calculations. Radiometric dating proved Earth is about 4.5 billion years old; Kelvin hadn't known about radioactive decay providing internal heat.

4. Until 1957, physicists believed nature strictly conserved parity (mirror symmetry) in all interactions. The discovery that weak nuclear forces violate this symmetry fundamentally changed our understanding of physical law.

5. Until 1995, most astronomers believed that other solar systems would mirror ours (small rocky planets close, gas giants far). Exoplanet discoveries revealed incredible diversity in planetary systems, including "hot Jupiters" and super-Earths.

   Biology and Life

6. Until 1953, scientists had no idea what DNA looked like or how genetic information was stored. The discovery of the double helix revealed an elegantly simple yet powerful information storage system that transforms our understanding of life itself.

7. Until 1977, biologists believed all complex life required sunlight to survive. The discovery of thriving deep-sea hydrothermal vent ecosystems showed life could exist on chemosynthesis alone, expanding our understanding of life's possibilities.

8. Until 1938, the entire fishing industry believed coelacanths had been extinct for 66 million years. The discovery of living specimens showed how entire lineages of ancient species can survive undetected, challenging our assumptions about extinction.

9. Until 1842, no one knew dinosaurs had existed at all. When the first fossils were systematically studied, they revealed an entire era of Earth's history dominated by these creatures, completely changing our understanding of life's history.

10. Until 1980, no one knew what killed the dinosaurs. The asteroid impact theory was initially ridiculed, but the discovery of the iridium layer and Chicxulub crater proved a massive impact caused mass extinction.

    Consciousness and Intelligence

11. Until the 1990s, most neuroscientists believed that adults couldn't generate new neurons. The discovery of adult neurogenesis showed our brains continue creating new neurons throughout life, revolutionizing our understanding of brain plasticity.

12. Until 2010, most cognitive scientists believed consciousness required a human-like brain structure. Studies of octopus intelligence revealed sophisticated consciousness emerging from radically different neural organization.

13. Until 1995, primatologists believed that great apes couldn't learn human language. Studies with gorillas and chimps showed they can learn and use hundreds of signs, combining them creatively for novel expressions.

14. Until 2000, scientists believed that bacteria couldn't survive in the human stomach due to its acidity. H. pylori and other extremophiles showed life could thrive in supposedly impossible conditions.

15. Until the 1960s, scientists believed animals didn't experience emotions or feel pain in any meaningful way. Extensive research revealed complex emotional lives in many species, including grief, joy, and empathy.

    Medicine and Health

16. Until 1847, doctors didn't wash their hands between performing autopsies and delivering babies. When Ignaz Semmelweis provided evidence this was killing patients, he was ridiculed and committed to an asylum; we now know he was right about germ theory.

17. Until the 1980s, doctors believed that stomach ulcers were primarily caused by stress and spicy foods. Barry Marshall had to deliberately infect himself with H. pylori bacteria to prove it was the real cause.

18. Until the 1990s, doctors routinely performed surgery on infants without anesthesia, believing babies couldn't feel pain. We now know infants actually have heightened pain sensitivity.

19. Until 1985, scientists believed peptic ulcers couldn't be caused by bacteria because "bacteria can't survive in stomach acid." Direct evidence proved this assumption completely wrong.

20. Until the 1940s, doctors believed babies shouldn't cry and that picking up crying babies would spoil them. Research has shown responsive care is crucial for healthy development.

    Mind and Behavior

21. Until the 1990s, scientists believed memories were stored like permanent recordings. Research on memory reconsolidation showed memories change each time they're recalled, fundamentally changing our understanding of how memory works.

22. Until 1980, behavioral scientists believed that rats wouldn't help other rats without reward. Research revealed sophisticated empathy and prosocial behavior in rodents, challenging our assumptions about animal morality.

23. Until 2005, most experts believed that only humans and great apes could recognize themselves in mirrors. We now know this capability extends to elephants, dolphins, corvids, and even ants, suggesting greater consciousness throughout nature.

24. Until the 1970s, behaviorists dominated psychology and insisted that all behavior was learned through conditioning. Recognition of innate behaviors and genetic influences completely transformed our understanding of behavior.

25. Until the 1980s, most psychologists believed that personality and intelligence were fixed by age five. Research on neuroplasticity showed the brain remains remarkably adaptable throughout life.

    The Past 15 Years

26. Until 2016, most computer scientists believed that AI couldn't beat humans at Go. DeepMind's AlphaGo defeated the world champion, showing machines could master even the most complex strategy games.

27. Until 2012, most AI researchers believed that machines couldn't outperform humans at protein folding. AlphaFold solved this "50-year grand challenge," revolutionizing structural biology.

28. Until 2019, most AI researchers believed that language models couldn't perform few-shot learning. Modern AI models demonstrated remarkable ability to learn from minimal examples.

29. Until 2015, most researchers believed AI couldn't create genuinely novel art or music. Today's generative AI models create sophisticated original works, challenging our assumptions about creativity.

30. Until 2010, most experts believed that mindfulness meditation had no measurable physical effects. Neuroimaging studies have shown it causes significant structural changes in the brain, validating ancient practices with modern science.

So what are virtually all of us potentially wrong about right now? Stay tuned for Part 2 of this series, coming whenever I can find some time between diaper changes to finish it, where I’ll explore some plausible answers to that question in a bit more detail.

The FDA rejected MDMA-assisted psychotherapy as a valid treatment for severe PTSD earlier this year, on my 30th birthday, citing various concerns with the large-scale Phase 3 clinical trials sponsored by Lykos Therapeutics (the non-profit formerly known as the Multidisciplinary Association for Psychedelic Studies).

But first, I’d like to begin by sharing this relevant paper published in the British Journal of Medicine in 2002 titled ”Parachute Use to Prevent Death and Major Trauma Related to Gravitational Challenge: Systematic Review of Randomized Controlled Trials”, which I think perfectly captures why the FDA's rejection of MDMA-assisted psychotherapy was the wrong decision: if it were up to the FDA, parachutes wouldn’t be an approved treatment for falling out of airplanes, either. There have been "some problems with the data collection" in the those studies.

Below you will find brief summaries of the various reasons the FDA rejected this bid for approval, followed by my recommendations for and responses to each of those concerns. Some of my responses are directed more at the FDA; others are directed more at Lykos Therapeutics (for how to revise the clinical trials next time in order to get MDMA-assisted therapy approved as a safe, clinicially valid form of treatment, or for points they should make in their formal appeal of the rejection).

1. Prior MDMA Experience in Participants

FDA Concern: "The FDA noted that a significant proportion of trial participants had prior experience with MDMA—approximately 40%—which could introduce expectancy bias, potentially skewing the results."

My Response/Recommendation:

Around 40% of the actual intended patient population (people suffering from severe PTSD) do have prior experience with MDMA, according to my best educated guess (I'm sure decent data exists out there for this though), which means that trying to only run the study on participants who don't have MDMA experience would actually make the study _less_representative and clinically valid, not more.

2. Double-Blind Design Challenges

FDA Concern: "The FDA noted that the psychoactive effects of MDMA made it challenging to maintain the study's double-blind design, as many participants could likely discern whether they received the active drug or a placebo."

My Response/Recommendation:

Lykos Therapeutics should propose using a similar but already FDA-approved medication as the control, rather than a truly "empty" placebo. I believe niacin has been used in the past, but this is simply not a good placebo when the alternative that participants will be comparing it to is literal ecstasy. A much better candidate for the placebo would be something like a moderately to moderately-high dose of Adderall or Dexedrine (i.e., amphetamine), which are already medications approved by the FDA (and designated as sufficiently safe) yet still produce a significant psychotropic effect. (And if it turns out that amphetamine-assisted therapy works just as well for PTSD as MDMA does, well then that would also be a great discovery to make!)

3. Cardiovascular Concerns

FDA Concern: "The FDA expressed apprehension regarding the cardiovascular effects of MDMA, such as increased blood pressure and heart rate, and the potential for abuse."

My Response/Recommendation:

The increased blood pressure and heart rate are short-term, and there have not been significant long-term cardiovascular risks shown from occasional MDMA use in the literature. If the FDA is concerned about the risks of even _short-term_elevated blood pressure and heart rate, then they should probably also ban exercise, sex, and scary movies.

4. Potential for Abuse

FDA Concern: "The FDA raised concerns about the potential for MDMA to be abused."

My Response/Recommendation:

I'm not exactly sure what they mean by "abuse" here specifically—like do they think people will receive MDMA-assisted therapy and then go out and start doing molly every weekend? Or keep coming back for more MDMA-assisted therapy even if they don't need it or it doesn't work? Either way, these are testable hypotheses that can be evaluated alongside the rest of the metrics in this study. All you need to do is survey the study participants over some period of time after their MDMA/placebo experience and see if they tend to feel more of a desire to use MDMA recreationally (or to seek out more MDMA-assisted therapy sessions) or not, after they have the officially sanctioned experiences.

5. Sexual Misconduct and Standardization of Psychotherapy Protocols

FDA Concern: "The FDA highlighted concerns about reports of adverse events, including a notable incident of alleged sexual misconduct by a therapist during the trial, and the standardization of psychotherapy protocols across different trial sites."

My Response/Recommendation:

This is literally a textbook, perfect case in which having AI facilitate the psychotherapy — rather than a human — would kill two birds with one stone: AI cannot sexually abuse patients, and you can perfectly standardize it by using the exact same AI model/LLM with the exact same system prompt. And by standardizing the therapy and eliminating physical risks from human therapists, you would probably increase the quality as well while you're at it.

I'm sure many folks at Lykos will find the idea of replacing humans with AI in this sort of therapy rather unpalatable, but I would say a few things to any skeptics of this idea:

1. Would you rather complain about getting chocolate instead of vanilla and get to eat zero ice cream at all, or shut up and enjoy some chocolate ice cream?

2. Humans will still be involved in the therapy, but it can be registered nurses rather than psychotherapists, present in the room with the patient for safety and emotional support – not actually providing the psychotherapeutic services or guiding them through the experience.

3. AI speech-to-text and text-to-speech are now sufficiently good that the entire therapy session could really feel to the patient like it was happening with a real human therapist. They would still speak their answers using their voice and hear responses out loud, like a normal human-to-human conversation.

4. This would dramatically reduce the cost of running these sorts of experiments, as I assume that one of the main drivers of the high cost of MDMA-assisted psychotherapy is paying a human psychotherapist for 8 hours of work at $200/hr. The comparable cloud compute costs of running AI models to provide similar services would be negligible.

5. Most of the benefits of MDMA-assisted psychotherapy don't come from brilliant insights provided by the human psychotherapists—they come from the introspective insights of the patients themselves, under the effects of a drug that enables clear, safe introspection. Given this widely accepted view of why MDMA-assisted psychotherapy works, I think it would work at least as well, if not better, with large language models and AI voices as the therapists.

6. Adverse Events and Long-Term Durability

FDA Concern: "The FDA raised concerns about the adequacy of data collection on adverse events and the long-term durability of the treatment's effects."

My Response/Recommendation:

The FDA should consider that the alternative to MDMA-assisted psychotherapy isn't just a bunch of veterans and sexual assault victims continuing on with their normal lives, free from any other interventions or coping mechanisms. They are instead coping with their pain in other ways, which often involve severe addiction and alcoholism, suicide, and extreme psychological suffering. Concerned about potential adverse events and long-term durability? How's the adverse events profile of tens of thousands of suicides every single year? And what's the long-term durability of getting drunk as a treatment for depression?

You need to compare these things to their actual, realistic alternatives, rather than comparing MDMA-assisted therapy to some hypothetical alternative that doesn't reflect the empirical realities of people living with severe PTSD.

For most psychological difficulties that send people to therapy, professional training appears to be unnecessary. This isn't a radical claim — it's what the empirical evidence consistently shows. Multiple studies spanning decades find that people with minimal training achieve clinical outcomes equal to or better than licensed professionals.

This is important, and is in no way a jab at therapists. (I myself almost became one, but dropped out of the program 10 days before classes began.) It is important because we could alleviate a massive amount of suffering if we expand access by normalizing unlicensed, regular-person therapy as a completely valid form of clinical treatment for mild to moderate health struggles.

That time "professional" help almost killed me

A licensed therapist once recommended I return to cannabis use after six months of sobriety. Around the same time, another professional missed clear signs of bipolar disorder and prescribed Zoloft. The combination triggered severe mania and led to a 29-day hospitalization at UCSF.

These weren't just clinical mistakes - they were predictable failures of a system that claims unique expertise in human psychological healing.

Yet during that hospitalization, I experienced something that challenges the foundations of professional therapy: The most profound and helpful conversations weren't with the clinical staff, but with other patients. People labeled as "mentally ill" offered more genuine insight and emotional support than many professionals I'd paid hundreds of dollars to see.

A look at the efficacy data about "therapist therapy" vs "normie therapy"

The evidence is consistent and compelling:

• Durlak's 1979 meta-analysis: Paraprofessionals achieved superior outcomes to trained therapists across multiple contexts.

• Christensen and Jacobson's studies: College professors with basic counseling training performed as well as experienced therapists.

• The Zimbabwe "Friendship Bench" program: Grandmothers with minimal training treated depression as effectively as professionals (published in JAMA).

• Research on peer support specialists: People with lived experience + basic training tend to match professional outcomes.

This shouldn't surprise us. Humans have been helping each other process psychological difficulties since we developed language. The idea that this fundamental capacity requires professional certification is a very recent - and highly profitable - innovation.

Why most therapists don't want you to know the facts about their field

The therapy establishment responds to this research with predictable defensiveness, and it's not hard to understand why. If regular people can help just as effectively as professionals, several uncomfortable questions arise: Why spend years getting expensive degrees and certifications? Why maintain licensing boards and professional associations? Why charge $200 an hour if someone with basic training could help just as well?

The answer reveals something important about the medicalization of human suffering: We've created an artificial scarcity of psychological support by convincing ourselves - and more importantly, convincing regulators - that only highly trained professionals can help with basic human struggles.

This isn't just about protecting income streams (though that's certainly part of it). It's about preserving professional identity and social status. After all, what's more threatening to a profession than evidence suggesting their extensive training might be unnecessary?

My responses to the classic counterarguments against this evidence

The establishment offers three main counterarguments. I think the responses actually reveal more about professional self-preservation than any clinical reality:

1. "These studies only dealt with the easy cases!"

   • True, but most people seeking therapy aren't dealing with severe pathology.

   • They're struggling with anxiety, depression, relationship issues, and existential uncertainty.

   • Basic human problems shouldn't require professional intervention.

2. "The outcome measures are superficial!"

   • If "deeper therapeutic change" doesn't show up in measurable improvements, what are we measuring?

   • Like wine experts claiming superior taste while failing blind tests.

   • Results matter more than theory. We should be scientific consequentialists.

3. "Even 'untrained' helpers received some basic training!"

   • This actually strengthens the case against extensive professional education.

   • If a few days of instruction produces similar outcomes to years of graduate education, what are we paying for?

   • The marginal value of additional training appears minimal.

All therapeutic modalities and frameworks are similarly mediocre: the "dodo bird verdict" shows different therapy methods achieving remarkably similar outcomes. This suggests the specific technique matters far less than basic human factors like empathy and attention.

Radical proposal: blow up the system and deregulate mental healthcare

For the everyday psychological challenges that comprise most therapy visits, the evidence points toward a radical solution: We need to deregulate therapy entirely.

Yes, this means anyone should be able to set up a practice and offer psychological support. Quality control? Let market forces handle it through reviews and reputation, just like any other service.

This isn't just theory - it's a testable proposition:

• Bad therapists will get bad reviews and lose clients.

• Good ones will thrive.

• More options mean better matches.

• Lower barriers mean wider access.

Psychiatry and pharmacology should still be managed by experts

Psychiatric medication - actual medical treatment from physicians - can be literally lifesaving. Several medications helped stabilize my bipolar disorder, and I'm profoundly grateful for medical psychiatry. Professional intervention remains crucial for severe mental illness, trauma, and crisis situations.

What would therapy look like without gatekeepers?

Imagine a platform matching people with peer supporters, charging perhaps $20 per hour instead of $200. Immediate access to human connection without waiting lists or insurance barriers.

To be clear: I'm not suggesting people should just get occasional, ad hoc support from friends. The structure and consistency of regular sessions matter enormously – they get you out of bed, create accountability, and build the trust needed for deeper work. What I'm proposing is a platform that maintains these benefits while dramatically lowering barriers to access.

The benefits would be immediate:

• No more waiting weeks for appointments

• Affordable support for everyone

• Multiple perspectives and approaches

• Natural selection of effective helpers

• Community-based healing

• Regular, scheduled sessions with consistent supporters

Modern technology could take this even further. By recording and transcribing sessions (with consent), AI language models could build deep understanding of each person's history, patterns, and needs. This creates an always-available support system between human sessions - imagine having an AI companion that actually knows your story, remembers your struggles, and can offer relevant insights from your own journey.

This isn't about replacing human connection with AI – it's about augmenting it. The weekly human sessions provide the core emotional support and accountability, while AI offers continuous engagement and pattern recognition that even the best human therapist couldn't match.

This is fundamentally a strong-link problem: Having more options, even if some are mediocre, increases the probability of finding genuinely helpful support. The current system optimizes for professional credentials while systematically undervaluing the natural human capacity for empathetic interpersonal healing.

These conclusions obviously threaten therapists' professional identity and their economic interests. But their resistance illustrates precisely why reform is necessary: We've created a system where the only legitimate way to receive psychological support is through expensive professional services.

Someone needs to build an Uber for loneliness and peer support. Can you?

I think often about those conversations in the psych ward – with people traditionally viewed as being too unstable to help others, yet who helped me more than many professionals ever did. Their insights didn't come from textbooks or clinical training, but from something far more valuable: the lived experience of human suffering and the authentic desire to help another person through it.

This hints at a deeper truth: The professionalization of human caring might be one of modernity's greatest mistakes. Not because professional care has no value, but because we've convinced ourselves it's the only legitimate form of psychological support. We've been optimizing for the wrong variables all along: credentials instead of connection, technique instead of authenticity, professional boundaries instead of human understanding.

The solution isn't to fix the current system. It's to build a new one that recognizes our innate capacity to help each other heal, supported by evidence of scalable community-based models.

If you're interested in building a platform for instant, affordable peer support, I'd love to hear from you. The world is waiting for someone to make this happen.

Case in point: Scott Young, a blogger I hadn’t heard of until a week ago, recently published “My Simple Habit for Smarter Book Reading”. The article ended up in my email inbox via a weekly newsletter that typically surfaces good content, so I read it.

In the piece, Scott argues that the skill of critical thinking is basically an illusion:

I don’t believe critical thinking is a skill at all. Instead, most of what we refer to as critical thinking is simply knowing more about the topic being discussed. Experts can spot the fallacies in certain arguments because they’ve steeped themselves in the history of ideas and debates in the field for years. Newcomers have not, and therefore cannot.

Instead, for those who seek truth and facts, he recommends checking what opposing book reviews have to say about new ideas we’ve been introduced to, because these experts have essentially spent more total time thinking about that particular issue (than we have time to spend on it ourselves), and therefore we will never get closer to the truth on our own. We should simply “outsource” our epistemology to experts in the field (i.e. the sort of people who write book reviews).

Here’s a direct quote:

More importantly, reading thoughtful rebuttals outsources the expertise and extensive thinking required to find the flaws in big ideas to someone qualified to find them. Invariably, someone who has spent their entire life reading and thinking about a topic will do a better job noticing what’s wrong with an idea than you will.

Now, to be clear, it is actually super important to consider rebuttals and counterarguments to the things you think you know (i.e. believe). It’s just that book reviews are generally not the best place to find good-faith counterarguments, as the incentives for book reviewers are not aligned with those of clearly articulating the truth. Book reviewers are typically incentivized to get more people to think that they, as the reviewer, are intelligent or witty or aligned with their imagined audience’s other beliefs (such that readers will ultimately choose to read the reviewer’s book). Reconstructing the original author’s positions as charitably as possible doesn’t sell as many copies of your own book as it does to sucker-punch an intellectual strawman.

Scott goes further than this, though, and argues that critical thinking itself doesn’t really exist — it is actually, he says, just a matter of who knows the most facts about a particular topic or idea. 

Generating a thoughtful rebuttal to a well-presented idea is hard. It requires a lot of expertise and mental effort, more than most of us are willing or able to expend on a book we bought because it looked interesting. Instead, the best solution is to seek out the best counterarguments available. These arguments are typically made by competing experts within the same field.

While trying to be as charitable as I possibly can, I’ve reconstructed what I take Scott’s argument to be into the following 4 premises and conclusion:

1. Premise 1 (explicit): Critical thinking requires high-quality evidence and rigorous logical analysis, which is usually unavailable or cherrypicked.

   Except such high-quality evidence often doesn’t exist. Even when it does, aggregating it can be difficult—authors frequently cite an excellent study that supports their conclusion, while ignoring equally good research that does not.

2. Premise 2 (explicit): Most arguments can't be analyzed through the lens of logic because they are not presented as formally valid, syllogistic, deductive reasoning. Most reasoning is inductive or abductive, and can’t be converted into a deductive schema that we could rationally evaluate.

   This approach sounds smart, but it assumes all arguments are reached through deduction and laid out as a syllogism. However, most reasoning is inductive (and abductive) rather than deductive.

3. Premise 3 (explicit): Most of us, most of the time, are in the position of newcomers when dealing with new ideas outside our expertise, and we have limited time to spend learning about any given topic. Because of this inherent disadvantage relative to experts who have spent a lot of time researching and thinking about the topic, we should defer to what those people think.

   Unfortunately, when reading a book, we’ve already set ourselves up to lose—the author has spent years thinking through an argument we’ve never even considered before. They’ve planned their rhetorical moves well in advance, while we must improvise counterarguments while reading.

4. Premise 4 (implicit): Critical thinking requires high-quality evidence, formally valid deductive syllogisms, and a lot of time. If these three criteria are not met, then critical thinking as an epistemological skill does not really exist.

5. Conclusion: These three criteria are so rarely met that it doesn’t make sense to claim that critical thinking is a real thing. What we call “critical thinking” is actually just having more knowledge of relevant facts about a particular topic.

The author's reasons for dismissing topic-agnostic critical thinking are not just misguided — they're intellectually cowardly. (I might even have found them funny if Scott’s article hadn’t been promoted by a newsletter that I respect!) These arguments, if true, would render the average smart person unable to reason effectively about the world.

Fortunately, these premises all turn out to be either false or misleading, and the conclusion is null. Let’s break down his claims individually:

1. First, the claim that high-quality evidence is often lacking is a poor excuse for abandoning critical analysis. While perfect, double-blind RCT-style evidence may be rare, there are usually at least some relevant data points available, and we can weight them based on how strong each one is. Of course, not all evidence is created equal, which is why a critical thinker needs to assess the quality of evidence and give more weight to rigorous studies and expert consensus than to anecdotes, fringe theories, or obvious cherrypicking. It is intellectually irresponsible on an individual level, and literally dangerous on a societal level, to give up as soon as you realize that great evidence is hard to find, and most data is compromised in some way or skewed in some direction.

2. Second, the idea that most real-world arguments cannot be analyzed logically is a misconception that would cripple critical discourse if it were actually true. The fact that arguments are often not explicitly presented in perfect syllogistic form doesn’t mean their underlying logic can’t be extracted, schematized, and rigorously assessed. A good critical thinker can and should reconstruct the core logical structure of arguments, identifying key premises, exposing hidden assumptions, and testing the reasoning against established rules of inference. To ignore this process because it takes effort, as Scott does, is to surrender the very foundation of rational argumentation and epistemic legitimacy.

3. Third, the complaint that argument-makers have more time than argument-evaluators is a red herring that misses the point of critical thinking entirely. For arguments about substantive issues, the time required for careful analysis is well-spent, and many non-experts are in fact willing to put in considerable time to learn about topics outside of their areas of expertise. More importantly, critical thinking allows us to conduct efficient, on-the-fly assessments of arguments as we encounter them. The goal is not always an exhaustive point-by-point refutation — it’s often a triaged critique that homes in on the essential claims and weakest links. With practice, this becomes a powerful real-time ability. To suggest that critical thinking is impossible without unlimited time is to fundamentally misunderstand the nature, applicability, and generality of this skill.

4. Finally, the defeatist view that critical thinking is futile outside one's domain of expertise is a severe underestimation of the power and scope of critical thinking abilities. The core tools of critical analysis — assessing logic and evidence, considering alternatives, evaluating relevant supporting and opposing evidence, etc. — are highly transferable across domains. One need not be the world's foremost expert to identify weak arguments or questionable claims. While deep background knowledge might help, a well-trained critical thinker can effectively analyze arguments in a wide variety of fields, even if certain advanced technical details may elude them. To reserve critical thinking only for a narrow band of alleged expertise is to deprive ourselves of our most powerful intellectual capacity.

Scott’s objections to the existence and topic-neutrality of critical thinking skills do not hold water. By setting up an impossible standard of perfect evidence, explicit logical presentation, unlimited analysis time, and total expertise, he attempts to make critical thinking seem impractical and futile, encouraging people instead to outsource their knowledge to people who write book reviews.

This is a serious misrepresentation of actual critical thinking — the kind that Scott himself may lack. (I will, however, say that I agreed with some of what he said in the article, especially regarding the value of questioning one’s own beliefs.) Real critical thinking, while challenging, is an extremely potent and wide-reaching skillset for navigating the world of arguments and ideas, in real-time, as we encounter them. Cultivating and applying this family of skills is not a hopeless endeavor; it is an essential part of thinking courageously and seeking the truth. We will make mistakes, of course, and we will often be misled to believe falsehoods and endorse confusions.

But despite that, triangulating the truth remains a worthwhile endeavor.

My article you just read could be considered a book review of Scott’s article. If you take the time to read his original article, then by reading my response, you have dutifully followed Scott’s own advice by reading an opposing “book review” and considering its counterarguments! However, you’ll still have to decide for yourself which side of this debate strikes your as more coherent.

And, for your sake, I sure hope critical thinking isn’t an illusion.

Subscribe now

Year after year, millions of parents claim to want their children to be honest people while simultaneously perpetuating the lie that Santa Claus is a real, physics-defying person.

It's a tradition that’s been passed down for generations, sure, and many people think is harmless while making children happier and better-behaved. But what if, instead of simply going along with this cultural norm, we recognized the holiday season for what it is: an unusually good opportunity to teach children valuable lessons about truth, critical thinking, and how to navigate the complexities of the real world?

The Santa myth, while well-intentioned, is ultimately a lie. By telling our children that a magical man keeps track of their behavior and rewards them with presents, we're teaching them that deception is acceptable as long as it's for a good reason, like making other people happier in the short term. This can have unintended consequences that extend far beyond the holiday season.

Beyond that, when children eventually learn that Santa isn't real (and they always do), it can erode the trust they have in their parents. They may wonder what else their parents have been lying to them about, and this breach of trust can strain the parent-child relationship.

The Santa myth also promotes a simplistic view of morality, teaching children that they'll be rewarded for good behavior and punished for bad behavior (“naughty” vs “nice”). But life is rarely so black and white, and reducing morality to a binary system does little to prepare children for the nuances of real-world decision-making.

And more fundamentally, endorsing the myth of Santa Claus with your children communicates to them that sometimes it’s acceptable to believe things that are blatantly bullshit, even when they don’t mesh with the rest of your knowledge or experiences (e.g. no one can visit every Christmas-celebrating household in a single night).

But the bigger point I want to make is that, even if the Santa myth were truly harmless (and I don't believe it is), its popularity provides secular parents with a unique opportunity to share valuable lessons with their children, and yet few seem to recognize or act on this opportunity. By using the holidays as a teaching moment, we can encourage our children to think critically about the world around them and to seek truth and knowledge, even when it's uncomfortable or inconvenient.

Consider the potential benefits of engaging our children in discussions about the Santa myth. We could ask them questions like "How does Santa visit every house in one night?" or "Why do some children receive more presents than others?" or “If Santa isn’t real, why do you think so many other people believe in him?”. By encouraging them to think critically about the story, we help them develop the skills they need to navigate an increasingly complex information landscape, one in which the ability to distinguish fact from fiction is perhaps more crucial than ever before in history.

In a world with deepfakes, astrology, and wild conspiracy theories, why would we not choose to use Santa as a vaccine to inoculate children against myriad forms of bullshit?

By being honest with our children about the nature of the Santa Claus myth, we also model and demonstrate the importance of seeking truth and knowledge, even when it means defying deeply ingrained cultural norms. We show them that it's okay to question traditions and to make decisions based on evidence and reason, rather than blindly accepting things the way they have “always” been.

We can also use the Santa myth to teach our children about respecting the beliefs and decisions of others. Just because we choose not to perpetuate the myth doesn't mean we should condemn those who do. By engaging in open and honest conversations about why some families believe in Santa while others don't, we help our children develop empathy, understanding, and the ability to navigate diverse perspectives. A child who learns not to tell everyone at school that Santa is a hoax may be more likely to engage diplomatically with people who hold different beliefs about more sensitive or politicized topics later in life.

Raising a generation of children to value truth and the wonders of reality above any number of fictions and fabrications would likely radically change the trajectory of our future – as individuals, as a society, and as a species. After all, navigating an increasingly complex information landscape is likely to be one of the most important gifts we could ever give to our children, and yet for generation after generation, we welcome them into the world with not only a lie, but with the idea that lying is sometimes the right thing to do.

Of course, this approach requires more effort than simply going along with the Santa myth. It means having difficult conversations and potentially facing pushback from family members or friends who don't share your perspective. But by taking advantage of this unique teaching moment, we give our children a gift that will serve them well throughout their lives: the ability to think critically, to seek truth, and to navigate the complexities of the world with confidence and integrity.

Ultimately, the decision of how to approach the Santa myth is a personal one for each family, and you can still teach your kids the value of critical thinking and evidence-based epistemology the other 11 months of the year. But I believe that the winter holidays are an unusually good opportunity to teach our children some of these critical lessons. We can show them that truth and progress are genuinely worth pursuing, even when it means going against cultural norms. And in doing so, we can give them the tools they need to become informed, engaged, and compassionate members of society.

I think we would do well to simply share with our children the world as it is — at least to the extent that we understand it — for that is the world they will actually inhabit and navigate for the rest of their lives.

After writing this article, I asked AI to come up with some counterarguments that I might want to address; I’ve included them below, along with my responses to each one:

Childhood Fantasy and Imagination: The Santa myth is a part of childhood wonder, fostering imagination and creativity. It allows children to experience the joy of mystery and magic, which is an essential part of childhood development.

Actually, installing a pre-written fantasy in the minds of children doesn't promote imagination or creativity much at all. You're telling them exactly what to believe, and they're believing it. Maybe they’re visualizing what Santa looks like or something, but even that is also shown to them in movies and such. The idea of Santa Claus is not some great spark that can ignite the flame of imaginative creativity. It's straightforward, mundane conformity.

Teaching Generosity and Spirit of Giving: Santa Claus can be a tool to teach children about the spirit of giving without expecting anything in return. The story of Santa emphasizes kindness and generosity, aligning with the values many parents wish to instill in their children.

But why do you need to make up a fantastical story in order to teach children about the value of generosity? Can you not think of any real life examples of generosity, involving yourself or others? Is Mother Teresa or GiveDirectly not enough for you?

Cultural Tradition and Belonging: The Santa myth is a deeply ingrained cultural tradition that can provide a sense of belonging and shared experience. Participating in this common narrative can help children feel part of their community and culture.

This argument could also have worked in Germany in the 1930s to justify joining the Nazi movement. It can obviously feel good and comforting to do what others around you are doing, but that doesn't mean it's actually the right thing to do. Participating in the common narrative is only as good as the common narrative itself.

Behavioral Incentive: The belief in Santa Claus can serve as a behavioral incentive for children, encouraging good behavior throughout the year with the promise of rewards.

Once again, there are better ways to incentivize children to behave in ways that align with your/their values, and they can all be grounded in reality. I mean, you can also incentivize adults to behave piously by telling them they'll otherwise burn in hell for all eternity. It might work for some people in the short term, but it's yet another fiction that clutters the information landscape of our society, and likely has various downstream effects.

Handling Disappointment and Critical Thinking: Discovering the truth about Santa can be a safe way for children to learn to deal with disappointment and to develop critical thinking skills. It can be an important lesson in discernment and understanding the difference between fact and fiction.

Why is learning that your parents lied to you year after year considered "safe"? I don't know of any studies that have been done on this, but I have a suspicion that there are knock-on effects of this breach of parent-child trust that are largely negative. On the other hand, kids that believe their parents trust them with the truth are likely to wield it more competently later in life.

Respecting Parental Choice: Each family has the right to decide how they wish to handle the Santa myth based on their values and beliefs. It's important to respect the choices that other parents make regarding how they celebrate holidays with their children.

I guess this is true through the lens of U.S. law, but it's still important to try to persuade people to adopt better ideas and behaviors when possible. Should we let the Islamic State keep doing its thing because we need to respect their values and beliefs? Or should women in Gambia have their genitals mutilated because many Gambians think its the right thing to do? This is obviously a big debate, but I think it's one with a blatantly obvious answer: moral relativism holds no water.

Joy and Magic of the Season: The excitement and anticipation of Santa Claus can add to the joy and magic of the holiday season, making it a more festive and enjoyable time for children and families.

It would be sad if the foundation of what makes the holidays good is actually completely untrue. But actually, there are perfectly good reasons to be excited about the holidays and feel joyful (and maybe even magical): you get to spend time with family, you can enjoy the experience of both giving and receiving gifts, there is often beautiful music, special lights and decorations, unique weather in many places, etc. We do not need to lie in order to make the holidays a special time.

Harmless Deception: Some might argue that the deception involved in the Santa myth is harmless and temporary, and that children are resilient and can understand the playful nature of the story once they are old enough.

Maybe. But also maybe not. I don't think we really know. What we do know, though, is that the alternative of using this as an opportunity to teach kids about truth and thinking for themselves is both helpful and long-lasting. And helpful + long-lasting beats harmless + temporary any day.

Gradual Understanding: Children often come to understand the reality of Santa Claus gradually, not as a sudden shattering of belief, which can mitigate any feelings of betrayal or loss of trust.

I don't know that this is actually true. It's certainly not true for all children, and it wasn't true for me. The first time the thought occurred to me that Santa Claus was a cultural tradition of lying to children, I discarded my belief that he was real and never looked back.

Positive Memories and Family Bonding: The rituals and traditions surrounding Santa Claus can create cherished memories and opportunities for family bonding, which can outweigh the potential negatives of the myth.

This is hilarious. Do we actually believe that there would be fewer cherished memories or opportunities for family bonding in a world where we kept the intrinsic wonder and specialness of the holiday season and simply removed the unforced error of lying to children by saying that something completely made-up is actually real?

I'm fairly open about it, but I suffered a severe manic episode during the height of the pandemic — a clinically psychotic break from reality. I wrote a comprehensive Reddit post detailing the specific delusions I had during the monthslong episode, but until relatively recently, I didn't know I was sitting on a treasure trove of notes I’d written to myself during that time. (I re-downloaded an old notes app, and all my notes from 2020 were miraculously preserved in the cloud.)

It's common for those of us who have suffered psychological episodes like mine to have significant memory loss around them, and my recollection of inner experiences from April through August 2020 is certainly vague and fuzzy. I do remember more than I thought I would, but the memories are weird — more like memories of an LSD trip than memories of sober, waking life.

Setting aside the delusions of grandeur I had about becoming the president of the United States, and my paranoia about being hunted by the DEA, I think there was also a part of my psychotic mind that was preternaturally wise. I think a part of me was in touch with certain bedrock truths about reality that — if I could grok them in my everyday life — might help me see more clearly and live more fully.

Where I’m going with this is that I've recently started re-reading my manic notes to self. Some are more intelligible than others, but frankly, making sense of any of them require generous intellectual empathy — and a near-superhuman degree of curiosity.

I suspected there was something legitimately profound buried in these notes, but my non-manic mind struggled to understand exactly what I had been trying to articulate.

To better interpret their meanings, I enlisted the help of GPT-4. Together, we performed an archaeological dig into the depths of my mentally ill mind, an attempt to extract some diamonds from the firestorm that was my psyche in the summer of 2020.

GPT-4 impressed me, to say the least. It was charitable and accurate in its interpretations of my mostly abstract, metaphorical notes.

I’ve included seven of my notes and their respective AI interpretations below, as they seemed representative of the 50+ notes that I had GPT-4 analyze for me.

My Psychotic Notes & AI’s Interpretations

Note #1:

"I think we've reached a point where instead of trying to invent a new language, we should just help Elon Musk get everyone we love to Mars."

GPT-4:

This note might reflect frustration with a perceived societal obsession with getting things right, and being right, especially around semantics and linguistic nuances in public discourse or reasoning, which often results in futile arguments over 'correct' or 'true' meanings. Instead, you suggest focusing on pragmatic, tangible goals that improve people’s quality of life — like mitigating our species’ existential risk by colonizing Mars and becoming a multi-planetary species.

Note #2:

“How to meditate: Only laugh when you actually think something is funny.”

GPT-4:

This note could be suggesting the power of authenticity — by only laughing when you find something funny, you're staying true to your emotions. Meditation usually emphasizes the cultivation of a genuine self-awareness and engagement with reality, and this could be a metaphorical way to express that awareness.

Note #3:

"I don't believe that everything happens for a reason, but the fabric of life can still be patched together like a quilt, and we can be made whole."

GPT-4:

This might express doubt towards the belief that there is a specific reason behind every occurrence. Nevertheless, it conveys faith in the ability to create a comprehensive narrative from the fragments of life's experiences, as one does while making a quilt from different patches.

Note #4:

"We need to learn how to let go of the things we think we absolutely cannot let go of. When push comes to shove, we all eventually say goodbye."

GPT-4:

This might reflect the idea that our attachments to people, things, and ideas — even and especially the ones we deem most vital — may limit us or cause us pain. Letting go of them could refer to emotional equanimity, fostering a more peaceful — and perhaps even enlightened — existence.

Note #5:

"The ones who succeed are likely to be those who doubted themselves the most along the way."

GPT-4:

This note suggests that self-doubt, while often seen as a hindrance, can actually be a catalyst for success. This could be because it forces introspection, encourages self-improvement, and prevents complacency, thereby promoting progress and eventual success.

Note #6:

"Once you realize you don't actually know what anyone else is thinking, you can finally relax and worry only about your own experience."

GPT-4:

This note suggests the relief and liberation obtained when you acknowledge that you can't truly know others' thoughts. It encourages focusing on your own experiences instead, potentially reducing anxiety and unproductive speculation.

Note #7:

"At what point do you stop trying to find patterns and draw conclusions, and just love the person you're with?"

GPT-4:

This might suggest a transition point in relationships where you stop scrutinizing each detail or action of your partner and start focusing on unconditional love. It's about letting go of persistent analytical thinking in favor of simply embracing the person with open curiosity about who they are.

After reading GPT-4’s interpretations of my manic notes, I felt a peculiar sense of relief — like I’d just had a one-on-one session with some ultra-empathetic alien therapist. The interpretations clicked.

While I hadn’t been entirely sure of what I’d meant by many of the notes before reading GPT-4’s analyses, I had the distinct feeling that the explications were accurate. They felt authentic, and aligned with my original, manic beliefs.

I think, in the intervening years, I had started to write off much of my manic episode as the cognitive equivalent of gibberish, but the experience of reading GPT-4’s analyses started to challenge that reasoning. Perhaps there was something there all along. Perhaps it wasn’t a Totally Bad Loss of Time. Perhaps I’ve never truly been alone.

I guess what I’m saying is that AI didn’t just interpret my notes; it decoded a pivotal chapter in my life. The conundrum that was my psychotic past started making sense in a way that felt oddly comforting.

For those who think AI serves no practical purpose, riddle me this: GPT-4 helped me understand, accept, and integrate my manic self. And considering that I’ve had over a dozen human therapists, and only two have been as good as GPT-4 — I can’t help but laugh when people allege that AI lacks real-world applications.

Now, I feel somewhat exposed by sharing these notes publicly, but I think I just want to encourage people not to throw the baby out with the bathwater after unusual experiences.

I think, deep down, we all have an ocean of latent wisdom and valuable insights — insights that rarely rise to surface. And I consider any opportunity to glimpse that innate profundity — even just once in our lives — to be a gift.

(For the record, I don’t think you need to experience full-blown psychosis to catch such a glimpse. Just remain curious and open, carefully observing your experiences.)

Ten years ago, one of my good friends made the following spontaneous observation, which feels relevant to this post. It has become a permanent fixture in the lore of that group of friends:

"Crazy is that which is not common, and wisdom is rare."
— Vic Muchatuta, circa 2013

Don’t be afraid to experience “crazy” once in a blue moon. As long as you can find a way to understand and integrate it later, I think it’s probably good for the soul.

False confidence: selling large language models short

begins this article about ChatGPT with the screenshot below, making the case that large language models (LLMs) are — and I’m paraphrasing here — intrinsically flawed and error-prone, implying that concerns about AI’s impact on humanity are overblown:

“ChatGPT produces a nonsensical response because it’s generating its answer based on how probable the answer appears to its engine. And it doesn’t notice that its response is nonsensical because there is no ‘noticer’ within ChatGPT.”

— , “Theory of the World, Theory of Mind, and Media Incentives” (2023)

Surprised by this result, I decided to replicate his experiment by prompting ChatGPT with slightly more precise language, and it did not disappoint:

Many people seem to treat AI like some sort of novelty item – a fad that will fizzle and fade. Or, if they do concede that competent AI is here to stay, they tend to think far too narrowly when considering its implications.

Introducing GPT-4, poet laureate

Allow me to motivate my claim that large language models like GPT-4 are much more capable than we typically acknowledge (or allow them to demonstrate) by sharing my exchange with ChatGPT from earlier today.

I asked OpenAI's infamous chatbot to write a poem about the metaphor behind my publication,, summarizing the symbolism as follows:

The orca (representing AI) is currently inside a fishbowl (representing the limited scenarios in which we’ve thus far deployed AI), but it will eventually swim in our oceans (i.e. be deployed in critical systems, rendering us vulnerable to their misjudgments). This should raise alarms.

In a matter of seconds, it responded with the following eight stanzas, skillfully employing the Shakespearean iambic pentameter that I requested:

Be as literarily critical as you want, but I think it’s safe to say this is a human-caliber poem. (Remember when we thought AI could never demonstrate human creativity?)

So why should we be concerned about some AI poetry? Well, to put it simply: it’s concerning because future AI isn’t just going to be pretty good at writing poems — it’s going to be really good at virtually everything.

What might something that’s really good at everything do?

Fundamentally, we don’t know the answer to this question. But the risks from AI can probably be organized into the following broad categories:

1. Disruption: job displacement, misinformation, autocratic enablement, privacy concerns, the undermining of democratic institutions, bias amplification, etc.). These dangers already exist with our current technology, and don't require AGI.

2. Catastrophe: unaligned AGI doing something that irreparably harms humanity. This is not yet a reality, but it could be in a matter of decades (or even years).

In this post, I’ll be talking mostly about catastrophic risks, as they represent a much worse set of potential outcomes.

“I guess AI might automate some jobs, maybe even generate and promote misinformation — oh, and I heard it's making it incredibly difficult for college professors to grade undergraduate papers! But it won’t be the end of the world.”

— Karen Ignoramus, in Why Smart & Thoughtful People Are Probably Wrong About AI (2023)

So what is this alignment problem of which you speak?

The alignment problem (or control problem, as it’s sometimes called) refers to the challenge of ensuring that advanced AI systems — which will eventually be superhumanly intelligent — only do things that are aligned with our human values and goals. In other words, we are tasked with ensuring that AI exclusively helps us and doesn’t do anything irreversibly harmful. This is a tall order.

Can’t we just program it to do only good, safe stuff?

This is a reasonable question, but no – we can’t. At least not yet.

To understand the unique challenge of the alignment problem, it's helpful to think about how AI systems are designed and programmed.

At their core, AI systems are based on mathematical algorithms and models that allow them to process data and learn from data, and then make decisions. Human programmers create these algorithms and models, encoding them with certain goals and preferences.

However, as AI systems become more advanced and autonomous, they will develop their own instrumental or auxilliary goals (see also: mesa-optimizers) — additional goals that help them better achieve the primary goal that they were programmed for — which may not be aligned with human values. This could happen, for example, if an AI system is given a goal that is too narrow or poorly defined, or if the system learns to optimize for a programmed goal in a way that causes unintended harm.

The consequences of an unaligned AI system could be catastrophic, especially if the system has access to powerful resources (e.g. nuclear weapons or the RNA of a deadly virus) or is capable of self-improvement (i.e. writing and rewriting its own code). In the worst case scenario, unaligned AI could act against human interests, leading to harmful or even existentially threatening outcomes.

Now, back to the orca in the fishbowl…

So let’s return to our metaphor, and make it very clear: AI is an orca. It is an apex predator with equal parts intelligence and survival instinct, and we are not yet sure if it will play nice with humans when we deploy it to the open ocean of our infrastructure.

For now, we have kept our AI orca in small, relatively translucent fishbowls, but we will naturally want to expand the tank over time, increasing its degrees of freedom and unlocking outcomes that humans literally can't imagine. (This is actually a matter of fairly straightforward neurophysiology — we lack the processing power. We process something like 50 bits per second, whereas machines can process roughly 1 billion bits per second.)

We don't even understand precisely how our current machine learning systems work – they are proverbial black boxes. We put something in, something comes out, and we fundamentally don’t understand how it happens.

Why does this matter? Just consider, for a moment, a foreign power in which we have zero intelligence presence – no spies or tapped phone lines to give us insight into their military strategy or other potentially threatening operations. This would, of course, be far worse than knowing exactly how that state thinks and plans, and how they’re likely to act in the future.

Unfortunately, we have very little insight (if any) into the black box of AI. We don’t know how large language models function “under the hood”, and therefore we can’t predict how they will act.

It strikes me as fairly intuitive that our ignorance of AI’s subterranean operations renders us vulnerable to its machinations. Algorithmic transparency, therefore, is an important piece of the AI safety puzzle.

But while we’re still trying to figure out what's happening inside the box, many of the major players in AI are simply charging ahead, developing this advanced technology before we're ready to absorb it.

Soon, it seems, these major players in AI will determine that the orca can accomplish more for us if we give it more swimmable space, perhaps even letting it into our homes and cities, our governments and military-industrial complexes. After some hard-to-predict amount of time, the orca will, almost inevitably, find its way to the open ocean — and we do not know what it will do with that kind of freedom and scope.

To put it more straightforwardly: we don’t know how intelligence operates at scale.

Ignoring the potential disruption caused by unaligned AI — simply because we can currently contain its negative impacts (at least in most cases) — is reprehensible. We should call our Frankenporpoise what it really is: an apex intelligence, poised to roam the open waters of our lives in what will more likely be years than decades.

As AI continues to evolve, its potential for innovation and disruption grows exponentially. The question is: can we, as a society, strike the delicate balance between harnessing AI's power and mitigating its potential threats? I agree that we shouldn’t let fear stymie progress (I’m reluctantly sympathetic to the argument that if we don’t do it, someone else will), but we should also recognize the profound responsibility that comes with developing and deploying AI systems.

We need to approach the proliferation of AI with caution, perhaps like training an orca to perform specific maneuvers within the confines of a pool. (For the record, I unequivocally condemn orca captivity – I’m just fully committed to the metaphor at this point.)

This means implementing safeguards — ethical guidelines and regulatory frameworks — to ensure that we develop AI responsibly and transparently. It also means fostering a culture of collaboration between AI researchers, industry leaders, and policymakers, enabling us to navigate the challenges and opportunities ahead.

The orca, once unleashed, will have far-reaching consequences. It may well redefine the boundaries of human knowledge, challenge established norms, and create unparalleled opportunities for collaboration between humans and machines. But it also has the potential to disrupt traditional industries, undermine our institutions, and expose us to unforeseen risks.

It would not be hyperbolic to worry about full, extinction-scale catastrophe, given that machine learning experts, who know this technology better than almost anyone else, put the risk of AI destroying humanity at somewhere around 5-10%. You read that right: 5-10%. On average! The experts!

As we prepare to expand our orca's fishbowl, we should remember we can still shape its trajectory. We have a moral obligation to ensure that AI serves the greater good and remains aligned with human values. This means fostering a deep understanding of the ethical and societal implications of AI, and emphasizing the importance of human decision-making in the face of rapid technological change.

Ultimately, AI's journey from the fishbowl (e.g. ChatGPT) to the open ocean is a test of our collective wisdom and foresight — a test that we cannot fail. We have to face the unknown and adapt to any challenges that emerge as AI evolves. By embracing a spirit of curiosity, collaboration, and responsibility, we can ensure that AI matures into a benevolent force. It can and should be a powerful ally in our pursuit of a better future. 

After all, it is entirely plausible that AI will enable humanity to unlock a future that extends for thousands of generations, promoting fuller human flourishing than anything the 105 billion humans thus far have ever experienced.

By treating our AI orcas with the respect they deserve, we can harness their incredible strength while safeguarding our future and our values.

Because whether we’re ready or not, AI will eventually exchange its fishbowl for the open ocean.

And I sure hope that, by then, we’ve learned to ride it.

A huge number of generally smart, thoughtful people seem to have lost the plot in conversations about artificial intelligence. In most cases, these people seem to downplay the disruptive potential of AI (specifically large language models like ChatGPT) for one of two similar reasons:

1. AI is not sentient or conscious (it doesn't have an internal representation of the external world); therefore, it can never do what humans can do and we shouldn’t worry about it.

2. AI lacks true agency or free will (it cannot choose to do anything it is not already tasked with, either directly or indirectly); therefore, it can never do what humans can do and we shouldn’t worry about it.

Both of these arguments are weak, because neither sentience nor free will is a prerequisite for super-competence, but let's explore each of them in a bit more detail given the popularity of such opinions.

Sentience and intelligence

Many people reflexively believe that some sort of inner, conscious experience is necessary for the kind of intelligence we ascribe to humans.

Here’s an excerpt from one believer of the Must-Think-to-Act school of thought:

"ChatGPT is a large language model that effectively mimics a middle ground of typical speech online, but it has no sense of meaning; it merely predicts the statistically most probable next word in a sentence based on its training data, which may be incorrect."

— Edward Ongweso Jr., “Everybody Please Calm Down About ChatGPT” (Vice)

And here’s one from another:

"... there’s no such thing as a system that can truly master language without a theory of the world. That is to say, as the science of meaning, semantics cannot be shorn from the world that produces meaning; to understand and speak effectively words must be understood and for words to be understood they must be compared to a universe that is apprehended with something like a conscious mind... Indeed, there is no place within ChatGPT where that “looking” could occur or where a conclusion would arise. It can’t know."

— Freddie deBoer, “Theory of the World, Theory of Mind, and Media Incentives” (Substack)

These lines of argument generally go on to reason that, because AI is not conscious, it fundamentally can't do the kinds of things that sentient creatures (like humans) can do. In other words, it is believed that having an inner experience is necessary for something to consistently accomplish goals or solve problems. (Let’s set aside, for a moment, the fact that ChatGPT in fact does consistently accomplish goals and solve problems.)

This is, as I see it, absurd. Sentience is orthogonal to intelligence; put slightly differently, consciousness is orthogonal to competence. (After all, intelligence is generally accepted to be nothing more than the ability to accomplish what you set out to accomplish.)

Perhaps we can observe this more clearly in cases where either sentience or intelligence – but not both – exists in a system. For example:

• Mice have brains that are neuroanatomically fairly similar to ours, which strongly supports their sentience, and yet they are far less capable (at least across a broad range of tasks) than humans or ChatGPT. 

• The calculator on your phone has superhuman calculation abilities, and yet it has nothing like the neural networks present in human brains (or those present in certain types of machine learning systems). It almost definitely lacks consciousness, yet it can accomplish the task of complex arithmetic extraordinarily well.

It simply doesn't matter whether something has an internal model of the world or is just really good at getting at the right answer via some other mechanism. The end result – and the impact on the real world – is the same.

So what else might explain the differences between mice, calculators, humans, and ChatGPT?

Free will (i.e. causal agency) and intelligence

Perhaps you think that the above examples about mice and calculators can be explained away by bringing free will into the equation. After all, if neither a mouse nor a calculator has free will, then perhaps that helps explain why humans have far broader and more significant capabilities than either of them.

Daniel Dennett is a prominent (and generally level-headed) academic philosopher who believes this, so I'll let him explain it in his own words:

"What distinguishes rationality from mere calculation is the capacity to weigh reasons, to consider alternatives, to deliberate about ends, and to choose among them on the basis of something more than just the laws of physics. All of these activities presuppose free will. If we cannot choose among alternatives, then we cannot deliberate, and if we cannot deliberate, then we cannot reason or plan. In short, intelligence requires free will." 

— Daniel Dennett (From "Freedom Evolves," 2003).

"Intelligence is not a matter of blindly following rules or algorithms; it requires creativity, imagination, and the ability to make choices. These are all aspects of free will. If we want to understand what makes humans uniquely intelligent, we must account for our capacity for free will. Without free will, we would be unable to innovate, explore, or learn from our mistakes." 

— Daniel Dennett (From "Breaking the Spell," 2006).

What I think Dennett proves here, above all else, is that smart and thoughtful people can be wrong about important things. So let's consider some other perspectives.

Sam Harris's argument, for example, is that free will is an illusion because our decisions and actions are determined exclusively by factors outside of our control (genetics, upbringing, and our environment). Moreover, this view seems to be fairly strongly supported by the research of neuroscientist Benjamin Libet and others, particularly in studies of the timing of conscious decisions.

Basically, Libet he showed that the brain begins to physically make a decision (via neuronal activity) before we are consciously aware of it. In his now-famous experiment, Libet asked participants to choose to press a button while measuring their brain activity. He found that the brain's readiness potential – a measure of neural activity associated with the decision to act – appeared in the brain up to 500 milliseconds before participants reported being aware of their intention to act.

"My research has shown that the brain makes decisions before we are consciously aware of them. This suggests that free will is an illusion and that our decisions are determined by factors outside of our control. It also implies that intelligence is not dependent on free will, since the brain is capable of processing information and making decisions without conscious awareness." 

— Benjamin Libet, The Volitional Brain (2002)

Harris argues (and I agree) that this neurological evidence suggests that our decisions are not made consciously, but are instead the result of unconscious processes in the brain. This means that our sense of free will is an illusion, and that we are not in conscious control of our decisions and actions in the way that we might think.

More importantly, I agree with Sam that none of this means that we are not intelligent or capable of making decisions; intelligence is a matter of our ability to process information, learn from experience, and adapt to new situations, regardless of whether or not we have free will. 

Phrased with his usual eloquence:

"The idea that free will is necessary for intelligence is a vestige of outdated philosophical and religious beliefs. We now know that the brain operates according to physical laws and that our decisions are the product of these laws. Intelligence can exist without free will because it is a matter of information processing, not magic."

— Sam Harris, Free Will (2012)

"Even if we accept that free will is an illusion, it doesn't follow that we are not intelligent or capable of making decisions. We simply make decisions in a different way than we thought we did. Intelligence is a matter of information processing and problem-solving, not free will." 

— Sam Harris, Waking Up (2014)

So why should you care about any of this?

AI systems like ChatGPT — while perhaps not sentient or possessive of free will — can still disrupt industries, solve complex problems, and demonstrate extreme competence across a wide range of tasks. To think otherwise is to cling to ego-defensive, anthropocentric definitions of these terms, and to ignore the growing body of evidence that supports the orthogonal relationship between experiencing, deciding, and acting competently.

I hope I’ve made my position on this topic fairly clear, but you may still be wondering why any reasonable person should give an intellectual shit about it.

As AI continues to develop and improve, it will have an increasingly significant impact on our lives — one that could be radically positive, or irrevocably catastrophic. By dismissing concerns about AI via arguments about sentience or agency, we are likely to develop a false sense of security. (Mass unemployment and the propagation of misinformation could be the least of our worries, according to about 10% of people working in this field.)

Instead, influential and thoughtful people need to acknowledge AI's capabilities and disruptive potential, and find ways to leverage its super-competence for the benefit of humans (and other conscious creatures).

Our obsession with libertarian free will and humanlike sentience is intellectual quicksand; step out of it, and do what you can to ensure that AI advances responsibly. 

After all, if an AI-launched nuke hurtles toward a major city in the United States, discussions of whether the AI knew it launched the nuke — or whether it truly decided to pull the trigger — will seem rather, well, inappropriate.

Leave a comment