Mission elapsed time 21,021 days.
Good morning, beloved -
I want to point you at this extraordinary work of analysis and synthesis that Erin Kissane posted on her wreckage/salvage site last night. It’s the fruit of what sounds like a yearlong effort on Erin’s part, to name and think about what’s happening to our collective ability to know things, and to do so rigorously, outside the received pop-epistemology frames of “information overload,” “filter bubbles” and so on. She’s pinned down something inchoate that I’ve been grappling with, alongside a number of other people I’ve been in conversation with this year, and put it into clear and expressive language better than just about anyone else has been able to. I think it’s really important.
You can and should read the piece itself, but if I can fairly summarize Erin’s concern here, it’s that certain features of the way we collectively develop and convey accurate understandings of the world are under existential pressure. She uses the model of the massive Valdez earthquake of 1964, of all things, to suggest some ways in which this pressure sets up some really scary conditions for the knowability of anything, or the survival of any society predicated on it.
Erin starts from the idea that the overwhelming majority of our knowledge of the world is received, neither personally developed by us through empirical inspection, nor derived logically from first principles. To know at all, then, is inherently, profoundly social: “We build our understanding of the world together,” she tells us, “from information assembled by others and then held in common,” with that common holding including all of the ways we’ve ever developed of getting knowledge out of our heads and into the world.
It may not be particularly controversial to point out that any model of a system accurate enough to afford intervention — what we might call “efficacious knowledge” — is smeared out across social and material space, rather than residing in any one individual consciousness. But one of the places where Erin’s piece really shines is in reminding us that the process of smearing-out consists of a double articulation, in which a phase of “expert knowledge production” is followed by “sensemaking” woven around that production.
In this context, I think we should understand the label “expert” to imply only what its etymology suggests: that the knowledge involved results from some more or less structured and iterative process of trial. It’s something developed through experience over time, in other words, involving the empirical testing of local reality. And that reality-testing can be pursued by anyone, and take any number of forms, from simply taking note of conditions as they change to the most elaborately instrumented probes of regimes far beneath or beyond the threshold of our own perception. This is the knowledge that’s inscribed in the world somehow, once produced, using any of the strategies for doing so our species* has yet devised, whether that means an oral tradition, cuneiform pressed into clay tablets, arXiv preprints or even gene sequences.
*Other species do this as well, of course, more of them and far more extensively than we still generally recognize.
But the greater part of us don’t ordinarily participate in this process of knowledge production ourselves. We, quite literally, leave that to the experts. And what Erin calls our attention to is how little of that efficacious knowledge we’re therefore actually able to call upon directly and for ourselves in the course of everyday life. When we’re confronted with a challenge that requires some tranche of knowledge-of-the-world, we consult the common archive — which, again, could mean anything from a litany passed down the generations, to a Carnegie-funded library of stone and heartwood, to Wikipedia — and we interpret what we find there locally and in real time, very often in conversation with similarly inexpert peers.
And we have a genius for this sensemaking, both individually and collectively! There’s all but invariably a cleverness and charm to our mental models of the way things work; I’m always reminded, in this respect, of the St. Petersburg cab driver who thought the Google service we were relying on for our conversation was underwritten by clever “Amerikanski techniki” listening in and translating it in real time, and maybe there’s more truth to that notion than service providers would have us believe.
But if the test of iterated, empirical, rigorously reality-tested expert knowledge is whether it is for all intents and purposes true, and that truth value is what permits it to be efficacious in deployment, sensemaking has a different, lower threshold to clear. That threshold is plausibility. And if all knowledge is social, historically contingent, both subject to and conditioning relations of power — as I am very far from alone in arguing it is — our sense of what is plausible is still more so. For some, it’s plausible that drinking the occasional glass of red wine helps to lower blood pressure; for others, that there’s a secret dungeon beneath a popular Washington DC pizza joint, spoken of by the powerful in only barely coded messages, in which the hapless victims of a global child-slavery network to this day cry out for rescue and go unheard.
And it’s the latter scenario that speaks to the particular aspect of contemporary sensemaking that most concerns Erin here, which has to do with the historically near-simultaneous breakdown of multiple systems that functioned to protect the place of expert knowledge across the better part of modernity. I should point out that this breakdown was, in part, the eventual result of a concerted effort by figures some of us continue to revere, perhaps most notably Ivan Illich, and undertaken for good and sufficient cause; I’m not at all sure we were better off under the regime of elite gatekeepers and credentialing institutions that for so long and so successfully connived at maintaining a certain cordon sanitaire around who got to advance knowledge claims in the public sphere and how those claims were evaluated. But I think it’s by now fairly plain to see that the eclipse of those institutions has not been an unalloyed good, or something that’s unfolded without giving rise to unintended consequences. The trouble, as I see it, is that the authority of these institutions was successfully and comprehensively eroded, without anyone devoting all that much thought as to the kind of just and responsive measures that might usefully replace it.
However partial, self-regarding and compromised that institutional reason might have been, its descent into sleep unquestionably produced monsters. What resulted from the eclipse of gatekeeping authority (very much including that inscribed tacitly, in agreements like common scholastic curricula) was a confusion at two levels: first that between what Erin refers to as “respectable and marginal sources of information” [emphasis added], and then that between primary knowledge production and the secondary act of interpretation we’re here calling “sensemaking.” And then the collapse of authority and the utter confusion over what constitutes a trustworthy source of information, both already robustly underway, get accelerated enormously by the mass global adoption of social-networking platforms in the years following 2008. In the aftermath, it’s very hard for anyone to discern, without a great deal of effort, what a given representation of fact is based on, or what relation that representation has with the real.
That effort is occasionally undertaken by autonomous actors like Forensic Architecture (who call this practice “counterforensics”) and Bellingcat (“open-source investigation”), or the few remaining viable commercial entities capable of conducting investigative journalism. But more often, in the absence of their intercession, complex events now very rapidly become shrouded in an epistemic fog — one composed not of too few explanations but of too many, with every proposition that can eventually be verified surrounded by a vast point-cloud of superficially plausible others which cannot.
What happens, when things have become confounded in this way, is that common sense just about everywhere tilts in the direction of self-reinforcing consensual hallucination, propagated by the social constellations Erin calls “marginal sensemaking communities”: those “that break the rules meant to keep mismatched frames like theology or mysticism out of zones like scientific inquiry.” (This is both an astoundingly clear and helpful definition, and a little frustrating, in that I think it cedes too much to the scientific enterprise’s own account of itself, but we’ll come back to that later.) In short, there is nothing left to stop those proffering locally plausible, but not true, explanations of reality from being taken seriously by very large, indeed societally salient numbers of people. This is how you get the pizza-dungeon story. This is how you get a hugely influential figure in midwifery telling her followers that “I actually don’t believe that gravity is true,” a chart-topping musician advancing his belief in a Flat Earth, or someone made responsible for feeding large groups of people spontaneously remarking that he “doesn’t believe in germ theory.” You know I’m the furthest thing from an unreflexive, XKCD-posting science stan, but it’s hard to read this as anything but fulfillment in every specific of what Carl Sagan had so presciently tried to warn us about in The Demon-Haunted World, way back in 1995.
If there’s anything we know, too, it’s that any such fog is very easily exploited by bad actors. when, indeed, it is not intentionally generated, and that’s part of why I think training in the practice of counterforensics is so vitally important. There’s a lot more to Erin’s argument than that, and, again, I really do recommend that you go and read the whole piece on her site. But that’s certainly one of the places I land after reading and thinking about it. There’s a couple of other things I want to say about it, as well, by way of accompaniment:
The first is that Erin’s account of cognition – “Most of the explanations that I think are in my brain are actually dotted lines in the shape of real explanations, but with very little inside them. They are ghosts of knowledge. And I won’t know what I don’t know until I fuck around and find out” – so closely mirrors what we now believe is going on with perception that it made the hairs on my arm stand up. As any number of both more and less scholarly observers have now pointed out — this is a point of view I’ve seen elaborated as recently as my reading of Iain McGilchrist’s The Matter with Things over the summer, but which I personally first encountered in Tor Nørretranders’ 1991 book The User Illusion in the late Nineties — human perception is a remarkably rasterized and low-resolution scan of the environment, especially outside the direct focus of attention, with interpretation a process significantly given over to interpolation and backfill that happens some distance downstream. It makes sense to me, then, that it should all be ghosts, all the way down. (If I had to make a profoundly inexpert guess, it’d be that this strategy of making-do-on-not-very-much evolved as an energy-conservation measure.)
I want to say, too, that forming efficacious mental models of the systems and environments we encounter is by no means straightforward or easy. Any such endeavor is perpetually vexed by the boundless complication of the world — even in its most trivial and concrete manifestations, let alone in its higher-order self-interaction as a grand system-of-systems. Developing a sense of cause and effect and consequence in, say, the tangle of utility lines running alongside a single backstreet block in Seoul is the work of days, even weeks, not minutes or hours. Extend that inquiry into “the way things work” to all the physical, informational, economic, political, institutional, cultural and social-psychological circumstances that condition the lay of those utility lines, and the way in which all of those circumstances evolve in real time, at their different characteristic rates, and you very quickly hit the limits of the individual human sensoriconceptual apparatus. In fact we may not be equipped with the means of developing anything but the most distant shadow of knowledge, delayed in our comprehension as even the pallid and partial account we are able to organize propagates back to us from a system that’s long since assumed a new state of being.
There’s also something in this line of thought that stirs up the old Donald Rumsfeld line about the “unknown unknowns,” the domain of things that we don’t know we don’t know. Part of what it feels like Erin is suggesting is that ignorance takes multiple forms, especially in this quadrant. There are places where that form closely resembles our naive understanding of it, where ignorance is constructed as a simple lack of knowledge. But we shouldn’t forget that ignorance also consists in the affirmative presence of spurious knowledge. And this may be the more dangerous of the two forms, because this is what underwrites that misplaced, Dunning-Krugerish confidence that we already understand the world adequately well, and have no need to investigate further.
And this goes to the final and maybe the biggest thought I have to offer, which is that if we are (and, by extension, everything we think of as civilization is) desperately reliant on reliable, reality-tested and therefore efficacious knowledge, the most dangerous thing one could imagine would be a machine that replaces that knowledge with a spurious, superficially plausible, costly-to-detect impostor. Erin discusses this, extensively, in a review of the literature on the psychology of news consumption, and how it relates to the subjective sense of being well-informed. I want to expand it to the way it feels to consume information from a different source.
§ One of the reasons why communities with a flawed or broken reality-testing apparatus generally cannot prosper indefinitely, whatever their short-term fortunes, is that eventually their consensual hallucinations run ashore on the rocks of the real thing. That’s the “find out” phase of FAFO, and it’s one I think MAHA, to take just one prominent example, will before long be entering, along with everyone else they’ve dragged along for the epidemiological ride.
But there’s something else in our lives which is responsible for a large and steadily increasing share of public knowledge production, and which also has no organic reality-testing ability, and that’s the large language model. This is central to what we mean when we argue that LLMs destroy knowledge: that for every verifiably accurate account of the world, LLMs produce a Borgesian library of superficially plausible alternates – sufficiently plausible, in fact, that considerable effort and energy must be invested in verifying or falsifying them. And past some horizon, it’s not inconceivable that it will take more energy and effort to falsify all the plausible-but-spurious interpolations contending with the actual explanation of some state of affairs than it would to develop that knowledge de novo in the first place.
The upshot is that there is now effectively a tax on the development and transmission of knowledge strongly anchored in the empirically verified real, compounded by the fact that the individual and collective bearers of that knowledge – whether we know them as civil engineers or curanderas – are forced to compete in a marketplace of attention and credibility with other agents, machinic or human, who paid no such tax, and who are free to advance and assert the truth value of any-notion-whatever. And indeed, as we’ve all learned this past decade or so, it will often be easier for those any-notions-whatever and their bearers to prosper, because they’ll appeal directly to what feels good in a chaotic and often terrifying world: yes, patriots are in charge; yes, this old, cheap, off-label drug will protect you from the disease running rampant; yes, investing in this currency will make you rich beyond your wildest imagining. Reality-tested knowledge, buzzkill that it so often is, struggles to compete with such promises.
So it feels to me like everything that Erin is concerned about in her piece, as distressing as it is, is twinned with another and a different kind of erosion of what we (think we) know, with which it now and henceforth becomes intertwined. The analogy of ultraprocessed food is irresistible here: what we are offered by LLMs, bite-sized and engineered for appeal and engagement as it is, clicks solidly into the docking sites we have for knowledge, but doesn’t deliver the payload we might expect. (This isn’t merely rhetoric, by the way: consider the 2025 finding that “when individuals learn about a topic from LLMs, they tend to develop shallower knowledge than when they learn through standard web search, even when the core information in the results is the same.”)
And I don’t know how one devises procedures for the production and transmission of what Erin calls “durable knowledge” under such circumstances, except by getting outside the sociotechnical frames we’re generally offered entirely. There is, of course, a decent body of contemporary thought on just this question, extending from the development of the long-term nuclear-waste warning messages (you know: “this is not a place of honor”) to Neal Stephenson’s novel Anathem. But I tend to dislike how much of this discourse resides with folks like Stephenson, or the Brand-Kelly-Eno axis: it just doesn’t feel right to me that knowledge practices meant to sustain nothing less than the long-term survival of high-complexity civilization should be devised by such a concentrated cohort, with so striking a degree of self-similarity among its members. You won’t be surprised to hear me argue, by contrast, that various Indigenous societies offer more interesting, potentially fruitful and literally time-tested methodological propositions — for example in the Tasmanian oral traditions which conserved accurate knowledge of astronomical and geological events for spans well in excess of ten thousand years.
Production of knowledge about the real, of course, isn’t nearly as straightforward as I’ve tended to suggest here. We know that reality is neither a static nor a passive object of inspection, that the act of measurement changes the thing that is measured, that the laws of the universe may well change over time, and that consciousness itself can appear to be entangled with certain objects of empirical analysis in ways we still only very poorly understand. We can imagine encodings of knowledge that are embodied, distributed and physical — that don’t require a subject, and aren’t “about” anything, let alone representational in the way our own records tend to be. Nevertheless, for the most part representation is how we currently do things. And when our representations of the real wander too far from local ground truth, the bill will and must eventually come due: as Phil Dick taught us so long ago, “reality is that which doesn’t go away when you stop believing in it.” There’s no guarantee whatsoever, of course, that any such reckoning will happen quickly enough to preserve the health of systems that have drifted into epistemic error themselves, or been exposed to that error as an endemic or field condition. But if we want to avoid that fate completely, we’ll have to devise practices and procedures that keep us from drifting too far from the real, alongside the institutions that carry them down through time. My very strong sense is that LLMs will not and cannot be a part of that assemblage, certainly not as they’re presently constituted.
§ Right now I'm reading Dune Messiah, and wondering what happened to Frank Herbert between 1965 and 1969 that had such a marked effect on his ability to imagine expansively; listening to a Christmas playlist that includes just about nothing recorded after 1965 save tracks by Low and Sinead O’Connor; and watching Pluribus and Shrinking and everything else I can on Apple TV while I have shared access to it.
That's about it for now. As always, I hope that you are well, that you have your feet planted solidly on the good Earth, and that you are taking care of those who need you, taking care of yourself, and letting others take care of you.
Merry Christmas, beloved,
ag
ldn
This is Dispatch from London No. 154 of Wednesday, 24 December 2025, offered to you under the terms of a Peer Production License. You are free to reuse this content, but must acknowledge Adam Greenfield as its author. ◊