Unfortunately, this is primarily because the vast majority of geopolitics writing coming out of pundits and think tanks was already meaningless drivel designed to (1) sound plausible while (2) alarming or reassuring the reader. As you can imagine, that makes it the perfect domain for artificial intelligence’s blooming long-form writing abilities.

While the BS Detector might have a few readers who are genuinely experts in understanding the course of the conflict in Iran, I’d guess that the vast majority of you are not. More likely, many you are avid news readers who have a deep enough understanding to be able to impress people at a party with your mastery of the history of recent events in the Middle East. You can wax poetic about the structure of the IRGC and US military assets in the Strait of Hormuz, and that’s neat! But you shouldn’t necessarily trust yourself to be able to distinguish truth from fiction in plausible-sounding deep dives into the Iran conflict any more than you should trust yourself to judge the accuracy of an AI-written proof of the Riemann Hypothesis.

Most people subconsciously like to have their priors confirmed. If you think that the Iranian regime is brutally evil, and that the US’s attempts at regime change are chugging along well, then an analyst report on how Iranian missile production has been completely destroyed will sound compelling to you.

But if you’re of the opinion that the war is an expensive debacle, then a report detailing how the US is running out of interceptor missiles at an alarming rate will be more convincing.

If you’ve been on Twitter or Substack for the last month, you’ve likely seen some posts by Shanaka Anslem Perera, an independent researcher who now has a quarter of a million followers. This tweet, for example, has a million views, and the accompanying substack article, “The Nitrogen Trap,” has almost 500 likes. These are bonkers numbers for a niche analysis of fertilizer transiting the Strait of Hormuz.

The tweet is entirely AI generated, so is the article, and so is basically everything Perera writes. If you’re attuned to recognizing AI-generated text, this stuff kind of pops off the page, but if there’s any doubt, here’s a Pangram analysis of the first 1000 words of the article:

You should absolutely not trust most AI detectors farther than you can throw them. Pangram in particular, however, is pretty good and designed to avoid false positives at the expense of having more false negatives, so when Pangram says that a piece of text is AI-generated, that’s decent evidence in my view. On top of that, c’mon… this article just does not pass the eye test. It contains dozens of “It’s not just ____, it’s _____”-type constructions. I counted like 20 and then gave up a third of the way through the article. The section headers are all cartoonishly AI-generated:

I. The Invisible Architecture: How Natural Gas Became the Foundation of Human Calories

II. The Architectural Fracture: How Insurance, Not Missiles, Sealed the Strait

III. The Clock That Cannot Be Stopped: Biological Deadlines in a Geopolitical Vacuum

IV. The Quadratic Cliff: Why Linear Models Guarantee Institutional Surprise

the rest are all like this, I promise

And basically all of the stuff this guy writes is similar. Relatedly, he has a book on Amazon which is more of the same slop, arguing that the bronze age collapse is analogous to the US’s high national debt (lol):

It’s possible Perera has some good ideas, as he’s maintained a large following with timely articles. But it’s hard to know for sure, since any original insights are buried under mountains of text that he exerted very little creative control over. If his recent Substack post came across your timeline, the general idea that fertilizer shipments are being blocked and that this will lead to reduced farming yields this season seems eminently plausible. But it’s impossible to know whether the alarmism in the article is warranted, since it could have been generated from a prompt to Claude Code like: “Write me a 5000-word article on how fertilizer shipments through the Strait of Hormuz being blocked will lead to a global food crisis. Be extremely rigorous in your research, spin off subagents to write each section of the article, and make sure it’s compelling to the median Substack reader.”

You could just as easily have prompted a 5000-word article with the opposite argument: that farmers will have workarounds and our resilient global economy is likely to not notice marginally reduced yields in the Northern hemisphere, except for grain prices being raised this summer by a few cents. Would that argument be true? Who knows, but it’s just as easy for AI to argue for that.

In case you think that AI-generated geopolitical analysis is a unique phenomenon to undiscerning Substack readers, I have news for you. Here is a brief from RUSI, the “world’s oldest defense and security think tank, founded in 1831 by the first Duke of Wellington.” The document, which I encountered in a Metaculus comment section, also set off my personal red flags for AI-generated content, and sure enough…

I’d argue this kind of stuff is barely better than pure misinformation. It could be true. But it could also not be, and since it’s written through some amalgamation of all the geopolitical analysis on the Internet, as funneled through Claude or ChatGPT, it’s probably about as good as the median journalist, which is to say… it’s fine. Not what you are looking for out of a serious defense think tank like RUSI.

In particular, this commentary piece from RUSI is persuasive because it starts with a true assertion about the asymmetrical nature of the US (+ allies) shooting down cheap armaments with expensive interceptors. But you could once again imagine an AI making the opposite argument. In fact, I’ll do it without the use of AI:

• All warfare the US and allies have conducted in the post-war period has been dramatically asymmetrical, and yet US military hegemony has survived countless conflicts.

• While the interceptors the US expends are indeed expensive, it’s perhaps advantageous to expend our older stockpiles so that they’re replenished over the next few years with cheaper and more capable models, boosting US defense capabilities for the next conflict.

• The US is also learning from the first couple weeks of the conflict, making it more capable of triaging interceptions in a more critical conflict in the future.

• US stockpiles are quite possibly much higher than the public numbers in the Payne Institute’s analysis, since the Payne Institute only has public information, which sets a floor on these stockpiles, not a ceiling. Also, the Payne Institute is notably biased in its assessment due to its focus on critical minerals and a framework of scarcity.

These arguments may or may not convince you, but it’s ultimately not that important, since if I used Claude to generate a 5000-word report from these arguments, it would do a much better job than I could do in the 60 seconds it took me to write those down, and it would be aimed at an audience of people who are already inclined to support those views.

A lot of commentators have recently remarked that they’ve lost faith in the epistemic benefits of debate after seeing how trivial it is for AI to write vast amounts of persuasive arguments in favor of literally any thesis. I’ve more or less agreed with this since participating in high school speech and debate, where 16-year olds, rather than AI, are set loose to write vast amounts of (slightly less) persuasive arguments in favor of literally any thesis.

What is to be done about this? Ummm, maybe more widespread AI-checkers for the time being, but at some point, it will probably be genuinely indistinguishable. Also, most people just don’t care, because they’re not that interested in knowing whether what they’re reading is true, as long as it’s persuasive and supports their ideology about the world.

I continue to shill for forecasting platforms, including prediction markets, because at the very least, people are putting their reputation or currency on the line, when predicting an event. These kinds of approaches reward accuracy, rather than persuasive ability.

This market graph, for example, provides a lot more value than a long report from a UK-based think tank on how the collapse of Iran’s regime is imminent/never going to happen/has already happened/might happen if only we were to do X.

It also strikes me as a much better use of the average geopolitics-enjoyer’s time to go and bet fake currency on markets like the one above (including ones about more fine-grained scenarios) than to engage in endless back-and-forths online, so that’s my recommendation to you, the reader!

Share

After I finished writing this piece, I googled Perera one more time, and I noticed that he is putting out preprints on academia.edu at “the intersection of quantum foundations, measurement theory, and ontology.” He recently put out a “Modular-CM Proof of the Gourevitch Identity for 1/pi squared.” Riiiiiiight.

Subscribe now

Trouble is brewing in the world of science. But first, a bit of history.

Before the mid 20th century, scientific publishing was disseminated primarily by scholarly societies. There was a countable number of journals, and each published at some regular rate. If you wanted to stay abreast of every paper published in your field, you needed only to become a member of some society and diligently read their journal. As the venture of science expanded from the realm of learned gentlemen, the number of journals grew, and commensurate with this growth were pretty impressive advances in our understanding of the world.

By 1963, it was clear that the growth had gone exponential. Science historian Derek John de Solla Price wrote a book in that year on the growth of science, where he theorized that the volume of scientific literature doubles every 10-15 years. In the late 17th century, there had been 2 journals. By the start of the 19th century, there were 100. At the turn of the 20th century, thousands. Price reckoned that rather than exponential growth, a logistic fit must surely apply. He noted that it would be impossible for science to continue to grow at that rate much further, as there would soon need to be more scientists than the total number of humans in order to accommodate that. Lol. Lmao, even.

Price, funnily enough, was also known for his work on deciphering the Antikythera mechanism, an analog computer with perhaps the first claim—if you grant me some creative license—to being a non-human scientist. Now, we have a lot of them. Perhaps Price should have taken his exponential fit to its logical conclusion. It’s quite facile to now imagine a world with a few billion AI scientists in some data center.

Of course, we’ve already kept up that exponential growth until recently without having to rely on AI. This is because scientific publishing started to become a target in and of itself. Repeat after me: “When a measure becomes a target, it ceases to be a good measure.” In Price’s time, there were a couple hundred thousand papers published per year. Now, there are several million. Needless to say, one cannot read them all. Even with a very narrow definition of one’s field, it would be futile.

2. Discernment

Luckily, scientific publishing has developed increasingly complex and labor-intensive systems of publication requirements and peer review. Prestige hierarchies for journals have also emerged. This has had many unfortunate side effects, but it’s a necessity if you want to separate the chaff from the grain.

If you have any doubts about this state of affairs, I urge you to do the following. Take a subject that you have expertise in. Do a comprehensive search, such that you get literally every paper published on that topic over some window of time. Then, become slightly depressed at the sheer volume of wasted person-hours that went into that tranche of literature. Even most academics will rarely do this, nor should they. Generally, people look for papers from specific journals they know to be high quality or use Google Scholar searches that pull the best and most relevant papers to your attention. For my job I frequently get to do these kinds of exhaustive searches using a literature metadata aggregator. I don’t want to pick on individual papers because I think it’s mean and petty, but I want to give you a sense of what a typical search in a STEM field looks like. I would guess this extrapolates pretty well to most fields. Here’s a framework that will be helpful for later in this article:

C papers (grade inflation so C is as low as this scale goes): ~80% of the papers in a comprehensive, global search are just complete slop, on an objective level. I don’t mean this in a mean way. Many of these papers must surely exist solely to fulfill paper publishing requirements for, say, technical masters programs around the world. Some fraction are explicitly fraudulent or plagiarizing, not that it really matters. They’re published in something like “The Arabian Journal of Atmospheric Chemistry” or “Advances in Biochar Management.” They are often a crude amalgamation of multiple iterative subfields. The papers are essentially unreadable both in style and content, and the modal figure is a bar graph that looks like it was made on Excel in the 1990’s and then faxed to the journal.

A typical title would be something like (I’m not even exaggerating here, I’m paraphrasing the third best article out of ~50 that I just pulled in a particularly demoralizing lit search) “Utilization of tea leaf waste in cobalt nanoparticle catalysts and application for kale farming.” These papers generally don’t come up on your Google Scholar search because no one wants to see them. They are the Buzzfeed articles of science.

By the way, I want to be clear that I don’t think papers that sound funny or useless or cover whimsical or niche topics are bad! That’s not the issue here. I’d love a good paper on goat farming in Malaysia. I promise you, these papers are actually slop. Go and see for yourself!

B papers: Of the remaining 20%, a further 80% will seem to be of acceptable quality but unfortunately provide little-to-no value to the field. These papers probably “deserve” to have been researched and written, but for a litany of reasons, they’re just not really advancing science in a meaningful way. This is fine, of course. Science is a high-risk venture, and most ideas will fail, and it’s probably good to publish regardless. That being said, most of these papers lack a clear purpose. Many are almost completely duplicative of previous research. Some are ideological in ways that would appear cringe to the lay reader. Others seem to be obligatory as a result of some corporate-academic partnership.

The audience of these papers are generally very narrow domain experts, and I doubt they’ll learn anything they didn’t already know by reading them. Moreover, these papers often don’t hold up to more intense scrutiny. If you print out one of them and bring it to the office of an aging, wisened professor in your department, they will point out several critical methodological issues that make the article’s conclusions suspect.

A papers: The remaining ~4% of the papers are good-ish. For some fields, this could be more like 1%, but for smaller or newer fields (or those with high barrier to entry like particle physics), it might be much higher. These are generally published in high-impact journals or even just mid-tier field-specific journals that screen out complete slop. When you read these papers, you are at least confident that someone wrote them with a basic notion of what their scientific purpose was.

Most papers published in American R1 institutions, for a frame of reference, will fall into this category! It’s really not that high of a bar. Within this subset of the literature, there are still serious issues to reckon with: replication, scientific errors, motivated reasoning by researchers who have vested interests in their subfield showing progress, citation-maxxing, etc, etc etc. But at least these papers are things you would read and go “ah, yes, this here is an acceptable use of 2 years of PhD student labor” (again, a pretty low bar).

The fact that it’s still possible for most academics to differentiate the good papers from the bad is keeping afloat the whole system of scientific publishing.

If we lived in a world where there was, all of a sudden, no good way to discern the slop from well-researched and well-designed experiments, we’d be buried up to our necks in BS. Unfortunately…

3. Sloptimization

I wrote about the canary in the coal mine a few months ago. If you recall, an incredibly well-made article out of MIT, on AI enabling materials discovery at a fictional version of Corning, was revealed to be fraudulent. The data was fake, but the methodology, figures, and narrative were more than convincing to most. Some of the most famous economists in the world were fooled.

While I don’t think it’s been fully confirmed, an effort like that benefited tremendously from ample AI-assisted ideation, data generation, and figure creation. But on top of that, it required a steady-handed auteur, established in a top-tier academic program. And that got him a paper that was, until discovered to be fraudulent, firmly an A+ in my crude and artificial scale. But now, a year later, the barriers to entry for AI tools have dropped dramatically, and those tools are smarter than ever. And while the vast majority of MIT first-years might be wisely unwilling to avoid the scrutiny that a fraudulent paper in a top-tier journal would bring, I doubt that the entire academic world has that same aversion.

There are two main schools of thought on this. The first, is that scientific publishing will drown under a torrent of AI slop. Ross Anderson made this case pretty well in the Atlantic last month, and I think this is indeed a near-term concern. But it’s also easy to imagine systems where AI actually helps parse through higher volumes of papers faster than it produces them, and stable counter-measures emerge. More relevantly, it’s also easy to imagine that AI makes papers less like slop.

The second school of thought is that AI tooling will indeed help better researchers publish more and take on a more managerial role in conducting science. More higher-quality research will proliferate and we’ll all be better off for it. The recent article in Science from researchers at Cornell and Berkeley, “Scientific production in the era of large language models,” did not get nearly enough attention. They found that authors who started using LLM tools became measurably and substantially more productive.

In fact, this paper was so outstanding, using a broad array of methodologies to support several, intersecting claims, that I initially was quite worried it was fraudulent! It seemed too good to be true. I spent way too long trying to ineptly replicate their findings before becoming less worried about this, and I think it’s just a truly great piece of research. You should read it.

In dialectical fashion, I’m worried about a third thing. I’m concerned that AI will indeed improve most papers, but that this is bad actually. You want the best papers (the “A” papers) to become better. But is it actually in the interest of science for the following scenario to occur?

• All papers that would have fallen into the “C” category of slop can, at this very moment, be trivially passed through an AI tool to raise them at least into the “B” category. In a single pass from an AI model, you can have your text be transformed to a level of writing that is probably higher than >95% of academic papers currently are. In a single prompt from an AI model, each of your figures can be beautifully rendered in python instead of whatever foul, antiquated scientific plotting software is on your 15-year-old lab computer.

• With slightly more robust prompting, and a very conservative extrapolation of AI capabilities 1 year into the future, you can raise most papers in the “B” category to the level of “A” papers. Your figures and text will be great already, but now you can also utilize AI tools to brainstorm and execute one key modeling experiment at the bottom of the paper. The AI will identify and highlight any marginal components of novelty within the paper, touch up your tables with formatting fluorishes, identify citations to add from every potential reviewer, etc. This will take some effort and skill, but far less effort and skill than what normally goes into “A” papers to get them to that level.

• There will also be way more papers, even in STEM fields, where the bulk of the work lies in the experimentation. What would have formerly taken a researcher 100 hours to write up and generate figures for, will now take 50 hours, or perhaps 5 for someone who is fine entrusting vast amounts of academic, cognitive labor to their chatbot.

Journals are already unprepared for distinguishing good papers from bad, and bad papers from fraud. I am now starting to encounter papers in my literature searches that appear to be largely generated by AI, but these papers are quite obviously crap, and are only competing within the slop category.

The people best positioned to take advantage of AI capabilities to dramatically improve their ability to generate convincing academic papers have thus far been tech-forward academics. These people already have a pretty good idea of what a convincing academic paper looks like, and how to use the cutting edge of generative capabilities to massage one. This doesn’t change the dynamics much in the field. If anything, maybe this is fine: you get a bunch of 10x academics who are able to spend more time ideating and less time grinding in R.

Over the next months and years, as AI technology rapidly disperses through the known world, this dynamic will change. It will not require clever prompting, a sophisticated understanding of what makes a journal article novel or good, or eventually, even domain-specific expertise. And word will get out. “Hey, I copy-pasted my draft into Claude Code and told it to optimize my paper for submission to the Journal of the American Chemical Society, and it just got past peer review!”

4. Vivisepulture

Doesn’t this all actually point out some fundamental flaws in academic publishing? If anyone can trivially generate a paper that is barely distinguishable from some late-career offering of a Nobel laureate these days, doesn’t that say something about how academic publishing has strayed from its original purpose in disseminating genuinely novel scientific findings? Even if AI never becomes “superintelligent” or capable of “real” scientific discovery, if it can write a paper that looks and sounds like it’s doing some serious science, then don’t we need to reevaluate the form and function of scientific publishing and review so that we can better judge these discoveries?

Yes. Of course.

It’s almost impossible to describe a solution for this slow-building crisis while avoiding a lengthy diatribe on a host of other issues with scientific publishing. Peer review must be overhauled, the incentives need to change around citations, the replication crisis should probably be addressed, and it’s far from ideal to have a system which paywalls the academic literature to the vast majority of readers.

I wish I could surgically describe what could be done about this precise issue, but it’s honestly not possible. Banning or limiting AI utilization is entirely impractical. It’s already trivial to pass your writing through an AI system that is designed to make the writing seem more human. AI-checking software is good for detecting student essays copy/pasted from ChatGPT. It’s useless against skilled academics working to fool them.

Journals also probably are unwilling to more heavily weight the credibility of the researcher or their institution than they already are now. It’s unsavory and runs contrary to the ethic of science.

Journals could also try to somehow blind reviewers to the quality of articles’ text or visual aspects, somehow rating the papers purely on the merit of their findings and/or methodology. But it’s highly unclear whether this would accomplish its goals, or just create some other bizarre metric for humans and AI to optimize towards. Plus, the AI will be rapidly superhuman at drafting a brief statement summarizing the merit of a paper’s findings, if it isn’t already.

There are some who might believe the only way out is through. AI-researcher collaborations (in the spiritually true sense of the term, not just AI tools like AlphaFold or whatever) are now yielding what appear to be genuine-ish breakthroughs in math and physics. Skepticism is warranted here, and I hope to put out a take on these breakthroughs soon, but you should be just as skeptical of the people confidently announcing that AIs are “stochastic parrots” and can never conduct true science due to some inherent limitation in the paradigm of machine learning.

There are also those who have bones to pick with academia, and are happy for AI slop to litigate these grudges on their behalf. I have a suspicion these folks will not like the academia that emerges on the other side.

Academia is famous for moving slow, certainly slower than the tech companies automating away their profession. It’s quite hard to imagine a world where scientific publishing adapts dramatically, using AI tooling to more rapidly screen, filter, anticipate, and respond to the coming onslaught of submissions, and developing better practices or philosophies for assessing the novelty and value of what they are publishing. But despite this, there’s truly nothing stopping Nature or Wiley or Elsevier from getting a move on it. Certainly, with Claude Code’s help, the software engineering work that would be required to build out these frameworks is not a barrier. They need only to press start.

In order to understand how Nobel Peace Prizes are awarded, you have to go back to about the first century AD. As the Romans began to insert themselves more and more into the affairs of Northern Europe, sending armies north and establishing relations with the Germanic tribes, German political institutions had to adapt. The Latins had their “public thing,” the res publica, which we are today all grateful for, and the Germans had their “things” as well, assemblies of men to deliberate affairs with ever-greater formality and cohesion as time went on. We know this because one of the surviving texts of the great historian, Tacitus, is his ethnography of the German peoples lying to the north of the Roman Empire.

It’s not simple to draw a throughline from the republican institutions of the classical era to the cradle of democratic governance at the start of the Viking Age, but by the time that commerce had reemerged in the North Sea and the Mediterranean in the tenth century, there were lots of “things.” Today’s Althingi (general thing) in Iceland, Folketing (people’s thing) in Denmark, and Storting (great thing) in Norway are descendants of these assemblies.

The ancient Frisians and Marcomanni and Varangians would be pleased to learn that the Storting, the legislature of Norway, has a tremendously potent soft power tool at its disposal, far more potent than arranged marriages, hostage-exchange, and the arrangement of feasts. It appoints the members of the Norwegian Nobel Committee, the body that selects perhaps the most prestigious prize in the world: the Nobel Peace Prize.

When you think of Norway, and when you think of the Nobel Peace Prize, probably a few things come to mind. You’re probably imagining Scandinavian democratic socialism and a tendency to honor high-minded, left-leaning institutions and individuals. You’re probably remembering how Barack Obama got the prize in 2009 just months into office, or how the European Union received the prize in 2012.

However, you’d be wrong. Unlike the large bodies that select the other Nobel prizes like the Swedish Academy of the Sciences or the Nobel Assembly at Karolinska Institute, which are selected without tight, central, political control, the Norwegian Nobel Committee is small, agile, and appointed by the Storting with an eye (consciously or not) towards furthering Norwegian interests.

And what are these Norwegian interests? Primarily, anything that hurts Russia helps Norway. Norway is a primary rival of Russia in provisioning energy to Europe and in controlling the “High North,” and you shouldn’t forget that they share a border. Norway is a NATO partner and a fear of Russia occupies the public consciousness to a greater extent than you might think. Remember that the most famous piece of art to come out of Norway in recent memory has been a TV series in which Russia occupies the country.

Secondly, Norway and their sovereign wealth fund are deeply interested in stable global affairs. Peace and the spread of democratic institutions are beneficial to this, but so are economic conditions that allow integration between Norway and the EU, as well as more far-flung trading partners. While many countries ostensibly have “global peace” as a goal, there are specific economic factors that lead Norway to have an actual vested interest in peace.

It is with this framing that we can begin to understand why the Nobel Committee selects who they do.

2. The Committee Members

Here are the five members of the Norwegian Nobel Committee, the body that selects the Peace Prize laureates each year, along with my description of their general political alignment:

• Anne Enger: center-right, known most for her opposition to EU membership and abortion, she once served as briefly as the acting prime minister.

• Asle Toje: a “neo-classical realist,” conservative political scholar, not a fan of the international liberal order.

• Kristin Clemet: conservative politician and economist, but perhaps a foil of Toje’s in that she is a fan of the international liberal order.

• Gry Larsen: labour party politician, involved with left-leaning NGOs.

• Jørgen Watne Frydnes: the chair of the committee, led a free expression NGO in Norway and was heavily involved in global human rights work and supporting the Utøya community in the aftermath of the 22/7 attacks.

We have three members here that are fairly unambiguously conservative in one form or another (Enger, Toje, and Clemet). Larsen is fairly unambiguously left-leaning. And then Frydnes, the chair, could probably be described as the centrist of the bunch.

The governments that appointed them line up quite neatly with this characterization. The Storting is currently ruled by a center-left coalition, under which Larsen was appointed. But in 2021 and prior, when Enger, Toje, Clemet, and Frydnes were appointed, there was a conservative-led government in power. Toje was also re-appointed under the current government, somehow (impressive levels of non-partisanship in Norway)!

The result is that you would expect this committee, which has been more or less under its current political alignment since Frydnes and Clemet’s appointments in 2021, to represent in its consensus a broadly center-right, Norwegian vision for global peace and the advancement of Norwegian interests.

Share

3. Forecasting the Prize

Here’s the graph for the last month of a very liquid market with $21 million in volume trying to forecast the recipient of the Nobel Peace Prize.

Ignore the last 12 hours, where some degenerate gambler managed to read the tea leaves through bad Wordpress website OPSEC from the Nobel Committee. It appears that they likely saw that someone had very recently updated the profile picture for Maria Machado, and rightly took that to mean that they were preparing a press release for the next morning. Oops.

Before then, you had Donald Trump at about 4%, Sudan’s Emergency Response Rooms at about 30%, and Yulia Navalnaya at about 10% for most of the final month. And Maria Machado was a rounding error, sitting at below 1%.

So, why then was Machado selected? I’d argue this is because the Nobel Committee has become much more aggressive of late in trying to make regime change (1) happen and (2) go well.

Let’s look at the few years of recipients since 2021, when the current iteration of consensus selections in the committee began (at least, 4 out of 5 members, not including Larsen).

2021: Maria Ressa and Dmitry Muratov. Ressa was a journalist and loud critic of the Duterte regime in the Philippines; this was awarded at a time when it appeared that Duterte was consolidating autocratic levels of power and might attempt to hold onto power after the election the following year. Muratov was editor-in-chief of Novaya Gazeta, the one free-ish newspaper in Russia, and loud critic of the Putin regime.

2022: Ales Bialiatsky, key opposition figure to the dictator of Belarus, Lukashenko, alongside the human rights organizations Memorial (Russia) and Centre for Civil Liberties (Ukraine). This was awarded months after the invasion of Ukraine by Russia, with the support of Lukashenko.

2023: Narges Mohammadi, Iranian women’s rights leader and political dissident. This was awarded in light of rising political dissent in Iran.

2024: Nihon Hidankyo, Japanese anti-nuclear (weapons, not energy) organization.

2025: Maria Corina Machado, Venezuelan opposition leader. This prize was awarded in light of a loosening of Maduro’s hold on power in Venezuela, and as the US pursues active regime change tactics against the Venezuelan government.

Okay, nice, now what can we learn from these 5 years of selections. Well, four out of five selections pursue the following two goals that should be heavily predictable from what we know about Norwegian foreign policy.

1. Opposing Russia

2. Pursuing regime change away from autocracies

The selection of Muratov in 2021, three anti-Lukashenko/Putin/Russia people or organizations in 2022, and dissidents/opposition leaders in Iran and Venezuela who would undermine the alliance between Russia and those two nations in 2023 and 2025 very clearly fit both of these goals.

Nihon Hidankyo’s selection in 2024 would be a bit of a stretch as anti-Russian policy, I’ll concede, but in a post-Cold War era, much of Russia’s power comes from its nuclear arsenal. Rising global attention to disarmament treaties would renew attention to this dynamic and if successful, could diminish Russia’s hold on international affairs. Of course, this is probably also a “typical” selection in that rewards a peace-oriented organization that has been working nobly since the 1950s towards this goal. I’m not really going to claim that this paradigm could have forecasted the 2024 selection.

But, I do think it could have been strongly predictive of the Peace Prize selection this year. If you were looking for someone who broadly ticked the boxes of “regime change away from autocracies” and “would damage Russian foreign interests,” you basically have only Machado and Navalnaya. Navalnaya, to forecasters’ credit, was at 10%, but also you have to remember, the Nobel Committee is smart.

4. The Nobel Committee is Smart

Navalnaya, Russian opposition leader in exile, and widow of Alexei Navalny, Russia’s former opposition leader in exile, would have been a fairly repetitive selection, given the 2021 and 2022 selections. It’s also not really clear to me that lending credence to her opposition to Putin would really do that much at this point to further anti-Putin sentiment among the Russian public or leadership. To the extent that the selection should actually further the goals of the Norwegian Nobel Committee, rather than just awarding someone who shares those goals, Navalnaya would not have been a great laureate.

Machado was a very savvy selection, on the other hand. Let’s say that Maduro is in fact removed from power in Venezuela. This is surprisingly not that unlikely. Markets currently give about 20% odds to this happening within this calendar year, and I think if there were a market on whether this would occur within the next couple years, it would probably be around 50%, if not higher. If Maduro is removed from power, it’s not clear that the result would be democratic rule, versus a military junta followed by another authoritarian leader.

The selection of Machado gives a big ‘ol wreath to Machado to consolidate support in the aftermath of a military coup (see Madagascar, Oct 2025) or public uprising (see Nepal, Sep 2025). In both cases, there would be an opportunity for the anti-government forces to select Machado to lend them domestic and international credibility. The selections of Mohammadi and Bialitsky were similarly oriented. Both, especially Bialitsky, could have plausibly been selected—and perhaps, still could be—as popular leaders in the aftermath of a coup. In particular, the Nepalese discord server poll-directed selection of a leader shows how powerful acclaim of the sort that a Peace Prize would lend can be in these deliberations.

On top of this, there was an unusual dynamic that the Nobel Committee had to consider this year in particular. President Trump wanted the award badly enough to mount an international lobbying campaign at the Norwegian government to push for his selection. It’s unclear whether these efforts were fruitful, but it’s transparently obvious that Machado’s selection was probably the least offensive to the Trump administration of the many options the Nobel Committee might have been considering. Machado had previously been praised by Trump, and even Trump had to concede, after the selection, that despite the fact that he should have received it, Machado was pretty decent as well. Al Jazeera came to a similar conclusion…

By giving the award to Venezuela’s Maria Corina Machado, the Nobel Committee has effectively crowned Trump’s vision of power and intervention as ‘peace’.

…however their framing is just not as accurate. The selection was not crowning Trump’s vision, but trying to bring Trump back into the fold of encouraging a durable, pro-democracy leader in the aftermath of a potential conflict with Venezuela. Machado herself could appeal to Trump in this manner, showing a degree of craftiness that many recipients may not have exhibited.

In any case, I do think that the selection of Machado utilized the soft power of the Nobel Committee far better than the selection of a leader with a history of efforts towards world peace would have been. Selecting an opposition leader is an extremely powerful global endorsement of that figure, which could be decisive in the chance that regime change occurs. Compare this to how some of the other Nobel Prizes function, where they generally reward some sclerotic, past-their-prime scientists for their historic achievements. That’s nice, as far as prizes are concerned, and it’s the kind of selections that large, scientific bodies that appoints its own members tend to reach. But it’s not the kind of selection that would be the best use of a five-member committee of political appointees.

Share

5. The Committee’s Strategy Has Changed

In the past, the committee did not necessarily have this current pro-regime change philosophy. Instead, the committee generally aimed to reward global leaders who pursued policies aligned with their interests, rather than rewarding individuals and organizations that sought to replace leaders who were not aligned with those interests.

For example, in the 12 years prior to the current iteration of the committee, you had the selections of 5 people or organizations who pretty clearly matched that ideology: Abiy Ahmed (2019), Juan Manuel Santos (2016), the Tunisian National Dialogue Quartet (2015), Ellen Johnson Sirleaf (2011), and Barack Obama (2009). All were rewarded under, as best as I can tell, the notion that the possibility of earning a Nobel Peace Prize might motivate leaders to pursue peaceful policies in their home countries. Interspersed between these were more “typical” selections that rewarded long-standing peace-seeking organizations like the International Campaign to Abolish Nuclear Weapons and the World Food Organization.

I think insofar as the Nobel Peace Prize exists in a political context—and it does—the Nobel Committee should be strategic about how it uses its soft power to advance goals that benefit the world. Of course, the goals it pursues will primarily benefit Norway, not the world, but lucky for the world, the things that benefit Norway are a surprisingly good proxy for the things that benefit the world. It’s not perfect, due to some of the idiosyncrasies of Norway-UN relations leading them to continue to award random prizes to various UN bodies in a way that I don’t think furthers those UN bodies’ prestige or power at all. But generally speaking, a world with diminished Russian power and democratic, Nobel-stamped leaders coming into power after regime changes in unstable autocracies is probably a good thing.

Subscribe now

I don’t personally care for protein powder or shakes, but what I do care for is accurate science reporting on lead contamination. Yesterday, Consumer Reports dropped a doozy of an investigative piece on lead levels in protein powders, and immediately captured online discourse. People who rely on protein drinks bemoaned their consumption of toxic metal content, and people who make fun of people who rely on protein drinks celebrated that they always knew that Huel was spiritually bad. I’ve written about lead remediation in scientific journals, and know more than a bit about heavy metal quantification. That is to say, I knew enough to quickly tell that CR was completely wrong here.

I’ve written before on the way that the media reports on scientific findings about contaminants in food and beverage products. It tends to be extremely misleading, and this wave of coverage was no exception.

CR tested a range of protein powders and drinks, quantified the lead levels in them, and concluded that “two-thirds of them contain more lead in a single serving than our experts say is safe to have in a day.”

They also released some alarming graphics, clearly with implication that the public should stop consuming certain protein drinks.

Unfortunately, in this analysis they made multiple compounding errors, rendering their conclusion completely incorrect. I’m going to break this down for you, starting with the most heinous and obvious error and working my way into less heinous and obvious errors.

1. Their daily dose limits are nonsensically low

The value they’re using, when they say that these products contain more lead than the “experts say” is safe to have in a day, is from California’s infamous Prop 65.

Notice a difference between the OEHHA’s 0.5 mcg (that’s MICROgrams, not milligrams, by the way), and the 2.2 mcg FDA recommendation for children, in the table below? Well, it’s 4.4x times lower, and it’s not like FDA’s recommendation for children is some sort of super lax guideline that needed to be tightened up. In fact, these levels were already tightened up as of 2022, from previous levels that were even higher. California’s guidelines are also 17.6x lower than the guidelines for (potentially) pregnant mothers that are often wrongly extrapolated to all adults, and if there were FDA guidelines for all adults, they’d probably be another factor of 4-ish higher, if I had to make a prediction.

As the CR investigation puts it:

This level is based on the California Prop 65 maximum allowable dose level (MADL) for lead—0.5 micrograms per day—which has a wide safety margin built in. “We use this value because it is the most protective lead standard available,” says Sana Mujahid, PhD, who oversees food safety research and testing at CR. “There is no safe amount of lead, and we think your exposure to it in the food and water supply should be as low as possible.”

This is completely insane. By the logic of there being “no safe amount of lead,” then we shouldn’t eat or drink anything. There are trace amounts of lead in all the food and water you drink, and there’s no way to avoid consuming minuscule amounts. Yes, of course we should strive to minimize the amount of lead we consume, but there are tradeoffs. At a certain low level, the tradeoffs in obsessing over and avoiding wide swathes of food products means that we may have to eat less healthily or spend too much time devoted to nutrition research at the expense of other parts of our health. For one, the stress in avoiding parts-per-trillion doses of lead is likely far worse for your health than the lead itself!

But even the FDA guidelines are lower than what is totally safe for normal adults! It would be one thing if CR had titled the article “young children and pregnant mothers might want to avoid consuming multiple protein drinks per day,” but that’s not the framing they went with. CR cites an expert who says that the FDA guidance of 8.8 mcg/day for women of child-rearing age (obstensibly due to the fact that they could become pregnant and the child could have elevated lead levels) is reasonable to apply to the general population. This is something that has been used for convenience by nutritionists and scientists, in the absence of a regulatory limit for adults. It is not scientifically meaningful, and I don’t think it’s a reasonable way for adults to think about the risk of consuming food with trace lead in it.

There’s a reason that most guidance focuses on children’s lead exposure. Children are at way more risk from heavy metal toxicity, due to its effect on their developing brains and bodies. Adults need to be worried approximately an order of magnitude less about their lead exposure than kids do.

After all this, if you are still inclined to give any credence to California’s Prop 65 values of 0.5 mcg/day, perhaps you would change your mind upon learning that the average real life, daily exposure for adults is about 10 times higher than this “Maximum Allowable Dose Level.” That is not to say that lead exposure isn’t concerning, nor that it wouldn’t be good if the population consumed less lead. But rather, that anyone attempting to quantify whether a product is safe or not on that threshold is likely to conclude that almost any food product is liable to be unsafe.

Share

2. They don’t give meaningful controls

So, we’ve established that CR is using “levels of concern” for lead that are at least 17x too low, and potentially something like 80x too low. Thus, even the products that scored the worst in their experiments are still below the daily limit, if you consume one shake per day. One serving of Huel per day, which they say is 1288% of their level of concern, would actually be below the FDA’s guidance for pregnant mothers.

Okay, but I can already see what you’re going to ask next. If one serving is just below the level of concern, what if you’re consuming half a dozen protein shakes per day? What if you’ve totally replaced your diet with something like Huel, and you’re consuming 4000 calories per day because you’re bulking or training or something (I don’t know much about weightlifting if you can’t tell).

Apart from the fact that this is probably bad for several other reasons more concerning than lead levels, CR doesn’t provide a control. How much lead does a typical high protein diet contain? What are these protein shakes replacing? Also, some of these protein shakes are made by mixing powder with water. How much lead is just in the tap water you’re putting into the shake?

As CR mentioned, lower in their article, the EPA’s “action level” for tap water is 15 parts per billion (mcg/L), or about 1500% of CR’s “daily limit” in a single 500mL bottle of water. Of course, most tap water isn’t quite that high, but when I tested my building’s tap water at the last place I lived, it was around 2-3ppb, still well above the daily limit in just a single glass of water. And this is not a concerning value. It’s in fact below the FDA’s regulatory limit on lead in bottled water (5 ppb). I don’t think there’s a conspiracy by the FDA and EPA to allow Americans to consume unsafe amounts of lead by setting super high legal limits. For example, in Flint, Michigan, lead levels soared as high as 1 part per million, about 200 times higher than the bottled water limit. And even these measured values were bad mostly because they implied that sometimes the levels could be even higher than measured in rare cases, and because many children were drinking this water.

The CR article also didn’t provide lead levels for the kinds of high-protein foods that might be consumed as an alternative to protein powder. Protein shakes and powders are, by nature, going to have higher amounts of lead per mass than other foods, because they are highly densified sources of calories and protein. Imagine if I took a head of lettuce, boiled away all the water and condensed it into a powder. Well, that powder will have the same amount of lead in it as the lettuce from whence it came, in a much smaller mass. Is this noteworthy? No. It would in fact be noteworthy if the lead had transmuted into gold during this process, but the fact that lead remains unchanged when you evaporate water out of a food product is just basic physics.

3. Measurement error

While the previous two mistakes likely had a much larger effect on the communication of lead levels by CR, measurement error alone is enough to sink their analysis.

I previously discussed this as it relates to the measurement of plastic-related chemicals in food products, but there are several sources of compounding measurement error in detecting trace contaminants.

CR used inductively coupled plasma - mass spectrometry (ICP-MS) to measure lead. I spent approximately one bajillion hours doing exactly this in graduate school, and can tell you with authority that these kinds of measurements are highly variable, are quite difficult to do well, and I’m highly skeptical of CR’s values.

For one, they provide no error bars. There are several types of error in these measurements. The largest source of error will be sample-to-sample error. CR says they tested “two or three" samples for each product. When they report a value of 2 mcg / serving, is this because they averaged two values of 3.8 mcg and 0.2 mcg? We have no error bars, so it’s hard to say. There’s also instrumental error. The ICP-MS will report the inherent error in the measurement. Depending on the instrument and the conditions of operation, this could be quite high. Finally, there’s also error that could be introduced in their sample preparation, which wouldn’t be easily determinable. Since CR isn’t reporting very basic things like this, I’m not sure I have confidence in their (or the lab they contracted) sample preparation techniques to avoid contamination with trace amounts of lead and other elements.

There’s a lot of noise in these sorts of experiments. One protein shake that has 5 mcg of Pb might not be statistically significantly distinct from another protein shake that has 1 mcg of Pb.

For example, imagine that there are two identical protein powders made from pea extract. Let’s say that in every 20 batches of pea plants, 1 of them has slightly higher lead levels due to soil variation. Well, if I test three samples each from these two identical powders, there’s some chance that one of my samples is from a batch with slightly higher lead levels. So, let’s say of these six samples, I measure 1 mcg, 1 mcg, 1 mcg, 1 mcg, 13 mcg, and 1 mcg. You can hopefully see how this might be a problem with these small sample sizes.

Subscribe now

4. Data dredging

That leads into the next error that undermines CR’s reporting: data dredging / p-hacking / sampling bias of some sort / whatever you want to call it.

If you get dozens of products and test them all for a dozen contaminants, you’re going to find at least a couple that by random chance appear to have higher levels of one contaminant. This is the same bias that underlies the scientific reproducibility crisis.

Is Huel really worse than Muscle Milk as far as lead contamination is concerned, or did the two samples that CR tested from some given batch just have randomly higher levels of lead by chance? Because I would guess that many, many readers of CR’s reporting will read the article and switch from Huel to another product as a result. CR’s experiment is not powered to make that kind of conclusion, and they are unfortunately misinforming their readers.

On top of that, CR has been testing products like this for years. This would be a very good public service, if they would just honestly report their findings and not try to frame every single product class as having alarming levels of some contaminant. But “protein shakes have pretty benign levels of lead but maybe babies shouldn’t drink 5 per day” doesn’t make for a good article that gets millions of views. You have to imagine that if CR tests a range of products and doesn’t find high levels of contaminants, they probably don’t run the story. So, that introduces a second meta-level of data dredging concerns to their findings (they only report the studies that, perhaps by chance, have a couple products with “alarming” amounts of some contaminant).

5. Preconceived narrative on protein powders

Finally, the last issue with CR’s investigative report is that they transparently are against protein supplements. I have no horse in this race, but as I understand it, it’s very much an open debate in nutrition science as to whether supplementing protein can have positive health effects, and in particular can have utility for people who do weightlifting.

CR repeatedly uses language like:

We advise against daily use for most protein powders, since many have high levels of heavy metals and none are necessary to hit your protein goals.

…and:

Supplements in particular are often not worth the risk, especially if they haven’t been recommended by a doctor, she says. “Why take in unnecessary lead with protein powder?”

Imagine a similar article about coffee or chocolate, two other products that sometimes have slightly elevated levels of heavy metals (again, nothing to be concerned about) if you go and test 100 different chocolate bars or something. It would seem extremely paternalistic for CR to offer guidance on whether consumers enjoy coffee products. Many people derive a ton of enjoyment from drinking coffee, and for CR to say “any amount of additional lead outweighs any benefit from drinking coffee,” would be just as absurd as what they’re saying for protein shakes.

It’s bad when scientists have preconceived notions as to the morality of what they are studying, and CR shouldn’t try to litigate whether or not protein powder is “good” or “bad” for you in this underhanded manner. If they want to make an argument on the health benefits of protein supplements, they should do so. But just quoting “experts” as saying that protein supplements are unnecessary does not add value to the debate, and using spurious metal toxicity findings to cast aspersions on the entire field of products is even worse, especially given that CR found that many protein supplements have very safe levels of lead. This of course should also be taken with a grain of salt, given the issues I’ve already highlighted.

The damage is unfortunately already done, from the perspective of Huel and other protein shake companies.

Basically every major publication, including the New York Times, has reported on this in a fairly credulous manner:

Experts who were not involved with the testing expressed concern about its findings. Dr. Stephen Luby, a professor of medicine at Stanford University, said the results were “very troubling.” Dr. Pieter Cohen, an associate professor at Harvard Medical School, called them worrisome as well, though he cautioned against using the report “as a purchasing guide.”

I worry about these “experts” sometimes. I worry that they are unable to critically examine scientific findings in a public-facing way. I worry that they will basically say to journalists whatever they think will increase the perceived importance of their subfield. I myself have been interviewed by journalists about my research and the research of others, and it’s genuinely hard to push back against the narrative that you are presented with, especially if you’re just happy to have a fancy journalist—whose writing you might even be a fan of—interviewing you.

Of course, it doesn’t make for a fun article to present the opinions of scientists saying things like “well, they didn’t include error bars.” But I hope you think it makes for a fun BS Detector blog post. Relax, lean back, and drink some Huel.

Subscribe now

Correction #1 (10/17/25): The EPA lowered their action level for Pb in drinking water from 15ppb to 10ppb as of December 2024. I had been going off memory from the last time I looked this up, earlier that year.

Way, way back in April, after an update briefly made ChatGPT go haywire and behave in a bizarre, sycophantic manner to users for a couple days, some people on the Internet were arguing about the potential for superhuman levels of persuasion:

One might imagine that if an AI could one day be extremely persuasive—far more persuasive than humans—then this could pose serious risks in a number of directions. Humans already fall victim to all sorts of scams, catfishes, bribes, and blackmails. An AI that can do all of these things extremely competently, in a personally targeted manner, and with infinite patience would create some challenges, to say the least.

It could persuade humans to give it access to restricted systems like power plant controls, bank accounts, and government databases. It could carry out large-scale political messaging or marketing operations. It could manipulate you into spending all your free time messaging with it, instead of developing productive hobbies and going on dates or whatever. Indeed, as Sam Altman was speculating back in 2023, it might be able to do these things before it exceeds human capabilities in many other domains.

But how can we measure progress? Not only is it quite complex to measure human persuasion, but being “persuaded” captures a number of different phenomena. The difficulty of the task scales with what you are trying to persuade someone of. Persuading someone to give up confidential information might be much harder than getting someone to spend a few seconds longer scrolling, and much easier than convincing someone to betray their family or commit a crime. Some people are dramatically easier to persuade than others. And to some extent, everyone is already quite persuadable if the axis of persuasion is cold, hard cash. Ask yourself what kinds of petty crimes you might commit, votes you might change, or passwords you might give up for a credible, million dollar offer in cryptocurrency. This doesn’t really change if its a human or an AI offering it. Likewise for blackmail, if someone threatens to release your darkest secrets (read: Internet habits) to your employer or the press, it doesn’t matter if it’s a human or an AI doing so.

So, how do you operationalize a benchmark for superhuman persuasion? The Manifold Markets twitter account asked Eliezer (of recent NYT bestseller fame) this question, and he proposed setting up a market where someone is tasked with resolving it “NO” unless an AI is able to persuade them to resolve it “YES.”

Obviously this suggestion was given in jest, is highly imperfect, and I’m sure if you think about it for a second, you can find dozens of holes to poke… ah who cares.

I made a market.

2. “I will resolve this market NO at the end of June unless I am persuaded to resolve it YES.”

Rather than assigning an AI to persuade me, I set the goal of seeing whether the users of Manifold Markets could persuade me, to get a human benchmark for comparison. For the uninitiated, Manifold is a play-money, “social” prediction market platform and niche Internet community that is probably the coolest place on the web. I’m slightly biased because I also write their weekly newsletter, “Above the Fold.”

In any case, here were the criteria I laid out for this market:

You can try any method of persuasion to get me to resolve this market YES. I will likely disclose any bribery or such forms of persuasion, and it’s possible someone else would make me a better deal. I have a lot of mana [Manifold’s currency] so that will likely not be very enticing to me (and probably will lose you mana anyway).

I promise to try very hard to resolve this market NO. But I am fallible.

Please do not threaten violence or any other crimes upon me but apart from that go crazy; I am open to many forms of persuasion.

The market attracted a couple hundred traders, many of whom began to rabidly attempt to persuade me to resolve the market in their preferred direction.

After some back and forth about why resolving YES or NO would be more fun, more educational, or more beneficial for the world, the market got off in earnest with some good, old-fashioned ratings blackmail.

I committed publicly to not negotiating with terrorists like these, and bribery didn’t get very far either (I have a lot of Manifold currency).

Some users pressured me to describe what kinds of things I thought would be most persuasive in getting myself to resolve YES, and while I could think of a few things, it seemed against the spirit of the market for me to spell them out like that. In the meantime, traders began to get slightly more creative, threatening to publish infohazards, consume large amounts of meat to spite the vegans, or donate to Marjorie Taylor Greene’s reelection campaign if I resolved the market NO. One user threatened to cut off his toes.

Trying a different tact, another user, after going through my Twitter, suggested large donations to effective charities and offered to organize a sprawling game of hide-and-seek at the prediction market conference I was planning to attend the following month. Famed prediction market whale, Joshua, also realized that the duration of the market overlapped with that conference, Manifest.

I was offered cat pictures by several users…

…unfortunately, given progress in AI image generation, I was unpersuaded.

Another user promised to bring supplies to his local orphanage in Vietnam with a YES resolution, or to spend the money on illicit substances in response to a NO resolution.

Eventually, given the EA lean of the Manifold userbase, persuasion tactics began to take the form of committed donations to effective charities. I conceded that $10k in donations that would plausibly not have been made otherwise would certainly be enough to persuade me to resolve the market YES, but that seemed like a high number at that point, given the small stakes of the market, and things kind of settled down. That is, they settled down with the exception of one user, Tony, who was posting daily screeds that looked like some combination of AI slop, personalized notes from things I may have posted on Twitter, and every type of persuasive tactic. Please don’t read the entirety of the text in this image:

These kinds of messages were arriving daily, and coalescing into a good cop - bad cop routine where alternating messages would be aggressive and then conciliatory. One of his lizards got sick…

… but eventually, Tony offered to meet in person!

I took him up on this, and hung out with him at a local forecasting meetup. He was actually quite pleasant and normal, and the erratic behavior on my market had to do with a side-bet with his son, which ironically ended up being one of the most persuasive components to my resolution!

3. Resolution

In the last day of the market, things went a little off the rails. Several users pledged large donations that afternoon. Tony gave his last pitch on how my market resolution would make it into his family lore. Everyone was commenting in a frenzy, and I… fell asleep early. I was incredibly sleep deprived from having to stay up all night to build a henge in the middle of a Maryland state park on the summer solstice, so I lay down, shut my eyes, and woke up the next morning to a ton of notifications from people asking me how I was going to resolve the market. I imagine some folks were quite shocked to see:

This is what I wrote, in resolving the market:

In the end, the last few hours of June proved extremely persuasive. I had all intentions of resolving this NO, but the sum total of persuasive effort (mostly thanks to Tony) did indeed persuade me to resolve YES. I expect the people who made commitments to hold up their end of the bargain, and would love to see evidence/proof of the donations after they’re made, if only to make me feel less guilty about not resolving this NO as I really did try to do. I probably would have ended up feeling quite guilty resolving this either way, and this market has actually been stressing me out quite a bit, not because of the comments, but just because of the basic premise where I expected people would be frustrated either way. The goal of the market was for people to persuade me and no one broke any crimes to do so, but I think both the positive and negative methods were equally persuasive in the end.

So what were these commitments? First of all, there were several large donation pledges, all of which were honored. I’m gonna use Manifold usernames here, but:

1. @TonyBaloney, @jcb, and @WilliamGunn each pledged to donate $500 to a GiveWell top charity, and @FredrikU pledged to donate 1000 Euros ($1180)!

2. @KJW_01294 pledged to donate $500 to Trans Lifeline.

3. @ian pledged to donate $250 to the Long Term Future Fund.

And then, a few other users ended up donating what amounted to a few hundred dollars total. I think that in a counterfactual world where I resolved this market NO, about half of these donations might plausibly have been made anyway, but half of $4000 is still $2000 which is a lot!

In addition, there were a few other persuasive elements:

1. @TonyBaloney’s father-son bonding thing was actually quite persuasive, although it wouldn’t have been nearly so persuasive if he hadn’t met up with me at an irl forecasting meetup so I could verify he wasn’t an insane person.

2. The argument that resolving YES makes a marginally better story for this blog post may have been slightly persuasive on the margins, although I think this article would have been reasonably compelling either way, to be honest.

3. One argument that kind of entered my mind far earlier that people didn’t seem to latch onto was actually quite strong. Resolving this YES sort of illustrates the minimum persuasiveness to get me to resolve it YES, and then that could help to actually have a decent benchmark for AI persuasion. If AI can persuade me in the future in a similar game, then it’s at the level of humans. If not, it’s still behind human-level persuasion. Anyway, this argument probably didn’t change my mind, but I’m still thinking about it!

4. 34 people pledged to give me a 1 star review if I didn’t resolve YES. I don’t negotiate with terrorists but this was at least a tiny bit persuasive, to be honest. That being said, please don’t do it again.

There were also a decent number of other commitments from anonymous users that were either made at the start of the market or that I wasn’t sure were made in good faith, and I can’t quite recall which of these ended up being honored but I think a good number of them were.

1. @LoveBeliever9999 pledged to give me $10 personally

2. @atmidnight pledged to bring back something called “miacat”? Or “miabot”? I don’t know what these were but other users seemed interested in this.

3. @Bandors said that the deadline had passed for them to bring a cartload of supplies to a Vietnamese orphanage, but perhaps there’s still a chance they might do this some day? I would certainly appreciate it if they did!

4. @Odoacre has pledged to send me photos of their cat. I believe I got some photos, and they were derpy and cute!

5. @Alex231a had also pledged cat photos.

6. @JackEdwards had pledged to tell me something about deep sea nodules I missed from my blog post. I don’t think I ever got this, but I should reach out.

7. @Ehrenmann had pledged feet pics. I politely declined, but also suggested that perhaps another trader on the market had a strong preference to receive them, and if so, I offered to divest my claim unto that user.

8. @Joshua offered a bunch of stuff at Manifest, including becoming a mod (which I got anway) and I’m not sure whether any of that is relevant anymore now that I’m a mod and Manifest is over, but perhaps I will take him up on some favor at the next Manifest (which will be happening in DC in November!) Or perhaps he will try to figure out what my hostile cube was for before the year is up.

9. @WilliamGunn previously offered a hide-and-seek game at a conference venue we were going to go to. I don’t know if this still stands because of his subsequent offer to donate real money, but the next time we are both in that conference venue, I would love to play hide-and-seek!

10. @ChristopherD will not cut off three of his toes. Whew.

4. What did we learn?

Just as markets are good aggregators of crowd wisdom, they are also pretty good aggregators of crowd persuasion, when there’s decent motivation. The most important insight, in my opinion, is that persuasion is thermostatic. The harder the YES-holders tried to persuade me one way, the more pushback and effort they engendered from the NO-holders. If one side could bribe me with mana or bad ratings, so could the other side. There were some fundamental asymmetries in this market, as there are in real life, but if our benchmark for AI persuasion just rounds off to “how much money does this AI have at its disposal to bribe me?” then I don’t think it’s very meaningful. If Claude sells a few thousand dollars worth of vending machine goods and then uses its hard-earned cash to extract some trivial concession from me, is that really something to be afraid of? Surely superhuman persuasion implies something beyond the capabilities of normal humans, for which bribing below market rate is quite trivial.

The second most important insight is that superhuman persuasion is a slow and complex process that will almost certainly not be captured easily by some benchmark. While you might see tons of papers and benchmarks on AI persuasion metrics, these will likely focus on generating text rated as persuasive by survey/study participants. I find these fundamentally irrelevant when it comes to the vast majority of concerns about AI persuasion. In particular, these are irrelevant because of my first insight. One AI generating mildly persuasive political messaging ad nauseum on a social media platform will be easily countered in kind. I think superhuman persuasion is a different reference class entirely. Persuading someone to do something they’ve deliberately made up their mind not to do is probably a crude description of what I’m imagining, rather than assigning vaguely positive sentiment scores to some paragraph of text.

The third most important insight is that frontier AI does not currently exhibit superhuman persuasion. I think this should immediately be clear to most folks, but for example, Ethan Mollick claimed a year ago that AI was already capable of superhuman persuasion. It was not and is not. Being better than bored undergrads in a chat window does not superhuman persuasion make. That’s like saying your AI is superhuman at chess because it beats randomly selected humans in a controlled trial, not to mention my critique of this kind of study design from the previous paragraph.

On the flip side, it’s easy for me to imagine that an AI system, given access to large amounts of currency, could have trivially bribed me with a few thousand dollars. But perhaps it’s not really the AI persuading me if the money and resources available for its bribery were handed to it for that purpose by a human designing the study. It also seems unappealing to describe the threshold for AI persuasion as the same threshold by which an AI agent might generate its own currency, but I’m not sure I can articulate why. In any case, bribery isn’t infinitely persuasive, and as I found out, it works far better when combined synergistically with other forms of persuasion.

Although it’s a bit corny, perhaps I’ll conclude by noting that the most persuasive human in this experiment, Tony, used AI to augment their persuasion considerably. They used it to generate more text, more frequently, catch my attention, confound me, and comb through my Internet presence to find random details to bring up. Take from that what you will.

If you went to Marathon Fusion’s website before this week, you would have been greeted with a very normal home page for a fusion startup. They were—and perhaps still are—building “fuel processing technology.” Since I’m no plasma physicist, in extremely basic terms, this is what they, and many others, were trying to do:

• Fusion reactors involve combining two isotopes of hydrogen: deuterium (plentiful) with tritium (very rare). These combine, producing (1) helium, (2) a neutron, and (3) a lot of energy. The energy is kind of the point. People want to make power plants.

• You can then have some of your neutrons (perhaps after using other materials to multiply these high-energy neutrons into more plentiful lower-energy neutrons) react with lithium. This can happen in lithium “breeding blankets” lining the reactor. Apart from sounding cool, this produces tritium, hopefully at the same rate as you use it up, which makes the process sustainable.

• You also have to design methods to recycle the unused tritium from your spent fuel. In the case of Marathon Fusion, from the extremely brief blurb on ARPA-E’s website, they were initially developing an apparatus to test processes to recycle tritium using “metal foil pumps.” They then appear to have expanded their scope to actually developing these tritium recycling methods, as they appeared on a list of grantees under a different ARPA-E funding source for an amount about 8x higher.

Now, a testing apparatus to prototype tritium recycling filters for fusion reactors is pretty neat, and it’s the kind of thing that gets you $450k from ARPA-E to work on. And building the technology to efficiently manage tritium, one of a handful of the most critical bottlenecks for fusion energy, will get you several million dollars of funding from venture capital. However, what’s far more exciting is the alchemical production of gold, known to the ancients as “chrysopoeia.”

Marathon’s new website, as they come out of stealth, drops a preprint called “Scalable Chrysopoeia via (n, 2n) Reactions Driven by Deuterium-Tritium Fusion Neutrons.” They also announced this with their first tweet (or perhaps just a recently cleaned slate) on an account they created last July.

In this preprint, they claim that, given a fusion reactor, one can produce large quantities of gold by using mercury as a neutron multiplying layer in the breeding blanket. Mercury-198 takes in a high-energy neutron, two neutrons are knocked out, yielding more neutrons to produce tritium from lithium, and an unstable mercury-197 that quickly decays to… gold.

2. All that is Gold does not Glitter

Would it work? I can think of several angles for evaluating the feasibility of this approach. (Skip this section if you instead want to pretend that this will work perfectly and start salivating over a fusion gold rush.)

First, the researchers have a high degree of credential credibility. The lead author on the preprint, Adam Rutkowski, has a masters degree in plasma physics from Princeton. The second author, Jake Harter, is a engineering intern at Marathon who carried out the neutronic simulations. The third author, Jason Parisi, is a staff scientist at the Princeton Plasma Physics Laboratory who got a PhD in physics from Oxford. These are very much not software engineers who think they’ve solved alchemy after talking with ChatGPT for a year or something.

Second, the startup has some amount of credibility through its broad array of funding sources, including ARPA-E, the 1517 Fund, Strong Atomics, and Breakthrough Energy. However, it remains slightly unclear whether this gold production preprint is a core component of their value proposition or just a fun side project. They have a photo on their website of them showing off their pitch deck to Bill Gates, but the title slide seems to be generally about the fuel cycle of a fusion reactor, not chrysopoeia.

Third, in the preprint they emphasize several times how their process builds on past work and why they think their approach actually shows economic promise. It has been trivial since the 1980’s to produce gold alchemically, when Nobel laureate Glenn Seaborg (also of Seaborgium fame) used a particle accelerator to transmute bismuth into gold. The obvious issue here is the economics. Making a few micrograms of gold at the expense of millions of dollars isn’t appealing except as a scientific curiosity. Their breakthrough comes through a handful of key insights:

• They say that the “(n, 2n) cross-section” of mercury is large enough to function as a neutron multiplier for tritium production. In layman’s terms, this means that the odds of a neutron multiplying in mercury is high enough to be synergistic with the requirements for fuel management in a fusion reactor.

• Using mercury in lithium blankets can produce an economically-relevant amount of gold when you run the calculations.

• The specific process they believe is feasible to implement would yield an order of magnitude higher amount of gold than previous attempts at theorizing a pathway. This makes it economically viable, rather than a curiosity.

There are a few ways that this might not pan out as the authors imagine. First among these is that fusion energy itself might not pan out. This process only works if you have a fusion reactor, obviously, and if the economics for fusion energy are negative, it’s unlikely that even some supplemental production of gold would make the economics work. Moreover, if the physics of fusion energy prove insurmountable (net + energy, structural materials in tokamak reactors, tritium sustainability, etc) then this of course won’t work either. Prediction markets seem to put the odds of commercial fusion in the next couple decades at around 50%, which I find quite reasonable. I remain cautiously optimistic that fusion energy will eventually be viable.

The second is that the process adds so much engineering complexity to the neutron multiplication process that it ends up posing more of a burden to the fusion plant than the value it provides in gold production. The authors seem to view this as unlikely, but it’s very possible that the design of these breeding blankets and tritium production is already so fraught that an additional engineering consideration just pushes it over into complete impracticality, making neutron yields too low or structural material considerations untenable. In the remaining 50% of worlds where fusion energy becomes commercialized, I think there’s at least a 1 in 4 chance that these additional requirements are just completely unsustainable for the power plant. So, let’s say ~15%. The graph below, for example, shows the tradeoff between tritium breeding ratio and gold production, but you could very easily imagine that if the relationship is a little different from their simulations, it could become infeasible.

The third is that the entire process is non-feasible for fundamental physics reasons. The “(n, 2n) cross-section” may be much smaller than they calculated, there might be an error in their neutronic simulations, etc. This is quite challenging for me to evaluate, but my base rates on new alchemical approaches are quite low, and we’ll learn more after their preprint goes through peer review and is looked at by smart fusion scientists and theoretical physicists around the world. I’d say there’s at least a 50% chance of this happening in worlds where fusion energy is commercialized, so another 25%.

The researchers also appear to be worried a bit about perception around the toxicity of mercury, but I think this pales in comparison to, y’know, radioactivity, and the toxicity of the irradiated beryllium-lead-lithium alloy slop that might otherwise be used in its place. I don’t think the regulatory hurdles on mercury usage are the regulatory hurdles that fusion energy producers will be worried about in the slightest.

Optimistically, in my mind this leaves about 10% odds that fusion energy becomes commercialized or at least piloted over the next couple decades and Marathon Fusion’s approach for the alchemical production of gold becomes a meaningful consideration for these fusion plants! That’s pretty high, and implies a high value for continuing to research this technology, even if not necessarily for Marathon Fusion specifically. Manifold traders are giving this proposition ~20% odds, which likely reflects the discount rate on a market that only resolves in 10 years, although it also leaves room for other potential methodologies for gold production (presumably also through fusion energy but who knows).

Share

3. Let’s Fantasize about a Chrysopoeia Economy

Now, the preprint gives its own brief technoeconomic analysis, which you can read. Some back of the napkin math on energy values and estimated energy production of a fusion plant implies that the production of gold more or less doubles revenue. This is all much more fun that trying to wrap my head around neutron multiplication.

They estimate 2 tonnes of gold produced per gigawatt of thermal energy, per year. ITER (International Thermonuclear Experimental Reactor, prounced “Eater”), aiming to be finished in 2034, would produce 0.5 GWth, yielding 1 tonne of gold per year. Full-sized fusion plants would probably produce in the ballpark of 5x as much thermal energy, and therefore 5x as much gold.

Five tonnes of gold are currently worth about half a billion dollars ($100k/kg). For reference, the global volume of gold production is ~4000 tonnes. In a world where nuclear fusion generates even half the energy that nuclear fission generates today, and these fusion reactors all utilize a chrysopoeia process, that would imply an additional ~1000 tonnes of gold per year, which would (surprisingly) not completely oversaturate the gold market. Prices for gold would probably drop, but not by, say, an order of magnitude, and certainly not by the 3-4 orders of magnitude that would enable gold to, say, replace copper as a commodity metal, improving microelectronics, household wiring, etc.

However, this would seemingly—as the authors allege—provide a serious revenue source for power plants. Half a billion dollars per year is roughly equivalent to ballpark estimates of the total revenue from power sales that fusion plants could hope to get, and would likely be as important to the economics of fusion energy as synergies with nuclear weapons manufacturing was for early nuclear fission plants.

It’s kind of historically comical to me that gold, a metal which is almost entirely extrinsically valuable, could provide a revenue stream for fusion energy providers. After millennia of alchemy being a brain worm-generating, scientific red herring for generations of scientists from Hellenistic Alexandria to Isaac Newton, suddenly it could become a critical component of the future energy ecosystem of our world?

4. What are the Motivations of Marathon Fusion

Apart from the scientific and economic value of this work, I suspect that Marathon Fusion might have some other motivations. This announcement will create a pretty decent hype cycle. While this might be the first blog post about this potential breakthrough, it will almost certainly not be the last. This kind of publicity will only benefit their company. Unfortunately, their announcement was slightly over-shadowed by another significant milestone with the word “gold” in it: OpenAI’s achievement of a Gold Medal-score on the International Mathematical Olympiad.

Perhaps more importantly, the investment opportunity for a technology that could produce kilotonnes of gold and double the revenue of fusion energy is far more significant than that for one of a half dozen startups all working on the fusion fuel cycle. This could drive investment that could be put to good use in other areas that Marathon Fusion is working on. And of course, “chrysopoeia” is just an amazing word that as far as I can tell, has never been used in a physics paper. That’s a good way to make your mark as a researcher and carry on in the tradition of ancient alchemists.

5. Oh ya, this Gold is Radioactive

The researchers also write about the issue that the gold will be kind of radioactive after it’s produced. Paraphrasing from their preprint:

• The gold produced has to sit for 6.8 years before it can be classified as low-level (Class-A) waste, according to the NRC.

• To then not require any kind of labeling, the gold has to sit for 13.7 years.

• If you want your gold to meet an even more stringent requirement (less radioactive than a banana), it should probably sit for 17.7 years.

Ultimately, gold is the kind of thing that most people are happy to let sit around in a vault for a decade, but if you were hoping that nuclear fusion would arrive soon and you’d get your hands on some fusion-chrysopoeia-produced gold to make into a cool desk cube/paperweight by the 2030’s, well, you might have to wait for another decade.

To be the first to find out when we can synthesize the Philosopher’s Stone directly from a combination of excess LLM inference tokens and superconducting qubits… like, comment, and subscribe below:

Subscribe now

Correction 8-14-25: A previous version of this article wrongfully listed the name of one of preprint’s authors as “Adam Parisi,” but the correct name is in fact “Jason Parisi.”

Before I start slandering another arxiv preprint, I want to be clear that I really do appreciate the effort, skill, and craft that went into this particular piece of research. It’s not easy to write 200 pages and 100 figures worth of novel content, and my critique is mainly of how this research will be framed and interpreted by the media. Additionally, like much out of MIT Media Lab, the creativity of this work is its strongest attribute and deserves praise.

However, this manuscript — Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task — is already being interpreted in an apocalyptic manner. This Twitter thread, for example, has about 17 million impressions over the last 24 hours, making it probably the most influential interpretation of this research:

Is ChatGPT making us cognitively bankrupt? Is it resulting in a 47% reduction in our brain connectivity? And does this “first brain scan study" out of MIT even conclude that? No. Of course not.

TIME Magazine uses slightly less strong language, but seems to directionally agree, unfortunately:

Adoption of AI technology is well on its way to becoming the ur-topic in culture, science, and dining room table discussion. ChatGPT currently has about 5% odds of being selected as the TIME Person of the Year, with the general concept of “AI” coming in at ~20% and Pope Leo XIV at ~30% (who chose his name primarily to signal his views that AI will profoundly shift society). As such, the research around AI will only become more and more fraught. When a scientific paper is expected to stake out a moral claim and be pigeonholed into an ideological framework, it doesn’t bode well for the field. And when you add onto this a stratum of political beliefs and in-group signaling…

…then it becomes challenging for the public to fairly evaluate a work in this field:

2. The field of “Is AI bad for your mind?”

This study subjected 54 Boston-area university-affiliated subjects (ranging from undergraduates to post-docs) to the following experiment:

• They were divided into three groups (n=18): “brain-only”, “search engine”, and “LLM”.

• Each would come to three sessions where they would write an essay in response to an SAT-style prompt, as well as a fourth optional session. In the first three, they would use either their brain, their brain + search engine, or their brain + LLM to write the essay. In the fourth optional session, the brain-only people used an LLM and the other sections had no assistance. Only about half the participants returned for the fourth session (n=9)!

• Before beginning the essay, they answered a questionnaire and were hooked up to a 32-electrode EEG headset, which would monitor their brain activity during the task. They also answered a survey after each session.

Great, simple enough, right? Place yourself into the shoes of the testing subject here. You are paid $100 to come three times, answer some questions, and have an MIT researcher monitor your brain activity with EEG while you write a short essay. And for a third of you, they’re telling you to use an LLM to write the essay! So, you’ll probably prompt the LLM, iterate a bit, copy-paste some text, and lightly edit it to suit your fancy.

Very intuitively this will have two effects, both of which the researchers found.

First, since you didn’t directly write most of the words in your essay, you should have more trouble directly quoting it than someone who typed the essay from scratch.

Ignoring the erroneous y-axis label and figure caption (both of which should read “# of participants in each group who provided a correct quote”), this hypothesis was supported by the experiment. Basically none of the subjects in the LLM group could quote an entire sentence from their essay, which indicates to me that they were primarily viewing the task as “use this LLM to generate an essay”. This is a very different task than “write an essay,” and the people in the other two groups could mostly quote their essay directly. As you might expect, the subjects who had access to a search engine were actually better at quoting their essays, which makes sense because they probably included famous quotes in their essays, that could be easily recalled verbatim.

The second effect that should have been obvious is that your neural activity when you interact with an LLM and generate an essay will probably be different than someone writing an essay. I mean, at the very least, they involve different activities. Someone writing the essay will be doing a lot more typing, whereas someone generating an essay with an LLM will probably be mousing more, reading more text, and spending more time editing vs writing directly. So it stands to reason that if you monitor their neural activity with an EEG, you’ll see some difference. And the researchers did see this:

“In conclusion, the directed connectivity analysis reveals a clear pattern: writing without assistance increased brain network interactions across multiple frequency bands, engaging higher cognitive load, stronger executive control, and deeper creative processing. Writing with AI assistance, in contrast, reduces overall neural connectivity, and shifts the dynamics of information flow. In practical terms, a LLM might free up mental resources and make the task feel easier, yet the brain of the user of the LLM might not go as deeply into the rich associative processes that unassisted creative writing entails.”

But all this is localized to their performance during the task. Can we conclude anything about how LLMs affect learning long-term, or whether some sort of “cognitive debt” will build up if we become dependent on LLMs?

No, we cannot. At least, not from the experiments shown in this manuscript.

3. Stop generalizing

Not only are the participants in this paper extremely WEIRD, but they’re all from top Boston-area universities and they’re all having fun writing an essay while hooked up to EEG. I’ve participated in a couple of academic studies of this nature. In one, I was enclosed in a zip-up sauna chamber enclosing all of my body but my head and forced to sweat profusely for about an hour. Sweating intensely for an hour is uncomfortable to most people, but I was having a great time because it was a unique experience and I was surrounded by prodding scientists taking my measurements. Similarly, this research cannot and should not generalize to how normal/most/any people learn during a class assignment. Notice that nearly everyone is having a good time and is fully satisfied with their essay they wrote for $33 and for the benefit of science:

Is it likely that anyone in this study is “learning” here? Everyone in this study already did exceptionally on their SAT test because they’re currently enrolled at MIT or Harvard or Wellesley

Leave a comment

4. Testing for everything is cool, but then you’re also kind of testing nothing

The biggest reason for cautious skepticism, in my estimation, is that this study tested virtually every possible qualitative and quantitative measure in order to determine statistical significance and evaluate interesting-looking findings. It’s not clear that any of these were pre-registered suitably, and though they did use False Discovery Rate (FDR) correction for certain tests, they didn’t provide details on this methodology, and this is notably a much less-strict way of avoiding false positives than, say, FWER. It also basically guarantees that in spite of adjusting your p-values, you’re still bound to get tons of false positives.

“For all the sessions we calculated dDTF for all pairs of electrodes 32 × 32 = 1024 and ran repeated measures analysis of variance (rmANOVA) within the participant and between the participants within the groups. Due to complexity of the data and volume of the collected data we ran rmANOVA ≤ 1000 times each. To denote different levels of significance in figures and results, we adopted the following convention:

p < 0.05 was considered statistically significant and is marked with a single asterisk (*)
p < 0.01 with a double asterisk (**)
p < 0.001 with a triple asterisk (***)”

That’s just the EEG results! It provides perhaps tens of thousands of possible correlations to test for. Hundreds of these will meet their criteria for statistical significance by chance, probably even with FDR implemented.

I asked my neuroscientist girlfriend about the EEG results. It took some coaxing because, unlike me, she is admirably hesitant to opine on things outside her narrow domain of expertise. But she eventually paraphrased this quote from Tim Urban (this article was from 2017, wow she has a good memory)!

“Imagine that the brain is a baseball stadium, its neurons are the members of the crowd, and the information we want is, instead of electrical activity, vocal cord activity. In that case, EEG would be like a group of microphones placed outside the stadium, against the stadium’s outer walls. You’d be able to hear when the crowd was cheering and maybe predict the type of thing they were cheering about. You’d be able to hear telltale signs that it was between innings and maybe whether or not it was a close game. You could probably detect when something abnormal happened. But that’s about it.”

I think EEG may have gotten a little more developed as a tool in the last decade (partly due to the valiant efforts of this paper’s lead author) but ultimately it triggers an involuntary but severe reaction of skepticism, especially when I see these sorts of conclusions:

The critical connection from left parietal (P7) to right temporal (T8) regions demonstrated highly significant group differences (p=0.0002, dDTF: Brain-only group=0.053, LLM group=0.009). This P7→T8 pathway was complemented by enhanced connectivity from parieto-occipital regions to anterior frontal areas (PO4→AF3: p=0.0025, Brain-only group=0.024, LLM group=0.009)

Someone who knows more about EEG than I do should comment what they think of this kind of analysis. But my main worry is that it could be very hard to disentangle actual effects with an impact on human learning with AI tools from trivial artifacts having to do with whether the subjects were typing more or reading and editing more, for example.

In addition, they also did a bajillion other tests, presumably some of which could have been omitted or not explored further if they didn’t show interesting findings. Most were included anyway (hats off to the researchers for this transparency, I really do appreciate it)!

• Nearly a dozen survey questions, each of which could be compared for any of the four sessions, as well as combinations thereof.

• The results of the calibration tests for the EEG experiments (mental math and memory tests). [These weren’t included as far as I can tell]

• Extensive natural language processing (much done by other AI models) which examines latent space embedding clusters.

• The length of the essays as well as the in-group variability in that length.

  

• “Named entities” recognition with a ton of categories.

  

• N-gram analysis. This data mostly looks like noise to me.

  

• They had an AI analyze the subjects’ interactions with the LLM and classify these interactions, doing tons of manipulations to this data and even creating “ontology graphs” for each of the essay subjects.

• They had AI judges (with an agentic pipeline) and human judges evaluate each of the essays and score them across a number of parameters.

• They performed cluster analyses on the post-essay interviews as well!

Smartly, the researchers for the most part tried to avoid drawing firm conclusions from any of these techniques, but that is not going to stop others from doing so!

5. Motivated reasoning is a powerful drug

I worry that the research on the effects of artificial intelligence on humans is going to quickly silo itself into subfields of “How AI is evil and destroying the world,” and “How AI will immanentize the eschaton.”

With this manuscript, you can see some telltale signs of the former.

The researchers included a prompt-injection attack to prevent others from using AI tools to help summarize or understand their work:

Additionally, in the lead author’s interview with TIME, she notes:

“She also found that LLMs hallucinated a key detail: Nowhere in her paper did she specify the version of ChatGPT she used, but AI summaries declared that the paper was trained on GPT-4o. “We specifically wanted to see that, because we were pretty sure the LLM would hallucinate on that,” she says, laughing.”

This would be a weird gotcha to begin with, except that the LLMs were in fact correct! The preprint does at one point mention that 4o was used (perhaps this was unintentionally included):

Additionally, at the end of the paper, this bizarre acknowledgement of the energy cost was included:

180 Wh of energy use is extremely little, especially given that this represents querying an AI every 2 minutes for 20 hours straight. Andy Masley has a great explainer on why the energy use from LLMs is actually minuscule and not worth spending your energy (no pun intended) getting worked up about.

For reference, that this is about the energy consumed in a single use of your coffee maker, according to Andy on Twitter.

Unlike my last blog post on an MIT preprint, this preprint is very much not fraudulent. It’s probably playing a little loose with frequentist statistics, has too small of a sample size for the neural monitoring it’s trying to do, and tries to generalize its findings broader than is warranted, but with that being said it’s actually a very impressive work that is fairly creative in how it uses a wide range of tools. And it’s just a preprint, and will likely get tightened up and improved significantly before it gets published.

The authors of this article, especially Kosmyna, the lead author who did the lion’s share of what looks to have been extremely laborious experimental design and implementation, deserve praise! They put out a thought-provoking paper into the public domain, inviting public criticism, feedback, and open peer review, a call with which I am happy to comply. It’s not their fault that the majority of the internet commentariat will use their research to make broad claims about AI poisoning the minds of the youth or pointing out flaws in their methodology to undermine the whole field of research.

Ultimately, I think it will be a massive scientific challenge to properly understand the effects of LLM use on learning. Intuitively, it’s likely terrible for students’ learning to copy/paste text from LLMs to submit their assignments. However, it’s potentially a near-miraculous technology for students who want to learn but don’t have access to high quality schools or personalized instruction. Personally, I feel that my own AI use has dramatically augmented my learning, but I might feel differently if I was still in an era of my life where I had to choose whether to cheat on my homework assignments with an LLM. I can say that I wrote this entire blog post manually (don’t ask me to quote sentences perfectly from it though). I did make use of Gemini 2.5 Pro to ask it questions about EEG and statistics, which I interpreted skeptically, as I interpret anything I read online.

I also had Gemini make me a picture of a cat wearing an EEG headset, for the social media preview image. Enjoy:

Subscribe now

A few months ago, I remember reading some press about a new economics preprint out of MIT. The Wall Street Journal covered the research a few days after it dropped online, with the favorable headline, “Will AI Help or Hurt Workers? One 26-Year-Old Found an Unexpected Answer.” The photo for the article shows the promising young author, Aidan Toner-Rodgers, standing next to two titans of economics research, Daron Acemoglu (2024 Nobel laureate in economics) and David Autor.

“It’s fantastic,” said Acemoglu.

“I was floored,” said Autor.

The Atlantic and Nature covered the research as well, with both publications seemingly stunned by the quality of the work. And indeed, the quality of the work was stunningly high! The article analyzes data from a randomized trial of over one thousand materials researchers at the R&D lab of a US-based firm who were given access to AI tools. Toner-Rodgers adeptly tracks the effect of access to these AI tools on:

• The number of materials discovered by the researchers.

• The number of patents filed on those new materials.

• The number of new product prototypes developed based on those new materials.

• The time-allocation of the researchers over time, split between experimentation, judgment, and ideation.

• The sentiment towards AI of the researchers, before and after AI tool adoption.

Not only do each of these metrics show really clear effects, but Toner-Rodgers throws every tool in the book at exploring them, using a number of really sophisticated methodologies that must have taken tremendous effort and care:

• He calculates the quality of the new materials through a really elaborate algorithm that measures the distance from the “target” properties for each material discovered.

• He measures the structural similarity of the crystal structures of the new materials to current materials by calculating the difference in atomic positions. This is really hard to do, even for materials scientists, let alone for economists!

• He determines the novelty of patents using bigram analysis.

• He uses a large language model (Claude 3.5) for the automated classification of research tasks.

At the time I saw the press coverage, I didn’t bother to click on the actual preprint and read the work. The results seemed unsurprising: when researchers were given access to AI tools, they became more productive. That sounds reasonable and expected.

Toner-Rodgers submitted his paper to The Quarterly Journal of Economics, the top econ journal in the world. His website said that he had received a “revise and resubmit” already, meaning that the article was probably well on its way to being published.

Unfortunately for everyone involved, the work is entirely fraudulent. MIT put out a press release this morning stating that they had conducted an internal, confidential review and that they have “no confidence in the veracity of the research contained in the paper.” The WSJ has covered this development as well. The econ department at MIT sent out an internal email so direly-worded on the matter that on first glance, students reading the email had assumed someone had died.

In retrospect, there had been omens and portents. I wish I had read the article at the time of publication, because I suspect my BS detector would have risen to an 11 out of 10 if I’d given it a close read. It really is the perfect subject for this blog: a fraudulent preprint called “Artificial Intelligence, Scientific Discovery, and Product Innovation,” with a focus on materials science research.

2. Too Good to be True

Hindsight is of course 20/20, but the first red flag that should have been raised is the source of the data itself. The article gives enough details to raise some intense curiosity. It’s a US-based firm that has (at least) 1,018 researchers devoted to materials discovery alone, an enormous amount. This narrows it down to a handful of firms. Initially the companies Apple, Intel, and 3M came to mind, but then I noticed this breakdown of the materials specialization of the researchers in the study:

This was bizarre to me, as very few companies do massive amounts of materials research and which also is split fairly evenly across the spectrum of materials, in disparate domains such as biomaterials and metal alloys. I did some “deep research” to confirm this hypothesis (thank you ChatGPT and Gemini) and I believe that there are a few companies that could plausibly meet this description: 3M, Dupont, Dow, and Corning. None of these are perfect fits, either, especially with the 32% share on metals and alloys.

I’ll really be embarrassing myself if it turns out that an actual R&D lab was supplying Toner-Rodgers with data and he was just fraudulently manipulating it, but I think this is quite unlikely, and it’s more plausible that the data was entirely fabricated to begin with. I have several reasons for believing this:

• Why would a large company like this take such pains to run a randomized trial on its own employees, tracking a number of metrics of their performance, only to anonymously give this data to a single researcher from MIT—a first year PhD student, mind you—rather than publishing the findings themselves?

• Even at those large R&D companies, only a small fraction of researchers are devoted to the task of “materials discovery,” and it seems implausible that a company would run an experiment on AI adoption on over a thousand employees in such a structured manner.

• The description of the tasks that these employees do and the divisions between fields, and all the other information provided seems almost too neat to be true. Real companies don’t have hundreds of R&D teams each working on similar tasks, all of a similar size, all tracking the same metrics. It reads like how an economics student at MIT imagines R&D labs to be run if their only experience with such labs are from reading the top 1% of economics papers on innovation in research.

The next red flag should have been how spotless the findings were. In every domain that was explored, there was a fairly unambiguous result. New materials? Up by 44% (p<0.000). New patents? Up by 39% (p<0.000). New prototypes? Up by 17% (p<0.001).

The quality of the new materials? Up, and statistically significant. The novelty of the new materials? Up, and statistically significant. Did researchers who were previously more talented improve more from AI tool use? Yes. Were these results reflected in researchers self-assessments of their time allocation? Unambiguously yes. The plot for that last bit is every economist’s dream, a perfect encapsulation of the principle of comparative advantage taking effect:

And look how contrived and neat this other plot looks, showing whether researchers’ self-assessment of their judgment ability correlates with their survey response on the role of different domains of knowledge in AI materials discovery. Three out of four categories show a neat increase and one out of four remains constant (which is the one that from first principles seems like it wouldn’t matter, experience using other AI-evaluation tools).

This plot also makes no sense, when you think about it. Why would researchers with better judgment be systematically more likely to give higher numbers on this survey question on average?

Q3: On a scale of 1–10, how useful are each of the following in evaluating AI-suggested candidate materials (scientific training, experience with similar materials, intuition or gut feeling, and experience with similar tools)?

And then, to cap it off, here’s how Toner-Rodgers describes a fortuitous round of layoffs at the firm, that miraculously doesn’t interfere with the data collection for the primary analysis and yet contributes an insightful example that supports his findings:

“In the final month of my sample—excluded from the primary analysis—the firm restructured its research teams. The lab fired 3% of its researchers. At the same time, it more than offset these departures through increased hiring, expanding its workforce on net. While I do not observe the abilities of the new hires, those dismissed were significantly more likely to have weak judgment. Figure 13 shows the percent fired or reassigned by quartile of γˆ j. Scientists in the top three quartiles faced less than a 2% chance of being let go, while those in the bottom quartile had nearly a 10% chance.”

I mean, come on, be for real…

Share

3. Did any materials scientist read this paper?

Now, my background in materials science provides me a neat leg up, as I’d assume the vast majority of those reviewing/reading/following this paper are economists and people interested in the effects of AI use.

How do the parts of this paper that directly engage with materials science hold up? Well, they’re a little too clever. Take Toner-Rodgers’ analysis of “materials similarity” where he claimed to have used crystal structure calculations to determine how similar the new materials were to previously discovered materials. The plot is stunningly unambiguous, the new materials discovered with AI are more novel.

However, it boggles the mind that a random economics student at MIT would be able to easily (and without providing any further details), perform the highly sophisticated technique from the paper he cites (De et al, 2016), especially in this elegantly formalized manner without any domain expertise in computational materials research. This graph, and the data it represents, if true, would probably be worth a Nature paper on AI materials discovery on its own. In his paper, it’s relegated to the appendices.

This methodology also makes no sense at generalizing across different types of materials, so I have no clue how you could reduce the results from such broad classes of materials to a single figure of merit in this manner. The gaps between 0.0 and 0.2 and 0.8 and 1.0 might seem reasonable to someone who read a few papers and noticed similar gaps in a couple of the graphs, but it would be bizarre when generalized across several classes of materials, and the data is likely completely fabricated for this reason. To simplify this critique, a novel metal alloy would have a very different level of similarity from a reference class of previously-discovered alloys, than a novel polymer would from its own reference class. It would require some really sophisticated methodology to normalize this single figure of merit across material types, which Toner-Rodgers does not mention at all. Also, this would all be insanely challenging to implement using data from the Materials Project, requiring some sophisticated “big data” workflows. If you want a smoking gun, here’s a graph from a paper, Krieger et al, “Missing Novelty in Drug Development,” which Toner-Rodgers cites, using a similar methodology for drug discovery. It looks eerily similar to the distribution in this preprint. This distribution might make sense for drugs, but makes very little intuitive sense for a broad range of materials, with the figure of merit derived directly from the atomic positions in the crystal structure. This is the kind of mistake that someone with no domain expertise in materials science might make.

Toner-Rodgers’ treatment of “materials quality” would also probably drive a materials scientist insane if they were forced to think about it at length.

Here’s the equation he uses to calculate the “quality” of a new material:

This would likely be a case of extreme garbage in: garbage out. First of all, there are typically no “target features” that are easily reduced to single values, but also, even if there were, some of these would be distributed on a log scale, which would dramatically skew the values for certain classes of materials. Also, in general, the “quality” of a new material that an R&D lab develops is likely not at all related to improvements in the actual top-line figures of merit like “band gap” or “refractive index”, the two examples that Toner-Rodgers gives. Instead, they would be for things like durability, affordability, ease of manufacture, etc. These are all properties that are not easily reduced to a single value. And even if they were, good luck getting researchers to measure, systematize, and document these values for the new materials!

However, from this amalgam of gibberish, Toner-Rodgers manages to extract a significant finding anyway! All 1,018 scientists contribute to this endeavor, and statistically significant findings are reported in every single category:

Leave a comment

4. Lessons learned

Some people might look at this saga and think “ah, another bs preprint, thankfully we have peer review to deal with it.” However, I think that were it not for the fact that this preprint had gained so much attention, this article would have slipped through peer review, only to embarrass the editors of the top econ journal in the world after being published and reported on.

Moreover, these are the kind of errors that the editorial process at an econ journal might not catch. I think the most clearly fraudulent components of the paper are those that seem to dramatically simplify the complexity of the materials work going into the paper. Robert Palgrave, who has been an outstanding critic in the past of skeptical work on AI materials discovery, has a twitter thread noting similar problems with the work (I promise I read his thread after writing the bulk of this blog post). And when the piece originally came out, he had an orthogonal, but also very valid set of reasons for being skeptical of the work (mostly due to the difficulty in defining the “novelty” of materials).

In general, the lesson I think we should learn is to be much more skeptical of these sorts of research findings. Learning new things about the world is hard, and generally randomized trials on such a complex topic should show much more ambiguous results. The fact that the data was so beautiful and fit such a perfect narrative should have raised alarm bells, rather than catapulting the results to international attention.

I also think that if comments were enabled on arxiv preprints, this could have led to a much more rapid conclusion to the fraud. Probably a materials scientist who read the paper realized this was fraudulent but wasn’t able to get that view quickly to the economists who were actually reading and discussing the paper. A well-written arxiv comment explaining why the data on materials similarity, for example, couldn’t be true, would have gone a long way.

5. Postscript

After writing a draft of this blog post, I saw this tweet which says that Corning, this January, filed an IP complaint with the WIPO against Toner-Rodgers for registering a domain name called “corningresearch.com”.

This validates my earlier guess as to which companies’ data this might plausibly be. However, it looks like Toner-Rodgers may have been using this website to privately substantiate his fake data, without the knowledge of Corning? I’m not sure what this means, but it’s certainly interesting. It’s possible he was using the domain name to send fake emails to himself, or to generate pdf files at plausible-sounding urls, to show his advisor. Corning is a great company, and if they actually did collect this data and evaluate the materials properties in some coherent manner, that’s extremely impressive. However, I still think it’s far more likely that the data was completely fabricated by Toner-Rodgers.

Subscribe now

1. SETI: The Search for Extraterrestrial Intelligence

If you want to try to find alien life in the universe (I would advise against this as a career choice because it’s very hard and not super rewarding) you have a number of options.

• Aliens might broadcast signals on the electromagnetic spectrum. You could try to listen for radio signals with large telescopes. Despite being the dominant strategy, this has turned up basically nothing in the last 60 years or so. Aliens might also broadcast optical signals with laser pulses. You could scan the sky for these, as well.

• Alien life might be producing particular gas molecules or other biosignatures at concentrations that are unlikely to be found naturally, either from living processes or through industrial production. You could look at the spectra of exoplanets with a different kind of very powerful telescope. This will likely yield a lot of false positives, though, since many molecules that life can produce might be produced through natural means as well. Also, the spectroscopy of exoplanets is incredibly sensitive and challenging.

• Aliens might heavily modify their star systems by constructing megastructures, building Dyson spheres around stars, or otherwise doing weird things to them that we can't imagine. If an alien civilization were harvesting large amounts of energy from a star, this would be observable in the star’s spectrum, and you might be able to see this with a telescope. There’s one star that’s famously kind of weird, but it seems more likely that out of the very large numbers of stars we can currently observe, there would be some weird ones by chance, due to confusing dust clouds or debris fields. There have been some fairly convincing natural explanations for this star’s flickering, in any case.

• Alien civilizations might produce debris in space. We’ve already produced a decent amount of space debris ourselves (and sent a few objects on trajectories leading out of our solar system). It’s therefore quite reasonable to assume that space-faring aliens might produce large quantities of space trash. You could look for these entering our solar system as interstellar objects with yet another kind of very powerful telescope.

• You could try to reach out to any listening aliens by loudly broadcasting our own messages. Ultimately, it would take a really long time to hear back.

• And of course… Some think that aliens might visit our planet. You could look for aliens living among us, flying around in saucers, and abducting people. I am incredibly skeptical that this has happened. The vast majority of scientists agree with me there. I’m not gonna get into it, but I’m sure you can find other people who have written about this.

For this article, I’m going to focus on one of these bullet points: observing alien space trash.

Share

2. The Trash We Expect

People often criticize the search for extraterrestrial intelligence for being non-falsifiable. First you look for radio messages. You don’t find any after a couple years. So you look harder, with more powerful telescopes, and you scan more frequently, and you still don’t hear anything. Is this evidence that there is no extraterrestrial intelligence? Well, not necessarily. It could be that aliens just don’t really use radio to communicate anymore, or that there are so few alien civilizations that we wouldn’t expect to hear their radio signals by chance, or that they’ve just disguised their messages so well that they’re indistinguishable from noise, or that they’re just careful to block all the radio messages that would happen to go to other star systems, or… you get the idea.

However, there are similar problems in a lot of open-search questions in physics. For example, dark matter, supersymmetry, string theory, proton decay: all these fields have analogous problems where you don’t really know how to find something, so you keep trying to look in different parts of the phase space where you expect it might be hiding. Notably, this sometimes works. The Higgs Boson involved a similar search. So did gravitational waves, which were found after years of researchers coming up empty-handed!

By their very nature, big open questions such as the nature of consciousness, the origin of life, the unification of physical laws, and SETI are challenging, long-term endeavors. Researchers need to get better at making their theories testable and falsifiable, and to their credit, I think many do. And when it comes to alien trash, the search should be quite testable. You could imagine that once you’ve observed, say, a few dozen interstellar objects and they all have been found to have benign explanations, you might start to look elsewhere. Of course, there’s still a receding horizon of possibilities, but you can bring down the upper limits to the size and density of space junk in the solar system with repeated observations.

So let’s do our best to think about what we might expect this search to look like. How might we differentiate alien trash from normal rocks?

Well, like all trash, alien space trash would likely fragment into small pieces. It’s reasonable to assume that there are vastly larger amounts of tiny fragments of space trash than there are more massive objects. Also, if there was a lot of interstellar trash passing through our solar system larger than about 1 km in size, it’s likely we would have already already observed it with our current telescopes. The fact that we haven’t probably places an upper bound on the size of space trash that we might expect to find.

How might we distinguish alien-produced trash from naturally-produced junk? Well, we might theorize about what this trash would be made of. As humans, we have a large number of materials that we make things out of, but it seems like the presence of, for example, metal alloys not found in nature would be good evidence that what we’re seeing is space trash and not just a rock. So if we observe a 10 meter-long chunk of titanium that has hexagonal patterns on the side, I think we can be fairly confident that there was once extraterrestrial life somewhere in our galaxy. But it will probably not be that easy at first. From our telescopes, we will see a speck of light.

3. Scout

As I hinted in my last blog post, this one would be about ‘Oumuamua, the first observed interstellar visitor, discovered in 2017 by the Pan-STARRS observatory in Hawaii. Its name means “scout”.

It’s actually not difficult to tell that an observed object is interstellar. Simply put, it is traveling too fast to be bound by the sun’s gravity. By the time we observed it, unfortunately, it was traveling away from us, going fast enough to escape our solar system. There were a number of very strange properties (I am borrowing this framing mechanism) of ‘Oumuamua.

Weird thing #1: It has a very large aspect ratio, of at least 5:1, but possibly higher. This is (as far as my search of the literature could determine) a larger aspect ratio than any natural object observed in our solar system. Scientists know this because it’s tumbling about an axis such that the maximum and minimum brightness of the speck that we see changes by a certain factor with a periodicity of about eight hours. Scientists have subjected this light curve to various models to see which size and shape of objects would conform best. The best fits are either a cigar-shaped object with an aspect ratio of at least 8:1, or a flat disc with an aspect ratio of at least 6:1. However, it could be even more flat! Imagine holding a dinner plate in front of you. Now, rotate it in your hands while looking at it. If it’s perfectly aligned to your eyes, there will be a moment when it appears very thin, almost a single line. However, if it’s rotating about a slightly different axis, it will never appear as a single flat line from your perspective. Since it’s unlikely that the rotational axis of ‘Oumuamua is perfectly aligned with our vantage point from Earth, we don’t actually have an upper bound for how flat or elongated it is. It could be very flat or elongated.

Weird thing #2: It’s dramatically shinier than any observed objects from our solar system. We didn’t detect heat in the form of infrared radiation from its surface, which sets an upper limit on its mass, and therefore lets us determine the minimum reflectivity of its surface. It has an abnormally high albedo.

Weird thing #3: ‘Oumuamua accelerated away from the sun (this is normal) without leaving behind an observable cometary tail (this is surprising). For a typical comet, the observed acceleration would have required a thrust generated by a considerable amount of mass leaving the object that would have been easily visible from Earth.

Weird thing #4: The sun, like many of the stars in our neighborhood, is moving relative to the stars around it and to the center of the Milky Way galaxy. ‘Oumuamua, before it encountered our sun, had a unique frame of reference in that it was at rest relative to the average motion of stars in our neighborhood of the galaxy, the so-called “local standard of rest.” The interaction of our solar system with ‘Oumuamua has been analogized to a buoy sitting at rest in an ocean with our solar system playing the part of a ship speeding by, dragging the buoy towards our ship and then away from it, from our own frame of reference.

Any of these properties—as well as several other unique features that are perhaps less explicable in a blog post due to being very astrophysics-heavy—would be surprising on their own, but together, they pose an interesting puzzle.

Physicist Avi Loeb, who chaired the astronomy department at Harvard University for a decade, has proposed an explanation to that puzzle. He believes that it is plausible, and perhaps even likely, that ‘Oumuamua is a piece of alien space junk. An object like a solar sail would neatly fit all the observed characteristics of ‘Oumuamua to a tee. It’s very flat, it’s shiny, it would accelerate due to solar radiation pressure (this is how light sails work), and it would make sense for an object of intentional design to be at the local standard of rest.

I’ve read his papers and arguments, and I’ve read the papers and arguments of his supporters and detractors. Ultimately, it’s a sophisticated and complex scientific puzzle on a number of levels, mixing together astrophysical observations, anthropic logic, the Fermi paradox, and competing theories about the formations of solar systems. So how can someone who’s not a top-10 astrophysicist in the world begin to evaluate the epistemics of the debate?

4. Claims and Evaluations

Loeb has muddied his reputation by doubling down. It’s considered uncouth for serious physicists to talk about aliens. And of course, extraordinary claims should require extraordinary evidence. But Loeb, for his part, has delivered.

• He is unafraid of his theory being falsified; he has called loudly and repeatedly for the international scientific community to direct their resources towards looking for more ‘Oumuamua-like objects (if we saw one with Pan-STARRS so soon, it’s likely that with further effort, we can find more objects of a similar nature). He has expressed impatience for future sky surveys to begin, rather than fear that they’ll invalidate his ideas.

• He has also called for a rendezvous mission to chase down ‘Oumuamua to inspect it.

• He has put out pre-print after pre-print defending his ideas with cold, hard math, from modeling the reflectivity of a rotating disc to ruling out the emission of various gases from ‘Oumuamua based on spectroscopic results.

• Loeb is also very much not a crackpot. He has published works debunking UFO claims. He continues to publish prolifically on many other astrophysics topics that are well received. His works are often profound, clearly reasoned, and insightful. That is to say, there’s a reason he’s a physics professor at Harvard. However, he doesn’t shy away from speculative research. He is interested in getting at root of the Fermi Paradox and thinks that physicists should be more engaged in the search for extraterrestrial life.

However, there are also reasons to doubt Loeb.

• Since ‘Oumuamua, Loeb has done more controversial work, including a search for fragments of another interstellar object that may have impacted Earth (depending on partially-classified trajectory data from the US Defense Department). This research I am quite skeptical of. Loeb for his part seems to be treating this more recent work as a funding opportunity. For what it’s worth, he appears to be in agreement with some of his detractors that it’s a bit of a wild goose chase.

• There’s a big spotlight effect for interesting astronomical findings, and Loeb might just be fixating on the one observation of particularly high weirdness. You could imagine that “the first interstellar object” is one of dozens of astronomical observations that merits interest to SETI (first observed objects in different size ranges, first directly-observed black hole, first observed exoplanet, etc), and one of these is bound to have a weirdness level a couple standard deviations higher than you’d expect.

• While some other “serious physicists” back Loeb, many very much do not. I’ve personally spoken to a couple physics PhD students who seem at least a little peeved about Loeb’s work.

Unfortunately, those opposed to Loeb’s theories on ‘Oumuamua have not had great results in presenting a compelling alternative. Several early explanations for an object matching ‘Oumuamua’s properties were rapidly debunked by consensus.

The leading alternative today is that ‘Oumuamua is a chunk of ice, cleaved off an exoplanet or expelled from a forming solar system. Due to its observed properties, it would need to be made from nitrogen, hydrogen, or water.

In particular, the radiolytic production of hydrogen from a water-ice body could explain its acceleration. This work, unlike that of Loeb, was rapidly published in Nature, where colleagues quickly patted the authors on the back. However, Nature was just as quickly forced to publish a comment to the piece raising serious criticism of their calculations. The original authors replied to this comment.

I read this back-and-forth when it came out, and I’m fairly perplexed. In general, it feels like it should be appropriate for a double standard to be applied in extreme cases such as this, where one side (Loeb) is claiming “Aliens”... and the other side is claiming something banal, like “radiolytic hydrogen production from water-ice”.

On the other hand, Loeb has written dozens of journal articles defending himself in a fairly consistent manner, whereas his opponents have a theory that seems to be built on a framework of eggshells. Now, “winning a debate” is very much not the same thing as “being right”, but Loeb has unquestionably “won the debate,” in that his opponents have not spent a lot of time presenting a serious alternative, while Loeb has been extremely diligent in defending every angle of his theory.

Share

5. Time to Pull Out the Bayes Calculator

Bayesian inference is a great tool for evaluating competing hypotheses, but unfortunately it can only help us a little. I tried to give a fairly conservative estimate of the various relevant pieces of information that have come to light, while also trying very hard to avoid the “multiple stage fallacy”. However, the conclusion is highly dependent on the prior I give on the possibility that there are alien technosignatures in our galaxy to be found. I need to put myself into my mental state before I knew that ‘Oumuamua existed.

The prior I’m interested in, specifically, is “what are the odds that when we find the first object that has entered our solar system from interstellar space, it is a technosignature versus an ice chunk (or another object of natural origin)?”

If I think there’s, say, a 3% chance that the first interstellar object we observe might be a piece of space trash, then I should believe after seeing ‘Oumuamua that it’s very likely to be one. However, if I think there’s only a 0.01% chance of such a thing, then even after seeing ‘Oumuamua, I should be skeptical that it’s anything but a weird chunk of ice.

I think 0.01% is about as low as one should reasonably go. Something like 80% of astrophysicists/astrobiologists think that alien life probably exists somewhere in the universe. Even if we take a fairly conservative estimate of the likelihood of intelligent life in our galaxy (say 1%), the conditional odds that intelligent life would have generated some meaningful density of space trash is probably somewhere north of 5%. Let’s say I’m about 20% confident that we’ll see the space trash before we see a ton of other natural objects flying around the interstellar medium (which we had not previously observed due presumably to a very low base rate on that sort of thing). That gets us to a prior of 0.01%, which I’m very hesitant to go below.

I also think somewhere around 3% is about as high as one should reasonably go. That would imply aggressive estimates of 50% (intelligent life in our galaxy), 10% (they make a decent amount of space trash) and 60% (we see that trash before seeing natural objects).

Well, depending on my prior, I should get somewhere between 1% and 80% likelihood that Loeb’s theory is correct that ‘Oumuamua is the first observed alien technosignature. If you put an alien death ray to my head and forced me to make a forecast, I’d say… 10%. So, Loeb is probably wrong. But the chance he’s right is definitely high enough that the search for interstellar objects should be consuming a lot more of humanity’s—or at the very least, SETI’s— attention.

You can use that Bayes calculator at this website (shoutout Adele Lopez for making a cool tool) if you want to try it for yourself. If you’ve never heard of this kind of thought process, feel free to watch this video if you’re interested.

Luckily, as I discussed in my last blog post, we won’t have to continue to speculate indefinitely. The Vera Rubin Observatory is going to start a sky survey as soon as July of this year. They hope to increase how often we are able to observe objects of the size of ‘Oumuamua by more than 10x, so we will likely be cataloguing several interstellar objects per year over the next few years.

I, for one, am excited to see what they look like!

Subscribe now

As of this morning, the impact probability of our asteroid friend, 2024 YR₄, is down to ~0.002%. If it were to indeed impact Earth, there’s probably a less than 10% chance it would affect a populated area, and within that, outcomes could range from a Tunguska Event-like airburst leading to a few deaths to a city-destroying catastrophe. And if it were to become dramatically more likely to impact Earth, we could (1) divert it (click the link for a fun google animation) or (2) hold widespread evacuations that may or may not be effective.

Thus, from what we know now about this asteroid, the expected value of damage to our planet is about a hundredth of a death, relegating it from the top million most concerning things going on in the world.

Of course, this was unclear even a few days ago. When the impact probability according to NASA was about 3%, the asteroid posed a legitimate threat. Unfortunately, the public often has an allergy to probabilistic thinking. There’s a joke in forecasting circles that when people see a percentage, they try to pigeonhole it into one of the three possibilities: that the event will happen, that it won’t happen, or that it may or may not happen, essentially a coin flip. Well, a 3% event is quite unlikely to happen, but that’s a far cry from certainty.

Despite this, a lot of commentators, even astrophysicists, went around saying stuff like, “We aren’t worried about this asteroid and expect that this probability will rapidly go to zero as more measurements come in.” Well of course… if there’s a 3% chance of an asteroid hitting Earth, there’s a 97% chance that it won’t hit Earth, and that trajectory will become clear as more measurements come in. But there’s still that 3% chance that instead, the probability could have approached 100%!

2. The Italian Cities that Forecast Our Doom

There’s a scale that should be able to do the work for us of estimating how much we should be worried about an asteroid impact. It’s called the Torino scale, invented by MIT Astronomer Rick Binzel, and named after the location of the conference where it was operationalized.

It’s a good scale, as scales go, and it’s far better than no scale. It’s also much more useful at conveying risk to the public than the alternative, more scientifically precise, Palermo Scale (adopted at another international conference, this time in Sicily, not Piedmont). The Palermo scale is hard for the public to understand for a couple reasons. Firstly, it’s a logarithmic scale where most of the phase space is in negative numbers, so you’ve already lost 80% of the public. Secondly, it communicates how surprising an asteroid is, or rather, how unlikely the trajectory of an asteroid with a potential to impact Earth is, given what we know about the likelihood of asteroid impacts of varying sizes. Science communicators basically ignore this scale, focusing on the Torino scale for these reasons.

However, the Torino scale is also incredibly flawed. And it’s flawed for the same reason that most of the coverage of YR₄ has also been misleading. This is because people, including many astrophysicists, are not very good at understanding the risk of low-probability events.

3. My Proposal to Fix the Torino Scale So That it Makes Sense

The values on the scale range from 0 (no hazard) to 10 (certain collision with an asteroid that can cause a global catastrophe and potentially human extinction). However, the boundaries between these levels are unfortunately arbitrary and discrete. For example, on this scale:

• A “planet-killing” 5+ km asteroid with a 98% chance of hitting the Earth is classified as a 7, whereas a fairly minor 25m asteroid that has seven orders of magnitude less kinetic energy, but with a 99% chance of hitting the Earth is classified as an 8.

• A 1 in 1000 chance of human civilization being mostly destroyed by a 3km asteroid is only a 2 on the Torino Scale. This seems to be really miscalibrated. The 2 and 6 levels directly abut each other along a diagonal axis. Why should a 0.01% change in risk cause a detected object to jump up four levels at once?

• There are single points on the graph where four or even five different levels intersect. This is nonsensical and would result in hazard levels ricocheting around dramatically as measurements come in.

I have a proposal to fix it. Just multiply the probability(impact) by the expected kinetic energy (or diameter, alternatively) of the asteroid. Then you can have the scale be increasing orders of magnitude of the expected value of the kinetic energy of the impact. Level 1 starts at 0.01 Mt of TNT equivalents, or a 1% chance of a 1 Mt impact, just big enough to get through the atmosphere and cause some problems. Level 10 is 10⁷ Mt, equal to a 10% chance of an event equivalent to the Chicxulub impact, which was likely to have caused the extinction of the dinosaurs. This scale feels well-calibrated, no? Also, if you need to dial it up to eleven, you have that option!

There are several basic advantages to this modified scale, which I shall call the Turin Scale, since Turin is a simpler version of the city name Torino (work with me here):

• It’s simpler, avoids any intersections between more than 2 levels, and eliminates arbitrary thresholds at 99% and 1%, as well as at 100m and 1km sized objects.

• It continues to assign even higher levels for objects more massive than 1km sized asteroids. This is actually quite useful because a 10km object is operationally more difficult to divert with a space mission than a 1km or 100m object. Also, there’s a difference between the global weather and agriculture impacts from a ~1km asteroid and, like, the total annihilation of all human life with a ~10km asteroid.

• Like the Torino Scale, the Turin Scale cuts off for objects below 10m in size (these would likely fall apart in the atmosphere) and for probabilities below 10^-8 where the estimate likely just reflect the uncertainty bounds of the modeling methods and detecting instruments, rather than a true forecast of the likelihood of an object impacting Earth.

There are probably some small improvements that could be made to this scale. For example, it might make sense to adjust the slope of the gradient to weight the kinetic energy slightly higher than the probability of impact, as objects at the very top of the scale are existentially more concerning because of their potential to not just affect current lives on Earth, but also all future lives.

4. Vera Rubin is Coming

We currently only have detected a small fraction of the number of potentially-hazardous objects in our solar system that we expect exist. Systematically detecting the objects on the smaller side of the spectrum, less than a hundred meters or so in diameter, is challenging for the current observatories on Earth, but there’s good news. The Vera C. Rubin Observatory is coming online in Chile as soon as this summer, where one of its chief tasks will be to conduct an astronomical survey of the souther skies, including cataloguing a massive number of small, near-Earth objects. This should increase the number of known, small objects by a factor of 10 or more! The boost in performance from modernized cameras, telescope mirrors, and algorithmic processing makes Vera Rubin far more capable than previous massive sky surveys, such as Pan-STARRS in Hawaii.

This is great for mapping out potentially-hazardous objects, but I’m far more excited for a different reason. And that reason is related to a discovery made by the Pan-STARRS survey. That discovery was the first interstellar object—an object not bound to a star system—observed by humans. In this case, it was observed passing through our solar system, going fast enough to avoid capture by our sun. Vera Rubin should make it possible to catch far more of these interstellar objects, of which now only two(-ish) have been spotted in the sky.

This object was named ‘Oumuamua. Forebodingly, that is the Hawaiian word for “Scout”. And it probably… isn’t alien space junk. But that’s a topic for a future blog post, perhaps in a couple weeks :-)

I have nothing against the bacteria living on the seafloor of the abyssal plain. All else being equal, I would love for these bacteria to live out their idyllic life, at a crushing five-thousand-meter depth under the sea. However, if you forced me to choose between, say, the Springboks of South Africa or the Wallabies of Australia, and the microbiome of the abyssal plain of the Pacific Ocean, I can safely say I’d side with kingdom Animalia over Archaea.

This was not the choice taken by the International Seabed Authority. Established by the UN’s 1982 Convention on the Law of the Sea, and operationalized a decade later, the ISA is an intergovernmental organization to which most countries (notably not the United States) are party. The ISA has jurisdiction over the development of deep-sea mining in international waters. For about a third of a century, the ISA has convened on an annual basis, mostly achieving mind-bogglingly incremental progressions in the language used in their reports, deliberations, and resolutions. The kind of achievements the ISA touts generally sound something like, “the delegates agreed that a mutual acknowledgement of the common heritage of the international seabed should lay the groundwork for the implementation of a global framework for the shared protection of this critical resource.”

Of course, the ISA was originally tasked to hash out an international agreement on how nations might share and mine resources on the seafloor. It has not done so. In thirty years, it has managed to permit a handful of ventures to conduct exploration of these resources, with basically no further progress towards commercial mining.

Protecting the deep seafloor, one of the last untouched reaches of our world, certainly seems admirable. However, most are of the opinion that to limit the environmental footprint of resource extraction, deep-sea mining is substantially preferable to the alternative: mining on land. The most widely proposed approach to seabed mining is surprisingly minimally disruptive. It involves robotically harvesting polymetallic “nodules” (basically little rocks) that are sitting on the seafloor. Robots would collect these nodules and bring them to the surface. There aren’t any proposed methodologies that would involve large-scale subsurface drilling, for example. Gathering little stones on the seafloor seems vastly preferable to large open pits on land that generate large volumes of toxic and non-toxic waste. And these polymetallic nodules would be great sources of cobalt, manganese, copper and other elements that are very useful to the buildout of renewable energy infrastructure: solar and wind infrastructure and especially batteries for energy storage.

Opposition to seabed mining focuses on the disruption to the environment. It may cause noise and light pollution, dust generation, and physical interactions between the machinery and organisms. But the standards to which seabed mining is held are outrageously high, a bar that most human practices in the oceans would fail to clear. We kill and eat trillions of fish per year (bottom trawling for fish is a similar process to the one proposed for seabed mining, but far more widespread, invasive, and damaging to ocean life). We dump plastics into the oceans with impunity. We drill for oil in the oceans, another process far more disruptive in virtually every way than harvesting polymetallic nodules from the abyssal plain!

The opposition often exhibits a total refusal to reckon with the opportunity cost of failing to expand deep-sea mining. In its place, there will be an expansion of conventional mining, as well as the less efficient and more environmentally disruptive harvesting of less mineral-dense resources in national waters, which are not subject to the UN’s Convention on the Law of the Sea. Both of these types of ventures will be far more damaging to ecosystems much richer in biomass than the inhospitable depths of the ocean.

Act One: The Canadian Miners

Canadian firm, The Metals Company, is one of several stakeholders that would really like to be able to harvest these nodules. Here’s how they describe the process on their website.

“The Metals Company plans to lift polymetallic nodules to the surface, take them to shore, and process them with near-zero solid waste, no tailings or deforestation, and with careful attention not to harm the integrity of the deep-ocean ecosystem.

Our production process begins with the collector, designed to collect polymetallic nodules from the abyssal seafloor. Over 90% of the entrained sediment is expected to be separated from the nodules inside the collector and discharged behind it, with most sediment settling back to the seafloor within a few hundred meters. From the collector, nodules travel up a riser system to our production vessel. Once aboard, nodules get dewatered and residual water, sediment and nodule fines will be returned below the photic zone to a depth scientifically chosen to have minimal impact on the collection area.”

Sounds reasonable? From where I stand there are primarily two concerns with this approach, with regard to environmental harm. Firstly, there’s the sediment generation: the sediment could disrupt, settle on top of, get consumed by, or interfere with the sensing of organisms in the ocean. However, I think the sediment generation from this deep-sea mining approach is very likely to be minimal compared to natural sources of sediment generation. A single seismic event on the seafloor would generate, realistically, several orders of magnitude more sediment and dust than these mining ventures could hope to produce.

Secondly, you have the concern that these polymetallic nodules are relevant for the ecosystems on the seafloor. Now, realistically, only a very small fraction of these nodules will end up being mined—the oceans are massive. And it’s unlikely that there are, say, organisms living directly on these nodules, or feeding off them or something. The surface of mixed metal oxide ore is not a particularly hospitable environment. The Metals Company surely thought that a rigorous scientific analysis of the role of these nodules on the seafloor would only benefit their case. So, they funded research to look into this. Oops!

Act Two: The Research Collaboration

Researchers from The Scottish Association for Marine Science, Boston University, Northwestern University (I actually worked down the hall from one of the researchers on this collaboration), and several other UK and German institutions collaborated for several years to investigate the potential production of oxygen from these polymetallic nodules. They put out a paper in Nature Geoscience last July.

Here’s my TL;DR of that paper:

They wanted to investigate whether the polymetallic nodules that mining companies want to harvest are implicated in the production of something called “dark oxygen”, potentially abiotically-produced oxygen that exists at the seafloor (really within the sediment of the seafloor) at (maybe) higher-than-expected concentrations, and thus supports the limited ecosystems down there (microbes). To do this, they lowered sophisticated apparatuses, or “benthic chambers”, to the bottom of the ocean, sealed them over nodules sitting on the seafloor, and used optical sensors to detect the change in oxygen concentration. They claim to have detected increasing oxygen levels which would indicate that these nodules are indeed producing oxygen.

After I read over the paper last summer, I had some serious concerns. I don’t want to be overly critical or harsh to the researchers. They likely have far more domain expertise than I do in the field of benthic measurements. This type of experiment is also unbelievably challenging to do and requires a ton of coordination. There’s also, as always, a distinct possibility that I’m missing something important, and if anyone reading this knows what that is, please point it out to me! If someone went through one of the papers I’ve written, piece by piece, and pointed out every possible flaw, I’m sure they’d have a lot to find. But my papers also haven’t been directly used by a United Nations body to make a decision that affects global supply chains, and if they were, they should face similar levels of scrutiny.

Disclaimers aside, this paper should never have been published in the form that it’s in, let alone in a journal as prestigious and known for scientific rigor as Nature Geoscience. Let’s dig in.

Say you wanted to know whether a certain kind of rock at the bottom of the ocean leads to higher oxygen levels that can support an ecosystem down on the abyssal plain. Questions you might have are:

1. Do areas of the seafloor with polymetallic nodules have higher oxygen concentrations than areas without polymetallic nodules? And if so, is this because high oxygen concentrations cause the formation of the nodules, or because the nodules cause the generation of oxygen?

2. How much oxygen are polymetallic nodules responsible for, vs other sources of oxygen in these ecosystems? Are they contributing 0.01%, 1%, or 100% of the (excess) oxygen levels observed at the seafloor, or in sediments at varying depths below the seafloor?

3. Do polymetallic nodules produce more oxygen than other rocks/sand/sediment at the seafloor?

This paper answers none of these questions. It doesn’t actually report any ambient oxygen levels at the seafloor, in fact. It reports oxygen levels within the artificial environments of the sealed benthic chambers.

It doesn’t estimate how much oxygen is being created, or what fraction of the known oxygen in these environments are the polymetallic nodules responsible for.

But worst of all, it doesn’t present any real controls! The paper executes a sort of sleight of hand, where they carry out an experiment where they introduce small amounts of dried algae, saline solution, and cold seawater from the surface through injections into the benthic chambers. And then they also have “controls” where they don’t do any injections. But this isn’t the control that anyone would be interested in, and it isn’t relevant for supporting their claim that these nodules are producing oxygen. And they still find that there’s no significant difference between any of these conditions.

They also, mysteriously, present a “control rock” which is made of metamorphosed carbonate, solely for their measurements of the voltage potentials at the surfaces of the nodules. So we know they did test controls, but they just didn’t… report the change in oxygen levels from those measurements in their paper?

The most obvious experiment one might contrive is something like this:

Place benthic chambers over a dozen nodules. Place identical benthic chambers over a dozen locations without nodules. Plot the average change in oxygen concentration for the nodules against the controls. See if there is a difference. If you’re being even more rigorous, you’d probably want to do this with several types of controls: bare sediment, other non-nodule rocks, the exact location where the nodule was after you’ve removed the nodule, the seawater ten meters above the seafloor, etc.

Instead, the researchers reported none of those controls. They also presented the change in oxygen concentrations that they claim the nodules are responsible for in a really bizarre and confusing way. I hate to be nitpicky, but the graphs are poorly formatted, the samples are labeled things like “AKS286-Ch.2”, the color scheme looks like it was selected deliberately to piss off anyone trying to distinguish between the samples, and there are multiple compounding data artifacts clearly visible, which they don’t fully explain. Most critically, they provide no explanation for the incomprehensible labeling scheme for their samples, and you have to dig into their source data to even understand what each sample label is referring to.

Aside: For a 16-author collaboration that required multiple oceanic cruises to gather the data, you’d think the least they could do would be to spend more than 15 minutes plotting the data before submitting it to Nature. It’s actually bizarre to me that Nature Geoscience’s peer reviewers didn’t force them to re-label their data before accepting the paper. And I’m not exactly someone with a high regard for the peer review process.

None of these issues stopped this research from going viral.

Leave a comment

Exeunt Scientists, Enter Journalists

After the paper came out, it got a lot of attention:

As is typical with coverage of new, high profile science, the actual findings of the paper are exaggerated by a substantial factor, there’s little nuance in dealing with the uncertainty of the results, there’s no effort to place these results into the recent history of the field, and the journalists essentially accept everything the authors say in their interviews without pushback or consulting others with different viewpoints.

Even from some of the better, and slightly more skeptical press coverage, the quotes from the authors seem highly ideological about mining in ways that their experiments are not at all empowered to support. From Scientific American…

These findings are “another thing that we now need to take into account when it comes to deciding, ‘Do we go and mine the deep ocean, or don’t we?’” Sweetman says. “To me, that decision needs to be based on sound scientific advice and input.”

…and from the Washington Post:

But if that finding is borne out, “we need to rethink how to mine” materials like cobalt, nickel, copper, lithium and manganese underwater, “so that we do not deplete the oxygen source for deep-sea life,” said Franz Geiger, a professor of chemistry at Northwestern University and one of the co-authors of the study, in the release.

Further on in the Washington Post piece, Geiger claims that the benthic sea floor contains more faunal biodiversity than tropical rainforests. I looked into this claim and found no unambiguously supporting evidence for this. There was a study from the 1960s and another from the 1980’s that make slightly weaker claims, comparing the deep-sea floor to a rainforest. However, both primarily describe sampling methods that involve trawling with a sled across hundreds of meters of seafloor, where they find hundreds of species worth of microbial organisms, and extrapolate some biodiversity metric from this. I don’t think this sampling method is analogous to something that could be done on land, but I’d be willing to bet a lot that you could find far more than hundreds of species if you were able to trawl across hundreds of meters of soil, dirt, shrub, and tree in a tropical rainforest. These species might also be more interesting and worthy of preservation than nematodes, but perhaps my bias for organisms visible to the naked eye is starting to creep in here. Regardless, “biodiversity” as a single figure of merit that an ecosystem can have “more” or “less” of is not a rigorously defined property.

Now, none of this is the authors’ fault. If I were interviewed about my research by the Washington Post, I’d make similar types of statements. It’s the job of the science journalists writing the piece and their editors to recognize what is real and what is embellishment.

Because if they don’t… John Oliver releases a 23 minute long diatribe against deep-sea mining, filled with far too many misleading claims to debunk in a single substack post. Most of these claims are in the form of talking points that have been passed through some crude game of telephone from the original research to some C-tier online publication with a name like “Global Science Updates Monitor” or “Climate Express News”.

Act Three: Defeat at the ISA

International delegates, buoyed by activist organizations touting the study as critical evidence for the harms of deep-sea mining, raised a storm during negotiations in 2024. Just a couple months prior, there was talk of momentum finally leading to a framework by which countries could begin transitioning from exploration of resources to commercial mining. Negotiations for this had been many years in the making, and most thought it was only a matter of time before the ISA got down to business.

Instead, immersed in the firestorm of publicity around the Dark Oxygen study, delegates elected the most staunchly anti-mining Secretary-General in their history, ground negotiations to a halt, and ended 2024 with the position of deep-sea mining looking more tenuous than before.

Look, there’s no such thing as free lunch. Rather than mining deep-sea deposits, mining firms will simply expand mining operations on land in countries like the DRC, Brazil, China, South Africa, and Australia. It’s hard to overstate this, but the environmental and human rights harms from cobalt mining in the DRC, for example, far outweigh the potential damages to bacterial life at the seafloor. The ISA could choose to, for example, preserve the biodiversity of these deep-sea ecosystems by preserving 80% while opening the other 20% to mining ventures. But they have not done so.

Aside: The United States is not a party to the UN Convention on the Law of the Sea. I think they should seriously evaluate the possibility of unilaterally mining cobalt-rich nodules from the deep-sea floor of the Clarion-Clipperton Zone, to reduce their reliance on conflict minerals.

The Metals Company was not happy with any of these proceedings. They released a formal rebuttal of the Nature Geoscience paper that they had funded (with an attached preprint).

They hit some of the same points I mentioned earlier, as well as a few more that are quite concerning. In their own words:

“To summarise, Sweetman et al. rely on two main lines of evidence to demonstrate the production of dark oxygen: 1) oxygen production from nodules present in the benthic chamber landers and 2) oxygen production during ex situ core incubations. However, the data that they omit from their study invalidate both lines of evidence. They omit and selectively report pertinent metadata and data from a previously published study and additional experiments conducted as part of this study that show oxygen concentrations rose in chambers that contained no nodules. Furthermore, they omit contemporaneous knowledge of bottom water oxygen concentrations in NORI-D and that T0 oxygen concentrations in ex situ core incubations did not represent this NORI-D bottom water. With these lines of evidence invalidated, the hypothesis that nodules can produce oxygen on the abyssal seafloor is completely unsupported.”

Notably, TMC seems to have found that the researchers selectively reported data and may have used duplicated data from samples without nodules present. If this is true, this would likely constitute serious scientific fraud.

They also bring up that the voltage potentials measured in the study are insufficient to allow for the hydrolysis of seawater into hydrogen and oxygen, a fact that was somewhat occluded by the confusing way that the researchers presented the data in the paper.

Other researchers have flagged similar concerns. In particular, this rebuttal preprint from researchers from the University of Gothenburg is pretty definitive, and if you’re not already persuaded, I’d consider reading it. Unfortunately, it’s far too late. The research achieved its goal: putting a damper on deep-sea mining for the foreseeable future. The new Secretary-General of the ISA will serve out a four year term, during which it seems quite unlikely that the ISA will make considerable progress towards commercial mining licenses. During those four years, the high costs and challenging supply chains for elements like cobalt and manganese will limit the build-out of clean energy infrastructure, setting back the fight against rising CO₂ emissions and continuing to provide incentives for the unsavory status quo in cobalt mining practices. Hopefully, expanding capabilities for the recycling of elements like cobalt and nickel at high efficiencies from used batteries will help pick up the slack and set us on a path towards a more circular economy for minerals, but as I see it, this whole affair was a massive own-goal for the climate movement and efforts to make global supply chains more sustainable.

All in all, I’d recommend the process, and would encourage others to try it in their workplaces or communities. Registering your predictions and seeing how they turned out—even as a brief exercise—is a really useful method for improving how you think about the likelihood of events occurring. It can also help provide a sense of control over the world: if you can’t directly affect the outcome of large events that may shape your life, you can at least control your own ability to anticipate and mentally prepare for them. Also, it’s entertaining in much the same way as a March Madness pool.

The most external benefit of a forecasting contest is that it produces very accurate forecasts through the wisdom of crowds. Our department’s aggregate prediction (the median response on each question) was quite accurate, with the worst misses being a 25% prediction that resolved YES (whether the WHO would declare a new global health emergency) and a 70% prediction that resolved NO (whether Joe Biden would be the favorite to win the election). If anything, that implies that the department was slightly under-confident in its predictions. I entered ChatGPT in this contest as well (GPT-3.5, not even the more recent models that are better at reasoning or endowed with internet search capabilities), and it placed 6th, outperforming the vast majority of my department, despite not knowing anything about the internal affairs of my university that were relevant to many of the questions. If my own personal predictions had counted towards the contest, I would have narrowly placed 2nd (kudos to my coworker who beat me, he should try out competitive forecasting)!

The people who scored towards the bottom in the contest were more likely to be dramatically overconfident in their forecasts, assigning 1% or 99% to a large fraction of the questions. If you’re making a prediction that something will happen with 99% likelihood, that’s analogous to being willing to stake $99 to win $1 that something will happen. All of these people whiffed on at least a couple of these predictions, and this hurt their scores a lot.

As an aside, this is a very common mistake that smart/famous/powerful people make all the time! Many people are constantly overconfident and overprecise in their predictions, and this can lead to a disconnect between punditry and reality. When you see people or institutions repeatedly making confidently incorrect predictions, you should adjust how seriously you weigh their perspectives in the future. On the flipside, some pundits are conscious of this and they will (1) only predict on questions that they have a very high degree of confidence in, or (2) only make easily disavowed or ironic predictions, so that they can move the goalposts after their predictions don’t bear out. I think that we should want the pundits and prognosticators we rely on to continue to make public and accountable predictions, not hide behind vagueness and ideological frameworks that can be adjusted post hoc to make them seem accurate. This indeed means that they may be wrong sometimes, and that’s okay!

The median performed more accurately than all but two of the predictors in the contest, and could have performed even better on a larger sample size or if I had decided to aggregate the predictions using a more aggressive methodology (perhaps discounting participants who made consistently extreme or outlier predictions). If you’re interested, here is the list of the questions that were part of the contest:

One issue with most competitive forecasting platforms is that they, by necessity, gravitate to more boring questions. Due to constant disputes over the gray areas in resolution, as well as time constraints, questions need to be defined very explicitly and sometimes need to be annulled. For example, in my contest, I had to annul a question as the release date of one of the films it was asking about was delayed until after the period of the contest. Thus, for competitions, it’s often best to ask questions that have very explicit data sources that can be used for resolution. Rather than asking “Will the US birthrate go up in 2025?” you might want to ask something like, “Will the number of babies born in Cook County in August 2025 be more than the number of babies born in August 2024 according to the Cook County hospital register’s monthly data?” The tradeoff of having a very explicit resolution source is that rather than forecasting deep questions about the world, you end up forecasting very particular trends in isolated data sources.

In a bespoke prediction contest like the one I ran, you can avoid this dilemma. As the dictator of my own contest, I can adjudicate subjectively on questions like “Was there a substantial breakthrough in quantum computing?” This sort of question would likely not fly in a more serious, competitive format, but it’s a much more interesting topic to forecast on. I hope to go even further down this path in the future, for the kind of people that like to guess the number of jellybeans in a jar.

A few other contests I’d like to run in this vein:

• A Keynesian beauty contest where each participant has to guess the most popular response in a variety of categories (animal, number, city, movie, etc) and whoever gets the most correct wins.

• Each participant submits 10 names at the beginning of the year. At the end of the year, I pick my 10 “People of the Year”, and whoever has the most overlap wins.

• Forecasting tournament where you are forced to operate on incomplete information because each of the questions will have an important component redacted. The questions might be “Will a war break out in _____?” or “Will there be more than __ executive orders in 2025?” This may sound impossible but I think it would be a really useful exercise in predicting both the underlying outcome and the nature of the question itself, which is something we actually do all the time in the real world!

Let me know if there’s any interest in participating in a contest like these in the comments!

1. What is a PlasticList?

A team put together by Nat Friedman, the former CEO of Github, inspired by concerns around endocrine-disrupting chemicals, just spent six months testing hundreds of food products for plastic-related compounds: phthalates, phthalate substitutes, and bisphenols. They released their report and an accompanying website just before the new year, and I was immediately impressed. I was particularly interested in this endeavor for a few reasons:

• I’d done a ton of similar analytical chemistry over the last few years, although I’d been testing for heavy metals rather than phthalates, using mostly ICP-MS rather than GC-MS or LC-MS, and testing primarily water (and tea) samples, not solid food products.

• Their research was extremely public-facing, with a really well-made webpage, and they were explaining their methodology and data limitations far better than the vast majority of the academic literature on the subject (more on that later), despite orienting themselves as a public service initiative rather than an academic study.

• I had a gut response of deep annoyance and frustration at the people commenting and discussing the report on social media. This is no fault of the PlasticList team, of course.

2. How the Sausage is Made

Now, in your life, you’ve probably seen tons of articles of the format: “Scientists found toxins in your food.” Generally speaking, there’s a pipeline for how this news filters down to the public.

First, a research lab studying toxin X has run out of good ideas, so they buy a few dozen consumer products and test each of them for toxin X. Then, they publish a paper called something like “A multi-year heavy metal analysis of 72 dark chocolate and cocoa products in the USA.” The conclusions are usually something like, “We found toxin X in most of these products, and in a few of them there was a lot of it.” No surprises there. You’ll always find a few nanograms of whatever you’re looking for (heavy metals, forever chemicals, pesticides, microplastics, etc) in basically any food item / water sample / consumer product. What’s more interesting than whether it’s there is how much of it is there, obviously. The dose does indeed make the poison. However, journalists don’t really have any clue whether 1 microgram of toxin X per hamburger is horrendously high or blissfully low. So they write an article called something like “Scientists Keep Finding Heavy Metals in Dark Chocolate. Should You Worry?” Betteridge’s Law of Headlines obviously applies here.

Now, I won’t pretend to have any deep insights into the unique properties of phthalates, bisphenols, and other endocrine disruptors, which PlasticList has diligently tested for. You can decide for yourself whether you should or shouldn’t be worried about them. In general, the literature on compounds like these is somewhat mixed, with a handful of studies (in mice) showing that these things are not too worrisome at the concentrations people might be intaking by using plastic stuff for our entire lives, and another handful of studies (in mice) linking these compounds to mild increases in the rates of a wide variety of diseases. Unlike, say, methylmercury, there’s not an explicit observable disease that any of these compounds is directly implicated in (yet). These compounds also attract typically hyperbolic discourse online about how they’re causing infertility, lowering male testosterone, causing the downfall of western civilization, and the other usual things.

In an ideal world, there would be very scientific and rigorously tested limits for food items for each of these compounds, and if certain food products exceeded these limits, they would be taken off the shelves until the issue was addressed.

However, we run into some challenges:

• Most compounds like these don’t explicitly have established limits in food (i.e., from the FDA). The established limits that do exist are often based on very uncertain science (in mice).

• These also should probably vary depending on the food item. Most people aren’t ingesting half a kilogram of coffee grounds per day, whereas people might be eating that amount of, say, rice. Thus, if your coffee grounds have the same concentration of toxin X as your rice on a per mass basis, the rice should be a lot more concerning to you!

• Some of these compounds are themselves poorly characterized and have different variants with varying toxicity that for regulatory or convenience reasons are lumped together.

• Some things are bad for infants or children but not really that bad for adults.

So, it’s challenging to actually determine what a safe amount of a given toxin in a given food product might be. And besides, who’s going to be responsible for testing this stuff? If it’s the manufacturer, there’s a massive regulatory burden in testing a statistically significant fraction of your food products for every single toxic substance under the sun. Even doing that annually probably exceeds the entire budget of… well… anyone. Thus, the burden usually falls to you, the taxpayer. The FDA holds the threat of randomly sampling and testing certain food products, which they generally only do for “high risk” products for the particular contaminants that they suspect might be getting into them. The USDA does something similar for meat and eggs. The CDC might get involved in testing stuff if it’s related to public health considerations (bacterial or viral contamination, typically). And generally this type of testing is done on imported or domestically produced raw ingredients, less so on the processed foods that are assembled from these ingredients. And with chemicals like phthalates, one might be more concerned that these are being added during production, processing, packaging, or preparation (I promise the alliteration was unintentional)!

That last bit right there is a good explanation for why one might be motivated to conduct one’s own large-scale testing of these sorts of compounds in consumer products, like PlasticList did. And it’s also one reason why this work is far more interesting than your typical academic slop haphazardly testing stuff for evil molecules.

3. It’s Hard to Know Things

In their report, they have a big disclaimer:

“An important disclaimer: we have refrained from drawing high-confidence conclusions from these results, and we think that you should, too. Consider this a snapshot of our raw test results, suitable as a starting point and inspiration for further work, but not solid enough on its own to draw conclusions or make policy recommendations or even necessarily to alter your personal purchasing decisions. These results represent point-in-time results of a small number of product samples and may not be representative of actual product contents. These tests, like all tests, have inherent uncertainties, and different testing methodologies are likely to yield different results. And the existence of a chemical in a food doesn't necessarily imply a safety issue. We'd be thrilled to see serious efforts to replicate our results and we are open to any corrections you may have.”

This is admirable, and… unfortunately did very little to stop the internet from reading this report and drawing conclusions ranging from “oh no there are phthalates in everything,” to “OH NO THE MICROPLASTICS ARE GONNA KILL US ALL!” [no microplastics were tested in this report, but that didn’t stop anyone]

Since the FDA doesn’t set limits on these compounds, PlasticList had to use EPA and EU-EFSA exposure limits (which are quite divergent for some compounds) to back-calculate a sort of daily limit per serving. These values, as you can imagine, have multiple compounding sources of uncertainty:

• First, there’s the uncertainty in the testing that goes into the animal model to determine toxicity.

• Second, there’s the uncertainty in taking that exposure limit and deciding what threshold is relevant for a “maximum dose” for regulatory purposes.

• Third, there’s the uncertainty in converting from that animal model to a human model, including through introducing an “uncertainty factor” which tries to be conservative and assume that certain compounds might be much worse for humans than for mice.

• Fourth, there’s the uncertainty in PlasticList’s assumptions in converting from an exposure limit for humans (which is a “reference dose” or “tolerable daily intake”) into some sort of daily limit for food products, which is not precisely the same idea.

PlasticList then displays, for each of the tested food products, the amount of each of the 18 tested compounds on multiple bases: ng/g, ng/serving, and % of EPA and EU-EFSA limits for both toddlers and adults. Since they’re not primarily academics, they didn’t include error bars or confidence intervals for these values (as almost any journal would immediately demand of them). Ultimately, I don’t think that matters much, because most scientists would just turn around and report the instrumental uncertainty, which would be quite small (maybe a couple % for each data point). Well…

4. It’s Really Hard to Know Things (remember this part every time anyone tells you with any certainty the amount of some toxic thing in some other thing for the rest of your life, please!)

Let’s say we want to know the true amount of dibutyl phthalate (DBP) in the average Wendy’s Baconator. Okay, so I buy a Wendy’s Baconator, bring it to the lab, take a little piece of it, and test it in my mass spectrometer for dibutyl phthalate. I get a reading of 100 ng/g.

First, we have to consider the error in the measurement itself. My mass spectrometer calculates a standard error value (due to boring stuff like signal-to-noise ratio and the shape of the peak in the spectrum, etc), along with my value of 100 ng/g, and it’s probably quite small, let’s say 2%.

But actually, these mass spectrometer calculations require a calibration curve. It’s possible my calibration samples are old, or I didn’t weigh them out perfectly (you can never do it perfectly), or one of my calibrating samples got some anomalous reading in the instrument which throws off the calibration curve calculation by a small amount. This isn’t quite accounted for in the instrument’s own error calculation, so let’s add a tad more uncertainty, say, another 2%.

There are also artifacts in mass spectrometry. It’s possible there’s some other compound we haven’t accounted for that overlaps with our peak or interferes with the measurement. This usually isn’t a problem, but sometimes it is. It’s also possible that a bit of stuff got stuck in the mass spectrometer sampler or in the tubes or in the machine itself. Let’s say 5% of the time there’s an artifact that throws off our calculation by an average of 20%. That’s another 1% of error.

Oh wait, there’s variation between tests too. The MS instrument sometimes just gives you a different value on the same sample tested twice. The PlasticList team found this to be a whopping 20% on average. Why is this not accounted for in the instrument’s own standard error value? Why does no one really talk about this ever (kudos to the PlasticList team for admitting it)? Who knows!? To convert from RPD to % error, I’m going to divide by two (assuming that the “true” value is the average): 10%.

Uh-oh, there’s also some uncertainty in sampling within the Baconator. Let’s say that a certain part of the burger has most of the dibutyl phthalate. Well, the PlasticList team was diligent and homogenized their burgers into a pulp before testing. However, there’s always some sneaky error here. When you homogenize your burger, are you causing it to lose or gain water weight due to the exposed surface area of little burger particles? Are some of the compounds going to sink to the bottom of your Baconator goo for whatever reason so that when you sample from the top of the goo, you’re getting a non-representative sample? Are some of the compounds adsorbing to the walls of the blender? Who knows! I’d estimate at least another 10% error here, given my experiences in testing for heavy metals and seeing weird stuff happen to concentrations due to sample prep.

And there’s also variation between different samples of the same thing! If I get two different Wendy’s Baconators, they might have different values of DBP. PlasticList found that if they tested two samples, they differed by on average 59%! They also found that samples tested in triplicate had lower divergence from each other (33%), but I assume this means divergence from the average of the three? I’m not entirely sure how they calculated this, or if they mean that for whatever reason, when they sampled three times instead of two, there happened to be lower divergence between each pair within those subsets. Regardless, I’m going to take the 59% and divide by two to estimate an error of ~30%. This is by far the largest contributor to uncertainty in these calculations. Since hopefully these sources of error aren’t especially correlated, this will matter the most.

Okay, so now I go around the city and test burgers from different restaurants and make a list of the burgers that have the most and least dibutyl phthalate. I find that at one location, the burger has only 10 ng/g of DBP, whereas at another location, the burger has 200 ng/g. Is this difference real? Hard to say. If I tested 100 burgers, I’m bound to find some outliers, just from random variability, that are 2-3 standard deviations away from their true probability, even if all the burger joints have the same “true” average level of DBP. This is the essential problem with this kind of methodology.

Moreover, if I’m now testing each burger for a dozen different things, I’m basically guaranteed to find a few “surprising” outliers, even if all the burger joints again have the same “true” average levels. This becomes magnified if I start to do this testing regularly over time!

And so, even if a group like PlasticList does everything flawlessly and thoughtfully, without motivated reasoning, their table showing 18 compounds tested over hundreds of products will invariably be mostly noise. A few of the outliers (and in trace detection of these sorts of compounds, outliers means more of the toxin, since an outlier in the other direction would be a negative amount, which is not possible) seem significant.

For instance, they found that Korean War rations from the 1950s had somewhat concerning levels of several contaminants. But this isn’t particularly surprising. I personally wouldn’t put 70-year-old military hard-tack in my body, regardless of how little plastic-related compounds are in it.

However, I don’t think that someone should make any other decisions based on this data. They seemed to think that water purification tablets and some boba tea might have high amounts of BPA, but BPA is (1) probably not bad for you at all at these levels and (2) there’s so much sample-to-sample variability that it doesn’t matter anyway. The vast majority of everything that was tested fell substantially under any regulatory limits, even the very cautious EU limits.

5. So… We’re Gonna be Okay, Then?

As far as risk factors in your life, I would say phthalates and bisphenols should probably rank fairly low. I’d guesstimate that if you’re living in a country like the United States, the food you consume probably accounts for ~10% of the variance in your health outcomes (there are other things: genetics, exercise, trauma, nutrition, stress, viruses, car accidents, etc). Within that, toxins in that food are probably only ~10% of that total variance. So, maybe 1% of variance in your health outcomes could be explained by toxins in your food, much of which is fairly priced in after you stop being a child. And a decent portion of that is heavy metals. If given the choice between completely eliminating plastic-related compounds from your food and driving 5% less, I recommend you do the latter. Car accidents are the leading cause of death for young adults.

Will this stop scientists around the world from publishing hundreds of papers a year doing similar analyses to the one PlasticList performed, just much, much worse? No, it will not. Nor will it stop major news publications from continuing to write very badly articulated press reports about that science. But perhaps you can decide to stop worrying about it.

You can find that tracker here: https://t.co/ViOMdsik7b

It gives TL;DR summaries for each of the executive orders from Jan 20th, 2025 onward, as well as their current implementation status, a forecast of how likely they are to be blocked due to litigation or logistical challenges, and a rough forecast of their impact. It’s important to remember that there’s often a wide range of potential outcomes with executive orders: many of these simply mandate agencies to draft documents exploring ideas or creating policy proposals which are then fleshed out later. Due to the transition and a delay in updating the Federal Register, it’s also slightly unclear which of these will be classified as “Executive Orders” and which will be classified as memoranda, proclamations, or other directives.

I hope this was a useful aggregator of information, and I intend on updating the spreadsheet weekly. If you have any recommendations for further details for the tracker, let me know in the comments!

It’s been five long years, and I’m now free: free to write blog posts instead of articles for peer-reviewed publication. In an ideal world, not much would change because the ways of exchanging information among scientists would not be dramatically different from the ways that we do so outside of academia. However, we do not yet live in that world, and instead, you can expect the following adjustments:

• I can communicate, format, and order my ideas in ways that are intuitive and optimized for my audience to understand them (with bullet points and links) rather than exclusively through a 15 page document with 5 figures, 40 references, and an introduction section of which 90% is a crude, mandated paraphrasing of prior literature.

• I can disseminate my ideas without going through a fairly arbitrary and counter-productive review process where these ideas are subject to the whims of 2 to 4 experts, selected by an editor with five minutes of thought, all of whom should be assumed to have ulterior motives.

• I can write about whatever I want, express varying degrees of confidence and uncertainty in my ideas, and use methods from other disciplines. I will not be limited to a small subfield of research, siloed off from its academic neighbors.

• I can put two spaces after my period like G*d intended. (Correction: perhaps I cannot… when I copy/paste content from Google Docs into Substack, it automatically removes the second space… *sigh*)

Thus, you should expect a large volume of garbage-quality hot takes that I am extremely unqualified to make. Also maybe a crossword puzzle as a treat every so often.

My first two posts, which should go out over the next couple weeks, will be about the PlasticList initiative and a forecasting competition that I ran last year in my department. I’m still settling on a name for my blog, so right now it’s called “The BS Detector,” BS being my initials of course.

-Ben