Unprotocolized Knowledge

IN 2021, a group of theoretical physicists and computer scientists uploaded a paper to the open-access database arXiv. In a swift series of mind-bending pages, “The Autodidactic Universe” outlines a bold theory:

We present an approach to cosmology in which the Universe learns its own physical laws.1

Drawing on the examples of biological evolution and deep-learning neural networks, its authors suggest that a dynamic system can be said to “learn” when rules emerge that alter its own internal processes. Many examples of learning appear to exhibit this property, from the vast learning system that is natural selection to intellectual learning among humans.

If the universe can learn, can societies? Personal learning doesn’t necessarily ladder up to a shared body of knowledge. How then is collective experience codified and maintained? How have we created a scientific commons and a sense of progress?

Wisdom and tradition reflect a social-scale learning which holds collective memory in the form of cultural practices. But our civilization has managed to create a globalized knowledge space that spans cultural boundaries. This knowledge differs from wisdom, tradition, and mere data. It is legitimized through the rules and procedures it passes through. It is only because of these shared, recognized protocols—teaching, refereed publication, credentials, and empiricism, to name a few—that our society construes itself as knowing anything at all.

By setting out the boundaries of global validity, knowledge protocols also determine the lines of conflict where new ideas and theories fight for inclusion.

In the recent past, universities and journals provided the stage for these conflicts. Today, they take place on the internet’s turf. We are no longer in the realm of enlightened discourse that characterized science’s genteel origins. Debates over the interpretation of evidence have spilled over into the Twitter trenches, the podcast pit, the comment cockfight.

The ideas generated by amateur scientific dialogues can now accumulate as much attention and awareness as those organized by academic institutions. Are they violations of our knowledge protocols or are they signs of protocol insufficiencies?

By some measures, the guardrails of scientific knowledge appear to be broken. While charismatic memers may produce regimes of “alternative” truth, the university is also guilty of falsification scandals and questionable findings. We don’t yet have methods of determining whether populist theories are any better than the results of perverse university incentives.2

Is it possible to change the protocols? Do we have agency in this process? Should “broscience”3 replace science? The history of our protocols’ painstaking evolution suggests that there has never been an infallible system of knowledge production, and there never will be. But as dissonance grows between collective concerns and validated truth, the protocols must evolve.

The Seed Oil Wars

Science doesn’t produce a consensus answer for every question it asks. Some esoteric puzzles have evaded resolution for centuries—for instance we still don’t have a complete understanding of how aspirin works.4 Other issues are longstanding debates, like the sources of America’s high rates of obesity and heart disease. These are cause for public conjecture and concern, giving rise to a host of controversial theories, some of which have gained a foothold within internet-­disseminated lifestyles.

Let’s take a look at one of these speculative theories: are seed oils bad for you? It has become nearly impossible to browse Twitter without encountering the seed oil war. The battle is an ongoing series of theoretical skirmishes, fought with various munitions. Interpretations of scientific studies are special artillery for both promoters and debunkers of seed oil narratives, but they are vastly outgunned—on both sides—by the heavy bombardments of lifestyle influencers and memetic guerilla warfare.

In all seriousness, the uncertainty of the seed oil question has gone from the butt of a joke to a real public interest issue. Examining the dynamics at play illustrate how today’s knowledge creation protocols lack trust and fall short when it comes to disputes that extend beyond the university.

Where did the issue originate? It’s difficult to say. One of the earliest catalysts of the anti-seed oil craze was Ray Peat, Ph.D.: dietary influencer and scientist of notable disrepute (1936–2022). Peat’s dissertation at the University of Oregon examined hormone production in hamsters and he maintained a focus on endocrinology throughout his life, publishing a number of papers and blog posts for which he gained notoriety. One of his most popular and reiterated theories is that the polyunsaturated fats (PUFAs) found in vegetable oils like Canola inhibit thyroid function.5

Peat’s record is spotty at best. He sold unprescribed progesterone to clients at a local weight loss clinic (the supplements were eventually seized by the FDA).6 His eclectic diet regimen has produced mixed results.7 And some have accused Peat of cherrypicking studies to support his claims about PUFAs.8 Peatism has nonetheless grown immensely popular, especially among young, right-leaning bodybuilders and dietary experimenters on the internet.9

There are influencers with stronger bona fides than Peat involved in the seed oil debate, but they can still be undisciplined when extrapolating from inconclusive scientific evidence. Dr. Cate Shanahan10 is another influential figure responsible for promoting the theory. A medical doctor by training, Shanahan has since become known as a celebrity nutritionist, designing diet programs for the NBA. Citing a few studies in each blog post, Shanahan links the ubiquitous frying oils to diabetes, points to the potential cancer risks of diets high in linoleic acid, and claims that seed oils increase sunburn.11 If you take Shanahan’s word for it, seed oils might be responsible for every health issue there is. Paul Saladino, another internet dietician, comes at the issue from a different angle. According to Saladino, our ancestors didn’t eat seed oils, so we shouldn’t either. Instead, he encourages his followers adopt a meat-heavy “ancestral diet,” and to avoid foods on his “Bullsh*t List.”12 Saladino’s appearance on Joe Rogan’s podcast played a major role in spreading mainstream awareness of seed oil theories.

Shanahan and Saladino are, at their core, influencers. There’s no doubt that they draw on published scientific collateral, but they do not prove their theories in a way that could be deemed conclusive. Others, meanwhile, have stepped up to refute their claims. Simon Hill, a nutritionist and creator of TheProof.com, has attempted a refutation of Saladino’s posts.13 Ironically, Hill himself uses his popularity and platform as a credentialed fact-checker to promote plant-based diets. A Patreon-supported nutritional science blogger who goes by The Nutrivore has written an extensive critique of what he calls “seed oil sophistry”14 and can frequently be seen on Reddit answering questions about the validity of anti-seed oil claims.

Another factor adding fuel to the fire is the vegetable oil industry’s indisputable influence on nutrition science. Procter & Gamble, the owners of palm- and soy-based Crisco, have prolifically claimed since 1911 that seed oils are healthier than other sources of fat.15 The company’s earliest claims were made without any concrete evidence, and later claims relied on studies performed by the American Heart Association—an organization that P&G provided startup funds to in 1948.16 Revelations like these are fuel for conspiratorial thinking.

This unhappy tableau depicts the challenges of knowledge production today. The hypothesis that seed oils are detrimental to health is an interesting one. Claimants and detractors each have convincing theories, but the science is disjointed; the topic’s varied hypotheses and approaches make it difficult for academics and non-academics alike to understand and assemble a cohesive narrative. This is the situation that our knowledge protocols are supposed to be able to resolve. Here they have come up empty-handed—in fact, most of the action is happening beyond their reach.

In the chaotic space outside the jurisdiction of scholarly protocols, in the subscriber lists of influencers’ Patreons, a horde of enthusiasts comes into play. Here is where the dynamics resemble less a scientific forum and more the “Internet of Beefs.”17 In many ways, the loyal keyboard warriors who represent avatars like Peat, Shanahan, and Saladino are the most critical players in the seed oil wars. With memes and shitposts flying from their fingertips, these commenters add a sharp political edge to the debate, forming ideological alliances to expand the battlefield. Are seed oils bad for you? We still don’t know, but the memes are strong, providing a false sense of consensus.

Populist Paradigms and Protocol Problems

The seed oil debate is an example of a large and seemingly growing category of popular, contested theories that jam up our knowledge-production system. It joins other longstanding subcultural preoccupations that have gained internet notoriety like vaccine-related health concerns. These “populist paradigms,” to twist Thomas Kuhn’s term,18 present a dual challenge. On the one hand, they obviously have spread largely outside the university, bypassing and thus failing to meet the rigorous standards for adjudication that typically accompany scientific claims. On the other, they expose both the perversions and the general limits of the “official” protocols for legitimating knowledge.

Influencers like Shanahan point to the American Heart Association’s record of obfuscating research findings and suggest they are hiding the harms of seed oils.19 There is no direct evidence, but readers have good reason to be suspicious. Consider the confusing vacillation of the World Health Organization on the artificial sweetener aspartame, which it first linked to increased diabetes risk, then later to cancer, and ultimately contradicting itself in a third report. Researchers have conclusively tied this wavering to corruption of the review process by beverage industry consultants.20 And a recent spate of data falsification issues compound this mistrust. Over summer 2023 alone, two academic luminaries, including a figure of no less prestige than the president of Stanford University,21 were suspended over fraudulent research.22 When research misconduct by authoritative sources comes to light,23 it becomes more difficult to discern what information is reputable in general.

As a consequence, we can observe—whether or not corruption is present—a growing mistrust in the nonpartisan ethos of science. And mistrust makes people more open to the righteous-sounding claims of charismatic influencers. On Twitter and Reddit, where “takes” are the coin of the realm, even the most rigorous studies are taken out of context and sensationalized. Science and health are particularly vulnerable to dramatization, since they are easily appropriated as propaganda for internet preachers and lifestyle movements.24 Combine this with “feral scholars” who factcheck, research, and circulate supposedly reputable information in response to the drama. It’s hard to know who to believe.

Faltering trust is only one symptom of the breakdown of institutionalized knowledge production. The academic establishment increasingly fails to support the development of novel paradigms. In Kuhnian terms, a paradigm is a dominant model of reality that underpins the methods and instruments of a scientific discipline. Paradigms are self-reinforcing, resisting theories and anomalies that undermine their validity. If enough of these anomalies accrue, it’s possible for new, non-paradigm hypotheses to gain attention in the scientific community and cause a paradigm shift. Today’s science funding systems excel in moving paradigmatic research toward clinical trials.25 But they are notoriously poor at incentivizing high-risk, high-reward frontier research.26 This is clearly visible in Alzheimer’s research, where scientists working outside the predominant amyloid hypothesis were systematically excluded from grants and publishing opportunities.27 The decades-long silent embargo forced some researchers to pivot to other topics, creating a class of “paradigm refugees” unable to continue their original work in a hostile environment.

So can we call our imperfect protocols “unreasonably sufficient?” In “The Unreasonable Sufficiency of Protocols,” their pilot study on protocols, Rao et al. use this term to describe protocols’ tendency to overperform relative to their scope and complexity.28 But sustained overperformance can create new conditions that outstrip those conditions the protocol came into existence to manage. Credentials are one protocol that have overperformed; the massive expansion of the knowledge-producing class has created needless field subdivisions, more competition, and pay-to-play dynamics across academia.29

The high education level of the populace has also opened up opportunities for lifestyle influencers to peddle less-than-credible infotainment. Journals, too, exhibit this overeffective quality, having extended far beyond their original function. The early emergence of journal publications was a leap forward for organizing scientific discourse; today, their importance to the careers of young researchers has led to predatory practices, like charging exorbitant rates for publication and employing “coercive citation”: editors strong-arming authors to cite articles that benefit their journals’ impact scores.30

Perhaps this quality of protocols might be called their reasonable insufficiency: The emergence of unmanaged maladaptive phenomena around too-successful protocols. Drew Austin’s research shows that while traffic protocols have succeeded in mitigating dangerous vehicle flows, they have also given rise to traffic jams. Ordered congestion can be seen as a success or a failure, depending on your perspective.31 It is a sign that more dangerous vehicle flows have been mitigated—a positive indicator of the protocols’ unreasonable sufficiency—but also a signal that further protocol adaptation and optimization may be required. So it may be that protocols are unreasonably sufficient and reasonably insufficient at the same time.

Can we view our overperformance and its consequences as a win? When a new story of fraudulent research appears almost weekly, it is no surprise to see increasing numbers of enthusiastic outsiders verifying academic claims and doing the research themselves. There’s something powerful about how online seed oil researchers have developed their hypotheses in an unprotocolized way. Seed oils are at the center of a variety of theories, identified as the culprit behind issues ranging from the obesity epidemic32 to chronic disease.33 There is no single canonical version of the seed oil narrative, and these ideas did not originate within the university, instead, they have mutated and iterated over a variety of blog posts from dieticians, entrepreneurs, amateur scientists (and yes, sketchy influencers). Nevertheless, these speculations may collectively be compelling enough to be considered a viable pre-paradigm theory.

The causes of America’s obesity and diabetes crises are questions that currently have no established paradigmatic answers. From this perspective, we consider the surging interest in tracing America’s health crises back to seed oils to be a positive development. Despite its subversion of the protocols, expanding investigation of these problems is worthwhile, even if some new theories fall outside the academic Overton window. The internet is not only an endpoint for academically certified knowledge. It is becoming an expanded space for contestation and debate of evidence. The internet is producing ideas worthy of empirical testing. We don’t have the protocols to do this today.

The Evolution of Knowledge Protocols

The scientific regime is simply unprepared for the challenges presented by populist paradigms like the ones surrounding seed oils. To members of the establishment, the internet’s superspeed and superabundance of information creates a condition of “epistemic tribalism” or “post-truth.”34 This perspective tends to ignore both the diligently scientific attitudes of many internet users and the role that full-time academics play in the social media landscape. The alternative view is more realistic and more optimistic: the internet merely collapses the context between academia and the public, leading to new information behaviors that our knowledge production protocols aren’t designed to handle.35

Luckily, there’s hope of evolution. The authors of “The Autodidactic Universe” write that

a system learns when it is able to alter its internal processes and actions in the world […] to further a goal, which is typically continued existence.36

The safety protocols outlined by fellow protocol researcher Timber Schroff37 display continuous development and accretion of new sub-protocols, adapting to new risk behaviors. Numerical addressing systems were created to render a population legible for tax collection purposes; this innovation represents an efficiency gain over an older protocol designed for the same reason, the census.38

The techniques that societies use to learn also display this tendency toward optimization. Modern science’s flagship protocols—among them empiricism, inductive reasoning, the peer review system, and open publishing—did not suddenly appear in their current form; they have evolved and accrued new layers in a piecemeal fashion over centuries39 For instance, although career advancement in today’s university system rests on publishing in well-regarded, peer reviewed journals, those editorial and curation systems sprouted organically from the exchange of hand-written notes between long-distance intellectual correspondents. As 15th-century scholars shared thoughts, news, and responses across national boundaries, they began to understand themselves as participating in a global scholarly conversation, the Republic of Letters.

Journals evolved as an organizing tactic for people to distribute their ideas, as well as reviews and responses to others’ work, within this accelerating information space. The format was quickly adopted by established institutions, starting with the Royal Society of London’s journal Philosophical Transactions in 1619.40 Philosophical Transactions had a massive effect on epistemic processes and the officiation of knowledge. As scholars began to see acceptance by Philosophical Transactions as an opportunity for critical commentary from fellow intellectuals, they began to write explicitly for publication in the journal. Thus spontaneous evolution of the format turned into mass adoption. As a host of similar publications arose from scientific academies around Europe, publishing in a journal became the de facto way that discoveries, papers, and inventions were validated.41

The overall effect of protocolization is to encode more and more information in a way that makes it dependable and reliable. Historian Steven Shapin argues in his book A Social History of Truth that the credibility of early English science relied in large part on faith in gentlemanly virtue and dignified disinterestedness.42 The new interface of journals enabled that person-to-person trust to be offloaded onto a standardized—protocolized—procedure. And the journal would soon be extended by much more elaborate institutional protocols: citations, refereed publication, and statistical techniques.

Other protocol evolutions have been made possible because of technical advancements, bringing new types of data under the regime of empirical credibility. Recent improvements in mobile sensors and smartphones have unlocked new opportunities for citizen science and crowdsourcing.43 Mobile internet devices allow for the standardized capture of everyday experiences like sleep and heart rate, and apps like iNaturalist enable the outer world to be documented and verified. These sorts of experiential data can now be recorded and leveraged to conduct large studies.44 The increasing datafication of health via biometrics points to future evolutions that may be possible for both scientific and internet health communities.

———

Some speculations may be made here about the origin and development of knowledge protocols. It seems that these protocols tend to appear, or perhaps “crystallize,” in environments supersaturated with information. Studies and methods for leveraging personal data in new ways are only possible in a data-rich networked world. The Republic of Letters itself was made possible by the spread of a standardized postal system throughout Europe in the 1600s.45 With an ever-increasing number of correspondents, the rise of journals to mediate and standardize these exchanges appears inevitable in retrospect.

It is intriguing that today’s internet resembles something of a Republic of Letters in which “philosophical transactions” occur between entire communities, not just individuals. Terminally online conservatives have picked up the importance of the endocrine system from testosterone-maxing bodybuilders, whose “broscience” is informed by nootropics and supplement experimenters who themselves are in dialogue with credentialed vitamin researchers. Amateur health science can in fact be highly rigorous, if solipsistic.46 Regrettably, only a fraction of self-experimentation is bundled and formatted into protocolized—peer reviewed, published, replicated—forms. Most self-reported data are stuck in Tweet threads, Substacks, defunct forums, and Reddit posts. The different normative standards of each platform force claims to mutate as they spread across channels, making them difficult to aggregate and assess.

What if the internet is facilitating an enlightenment of hobbyist intellectuals and scientists, but we don’t have the tools to make sense of their contributions? Our current oversaturated information environment may be a result of the runaway success of knowledge protocols past—in particular, credentials and open publishing. The products of intellectual labor now live freely online, inspiring swaths of DYOResearchers. As it stands, we lack the organizing models necessary to deal with the epistemic uncertainty they create to validate their ideas. In the past, similar situations have prompted protocol transformations. In other words, further evolution is required.

Desiderata for a Protocol Evolution

There are existing efforts to move authoritative knowledge production at least partially online: reviewed preprints47 and citizen science. We don’t think these are particularly ambitious, and they don’t come close to solving the sorts of problems we’ve discussed in this paper. New refereeing techniques don’t address the inclusion of outsider ideas and theories and citizen science is only permitted by the ivory tower because of the high degree of confidence that exists in a paradigm. The information citizen science gathers and makes fact is already theoretically predicted or only needs crowdsourced confirmation. Research efforts to understand animal migratory patterns or weather systems are examples of necessary, but not paradigm-shifting work.

For the contested theories promoted by amateurs to be subjected to rigorous experimentation, different methods will be required. If we want to experiment with new internet-native knowledge protocols, we must not try to squash internet-driven theories in the name of epistemic integrity. Instead we need to adapt and extend the existing protocols to the internet and its engaged communities. A first step is acknowledging the deep bench of untapped talent. In seed oils discourse, we can see that the parties most interested in producing and evaluating the knowledge are excluded from the protocols. They would participate if they could, but they’re limited to populist forums. How could seed oils be anything besides a populist movement?

Here is one speculative idea for a potential approach. Some of the most important recent scientific findings have come from multi-decade studies of participant cohorts.48 Applying this approach to nutrition studies is challenging because the field is young and methods aren’t yet agreed upon. Even so, it may be possible to design a participatory, multi-year study that leverages the strong interest in diet lifestyles such as cutting out seed oils. The sheer popularity of these theories suggest that many amateurs would be willing to participate in the design, execution, and collection of evidence. Widely available biomonitoring devices could bring down the high cost of quantified selfhood.49 The study would need to be heavily protocolized to accommodate data collection across different diet regimes, locales, and levels of participant expertise.

It sounds challenging, but not impossible. There are already new proposals to advance clinical trial design by incorporating self-experimentation data from online communities.50 Similar methods have achieved promising results.51 This sort of participatory public health moonshot is exactly the sort of experiment needed to build consensus and resolve the institutional distrust that plagues our medical system. America’s public health crises demand interpretation free from industry agenda and bias. Going through a health scare can make anyone radically aware of gaps in medical information and systemic corruption. Amateur scientists are developing theories—about seed oils, about glyphosate, about aluminum adjuvants in vaccines—precisely because the health impacts of these areas are under-researched.

Another key to extending our knowledge protocols is to leverage the internet’s speed for rapidly disseminating research outcomes. This property of the internet, often viewed as an engine of disinformation, has equal potential to facilitate rapid science, studies, and collaboration.52 The recent LK-99 replication craze put this potential on full display. When Korean researchers posted a preprint detailing a possible room-temperature superconductor, dozens of labs across the world jumped into a replication effort, documenting their work live on Twitter. While the efforts resulted in failure, something was still learned. More importantly, Tim Hwang reflects, the phenomenon can claim success not only in terms of growing public awareness, but as the internet making possible a “global spasm of participatory science.”53

This type of public, networked replication effort works best for hypotheses that are quickly falsifiable. Would this approach work for something like health research, where multi-year periods of uncertainty cannot be eliminated? Here LK-99 has another lesson to teach us. With a participatory process, we can tolerate more epistemic doubt because we are watching the work happen in real time. We would even speculate that it is because amateurs and professionals were all in on the game that consensus was reached, settled, and accepted by the public so quickly. When the academy comes to a conclusion in a way that excludes outsiders, it’s just more fodder for the conspiratorial element.

A word to the wise: some efforts to protocolize collaborations between the public and professional scientists have backfired. Take the project ResearchHub, a frankensteined amalgam of cryptocurrency protocols and scientific publishing.54 The website leverages financial incentives to drive user behavior; users receive “ResearchCoin” in exchange for posting papers and sharing comments, incentivizing bot behavior and bogus contributions. A major lesson of cryptocurrency protocols is that when you design something with a trust-minimized architecture, people approach it with a mercenary perspective.55

Optimistic calls for decentralizing science should always take these behaviors into account.56 There is one thing that “DeSci” does get right though: expanding the availability of project funding. David Lang’s Experiment Foundation provides crowdfunding opportunities and philanthropic angel capital to indie scientists looking to take on self-initiated projects.57 Emergent Ventures experiments with a similar model.58 Notably, FastGrants leveraged the same infrastructure to quickly deploy capital for urgent Covid-19-related projects in the early Covid-19 pandemic.59 Could we imagine crowdsourced dollars and angel checks directing labs to focus on the internet’s standout public health concerns? Money is leverage over what gets studied, and there’s no reason to overdesign complicated solutions to capital allocation problems. Lab agendas are routinely shifted by industry and philanthropic funding. Impetus Funding has succeeded in convincing nearly fifty labs to pursue longevity research; with coherent hypotheses and competent program managers, we could do the same with any field.60

It’s easy to be inspired by new, optimistic funding patterns. But there is also intrinsic value in the emergent conflicts between scientific knowledge communities and public audiences. Entrenched, paradigmatic hypotheses like amyloid in Alzheimer’s studies are still incomplete, and research agendas may be stagnant. These might be shifted more readily by the unconventional methods of outsiders than by instruments within the university context. When it comes to challenging paradigms and disputing studies, diligence and scruple demand willingness to go beyond any protocol.

As we’ve seen, there are people outside the academy today with as much interest in the validity of scientific claims as members of historical intellectual communities like the Republic of Letters. Institutional asymmetries make it hard for these parties to integrate with the protocols. Some groups attempt to cut through the red tape by conforming to the shape and aesthetic of sanctioned protocol outputs—citations, methods, and academese. Other noble vigilantes use plain language to fact check and invalidate the claims of institutions that many view as irreproachable.

We think there’s more to be done. Whatever pressures exist, the shape of future protocols is not predetermined. New evolutions are driven by people with a dedication to the content of science and the pursuit of truth. We are in agreement with the authors of “The Autodidactic Universe.” Whether designed or emergent, there are certain rules that societies use to learn, and these rules themselves exhibit evolutionary properties. The methods of scientific advancement have produced world-changing knowledge. But it would fly in face of history to claim that they are fixed or complete. What is “legitimate” is outstripped by what is “known.” What is known is greatly outstripped by what is unknown. In the face of unknowability, we can commit to only one thing: the continuous tradition of learning. Δ

ACKNOWLEDGMENTS Thanks to Eric Alston, Alice Noujaim, Venkatesh Rao, Chenoe Hart, Chris Beiser, Wydna777, and Kyle Shorin for their feedback and contributions.

Kara Kittel is a researcher and creative producer based in New York. She is a co-founder of Team Rolfes, an experimental 3D animation studio, and Other Internet Research Institute, an in-the-world lab investigating the psychological, political, and economic effects of digitalization. Her current research interests include collaborative authorship and informal coordination practices in online communities. www.linkedin.com/in/kara-kittel-81a2598a

Toby Shorin is a writer and technologist based in New York and the cofounder of Other Internet Research Institute. His research examines the moral genealogy of technological and cultural evolution. tobyshorin.com

———

---

1. 1. Stephon Alexander, William J. Cunningham, Jaron Lanier, Lee Smolin, Stefan Stanojevic, Michael W. Toomey, Dave Wecker, “The Autodidactic Universe.” doi.org/10.48550/arXiv.2104.03902

1. 2. Marc A. Edwards and Siddhartha Roy, “Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition,” Environmental Engineering Science 34, no. 1 (2017): 51–61. www.liebertpub.com/doi/full/10.1089/ees.2016.0223

1. 3. “Broscience Meaning & Origin,” Dictionary.com, July 23, 2018. www.dictionary.com/e/slang/broscience

1. 4. “Scientists Just Learned Something New about How Aspirin Works,” Gizmodo, March 28, 2023. gizmodo.com/how-aspirin-works-inflammation-1850274046

1. 5. “Unsaturated Vegetable Oils: Toxic,” Raypeat.
   raypeat.com/articles/articles/unsaturated-oils.shtml

1. 6. U.S. Congress, Senate, Committee on Commerce, Science, and Transportation, Hearing on Consumer Fraud and the Elderly, 98th Cong., 2nd sess., October 22, 1984, Serial No. 98-115. www.ojp.gov/pdffiles1/Digitization/97592NCJRS.pdf

1. 7. Santiago, “BAP is Right the Ray Peat Diet Does Make Most People (Me) Obese,” Twitter, August 28, 2023. twitter.com/criticalmasc/status/1695876575198126540

1. 8. RationalWiki, “Ray Peat.” rationalwiki.org/wiki/Ray_Peat

1. 9. Joshua Citarella, “Raw Eggs, Pink Pills, and Embodied Identity: Online Communities Create Their Own Proof in a Vacuum of Truth,” Document Journal, December 19, 2022. www.documentjournal.com/2022/12/raw-eggs-pink-pills-and-embodied-identity-online-communities-create-their-own-proof-in-a-vacuum-of-truth

1. 10. Cate Shanahan, “PUFA-Project: Scientific References on Seed Oil Toxicity,” Dr. Cate, September 3, 2020. drcate.com/pufa-project

1. 11. Cate Shanahan, “Seed Oil Makes Sunburn Worse and Ages Your Skin,” Dr. Cate, August 21, 2022. drcate.com/pufa-project/, https://drcate.com/seed-oil-makes-sunburn-worse-and-ages-your-skin/

1. 12. Paul Saladino, “Bullshit,” Paul Saladino, MD.
   www.paulsaladinomd.co/bullshit

1. 13. Amy Bach, “Fact-Check: Paul Saladino on Seed Oils,” The Proof, February 24, 2023. theproof.com/fact-check-paul-saladino-on-seed-oils

1. 14. Nick Hiebert, “A Comprehensive Rebuttal to Seed Oil Sophistry,” The Nutrivore, November 1, 2021. www.the-nutrivore.com/post/a-comprehensive-rebuttal-to-seed-oil-sophistry

1. 15. Drew Ramsey and Tyler Graham, “How Vegetable Oils Replaced Animal Fats in the American Diet,” Atlantic, April 26, 2012. www.theatlantic.com/health/archive/2012/04/how-vegetable-oils-replaced-animal-fats-in-the-american-diet/256155

1. 16. Nina Teicholz, “A Short History of Saturated Fat: The Making and Unmaking of a Scientific Consensus,” Current Opinion in Endocrinology & Diabetes and Obesity 30, no. 1 (2022): 65–71. doi.org/10.1097/med.0000000000000791

1. 17. Venkatesh Rao, “The Internet of Beefs,” Ribbonfarm, January 16, 2020. www.ribbonfarm.com/2020/01/16/the-internet-of-beefs

1. 18. “Thomas Kuhn,” Stanford Encyclopedia of Philosophy, October 31, 2018. plato.stanford.edu/entries/thomas-kuhn/#ConcPara

1. 19. Cate Shanahan, “Cholesterol: What the American Heart Association Is Hiding from You (Part 3),” Dr. Cate, January 14, 2021. drcate.com/cholesterol-what-the-american-heart-association-is-hiding-from-you-part-3

1. 20. Tom Perkins, “Revealed: WHO Aspartame Safety Panel Linked to Alleged Coca-Cola Front Group,” Guardian, August 17, 2023. www.theguardian.com/business/2023/aug/17/who-panel-aspartame-diet-coke-guidelines

1. 21. Max Kozlov, “What the Stanford President’s Resignation Can Teach Lab Leaders,” Nature (2023). www.nature.com/articles/d41586-023-02438-3

1. 22. Rahem D. Hamid, “Harvard Business School Professor Francesca Gino Accused of Committing Data Fraud in at Least Four Papers,” Harvard Crimson, June 23, 2023. www.thecrimson.com/article/2023/6/23/alleged-data-fraud-gino

1. 23. Stephanie M. Lee, “A Famous Honesty Researcher Is Retracting a Study over Fake Data,” Buzzfeed News, August 20, 2021. www.buzzfeednews.com/article/stephaniemlee/dan-ariely-honesty-study-retraction

1. 24. Toby Shorin, “Life After Lifestyle,” Subpixel Space, September 14, 2022. subpixel.space/entries/life-after-lifestyle/

1. 25. P. M. Rothwell, “The High Cost of Not Funding Stroke Research: A Comparison with Heart Disease and Cancer,” Viewpoint 357, no. 9268 (2001): 1612–1616.

1. 26. Paula Stephan, Andrew Young, and Chiara Franzoni, “Encouraging High-Risk High-Reward Research at NIH,” Building a Better NIH, May 2023. www.brookings.edu/wp-content/uploads/2023/05/StephanFranzoniFinal-3.pdf

1. 27. Sharon Begley, “The Maddening Saga of How an Alzheimer’s ‘Cabal’ Thwarted Progress toward a Cure for Decades,” STAT, June 25, 2019. www.statnews.com/2019/06/25/alzheimers-cabal-thwarted-progress-toward-cure

1. 28. Venkatesh Rao, Tim Beiko, Danny Ryan, et al., “The Unreasonable Sufficiency of Protocols,” Summer of Protocols, 2023. summerofprotocols.com/research/module-two/the-unreasonable-sufficiency-of-protocols

1. 29. Randall Collins, The Credential Society: An Historical Sociology of Education and Stratification (New York: Columbia University Press, 2019).

1. 30. Allen W. Wilhite and Eric A. Fong, “Coercive Citation in Academic Publishing,” Science 335 (2012): 542–543. doi.org/10.1126/science.1212540

1. 31. Drew Austin, “A Protocol Pattern Language for Urban Space,” Summer of Protocols, 2023. summerofprotocols.com/research/a-protocol-pattern-language-for-urban-space

1. 32. Anonymous, “Seed Oils Explain the 8 Mysteries of Obesity,” Experimental Fat Loss, July 22, 2023. exfatloss.substack.com/p/seed-oils-explain-the-8-mysteries

1. 33. Jeff Nobbs, “What’s Driving Chronic Disease?,” Jeff Nobbs, March 28, 2020. www.jeffnobbs.com/posts/what-causes-chronic-disease

1. 34. David Roberts, “Donald Trump and the Rise of Tribal Epistemology,” Vox, March 22, 2017. www.vox.com/policy-and-politics/2017/3/22/14762030/donald-trump-tribal-epistemology

1. 35. Alice Marwick and Danah Boyd, “I Tweet Honestly, I Tweet Passionately: Twitter Users, Context Collapse, and the Imagined Audience,” New Media & Society 13, no. 1 (2010): 114–33. doi.org/10.1177/1461444810365313

1. 36. Alexander, Cunningham, Lanier, et al., “The Autodidactic Universe,” p. 7.

1. 37. Timber Schroff, “Safe New World,” Summer of Protocols, 2023. summerofprotocols.com/research/module-three/safe-new-world

1. 38. Chenoe Hart, “Addressable Space,” Summer of Protocols, 2023 (summerofprotocols.com/research/addressable-space); Anna Diamond, “The Enumerated Story of the Census,” Smithsonian Magazine, April 13, 2020 (www.smithsonianmag.com/history/enumerated-story-census-180974648); and
   Anton Tantner, “Addressing the Houses: The Introduction of House Numbering in Europe,” Histoire & Mesure 24, no. 2 (2009): 7–30.

1. 39. Edwards Zilsel, “The Genesis of the Concept of Scientific Progress,” Journal of the History of Ideas, 6, no. 3 (1945): 325–349; and Drummond Rennie, “Editorial Peer Review: Its Development and Rationale,” Peer Review in Health Sciences (2003): 1–13. firstmonday.org/ojs/index.php/fm/article/download/661/576

1. 40. “History of Philosophical Transactions,” A History of Scientific Journals. arts.st-andrews.ac.uk/philosophicaltransactions/brief-history-of-phil-transe

1. 41. W. Eamon, “From the Secrets of Nature to Public Knowledge: The Origins of the Concept of Openness in Science,” Minerva 23 (1985): 321–347; and H. Brown, “History and the Learned Journal,” Journal of the History of Ideas 33 no. 3 (1972): 365–77.

1. 42. Steven Shapin, A Social History of Truth (Chicago: Chicago University Press, 1994).

1. 43. Eric Graham, Shikeshia Henderson, Anette L. Schloss, “Using Mobile Phones to Engage Citizen Scientists in Research.” Eos, Transactions American Geophysical Union 92, no. 38 (2011): 313–15. doi.org/10.1029/2011eo380002

1. 44. Hannah Schmitz, Carol L. Howe, David G. Armstrong, and Vignesh Subbian, “Leveraging Mobile Health Applications for Biomedical Research and Citizen Science: A Scoping Review,” Journal of the American Medical Informatics Association 25, no. 12 (2018): 1685–95. doi.org/10.1093/jamia/ocy130

1. 45. Wolfgang Behringer, “Communications Revolutions: A Historiographical Concept,” German History 24, no. 3 (2006): 333–374; and Brian Ogilvie, “Correspondence Networks,” in A Companion to the History of Science, ed. Bernard V. Lightman (Chichester: Wiley-Blackwell, 2016), pp. 358-71.

1. 46. Joshua Citarella, “Raw Eggs, Pink Pills, and Embodied Identity: Online Communities Create Their Own Proof in a Vacuum of Truth,” Document Journal, December 19, 2022. www.documentjournal.com/2022/12/raw-eggs-pink-pills-and-embodied-identity-online-communities-create-their-own-proof-in-a-vacuum-of-truth

1. 47. Reviewed Preprints, eLife. elifesciences.org/reviewed-preprints

1. 48. Kjetil Bjørnevik, Marianna Cortese, Brian C. Healy, et al., “Longitudinal Analysis Reveals High Prevalence of Epstein-Barr Virus Associated with Multiple Sclerosis,” Science 375, no. 6578 (2022): 296–301. www.science.org/doi/full/10.1126/science.abj8222

1. 49. Ashlee Vance, “How to Be 18 Years Old Again for Only $2 Million a Year,” Bloomberg, January 25, 2023. www.bloomberg.com/news/features/2023-01-25/anti-aging-techniques-taken-to-extreme-by-bryan-johnson

1. 50. Vivek Subbiah, “The next Generation of Evidence-Based Medicine,” Nature Medicine 29 (2023): 1–10. www.nature.com/articles/s41591-022-02160-z

1. 51. Paul Wicks, Timothy E. Vaughan, Michael P. Massagli, and James Heywood, “Accelerated Clinical Discovery Using Self-Reported Patient Data Collected Online and a Patient-Matching Algorithm,” Nature Biotechnology 29, no. 5 (2011): 411–14.
   doi.org/10.1038/nbt.1837

1. 52. Renee DiResta, “The Digital Maginot Line,” Ribbonfarm, November 28, 2018. www.ribbonfarm.com/2018/11/28/the-digital-maginot-line/

1. 53. Tim Hwang, “It’s so Over. Now What?,” Macroscience, August 9, 2023. www.macroscience.org/p/its-so-over-now-what

1. 54. ResearchHub. https://www.researchhub.com

1. 55. Toby Shorin, Sam Hart, Laura Lotti, “Crypto’s Three Body Problem,” Other Internet, January 15, 2024. otherinter.net/research/three-body-problem/

1. 56. Sasha Shilina, “Decentralized Science (DeSci): Web3-Mediated Future of Science,” Paradigm, January 23, 2023. medium.com/paradigm-research/decentralized-science-desci-web3-mediated-future-of-science-2547f9a88c40

1. 57. Experiment Foundation. www.experiment.foundation

1. 58. Tyler Cowen, “The Philosophy and Practicality of Emergent Ventures,” Marginal Revolution, November 12, 2018. marginalrevolution.com/marginalrevolution/2018/11/philosophy-emergent-ventures.html

1. 59. Fast Grants. fastgrants.org

2. 60. Lada Nuzhna, “Impetus Grants: Reflections on 2 Years of Going after Risky Aging Science,” Lada Nuzhna, July 11, 2023. www.ladanuzhna.xyz/writing/2-year-of-running-impetus