Predicting Problems; Buddhism for Vampires; Homebrew Gene Therapy
"Within us all lies some particle of the dove, kneaded into our frame, along with the elements of the wolf and serpent"
machines + society
Mako Shen | June 14, 2021
A Meditation on Prediction
Nostradamus: original portrait by his son Cesar
I.
We learn by surprise. We have a prediction of what is about to happen in the world — that cup near the edge of the table is going to be knocked over soon, our text about 'opening up' the relationship is going to infuriate our spouse — and we update this model if we get surprised.
The problem is that often our predictions are mis-calibrated or unverifiable. Sometimes we aren't as surprised as we should be, and sometimes we don't get surprised at all.
Through making beliefs pay rent, probabilistic predictions are a great way to calibrate and sharpen our personal model of the world. Here are some principles I'd like myself to keep in mind when predicting:
Process over outcome
Rather than whether my prediction comes true, I should focus on the system that I used. Base rates,
Precision
It's really easy to fool myself. "I will like my new sales job" is vague enough that my mood can pretty easily determine whether I think I'm right or wrong. To fight against this, I need to make an effort to precisely specify the state of the world that I'm predicting.
Calibration
Reliable confidence is as important as getting the predictions directionally right. It's no good if most of my 99% prediction is right just as much as my 51% prediction.
Regularity
To get accurate feedback, I should regularly write down what I think
I've been convinced for a long time that keeping track of my predictions is really important for navigating the world in a skilled way, but I discovered that doing this regularly is harder than I’d thought.
Difficulty 1: Question Generation
Generating questions that are both interesting and precise is often harder than making the actual predictions. When I say 'harder', I mean that it requires both more creativity and cognitive effort. This applies especially to making regular predictions that are relevant to our personal interests, which are by nature idiosyncratic and applicable mostly to us.
There are a number of factors that make question generation especially difficult. For one, we can only create questions using the knowledge we currently have access to. For instance, "what will the top social media app be in 2050?" is not an especially useful question in part because there's a good chance that the top social media app doesn't exist yet, so we can’t predict it.
Another way of stating this is that prediction questions— to the best of my knowledge— cannot account for unknown unknowns.
Human thought-space is but a small fraction of the set of all thinkable thoughts. This makes the unknown-unknowns quadrant especially important. Image source.
Forecasting questions do a lot of conceptual work and necessarily make assumptions that constrain the future. "What will the top social media app be in 2050?" assumes that both social media and apps exist in 2050. It could well turn out to be analogous to someone in 1873 asking the question “which telegraph company will be the largest in 2000?”
Assumptions are a type of mini-forecast of their own; each individual assumption may seem reasonable on its own, but it is chained to other assumptions, compounding errors are increasingly likely. “Will the top Drum and Bass dancer on MySpace win the 2040 DnBStep contest?” is dead on arrival.
Difficulty 2: Inherent Uncertainty
It doesn’t help that in many ways, the future is inherently unpredictable.. Edward Lorenz (father of Chaos theory, coiner of 'The Butterfly Effect’) made a useful point:
When the present determines the future, but the approximate present does not approximately determine the future.
He was talking about chaos theory, which studies systems that are very sensitive to initial conditions (i.e. chaotic). A strain of research within chaos theory concerns itself with proving the inherent unpredictability of specific systems. Lorenz showed in 1963 that even for a set of simple nonlinear equations, minute changes in initial conditions enormously changed the relevant outcomes. In other words, beyond a certain very short period of time, forecasts had to become probabilistic rather than deterministic.
“Inherently unpredictable” is doing a lot of work here. Let me explain. “Inherent unpredictability” doesn’t refer to our state of understanding of the world, but rather is a property of the correspondence between maps of reality and reality itself. Models are a lossy compression of reality, and as long as the model is not reality itself, there will be small errors when using the model to predict. The claim within the aforementioned paper is that even when these errors seem really really tiny, they snowball and make accurate long term forecasts effectively impossible in meteorology, anthropology, sociology, environmental science, computer science, engineering, economics, ecology, etc.
According to chaos theory, no matter how much we improve our models, we will never be able to precisely predict a system like the weather very far into the future. So chaos theory itself presents a powerful prediction about the limits of prediction.
These model errors tend to compound superlinearly so predicting an outcome 10 years away is more than twice as hard as predicting it 5 years away. [1]
Difficulty 3: Probability theory is imperfect
There’s also the fact that probabilistic reasoning, for all its power, is highly limited. The world is not a casino, and probabilism can’t help us with many of the topics we care most about.
I can’t put a probability on “will I be happy in my marriage?” because I can’t reduce “happy in my marriage” to a binary outcome. The Wednesday night arguments over money can’t be meaningfully subtracted from the joy of raising a child. The world is nebulous, and probability theory is a very specific tool useful in very specific contexts (the ideal setting is a perfect information game which is non-chaotic, and has a small number of actions/actors).
The best tools we have for augmenting forecasts — using base rates, averaging over multiple predictions — are predicated on probabilistic reasoning. This is a problem if we seriously consider the aforementioned objections to probabilism. [2]
II.
It’s clear that making useful forecasts in general is very hard. So what can we do about it? We can get better at questioning, just do the predictions anyway, and try and apply predictions in our personal lives.
Learn how to ask good questions
There are many workshops on probabilistic thinking, but far fewer on asking good forecasting questions. I think this is a mistake. More practice in learning to ask good questions (whether good = interesting, precise, or creative) would go a long way to advancing our handle on the future.
I have a few thoughts on how one might learn to ask better questions, but I’ll have to save it for a followup post.
Forecast Anyway
There’s this saying that plans are often useless, but planning itself is indispensable. I think we should treat long term forecasting like that. It really clarifies your map of the future that loosely talking or writing about it does not.
In that spirit, here are some interesting prediction questions for the short to medium term future. Some are taken directly from other resources while others are my own. My aim here is breadth, rather than accuracy.
Technology
Ethereum will be the main platform on which decentralized financial applications are built (>60% of most used DeFi apps on Ethereum) in 2025. 80%
More than 20% of web traffic be directed to websites on the Web3 ecosystem by 2030. 40%
Amazon will deliver packages in part of a major city of the U.S. by drone (either terrestrial or aerial) before 2025 outside of trials? 60%
Direct air carbon capture technology will cost less than $50 per metric ton of CO2 before 2035. 50%
Chinese companies will have semiconductors at least as good as that of overseas companies by 2030. 60%
ASML will obtain an export license from the Netherlands to export an Extreme Ultraviolet Lithography System to the People's Republic of China by 2025. 15%
RSA-240 will be factored by a quantum computer before 2040? 5%
Biotechnology
By the end of 2021, a major U.S. government agency release a statement indicating greater than 50% belief that COVID originated in a lab in Wuhan. 65%
The first adult genetic disease will be cured via viral gene therapy before 2040. 70%
A bioterrorist/biowarfare attack using an engineered virus happen by 2040. 60%
Havana syndrome will be attributed to a foreign government weapon by a U.S. government agency by 2025. 35%
The first person who has been cryogenically frozen will be resuscitated by 2050. 5%
Economics
The unadjusted consumer price index (i.e. inflation) will exceed 3% by the end of 2021. 60%
Any interesting questions of your own? Add them to the comments section of the Substack (click the title of this article).
Use predictions in your personal life
When we think about forecasting, we usually talk about forecasting events distant from our personal lives (“will China invade Taiwan by 2025?”), but it is potentially much more directly useful to think about making predictions in your own life.
Julia Galef has a great post about 16 types of predictions that you can use directly in your life. Here are some of my favorites:
Predict how long a task will take you. This one's a given, considering how common and impactful the planning fallacy is. Examples: "How long will it take to write this blog post?" "How long until our company's profitable?"
Predict how you'll feel in an upcoming situation. Affective forecasting – our ability to predict how we'll feel – has some well known flaws. Examples: "How much will I enjoy this party?"
Pick one expert source and predict how they will answer a question. This is a quick shortcut to testing a claim or settling a dispute. Examples: "Will Cochrane Medical support the claim that Vitamin D promotes hair growth?"
When you meet someone new, take note of your first impressions of him. Predict how likely it is that, once you've gotten to know him better, you will consider your first impressions of him to have been accurate.
Predict whether a surprising piece of news will turn out to be true.
In the same article, she points out that there is often this obsession with generative objective metrics that can resolve cleanly to a 0 or 1, but that it is often still very helpful to generate subjective forecasts. I agree. We should obsess less about the evaluation and more about the process of asking and reflecting.
One thing I’ve just implemented as a result of writing this article is a prompt in my weekly self check-in for predictions. I think I’ll set up a sheet with predictions sorted by date that I go in and evaluate every couple months.
If you end up being inspired to make some predictions yourself, I’d love to hear about what they are/how they turn out. You can email me directly or reply in the comments section of this post.
Notes:
Public interest in prediction is bubbling along at a nice simmer. Philip Tetlock in a big way sparked this with his book Superforecasters and his Good Judgement Project. Now prediction forums are popping up all over the place. Those without currency involved include Metaculus, Good Judgment Project and CSET's Foretell. Then there are those where you can actually make money: PredictIt is a low-cap academic project/betting market run out of Wellington that Americans can take part in. Augur and Polymarket are two decentralized prediction markets that are trying to make question proposal really easy. But you don’t need platforms to start making predictions.
[1] The philosopher Nick Bostrom accepts that specific individual predictions in the short to medium term are hopeless but argues that in the very long term, it is possible to predict with a high degree of certainty. This is part of what he calls the Technological Completion Conjecture.
[2] Other problems of probabilism include the problem of priors and more generally the fuzziness of subjective probability. My favorite resource on this is David Chapman’s In the Cells of The Eggplant.
Prompts:
What does it mean for the future to be inherently uncertain?
How could do you incorporate personal predictions into your own routine?
What is a current shortcoming of probability theory?
📰 Assorted Links 📰
“It’s almost impossible to predict the future. But it’s also unnecessary, because *most people are living in the past*. All you have to do is see the present before everyone else does.” Crawford tweet thread. Endorsed.
Videos
"A little over 2 years ago, my life changed when I took a home brewed gene therapy to attempt to get rid of my lactose intolerance. within days of taking the therapy I went from being violently ill if I ingested even the slightest hint of lactose, to being able to eat a quart of ice cream without so much as a second thought." Simultaneously horrifying and exciting. From a notable biohacker.
Papers
Jay-Z’s 99 Problems, Verse 2: A Close Reading with Fourth Amendment Guidance for Cops and Perps. [h/t Gwern]
Nothing makes sense except in light of individual variable response. This paper demonstrates a massive range in response of cardiovascular fitness to a very standard specified fitness program. “large individual differences in CRF response (range: −33% to +118%) have been observed across the 8 exercise training studies independent of exercise duration” [h/t Gwern]
Articles
Andrew Gelman on Noise, the latest book by Daniel Kahneman and Cass Sunstein. “These guys get too much deference, more than is good for you... In celebrity academia, once you’re high enough in the stratosphere, you can stay afloat forever.”
*Are U.S. Officials Under Silent Attack? On Havana Syndrome: “[U.S.] officials described being bombarded by waves of pressure in their heads. Some said they heard sounds resembling an immense swarm of cicadas, following them from room to room—but when they opened a door to the outside the sounds abruptly stopped. A few reported feeling as if they were standing in an invisible beam of energy. The aftereffects ranged: debilitating headaches; tinnitus; loss of vision and hearing; vertigo; brain fog; loss of balance and muscle control… [t]heir working hypothesis is that agents of the G.R.U., the Russian military’s intelligence service, have been aiming microwave-radiation devices at U.S. officials to collect intelligence from their computers and cell phones”. [New Yorker]
The 2009 DARPA Networking challenge: "In the competition, teams had to locate ten red balloons placed around the United States and then report their findings to DARPA. ... The contest was concluded in under nine hours, much less than expected by DARPA, and had many implications with regards to the power of online social networking and crowdsourcing in general."
Need for cognition as a general personality trait. Many of my friends have in common a high need for cognition. Among my peer group, I have quite a high need for cognition, despite certainly not being the most intelligent. I'm glad I stumbled across this concept.
"I sent an e-mail to the proprietors at Penguin Warehouse asking all the questions I had, and telling them I was ready to order. They had a variety of breeds and ages available, and after a couple hours of research, I decided on a Snares Island penguin. I was going to name him Magellan… ‘I know I always tell you not to do your crazy ideas and you usually do anyway, but listen to me on this one. Don't do it’ …" [Link]
Fiction
Buddhism for Vampires. My favorite online fiction series. From the great David Chapman. A great introduction to Buddhist tantra with a compelling eastern fantasy setting. Relatively short also.
🎧 Music 🎧
Silva e Ludmilla - Um Pôr do Sol na Praia. Dance to this.
Lamparina e A Primavera - Não Me Entrego Pros Caretas. Dance also to this.
Ahmoudou Madassane - Zerzura Theme II. Hike through the desert listening to this.
Julien Chang — Deep Green. Feel the stirrings of youth listening to this. Ethereal guitar + stirring words echoed throughout.