Hello! I like making predictions and bets. I like measuring objectivity. I view it as a form of self improvement and a way of building self awareness of how well I know things. I write down what I think and what I expect and then follow up with my own records years later to see where my thinking went wrong. Then I try not to be wrong again. When I started doing this, losing would really hurt, but at this point I get a lot of joy from dissecting where I go wrong.
In this document I am going to go over bets and predictions I made that came to completion in 2020. Here are some previous write-ups I have done on previous years of predicting.
- here is my 2019 Prediction checkin
- heres my most recent set of calibration predictions
- heres my 2018 betting market run down
- heres my 2017 betting market run down
I think at least okay! If not good. I am really proud for how early I knew covid was going to be a big problem in particular (early february). But then 2020 wasn't without it's errors either.
I made two bets with friends regarding Covid in 2020, and one predictit.com bet. Among the personal bets I won one and lost one, and then I lost the predictit.com bet.
I started making little models predicting covids growth late january. By early february I flipped from "forecasting this for fun" to "forecasting because I'm actually getting worried about how this will impact my own future". In early february it seemed to be a very infectious disease, and there was the cold undeniable reasoning that a new highly infectious disease that we are all vulnerable to will continue infecting larger and larger numbers of people until some huge fraction of society has been infected. Seems like simple reasoning that I could find no real counter argument to. There have been other viruses that show up in the news, like SARs and ebola, but beneath the headlines those diseases were never very infectious to begin with. Covid was seemed to be truly different. But I still knew very little, so I leaned quite a lot on comparisons to other diseases, such as the flu, spanish flu, and measles. The way the bet came about, was that I said something like this (because I was saying things like this every day):
Oh man, the spanish flu infected 33% of the US, and it killed 0.5% of Americans. If Covid is half as bad we are still talking about a million deaths.
And my friend's response was very much like "Yeah right..", just like everyone else I talked to.
What worked for me here? I think staying focused on the pure reasoning helped me win this bet. The conclusion- that an infectious disease whos case count is doubling every week, and that was already numbered in the thousands of cases is virtually destined to infect everyone else- seemed bizarre and highly unusual; but there was just no way around the simple water tight logic. It did seem like covid could have been stopped in its tracks by a fast and harsh crackdown, but from my perspective in early february 2020 that clearly wasnt going to happen. So, if the only thing that would have stopped covid, would not occur, then covid would not be stopped. End of story.
I think people tend to dismiss straight forward reasoning when it points to highly unusual conclusions. I have that tendency in me too, but perhaps to a lesser degree. That helped me win this bet.
Also, "its unusual therefore its wrong" actually isnt a half-bad way of thinking in my opinion. It is in fact generally more likely that an bizarre conclusion is wrong in unclear ways than it is for a highly unusual event like covid to occur. Dismissing unusual conclusions is like an energy-efficient form of reasoning that fails only in edge cases.
Big kudos to my friend who made this bet with me. He had the normal view among everyone else I talked to, but he was the one who put the money down and owned up to it. I admire him for this.
I lost this one to my friend Ryan McDermott. We came up with the following terms for what would qualify as a vaccine or vaccine-like treatment:
There will NOT be a treatment for Covid 19 in the next 1.5 years that meets the following conditions:
- It is used by at least 10% of the US population
- It is at least 80% effective at granting immunity to Covid 19
- It induces side effects that are below 25% as bad as catching Covid 19
I think the current vaccines basically meet these conditions. So I lose. What was I thinking?
I remember from before I made this bet, one guy I like on twitter, Razib Khan, who has a lot of credibility in my opinion, said that the fastest turn around from a disease outbreak to a vaccine was for mumps, which took four years. I also knew that some viruses have never gotten vaccines (like herpes, and HIV).
So in my mind, 4 years was the absolute best development time for a vaccine and "we will never get a vaccine" was a possibility. It seemed unlikely to me that covid researchers would beat the absolute best development time. Looking for good arguments for why it would take quicker than 4 years, I recognized that the mumps vaccine was developed like 70 years ago, so perhaps modern vaccine technology would enhance the rate of development. I talked myself out of this counter argument by thinking that even if modern technology sped things up by 400%, it still wouldnt be enough, and then there is still testing and approval, which take years.
So where did I go wrong?
At first, I was thinking "Well, thats the last time I listen to Razib Khan", but then I started to realize it wasnt just Razib who got this prediction wrong. A survey among super forecasters (who are the very elite of people who competitively predict things) seemed to estimate the development time of a covid vaccine as taking 2 years. Other people who have a lot of credibility in my view also got this one wrong. So, I can't just blame Razib Khan.
I think I got the facts really wrong regarding the vaccine process. I have since learned that vaccine development takes mere days, not years, with modern technology; and it seems like the FDA is willing to expidite approval during an emergency. So making a vaccine isnt a two step process where each step takes years, but a two step process where the development step is basically instantaneous, and the second step is totally up to the discretion of the FDA.
I am sure someone out there who knows about vaccines would be laughing at me if they read what I am saying; because they knew from day 1 that a covid vaccine would plausibly take less than a year. In hindsight, its like I was treating this prediction as a mysterious question that no one had discovered the answer to, when in reality, some people knew exactly what the answer was because making vaccines is their profession. The problem was that the information in their head had not transmitted into my head. A lesson I will learn from losing this bet is that when trying to predict something, I should try and imagine who in this world actually knows the answer to whatever question is on my mind and then just go ask them. Its a big world, and a lot of the time the problem is just that the person who knows critical information hasnt distributed that information to people who want to know; rather than the information not existing among anyone. Learning about modern vaccine development from someone who does it would have been way more informative than trying to compare the mumps vaccine to the covid vaccine, which is sort of meaningless in hindsight.
Again, big kudos to Ryan who made this bet with me. He thought something, wrote it down in clear terms with me, and stood for it. I admire him for this.
I lost this one. I figured the market was largely debating whether or not covid would become a pandemic, and in my mind I was certain it would. So I figured the WHO, the authority on that matter, would soon declare it as a pandemic since that was the truth. That did not happen by March 6th however.
So it seemed like I approached this prediction as largely being a question of Covid being a pandemic or not, and that was just the wrong thing to focus on. What actually mattered were the political or institutional factors that would delay the announcement.
In previous prediction reviews I have noted how hard it is to win predictions in new markets that will resolve quickly. I do really poorly in prediction markets that are like "Hey, this new thing, what will it be like in 2 weeks?".
In late 2018 I started saying there will be a recession soon. I made two bets in 2019 on predictit.com: (0) there will be a recession in 2019, and (1) there will be a recession during Trump's first term. I won the Trump one and lost the 2019 one, but since both were high risk bets my overall $$$ return was high. In 2018, I had a conversation with someone that went like this:
Me: I think there will be a recession soon
Them: How do you know?
Me: Well, recessions happen periodically, almost always when unemployment is unusually low, and after periods of maybe 4-8 years of gradually declining unemployment. Thats about where the economy is right now.
Them: But you dont actually know what specifically will cause the recession?
Me: No
Them: Then I dont think you really know that a recession will occur
After I won my bet I had an astonishingly similar conversation with someone else:
Me: I won my bet that there would be a recession
Them: But how could you have known, since you didnt know covid would happen?
Me: Well, recessions happen periodically, almost always when unemployment is unusually low, and after periods of maybe 4-8 years of gradually declining unemployment. Thats about where the economy was at.
Them: So you didnt actually know that covid specifically will cause the recession?
Me: No
Them: Then I dont think you really knew that a recession would occur
So some people seem to think I just got lucky. I dont think I got lucky. Let me try and convince you.
Recessions are kind of like forest fires, in that, if new trees keep growing and forest density keeps increasing then you know a forest fire is inevitable. You know a forest will happen and maybe even approximately when it will happen, even tho you cant predict where it will start or what will cause it. You can still know that a forest fire will happen, even if you dont know whether lightning, illegal burning, fireworks, or powerlines will cause it.
Recessions are also very much like car engines breaking, in that you know it will happen inevitably, and its much more likely to happen after certain amounts of mileage. However, despite knowing that it will happen, you don't really know in advance which part of your engine will break, and you cant really point back to specific events that wear down your engine since its a gradual increase in mechanical stress over time. Nonetheless car engines breaking is a predictable event.
For both car engines breaking and forest fires, little events gradually accumulate over time (mechanical stress, new flammable trees sprouting up) that individually seem harmless with no adverse outcome, until some level of error is reached that brings down the entire system (the mechanical stress leads to a major component of an engine breaking, and a fire starts that can spread to every other tree in the forest).
The economy is the same way. And it has its own little micro-errors that accumulate into big cascading problems. The economy grows, which manifests itself as new jobs, companies, and projects. Most of this growth is good, but some of this growth ends up being a mistake. By "mistake" I mean something like, a factory is set up to make a new product, but then the product doesnt sell very well, and it turns out to that the factory could have been making something more valuable. Here is something that happens more often: a new employee is hired, but then their performance wasnt as good as the company expected. For whatever reason, these mistakes tend to not get corrected; factories dont get shut down or repurposed, and people dont get fired. It is as if its just fundamentally easier for business leaders to say "yes" than it is to say "no". When businesses make a mistake they tend to just live with them forever instead of correcting them. This stress keeps accumulating in all parts of the economy and eventually there is some kind of tipping point where the economy as a whole just cant take it, and unloads all this accumulated stress at once. To put it in concrete terms, if a company occasionally makes mistakes when hiring new employees, and it tends to retain bad employees in jobs they arent suitable for, then eventually the business will simply fail due to the underperformance of all the personel mismanagement. When that company goes bankrupt, its customers and suppliers are hurt, and since they themselves have been accumulating their own economic errors, the damage of their partner going bankrupt is enough to make them go bankrupt too. This cascades out everywhere. This basically what happens during economic recessions in my view.
So, coming back to my bet. No, I didnt see covid coming. But I did see that the economy was at that point in the economic cycle where something could (and inevitably would) knock the economy off track.
This was a loss for me on predictit.com and a great learning opportunity. I made this bet right after Trump got into office in 2017. I could see that the news and politics regarding Trump was going to be extremely ugly. I really did not want to be sifting through lots and lots of news in real time trying to see how the latest headlines impact whether or not Trump would be re-elected. So my strategy was to start from a basic theory that "the Trump presidency will go like every other presidency", and then be extremely reluctant to change my mind. For this particular bet, I considered that most presidents get a second term. So my theory would indicate that it was ~75% likely that Trump would get re-elected, and in 2017 the market odds were like 30% that he would get re-elected.
Maybe that sounds ridiculous to treat the Trump presidency as like any other. I can easily imagine people screaming the exact opposite "This is not normal!!! This is not what the US president is supposed to be like!!". Rather than defend the claim that the Trump presidency truly was normal, let me appeal to the effectiveness of the theory by listing out some bets I won on predictit.com just by asking myself if it was normal for a thing like that to happen to a US president, and reading as little news as possible:
- Trump will be the republican presidential candidate in 2020
- Trump Jr will not be charged with a crime
- Jared Kushner will not be charged with a crime
- Rudi Guiliani will not be charged with a crime
- Trump will not pardon himself
- Trump will not resign from office
- James Comey will not be charged with a crime
- Hilary Clinton will not be charged with a crime
- Hunter Biden will not be charged with a crime
The only one where I lost using my "this is normal" theory was "Oliver Stone would not be charged with a crime". I did well on these "charged with a crime" predictions generally, because I was strongly prejudiced to not believe anyone would ever get charged with a crime, because not being charged with a crime is normal and normal by definition is what usually happens. By the way, these were not low risk bets! People really believed with fairly high confidence that many of these things would happen.
Enough about where I did well, because I ultimatley did not do well predicting the 2020 election. What went wrong?
-
So, I have been enjoying prediction strategies, where I make a good guess early, and then strongly resist ever changing my mind. Its fun to see that work against strategies where people change their mind every day to every new headline. But, I think I have exhausted the effectiveness of not changing my mind, and indeed I have just become lazy. I am starting to see that there is a big practical difference between "Literally never changing your mind" and "Changing your mind extremely reluctantly". For example, I basically tuned out of predictit.com completely during 2020 and did not visit that website or put any thought into anything I bet there. In hindsight, if I ever get to that point regarding a bet I made on a prediction market, where I am not even paying attention, I should cash out immediately.
-
The facts that should have indicated Trump was not going to get re-elected, I think include his low approval rating and the fact that the economy was in the dump. Those are just two really ordinary and reliable indicators that someone isnt getting re-elected.
-
If I ever make a prediction like this again where I plan on predicting early and digging in my heels against changing my mind, I should pre-register things that will make me change my mind so I dont have to think about it in the heat of the moment. I should have done something from the start like "most presidents dont get re-elected when the economy is doing poorly and they have a low approval rating. If either of those conditions are met I should cash out immediately". Thats really objective, and clear, and I dont have to read the news every day to see if those conditions are met.
This isn't really an explicit prediction exactly, but I did go through all the same motions of predicting bitcoin price and then buying and selling based off my predictions. Altho I did well in 2020 buying and selling bitcoin, I also feel like I learned a lot.
I bought some bitcoin in April of 2020, and then sold it in December of 2020 at roughly 300% what I bought it. When I talk to people about bitcoin and tell them "I sold in December" invariably I get a response like "Oh Im sorry I bet you feel so bad". No! I dont feel bad! 300% is really good! Do you know how many people lose money on bitcoin? Lots!
I get what they are saying tho. I sold before the peak bitcoin price, so if I instead sold even a little bit later 300% could have been 3,000%. I am not bummed tho. I am really not emotionally invested in whether or not I did a perfect job selling bitcoin. How could I hold myself to that standard?
What is interesting me, is that this is a pattern for me. The previous two times I have bought and sold bitcoin, I also made roughly 300%, and I also could have made way way more if I just didnt sell so early. So, I clearly need to learn a lesson here. What is going on?
My buy timing is really good. I think I have an alarm clock in my mind that goes off if I have not heard anyone say "bitcoin" for 8 months, so I notice when the bitcoin is stable and mellow, which comforts me into investing a little bit.
My sell timing on the other hand is so-so. My model is that it takes 2 to 3 years for people to collectively forget about bitcoin. Once bitcoin is collectively forgotten, people get interested again as a form of entertainment fundamentally, but also because they actually think they are on the verge of a bitcoin infinite money singularity. The price starts exploding only because these new excited people are buying it. It takes ~6 months for them to get bored of dumping their cash into bitcoin, whereafter the price crashes, and then most of these people lose money. They buy high and sell low.
When I see people talk about bitcoin a lot, and I see the price going up I think "Yikes, I really dont want the value of my savings to be dependent on whatever insane people are doing" so I sell ASAP.
But I am clearly too sensitive to whatever insane people are doing, because if I sold just a few months later in any of these cases- or better yet, just never sold ever, I would be doing super-super-good instead of merely regular-good. Moving forward I think I will consciously delay any sell decisions by 2-4 months.
In the least year I have seen other people express the very same perspective as I have, so I am pretty sure its just becoming common knowledge that this is how bitcoin has worked thus far. My strategy has worked three times in a row, but I cannot believe it will work again. The expectation that the price will go up during a future hype train, and then go down afterwards, will be internalized in the price, and no one will be able to beat the market with that understading alone.