In the last year I have gotten hyped up about predictions. Why? heres a few reasons:
- Statistics is fun. Rationality and cognition are fun topics.
- It exercises self awareness. We all have beliefs about the world around us. How often do we deliberately test the accuracy of our beliefs? If you dont test yourself, how do you know your beliefs arent far off the mark? If your beliefs are really off the mark, is that going to work out well for you?
- It exercises objectivity. We all feel certainty some of the time about some of the things. If you feel really certain, is that 70%? Is it 90%? 20 points is a big difference! If you can reliably put your certainty into a number, then all the powers of math are at your disposal.
- Seems like being able to reliably predict useful things would be a pretty good ability to have, so its probably worth practicing.
To self improve, I am doing two sorts of predictions.
Calibration Predictions To calibrate myself, one thing I can do, is write down as many statements as I can think of about how the world might be in one years time. Then, I assign percentages to them. Once the year expires I see how many statements came true. If I am well calibrated then 50% of the ones I assigned 50% should come out true. For the ones I assigned 100%, all of them should come out true. If my 90% predictions happen only 70% of the time then I guess I am too confident or biased in some way, and I should probably figure out why. Also, since I am making predictions in many different areas, I can see what domains I am over or under confident in. Callibrating isnt about "winning" exactly, its about demonstrating that my own certainty is aligned with the actual probabilities of these events.
Winning Predictions Another form of predicting is competitive, and actually is just about winning. These are just regular cash bets, and winning these bets is defined by revenue, not percentage of the time I am right. To demonstrate what I mean, I could lose a majority of the time, but only make super long shot bets where my winnings are huge, and still make money. That would suggest I know what super unlikely events are under-estimated better than other people betting on those same outcomes. If I have some higher awareness of how the world works (relative to other people who bet), then you would think I would be able to make money. Making money is a way of verifying that.
In 2017, I put about $100 into the website predictit.com, which is a political betting website. Im not particularly drawn to politics as a betting subject matter, but its one of the few things people are actually willing to bet on. I invested that $100 into 30 or so bets rather than just a few, so I could minimize the possibility of just getting lucky/unlucky on the few bets I made. Slowly those bets are resolving, and as they resolve I would like to reflect on them in a diary-like way. Here are the first eight:
I bought in this bet when the market thought there was an 82% chance Trump would make it to the end of 2017. More than 82% of presidents last their first year in office, so whats pulling the market down to 82%? Here are my answers:
- People just arent that rational about politics
- People believe the news and they shouldnt, in part because they end up believing information they enjoy indulging
- People over estimate how much the crazy things they hear in the news will matter in determining the length of Trumps presidency.
I didnt really want to sell this bet, but I wanted to buy another bet and I didnt have any available cash in my predictit account, so I just sold off some of my Trump bets to free up some money.
My guess is that the big questions regarding this market are if some scandal will come out, or if he will die before 2020. Probably not. Assuming he doesnt die and no scandals come out, I assume he will run like he always does, and this market will slowly creep to 100% odds that he will run for president. But 2020 is a while off, so theres going to be a lot of random noise-y events that bump the price of this market up and down. I think some news story came out that bumped the price up a bit. Cool. So, I sold while its high, and I will try to buy it again later when some noise-y event knocks the price down a little bit. By buying and selling like this I am committing myself to the idea that people are too sensitive to short-term news events.
The market odds for this bet were 2%. No one thinks Sykes will be nominated. However, the next SCOTUS nomination is a long ways off, so like the Sanders bet above, a lot of random stuff could happen in the mean time. Is there any reason to think 2% is under estimated? Well, Sykes was the first person Trump ever named as a possible Scotus nominee, long before he published a list of 20 candidates, so it seems to suggest to me, that she is in a different logical position in whatever judgement Trump and his staff make about SCOTUS nominees generally. Maybe shes an obviously good option, and the other 20 are candidates Trump struggled to come up with. Maybe shes just the easiest name Trump was able to recall, and the other 20 are better and more deliberated options. Who knows. Point is the odds that she will get nominated probably rest on fewer statistical variables than the other possible nominees, which ceteris paribus I expected to be higher than 2%. My expectation was that nothing would happen in this market, but because the odds were so low (2%) I expected that if anything did happen, the price would go up a lot. I bought, then the market bumped up to 3% so I sold with a return of 3%/2% = 150%. The odds went up to 5% so I guess I missed out.
Only a 5% return, so not a big gamble really. For these types of questions I just add up a lot of numbers (percentages of SCOTUS justice mortality, retirement rates, etc), and then guess which numbers other people will get wrong. The number I think people will get most-wrong is the justice mortality rate, which I suspect people will over-estimate, leading me to think the next vacancy is farther off than what people think.
I found this one right when the market opened. Usually right when markets open, the price quickly adjusts itself to a very different level. That means if I bought fast with even a vague sense if it was going up or down, I could make easy money. I bought the bet that Trump wouldnt mention Gorsuch for four reasons:
- I skimmed over some old George Bush SOTU address and looked for any reference to supreme court justices. I found none.
- It doesnt really seem like the kind of thing you would say in a SOTU. My expectation is for the president to say only boring, unifying, forward looking, and vague statements meant to be understood by your average American. Most Americans have no idea who any of the justices are so talking about the SCOTUS will be confusing for them. A SCOTUS nomination has already happened, so its not forward looking. Fighting about the judiaciary is really ugly and divisive, so it seems like something to steer away from.
- There was another market that put 74% odds that Trump would mention senator Schumer, and that to me seemed a lot more likely than Trump mentioning Gorsuch.
- I tried to imagine what Trump could possibly say about Gorsuch, and the only thing that came to mind was something like bragging about it in some way. That seems to follow in line with the expectation that Trump will act like Trump, and I would like to bet against that expectation.
Anyway I won. I bought at 65 cents a share and sold at 80 cents a share. Trump didnt mention Gorsuch, so I would have won bigger if I held onto it until completion, but I knew going into this bet that I didnt want to wait to completion, I just expected the market to go well above 65.
This was an interesting and educational loss. To research this bet, I looked up who the FCC chairmen were in the FCC, and then looked for statements those individuals had made on net neutrality. For 3 of the 5 chair people I found, I found strong anti net neutrality statements from them. Figuring their statements will correspond to their votes, I concluded that 3 of the 5 would vote no on repealing net neutrality and made the bet that they wouldnt repeal at 50% odds. Some time later, I looked into it again and realized that I mistook Tom Wheeler as a member of the FCC, when in fact he retired in January. So, I sold my bet for a loss, and bought a new bet that they would repeal net neutrality at 70% odds. Sure enough they did vote to repeal net neutrality, but unforuntately for me the bet didnt resolve. It turned out that the terms of the bet specified that the bet wouldnt resolve until they published the rule It seems like a lot of people make this mistake. At this point, there were only a few more weeks in 2017, and so it was pretty clear they wouldnt publish the rule in 2017. I lost once when I flip flopped on my bet due to bad information on who were the FCC chairmen, and then I lost again on a poor understanding of the terms of the bet.
I screwed up. Here are my lessons:
- Dont just go to FCC.gov and think the people listed are the actual FCC chairmen
- actually study the exact conditions of the bet
- Other people are going to make these mistakes, maybe I should take that into consideration. For example, I could apply this technique to the market on whether Trump will get impeached. Most people seem to think of impeachment as "the president gets convicted and kicked out of office", but actually the definition is much weaker; "impeachment" just means that a charge is brought against him by congress. If people are betting with this misunderstanding of impeachment, they are underestimating the possibility of impeachment happening.
I saw the news that Jerome Powell was confirmed by the senate right when it rolled in. I remembered that there was a prediction market on that topic, so I anticipated that the price was going to jump up. Before buying tho, I did my research on the resolution criteria of the bet (learning from the FCC bet above) and realized it was pretty messy. The problem is that Yellens term ends on the 3rd, and its not clear that Powell automatically becomes the chairman on the 4th just because he was confirmed by the senate. Why not? The 4th is a sunday, and public officials often just dont do their swearing in on sundays. Also February 4th is superbowl. Furthermore, chairman Yellen's term didnt start until several days after Ben Bernake's term end; there was like 3 or 4 days where she was the "acting" chairman, but was still only a vice-chairman because she hadnt sworn in or gone through whatever ceremony they do. Reviewing federal officials generally, indicates to me that its unlikely Powell will actually be the chairman on the first day of the term.
So I thought I was going to rush over to this market and bet "yes", but instead I rushed over, researched the truth, lost my confidence in "yes", waited a day for the excitement of Powell's senate confirmation to clear, and then I bought "no" the next day. Over the next week, the market odds went from 60% sure Powell would be the chariman on Feb 4th to 15% sure. Seems like the market caught up with me in its understanding of how things will go. I cashed out before the 4th, because I dont actually want to gamble on how the resolution criteria would be interpretted, but still at a much higher price than what I bought in at.
Thats only 8 results, but if you add up all my returns, I am at about 70% profit. Thats looking pretty good! Truth be told tho I am a little below the $100 I put in tho, because the FCC bet I lost was the biggest investment. That sucks. There wasnt even a good reason for it to be the biggest, Ive just been betting random amounts based on what cash I have available in my predictit account in the moment. Moving forward I would like to be more mindful about the amounts I invest. I will try and invest to the degree I think bets are underpriced, and otheriwse try and standardize all my bets to the same amount to make the numbers easier to analyze. Cumulatively, these 8 bets only represent maybe 18% of the $100 I put in, so there are a lot more results to come through. I look forward to it!