Grading 2019 Predictions

I make predictions every year to put empirical tests on my model of the world. I tend to do a lot of predictions, in order to get a larger dataset, and at the end of the year, I grade them. These were made last year in March.  I’ve placed levels of confidence for each prediction with the odds I would bet on those outcomes in the vein of Bryan Caplan. I’ve created a chart at the end to show my calibration versus perfect calibration.

  1. Trump Approval Rating end of year <50% (Gallup): 95% ✔️
  2. Trump Approval Rating end of year <45% (Gallup): 90% (was 45% exactly)
  3. Trump Approval Rating end of year < 40% (Gallup): 70%
  4. US will not get involved in any new major war with death toll of > 100 US soldiers: 60% ✔️
  5. No single terrorist attack in the USA will kill > 100 people: 95% ✔️
  6. The UK will not leave the EU this year: 80% ✔️
  7. North Korea will still be controlled by the Kim dynasty: 95% ✔️
  8. North Korea will not conduct a nuclear test this year: 60% ✔️
  9. North Korea will not conduct a missile test this year: 60% They conducted 10, with several launching many missiles
  10. North Korea will not agree to give up nuclear weapons entirely, contingent on US troops staying in the Korean peninsula: 99% ✔️
  11. North Korea will not agree to give up nuclear weapons as a result of any negotiations: 90% ✔️
  12. Yemeni civil war will still be happening: 70% ✔️
  13. S&P 500 2019 >10% growth (from 2506 on Jan 1): 60% ✔️
  14. S&P 500 will be between 2400 and 3100: 80% (80% confidence interval) was 3231
  15. Unemployment rate December 2019 < 6%: 80% ✔️
  16. Unemployment rate December 2019 < 5%: 70% ✔️
  17. WTI Crude Oil price up by 10% (from $45.41): 70% ✔️
  18. Price of Bitcoin in dollars up over the year (Coinbase – 3823 Jan 1): 70% ✔️ was $7163
  19. Price of Bitcoin < $8,000 (does not double): 60% ✔️
  20. Price of Bitcoin > $1900 (does not lose half value): 70% ✔️
  21. Price of Bitcoin < $12,000 (does not triple): 70% ✔️
  22. Drivechain opcodes not soft-forked into Bitcoin: 80% ✔️
  23. No drivechains soft-forked into existence: 99% ✔️
  24. US government does not make Bitcoin ownership or exchange illegal: 95% ✔️
  25. Self-driving cars will not be available this year for general purchase: 95% ✔️
  26. Self-driving cars will not be available this year to purchase / legally operate for < $100k: 99% ✔️
  27. I will not be able to buy trips on self-driving cars from Uber/Lyft/Waymo in a location I am living: 95% ✔️
  28. I will not be able to order groceries on self-driving cars in a location I am living: 90% ✔️
  29. I will not be able to buy a trip on a self-driving car from Uber/Lyft/Waymo without a backup employee in the car anywhere in the US: 80% ✔️ This is tough. You can get self driving cars in Phoenix, but only if you’re part of the Waymo beta and so far they are free, so no “buying”.
  30. The artificial general intelligence alignment problem will not be seen as the most important problem facing humanity: 99% ✔️
  31. Humans will not be in lunar orbit in 2019: 99% ✔️
  32. SpaceX Falcon Heavy rocket will launch again this year: 90% ✔️
  33. SpaceX will bring humans to low earth orbit: 60%
  34. SpaceX will test the “Starship” mock up this year: 70% ✔️ (pretty sure I just meant this giant water tower thing, not a real launch)
  35. Mexican government does not pay for wall: 99% ✔️ (lol)
  36. Border wall construction not complete by end of 2019: 99% ✔️ (some construction occurred, mostly replacing existing wall)
  37. National Debt increases by >$1 trillion (from
    $21,943,897,000,000): 90% ✔️ (was $23.201 trillion on Jan 1 2020)
  38. There will not be a significant decrease in trade barriers between US and China from pre-2017 tariff levels: 90% ✔️
  39. Democratic RCP front runner will not be Bernie Sanders: 80% ✔️ (front runner on Jan 1 was Biden)
  40. Democratic RCP front runner will not be Kamala Harris: 80% ✔️
  41. Democratic RCP front runner will not be Beto O’Rourke: 80% ✔️
  42. Trump not removed from office or resign: 95% ✔️
  43. Trump not impeached: 70% I was not expecting this
  44. No CRISPR edited babies will be born: 80% (it turns out the researcher responsible for the two 2018 CRISPR edited babies had already treated a third unborn child in 2018 when the story broke. Apparently the third baby was born in 2019 if you carefully read Xinhua, so technically this prediction is wrong, although I meant no other researcher would do anything. Remember to properly word your predictions!)
  45. No full year US government budget will be passed (only several months spending): 90% ✔️ (they basically only do continuing resolutions now)
  46. Some tariffs raised: 90% ✔️ (like a bunch)
  47. Trump administration does not file a lawsuit against any news organization for defamation: 90% ✔️
  • I got 4 of 6 predictions correct at 60% confidence
  • I got 7 of 9 predictions correct at 70% confidence
  • I got 7 of 9 predictions correct at 80% confidence
  • I got 8 of 9 predictions correct at 90% confidence
  • I got 7 of 7 predictions correct at 95% confidence
  • I got 6 of 6 predictions correct at 99% confidence

Overall, not bad at all, and we should note that from last year’s grading, my 60% confidence predictions have tended to be overconfident. I only had 6 of those predictions this year, so actually 66% is the closest I could have been to perfect calibration. 70% also ended up being a bit overconfident, but a single additional missed prediction here would have dropped me down to 66% as well. Had I moved one of my correct 70% predictions to 80%, I would have been perfectly calibrated.

Combining this data and data from last year gives:

  • I got 11 of 15 predictions correct (73%) at 60% confidence
  • I got 14 of 20 predictions correct (70%) at 70% confidence
  • I got 14 of 17 predictions correct (82%) at 80% confidence
  • I got 14 of 15 predictions correct (93%) at 90% confidence
  • I got 16 of 17 predictions correct (94%) at 95% confidence
  • I got 10 of 10 predictions correct (100%) at 99% confidence

In 2018, as I noted in the post last year, I should have made some of my 60% predictions at a higher confidence, but other than that, these predictions are remarkably well calibrated if I do say so myself.

I hope to post my 2020 predictions soon.

2019 Predictions

I’ve made predictions for the past several years, and here are my predictions for 2019, a bit late. I’ve noticed that when discussing politics or difficult subjects with other people with whom I have strong differences, a possible avenue of understanding is to make a prediction about the world with odds. Most people don’t accept these bets or predictions, which I think are one of the best ways to test different models of the world against each other. Nonetheless, I think it’s personally beneficial to list predictions and accompanying odds each year to see test my model of the world.

  1. Trump Approval Rating end of year <50% (Gallup): 95%
  2. Trump Approval Rating end of year <45% (Gallup): 90%
  3. Trump Approval Rating end of year < 40% (Gallup): 70%
  4. US will not get involved in any new major war with death toll of > 100 US soldiers: 60%
  5. No single terrorist attack in the USA will kill > 100 people: 95%
  6. The UK will not leave the EU this year: 80%
  7. North Korea will still be controlled by the Kim dynasty: 95%
  8. North Korea will not conduct a nuclear test this year: 60%
  9. North Korea will not conduct a missile test this year: 60%
  10. North Korea will not agree to give up nuclear weapons entirely, contingent on US troops staying in the Korean peninsula: 99%.
  11. North Korea will not agree to give up nuclear weapons as a result of any negotiations: 90%
  12. Yemeni civil war will still be happening: 70%
  13. S&P 500 2019 >10% growth (from 2506 on Jan 1): 60%
  14. S&P 500 will be between 2400 and 3100: 80% (80% confidence interval)
  15. Unemployment rate December 2019 < 6%: 80%
  16. Unemployment rate December 2019 < 5%: 70%
  17. WTI Crude Oil price up by 10% (from $45.41): 70%
  18. Price of Bitcoin in dollars up over the year (Coinbase – 3823 Jan 1): 70%
  19. Price of Bitcoin < $8,000 (does not double): 60%
  20. Price of Bitcoin > $1900 (does not lose half value): 70%
  21. Price of Bitcoin < $12,000 (does not triple): 70%
  22. Drivechain opcodes not soft-forked into Bitcoin: 80%
  23. No drivechains soft-forked into existence: 99%
  24. US government does not make Bitcoin ownership or exchange illegal: 95%
  25. Self-driving cars will not be available this year for general purchase: 95%
  26. Self-driving cars will not be available this year to purchase / legally operate for < $100k: 99%
  27. I will not be able to buy trips on self-driving cars from Uber/Lyft/Waymo in a location I am living: 95%
  28. I will not be able to order groceries on self-driving cars in a location I am living: 90%
  29. I will not be able to buy a trip on a self-driving car from Uber/Lyft/Waymo without a backup employee in the car anywhere in the US: 80%
  30. The artificial general intelligence alignment problem will not be seen as the most important problem facing humanity: 99%
  31. Humans will not be in lunar orbit in 2018: 99%
  32. SpaceX Falcon Heavy rocket will launch again this year: 90%
  33. SpaceX will bring humans to low earth orbit: 60%
  34. SpaceX will test the “Starship” mock up this year: 70%
  35. Mexican government does not pay for wall: 99% (lol)
  36. Border wall construction not complete by end of 2018: 99%
  37. National Debt increases by >$1 trillion (from
    $21,943,897,000,000): 90%
  38. There will not be a significant decrease in trade barriers between US and China from pre-2017 tariff levels: 90%
  39. Democratic RCP front runner will not be Bernie Sanders: 80%
  40. Democratic RCP front runner will not be Kamala Harris: 80%
  41. Democratic RCP front runner will not be Beto O’Rourke: 80%
  42. Trump not removed from office or resign: 95%
  43. Trump not impeached: 70%
  44. No CRISPR edited babies will be born: 80%
  45. No full year US government budget will be passed (only several months spending): 90%
  46. Some tariffs raised: 90%
  47. Trump administration does not file a lawsuit against any news organization for defamation: 90%

I’d like to comment on every prediction, but I’m afraid it would take too much time. The predictions I find most interesting are whether the UK will leave the EU. I’m going by what predictions markets say, but I’m not really sure it will play out with only a few weeks to go and no deal. I’m also quite interested in the Democratic Presidential primary race, but I’m afraid the nomination will be trending towards someone pretty left wing. I’m also interested in whether Trump will be removed from office. I put the chances of Trump being impeached at 30% to stay in line with prediction markets, but my gut is that I should put that chance even lower.

Twitter Distractions

The U.S. military launched drone strikes on Libya on Friday, the first in Libya since January.  Trump has yet to mention these airstrikes as he’s been too busy fighting with professional athletes about how they protest.  If I’m counting correctly, there have been six Middle Eastern countries Trump has authorized military strikes in despite no authorization from Congress (and seven if you include Somalia): Libya, Syria, Yemen, Iraq, Afghanistan, Pakistan. Not to mention Trump has praised Saudi Arabia, a state that directly funds Wahhabism and an oppressive war in Yemen that does nothing to reduce radicalization.

Important criticisms of Hillary Clinton last year included her foundation receiving millions of dollars of support from the human rights disaster Kingdom of Saudi Arabia. But I’m not sure which is worse: taking bad people’s money or actively praising them. In fact, in what meaningful way is Trump’s Middle Eastern policy different from Clinton’s? Clinton was for a two state solution, while Trump didn’t seem to know what that meant–is that it?

Trump’s foreign policy has been pretty incompetent in other areas outside the Middle East. He’s failed to provide appointments for many ambassador positions, including South Korea. Speaking of which, Trump said he would control North Korea, but the DPRK has conducted more missile tests during his presidency (that’s 7 months) than any presidency in history. Even by using his own stated (terrible) goals of renegotiating NAFTA, tearing up the Iran nuclear deal, and reducing sanctions on Russia, he has failed to do what he said he would. In the case of Russia sanctions, this came at the hands of his own party overruling him in Congress.

Trump is a loud, robust failure in foreign policy. And rather than spend any energy actually trying to end military involvements like he said he would, or even do routine things like appoint ambassadors, he is igniting culture wars on Twitter. I think he prefers these to actual policy because there are no metrics to success when engaging in a cultural flame war online. It’s just “our tribe” vs “their tribe”, and no one can win because we’re not actually discussing anything. I think there are nuances to be had in this week’s particular flare up with the NFL and the national anthem, but they’re not worth teasing out because it’s so easy to get bogged down in an emotional fight.

So rather than engage with Trump’s culture war cage match this week, I think it’s more productive to point out that there are real issues he’s supposed to be dealing with, and he’s failing miserably. We’ve been at war for 16 years now. Soon, recruits will be traveling to battlefields that Americans have been fighting in since before these soldiers were born. But Trump would rather tweet about football players protesting.

 


Leave a comment on the official reddit thread.