Grading 2019 Predictions

I make predictions every year to put empirical tests on my model of the world. I tend to do a lot of predictions, in order to get a larger dataset, and at the end of the year, I grade them. These were made last year in March.  I’ve placed levels of confidence for each prediction with the odds I would bet on those outcomes in the vein of Bryan Caplan. I’ve created a chart at the end to show my calibration versus perfect calibration.

  1. Trump Approval Rating end of year <50% (Gallup): 95% ✔️
  2. Trump Approval Rating end of year <45% (Gallup): 90% (was 45% exactly)
  3. Trump Approval Rating end of year < 40% (Gallup): 70%
  4. US will not get involved in any new major war with death toll of > 100 US soldiers: 60% ✔️
  5. No single terrorist attack in the USA will kill > 100 people: 95% ✔️
  6. The UK will not leave the EU this year: 80% ✔️
  7. North Korea will still be controlled by the Kim dynasty: 95% ✔️
  8. North Korea will not conduct a nuclear test this year: 60% ✔️
  9. North Korea will not conduct a missile test this year: 60% They conducted 10, with several launching many missiles
  10. North Korea will not agree to give up nuclear weapons entirely, contingent on US troops staying in the Korean peninsula: 99% ✔️
  11. North Korea will not agree to give up nuclear weapons as a result of any negotiations: 90% ✔️
  12. Yemeni civil war will still be happening: 70% ✔️
  13. S&P 500 2019 >10% growth (from 2506 on Jan 1): 60% ✔️
  14. S&P 500 will be between 2400 and 3100: 80% (80% confidence interval) was 3231
  15. Unemployment rate December 2019 < 6%: 80% ✔️
  16. Unemployment rate December 2019 < 5%: 70% ✔️
  17. WTI Crude Oil price up by 10% (from $45.41): 70% ✔️
  18. Price of Bitcoin in dollars up over the year (Coinbase – 3823 Jan 1): 70% ✔️ was $7163
  19. Price of Bitcoin < $8,000 (does not double): 60% ✔️
  20. Price of Bitcoin > $1900 (does not lose half value): 70% ✔️
  21. Price of Bitcoin < $12,000 (does not triple): 70% ✔️
  22. Drivechain opcodes not soft-forked into Bitcoin: 80% ✔️
  23. No drivechains soft-forked into existence: 99% ✔️
  24. US government does not make Bitcoin ownership or exchange illegal: 95% ✔️
  25. Self-driving cars will not be available this year for general purchase: 95% ✔️
  26. Self-driving cars will not be available this year to purchase / legally operate for < $100k: 99% ✔️
  27. I will not be able to buy trips on self-driving cars from Uber/Lyft/Waymo in a location I am living: 95% ✔️
  28. I will not be able to order groceries on self-driving cars in a location I am living: 90% ✔️
  29. I will not be able to buy a trip on a self-driving car from Uber/Lyft/Waymo without a backup employee in the car anywhere in the US: 80% ✔️ This is tough. You can get self driving cars in Phoenix, but only if you’re part of the Waymo beta and so far they are free, so no “buying”.
  30. The artificial general intelligence alignment problem will not be seen as the most important problem facing humanity: 99% ✔️
  31. Humans will not be in lunar orbit in 2019: 99% ✔️
  32. SpaceX Falcon Heavy rocket will launch again this year: 90% ✔️
  33. SpaceX will bring humans to low earth orbit: 60%
  34. SpaceX will test the “Starship” mock up this year: 70% ✔️ (pretty sure I just meant this giant water tower thing, not a real launch)
  35. Mexican government does not pay for wall: 99% ✔️ (lol)
  36. Border wall construction not complete by end of 2019: 99% ✔️ (some construction occurred, mostly replacing existing wall)
  37. National Debt increases by >$1 trillion (from
    $21,943,897,000,000): 90% ✔️ (was $23.201 trillion on Jan 1 2020)
  38. There will not be a significant decrease in trade barriers between US and China from pre-2017 tariff levels: 90% ✔️
  39. Democratic RCP front runner will not be Bernie Sanders: 80% ✔️ (front runner on Jan 1 was Biden)
  40. Democratic RCP front runner will not be Kamala Harris: 80% ✔️
  41. Democratic RCP front runner will not be Beto O’Rourke: 80% ✔️
  42. Trump not removed from office or resign: 95% ✔️
  43. Trump not impeached: 70% I was not expecting this
  44. No CRISPR edited babies will be born: 80% (it turns out the researcher responsible for the two 2018 CRISPR edited babies had already treated a third unborn child in 2018 when the story broke. Apparently the third baby was born in 2019 if you carefully read Xinhua, so technically this prediction is wrong, although I meant no other researcher would do anything. Remember to properly word your predictions!)
  45. No full year US government budget will be passed (only several months spending): 90% ✔️ (they basically only do continuing resolutions now)
  46. Some tariffs raised: 90% ✔️ (like a bunch)
  47. Trump administration does not file a lawsuit against any news organization for defamation: 90% ✔️
  • I got 4 of 6 predictions correct at 60% confidence
  • I got 7 of 9 predictions correct at 70% confidence
  • I got 7 of 9 predictions correct at 80% confidence
  • I got 8 of 9 predictions correct at 90% confidence
  • I got 7 of 7 predictions correct at 95% confidence
  • I got 6 of 6 predictions correct at 99% confidence

Overall, not bad at all, and we should note that from last year’s grading, my 60% confidence predictions have tended to be overconfident. I only had 6 of those predictions this year, so actually 66% is the closest I could have been to perfect calibration. 70% also ended up being a bit overconfident, but a single additional missed prediction here would have dropped me down to 66% as well. Had I moved one of my correct 70% predictions to 80%, I would have been perfectly calibrated.

Combining this data and data from last year gives:

  • I got 11 of 15 predictions correct (73%) at 60% confidence
  • I got 14 of 20 predictions correct (70%) at 70% confidence
  • I got 14 of 17 predictions correct (82%) at 80% confidence
  • I got 14 of 15 predictions correct (93%) at 90% confidence
  • I got 16 of 17 predictions correct (94%) at 95% confidence
  • I got 10 of 10 predictions correct (100%) at 99% confidence

In 2018, as I noted in the post last year, I should have made some of my 60% predictions at a higher confidence, but other than that, these predictions are remarkably well calibrated if I do say so myself.

I hope to post my 2020 predictions soon.