site banner

The State of Forecasting: Dynamics, Challenges, Hopes

forecasting.substack.com

In short…

  • Forecasting platforms and prediction markets are partially making the pie bigger together, and partially undercutting each other.
  • The forecasting ecosystem adjusted after the loss of plentiful FTX money.
  • Dustin Moskovitz’s foundation (Open Philanthropy) is increasing their presence in the forecasting space, but my sense is that chasing its funding can sometimes be a bad move.
  • As AI systems improve, they become more relevant for judgmental forecasting practice.
  • Betting with real money is still frowned upon by the US powers that be–but the US isn’t willing to institute the oversight regime that would keep people from making bets over the internet in practice.
  • Forecasting hasn’t taken over the world yet, but I’m hoping that as people try out different iterations, someone will find a formula to produce lots of value in a way that scales.
5
Jump in the discussion.

No email address required.

Good writeup I read your article.

The biggest gap I see in prediction markets is a lack of actual large stakes betting without a house spread. It is either play money, or amounts too small to matter, or it is rigged against you by spreads in Vegas.

I would say the biggest betting markets you've missed are the actual financial, governmental and business markets. That is where the rubber hits the road and huge fortunes are gambled on directional bets. If every country that can is investing in nukes then nukes are probably important on the world stage; if every company that can invest in AI is doing so, the AI is probably key to business going forward.

There is another problem with forecasting that I'm sure people have written doctorial dissertations on. If it gets really good and really common then how would one extract any value from it? It would lack the informational asymmetry that might bring you value today if you had the only good predictive model for something important.

The final issue is that if it is common and good then it will alter the very things it is trying to predict. Does predicting it make it true when we trust predictions at a 99.9% confidence ratio? Is there then a rebound effect where they become worthless and you need a meta meta meta meta meta prediction market to determine the accuracy of the prediction market you're trusting to verify the accuracy of prediction market that you're using to make the initial prediction?

Postscriptum for my deterministic peeps out there-

Eventually everyone will have to accept total hard determinism, because after all, what are scientific physical laws but predictions that are right 100% of the time. But until then, there are hurdles to jump over in finding value in this space.

If it gets really good and really common then how would one extract any value from it?

If there's one answer that everyone agrees on and no one betting against each other, all the value would come from whomever posted the question and sponsored it so that anyone who guesses correctly gets proceeds. If people disagree on the answer, then people who're right also get value from whomever's wrong losing their money to the people who are right.

If people disagree on the answer then they need better forecasting.

Sure, but I don't think we're going to ever get to that point anytime soon, because some questions are real hard to forecast. I don't think anyone today knows when say the Israel-Palestine conflict will next have a ceasefire, but that's something a lot of people would like to know. If we get to a point where we can forecast so well everyone agrees on when it's most likely to end, then that's mission accomplished.

If we get to that point then the conflict would never have happened in the first place.