I will keep tracking the accuracy of AI predictions and give an update once rounds 2 and 3 of the group stage are over.

FIN/

Oh.....and I forgot to say #FIFAiscorrupt

Basically the predictions of the fancy AI model trained on >100,000 datapoints using Google Cloud's Vertex AI does not outperform a simple model 🤷

Summary: AI can be used for many things, but be wary of the hype! It is important to always compare AIs to benchmark models.

Much simpler models, can perform equally well. They are often much more transparent, easier to train, affordable, and emit less CO2!

5/

Let's put that into context.
To do so we need a reference model.
The simplest I can think of is using FIFA's own ranking of teams prior to the world cup (from 6 Oct 2022).

https://www.fifa.com/fifa-world-ranking/men

Using this we build a simple model that always predicts the highest ranked team to win (we dont care about draws)

How well does this model perform?

Well, it also achieves a 56.25% accuracy 🤷

In fact, the predictions are exactly the same!

#AI #WorldcupQatar2022

4/

The predictions look like this 👇
It shows the probability of which team is most likely to win. Since there can only be one winner we select the most likely team to win as "the prediction"

E.g. for this match we say the model has picked the Netherlands as winner

So how good is the model?

Well out of the 16 matches in the first group round the model correctly picked the winner 56.25% of the time.

Not a great accuracy ❌⚽️

3/

With the first part of the group stage over, its time to take a look at the predictions

Yet, its surprisingly hard to find predictions for individual matches, most models look like this👇 and show predicted group standings and later predictions for the knock-out matches

We will focus on the knock-out matches later, however, i did manage to find individual match predictions from @aljazeera's AI model

https://www.aljazeera.com/wc2022experience/

2/

There's a lot of HYPE around AI, and recently people are using it to predict who wins the FIFA 2022 world cup ⚽️🏟

These models have gotten a lot of press, even @Nature (uncritically) covered them nature.com/articles/d4158…

But how good are these models? 👇🧵
TLDR not great

1/