Fermi Poker – Gambling for quants and data scientists
This article is originally published at https://cartesianfaith.com
Fermi problems are well known for honing your ability to estimate quantities that are difficult (or impossible) to measure. Named after physicist Enrico Fermi, these eponymous problems can be used for more than estimating the strength of an atomic bomb and are great for any quantitative field, including business. These problems teach you to think carefully about your assumptions, resulting in better estimates. They also provide a frame of reference to sanity check actual data. But how certain should you be about this estimate? While the error tends towards a binomial distribution, Fermi problems can’t teach you to quantify the uncertainty in your estimates. But Fermi poker can.
The exemplary Fermi problem is “how many piano tuners are there in Chicago?”
I’ve used Fermi problems during interviews and also as exercises for teams I manage. They are very effective in understanding people’s thought process and what types of assumptions they make. It also helps identify their level of self-confidence and self-awareness. As a manager it’s a great tool. Lately I’ve wanted to go further and help my staff measure whether their confidence in their estimates is justified. This boils down to knowing how good your underlying assumptions are. The challenge is how to train people to accurately quantify this concept and then improve their estimates of this quantity.
I developed Fermi poker as an extension of Fermi problems to teach people how to quantify their certainty around their assumptions. It works by making the margin of error front and center in computing the payout of a poker round. Initial tests (i.e. games) show that it is effective in helping people not only estimate their uncertainty in their assumptions but also in internalizing orders of magnitude.
Rules of play
Fermi poker needs a minimum of two players and can also work in a group setting. Here’s how it works for two players. Start with a base ante, say x = $0.25. Players alternate being on offense (A) or defense (B).
A Fermi problem is randomly selected. The most appropriate Fermi problems are the ones where the answer is a power of 10 (see www.fermiquestions.com). Raw quantities (how many toothpicks does it take to cover a basebakl diamond) or ratios (how many times more expensive is an F-16 fighter plane versus a 5 year old Harley Davidson) work well. Player A has 150 seconds to make an estimate and determine a margin of error. Once the time has elapsed, Player A either states her answer and margin or forfeits the ante.
Player B has 15 seconds to accept or forfeit the ante. Player B can optionally raise the ante. Player A can raise, call, or fold. Continue until call or fold.
If the game is on, the answer is given. If the answer is within the margin of error, then Player A wins, and Player B must pay the payout less the ante. In other words, a 1x payout is just the ante, while a 3x payout requires an additional 2x.
The payout is loosely based on odds. The idea is that a lower margin of error is related to higher odds of losing. In the case of 0 margin of error, player A believes their guess is exactly right and has only one value that is in the money. Player B on the other hand has infinitely many chances to win. The risk is thus asymmetric between Player A and Player B, which is reflected in the payout scheme. In the 0 margin case, Player B can only win 1x but can lose 8x, while Player A can win 8x but only has 1x at risk.
The recommended payout table is below. The loser pays the winner less the ante. Note that the payouts are intentionally not normalized. This is because there is no house, so both players need to have variation in their payouts.
| Margin of error | A wins | B wins |
|---|---|---|
| +-0 | 8 | 1 |
| +-1 | 5 | 2 |
| +-2 | 2 | 5 |
| +-3 | 1 | 8 |
Multi player variant
For more than two players, these are the recommended rules. Each player has 150 seconds to calculate an estimate and decide a margin of error. After the 150 seconds has elapsed, each player announces their margin of error but keeps their estimate a secret. (A different player should start this each round since the last player has the most information.) The player with the smallest margin of error is Player A. If more than one person has the smallest margin of error, the tie must be broken. Ties are broken by increasing the Player B payout (in 0.5 increments). This caps upside payout but increases downside risk. More confident players will be willing to risk a larger loss for the same payout, since their expected value should be based primarily on their alpha (secret information).
Once Player A is chosen, she announces her estimate. An optional betting round then starts. Players can either fold, call, or raise the bet. As the stake increases, each player must decide how confident they are in her estimate vis a vis the offensive estimate. If the estimates are within the stated margin of error, then defense should fold. Otherwise she should call or raise depending on how confident she is of her estimate. On the other hand, Player A can fold if they aren’t willing to risk more than the current payout scheme. In this case, they pay what they’ve committed to each defensive player. (Any money derived from folding can be split equally amongst the winners of the round.)
Once the betting has finished, the answer is revealed and payouts computed. A convenient method is to tabulate each round and record the amount won or lost per player. At the end of the game, anyone with a negative balance pays into the pot, while those with a positive balance collect from the pot.
Results and observations
Anecdotal evidence shows that after only a few rounds, participants quickly learn to quantify the uncertainty in their estimates. What’s interesting is that some players focus on tying their uncertainty to the odds, while others attempt to determine a range of possible values to compute a margin of error. The time required to make calculations also decreases. Initially 150 seconds doesn’t seem like enough time, but after a few rounds, most people have plenty of time. This phenomenon can be partly explained by players learning how to quickly make calculations using exponents.
If you play, please write in the comments any feedback/observations. Also, if you have rule variations, feel free to suggest them!
Thanks for visiting r-craft.org
This article is originally published at https://cartesianfaith.com
Please visit source website for post related comments.
