How can you trust a bettor?
Two friends of mine are trying to convince me that they are good at betting. They infact have a certain profit after n bets. They provided me with all the bets they have made so far: odds, outcome and amount bet.
- How can I check if their wins are statistically significant or it's just variance that brought them up?
- Pretend I want to invest some bucks, how can I deterimine how to divide this amount between them? Should I invest more in the one who has the higher ROI (return on investment)?
- Is there a way to suppose or prove that if they have been winning (if there is statistical significance) they will continue winning at the same rate? Is there a way to monitor their future performances?
Say you have 32 friends and after betting on coin tosses five times (always betting all their money), one of them approaches you and says: "See? I have prophetical powers. A predicted the outcome of a fair coin five times in a row and made 320\$ from my original 10\$ - you can't explain that with sheer luck. And as you can see, it's really not easy: All those other 31 folks lost their bet at some point and are bankrupt now. Bet all your money with me and you're sure to double it. You can't lose if you do as I do..."
Please don't fall for that fallacy.
Without knowing more specifics about the game itself, it is very difficult to establish a correlation between data and confidence. but I will try to give you an upper bound for a broad class of games. That is games with the condition that there is some factor of your bankroll that you cannot make more than in any given iteration. (there are many games that do not truly satisfy this condition),
more specifically, if we assume that your friend is a martingale (that the expected value between his bets doesn't change, and is always zero), we can then invoke the Martingale Central Limit Theorem on the logarithm of the values in his data.
This link give some bounds on the rate of convergence of a martingales bankroll to a normally distributed bankroll.
So for "sufficiently large N (dependent on the maximum factor by which we can lose or make money)" our log-scale data is approximately normal, and our measured standard deviation is close to the actual deviation, and that allows us get a probability that your friend is a martingale based on the logarithm of his profits.
this is insufficient though, because you have to consider your own selection bias. This is generally very hard to correct for, but you may always assume that your sample size is the population of earth, and you are simply looking at the most extreme case. More likely though, you aren't considering people that you don't know. and you only have to consider a sample size of all of your friends who play such games, but there is a lot to consider in order to make that claim. If you want to determine whether the wealthiest poker player is/isn't a martingale, you should treat your sample size as the total number of poker players since the choice of "the best poker player in existence" is not a uniformly random one, in fact if the best poker player in existence is just lucky, it is probable that he will be at least 5 standard deviations from the mean (log scale). 6 standard deviations is 1 in a billion, and more than this is evidence of non-martingaleness no matter how large your selection bias is.
but ultimately you need a big N, and a big profit. just how big depends on the game.
A very naive way would be to simply compute the mean return per bet, compute the standard error of that mean, and compute confidence intervals with the appropriate t-distribution thresholds. If zero is outside the interval, their return is "significant". That is, with your confidence level you would say they are better than chance (indeed, if odds are not fair, even with a zero mean return they might be better than chance). But be mindful, even if at the 5% confidence level they appear to have beaten the odds, 5% of bettors with zero expected return will be favorably tested like this.
If you are risk neutral (want to maximize expected value), then by definition you choose the buddy with the highest ROI (=expected return). However, if you are risk averse, then you would probably do what most empirical finance people do, and compute some kind of risk adjusted return. This just controls for the fact that it is easier to get a high ROI by taking very risky bets. (Why? Suppose a nonrisky and risky asset give the same expected return, then no risk averse agent would take the risky choice. Hence, the risky asset must yield more return for at least some agents to be willing to take it).
Neither financial markets, nor betting markets, are perfectly efficient in the sense that price or odds reflect the true value/odds. In betting markets, the so called favorite-longshot bias is found consistently in many betting markets. It holds that favorites are underbet, while longshots are overbet. If it weren't for fees, you could actually exploit the bias by always betting on the favorite, thus gaining a positive expected return. (Until everybody recognizes this, at which point there will be a reverse favorite-longshot bias, etc...). In any case, there may be strategies that consistently outperform the crowd. At the very least insider knowledge would be such a strategy. Still, it is of course unlikely that someone without such a strategy will consistently earn positive returns. Thus: ask them for their strategy. If it is "a gut feeling" that determines their bets, I would rather buy stocks than invest my money with them.