How to calculate odds in finance
As noted in David's comment, we cannot say which option is better due to the risk preference of the individual.
One can argue that we should choose the one with the higher expected value.
If we assume that we initially invested $1$ unit of money, the expected value would be $$P(\text{10 months consecutive success}) \cdot (1 + \text{monthly interest rate})^{10} + P(\text{fail somewhere in the 10 months}) \cdot 1.$$
For the first option, we get $$0.8^{10} \cdot (1 + 0.05)^{10} + (1 - 0.8^{10}) \cdot 1 \approx 1.0675,$$ while for the second option, we get $$0.95^{10} \cdot (1 + 0.015)^{10} + (1 - 0.95^{10}) \cdot 1 \approx 1.0961.$$
With that, you can argue that the second option is better.
However, one can argue that the above is not reasonable.
Consider the following scenario. I offer you two options:
- I give you one billion US dollars ($10^9$ USD).
- I throw a fair coin. If it's heads, then I give you ten billion US dollars ($10^{10}$ USD). Otherwise, I give you just $1$ USD.
Realistically speaking, which will you choose?
If we compute the expected values, the first option gives $$100\% \cdot 10^9 = 10^9,$$ while the second option gives $$50\% \cdot 10^{10} + 50\% \cdot 1 = 5 \cdot 10^9 + 0.5.$$
So, it seems the second option is better.
Yet, I believe that most people will choose the first option. Even though $10^9$ USD is smaller, most people can live comfortably with $10^9$ USD in their bank account. So, it's reasonable to take the guaranteed $10^9$ USD rather than the uncertain $10^{10}$ USD.
This is where utility function comes into play. Instead of computing the expected value, we instead compute the expected utility value. Roughly speaking, you define the 'worthiness' of a money using a utility function $U(x)$, where $x$ is the amount of money.
One common example is $U(x) = \log x$. This is a reasonable choice, since one can argue the worthiness of money follows a logarithmic scale, i.e., it diminishes in value. For instance, give a billionaire and a poor person one million USD each; the poor person will likely be much happier than the billionaire.
If we set $U(x) = \log_{10} x$, the expected utility value of the second option becomes $$50\% \cdot U(10^{10}) + 50\% \cdot U(1) = 50\% \cdot 10 + 50\% \cdot 0 = 5,$$ while the expected utility value of the first option becomes $$100\% \cdot U(10^9) = 9.$$ So, the first option now seems better!
Coming back to your original question, my answer is: it depends on the utility function.
Different individuals have different utility functions, and it affects which choice is better for those individuals. We see this a lot in real life, e.g., some people play risky and invest in stocks, while others just place their money in a savings account with a smaller interest rate but less risk. This is particularly similar to your question too.
Is it better to choose a relatively high risk option that has high reward, or choose the one that is less risky but less rewarding? It depends.
Disclaimer: I am not saying that this expected utility concept is the 'best' way to judge the worthiness of a resource. This is just simply one common way to do so — at least, in my basic financial mathematics classes.
P.S. We can try finding the expected utility of the two original options you gave using several utility functions. For example, with $U(x) = \log_{10}(x)$, the second option is still better, while with $U(x) = x^3$, the first option is better.
P.P.S. The function $U(x) = \log_{10}(x)$ isn't exactly a good choice for a utility function, since it's undefined when $x$ is not positive. I chose this function because it's relatively easy to use for the computations.
Edit:
The answer above assumed you will lose all interest from all months when you fail once. If you only lose the current month's interest, then the expected value would be different.
If $p$ is the probability of a success for a month, then the probability of exactly $k$ successes among $10$ months is $\binom{10}{k} p^k (1-p)^{10-k}$. With these number of successes, the investment gives you $I \cdot (1+r)^k$, where $I$ is the initial investment value and $r$ is the monthly interest rate.
Hence, the expected value would be $$\sum_{k=0}^{10} I(1+r)^k \binom{10}{k} p^k (1-p)^{10-k}.$$ With this interpretation, if we set $I = 100$, the first and the second option gives approximately $148.02443$ and $115.19939$ respectively.