How can I mathematically model and analyze an incremental game like Cookie Clicker?

Suppose that you have a sequence of purchases. Prove that the optimum time to buy each element in the sequence is as soon as you can afford it.

With that lemma in the bank, you can consider a set of states where a state is a tuple of the number of cursors, grannies, farms, cursor upgrades, and granny upgrades purchased. Form a digraph where each state has edges to the state with one more cursor, the state with one more granny, etc. Therefore each state has up to five edges. Add a new vertex "Target" and add an edge from each state to the target. Give the edges weights as the cost of the upgrade divided by the production rate of the source state.

Then you just need to do a standard shortest path search (Dijkstra's algorithm, A*, etc) in the graph. This should be much much much more efficient than the brute force you describe.