How do you know when trying to find a proof/counterexample isn't worth it?

Solution 1:

Congratulations. You're learning how real mathematical research works.

You alternate between believing and doubting. When you are stuck on one you suspect the other and switch modes.

Over time you sense how to choose problems that are goldilocks for you: hard enough to be interesting, not so hard as to be impossible. But it takes time to develop that intuition.

One suggestion for now: talk to people - fellow students, faculty you know. Sometimes explaining your conjecture to a new listener will trigger an insight. Perhaps try explaining to your rubber duck. Or ask about a fragment of your puzzle here on stackexchange.

Finally: you might be able to write an excellent final year project in which you present evidence showing that your conjecture is interesting and difficult.

Solution 2:

All the possibilities that you were mentioned can be true based on the problems that occur. In mathematics, some problems were computationally out of reach, as you mentioned, due to computational technology. It might give you the counter-example you want via computational technology in the future, but not now. Also, it could be why the proof is beyond your mathematical abilities, but this does not mean you need to give up. The proof can also be actually within your reach, but you still cannot find the pattern. My suggestion is to find another way to visualize your problem to seek proof or a counter-example. The $way$ that I mentioned does not guarantee that you will find the solution you want, but in maths, a lot of trial and error (just like Terence Tao mentioned in this video) is needed to solve a specific problem. Thank you. Never give up.