Can a new battery go bad if unused
Solution 1:
Li-ions self-discharge, although very slowly. If you leave it for a very long time (probably many months) it could discharge so far that its "electronic fuse" will open. Once that happens, it's a paperweight.
It probably came to you with a charge in the 40 to 60% range, as that is supposedly the most stable for storage.
If you're not going to use it, I would check it every month or so. If it's dropped below about 20%, then charge it to bring it back up to 60% or so before storing it again.
If it were me, I'd switch to the new one and keep the old as a "just in case" spare. The new one should give you better runtime.
Solution 2:
In my opinion - you should swap these batteries once in a month and discharge battery to 40-60% before storage.
Lithium Ion batteries "go bad" when they are stored in discharged state.
It is all about battery voltage. If voltage is too low - undesireable chemical reactions will happen and battery will degrade.
If battery is not empty and not used for long time - it will be fine. However batteries are not perfect and they slowly discharge without load. If you leave full battery for few months - it may self-discharge and when voltage drop to "almost empty voltage" - it will start degrading and loosing capacity.
If it is stored near empty state - it will degrade and loose capacity.
More about battery storage (including Li-Ion):
BU-702: How to Store Batteries (at batteryuniversity.com)
(added later: batteryuniversity.com is a website created by some company, do not consider this as 100% reliable source of scientific informations)
There are many opinions about "ideal" charge for battery storage. Some people say 40% is the best, some people say 60%. In article linked above:
Lithium-ion must be stored in a charged state, ideally 40 percent. This assures that the battery will not drop below 2.50V/cell with self-discharge and fall asleep.