Will my game hold a save even without a battery as long as the system's still turned on?

Yes, if your console is powered on.

As far as I experienced and as the internet tells, the save game will not be lost, as long as the cardridge is in your GameBoy AND it is powered on (like being on the title screen of the game). Only plugging a loading cable in it will not do this work.

As reference I take this guide, where the writer replaces the battery while having the cardridge plugged into his GameBoy, powered on, and not losing his save state.

If you're really that paranoid about losing your save state, I'd recommend you to buy two accessoiries: The "Mega Memory Backup Cartridge" and the "GB USB Smart Card 64M". With these two tools you can save your save states onto a memory card and then archive it to your PC. You also would have the advantage to play your save state onto several emulators and then switching it back to your original game cardridge, if you want. If you need further information, this guide is about backupping a GameBoy game to your PC.