In Banjo-Kazooie, what happens if I run out of lives?

Game Over.

You're presented with a cutscene as if Banjo had failed to rescue Tootie and Grunty has succeeded. Your game is saved as if you had exited the level normally.

After the cutscene is over, you're sent back to the title screen, and can select your play file. Your note score on the level, if you had died with a higher score than your previous best, is saved as your best score. Any moves you learned on that level and any jigsaw pieces you've obtained are saved as well.

You will respawn in the same place as you would as if you had exited normally. Probably the entrance to Grunty's lair.