How does Skyrim determine if a dungeon is "cleared"

Solution 1:

Most dungeons are cleared when you have killed all important inhabitants, and as a result of doing that, you have seen the whole dungeon. The game doesn't force you to loot stuff (e.g., Werewolf form can't loot) so it comes down to just killing; you don't have to do your quest.

A victorious sound will play once you clear the dungeon, and the ambient sound will stop.

If you don't kill all important inhabitants, inhabitants might respawn and the dungeon will not be seen as cleared. Another occasion where inhabitants respawn in a dungeon is when you start a new quest that involves that dungeon; however, it won't change the cleared state of a dungeon.

There are reports that some dungeons don't get cleared, or are cleared a different way. Some report that whatever they try to do, the dungeon doesn't get cleared; this might be because they still have a quest to do in the dungeon. Other report that a dungeon gets cleared after looting a chest or learning a new dragon shout word. Please note that not every dungeon has a word wall.

Solution 2:

The other answer is actually incorrect.

With the Dungeon bounty given from WhiteRun (Valheim Towers), I went to the mountain top adjacent to the Keep, and from there was able to kill the chief bandit in that keep (I have mad archery skills). Once I did that, the dungeon showed as 'cleared' on my screen, and I was able to collect the bounty.

However, I didn't kill all the inhabitants. I killed 3 of them (there are six inhabitants to that keep, plus the outdoors guard on the road if you approach the keep from the road like you're supposed to).

The dungeon is considered cleared once you kill the 'boss' for that dungeon, whether or not you kill all the inhabitants.