Is there any way to completely empty your inventory?

These all seem like quest items, which can't be removed without console commands or by completing their related quest. That said, while they should be removed when you complete their related quests, I found myself stuck with some of these items after the quests were complete with no way to drop them as well.

Searching the UESP for the items may help you understand which quests these items belong to. For instance the Elder Scroll can be sold to the librarian at the College of Winterhold (Urag gro-Shub). The Gauldur Amulet Fragment and the Ivory Dragon Claw are part of the quest Forbidden Legend.

If this sounds like a lot of work, and you're willing to deal with your cluttered inventory, remember that quest items weigh nothing (despite their listed weight).