Which books are valuable to retain or are quest goals?

Solution 1:

You will not be able to drop unique books that are required for specific quests, so don't worry about that.

There are some books that are not unique, such as the Last Scabbard of Akrash, which you can drop or sell if the quest that requires them is not active. Though, even if you do, once the quest becomes active you'll get a quest marker to another copy of it somewhere in the world.

Additionally, there are quests that will be bugged if you already have the book before you're given the quest, such as Fetch Me That Book!

Therefore, it's probably best if you don't try to hold on to books. If you can't sell it, don't, if you can, go for it.

Solution 2:

Captain Aldis at Castle Dour in Solitude may ask for a copy of The Mirror. If you need a place to store stuff, there are several with safe storage you can use without having to buy or build a house.

If you join the College of Winterhold, you get a cubicle in the Hall of Attainment with several safe containers (barrels there aren't safe, though). If you start the quest House of Horrors in Markarth (talk to Vigilant Tyranus outside the Abandoned House) you get can get access to a property with more storage than any house you can buy in vanilla Skyrim with apprentice robes and hood to boot (not sure about the houses you can build in the Hearthfire DLC, though. They might have more storage options depending on the wings you build and how you furnish them). You can get a free bed and chest in the Ragged Flagon Cistern by joining the Thieves Guild, though I don't feel comfortable leaving anything there with a value to weight ratio of 10 or more (many Hearthfire building materials are okay though).

Solution 3:

You cannot lose quest items in Skyrim. You also cannot preemptively acquire said item and break the quest.

If a quest requires certain items, it will simply spawn said item at quest start. If you happen to have a different source of it stashed already (say a common Mammoth Tusk), you will simply just retrieve it from your storage instead of following the quest markers.

Assuming you picked up a quest related book, the game will still spawn a new copy of the quest item when the quest start. You can do anything with your existing copies without any consequence.