How can I remove these trash items / Why these things don't disappear?

Take them and sell them to a trader or put them in a respawning container (in a random cave somewhere or some such place). Dropping items in a non-respawning cell means they will stay there.

Well, I just clean up them by collecting ALL of them, and drop them to my house's big lake. FYI, this took nearly one hour(real life).

After that, I found that I can't swim into the house's lake anymore(PC crash into desktop when I go into the lake). Too bad :(

Now, I'm moving my garbage items to imperial camp's chest :P (I'm not imperial side, guards are attacking me. I give them a "gift".)

What room is so big to fit all this junk? – Vaishali That room is not mine. Hint: Windhelm, Right-Below's house, 1F

@Falskaar, Go clean your room!

Open the console and aim for your pile of junks, and use this command:

MarkForDelete Permanently deletes the selected item either instantaneously or upon reload. This command is cleaner than disable, as it removes the item completely instead of just hiding it from view.

And repeat for all items in the pile.

A long answer incoming!

The code of Skyrim is such that any object in the game existed always and will exist forever. New objects can be created in-game, items that can be bought, for example. Enchanted weapons and apparel can be destroyed but not the ordinary ones. If any item is dropped anywhere in the world, it'll stay there forever.

However, there's a way to get around this: Using the console that is built into the game. The MarkForDelete command does the job just fine. To open the console, press ~ key(just below Esc).

However I think it's better to keep them in cupboards and chests. The MarkForDelete command is irreversible. Once gone, is gone forever. It sets a permanent flag on the object's code asking the console to never render and scripts to never include it in the game. Chests, cupboards, end tables, etc. are "bottomless". You can store the things in chests and make money out of them when you don't need them.

If you become a Thane in Riften, you'll already have bought a home, probably with a chest or you can store all the things in the cupboard beside your bed in Jorrvaskr.

You may accidentally delete the code for a cell or a room while using MarkForDelete, and it is irreversible. Hence, it is always to wise to keep things. I use multiple cupboards and chests to organize the collected things. I put all the weapons in chest, armor in a closet, etc.