What does unplugging my DSL modem do?
What is happening here, is that your modem is slowly desynchronizing from the ISP. This desynchronization causes packets you are sending and receiving to become corrupt, requiring them to be resent, and thus, lowering your speed.
To fix this issue easier, I suggest a simple solution. Get one of those outlet timers that people use for when they go on vacation and want to have lights turned on. Then set it to power the router from, say, 4:16am to 4:15am, so that it is powercycled during a time you're not going to be using it in the first place.