Is a clean install really better than an upgrade?

What makes a clean install of an OS better than just upgrading? (In the context of MS Windows) Is it all in our heads?

Actually, upgrading is always better from a user's point of view. Who wants to keep starting from scratch, restoring one's programs, documents etc every time one gets a new computer or OS version...

That said, you probably used the word "better" in the sense "more stable" here in this context :)

Now the reality with Windows is that a clean install is very often less problematic than an upgrade. In my experience, I have always had trouble with upgrades, not one was successful for me so far, I've always had to basically redo a clean install after an upgrade. What I call an unsuccessful upgrade is one that I have to scratch and reinstall from scratch within a month because Windows proves to crash too often after the upgrade, and I couldn't find a solution to the problem within that time... (an unstable machine for me is a machine that can't run for at least 2 weeks without crashing).

I've done about 6 attempts at upgrades so far (since ~1998). My take on it is: you run into problems with upgrades if you install/uninstall/reinstall a lot of programs (keeping up with new versions of those programs etc). If on the other hand you don't do much with your programs besides using them (creating/editing documents etc), then chances are that an upgrade will go well for you.

Clean installs are more difficult (you need to reinstall software), but will result in a faster and more stable system when you are done. This is true on any OS, but there are some operating system differences. On Windows, particularly if you install and uninstall software frequently, it is very important to do a clean install periodically as the registry gets full of unused entries, which slows down performance and affects reliability. There are also OS authentication issues to consider if you are reinstalling from a previously used Windows CD (not a problem if you are using your original restore CD or upgrading to a new OS).

On a Mac, it is generally not needed to do a clean OS install as often, although I still prefer to do the clean install with each new OS as I install and uninstall software constantly. You do have the advantage in that you can drag applications over from a backup and don't have to reinstall apps.

yes, if your data is seperated and have backups of data is available, clean install is always a better option. In my experience, upgrade to vista failed because of some constraints but clean intall worked fine

The reason Windows has this reputation is rooted in history: Windows has no centralised way to keep track of all the files and data it thinks it looks after. Contrast to the vast majority of Linux distributions: at the core of their package system is a database of every file it has installed and what package it belongs to. There used to be stories of people installing very early versions of Debian and upgrading through the versions to the current one, with the end-effect almost identical to a clean install. It is the lack of this in Windows that contributes to things like "DLL hell" where mulitple applications might install different versions of a DLL into the system directory, to the detriment of each other (but it is true that recent versions of Windows have ways of addressing this).

Windows also does not manage its Registry all that well. The simple effect of that is that the raw size of the Registry grows over time, as it contains empty space and entries to software that is no longer installed or has been moved or upgraded (installers of applications can also be bad at keeping their own registry entries up to date). These are the main things that registry cleaners fix.

Because of these twin problems, a clean install is what is called "a known starting point". An upgrade has to be a lot more defensive about what it finds on the disk and inevitably makes assumptions that are not correct. Some of these a new version can find and fix (even after the same version made the mistake), some are relatively harmless, but others completely mess up the system. The other half of this problem is that Windows to some extent tries to second-guess what you're really doing. This is why you can feel like you're going round in circles trying to fix some oddity like your microphone not working.