PSU failing or Mainboard failing?

I am having some troubles lately powering on my desktop workstation.

While starting up the PC after being off for hours (usually at least 8 hours) it randomly fails to do so.

What happens is that :

1. I press the power button; nothing happens
2. I can hear a moderate buzzing noise at the back of the PC (near the PSU); but I can't say for sure that it's not from the mainboard.
3. If I insist pressing the power button a few times in 1-2 minutes it'll start

Another route would be that instead of (3) I will plug off the power cable from the PSU and wait for 30 seconds. Then I will press the power on and keep it for 30-60 seconds (I had some success at notebooks with a similar approach). Then I will plug back the cable in the PSU, press only once the power button and it will start normally.

Also while running normally I keep hearing some low buzzing which seems to be fan-RPM-related (i.e. when processing images or doing CPU intensive work).

What should I look into?

UPDATE

It's getting worse. It took more than 10 retries today and almost 20 minutes to start the computer. I tried the paperclip trick and the PSU behaves perfectly. I managed to start the computer like so : I pressed the on-button a few times and then left the PC in a pre-startup state (the fans were working the buzzing noise was strong and I went to eat. I thought I won't lit the house on fire so fast and without smelling. Back, after 10-15 min the computer booted up! Discussed with a fellow at Intel and he told me the capacitors on the mainboard are probably a bit shot. If they are shot, he said, it should start up warm perfectly. So I did restart it, warm, a few times (5 sec cooldown and then 40 sec cooldown and it started up perfectly). I can either replace the capacitors on the mainboard (doesn't sound worth it or replace the mainboard (this one sucks too :)) )

FINAL INFO : It was the PSU after all. Although it was powering the IDEs and SATAs the Mainboard power module was failing. I bought another mainboard just to find out that this wasn't the cause. Now I'll have to return it somehow. The spare PSU is now in the computer and doing well.. Although larger (500W), it's like a plane taking off.. I need a better one.

Your power supply is probably broken,

high pitched noise is usually one of best signs that could indicate that PSU is going to broke or already broken.

Pressing power button many times also speaks toward broken PSU case as when you press and release button it will discharge capasitors and then immediately charge them again, doing this many times could still make almost broken capasitor to work once again for some moments.

If you want to check PSU first

then go ahead, here is few things that you may want to look for after removing protective covers from PSU:

Check if there is burn marks anywhere, check both sides of circuit board.
Check for broken / blown capasitors (all three is going to blow up):

As you can clearly see its usually (but not always) pretty easy to visually determine broken capasitors. Capasitor top cover should be flat, yes flat and that flat is completely flat.

If you have some experience (or someone with experience) with multimeter then you could use it to test some parts or if you happen to have oscilloscope then you can use it to diagnose whole PSU and/or defective parts.

Actually, I'm trying to tell that your PSU is almost dead

at least every aspect in description of your problem is telling that some of power circuit capasitors are failing. There is also other power regulation circuits in mother board, mainly supplying power for CPU but usually if they are broken it may start up fans and maybe flash some lights and then immediately go down again.

Check your bios battery. When you your problem happens, try instead to remove power, and press power button(without power) for 5/7 seconds. If it solves the same way as you solution does, its probably a Moterboard bug/Battery discharged.

You may also check if your PSU is supplying enough power here.

Did you already tried to monitor temperatures? Check lz_sensors on linux or Speedfan on windows, and see if that "fan issue" is not related to a overheating problem.