Do I need to replace my CPU or Motherboard, PC not powering on
Solution 1:
You haven't specified if this is a new build, or if this is your existing system, but here are a few non obvious things to check:
Before I offer some suggestions, here is why putting your motherboard in an oven is a REALLY BAD IDEA
Capacitors are full of electrolytes which react to heat just like water. Putting a capacitor into an oven could figuratively boil off all the liquid inside. You do not want to heat up capacitors.
Ever heard of xbox's getting the red ring of death, and that you can fix it with the 'towel trick'? Bad idea.
https://electronics.stackexchange.com/questions/18866/can-capacitors-be-exposed-to-temperatures-above-the-max-operating-temp
4 Pin Connector Since removing the 4 pin connector causes it to work (as far as you have stated), then this is likely a power problem. Two very important things to consider.
Accidentally using the power supply 4 pin
Many 24 pin power connectors are also 20 pin connectors. Those 4 extra pins will fit into the cpu 4 pin header, but they have the wrong pinout. While this is a very slim chance it is the answer, I have seen this before from new system builders.
Accidentally using a GPU 4 pin connector
I've also seen 6 pin and 8 pin gpu headers plugged into the cpu 4 pin headers.
Bent CPU pin
A bent pin on the cpu can also cause the problems you are describing.
Bad Capacitors
If this isn't a new build, check to see if you have any blown capacitors.
http://en.wikipedia.org/wiki/Capacitor_plague
Modular Power Supplies
If you replaced your power supply and it happens to be modular, make sure you use the cables that come with the PSU. There is no standardization between manufactures for modular pinouts. You can kill your components if you don't.
See this super user question How do I interpret the Asus motherboard LED and beep patterns?
Is the CPU bad?
I've repaired computers for 15 years. I've only ever seen 2 cpu's 'go bad'. It is almost always the power supply then the motherboard. I suggest that you buy a new motherboard.