Term for main part of desktop computer
Solution 1:
That 'part' is the computer. The other devices you describe are peripherals connected to it.
If the form factor is a traditional, vertical case, as pictured, 'tower' is often used as well. And of course, if you're looking for a term specifically for the housing, well, that'd be the 'case'.
Solution 2:
The outer box can be called any of the following:
- computer case
- computer chassis
- tower
- system unit
- base unit
Though most of the times, people would refer it by just chassis or 'CPU' which is technically incorrect but widely accepted.