Why are USB ports oriented differently on different devices?

For example on one machine the ports have the empty side on the left and on another it's on the right. Why don't manufacturers make this consistent?

There are two simple reasons. First, there is no defined community standard. Each manufacturer makes their devices however they see fit. So, ports are put wherever they desire.

The second reason is because designers of devices, especially small ones, must first lay out their circuit boards and other internal components so that they can squeeze as much hardware into the smallest amount of space. Ports are a major part of that process, but there is also a bit of aesthetic trade-off when it comes to laying out all the hardware in the smallest footprint possible while also remaining functional.

Most USB receptacles are mounted to a circuit board, and in general, the "empty" space in the connector is on the side towards the board. This seems to be the de-facto standard construction (if not an actual standard) of PC-board-mount USB connectors, just like D-SUB connectors always have the narrow edge toward the board, and dual PS2 connectors always have the keyboard connector towards the board. (Hint: this is useful to know when blindly connecting cords behind a desk.)

When a USB cord is plugged in, the USB icon will be "on top" of the connector, if the board is toward the bottom. Likewise, the indicator LED on a USB thumb drive will be "on top."

The difficulty is that some manufacturers (AHEMDellAHEM) seem to always orient the assembly so the circuit board is above the connector when the PC is in normal orientation. This means that the device plugged into the USB connector will be upside down, so you cannot view the indicator LED.

What I find even more irritating is that USB connectors are allow a limited amount of space for the plug; most thumb drives and many cables are unnecessarily fat so plugging them in blocks other, adjacent USB ports.

As already pointed out, specs are not always followed, especially with something as ubiquitous as USB.

But an important point is also that specs are not always clear and precise, and are left open to interpretation.

This means that you can't rely on anything.

However, I presume by "empty side" you're referring the following:

This is actually somewhat covered by the USB spec. Section 5.3.1.1 of the USB 3.1 spec has the following image:

... and Section 5.3.1.2 has the following images:

This implies that to be "fully compliant", the connector must be mounted with the "empty side" (as you put it) at the bottom... in relation to the PCB's top/bottom.

I say implies because I can't see any specific claim that to be compliant, you must use these orientations, and in fact the USB 2.0 spec omits the example PCB footprints entirely.

To be honest, every connector I can think of follows the rule above, excluding only some Micro USB connectors which have been mounted upside down, notably by Samsung on phones including the S4.

Unfortunately this then leads you on to ask "what is the top-side of the PCB?" - a question that from the outside of a product can be difficult to answer.

There are then other standards at play - PC form factor standards such as ATX vs. BTX lead to products which have the motherboard mounted on the other side and add-in cards mounted upside down when compared to each other. This leads not only to the connectors being "upside down", but also the motherboard being mounted on the "wrong side".

These things aren't consistent because there are too many things at play here.

Take the S4 example I gave above... perhaps the engineers working on that PCB considered the "top side" to be the side facing away from the user - it would make sense, as lots of the components are mounted on that side to leave a flush bottom side against the display module.

I believe the USB symbol on the plug and next to the receptacle are intended to indicate which way the plug has to face to be inserted into the receptacle.

A USB spec says

The USB Icon on the plugs provides tactile feedback making it easy to obtain proper orientation.

However I couldn't find anything about receptacle orientation and, in practice, not all receptacles are marked with the symbol.

Perhaps you would like a flipper plug