If a soldier is behind both high and low cover, which one counts?

Consider the situation depicted below. An XCOM soldier (x) is next to both high (H) and low (L) cover. An enemy (e) is positioned such that the low cover is what prevents the soldier from being flanked.

...e....
........
........
...LL...
...xH...

When the enemy shoots at the soldier, does the game use the low or the high cover to calculate its chance to hit?

If the game has to choose between high and low cover, it will choose high. However, in your example the only choice is "low".

Please see this image describing how cover works in the game - the cover system is actually fairly complicated. If the enemy were 2 spaces to the right in your example, the game would have to choose between "high" and "low" and would thus choose "high".

It is based off of the cover that is between the enemy and your soldier. Horizontally and Vertically. That is why flanking is important, to exploit lower or no cover.

In your scenario the enemy has a line of sight that goes over low cover, thus is will calculate low cover into the accuracy.

If an enemy ran to his right, it would account for the high cover.

If it was on an angle, and both can be account for, it would choose the higher cover as mentioned by @Blue Raja.