"ou" versus "o" in spelling words like "color"/"colour"

You're correct that "o" is US and "ou" is non-US. It'd be considered bad style to switch between them in the same text. Generally, you should just choose one style and use it consistently, and you will be understood. I've heard a rule that if you're writing for a mostly American audience, you should use the American spelling, and otherwise use the international forms, but that may not even be necessary.

One place that mixing styles is allowed is when quoting verbatim from text, or in technical literature where spellings must be retained exactly:

I asked him what colour he wanted, and he said "I'm no good at picking colors".

The color: #ffffff; property indicates a text colour of white.

As several people have stated, the 'o' form is American. You should pick a format and stick with it. If you write 'o' your writing will be perceived as American. Otherwise it will be perceived as non-American. Does that matter to you, and do you think it matters to your audience? If so, pick the appropriate style that best suits you or your writing.

As for trends, one thing I've noticed as a Canadian is that many computer programs only recognize US and UK English and not Canadian or Australian or New Zealand or.... Also the default for lots of software tends to be US. So if you want to pick the lazy, pragmatic route, the American spelling will probably be more convenient.

So, first, yes, the o variations are preferred in the US while the ou variations are preferred in Commonwealth realms (at least, the ones I'm familiar with. Any counterexamples would be welcome).

As to the question of whether it matters, it depends greatly on the type of communication, the purpose, the reader, etc. While mixing and matching shouldn't hurt comprehension, as a matter of style, I would suggest consistency within a text in any formal or business-related writing, especially the color / colour type. In informal emails, forum posts, etc. I wouldn't spend a ton of time worrying with it.

As far as which you should choose, I would keep the following in mind: if you choose the reader's preferred spelling, it will likely not have any particular effect on them as they read it. However, if you choose the UK spelling for a US audience or vice versa, it will potentially be noticed as an explicit difference by the reader. This can have a couple of side-effects: it can take them out of the flow of the material, breaking up a nicely flowing sentence and distracting slightly from the content; it can also inform the reader of your background. While neither of these may matter if you're posting a question on a computer help forum, if it was a patriotic statement for a UK MP, I certainly wouldn't want it to appear to have been written by a yank, for instance.

Others have pointed out the difference, as being American vs British. You asked also about the trend. One way to get an idea, yourself, is to use Google Ngram.

If you type color and colour into it, for example, you get this graph, which shows that color seems to be gaining in usage over colour in both US English and British English. But not so, for labor vs labour.

I'm not even going to mention the fact that "o" is American English and "ou" is British English, as it has been mentioned by every other answer. (Whoops! I just mentioned it;)

But since you asked for trends -- I've noticed that in America, "o" is used almost exclusively, except for instances where people try to sound fancy and/or formal, such as wedding invitations (as in "We request the honour of your presence.")

First hit on Google images for "wedding invitation":

_{^^ Note that this is an American invitation; the wedding is in California ;)}