What is a word that describes the cultural attachments that come with a language?

For example, generally, if someone speaks American English fluently, that person would be familiar with not only common American phrases but also could relate to it.

Example:

Two strangers cross each other's path and one of them in a friendly gesture, playfully says...Boo! The other person if startled would immediately afterwards realize that it was a joke because the person used "boo" and therefore reciprocate the friendly intent of the gesture by a gesture of acknowledgment...perhaps through a smile, laugh, saying "you got me!", saying "shut up! (while smiling), respond with keeping the banter...etc.

On the other hand, if it were someone that was not familiar with "boo" and what it implied (perhaps due to learning it through text book, rather than experience), they might take it as a malicious gesture. Even though the person might be more fluent than the first person in English, they come from a different culture of where that particular language is used, thus they don't have the cultural understanding that comes with many parts of it.

What is that cultural understanding called?

For instance, if it was called ooly. I would say..."He's good in English, but still, be mindeful that there could be misunderstandings due to ooly."

If this was restricted to the meanings of individual words or phrases, you could call them idiomatic differences.

An idiom is a phrase or expression that typically presents a figurative, non-literal meaning attached to the phrase; but some phrases become figurative idioms while retaining the literal meaning of the phrase. Categorized as formulaic language, an idiom's figurative meaning is different from the literal meaning. - wikipedia

Although some consider idiomatic differences to be "between different languages", the word "language" is sufficiently plastic to cover dialects of the same language.

Here's one example: what does it mean to table a document?

• In the United States, to "table" usually means to postpone or suspend consideration of a pending motion.
• In the rest of the English-speaking world, to "table" means to begin consideration (or reconsideration) of a proposal. - wikipedia

The difference in the idiomatic usage of the term table can lead to exactly the sort of misunderstanding your sample sentence warns against.

However, in something like your "Boo!" example, there's more at play than just the meaning of a word or phrase. The stylised play that distinguishes innocent fun from a form of verbal assault includes body language and even situational context. In this case, it's cultural differences that your sample sentence warns about.

Here's an example of the term in use (emphasis, mine):

But visible cultural differences are only ten percent of our cultural identities: hidden cultural differences including values, assumptions, and beliefs represent the remaining ninety percent of our cultural identity. - cultureplusconsulting.com