Are there rules about using "that" to join two clauses?
Solution 1:
That can almost always be dropped. In your example, that is being used as a conjunction, i.e. it is introducing a subordinate clause as the object of the main sentence. In most situations where this is the case, it can be dropped. I cannot think of any where it can't be dropped.
When that is used as a demonstrative pronoun, e.g. "that was a nice question," it must be kept or replaced with another pronoun, e.g. "yours was a nice question."
When used as a relative pronoun, it can usually be dropped. For instance, "several people read the question that you wrote" can also be "several people read the question you wrote". But if used in a question with who, it should be kept. For example, "Who was the person that wrote this question?" cannot be *"Who was the person wrote this question?".
I'm sure I missed something, but the comments should keep me honest.
Solution 2:
For relative clauses: If the missing element (which is said to be "relativized") of the relative clause is the subject, "that" is obligatory, otherwise, it is optional:
Peter tasted the wine (that) his mother bought for him
Peter tasted the wine that/*() had cost only 12$.
Solution 3:
"That" is a conjunction in this type of sentence. (In "the cat that jumped over the wall", it is a relative pronoun.) It is in general OK to leave out the conjunction "that" now and then, as long as no ambiguity arises. Everybody does it all the time, even in formal style, though it happens more often in informal writing and speaking. It is possible too in German, though not in Dutch, as you probably know.
*She believed the judge, who was older than her father, was putting the moves on her.
In this sentence, the reader is led onto a false scent because "that" is omitted after "believed". This is a serious mistake. The reason why it is bad here is that "to believe" can have an object, so that the reader thinks "the judge" is simply the object of the verb, instead of the subject of a dependent clause, which in fact it is. When he finds out he is getting the sentence wrong, he needs to read back to repair the damage.
They say she was unable to realize her husband had left her.
This sentence is doubtful, because "that" is left out two times in a row. It makes the sentence a bit too loose, a bit messy: though the reader will probably get it right in one go, he will still need to spend a little more energy on it than necessary.
Solution 4:
In the example you gave, "I was not joking" is a subordinate clause. One way to think about this is that there are two sentences
- He will understand X.
- X = I was not joking.
"that" is not exactly a conjunction as Vincent said. A conjunction joins two independent clauses together, but here 2. is sort of the object of 1. Any time you want to use a clause as the object of a sentence, you can optionally precede it with "that." However, if you want to use a clause as the subject of a sentence, it will always be preceded by "that"
That I was not joking will be clear to him.
*I was not joking will be clear to him.
Sometimes clauses modify nouns. In these cases, they're called relative clauses, and what pronoun appears before the clause varies. One of the most important factors of whether or not "that" or "which/who" precedes the clause is whether the clause is "restrictive". To modify somee examples from Wikipedia:
(Restrictive) The house that Jack built fell down.
(Restrictive) The house __ Jack built fell down.
(Non-restrictive) The house, which we all thought was in fine condition, fell down.
In the first two examples, the relative clause picked out a particular house from all possible houses. In these sentences, there is a tendency to use "that" or nothing. In the third example, the particular house was already determined, and the relative clause simply introduced extra information. These relative clauses tend to be preceded by a wh- word.
Solution 5:
As most of the other answers here point out, including that in a sentence like "He will understand that I was not joking" is optional. But I'd like to offer some examples where that, despite performing the same basic function that it does in "He will understand that I was not joking," either is crucial for sense or vastly improves the flow of the sentence. My examples fall into four categories.
Awkwardness/Unnaturalness
I appreciate [that] CAN-SPAM creates a single set of rules for an industry previously struggling with a hodgepodge of 34 existing state antispam laws.
Long Beach Opera recognizes [that,] in order to preserve and strengthen opera's significant in American culture, it must continue to develop new approaches in programming.
He claimed [that] customers complain [that] the new product is heavier than the iPad2 and lacks any major improvements from the old version.
In each of these sentences, readers can supply the implied that themselves, but with more effort than the author should require them to make.
Parallelism
But he insists [that] the charges are false, and that he's sending as many e-mail messages now as he did before the suit was filed.
Google says [that] new movies will be $3.99 and older ones $2.99, and that most rentals will be for 24 hours.
Political consulting firm Election Data Services, which tracks voting-machine usage nationwide, estimates that 40 percent of voters will cast ballots on touch-screen machines this year, and [that] about 42 percent will cast them either on optical-scan machines or on traditional paper ballots.
In these sentences, using that in one branch of a parallel construction calls for using it in another, for balance and clarity.
Ambiguity: Modifier Placement
You’d think [that] by now [OR HERE?] I’d be used to Windows’ mystifying default behaviors.
The ISPs promise [that,] under the new agreement, [OR HERE?] content providers will not be given your subscriber information such as your name, address, telephone number, and e-mail address.
A spokesman for the ISP says [that] online [OR HERE?] criminals are stealing AOL users’ account passwords and using the profiles to advertise their sleazy wares.
In these sentences, that could fall in either of two plausible places, and the meaning of the sentence would change somewhat depending on its placement.
Ambiguity: Attribution
Microsoft says [that] the transition to a 64-bit OS on the desktop will take a few years, and [HERE, TOO?] typical users will be on the far end of that distribution curve.
AOL says [that] it will offer online backups, but [HERE, TOO?] the feature won’t be included in the July 13 beta.
Providers of the service say [that] the bugs have been all but exterminated and [HERE, TOO?] the service is ready for prime time.
In these sentences, the ambiguity relates to whether both clauses are assertions made by the named source, or whether the first is from that source while the second is a follow-up remark made by the author.