What exactly falls under the label of "complement"?

Alex, I think part of the problem understanding complementary relations is that there are a lot of different theories and terminology at work in traditional English instructional grammars, they just aren't unified and they often use similar ideas and terms in different ways. It's enough to make one's head spin. Linguists tend to have this pretty well sorted out but grammarians and language instructors often do not, and students take the brunt of the confusion that arises because they often consult multiple sources, teachers and websites on their language acquisition quests.

In my view, anytime a word impacts the meaning or interpretation of another word, there is a complementary relation there; within word phrases, between word phrases inside of sentence phrases, as well as between a main verb phrase and word phrases in the sentence complement phrase. Rather than distinguish between adjunct modifiers and complements, I focus on what is core, or, in other words, what words and phrases are essential to the syntax and meaning and are required to form a complete sentence. Some complements are core, like those that complete the meaning of verbs, and others are not, like subordinate clauses that add non-essential complementary information to subjects or objects.

Complements are content laden word phrases (and whole clauses), complementizers are function words (conjunctions, pronouns, prepositions, wh-words) that determine the logical relations between phrases, they connect complementary phrases and objects in defining ways much like prepositions define relations between their objects and the verb or noun they complement. But they are empty of content themselves and only acquire meaning through the logical possibility of relations between the complement and the complemented.

After a lengthy discussion with a talented user, BillJ, on this site, I have gotten a pretty good look at complements and how they function, so I have followed through with the suggestion to post an answer to my own question:

When examining complements, I find it best to look at structures syntactically, more so than semantically, as can be alluded to by the definition of "complement" -- "A syntactic element seen as completing the construction of another element," according to Matthews in The Concise Oxford Dictionary of Linguistics. Furthermore, as I discovered, complements are best described as obligatory elements licensed by some other element such as a verb.

Being able to differentiate between complements and adjuncts (or modifiers) is obviously one of the keys to defining complements. If you remove a complement from a sentence, it becomes ungrammatical or at least lacking of essential meaning.

From an example provided from BillJ,

"Sue used the cheese." (Obligatory, Complement)
"Sue ate the cheese." (Modifier)

In the first sentence, the verb "used" obviously calls for an object--or at least something to fulfill the action, as "Sue used" is not complete without "the cheese". The second sentence, however, does not need the object, "the cheese", because "Sue ate" is perfectly acceptable.

But, this is sometimes a lot more complicated, and it can be hard to understand what exactly is obligatory. Additionally, this is why it is important to focus on the sentence from both a syntactic viewpoint and one where you recognize the call for licensing that certain elements possess. I am just now starting to get the hang of distinguishing between complements and modifiers.

Moving on, it is also important to note that complements can function in many different layers of sentence structures. One thing I did discover was that relative clauses and pronouns are indeed modifiers for structures as a whole. But, within these clauses are complements. For example, consider the sentence "The walk which I had taken my dogs on was wonderful." Within this sentence, we have the relative pronoun "which" introducing the relative clause "which I had taken my dogs on". This relative item, in regards to the rest of the overall sentence, is not a complement. It is a modifier. However, if you look closer, you will see that "which" does indeed perform the role of complement...in the relative clause to the preposition "on".

Another example would be the sentence, "The boy whom I loved cared nothing for me." In this sentence, the relative clause headed by the relative pronoun "whom" is again a modifier, but, again, the relative pronoun can be said to be acting as complement within its own clause, the relative clause, to the verb "loved", as an object.

One might think that the mentioning of these relative clauses is not beneficial to the understanding of the term "complement". However, It is essentially what allowed me to understand complements, as I was able to recognize how they operate.

After conducting some additional research, it was found that there is a alternate, minority belief system on complementizers and their defining points, which I believe to be a reason why so many people have a hard time with them. The details of such beliefs can be found here: https://www.thoughtco.com/what-is-complementizer-1689770

But, they basically call subordinating conjunctions, relative pronouns, and relative adverbs complementizers. The viewpoints are based mostly upon Laurel J Brinton’s The Structure of Modern English: A Linguistic Introduction.