Are adjectives and adverbs just collapsed version of adjuncts?

Modifiers for verbs/ nouns can come in 3 main types: adjectives, adverbs & adjuncts. These all provide specific details about corresponding noun/ verbs e.g:

• Manner, means (instrumental) - with, using, by (way of)
• Positional (in space, time) - to, at, in, on,under
• stative (being) - is, are
• etc...

take the phrase: "he draws With a pencil", the adjunct modifier comprises:

• A Word communicating the nature of the modification (e.g. Spatial vs instrumental) + noun i.e. With + a pencil.

Such constructions make more sense than having separate words that each mean "with a paintbrush" "with a pencil" etc.

Instances where particular nouns are used commonly in these constructions it makes sense to collapse the construct into a single word which captures both meanings, e.g. "with + speed" = quickly. The same goes for adjectives - where "red" = is + red.

"A Word communicating the nature of the modification" generally words that fulfil this role are considered to be prepositions, but somethimes words which traditionaly fall into other word classes can be used - like verb participles.

Is this a reasonable summery? or am I seriously misunderstanding something?

At the most fundamental level, you are confusing the concept of a part of speech with the concept of grammatical function.

(You are also somewhat mixing up syntax and semantics, but let me say no more about that right now. Depending on how you choose to make your question more focused, semantics may or may not turn out to be relevant.)

Adjectives and adverbs are parts of speech that serve many kinds of grammatical functions. Just to give two examples, adjectives may serve as predictive complements (She is smart) or as pre-head modifiers within noun phrases (a smart lady).

Adjuncts are particular parts of sentences that can be realized in many kinds of ways by many kinds of parts of speech. As an example, consider adjuncts of temporal location. They may be realized by preposition phrases (I spoke to her before the meeting), noun phrases (I saw her yesterday), adverbial phrases (She had resigned three days earlier), gerund-participial clauses (Driving along the highway, we passed a long line of lorries), and past-participial clauses (This done, he walked off without another word). (See CGEL, pp. 696-699.)

You are also missing some concepts. Here is a sketch (borrowed from this answer of mine) that illustrates some of the relevant terminology used to describe different kinds of parts of a sentence:

In short: adjuncts may or may not be integrated into the syntactical structure of the sentence. If they are, they are called modifiers; if they are not, supplements.

There is also a category that includes both modifters and complements, but excludes supplements. These are dependents, which are either complements, or modifiers, or determiners.

Again, this is the terminology used in CGEL; other grammars use these terms a bit differently.

From CGEL:

adjuncts may be dependents (modifiers), … or supplements, elements that are more loosely attached to the clause. (p. 215)

Supplements are parts of a sentence that aren't integrated into the syntactical structure of it, but rather appear as interpolations or appendages (p. 1350):

Pat—the life and soul of the party—had invited all the neighbours.
The best solution, it seems to me, would be to readvertise the position.
Jill sold her internet shares in January—a very astute move.

In particular, adjectives can appear in most of these functions: as supplements (The editor has been sacked and, worse, they're imposing strict censorship), modifiers (a smart lady), and complements (She is smart). They can't serve as determiners (but can serve as predeterminers, which are a kind of modifier: such a nuisance).