Function "evaluation" just means "composition"?

That’s right, evaluation is not in any serious way distinct from composition with a function with singleton domain—or to remove all talk of elements entirely, a function whose domain is a terminal object. This kind of function can entirely replace the role of elements, which is at the heart of the categorical axiomatization of set theory called ETCS.

Basically, yes, what you are saying is that in the category of sets and functions the action of composition of $f\colon A\to B$ with $(a)\colon \star \to A$ is precisely encoding function evaluation $f(a)$. So this can indeed neatly be written as $f(x)=f\circ (x)$. This paves the way to thinking about functions and elements in a dual way: a function $f\colon A\to B$ is a machine that converts elements of type $A$ into elements of type $B$; dually an element of type $A$ is a thing that any machine $f\colon A\to B$ can act on to turn into something of type $B$. This tells us that we can take elementhood as primitive and define functions in terms of elements (as is classically done) or that we can treat functions as primitive and define elementhood in terms of functions (as is categorically done). In some sense, for sets and functions (at least naively) the two approaches agree.

However, in many other situations the two options are very different. Suppose that instead of sets and functions your world consists of the non-negative reals $[0,\infty ]$ (with $\infty $ thrown in) and you keep track of magnitude by means of $x\to y$ iff $x\ge y$. The special element $0$, the additive neutral element, serves here as the 'singleton' $\star$ (the reason is that a singleton set is neutral (up to isomorphism) with respect to the cartesian product of sets; similarly $0$ is neutral with respect to addition of real numbers). Now, in the world of sets the arrows $\star\to A$ encode the elements of $A$. What is encoded in the world of non-negative reals? Well, $\star \to a$ is no simply saying that $a=0$. So, if $a=0$, then it has a unique 'element' while any $a>0$ has no elements at all.

This is probably not surprising since real numbers are quite different from sets. Places that behave more like sets than real numbers are called toposes. But there is really no need to abandon the "composition=evaluation" paradigm in non-toposes. Indeed, evaluation can further be understood in terms of a much simpler operation. Going back to sets, if $[A,B]$ is the set of all functions $f\colon [A\to B]$, then the evaluation mapping is $\mathrm {ev}\colon [A,B]\times A\to B$ given by $\mathrm {ev}(f,a)=f(a)$. This evaluation function has a very special universal property. If $f\colon T\to [A,B]$ is any function from any set $T$ we can form the composition $\mathrm {ev}\circ (f\times \mathrm {Id}_A)$ to get a function $h\colon T\times A\to B$. But also conversely, any such function $h$ arises uniquely in this way from a function $f\colon T\to [A,B]$.

In other words, there is a natural bijection between functions $f\colon T\to [A,B]$ and functions $h\colon T\times A\to B$. The translation is performed universally by the evaluation function. So, we observe that the internal notion of the set of all functions $[A,B]$ together with the notion of evaluation is fully encoded by the much simpler notion of the cartesian product of sets. The encoding is done by stating the natural bijectiveness above. The technical way of saying it is that evaluation is the counit of an adjunction whose right adjoint is cross multiplying with a fixed set $A$.

Now we can replace the world of sets and functions with their cartesian product by other things. For instance, non-negative real numbers, magnitude comparison, and addition. It turns one (nice exercise) that there is here too an internal notion of "the real numbers of all magnitudes from $a$ to $b$", namely $[a,b]$, complete with an evaluation mapping with formally the same properties as for sets. Such places are called closed monoidal categories.

Is there anything more to evaluation, or can I exactly equate evaluation with composition and forever de-clutter my mind of the term "evaluation"?

Frankly, there is no way to non-circularly 'get rid' of the notion of "evaluation". A rose by any other name is still a rose. Even Lawvere's axiomatization of ETCS explicitly axiomatizes the evaluation mapping in Axiom 2, just phrased in category theory terms.

I think what you must realize is that certain mathematical notions are core notions and whatever implementation you use for them does not make them disappear at all. For example, it makes no sense to try to argue that ZFC set theory has eliminated the notion of function application because instead of "$f(x)$" we can use "$\bigcup \{ y : ⟨x,y⟩∈f \}$" given by the Union axiom and the Replacement schema (or a slightly more complicated expression if you use Specification and Powerset instead of Replacement).

The whole point is that we design foundational systems to be able to do the mathematics that we want to do. Since we want to have the usual natural numbers with the usual induction, we also want our foundational system to be able to construct a model of PA (for suitable encoding of "model"). And since we want to be able to reason about functions on $ℕ$, functions on $ℝ$ and so on, with the usual notion of "function", including evaluation, composition, function types and equality, we make sure that our foundational system is able to construct representations of the usual functions and prove those usual properties! See also this related post explaining why we care about abstractions rather than how things are concretely implemented in the foundational system.

There is nothing wrong with thinking about weaker systems that cannot do some of these things, but most logicians would not consider these systems viable as foundations for all mathematics, precisely for that reason!