Find the expansion for $\det(I+\epsilon A)$ where $\epsilon$ is small without using eigenvalue.

We note that the determinant is defined by the Leibniz formula:

$$\det(\mathbf{I}+\epsilon\mathbf{A})=\sum_{\sigma \in S_{n}}\prod_{i=1}^{n}(\mathbf{I}+\epsilon \mathbf{A})_{i,\sigma_{i}} \cdot \operatorname{sgn}(\sigma)$$

We note that along the main diagonal we have:

$$\prod_{i=1}^{n}(\mathbf{I}+\epsilon \mathbf{A})_{i,i}=1+\epsilon (A_{11}+\cdots+A_{nn})+\mathcal{O}(\epsilon^{2})$$

And along the other possible permutations we have at best $\mathcal{O}(\epsilon^{2})$ as there must be at least 2 off-diagonal entries for it to be a vaid permutation. Therefore we can say:

$$\det(\mathbf{I}+\epsilon\mathbf{A})=1+\epsilon\operatorname{Tr}(\mathbf{A})+\mathcal{O}(\epsilon^{2})$$

As required.

Consider this expression of the determinant: $$ \det(B) = \sum_{\sigma \in S_n} (-1)^\sigma \prod_{i=1}^n b_{i,\sigma_i} $$

For $\sigma=id$, we have the term $b_{11}b_{22}\cdots b_{nn}$. Now take $B=I+\epsilon A$ and this term becomes $(1+\epsilon a_{11})(1+\epsilon a_{22})\cdots(1+\epsilon a_{nn})=1+\epsilon \, \rm{tr}\,A+$ higher-order terms.

All other terms in the expression above contain $\epsilon^{2}$.

How this is done would depend on which characterizations of the determinant function you've seen. One of those is $$ \sum\left\{ \prod_{k=1}^n (\pm1)a_{k,\sigma(k)} : \sigma\text{ permutes }\{1,\ldots,n\} \right\} $$ where the sign is $+1$ if $\sigma$ is an even permutation and $-1$ if it is odd. When $\sigma$ is the identity permutation, the product is $$ (1+\varepsilon a_{11})\cdots(1+\varepsilon a_{nn}) = 1 + \varepsilon(a_{11}+\cdots+a_{nn}) + \varepsilon^2(\cdots) + \varepsilon^3(\cdots)+\cdots. $$ When $\sigma$ is any other permutation then the product is $$ \varepsilon a_{k,\sigma(k)} \varepsilon a_{j,\sigma(j)}\cdots+\cdots, $$ i.e. there are at least two indices $k$ for which $\sigma(k)\ne k$, so that $\varepsilon^2$ is a factor in this term. Thus the determinant is $$ 1+\varepsilon\operatorname{tr}(A)+(\text{terms of degree $\ge2$ in $\varepsilon$}). $$