How did Newton find derivative of basic functions before formulating systematic Calculus?
The earliest formulations of something that looks like calculus were not in terms of functions but of curves given by algebraic relations. So if we have $$ y=x^2 $$ and $(y+p,x+q)$ is another point on the curve, we would also have $$ (y+p) = (x+q)^2 $$ and if we multiply these equations out and subtract we get $$ p = 2qx + q^2 $$
Now what happens in the earliest sources is that it is simply postulated, without any backing by theoretical definitions or limits that when $p$ and $q$ are small, we can remove their higher powers, so ignoring the $q^2$ term yields us $$ p = 2qx \qquad\text{or, in other words,}\qquad p:q = 2x:1 $$ which can be used to draw a tangent.
The justification for this procedure was initially just that it worked in practice, but the unexplained ignoring of terms like $q^2$ drew quite a lot of contemporary criticism.
This was in the generation before Newton (chiefly Fermat and Descartes). They could produce painstaking geometric proofs in the Euclidean tradition (which was the gold standard for proofs in those days) that for each of the actual curves they considered what came out of this procedure was the right result -- but they didn't have the definitions and machinery to explain or prove rigorously why the procedure always works.
Newton brought in the new idea of a quantity $x$ that varied with time and its time rate of change $\dot x$, but he was still building on the "ignore higher powers of the increments" method. But he did it masterfully and was able to get more out of it than his predecessors did.
Some of these were likely found through a combination of intuition and geometric reasoning.
Consider $y = x^2$. Notice that the square of side length $x$ has area $y$. Suppose that we change the side length by $dx$. Then what is the change in the area of the rectangle $dy$?
I apologize for the crudely drawn picture, but notice that the change in area that we get is,
$$dy = xdx+xdx+(dx)^2 = 2xdx+(dx)^2 \approx 2xdx$$
for small enough $dx$ (i.e. the little black square is insignificantly small). And thus,
$$\frac{dy}{dx} = \frac{2xdx}{dx} = 2x.$$
Similar reasoning can be used to show that $\frac{d}{dx} (x^3) = 3x^2$ by considering the change in volume of a cube.
EDIT: Figured it was worth mentioned that this same square technique can be used to intuitively see why $\frac{d}{dx}(f(x) \cdot g(x)) = f(x)g'(x)+f'(x)g(x)$.
EDIT 2: After you have the product rule, it is easy to derive the power rule (over the natural numbers) using induction. The base case $\frac{d}{dx}(x^1) = 1$ is obvious. If we assume $\frac{d}{dx}(x^k) = kx^{k-1}$, then, $$\frac{d}{dx}(x^{k+1}) = \frac{d}{dx}(x^{k} \cdot x) = \frac{d}{dx}(x^{k})\cdot x + x^{k} \cdot \frac{d}{dx}(x)$$$$ = kx^{k-1}\cdot x+x^k\cdot 1 = kx^k+x^k = (k+1)x^k.$$
Here is some historical information cited from Oskar Beckers Grundlagen der Mathematik in geschichtlicher Entwicklung (say: Foundations of mathematics in historical development).
The first matured presentation of Newtons (1643 - 1727) method of fluxions which was his name of differential calculus dates from 1670/71. The title was Methodus fluxionum et serierum infinitarum and has been posthumously published in 1736.
Newton started to think about fluxions some years before. Here is a small excerpt from one of his early notes cited from O. Becker [ch. Fluxionsmethode p.147].
Newton: (from 23.11.1665)
...If two bodies $A$ and $B$ move uniformly, one of them from $A$ to $C,E$ and $G$, the other from $B$ to $D,F$ and $H$ along the same line, then the line segments $\overline{AC}$ and $\overline{BD}$, $\overline{CE}$ and $\overline{DF}$, $\overline{EG}$ and $\overline{FH}$ behave in the same manner as their velocities $p$ and $q$ and, if they do not uniformly move, then the infinite small line segments behave in each momentum as their velocities in each momentum.
If the body $A$ with velocity $p$ describes the infinite small distance $o$ in a momentum, then the body $B$ will describe in the same momentum with velocity $q$ the distance $\frac{op}{q}$. Since $p:q=o:\frac{oq}{p}$. So, if the line segments already described within one momentum are $x$ and $y$, the next will be $x+o$ and $y+\frac{oq}{p}$.
Now I'm allowed, if the equation of the line is represented by \begin{align*} rx+xx-yy=0 \end{align*} to substitute $x+o$ and $y+\frac{oq}{p}$ for $x$ and $y$ in the equation. This is valid, since they both mean in the same way as $x$ and $y$ the travelled distance described by the bodies $A$ and $B$. We obtain: \begin{align*} rx+xo+xx+2xo+oo-yy-\frac{2qoy}{p}-\frac{qqoo}{pp}=0 \end{align*} But, according to the assumption we have: \begin{align*} rx+xx-yy=0 \end{align*} after subtraction: \begin{align*} ro+2xo+oo-\frac{2qoy}{p}-\frac{qqoo}{p}=0 \end{align*} and after division by $o$: \begin{align*} r+2x+o-\frac{2qy}{p}-\frac{oqq}{p}=0 \end{align*}
Now, the terms which contain $o$ are infinitely smaller than the terms which do not contain $o$. So, after cancelling these terms we obtain: \begin{align*} r+2x-\frac{2qy}{p}=0\qquad\text{or}\qquad pr+2px=2qy \end{align*}
...
This is an early example how Newton treated problems of differentiation of functions like $y=x^2$. It indicates the development of differential calculus was driven solely to solve physical problems and calculation with infinitesimals was reasoned by physical evidence.
According to O.Becker it was Newtons teacher Isaac Barrow who inspired him to develop these first ideas. On the other hand intensive studies indicate that especially Blaise Pascal inspired Leibniz in his development of differential calculus methods.
Here was how for $f(x)=x^2$:
$$\frac{f(x+o)-f(x)}{o}=2x+o.$$ Now if $o$ gets small, the quotient gets close to $2x$, so Newton declared that when $o$ "gets to" 0, the quotient "gets to" $2x$.
Note, though, that in Newton's times there was no rigorous formulation of limits. So perhaps he was drawing conclusions somewhat from intuition and experimentation. But also keep in mind that intuition was where calculus stemmed from.