Proving that the genus of a nonsingular plane curve is $\frac{(d-1)(d-2)}{2}$

Let me suggest a more geometric approach than the one you are using:

Let $C$ be your degree $d$ plane curve. Choose a point $P$ not on $C$, and not on the $x$-axis. Let $\mathbb P^1$ be the $x$-axis.

Now let $\phi:C \to \mathbb P^1$ be projection from $P$ to the $x$-axis, i.e. given any point $Q$ in $C$, draw a line from $P$ to $Q$, and see where it intersects the $x$-axis; that point is the value $\phi(Q)$.

This will be a map of degree $d$ (if you take a point $R$ in the $x$-axis, and count how many points lie in its preimage, you draw the line through $P$ and $R$, and count how many times this intersects $C$; the answer will be $d$ times, because $C$ is of degree $d$).

Now you can apply the Riemann--Hurwitz formula; the only problem is to work out the ramification points.

Ramification will occur when the line through $P$ and $Q$ has a multiple intersection with $C$ at $Q$, i.e. is tangent to $C$ at $Q$. If you choose everything generically (e.g. choose $P$ generically, and/or change coordinates so that $C$ is in a general position with regard to the $x$-axis), then these intersections multiplicities will never be more than $2$ (i.e. you will get tangencies, but never higher order tangencies), and so $e_{\phi}(Q)$ will be either $1$ (if the line through $P$ and $Q$ is not tangent to $C$ at $Q$) or $2$ (if the line through $P$ and $Q$ is tangent to $C$ at $Q$).

Now you have to figure out how many times a tangency occurs.

Of course, you can work this out by assuming the answer (i.e. that $g(C) = (d-1)(d-2)/2$, and working backwards). I suggest that you also write down an explicit conic, like $x^2 + y ^2 = 1$, and then perhaps a higher degree curve, and concretely apply the above procedure and see directly how many points of tangency occur. After doing all this, you will hopefully figure out in general how many $Q$s there are for which the line through $P$ is tangent to $C$, and also see how to prove that what you have worked out is correct.

To fill in all the details you will either need to make the "general position" argument above rigorous, or deal with the possibility of higher order tangency (i.e. the case when $e_{\phi}(Q) > 2$). But I would worry about this later, after you understand the basic geometry of the situation.