Number of isoceles triangles formed by the vertices of a polygon that are not equilateral
QUESTION: Let $A_1,A_2,...,A_n$ be the vertices of a regular polygon with $n$ sides. How many of the triangles $△A_iA_jA_k,1 ≤ i < j < k ≤ n,$ are isosceles but not equilateral?
MY APPROACH: Firstly, I need to know that why is there only one unique isoceles triangle if I choose any $2$ vertices out of the $n$ vertices.. If it is so then the answer would be $$\binom{n}{2}- (the \space number \space of \space equilateral \space triangles)$$ Now how do I find out the number of equilateral triangles?.. I came across a formula which says the answer of this is $$\frac{n(n-4)}2$$ How is this derived?
For ex: if the polygon has $18$ sides then the answer is $126$..
Can anyone please help me out? Thank you so much.
I think one possible approach would be the following.
Let us start start by neglecting the fact that there may be equilateral triangles.
Consider the vertices of your polygon to be labelled by integers from $1$ to $n$, so that $(j,j+1)$ represent one side for each $j\leq n-1$ and $(n,1)$ is the remaining side. Consider an isosceles in the polygon. It will have two equal sides. One of this sides may be $(1,2)$ and the other $(2,3)$. There is an equivalent triangle with equal sides given by $(2,3)$ and $(3,4)$. And you can go on until $(n-1,n)$ and $(n,1)$. So you have $n$ isosceles such that each of the equal sides join two subsequent vertices of the original polygon.
Then you can consider triangles which has these two sides given by $(1,3)$ and $(3,5)$. And all the equivalents where these sides join vertices $j$ to $j+2$. Again you have $n$ of them.
You can go on finding sides joining $j$ with $j+i$. When should you stop? You must have that $i<n/2$. So you have $n/2-1$ possible choices for $i$ if $n$ is even and $(n-1)/2$ if $n$ is odd. This can be rewritten as $\lfloor (n-1)/2\rfloor$ possible choices for $i$.
So for now we would get that the number of isosceles is $$N = n\lfloor(n-1)/2\rfloor\,,$$ where $\lfloor\cdot\rfloor$ is the floor function (giving the integer part of a positive real number).
Now we have a problem. Indeed if $n$ is divisible by $3$ then we have constructed $n$ equilateral triangles taking $i=n/3$. (Moreover we have constructed 3 times the number of possible equilateral triangles, but this is not important for your problem). So, in order to get the number of the isosceles triangles which are not equilateral you need to subtract $n$ to the previous result, but only if $n$ is a multiple of $3$.
To summarise, the number $N$ of isosceles triangles is given by $$N = n(\lfloor(n-1)/2\rfloor -q)\,,$$ where $q=1$ if $n$ is a multiple of $3$ and $0$ otherwise.
As a proof check, for a square from this formula you get $$N = 4\times \lfloor 3/2 \rfloor = 4\,,$$ which seems to be the right result.
As for $n=18$, you have $$N = 18\times (\lfloor 17/2 \rfloor-1) = 18 \times 7 = 126\,,$$ which seems to correspond to your formula $N=n(n-4)/2$.
However the two formulae are equivalent only if $n$ is a multiple of $6$.
Let us try with $n=7$. Your formula gives $N = 7(7-4)/2 = 10.5$, which is clearly not possible since it's not an integer. I guess that $$N = 7\times \lfloor 6/2 \rfloor = 7 \times 3 = 21$$ makes more sense.