probablity of random pick up three points inside a regular triangle which form a triangle and contain the center

what is the probablity of random pick up three points inside a regular triangle which form a triangle and contain the center of the regualr triangle

the three points are randomly picked within the regular triangle and then form a new triangle and the new triangle have to contain the center of the original regular triangle what is the probability

Let $R$ be a convex region of the plane with unit area, and choose $n\ge 3$ points at random from the region. We will derive a general expression for the probability $P_{n}$ that the convex hull of these points contains a particular point $\tau \in R$ (which we will take as the origin of our coordinate system). We can then evaluate that expression for the particular question posed (in which $n=3$, $R$ is an equilateral triangle, and $\tau$ is its center).

Suppose the hull does not contain $\tau$, and assume that $\tau$ is not collinear with any two of the points (this is true with probability 1). Then we can draw a ray from the center point through one of the $n$ points (the "leftmost" point) such that the remaining $n-1$ points are to the right of the ray. (Note that this condition is both necessary and sufficient. Note also that if the convex hull does contain $\tau$, then there is no "leftmost" point.) We can calculate the desired probability by integrating over all such configurations. In particular, take $f(\varphi)$ to be the area of the subregion that lies to the right of the radial ray at angle $\varphi$, and take $r(\varphi)$ to be the distance from $\tau$ to the boundary of $R$ along that ray. Then the probability that a particular point will lie in the angular interval $[\varphi, \varphi+d\varphi]$ is $da = \frac{1}{2}r(\varphi)^2 d\varphi$, and the probability that each of the other $n-1$ points will lie to its right is $f(\varphi)$. Since there are $n$ points to choose as the leftmost, the overall probability of not capturing $\tau$ in the convex hull is $$ 1-P_{n} = n \int f(\varphi)^{n-1} da = n \int_{-\pi}^{\pi} \frac{1}{2} f(\varphi)^{n-1} r(\varphi)^2 d\varphi, $$ We can make a few observations at this point. First, if the region $R$ is symmetric under reflection through $\tau$, then $f(\varphi)$ is identically $1/2$: each radial line splits the region in half. The result then is $$ P_{n} = 1 - \frac{n}{2^{n-1}}, $$ which gives the known result $P_{3} = 1/4$ for polygons with an even number of sides. Second, if the region $R$ instead has bilateral symmetry across the $x$-axis, then $r(\varphi)$ is an even function, and $f(\varphi) = 1/2 + f^{-}(\varphi)$, where $f^{-}$ is an odd function. Then terms in the integral with odd powers of $f^{-}$ must vanish: in particular, we have $$ P_{3} = \frac{1}{4} - 3 \int{{f^{-}(\varphi)}^2 da} = \frac{1}{4} - 3\int_{-\pi}^{\pi}\frac{1}{2}{f^{-}(\varphi)}^2 r(\varphi)^2 d\phi \le \frac{1}{4}, $$ and interestingly the relation $P_{4} = 2P_{3}$ continues to hold.

It remains to evaluate $f^{-}(\varphi)$, $r(\varphi)$, and the resulting integral in the case of an equilateral triangle. The figure above shows an equilateral triangle with vertices at $(-x,0)$ and $(x/2, \pm x\sqrt{3}/2)$. For $\varphi \in [0,\pi/3]$, the function $f^{-}(\varphi)$ is the area of the blue triangle minus the area of the red triangle, $$ f^{-}(\varphi) = \frac{x^2}{8}\tan\varphi - \frac{x^2}{2}\frac{1}{\cot\varphi + \cot\frac{\pi}{6}} = \frac{x^2}{8}\tan\varphi \left(1 - \frac{4}{1 + \sqrt{3}\tan\varphi}\right); $$ and the radius $r(\varphi) = \frac{1}{2}x\sec\varphi$ over the same domain. By symmetry, the full integral is just six times its value over $[0,\pi/3]$: $$ \begin{eqnarray} P_3 &=& \frac{1}{4} - 18\int_{0}^{\pi/3}\frac{x^6}{512}\tan^2\varphi\left(1 - \frac{4}{1 + \sqrt{3}\tan\varphi}\right)^2 \sec^2\varphi d\varphi \\ &=& \frac{1}{4} - \frac{1}{36\sqrt{3}}\int_{0}^{\pi/3}\tan^2\varphi\left(1 - \frac{4}{1 + \sqrt{3}\tan\varphi}\right)^2 \sec^2\varphi d\varphi \\ &=& \frac{1}{4} - \frac{1}{12}\int_{0}^{1} u^2\left(1 - \frac{4}{1+3u}\right)^2 du, \end{eqnarray} $$ where we have used $x = 2/\sqrt{3\sqrt{3}}$ (in order for the triangle to have unit area) and introduced the transformed variable $u = \tan\varphi / \sqrt{3}$ (so $du = (\sec^2\varphi / \sqrt{3}) d\varphi$). The final integral is a straightforward exercise for the reader; the result is $$ P_3 = \frac{1}{4} - \frac{1}{324}\left(57 - 80\ln{2}\right) = \frac{2}{81}\left(3 + 10\ln{2}\right) = 0.2452215..., $$ in agreement with the brute-force and numerical results already given. This result can be generalized easily to the regular $m$-gon for any odd $m$ (changing only the values of some constants), and to the case where $n>3$ (complicating the final integral).

The answer is $p = \frac{2}{81} (3+\log (1024)) \approx 0.24522152606418402702 < \frac{1}{4}$.

I basically did a brute force computation. Fix the regular's triangle coordinates at $p_1 = (0,0)$, $p_2 = (0,1)$, $p_3 = (\sin(60^\circ), \cos(60^\circ))$. Let's parametrize points inside the right triangle as $p(u,v) = p_1 + (p_2-p_1) u + (p_3-p_1) v(1-u)$, with $0<u<1$ and $0<v<1$.

A uniform distribution within so parameterized triangle has the following density function $f(u,v) = 2(1-u) \mathbf{1}_{0 < u <1} \mathbf{1}_{0 < v <1}$.

Let $q_1$, $q_2$ and $q_3$ be randomly selected points within the regular triangle. The condition that point $p_c$ is within this triangle is $\ell_{q_1,q_2}(p_c) \ell_{q_1,q_2}(q_3)>0 \land \ell_{q_1,q_3}(p_c) \ell_{q_1,q_3}(q_2)>0 \land \ell_{q_2,q_3}(p_c) \ell_{q_2,q_3}(q_1)>0$, where $\ell_{q_1,q_2}(x,y)$ is the equation for the line through points $q_1$ and $q_2$.

Now, I plug everything in, and wait:

This still doesn't answer the question asked, but is getting too long for comments.

Here's an argument that the probability is always $\frac{1}{4}$ as long as the arena is centrally symmetric about the center point. Especially this is the case for a circle, a square, and all even-sided regular polygons:

As Didier noticed, only the directions from the center to each of the three points matter. The triangle they form contains the center iff their projections along radii to a (concentric) circle divides the circle's perimeter in three pieces each of which is less than 180°.

Draw a vertical diameter through the center point. Due to the central symmetry, any of the three points has equal probability to land left or right of the diameter. If all three of them are to the same side of the diameter (which happens with probability $\frac{1}{4}$), the center clearly isn't in the triangle. So we need only to analyze the situation, with probability $\frac{3}{4}$, that two of them are on one side and one on the other. Let's say, without loss of generality, that there is one to the left and two to the right.

Now, if we take the line through the center of the left point and continue it through the right side of the arena, we win if this line passes between the two right points. Also, since the arena is centrally symmetric, we might as well have chosen the left point by picking its reflection uniformly on the right. So the case is equivalent to picking two red points and one blue one on the right side uniformly on the right side of the arena and then asking whether the blue is between the reds. Since these points are i.i.d., the exact form of the distributions doesn't matter for their relative order, and so the probability of success here is $\frac{1}{3}$.

In total, the probability of the triangle containing the center is $\frac{3}{4}\times\frac{1}{3}=\frac{1}{4}$ -- as long as the arena is centrally symmetric!

What implications does this have for the triangular arena? We can still find a diameter that bisects the arena, but the idea of choosing all three points to the right and then reflecting one doesn't work anymore. In fact, heuristically, the more the red and blue distributions (of directions) differ, the less ought the probability of having the blue point end up in the middle to be. So we should expect a proability less than $\frac{1}{4}$ in the triangular case.

By continuity, odd regular polygons with many sides will give answers that approach $\frac{1}{4}$, so we should expect the triangle to be the odd-sided regular polygon with the least probability to have a random triangle contain the center.

By dividing the equilateral triangle in six congruent 30-60-90 triangles and counting combinations, I have found that the probability is $\frac{2}{9}$ plus $\frac{1}{12}$ of the probability in the case where two points are in one of the hexants and the third is in the diametrically opposite one. Further subdivision into symmetric sub-cases shows the latter probability to be between $\frac{1}{6}$ and $\frac{2}{3}$. So we have the lower bound $$P>\frac{2}{9}+\frac{1}{12}\times\frac{1}{6}=\frac{17}{72}=\frac{1}{4}-\frac{1}{72}\approx 0.2361$$