How to “shrink” a triangle

Solution 1:

Green triangle (result) is obviously a homothety of the given (blue) triangle with respect to a certain point $Q$ inside the triangle, and a certain coefficient $k$. As you wish to find $Q$ and $k$ providing the distance $X$ from the original edges, then the direction of the moveout of the vertices lays on the original triangle's bisectors (proof is trivial for every vertex of the green triangle). The intersection point of triangle bisectors is the center of the inscribed circle. So, the homothety center $Q$ is the center of the inscribed circle. The homothety coefficient $k$ is the fraction of $X$ with respect to inscribed circle radius $R$: $k=\dfrac{R}{R-X}$. Thus, given the original triangle vertices $A, B, C$, find the inscribed circle $Q, R$, then find the scale $k$, then moveout the vertices: $A'=A+(Q-A)\times k$ - same for $B, C$ - in vector form.