Given a width, height and angle of a rectangle, and an allowed final size, determine how large or small it must be to fit into the area

In other words, if I had a rectangle of $10\times 10$ and an angle of $45$, and the allowed area was $100\times 100$, the rectangle would be about $70\times 70$.

The allowed area is $100\times 100$ in red and the final rectangle is the blue $70\times 70$ rotated rect. I want to know how to get that "$70\times 70$" for any rect, angle, and area.

I believe the rectangle's original width and height are not important, only their ratio is.

Getting the above is easy if its a square, but I don't know how to do it if it is a non-square rectangle.

So what if I had a rectangle of $2\times 3$ or $200\times 300$ and an angle of $90$ degrees, that must fit into a space of $100\times 20$? Or some other arbitrary space?

Is there a general formula for determining the correct size of the original rectangle from the final area it takes up?

We will be referring to the primitive picture above. The original rectangle had horizontal side $a$ and vertical side $b$. It was rotated through an angle $\theta$ ($t$ in the picture).

We start with an $a \times b$ rectangle, with horizontal and vertical sides, such that the sides of length $a$ are horizontal. We rotate the rectangle counterclockwise about its center, through an angle $\theta\le \pi/2$.

How much horizontal and vertical space is occupied by the rotated rectangle? (In general, the minimal containing rectangle will not be a square.)

The diagram above can be used to see that the horizontal space occupied is $$a\cos\theta+b\sin\theta,$$
and that the vertical space occupied is $$a\sin\theta+b\cos\theta.$$

For suppose that the side labelled $a$ has length $a$, and the side labelled $b$ has length $b$. Then the horizontal side $PR$ of the containing rectangle is made up of two parts. By basic trigonometry, the part $QR$ has length $a\cos\theta$, and the part $PQ$ has length $b\sin\theta$. Add up. A similar argument deals with the vertical side of the containing rectangle.

If we want to fit the rotated rectangle into a square of side $s$, then the least $s$ that will work is given by $$s=\max(a\cos\theta+b\sin\theta, a\sin\theta+b\cos\theta).$$

If we have a square of fixed side $d$ (in the post, $d=100$), we may want to scale the original rectangle to make the rotated version fit. Let $\lambda$ be the (common) scaling factor of the sides of the original $a\times b$ rectangle that will make the fit in the $d\times d$ square snug horizontally and/or vertically. Then we need $$\lambda \max(a\cos\theta+b\sin\theta, a\sin\theta+b\cos\theta)=d,$$ and now we can compute $\lambda$.

Comment: The same basic formulas can be used to solve some of the more general problems that you mentioned. We obtained separate formulas for the horizontal and vertical space occupied by the rotated rectangle. Suppose that our target rectangle has given horizontal and vertical sides. We can compute the scaling factor $\lambda_h$ that will give a horizontal fit, and the scaling factor $\lambda_v$ that will give a vertical fit. Then the desired scaling factor for both a horizontal fit and a vertical fit is $\min(\lambda_h, \lambda_v)$. (Here as usual we are working on the assumption that the ratio of the sides must be maintained, so a common scaling factor must be applied to each.)

The important observation is that the width of a rotated rectangle is the greater projection of one of its diagonals on the horizontal axis. If you have a $w \times h$ rectangle, its diagonals are $\sqrt{w^2+h^2}$ long inclined at $\pm \arctan \frac{h}{w}$ If you rotate the reactangle by $\theta$ clockwise, the angles become $\arctan \frac{h}{w}-\theta$ and $-\arctan \frac{h}{w}-\theta$ and the overall width is the maximum of $\sqrt{w^2+h^2} \tan (\arctan \frac{h}{w}-\theta)$ and $\sqrt{w^2+h^2} \tan (-\arctan \frac{h}{w}-\theta)$. The width scaling factor is just this divided by the desired width. A similar computation can be done in height, giving the heights $\sqrt{w^2+h^2} \tan (+\arctan \frac{w}{h}-\theta)$ and $\sqrt{w^2+h^2} \tan (-\arctan \frac{w}{h}-\theta)$. and you just choose the minimum scaling factor to make sure it fits.