Solve the system $ x \lfloor y \rfloor = 7 $ and $ y \lfloor x \rfloor = 8 $.
Solution 1:
Reformulate: Let $a,b \in [0,1)$ and $m,n \in \mathbb Z$. Solve $$(m+a)n = 7\\ (n+b)m = 8$$ Rearrangement gives $$nm = 8 - bm = 7 - an$$ The product on the left is an integer so we already know that $bm, an\in \mathbb Z$ or in other words $$a = \frac kn; \quad b = \frac lm$$ With $k,l\in\mathbb Z$. Substituting this back gives us $$mn = 8-l = 7-k$$ Eliminating $l = k + 1$ so we get a solution by solving the integer equation $$mn = 7-k; \quad k < \min(n, m-1)$$ Chose $k=1,m=3,n=2$ and get $l=2$ and finally $$x = m + a = m + \frac kn = 3 + \frac12 = \frac 72\\ y = n + b = n + \frac lm = 2 + \frac 23 = \frac 83$$
Thanks to @Berrick Fillmore for the full set of solutions obtained from the last system:
$$\left\{ \left( 7,\frac{8}{7} \right), \left( \frac{7}{2},\frac{8}{3} \right), \left( - \frac{7}{8},-8 \right), \left( - \frac{7}{4},-4 \right), \left( - \frac{7}{3},- \frac{8}{3} \right), \left( - \frac{7}{2},-2 \right) \right\}$$
Solution 2:
Note: Here we use the following characterization of the floor function \begin{align*} \lfloor x\rfloor = p\qquad \text{ with } \qquad p \leq x < p+1, \quad p\in \mathbb Z \end{align*} We use this relationship to transform the system of equations into a system of inequalities which can be easily analysed.
Let $x,y \in \mathbb{R}$. The system of equations \begin{align*} x \lfloor y \rfloor & = 7 \tag{1}\\ y \lfloor x \rfloor & = 8 \end{align*} is equivalent with \begin{align*} x p & = 7 \qquad \text{ and }\qquad p\leq y < p+1, \quad p\in \mathbb{Z}\\ y q & = 8 \qquad \text{ and }\qquad q\leq x < q+1, \quad q\in \mathbb{Z} \end{align*} We substitute $x=\frac{7}{p}$ and $y=\frac{8}{q}$ and get the following system of inequalities with $p,q\in \mathbb{Z}$ \begin{align*} p\leq \frac{8}{q} < p+1\tag{2}\\ q\leq \frac{7}{p} < q+1\tag{3} \end{align*}
Note that the two inequalities are equivalent with the two equations in (1).
We are now looking for integer solutions of these inequalities. First we see from (1) that $x\neq 0$ and $y\neq 0$. Then we deduce from (2) that either both $p$ and $q$ are greater zero or both are less than zero.
First step: $p,q>0$
Since $q > 0$ we conclude from (3) that $1\leq p \leq 7$ and we obtain
\begin{align*} p=1 \qquad \rightarrow \qquad q\leq &\frac{7}{1}<q+1 \qquad \rightarrow \qquad q=7\\ p=2 \qquad \rightarrow \qquad q\leq &\frac{7}{2}<q+1 \qquad \rightarrow \qquad q=3\\ p=3 \qquad \rightarrow \qquad q\leq &\frac{7}{3}<q+1 \qquad \rightarrow \qquad q=2\\ p=4 \qquad \rightarrow \qquad q\leq &\frac{7}{4}<q+1 \qquad \rightarrow \qquad q=1\\ &\ldots\\ p=7 \qquad \rightarrow \qquad q\leq &\frac{7}{7}<q+1 \qquad \rightarrow \qquad q=1\\ \end{align*}
Checking these pairs $(p,q)$ in inequality (2) we see, that only $p=1$ and $p=2$ are valid.
\begin{align*} (p,q)=(1,7) \qquad \rightarrow \qquad 1\leq \frac{8}{7} < 2\tag{4}\\ (p,q)=(2,3) \qquad \rightarrow \qquad 2\leq \frac{8}{3} < 3\\ \end{align*}
Since $x=\frac{7}{p}$ and $y=\frac{8}{q}$ we get from (4) the solutions with positive $x,y$ \begin{align*} A=\left\{\left(7,\frac{8}{7}\right),\left(\frac{7}{2},\frac{8}{3}\right)\right\} \end{align*}
$$ $$
Second step: $p,q<0$
First we note, that if $p<-7$ it follows from (3) that $q=-1$. Putting $q=-1$ into (2) we obtain $p\leq -8<p+1$ and $p=-8$ follows.
We conclude that in the case $p<-7$ only $(p,q)=(-8,-1)$ is a valid solution.
Next we consider $-7 \leq p \leq -1$ similar to above
\begin{align*} p=-1 \qquad \rightarrow \qquad q\leq &-\frac{7}{1}<q+1 \qquad \rightarrow \qquad q=-7\\ p=-2 \qquad \rightarrow \qquad q\leq &-\frac{7}{2}<q+1 \qquad \rightarrow \qquad q=-4\\ p=-3 \qquad \rightarrow \qquad q\leq &-\frac{7}{3}<q+1 \qquad \rightarrow \qquad q=-3\\ p=-4 \qquad \rightarrow \qquad q\leq &-\frac{7}{4}<q+1 \qquad \rightarrow \qquad q=-2\\ &\ldots\\ p=-7 \qquad \rightarrow \qquad q\leq &-\frac{7}{7}<q+1 \qquad \rightarrow \qquad q=-1\\ \end{align*}
Checking these seven pairs $(p,q)$ in (2) shows that the following three pairs are valid \begin{align*} (p,q)=(-2,-4)\qquad\Rightarrow\qquad -2\leq- \frac{8}{4}<-1\\ (p,q)=(-3,-3)\qquad\Rightarrow\qquad -3\leq- \frac{8}{3}<-2\\ (p,q)=(-4,-2)\qquad\Rightarrow\qquad -4\leq- \frac{8}{2}<-3\\ \end{align*}
Together with $(-8,-1)$ we derive from $x=\frac{7}{p}$ and $y=\frac{8}{q}$ the following $(x,y)$ pairs with negative solutions \begin{align*} B=\left\{\left(-\frac{7}{2},-2\right),\left(-\frac{7}{3},-\frac{8}{3}\right),\left(-\frac{7}{4},-4\right),\left(-\frac{7}{8},-8\right)\right\} \end{align*}
Note that $A\cup B$ coincides with the solution of @AlexR.
Solution 3:
Combining equations can work too. Eliminating $x$: $$y\left\lfloor\frac{7}{\left\lfloor y\right\rfloor}\right\rfloor=8\text{.}$$
In this equation, you are taking the floor of a rational number with $7$ as its numerator. If $y$ is large in absolute value ($8$ or above, or smaller than $-6$), then $\left\lfloor\frac{7}{\left\lfloor y\right\rfloor}\right\rfloor$ will either be $0$ or $-1$. So that leaves $y\in[-6,8)$. Of course $y$ cannot be in $[0,1)$ or the first original equation is false.
For $y\in[-6,0)\cup[1,8)$, the possible values of $\left\lfloor y\right\rfloor$ are $\{-6,-5,\ldots,-1\}\cup\{1,2,\ldots,7\}$.
Running through these, the values of $\left\lfloor\frac{7}{\left\lfloor y\right\rfloor}\right\rfloor$ can only be $\{-2,-3,-4,-7,7,3,2,1\}$. So all together with the earlier noted $0$ and $-1$, $\left\lfloor\frac{7}{\left\lfloor y\right\rfloor}\right\rfloor$ can only take the values $$\{-7,-4,-3,-2,-1,0,1,2,3,7\}\text{.}$$
Put these values back in to $$y=\frac{8}{\left\lfloor\frac{7}{\left\lfloor y\right\rfloor}\right\rfloor}$$ to determine that $$y\in\left\{-\frac87,-2,-\frac83,-4,-8,8,4,\frac83,\frac87\right\}$$ Since $x=\frac{7}{\lfloor y\rfloor}$, this corresponds to $$x\in\left\{-\frac72,-\frac72,-\frac73,-\frac74,-\frac78,\frac78,\frac74,\frac72,7\right\}$$ Out of these 9 possible solution pairs, checking each against the original two equations reveals that six are in fact solutions. The pairs that fail are $(-7/2,-8/7), (7/8,8)$, and $(7/4,4)$. So the solutions are $$\left\{\left(-\frac72,-2\right),\left(-\frac73,-\frac83\right),\left(-\frac74,-4\right),\left(-\frac78,-8\right),\left(\frac72,\frac83\right),\left(7,\frac87\right)\right\}$$
Solution 4:
Let $x=m+\xi$, $y=n+\eta$ with $m$, $n\in{\mathbb Z}$, and $\xi$, $\eta\in[0,1[\ $. Then we have to solve $$(m+\xi)\>n=7,\qquad(n+\eta)\>m=8\ .\tag{1}$$ (i) If $m\geq1$, $n\geq1$ then $(1)$ implies $$m=\left\lfloor{7\over n}\right\rfloor,\qquad n=\left\lfloor{8\over m}\right\rfloor\ .\tag{2}$$ The following table contains in the first row possible values of $n$, in the second row the resulting values of $m$ according to $(2_1)$, and in the third row the resulting values of $n$ according to $(2_2)$: $$\matrix{ n&&1&2&3&4&5&6&7&\geq8 \cr m:={\displaystyle\left\lfloor{7\over n}\right\rfloor} &&7&3&2&1&1&1&1&0\cr {\displaystyle\left\lfloor{8\over m}\right\rfloor} &&1&2&4&8&8&8&8&-\cr}$$ A value $n$ in the first row is usable if the corresponding entry in the third row coincides with $n$. This leads to the usable pairs $(m,n)\in\bigl\{(7,1), (3,2)\bigr\}$, from which one computes the candidates $$(x,y)\in\left\{\left(7,{8\over7}\right), \left({7\over2},{8\over3}\right)\right\}\ .$$ (ii) If $m':=-m\geq1$, $n':=-n\geq1$ then $(1)$ implies $$(m'-\xi)\>n'=7,\qquad (n'-\eta)\>m'=8\ ,$$ or $$n'=\left\lceil{8\over m'}\right\rceil,\qquad m'=\left\lceil{7\over n'}\right\rceil\ .\tag{3}$$ The following table contains in the first row possible values of $m'$, in the second row the resulting values of $n'$ according to $(3_1)$, and in the third row the resulting values of $m'$ according to $(3_2)$: $$\matrix{ m' &&1&2&3&4&5&6&7&\geq8 \cr n':={\displaystyle\left\lceil{8\over m'}\right\rceil} &&8&4&3&2&2&2&2&1\cr {\displaystyle\left\lceil{7\over n'}\right\rceil} &&1&2&3&4&4&4&4&7\cr}$$ A value $m'$ in the first row is usable if the corresponding entry in the third row coincides with $m'$. This leads to the usable pairs $(m',n')\in\bigl\{(1,8), (2,4),(3,3),(4,2)\bigr\}$, from which one computes the candidates $$(x,y)\in\left\{\left(-{7\over8},-8\right), \left(-{7\over4},-4\right),\left(-{7\over3},-{8\over3}\right),\left(-{7\over2},-2\right)\right\}\ .$$ It remains to check that all six candidate pairs $(x,y)$ that we have found indeed satisfy the original equations.