Why aren't there two equivalent ways to set up the limits of integraion for P(X>Y)?

Let's say I have a joint probability density function for two random variables $X$ and $Y$ given by $f_{XY}(x,y)=\frac{6}{7}(x^2+\frac{xy}{2})$. The question is to find the probability that $X>Y$. To do this I set up the following integral:

$$P(X>Y)=\int_0^2\int_y^1f(x,y')\,dx\,dy'$$

When I evaluate this integral I get -1 as the result, and I double checked my work with wolfram alpha. So I tried setting up the other way the integral makes sense:

$$P(X>Y)=\int_0^1\int_0^xf(x',y)\,dy\,dx'$$ and I got $\frac{15}{56}$ Which makes much more sense as an answer.

However I would have expected both integrals to evaluate to a valid answer, since the limits and order of integration for both seem to capture the idea that we are integrating over a region where X>Y. To me this is just the difference of saying X>Y or Y<X....I must be missing something simple!

You have missed to type the support of the density function. Based on your work and the given density function, I assume it is $0 \lt x \lt 1, 0 \lt y \lt 2$

$f_{XY}(x,y)=\frac{6}{7}(x^2+\frac{xy}{2}), 0 \lt x \lt 1, 0 \lt y \lt 2$

The second integral to evaluate $P(X \gt Y)$ is correct. In the first integral, the bounds of $Y$ is not correct . As $X \gt Y$ and $X \lt 1$, we must have $Y \lt 1$.

So the first integral should be,

$ \displaystyle \int_0^1 \int_y^1 \frac{6}{7} \left(x^2 + \frac{xy}{2}\right) dx ~dy = \frac{15}{56}$

See the diagram for further clarity.