How to calculate the expectation of $XY$?
Solution 1:
In general, for jointly continuous random variables $X$ and $Y$ with joint pdf $f_{X,Y}(x,y)$, $$E[g(X,Y)]=\int_{-\infty}^\infty\int_{-\infty}^\infty g(x,y)f_{X,Y}(x,y)dx dy.$$ In the special case you are considering, this becomes $$E[XY]=\int_{-\infty}^\infty\int_{-\infty}^\infty xyf_{X,Y}(x,y)dx dy.$$
If $X$ and $Y$ are jointly continuous random variables with joint pdf $f_{X,Y}(x,y)$, and $f_{X,Y}(x,y)$ factors into the product of the marginal pdfs $f_X(x)$ and $f_Y(y)$, then $X$ and $Y$ are said to be independent random variables. More useful is the reverse implication: if we assume that $X$ and $Y$ are independent continuous random variables with known pdfs (e.g. standard normal), then they are jointly continuous with joint pdf $f_{X,Y}(x,y)$ equal to the product $f_X(x)f_Y(y)$ of their individual pdfs.
Your expression $\displaystyle E[XY] = \int_{R\times R} xyf_X(x)f_Y(y)dxdy$ is incorrect in the general case, but is correct when $X$ and $Y$ are independent continuous random variables since $f_{X,Y}=f_X(x)f_Y(y)$ in this case. Indeed, if your expression were correct in general, then we would have $$E[XY] = \int_{R\times R} xyf_X(x)f_Y(y)dxdy = \int_{R} xf_X(x)dx \int_{R} yf_Y(y)dy = E[X]E[Y]$$ so that $\text{cov}(X,Y)=E[XY]-E[X]E[Y] = 0$ for all random variables, which is clearly not true. So we have the following.
If $X$ and $Y$ are independent random variables, then $E[XY]=E[X]E[Y]$.
Note that this holds for all random variables, not just continuous random variables. Also, as you probably know, the converse is not true: uncorrelated random variables need not be independent.
With regard to your second question, $X$ and $Y$ are independent if you can find $g(x)$ and $h(y)$ such that the equality $f_{X,Y}(x,y)=g(x)h(y)$ holds at all points $(x,y)$ in the plane, not just at some points. If the joint pdf is nonzero only for $0<x<1,x<y<2x$, then $X$ and $Y$ are dependent random variables; no need to try and see if you can express $f(x,y)$ as $g(x)h(y)$.
Finally, note that all of the above applies provided the various integrals and expectations are defined or exist. $E[XY]=E[X]E[Y]$ does not apply to independent Cauchy random variables, for example, because $E[X]$ and $E[Y]$ are undefined for Cauchy random variables $X$ and $Y$.