Difficulties with the limit of difference quotients in metric spaces

To expand on the comments already made by Henno Brandsma, for $X=Y=\mathbb{R}$ with the usual Euclidean metric your definition would yield $$f'(x_0)=\lim_{y\to x_0} \frac{|f(x_0)-f(y)|}{|x_0-y|}$$ This means that your definition does not generalize the ordinary definition from Calculus, which is a bad thing. For instance we'd get that the derivative of $y=x^2$ would be $$f'(x) = \lim_{y\to x} \frac{|x^2-y^2|}{|x-y|}=\lim_{y\to x}{|x+y|}=2|x|$$ One of the primary applications of the derivative is to determine in what intervals a function is increasing or decreasing by looking at the sign of $f'(x)$. This definition would not allow us to check this since the derivative would always be non-negative.

It could be that this definition would turn out to be interesting or useful after all, but you shouldn't call it a derivative, because it does not generalize the usual derivative from Calculus.

For another distinction, notice that for $f:\mathbb{R}^2\to\mathbb{R}^2$ we approximate a given function around a point using a linear operator from $\mathbb{R}^2$ to $\mathbb{R}^2$ (the differential, or in terms of matrices, the Jacobian matrix). The definition above would not provide us with this $2$ dimensional structure and only yield a $1$ dimensional answer.