Why not define 'limits' to include isolated points?
If I understand correctly, most definitions of 'limits' require that the function either a) be defined in an open neighborhood around the relevant point or b) more permissively, that the relevant point is a limit point; the definition of 'continuity' is then given a special case so that functions are continuous at isolated points. Why not extend the notion of 'limit' so that the limit of a function at an isolated point is just whatever the function's value is there? Is there some good reason not to?
Solution 1:
You could make that definition I suppose, but what use would it have, and how would it relate to the usual notion of limit?
Let's look at what a limit of a function $f$ at a point $x$ should mean (let's say for a real-valued function on a metric space). You want the limit of $f$ at $x$ to be a real number $L$ such that for all $\varepsilon>0$ there exists a $\delta>0$ such that $0<d(x,y)<\delta$ implies that $|f(y)-L|<\varepsilon$. Now if $x$ is an isolated point, then you could take any $L$ you want, because whatever $\varepsilon$ is, you can choose $\delta$ sufficiently small so that $|f(y)-L|<\varepsilon$ is vacuously true for all $y$ with $0<d(x,y)<\delta$; that is, just make sure that there are no $y$ satisfying the latter condition. For non-isolated points, limits are unique. For isolated points, trying to extend the usual definition leads to making every real number a limit.
For one of the applications of limits, namely continuity, not defining the limit at an isolated point causes no problem. You can say that $f$ is continuous at a point $x$ in the domain if for all $\varepsilon>0$ there exists a $\delta>0$ such that $d(x,y)<\delta$ implies $|f(y)-f(x)|<\varepsilon$. If $x$ is an isolated point, then this will always be true, because for sufficiently small $\delta$ the only $y$ with $d(x,y)<\delta$ is $x$.
Added: I was writing when Alex posted, and part of my post makes a similar point to his. Qiaochu's comment on Alex's post gives an answer to my question at the beginning of my post. Making this definition allows continuity to be defined in terms of respecting limits without making isolated points a special case, something I had overlooked.
Nonetheless, continuity can be defined in terms of respecting limits without actually defining the limit of a function at a point. A function $f$ between metric spaces [resp. topological spaces] is continuous at $x$ if for every sequence [resp. net] $(x_n)_n$ in the domain converging to $x$, $\lim_n f(x_n)=f(x)$. In case $x$ is an isolated point, a sequence converging to $x$ is eventually constantly equal to $x$, so this will be satisfied.
Solution 2:
Okay, now that I've checked Rudin to see that this really is the definition he gives of a limit of a function, I have an answer, but I don't like it. The motivation behind the definition of $\lim_{x \to a} f(x)$ is that you want to understand what $f$ is doing in a neighborhood of $a$ in order to compare it to what is happening at at $a$. If $a$ is an isolated point, there's nothing to compare.
Solution 3:
The entire point of the notion of a 'limit' is to capture the behavior of a function as it gets "close" to a point, which is not possible for isolated points, thus there is no utility to extending the notion in the way you described. Your definition would make "limit" the same as "value" at these points, which is not very useful.