Graduate school self-doubt (currently an undergraduate)

You've gotten some good advice so far; permit me to chime in with the perspective of someone on the other side of the lectern. I've been teaching at small liberal arts colleges for longer (far longer) than any of my current students have been alive and over the years my colleagues and I have seen many of our majors accepted at good-quality graduate schools, both for Master's and Ph.D. programs. Let me get your major question out of the way immediately

[I]f you go to a small school, are you doomed to fail right from the get-go?

This is easy to answer: no! as I've mentioned above.

Now that that's out of the way, I first need to back up a little and ask you the question you should ask yourself,

Why do I want to get a Ph.D.?

The process of obtaining a Doctorate is long and, frankly, painful. If you've decided put your life on hold for five to seven hard and exasperating years, you'd better have a good reason. If your goal is to continue doing mathematics and eventually producing results of your own that add to the body of what is known, that's commendable. If you view the Ph.D. as a high-falutin' teaching certificate and see yourself as primarily a college professor, that's also commendable. If, on the other hand, you're contemplating merely drifting into a way to keep doing what you're good at, it might be a good idea to ask yourself whether there might be some other way to spend the next few years, since a Doctorate in mathematics won't help you very much in the Real World outside of academia and may actually harm more than help in eventually landing a job out there.

For the purpose of discussion, let's assume that you have thought long and hard and decided, "Yup, I really want to get that diploma," how can you get your foot in the door? You've earned superb grades in what looks to me to be a reasonably solid undergraduate curriculum. For all but the applicants for the very top schools, that will work in your favor, though you're correct in your assessment that good grades will be a small part of the admissions committee's view of you. Similarly, the GRE will also be a small part of your total package---those scores will primarily be a check for the Committee to ensure that you're not a hopeless idiot. On the other hand, you probably won't have any Fields medalists among your letters of reference, but that's not as bad as it may seem, especially if any recent graduates from your college have made it into a school for which you're applying.

Reference letters are the single most important part of your package, hands down. No matter where you did your undergraduate work and no matter who's writing your letters, the sentence "X was the best student I've seen in ten years" will loom large in the eyes of the Committee. If your English is excellent, make sure one or more of your letters mentions that, since universities are always on the lookout for good teaching assistant material.

Is there an area of math that really excites you? If so, check for universities with strong programs in that area and mention that in your personal statement. It's already been mentioned that it might be a good idea to write to faculty members at your target school and ask about what they're doing in the area of your interest before you submit your application.

That said, where should you apply? I agree that a top-tier graduate school is probably a stretch, though that shouldn't rule out trying for one or two. Who knows, you might get lucky? On the other hand, you probably shouldn't limit yourself to universities whose motto on their seal is "Plenty of Free Parking." There are plenty of respectable universities between those two extremes and for the time being they still need a crop of good applicants (which you are). As I said, we've sent a lot of students off to grad schools and I can't recall a case where one of our students failed to get into at least one of the schools to which they applied. Your task over the next few months is to research those middle-tier schools that look like the best fit for you.

As Ragib and Francis mentioned, getting a Master's degree first isn't all that bad an idea. At least, it might give you an idea about whether going on for the Doctorate is what you want to do (that's what I did, though for different reasons). If you go that route, do your best to make yourself stand out from the crowd, so your subsequent letters will reflect that. Keep in mind, though, that you'll have to dig up the money to pay for your education, since graduate schools rarely offer financial support for Master's students (though many will provide full support for Ph.D. students).

Finally, I fully agree with some of the other answers: don't be discouraged. Things are nowhere near as bad as you've painted them. If that's what you want, you'll almost certainly succeed. Best of luck---keep us posted.

I think a good place to start is not to be so down on your school. You didn't go to princeton, but it doesn't sound like you went to a bad school. And at your school it sounds like you have done exceptionally well and have tried to do as much as possible within the limitations of your situation. If you had gone to an "obscure" school and done poorly, then there might be an issue.

One thing to emphasize, both in whatever statements you write for the applications and in your recommendations is your potential. Tell the people writing your letters about all the extra work you have done to try and empress upon them the effort you have put in so far and would be willing to put in at a graduate school. While your recommenders may not be well known, at a small school they do have the chance to get to know you really well and that benefit shouldn't be underestimated.

Once you have an idea of what you would like to study and where you would like to go, try contacting some people at those schools in those areas. Tell them who you are, what you would like to do, and if they had any advice for someone preparing for graduate school. Ask for some papers that would serve as good introductions to research. Many will be happy to do that for you, and you will have a contact at that school that knows who you are and sees that you already want to start doing some work. These are both good things.

And when you are looking at schools, apply to a wide range. When I applied at the end of my senior year (also at a small midwestern liberal arts school), I didn't get in anywhere. So I took a year off, took a grad class at the University of Iowa, got a job at a community college, and reapplied. The second time I lowered the general range of schools, still applying to some good ones but including more lower tier programs, and I got in.

I think it worked out for me. Just because you don't go to a great school doesn't mean you can't get in somewhere. And once you are in you can do the work to make a name for yourself. You can try a masters first, or go straight to a PhD, but you can still succeed.

Currently the road I'm considering: Aim for smaller steps up the ladder.

If we were to divide schools up into 3 levels, where your current one is 3, your desired PhD program at 1, then apply to do Masters degrees at schools of level 2. It will probably have to be a 2 year Masters (instead of a 1 year) because otherwise you have to file PhD applications before you can get good recommendations.

This isn't necessarily 2 years wasted, because in this time you do many things you would otherwise do in the first year or two of a PhD program, such as learning more mathematics and honing in on what area you want to research in. If you use those 2 years wisely, impress the faculty and work your butt off, then you could end up with a strong enough case to get into your goal PhD school.

Digression: I'd just like you (the OP) to know, this is a problem that keeps me up at night as well. The best we can do is keep trying, work hard and have hope.

Encouragement: Have more self-confidence!

Bertrand Russell: “The trouble with the world is that the stupid are so confident while the intelligent are full of doubt.”

Despite your self-doubt, you are clearly an intelligent guy (I believe few people take topology in their first year).

Anyway, I believe that not being in a famous big-name school is not the end of the world. Many mathematicians (recently see http://en.wikipedia.org/wiki/Liu_Lu who solved the Seetapun Enigma) did not come from the best of universities. Even Galois was rejected from Ecole Polytechnique -- twice.

Ultimately, I think it is the individual that matters most; the university is important but still a secondary matter.

Wishing you all the best of luck. I need it too -- I am in the same situation as you, only worse as I am not studying in U.S.