You shouldn't need much. Almost all you need to know about topology (especially of the point-set variety) should have been covered in a course in advanced calculus. That is to say, you really need to know about "stuff" in $\mathbb{R}^n$. (The one main exception is when you study instantons and some existence results are topological in nature; for that you will need to know a little bit about fundamental groups and homotopy.) The reason is that differential topology and differential geometry study objects which locally look like Euclidean spaces. This dramatically rules out lots of the more esoteric examples that point-set topologists and functional analysts like to consider. So most introductory books in differential geometry will quickly sketch some of the basic topological facts you will need to get going.

In terms of topology needed for differential geometry, one of the texts I highly recommend would be

  • J.M. Lee's Introduction to Smooth Manifolds

It is quite mathematical and quite advanced, and covers large chunks of what you will call differential geometry also. One can complement that with his Riemannian Manifolds to get some Riemannian geometry also.

But since you are asking from the point of view of a Physics Undergrad, perhaps better for you would be to start with either (or both of)

  • Nakahara's Geometry, Topology, and Physics
  • Choquet-Bruhat's Analysis, Manifolds, and Physics: Vol. 1 and Vol. 2

and follow-up with

  • Greg Naber's two book series: Topology, Geometry, and Gauge Fields: Foundations and Topology, Geometry, and Gauge Fields: Interactions