Does it matter where my Bus Depot is?

I think you've got it right, it really is that simple. A single bus depot is sufficient to supply enough buses to a city of any size¹, and its position isn't really important as long as it is connected by road to any bus routes you create (if your city isn't entirely road-connected you'll need a depot in each area you want bus routes in). 

The only other thing to be concerned about is that depots create 50 noise pollution, so you probably don't want them nestled into the middle of your residential areas.

If you care about fine details, buses will need to travel from/to the depot to get to/from bus routes, this only happens when routes are created/edited/deleted (as you say, they never rest). This means new bus routes set further away from a depot take longer to start working. Also, if you change your city's bus funding (changing the number of buses on all of your lines), you might find a swarm of buses around your depot. These are temporary issues that you can probably ignore.

_{Also, I guess if you only have one depot and it catches fire you might have a problem, but meh.}