How important are Harbors in Civ V really?

I remember Harbors were quite crucial buildings in Civ IV, but they don't seem to be anymore in Civ V. They only provide a 25% production increase for building sea units and establish a trade route, if one doesn't already exist. Usually one or two advisors are picking the Harbor as the next important building to build, and insist on their choice for centuries.

If I already have an established trade route through roads and am not particularly interested in building sea units though, am I missing out on anything important if I don't build one?

No. I haven't found the advisors to be particularly good at choosing buildings. Most buildings are much more situational now due to the maintenance costs, so you shouldn't build them unless they seem like a good match with what you're planning for that city.

Harbors are best thought of as an alternative to roads and rails, and at times a superior alternative. They essentially turn the sea into a trade route connected to all other cities with harbors. If one of those cities is your capital or a city connected by road/rail to your capital, then building a harbor immediately creates a trade connection. Furthermore, this connection is considered a "railroad" connection, which means a 50% production bonus the moment you research railroads. They accomplish this while only costing 3 gold per turn. Often, it makes economic sense to rip up roads (or at least not lay down railroads) to save on maintenance, as it only takes two tiles of railroads to surpass the expense of a Harbor. The downside, obviously, is a loss of interior lines for military movement.

Also, Harbors are required to connect any cities (captured or otherwise) on landmasses not contiguous with your starting land mass to your trade routes.

Yes... not only can they obviate the need to build roads (which cost per unit length) but also, as well as the trade and production value, importantly they can contribute to happiness... definitely worth while when they can be upgraded to Sea Ports and once again they boost production...