English notation for hour, minutes and seconds

It's not particularly common for expressions of time.

It's similar to degrees-minutes-seconds: instead of decimal degrees (38.897212°,-77.036519°) you write (38° 53′ 49.9632″, -77° 2′ 11.4678″). Both are derived from a sexagesimal counting system such as that devised in Ancient Babylon: the single prime represents the first sexagesimal division and the second the next, and so on. 17th-century astronomers used a third division of 1/60th of a second.

The advantage of using minute and second symbols for time is that it obviously expresses a duration rather than a time.

From the time 01:00:00 to the time 02:34:56 is a duration of 1 hour, 34 minutes and 56 seconds (1h 34′ 56″)

Prime markers start single and are multiplied for susbsequent appearances, so minutes use a single prime ′ and seconds use a double-prime ″. They are pronounced minutes and seconds respectively in the case of durations like this.

Note that a prime ′ is not a straight-apostrophe ' or a printer's apostrophe ’, although straight-apostrophes are a reasonable approximation and printer's apostrophes do occur as well.

The ' and " are widely used in maps. They're hardly ever used to indicate time anymore. If you use them, be prepared for some strange looks.

The best option is hh:mm:ss. If you're only showing a pair of digits, the context will tell the reader whether it's hh:mm or mm:ss.

You can also suffix the digits with the unit, such as 1h 12m 23s but this gets long and if you're aligning many such intervals it can be difficult to compare.