Is "especially" always an adverb?

We must be wary of your surroundings, especially in the city.

Yes, "especially" is an adverb (a type of focusing modifier), but it's not a modifier of the VP; rather, it's a modifier in the structure of the PP "especially in the city".

The comma marks the PP as a supplementary adjunct, a loosely attached expression set off by intonation and by punctuation, presenting supplementary non-integrated information.

Supplements don't modify other elements but have a semantic 'anchor' that they refer to. Here, the anchor is the entire preceding clause "we must be wary of your surroundings"