Why is Heimskr allowed to continue preaching, if the Empire wins the war?

I've completed the Civil War questline in the Empire's favor in one game. Since then, I've noticed several indicators that Talos worship still hasn't been as thoroughly eradicated as one would have expected. One strong indicator of this is that Heimskr seems to be allowed to continue preaching in Whiterun. Other examples are the continued existence of Shrines of Talos in Markarth and Riften. One might expect these to still remain in the wild somewhere, but not in major cities.

Is there any canonical explanation for this?

According to some NPCs, the real reason for the Thalmor "witch hunts" is the bravado with which Ulfric & Co were professing their faith, and before that the worship of Talos wasn't actually persecuted. Perhaps the Empire is managing to hold off the Justicars to prevent even more tension? Either that or the devs just didn't get to changing those things.

Heimskr was actually supposed to be sent to jail if the Imperials win the battle, as per the Creation Kit. A bug prevents this, but the Unofficial Skyrim Patch (USKP) corrects this.

From the USKP v1.2.4 - (2012-11-21) version history:

When the Imperials get control of Whiterun during the Civil War, Heimskr will now be sent to jail as was intended.

Sources:
UESP wiki, "Heimskr" article
USKP version history

You have a point about the temples. But if you talk to Heimskr he will say how he had to bribe some jailors to continue preaching about TALOS!