What could be causing twitterbot to take 6s to download small image assets?

Solution 1:

Flags [E] and [W] in the tcpdump output are Explicit Congestion Notifications.

When you see theirsite.36652 > mysite.https: Flags [.E] it means that the receiver detected link congestion and asks the sender (your site) to slow down and the sender responds with Flags [.EW] indicating that it acknowledged the request and reduced TCP window.