What do the different colors of my fleets' firepower mean in Stellaris?

The red fleet power number is telling you that the fleet in its current state is significantly below the power of its template in the Fleet Manager. This can be because either the fleet needs upgrading or reinforcing. The Up arrow to the left of the fleet power indicates that you could use an upgrade.

Your "1st Fleet" on the other hand can also use an upgrade, but as its 14 size is greater than its template's size "(13)", its fleet power is at least that of its template.

I'd need to see more information to explain why your "2nd Fleet" of size 13 has at least the fleet power of its size 15 template.

The new Fleet Manager is a very cool tool. I definitely recommend you get comfortable using it.