How much can I improve the Nightingale Bow?

The infinite loop can indeed make the Nightengale Bow the strongest bow in the game. Any bow is capped for damage at a number that's essentially some huge power of two, minus one. Attempting to exceed this number, or have the game display such a number, causes a crash. If people aren't crashing the game they're not using the same loop you've read about.

The infinite loop requires four things: ingredients for restoration potions (cyrodilic spadetails, abecean longfins, salt piles), ingredients for enchanting potions, soul gems, and knowledge of the "fortify alchemy" enchantment. Fortify Smithing would also be useful for you.

For best results, have four (or as many as you can manage) +alchemy skill items as the only items on your favorites bar. Equip them. Make a fortify restoration potion. Drink the potion. Unequip the items using the favorites thing. Re-equip. Make a new restoration potion. Drink it. Repeat. Before long your potions will be 10000% plus. That's when you make the fortify enchanting potions. Use one of them to make a +smithing skill ring. Use the other to make a very high % alchemy skill ring, so you don't have to loop in the future.

If you have trouble with this loop three things help: get alchemy skill perks. You shouldn't need more than two or three. Get enchanting perks. You'll definitely want the +25% skill perk. Go fast. Remember that restoration potions multiply each other a little bit. If your alchemy skill is extremely low for your level some potion sellers might be selling higher skill fortify restoration potions than you can actually make yourself. Go buy one, if you can find it.

Because every weapon has the same theoretical maximum damage (which will display as negative because of how integers work in games, thanks to the convention of the leftmost digit in binary corresponding to positive or negative), it doesn't really matter what weapon you use. If you're using this glitch. Besides, you'll one shot anything in the game once you get your bow over 10000 damage or so, so a simple +50000% smithing enchantment ought to suffice for your weapons.

If you plan on using this loop with the double-enchanting perk to get awesome enchantments on every slot I can't stress enough how important it is to plan them out ahead of time so you don't waste a slot.

First off, bottom line: If you have Daedric Smithing, the max bow you can make will be about 60% stronger than the Nightingale Bow. If you have any Smithing perk, via the Alchemy-Enchanting-Smithing loop you can make that type of bow stronger than the Nightingale Bow simply because the Nightingale Bow is unaffected by Smithing perks (this is true for any Smithing value over 129; max is about 533).

Everyone talks about how armor can be improved infinitely using the Alchemy-Enchant-Smithing loop (thus it's possible to have hide armor that's as strong as dragonscale).

Armor is affected by a cap, so once you get above 567 (80% damage reduction) no amount of improvement matters. That's why weaker armors can effectively be the same as Daedric/Dragonplate/Dragonscale. Weapons have no such cap.

Using the Alchemy-Enchanting loop, you can indeed improve the Nightingale Bow drastically, however, keep in mind that it is not affected by any perks. Although the best version has the same base damage (18) as a Daedric Bow, if you're taking perks in things like Daedric Smithing, the latter will be significantly stronger.

It's impossible to predict how "strong" it will appear in your inventory because that calculation depends on factors like Archery skill level and perks.

With the loop, by the way, I've been able to create a Potion of Fortify Smithing +147% and 4 pieces of Fortify Smithing +29%. I have the Smithing perk tree filled out going the Heavy route, so I have Daedric Smithing. I then improved my Daedric Bow to where it's doing 599 damage for me (a bunch of Fortify Archery +48%, nearly full Archery perk tree). When I did the same for the Nightingale Bow, of which I have the best version, I get to 379 damage. So the max Nightingale Bow will be around 37% weaker than the max Daedric Bow.

Math
Just to think about the math for a second, I believe that gets me to

smithing = base skill * (1 + alchemy fort) * (1 + enchanting fort)
         = 100 * (1 + 1.47) * (1 + 0.29 * 4)
         = 533

Plugging that into improvement with/without the Smithing perk (Daedric Bow = Daedric Smithing, Nightingale Bow = none/without):

with    = floor((smithing - 5) / (103 / 6)) * 1.8
        = floor(30.76) * 1.8
        = 54
without = floor((smithing + 11 / 3) / (103 / 3)) * 1.8
        = floor(15.6) * 1.8
        = 27

Since the base for both is 18, that brings us to 72 for the Daedric Bow and 45 for the Nightingale Bow, or 37.5% less (and the ratio stays the same regardless of skill level/perks). Note also that the above 27 damage difference is greater than the difference between the weakest and strong base damages for bows, therefore the strongest bow will be the best bow type for which you have a Smithing perk. This should always be true if you can buff Smithing over 129 using enchanting/alchemy.

Also, FWIW, by the time you have 100 Enchanting you really should take the Extra Effect perk. That means the bow above can have the same effects as the Nightingale Bow. Not that they matter much when you're one-shotting everything.