Is there a downside to going multiple days without sleeping?

While adventuring and completing side quests, I'd realized I'd gone days without stopping to rest and cashing in my experience (thanks to those nighttime hunts). My party has started complaining of being tired and looking forward to a good night's rest. Are their conversations just flavor or are there any negative effects to keeping my party up for multiple days without sleeping?

Apparently there is a penalty for going more than 3 days without sleep, according to the wikia page on Lodgings:

The party can go 3 days without sleep, after which they lose any status buffs.

According to an an interview with Final Fantasy XV's director:

The way it works is, if you don't sleep for long periods, you don't get a negative status – rather it's the opposite. "Buffs" are statuses with positive effects, and you stop getting those. Basically, things like increased spirits or feeling good or other added bonuses stop getting applied if you don't sleep for long periods of time.

However, you can maximize your EXP gains by refraining from sleeping. If you rack up a ton of EXP and spend 30k to sleep in the nice hotel that gives you 3x EXP boost (it's in Altissia, talk to the receptionist on the right), you will be able to level up a lot more than if you sleep every night, as you probably can't afford to shell out 30k every night.

Past chapter 9, you can return to sleep in this hotel by talking to Umbra.

From personal experience, I've run for 2 in-game days straight (gotta max out Survival!) and fought every dropship along the way, as well as captured an outpost. I didn't notice any penalties during these fights.

Apart from the occasional complaint of your crew there isn't really any penalty involved with being awake for several days.

Now this doesn't mean it's a good idea to stay awake forever since the only way to level up is sleeping.

After some time you will simply be too low of a level to fight certain enemies (especially enemies that can lock onto you with ranged attacks) and you might not be able to progress any further without the extra stats you'd get from leveling up.

Also If you decide to stay awake for longer during low levels there isn't really much to do during the nighttime apart from running away from high levelled monsters.

Sources: Haven't slept in 12 days.