What's a word for being curious in a worried manner?

Concerned:

• feeling or showing worry or solicitude.

The Free Dictionary

What do you think about anxious - worried and tense because of possible misfortune, danger, etc; uneasy.

Alarmed, I leaned over to have a look for myself.

If curious is your best adjective and you have already stated quite clearly the type of curiosity, I would suggest "apprehensively curious" for your solution.

There is no word that exemplifies curiosity with an undertone of worry. Perhaps this is because the two are somewhat in opposition. Saying "curious, yet apprehensive" might actually be more appropriate.

Would worried about fit?

worry about [sth/sb] vtr phrasal insep

be concerned or anxious about