Is it possible to change orientation of JPG image without rotating it (and thus losing information)?

JPG can be rotated using only EXIF data, so no re-encoding is necessary.

I found this tutorial on the orientation flag - JPEG Rotation and EXIF Orientation

These days, most image apps can handle this flag, many can also write it.

Exiftool is a very comprehensive multi-platform solution that can handle all exif data - see How can I modify the EXIF orientation tag of an image?