Is there any drawback to just renaming jpeg files to jpg?

JPEG stands for Joint Photographic Experts Group. JPG (pronounced jay-peg) is the most commonly used file extension used to identify files created with this lossy format, and is the same as .jpeg. JPEG is a bitmap compression format for picture and image files with compression ratios ranging from 10:1 to 20:1. Older DOS-based computers were designed to handle a maximum "3-character file extension" which is why JPG was attributed to compressed image files. Newer Operating systems such as Windows XP and Vista allow for longer file extensions as evidenced by ".html". Accordingly, the JPG file extension was upgraded to the JPEG file extension which is the true acronym for Joint Photographic Experts Group. Just as a side note, XP and Vista will also support the older JPG file extension.

Reference

Extensions have no effect on the file itself.

The difference might be having different applications associated with them, in which case renaming the file might cause it to be opened by a different application by default.