Is there a reliability difference between 2.5" and 3.5" HDDs?

No not really, 2.5 and 3.5 inch drives are mostly the same, including interface and whatnot but 3.5 inch drives are usually faster and larger in space because there is more stuff you can fit into 2.5 vs 3.5 inches. 3.5s are cheaper because of space constrains but usually no more reliable than a 2.5 and vise versa. (in practice)If you could fit a 3.5 drive do it, it'll be cheaper and bigger(storage wise)

You can't base that just on the form-factor. What's way more important is how the vendor dedicates the hard disk. There are consumer disks with both sizes, which are mainly meant for running just a few hours a day and are lightly loaded.

Enterprise disks (SAS, nearline-SATA) are dedicated for 24/7 usage and higher loads. So if you need reliability, use those.

The most important characteristic to consider when choosing between 3.5" and 2.5" is typically size and energy consumption. If you are tight on both, you should choose 2.5", else you could go with 3.5" which are typically cheaper per GB.

The real question is what are you you trying to do? If you have a choice between two enclosures, check with the vendor(s) to see what disks are being put in, and see what the warranty period and the manufacturers MTBF on the drives.

Personally, I have seen 2.5" drives fail on me more frequently than 3.5" drives, and I have seen 2.5" drives have 1-3 year warranties versus 3-5 year warranties for 3.5" drives. To be sure, 2.5" drives are more likely to be installed in laptops, which tend to bounce around and have less ventilation.