Is it a good idea to seal off the optical drive bay after installing a second HDD?

Depends on your bracket/caddie style, but for peace of mind I would advise using electrical tape over the gap on the inside, any exterior covering cannot be less intrusive than just ignoring the hole. Airflow etc is not an issue.

Debris entering the Mac isn't a huge concern as there are slots larger than the optical bay and unless your optical drive has massive amounts of conductive metal debris in it - leaving the felt-like skirt in place should suffice for all but the most messy environments.

Unless you are also sealing the heat exhausts by the display hinge - sealing the optical slot won't matter either for good or for bad, so I'd say it's not a good idea to spend time/effort to further seal that slot.