There is a "$REYCLE.BIN" folder in my external hard drive; can I delete it?
Personally I would not delete it, I would just disable it for that drive. It won't hurt to have on empty hidden folder on your drive.
Right click on Recycle Bin>Properties and highlight the drive. Then change the "Settings for selected location" to "Don't move files to the Recycle Bin. Remove files immediately when deleted."