Does gatekeeper not check apps stored in /Applications/?

If I remember correctly executables are only checked by Gatekeeper if they have the quarantine filesystem attribute attached. And the quarantine filesystem attribute is (optionally) added by applications which download files from the internet.

So if Gatekeeper is not checking files it could be because the quarantine attribute has already been removed (after a successful check) or because they were downloading using an application that doesn't apply them in the first place (for example I think some torrent applications don't apply them, whereas most if not all web browsers do).

For example I just downloaded Firefox using the Safari web browser so if I run ls -l@ Firefox\ 54.0.1.dmg to show any attached file system attributes I get the following, which includes the quarantine tag...

-rw-r--r--@ 1 alistair  staff  59438170  7 Jul 00:23 Firefox 54.0.1.dmg
    com.apple.metadata:kMDItemDownloadedDate          53 
    com.apple.metadata:kMDItemWhereFroms         203 
    com.apple.quarantine          61 

This is why you see people suggesting removing the quarantine tag as a way to get around the Gatekeeper security prompts.