Who installed this Emacs?

I have:

$ /Applications/Emacs.app/Contents/MacOS/Emacs --version
GNU Emacs 24.3.1
$ /usr/bin/emacs --version
GNU Emacs 22.1.1

How do I get rid of the second installation of Emacs?

The second emacs came with OS X. For files that are logged in the package database, you can use pkgutil:

$ pkgutil --file-info /usr/bin/emacs
volume: /
path: /usr/bin/emacs

pkgid: com.apple.pkg.BSD
pkg-version: 10.9.0.1.1.1306847324
install-time: 1382483268
uid: 0
gid: 0
mode: 555

A list of packages can be obtained via pkgutil --packages. However, note that—again—this only covers items in the package database.

You should not modify items that are part of the core operating system. If you prefer a locally-installed program to override one that came with the operating system, you should adjust your PATH to prefer that program before system paths such as /usr/bin are checked.

One way to do this is to have /usr/local/bin be first in your path and link your /Applications version of emacs to /usr/local/bin so that it will be called first unless a program explicitly calls /usr/bin/emacs

The file /usr/bin/emacs is installed as part of Mac OS X. It is generally not a good idea to change or modify files that are installed as part of the default OS installation; other programs can rely on their presence.

Utilities installed as part of the operating system can be invoked by other programs with the expectation that the version installed will match the operating system spec. Removing them or replacing them with a different version could cause other programs to fail to operate as designed.