Disabling Zeitgeist
Solution 1:
Run sudo apt-get remove zeitgeist* in a terminal. It might remove a few related packages, but so long as you don't see anything suggested for removal that you don't want, you should be fine.
As per comments: you will lose unity functionality (I was not sure how much when I first wrote the answer) if you remove zeitgeist. So, I suggest you just disable logging.
You can also turn off logging altogether from system settings.
Solution 2:
There is an easy way described here -- which does not include deleting any package, but simply de-activating the corresponding services. I didn't try it myself, but this way side-effects should be minimal (and probably even wanted, as e.g. no "recent items" being available etc. Moreover, it is easily revertable in case of "unwanted" side effects.
The author of that article explicitly states: Here is a way to disable this logging without breaking Unity or any other part of the system, execute these commands in a terminal:
sudo mv /etc/xdg/autostart/zeitgeist-datahub.desktop /etc/xdg/autostart/zeitgeist-datahub.desktop-inactive
rm ~/.local/share/recently-used.xbel
mkdir ~/.local/share/recently-used.xbel
rm -rf ~/.local/share/zeitgeist
Reverting this (if one wants to enable Zeitgeist again) thus should be done like this:
mkdir ~/.local/share/zeitgeist
rmdir ~/.local/share/recently-used.xbel
# no action for the .xbel file, as it will be re-created automatically
sudo mv /etc/xdg/autostart/zeitgeist-datahub.desktop-inactive /etc/xdg/autostart/zeitgeist-datahub.desktop
which is simply "reversing" the steps of the de-activation.
Remark: The ~/.local/share/recently-used.xbel has nothing to do with Zeitgeist itself, it just falls into the same category somehow. The article mentioned is about "disabling activity logging", and thus included this. If you just want to disable Zeitgeist, you can skip the two middle steps.
Solution 3:
If you remove Zeitgeist then the menu in Unity wont work properly, it stops you searching for programs by typing into the box.