list files not originating from packages

Solution 1:

To answer your question literally, you can list files that do not come from packages this way:

find / -xdev -type f | sort >/tmp/root.files
sort /var/lib/dpkg/info/*.list >/tmp/installed.files
comm -23 /tmp/root.files /tmp/installed.files

That will of course include every file in home directories. You can skip some directories in the find command by adding -prune directives (you'll also need to add -print to print the rest, as the -print action is only implied if there is no action).

find / -xdev -type f -path /home -prune -o -path /var -prune -o -print | sort >/tmp/root.files

To check if a file has changed, you can compare its checksum against the one in /var/lib/dpkg/info/$PACKAGE.md5sum. If you want to compare every file (which will take a very long time):

for p in /var/lib/dpkg/info/*.list; do
  diff <(md5sum $(cat $p) | sort) <(sort ${p#.list}.md5sums);
done

You should not need this, because you should not modify files that come from packages. You should not add your own files in system directories either.

If you modify a configuration file, it lives in /etc; this is the only place where you should modify system files. Install the etckeeper package to keep track of your modifications in /etc. Run sudo etckeeper init, and /etc will be under version control (Bazaar by default).

If you install software system-wide, install it under /usr/local. Don't touch anything in /bin, /sbin, /lib (except manually-installed kernel modules if you need them because you have unusual hardware), or /usr (except /usr/local which is for your own use).

All files in your home directory were created by you (except for a handful that were copied from /etc/skel when you created your account). There's no general way to keep track of which ones are application defaults, which ones result from you selecting configuration options and which ones record current state (open files, command history, etc).

Solution 2:

For your second point, you can use debsums -ac (from the package of the same name) to list all package files (including configuration files) that have been modified.