How to get full control of umask/PAM/permissions?

// Updated Feb 8 - Outstanding issues in brief:

• How to umask directories differently from files?
• How to umask on Nautilus copy/paste?
• How to set umask for SSHFS?

OUR SITUATION

Several people from our company log in to a server and upload files. They all need to be able to upload and overwrite the same files. They have different usernames, but are all part of the same group. However, this is an internet server, so the "other" users should have (in general) just read-only access. So what I want to have is these standard permissions:

files: 664
directories: 771

My goal is that all users do not need to worry about permissions. The server should be configured in such a way that these permissions apply to all files and directories, newly created, copied, or over-written. Only when we need some special permissions we'd manually change this.

We upload files to the server by SFTP-ing in Nautilus, by mounting the server using sshfs and accessing it in Nautilus as if it were a local folder, and by SCP-ing in the command line. That basically covers our situation and what we aim to do.

Now, I have read many things about the beautiful umask functionality. From what I understand umask (together with PAM) should allow me to do exactly what I want: set standard permissions for new files and directories. However, after many many hours of reading and trial-and-error, I still do not get this to work. I get many unexpected results. I really like to get a solid grasp of umask and have many question unanswered. I will post these questions below, together with my findings and an explanation of my trials that led to these questions. Given that many things appear to go wrong, I think that I am doing several things wrong. So therefore, there are many questions.

NOTE: I am using Ubuntu 9.10 and therefore can not change the sshd_config to set the umask for the SFTP server. Installed SSH OpenSSH_5.1p1 Debian-6ubuntu2 < required OpenSSH 5.4p1. So here go the questions.

1. DO I NEED TO RESTART FOR PAM CHANGS TO TAKE EFFECT?

Let's start with this. There were so many files involved and I was unable to figure out what does and what does not affect things, also because I did not know whether or not I have to restart the whole system for PAM changes to take effect. I did do so after not seeing the expected results, but is this really necessary? Or can I just log out from the server and log back in, and should new PAM policies be effective? Or is there some 'PAM' program to reload?

2. IS THERE ONE SINGLE FILE TO CHANGE THAT AFFECTS ALL USERS FOR ALL SESSIONS?

So I ended up changing MANY files, as I read MANY different things. I ended up setting the umask in the following files:

~/.profile -> umask=0002
~/.bashrc -> umask=0002
/etc/profile -> umask=0002
/etc/pam.d/common-session -> umask=0002
/etc/pam.d/sshd -> umask=0002
/etc/pam.d/login -> umask=0002

I want this change to apply to all users, so some sort of system-wide change would be best. Can it be achieved?

3. AFTER ALL, THIS UMASK THING, DOES IT WORK?

So after changing umask to 0002 at every possible place, I run tests.

------------SCP-----------

TEST 1:

scp testfile (which has 777 permissions for testing purposes) server:/home/
testfile                                      100%    4     0.0KB/s   00:00   

Let's check permissions:

user@server:/home$ ls -l
total 4
-rwx--x--x 1 user uploaders 4 2011-02-05 17:59 testfile (711)

UPDATE: fixed by ONLY setting umask in pam.d/common-sessions (see comments)

---------SSH------------

TEST 2:

ssh server
user@server:/home$ touch anotherfile
user@server:/home$ ls -l
total 4
-rw-rw-r-- 1 user uploaders 0 2011-02-05 18:03 anotherfile (664)

--------SFTP-----------

Nautilus: sftp://server/home/

Copy and paste newfile from client to server (777 on client)

TEST 3:

user@server:/home$ ls -l
total 4
-rwxrwxrwx 1 user uploaders 3 2011-02-05 18:05 newfile (777)

Create a new file through Nautilus. Check file permissions in terminal:

TEST 4:

user@server:/home$ ls -l
total 4
-rw------- 1 user uploaders    0 2011-02-05 18:06 newfile (600)

I mean... WHAT just happened here?! We should get 644 every single time. Instead I get 711, 777, 600, and then once 644. And the 644 is only achieved when creating a new, blank file through SSH, which is the least probable scenario.

So I am asking, does umask/pam work after all?

UPDATE: fixed test 4 by ONLY setting umask in pam.d/common-sessions (see comments)

4. SO WHAT DOES IT MEAN TO UMASK SSHFS?

Sometimes we mount a server locally, using sshfs. Very useful. But again, we have permissions issues.

Here is how we mount:

sshfs -o idmap=user -o umask=0113 user@server:/home/ /mnt

NOTE: we use umask = 113 because apparently, sshfs starts from 777 instead of 666, so with 113 we get 664 which is the desired file permission.

But what now happens is that we see all files and directories as if they are 664. We browse in Nautilus to /mnt and:

• Right click -> New File (newfile) --- TEST 5
• Right click -> New Folder (newfolder) --- TEST 6
• Copy and paste a 777 file from our local client --- TEST 7

So let's check on the command line:

user@client:/mnt$ ls -l
total 8
-rw-rw-r-- 1 user 1007    3 Feb  5 18:05 copyfile (664)
-rw-rw-r-- 1 user 1007    0 Feb  5 18:15 newfile (664)
drw-rw-r-- 1 user 1007 4096 Feb  5 18:15 newfolder (664)

But hey, let's check this same folder on the server-side:

user@server:/home$ ls -l
total 8
-rwxrwxrwx 1 user uploaders    3 2011-02-05 18:05 copyfile (777)
-rw------- 1 user uploaders    0 2011-02-05 18:15 newfile (600)
drwx--x--x 2 user uploaders 4096 2011-02-05 18:15 newfolder (711)

What?! The REAL file permissions are very different from what we see in Nautilus. So does this umask on sshfs just create a 'filter' that shows unreal file permissions? And I tried to open a file from another user but the same group that had real 600 permissions but 644 'fake' permissions, and I could still not read this, so what good is this filter??

5. UMASK IS ALL ABOUT FILES. BUT WHAT ABOUT DIRECTORIES?

From my tests I can see that the umask that is being applied also somehow influences the directory permissions. However, I want my files to be 664 (002) and my directories to be 771 (006). So is it possible to have a different umask for directories?

6. PERHAPS UMASK/PAM IS REALLY COOL, BUT UBUNTU IS JUST BUGGY?

On the one hand, I have read topics of people that have had success with PAM/UMASK and Ubuntu. On the other hand, I have found many older and newer bugs regarding umask/PAM/fuse on Ubuntu:

• https://bugs.launchpad.net/ubuntu/+source/gdm/+bug/241198
• https://bugs.launchpad.net/ubuntu/+source/fuse/+bug/239792
• https://bugs.launchpad.net/ubuntu/+source/pam/+bug/253096
• https://bugs.launchpad.net/ubuntu/+source/sudo/+bug/549172
• http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=314796

So I do not know what to believe anymore. Should I just give up? Would ACL solve all my problems? Or do I have again problems using Ubuntu?

One word of caution with backups using tar. Red Hat /Centos distributions support acls in the tar program but Ubuntu does not support acls when backing up. This means that all acls will be lost when you create a backup.

I am very willing to upgrade to Ubuntu 10.04 if that would solve my problems too, but first I want to understand what is happening.

Many things could be going on here.

First thoughts:

• yes, pam.d changes take effect immediately
• /etc/pam.d/common-session is the best place to set a default umask
• any pam.d umask would get overridden by any entry in .bashrc,
  but .bashrc only gets read under certain circumstances (interactive, non-login shell)
• testfile (711) is very strange
  • how is /home mounted, and are you using ACLs?
    (e.g. what do ls -ld /home and getfacl /home print?)
  • did testfile already exist before you did the copy, because scp won't change the permissions on a file that already existed (unless you use the -p flag)
• Nautilus is known to create files a different way, not sure why, or what the rules are
• umask=0113 will probably cause problems
• are the server and client running the same operating system?
  for example, if the client has ACLs enabled, or is Cygwin, the behavior can be different
• the best way to force sane permissions is to use default ACLs, exactly because, as you discovered, umask can be overridden by the user in .bashrc and .bash_profile.

Update:

• umask=0113 for sshfs is wrong.
  1. Try mounting without specifying a umask
  2. Create a new file inside the mount point using touch.
  3. You should see it only gets e.g. -rw-r--r--, without x bits
  4. By masking out x bits, you might break directories
     and compilers may be unable to create executable files properly

Workaround:

If we can't think of anything better, you could either use fam or gamin to watch for new files being created and fix the permissions on them, or even just a script that runs periodically and sets the permissions on all files.

ACLs should work fine for you.

Set a default ACL on all folder for the group, which will then be inherited by all future files and directories.

Something like setfacl -m d:g:uploaders:rwx should work.

To fix your existing permissions:

find /shared/folder -type d -exec setfacl -m d:g:uploaders:rwx {} \;
find /shared/folder -type f -exec setfacl -m g:uploaders:rw {} \;

Looks like though if "preserve" is set when a file is copied, the default ACL doesn't work. In that case, I can only suggest running the find commands in cron or monitoring the filesystem for changes.