What is the meaning of "POSIX"?

What is POSIX? I have read the Wikipedia article and I read it every time I encounter the term. The fact is that I never really understood what it is.

Can anyone please explain it to me by explaining "the need for POSIX" too?

Solution 1:

POSIX is a family of standards, specified by the IEEE, to clarify and make uniform the application programming interfaces (and ancillary issues, such as commandline shell utilities) provided by Unix-y operating systems.

When you write your programs to rely on POSIX standards, you can be pretty sure to be able to port them easily among a large family of Unix derivatives (including Linux, but not limited to it!); if and when you use some Linux API that's not standardized as part of Posix, you will have a harder time if and when you want to port that program or library to other Unix-y systems (e.g., MacOSX) in the future.

Solution 2:

The most important things POSIX 7 defines

1. C API

   Greatly extends ANSI C with things like:

   • more file operations: mkdir, dirname, symlink, readlink, link (hardlinks), poll(), stat, sync, nftw()
   • process and threads: fork, execl, wait, pipe, semaphors sem_*, shared memory (shm_*), kill, scheduling parameters (nice, sched_*), sleep, mkfifo, setpgid()
   • networking: socket()
   • memory management: mmap, mlock, mprotect, madvise, brk()
   • utilities: regular expressions (reg*)

   Those APIs also determine underlying system concepts on which they depend, e.g. fork requires a concept of a process.

   Many Linux system calls exist to implement a specific POSIX C API function and make Linux compliant, e.g. sys_write, sys_read, ... Many of those syscalls also have Linux-specific extensions however.

   Major Linux desktop implementation: glibc, which in many cases just provides a shallow wrapper to system calls.

2. CLI utilities

   E.g.: cd, ls, echo, ...

   Many utilities are direct shell front ends for a corresponding C API function, e.g. mkdir.

   Major Linux desktop implementation: GNU Coreutils for the small ones, separate GNU projects for the big ones: sed, grep, awk, ... Some CLI utilities are implemented by Bash as built-ins.

3. Shell language

   E.g., a=b; echo "$a"

   Major Linux desktop implementation: GNU Bash.

4. Environment variables

   E.g.: HOME, PATH.

   PATH search semantics are specified, including how slashes prevent PATH search.

5. Program exit status

   ANSI C says 0 or EXIT_SUCCESS for success, EXIT_FAILURE for failure, and leaves the rest implementation defined.

   POSIX adds:

   • 126: command found but not executable.

   • 127: command not found.

   • > 128: terminated by a signal.

     But POSIX does not seem to specify the 128 + SIGNAL_ID rule used by Bash: https://unix.stackexchange.com/questions/99112/default-exit-code-when-process-is-terminated

   See also: What is the meaning of $? (dollar question mark) in shell scripts?

6. Regular expression

   There are two types: BRE (Basic) and ERE (Extended). Basic is deprecated and only kept to not break APIs.

   Those are implemented by C API functions, and used throughout CLI utilities, e.g. grep accepts BREs by default, and EREs with -E.

   E.g.: echo 'a.1' | grep -E 'a.[[:digit:]]'

   Major Linux implementation: glibc implements the functions under regex.h which programs like grep can use as backend.

7. Directory structure

   E.g.: /dev/null, /tmp

   The Linux FHS greatly extends POSIX.

8. Filenames

   • / is the path separator
   • NUL cannot be used
   • . is cwd, .. parent
   • portable filenames
     • use at most max 14 chars and 256 for the full path
     • can only contain: a-zA-Z0-9._-

   See also: what is posix compliance for filesystem?

9. Command line utility API conventions

   Not mandatory, used by POSIX, but almost nowhere else, notably not in GNU. But true, it is too restrictive, e.g. single letter flags only (e.g. -a), no double hyphen long versions (e.g. --all).

   A few widely used conventions:

   • - means stdin where a file is expected
   • -- terminates flags, e.g. ls -- -l to list a directory named -l

   See also: Are there standards for Linux command line switches and arguments?

10. "POSIX ACLs" (Access Control Lists), e.g. as used as backend for setfacl.

    This was withdrawn but it was implemented in several OSes, including in Linux with setxattr.

Who conforms to POSIX?

Many systems follow POSIX closely, but few are actually certified by the Open Group which maintains the standard. Notable certified ones include:

• OS X (Apple) X stands for both 10 and UNIX. Was the first Apple POSIX system, released circa 2001. See also: Is OSX a POSIX OS?
• AIX (IBM)
• HP-UX (HP)
• Solaris (Oracle)

Most Linux distros are very compliant, but not certified because they don't want to pay the compliance check. Inspur's K-UX and Huawei's EulerOS are two certified examples.

The official list of certified systems be found at: https://www.opengroup.org/openbrand/register/ and also at the wiki page.

Windows

Windows implemented POSIX on some of its professional distributions.

Since it was an optional feature, programmers could not rely on it for most end user applications.

Support was deprecated in Windows 8:

• Where does Microsoft Windows' 7 POSIX implementation currently stand?
• https://superuser.com/questions/495360/does-windows-8-still-implement-posix
• Feature request: https://windows.uservoice.com/forums/265757-windows-feature-suggestions/suggestions/6573649-full-posix-support

In 2016 a new official Linux-like API called "Windows Subsystem for Linux" was announced. It includes Linux system calls, ELF running, parts of the /proc filesystem, Bash, GCC, (TODO likely glibc?), apt-get and more: https://channel9.msdn.com/Events/Build/2016/P488 so I believe that it will allow Windows to run much, if not all, of POSIX. However, it is focused on developers / deployment instead of end users. In particular, there were no plans to allow access to the Windows GUI.

Historical overview of the official Microsoft POSIX compatibility: http://brianreiter.org/2010/08/24/the-sad-history-of-the-microsoft-posix-subsystem/

Cygwin is a well known GPL third-party project for that "provides substantial POSIX API functionality" for Windows, but requires that you "rebuild your application from source if you want it to run on Windows". MSYS2 is a related project that seems to add more functionality on top of Cygwin.

If the only thing from POSIX that you need are the command line utilities, also consider: https://github.com/shelljs/shelljs which re-implements a bunch of CLI utilities in Node.js, which already essentially implements a portability layer for the simpler system calls like mkdir etc. Many people use that project in the package.json of their project to allow running the project on Windows as well. Of course, it requires users to install the Node.js runtime, but given the popularity of Node.js, I don't expect that to break/be hard to satisfy anytime soon.

Android

Android has its own C library (Bionic) which does not fully support POSIX as of Android O: Is Android POSIX-compatible?

Bonus level

The Linux Standard Base further extends POSIX.

Use the non-frames indexes, they are much more readable and searchable: http://pubs.opengroup.org/onlinepubs/9699919799/nfindex.html

Get a full zipped version of the HTML pages for grepping: Where is the list of the POSIX C API functions?