What is it with printf() sending output to buffer?

I am going through "C PRIMER PLUS" and there is this topic about "OUTPUT FLUSHING". Now it says:

printf() statements sends output to an intermediate storage called buffer. Every now and then, the material in the buffer is sent to the screen. The standard C rules for when output is sent from the buffer to the screen are clear:

1. It is sent when the buffer gets full.
2. When a newline character is encountered.
3. When there is impending input.

(Sending the output from the buffer to the screen or file is called flushing the buffer.)

Now, To verify the above statements. I wrote this simple program :

#include<stdio.h>

int main(int argc, char** argv) {

printf("Hello World");

return 0;
}

so, neither the printf() contains a new line, nor it has some impending input(for e.g. a scanf() statement or any other input statement). Then why does it print the contents on the output screen.

Let's suppose first condition validated to true. The buffer got full(Which can't happen at all). Keeping that in mind, I truncated the statement inside printf() to

printf("Hi");

Still it prints the statement on the console.

So whats the deal here, All of the above conditions are false but still I'm getting the output on screen. Can you elaborate please. It appears I'm making a mistake in understanding the concept. Any help is highly appreciated.

EDIT: As suggested by a very useful comment, that maybe the execution of exit() function after the end of program is causing all the buffers to flush, resulting in the output on the console. But then if we hold the screen before the execution of exit(). Like this,

#include<stdio.h>

int main(int argc, char** argv) {

printf("Hello World!");
getchar();

return 0;
}

It still outputs on the console.

Solution 1:

Output buffering is an optimization technique. Writing data to some devices (hard disks f.e.) is an expensive operation; that's why the buffering appeared. In essence, it avoids writing data byte-by-byte (or char-by-char) and collects it in a buffer in order to write several KiB of data at once.

Being an optimization, output buffering must be transparent to the user (it is transparent even to the program). It must not affect the behaviour of the program; with or without buffering (or with different sizes of the buffer), the program must behave the same. This is what the rules you mentioned are for.

A buffer is just an area in memory where the data to be written is temporarily stored until enough data accumulates to make the actual writing process to the device efficient. Some devices (hard disk etc.) do not even allow writing (or reading) data in small pieces but only in blocks of some fixed size.

The rules of buffer flushing:

1. It is sent when the buffer gets full.

This is obvious. The buffer is full, its purpose was fulfilled, let's push the data forward to the device. Also, probably there is more data to come from the program, we need to make room for it.

2. When a newline character is encountered.

There are two types of devices: line-mode and block-mode. This rule applies only to the line-mode devices (the terminal, for example). It doesn't make much sense to flush the buffer on newlines when writing to disk. But it makes a lot of sense to do it when the program is writing to the terminal. In front of the terminal there is the user waiting impatiently for output. Don't let them wait too much.

But why output to terminal needs buffering? Writing on the terminal is not expensive. That's correct, when the terminal is physically located near the processor. Not also when the terminal and the processor are half the globe apart and the user runs the program through a remote connection.

3. When there is impending input.

It should read "when there is impeding input on the same device" to make it clear.

Reading is also buffered for the same reason as writing: efficiency. The reading code uses its own buffer. It fills the buffer when needed then scanf() and the other input-reading functions get their data from the input buffer.

When an input is about to happen on the same device, the buffer must be flushed (the data actually written to the device) in order to ensure consistency. The program has send some data to the output and now it expects to read back the same data; that's why the data must be flushed to the device in order for the reading code find it there and load it.

But why the buffers are flushed when the application exits?

Err... buffering is transparent, it must not affect the application behaviour. Your application has sent some data to the output. The data must be there (on the output device) when the application quits.

The buffers are also flushed when the associated files are closed, for the same reason. And this is what happens when the application exits: the cleanup code close all the open files (standard input and output are just files from the application point of view), closing forces flushing the buffers.

Solution 2:

Part of the specification for exit() in the C standard (POSIX link given) is:

Next, all open streams with unwritten buffered data are flushed, all open streams are closed, …

So, when the program exits, pending output is flushed, regardless of newlines, etc. Similarly, when the file is closed (fclose()), pending output is written:

Any unwritten buffered data for the stream are delivered to the host environment to be written to the file; any unread buffered data are discarded.

And, of course, the fflush() function flushes the output.

The rules quoted in the question are not wholly accurate.

1. When the buffer is full — this is correct.

2. When a newline is encountered — this is not correct, though it often applies. If the output device is an 'interactive device', then line buffering is the default. However, if the output device is 'non-interactive' (disk file, a pipe, etc), then the output is not necessarily (or usually) line-buffered.

3. When there is impending input — this too is not correct, though it is commonly the way it works. Again, it depends on whether the input and output devices are 'interactive'.

The output buffering mode can be modified by calling setvbuf() to set no buffering, line buffering or full buffering.

The standard says (§7.21.3):

¶3 When a stream is unbuffered, characters are intended to appear from the source or at the destination as soon as possible. Otherwise characters may be accumulated and transmitted to or from the host environment as a block. When a stream is fully buffered, characters are intended to be transmitted to or from the host environment as a block when a buffer is filled. When a stream is line buffered, characters are intended to be transmitted to or from the host environment as a block when a new-line character is encountered. Furthermore, characters are intended to be transmitted as a block to the host environment when a buffer is filled, when input is requested on an unbuffered stream, or when input is requested on a line buffered stream that requires the transmission of characters from the host environment. Support for these characteristics is implementation-defined, and may be affected via the setbuf and setvbuf functions.

…

¶7 At program startup, three text streams are predefined and need not be opened explicitly — standard input (for reading conventional input), standard output (for writing conventional output), and standard error (for writing diagnostic output). As initially opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device.

Also, §5.1.2.3 Program execution says:

• The input and output dynamics of interactive devices shall take place as specified in 7.21.3. The intent of these requirements is that unbuffered or line-buffered output appear as soon as possible, to ensure that prompting messages actually appear prior to a program waiting for input.

Solution 3:

The strange behavior of printf, buffering can be explained with below simple C code. please read through entire thing execute and understand as the below is not obvious (bit tricky)

#include <stdio.h>
int main()
{
   int a=0,b=0,c=0;
   printf ("Enter two numbers");
   while (1)
   {
      sleep (1000);
   }
   scanf("%d%d",&b,&c);
   a=b+c;
   printf("The sum is %d",a);
   return 1;
}

EXPERIMENT #1:

Action: Compile and Run above code

Observations:

The expected output is

Enter two numbers

But this output is not seen

EXPERIMENT #2:

Action: Move Scanf statement above while loop.

#include <stdio.h>
int main()
{
   int a=0,b=0,c=0;
   printf ("Enter two numbers");
   scanf("%d%d",&b,&c);
   while (1)
   {
      sleep (1000);
   }
   a=b+c;
   printf("The sum is %d",a);
   return 1;
}

Observations: Now the output is printed (reason below in the end)(just by scanf position change)

EXPERIMENT #3:

Action: Now add \n to print statement as below

#include <stdio.h>
int main()
{
   int a=0,b=0,c=0;
   printf ("Enter two numbers\n");
   while (1)
   {
      sleep (1000);
   }
   scanf("%d%d",&b,&c);
   a=b+c;
   printf("The sum is %d",a);
   return 1;
}

Observation: The output Enter two numbers is seen (after adding \n)

EXPERIMENT #4:

Action: Now remove \n from the printf line, comment out while loop, scanf line, addition line, printf line for printing result

#include <stdio.h>
int main()
{
   int a=0,b=0,c=0;
   printf ("Enter two numbers");
//   while (1)
//   {
//      sleep (1000);
//   }
//   scanf("%d%d",&b,&c);
//   a=b+c;
//   printf("The sum is %d",a);
   return 1;
}

Observations: The line "Enter two numbers" is printed to screen.

ANSWER:

The reason behind the strange behavior is described in Richard Stevens book.

PRINTF PRINTS TO SCREEN WHEN

The job of printf is to write output to stdout buffer. kernel flushes output buffers when

1. kernel need to read something in from input buffer. (EXPERIMENT #2)
2. when it encounters newline (since stdout is by default set to linebuffered)(EXPERIMENT #3)
3. after program exits (all output buffers are flushed) (EXPERIMENT #4)

By default stdout set to line buffering so printf will not print as the line did not end. if it is no buffered, all lines are output as is. Full buffered then, only when buffer is full it is flushed.