how to control popen stdin, stdout, stderr redirection?

popen(3) is just a library function, which relies on fork(2) and pipe(2) to do the real work.

However pipe(2) can only create unidirectional pipes. To send the child process input, and also capture the output, you need to open two pipes.

If you want to capture the stderr too, that's possible, but then you'll need three pipes, and a select loop to arbitrate reads between the stdout and stderr streams.

There's an example here for the two-pipe version.

simple idea: why not add "2>&1" to the command string to force the bash to redirect stderr to stdout (OK, writing to stdin still is not possible but at least we get stderr and stdout into our C program).

The return value from popen() is a normal standard I/O stream in all respects save that it must be closed with pclose() rather than fclose(3). Writing to such a stream writes to the standard input of the command; the command's standard output is the same as that of the process that called popen(), unless this is altered by the command itself. Conversely, reading from a "popened" stream reads the command's standard output, and the command's standard input is the same as that of the process that called popen().

From its manpage, so it allows you to read the commands standard output or write into its standard input. It doesn't say anything about stderr. Thus that is not redirected.

If you provide "w", you will send your stuff to the stdin of the shell that is executed. Thus, doing

FILE * file = popen("/bin/cat", "w");
fwrite("hello", 5, file);
pclose(file);

Will make the shell execute /bin/cat, and pass it the string "hello" as its standard input stream. If you want to redirect, for example stderr to the file "foo" do this first, before you execute the code above:

FILE * error_file = fopen("foo", "w+");
if(error_file) {
    dup2(fileno(error_file), 2);
    fclose(error_file);
}

It will open the file, and duplicate its file-descriptor to 2, closing the original file descriptor afterwards.

Now, if you have your stdout closed in your parent, then if the child calls open it will get 1, since that's (if stdin is already opened) the next free file-descriptor. Only solution i see is to just use dup2 and duplicate something into that in the parent, like the above code. Note that if the child opens stdout, it will not make stdout open in the parent too. It stays closed there.