Python Run a daemon sub-process & read stdout

#!/usr/bin/python
from subprocess import Popen, PIPE, STDOUT
import pty
import os

cmd = 'socat -d -d PTY: PTY:'

master, slave = pty.openpty()

p = Popen(cmd, shell=True, stdin=PIPE, stdout=slave, stderr=slave, close_fds=True)
stdout = os.fdopen(master)
print stdout.readline()
print stdout.readline()

There are two problems with your version. Firstly, you call read without argument which means it will attempt to read everything. But since socat doesn't terminate, it never decides that it has read everything. By using readline, python only reads until it finds a newline. From my understanding of your problem that is what you need.

The second problem is that the C standard library will buffer outputs on pipes. We solve that by creating a pty with the openpty() function and passing it to both stdout and stderr of the subprocess. We use fdopen to make that file descriptor into a regular python object and we get rid of the buffering.

I don't know what you are doing with the socat, but I wonder whether it could replaced by using the pty module. You are copying one pty to another, and openpty is creating a pair of ptys. Perhaps you can use those directly?

The subprocess probably never closes stdout, so the read() call waits forever. To make matters worse, it will probably buffer its output when it figures out that it's a pipe instead of a console (the standard C library does this automatically, so this isn't a function of how cleverly written the app is). If so, probably the only option is to use an expect-style library such as Pexpect.