reopening sys.stdout after it is closed by with statement

I'm having trouble with a printing information that is input from a yaml file using PyYAML. I'm trying to reduce line-count, without affecting functionality. In some runs the output has to be appended to a file, in others to stdout.

At first I used this multiple times in my function processData:

    if logName:
        fp = open(logName, 'a')
    else:
        fp = sys.stdout
    print(........, file=fp)
    print(........, file=fp)
    if logName:
        fp.close()

That worked, but has the disadvantages of not using the with statement when something goes wrong.

The actual problem is not the complex print statements, but that I

1) don't want to duplicate code when printing to file or to sys.stdout
2) want to use the with statement so that files get closed if there are print errors
3) there are several such blocks, I don't want to call a different function for each of them, and so preventing code duplication

Then what I tried is:

def processData(yamlData, logName=None):
    ......
    with open(logName, 'a') if logName else sys.stdout as fp:
        print(........, file=fp)
        print(........, file=fp)
    .....
    with open(logName, 'a') if logName else sys.stdout as fp:
        print(........, file=fp)
        print(........, file=fp)

If there is not a logName, this errors to "ValueError: I/O operation on closed file". Any suggestions on how to get this to work without the original duplication? Can I reopen sys.stdout?

Solution 1:

You can "wrap" sys.stdout in a class to prevent it from being closed in the first place.

The with statement calls __enter__ and __exit__ on instances of that class at the beginning and end resp., so just make sure the __exit__ doesn't do anything:

class StdOut:
    def __enter__(self):
        return sys.stdout

    def __exit__(self, typ, val, trace):
        pass

stdout = StdOut()

and then use stdout instead of sys.stdout

Solution 2:

Literal question - reopening stdout

At the lowest C level, stdout is a well-known file descriptor (an integer pointing to an entry in the runtime- or system-managed descriptor table), initialized in a process upon its creation. It cannot be reopened (with standard C means) once it's closed and must be duplicated beforehand if you still need it.

A disposable copy of sys.stdout can be created like this:

stdout_copy=os.fdopen(os.dup(sys.stdout.fileno()))

_{(In Python 3, os.fdopen() has been merged into open() and is an alias to it.)}

You may need to use sys.__stdout__ instead if sys.stdout has been replaced.

The other question - wrapping functionality into with logic

First of all, consider the standard way to log - namely, the logging module - to avoid reinventing the square wheel. Opening and closing the file on demand can very well be implemented with its machinery, and it's not even needed in the vast majority of cases.

Now, the only way to cut on the repetitive parts of the code is to wrap the repeating part into a subroutine (or a code block that processes a list with elements describing what it should do each iteration, but it can only be used once). There are three conceptual parts here, regardless of syntax (it can be try/finally just as well as with):

• the wrapping construct
  • incl. exception handling
• the opening+closing code
• the wrapped code

• Wrapping just "the opening+closing code" is the easiest, the other answer is one possible way, but it leaves the repeating with and print(........, file=fp) parts.

• Wrapping the entire construct is harder since you'll have to pass a code chunk to your would-be subroutine, and Python intentionally omits anonymous code blocks - you'd have to def and then immediately use it which is rather awkward.

  • a decorator or passing your code as a callback are two possibilities.
  • If your code can be reduced to a pattern (like a set of messages), you can pass just that pattern instead and have the subroutine process it.