Print LF with Python 3 to Windows stdout

How to get \n printed to stdout on Windows? This code works in Python 2, but not with Python 3:

# set sys.stdout to binary mode on Windows
import sys, os, msvcrt
msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)

# the length of testfile created with
#     python test_py3k_lf_print.py > testfile
# below should be exactly 4 symbols (23 0A 23 0A)
print("#\n#")

Solution 1:

Python 3 already configures standard I/O in binary mode, but it has its own I/O implementation that does newline translation. Instead of using print, which requires a text-mode file, you could manually call sys.stdout.buffer.write to use the binary-mode BufferedWriter. If you need to use print, then you'll need a new text I/O wrapper that doesn't use universal newlines. For example:

stdout = open(sys.__stdout__.fileno(), 
              mode=sys.__stdout__.mode, 
              buffering=1, 
              encoding=sys.__stdout__.encoding, 
              errors=sys.__stdout__.errors, 
              newline='\n', 
              closefd=False)

Since closefd is false, closing this file won't close the original sys.stdout file descriptor. You can use this file explicitly via print("#\n#", file=stdout), or replace sys.stdout = stdout. The original is available as sys.__stdout__.

Background

Python 3's io module was designed to provide a cross-platform and cross-implementation (CPython, PyPy, IronPython, Jython) specification for all filelike objects in terms of the abstract base classes RawIOBase, BufferedIOBase, and TextIOBase. It includes a reference pure Python implementation in the _pyio module. The common denominator for the raw io.FileIO implementation is the set of low-level POSIX system calls such as read and write, which eliminates the problem of CRT stdio inconsistencies. On Windows, the POSIX layer is just the low I/O layer of the CRT, but at least that's limited to the quirks of a single platform.

One of the Windows quirks is having non-standard text and binary modes in its POSIX I/O layer. Python addresses this by always using binary mode and calling setmode on the stdio file descriptors ¹.

Python can avoid using the Windows CRT for I/O by implementing a WinFileIO registered subclass of RawIOBase. There's a proposed patch for this in issue 12939. Another example is the win_unicode_console module, which implements WindowsConsoleRawReader and WindowsConsoleRawWriter classes.

_{1. This has caused problems for programs that embed Python and expect stdio to use the default text mode. For example, in binary mode printing wide-character strings no longer casts to char as it does in ANSI text mode, and it certainly doesn't print using WriteConsoleW as it would in UTF-16 text mode. For example:}

Python 2.7.10 (default, May 23 2015, 09:44:00) 
[MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys, os, msvcrt, ctypes 
>>> ctypes.cdll.msvcr90.wprintf(b'w\x00i\x00d\x00e\x00\n\x00') 
wide
5
>>> msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY) 
16384
>>> ctypes.cdll.msvcr90.wprintf(b'w\x00i\x00d\x00e\x00\n\x00')
w i d e
 5