Read full lines from stdin including \n until end-of-file

I want to convert this Python code to Rust:

for line in sys.stdin:
   do something to the whole line including \n

But I can only find examples that read a single full line (including \n) or examples that do not read the \n.

It seem this ought to be some of the simplest in the world, but I have been unable to find it.

Solution 1:

Both Python and Rust offer convenience API for iterating over the lines in a file, they just chose different tradeoffs of convenience vs completeness. Rust has chosen extra convenience and strips the newlines at the expense of being able to distinguish whether the final line ends with a terminator. Python has chosen the opposite, at the expense of all line parsers having to take the final \n into account, and also take into account that it's optional.

But BufRead::lines() is just a convenience AP; if it doesn't meet your needs, you can always drop to the lower-level read_line() method:

let mut line = String::new();
while input.read_line(&mut line)? != 0 {
    let line = std::mem::take(&mut line);
    // ...
}

If you use this kind of code in multiple places, or just want the convenience of a for loop, you can abstract it into a utility function that returns an iterator, such as:

fn full_lines(mut input: impl BufRead) -> impl Iterator<Item = io::Result<String>> {
    std::iter::from_fn(move || {
        let mut vec = String::new();
        match input.read_line(&mut vec) {
            Ok(0) => None,
            Ok(_) => Some(Ok(vec)),
            Err(e) => Some(Err(e)),
        }
    })
}

Then you can use a for loop similar to the one in Python:

for line in full_lines(io::stdin().lock()) {
    let line = line?;
    // ...
}

With additional effort it is even possible to make full_lines a method on anything that implements BufRead:

trait FullLines: BufRead + Sized {
    fn full_lines<'a>(self) -> Box<dyn Iterator<Item = io::Result<String>> + 'a>
    where
        Self: 'a,
    {
        Box::new(full_lines(self))
    }
}

// Provide a blanket implementation of FullLines for any T
// that implements BufRead
impl<T: BufRead> FullLines for T {}

// Usage:

use FullLines;

for line in io::stdin().lock().full_lines() {
    let line = line?;
    // ...
}