Identifying and removing null characters in UNIX
I have a text file containing unwanted null characters (ASCII NUL, \0
). When I try to view it in vi
I see ^@
symbols, interleaved in normal text. How can I:
Identify which lines in the file contain null characters? I have tried grepping for
\0
and\x0
, but this did not work.Remove the null characters? Running
strings
on the file cleaned it up, but I'm just wondering if this is the best way?
I’d use tr
:
tr < file-with-nulls -d '\000' > file-without-nulls
If you are wondering if input redirection in the middle of the command arguments works, it does. Most shells will recognize and deal with I/O redirection (<
, >
, …) anywhere in the command line, actually.
Use the following sed command for removing the null characters in a file.
sed -i 's/\x0//g' null.txt
this solution edits the file in place, important if the file is still being used. passing -i'ext' creates a backup of the original file with 'ext' suffix added.