Prevent diff from checking for newline at end of file

Solution 1:

You basically need to compare two files, conditionally ignoring the trailing byte. There isn't a 'diff' option to do this -- but there are a number of ways it could be done (e.g, hex diff comes to mind as well.)

To use 'diff', you basically have to modify the files that are missing the newline at the end of a file, and then compare. You could create a temporary directory with the modified files, or with a little bit of scripting it could be done in memory. (As to which is preferred depends on preference, file size, number of files...)

For example, the following will modify a file's contents (use sed -i to modify in-place, this just prints to stdout) to add a newline if one is missing (or leave the file unchanged if there already is a newline):

sed -e '$a\'  file1.txt

And just to review 'diff' syntax (returning true means they are the same, false means different):

$ diff a/file1.txt   b/file1.txt  \
      && echo '** are same' || echo '** are different'
2c2
< eof
---
> eof
\ No newline at end of file
** are different

Verify that only whitespace is different:

$ diff --ignore-all-space  a/file1.txt   b/file1.txt \
     && echo '** are same' || echo '** are different'
** are same

In bash, we can use 'sed' to manipulate the file contents as it is passed to 'diff' (original files left unchanged):

$ diff <(sed -e '$a\' a/file1.txt) <(sed -e '$a\' b/file1.txt) \
     && echo '** are same' || echo '** are different'
** are same

Now all you have to do is emulate diff -r to recursively compare directories. If comparing directories a and b, then for all files in a (e.g., a/dir1/dir2/file.txt) derive path to file in b (e.g., b/dir1/dir2/file.txt) and compare:

$ for f in $( find a -type f  )
> do
>    diff <(sed -e '$a\' $f) <(sed -e '$a\' b/${f#*/})
> done

A slightly more verbose version:

$ for f in $( find a -type f  )
> do
>   f1=$f
>   f2=b/${f#*/}
>   echo "compare: $f1 $f2"
>   diff <(sed -e '$a\' $f1) <(sed -e '$a\' $f2) \
>       && echo '** are same' || echo '** are different'
> done && echo '** all are same' || echo '** all are different'
compare: a/file1.txt b/file1.txt
** are same
compare: a/file2.txt b/file2.txt
** are same
** all are same

Solution 2:

I solved the problem by adding a newline to each of the files and ignoring blank lines in the diff (option -B). This solutions may not be suitable for your use case but it might help others:

echo >> $FILE1 
echo >> $FILE2
diff -B $FILE1 FILE2