compare and find differences of 2 huge text files (50 gb)
Solution 1:
wxHexEditor
wxHexEditor is both free and able to Diff large files up to 2^64 bytes (2 ExaByte). Has a GUI. Cross-platform. Lots of features.
To get it for free, choose one of the following options:
- Option: SourceForge
- Option: GitHub
- Option: Both Debian & Ubuntu repositories
Below is the same suggestion as above. But with details if you're interested in those.
Screenshot
Strength
• Hexadecimal (Hex) Editor. Which is helpful for doing reverse Engineering.
• Cross-platform. Linux, Mac OS, Windows
• Easy to use Graphical User Interface (GUI)
• Supports very large files up to 2^64 bytes (2 ExaByte)
• Compare two large files side by side (diff). Optionally list and search all diff.
• Very fast search
• Use small amount of RAM
• Do not create temporary files. So it used a very small amount of storage space.
• Dark or bright theme
• Multilingual 15 languages
• Open source. If you are not familiar with "open source", it means this software has both stronger security & stronger privacy. Because its code is publicly available for review and contributions to GitHub at https://github.com/EUA/wxHexEditor or at SourceForge at https://sourceforge.net/p/wxhexeditor/code/
• Attractive GNU General Public License version 2. This means the software code of this extension is owned and supported by a friendly not-for-profit community. Instead of a for-profit corporation. https://github.com/EUA/wxHexEditor/blob/master/LICENSE
Challenge
• Confusion between the two code repositories. At the time of this writing, August 2021, the GitHub repository seems to be more recent. As it was last updated in 2021 at https://github.com/EUA/wxHexEditor In comparison, the SourceForge repository at https://sourceforge.net/projects/wxhexeditor/ was last update of wxHexEditor was December 31st, 2017.
Show Your Support
• If you enjoy this application, show your support to the authors & contributors with:
___• Donation at https://www.paypal.com/cgi-bin/webscr?item_name=Donation+to+wxHexEditor&cmd=_donations&business=erdem.ua%40gmail.com
___• Support with ticket at https://sourceforge.net/projects/wxhexeditor/support
___• Support with forum at https://sourceforge.net/p/wxhexeditor/discussion/
___• Patch at https://sourceforge.net/p/wxhexeditor/patches/
Using
• wxHexEditor 0.23 • Debian 10 Buster • GNOME 3.30.2