Why does Mercurial think my SQL files are binary?

I just scripted out my SQL Server stored procs, table definitions, etc using SQL Server Management Studio, and tried to add them to my Mercurial source control repository. They got added just fine, but now when I change and diff them, Mercurial calls them "binary files" and doesn't give me a proper unified diff.

I thought the encoding might be a problem, so I tried regenerating the scripts and specifying ANSI for the text file output, but I get the same behavior. I can view them just fine in notepad without any odd-looking characters showing up. Why does Mercurial think these files are binary?

Otherwise, if someone can recommend a good tool for scripting out a SQL Server database that might not cause this issue, that might work, too.

Solution 1:

I've run into this problem because SQL Server Management Studio saves the files as Unicode. The first two bytes (most of the time) of a Unicode text file define the encoding. Most newer text editors (e.g. Notepad) handle this transparently.

The first two bytes are probably where your problem is. They may look like ÿþ. Or FF FE in hex.

On the "Save" button on the Save dialog is a pick list. Choose "Save with Encoding..." and select "US-ASCII-Codepage20127". I believe this setting is sticky and will remain for future saves.

Solution 2:

According to the docs, it's considered binary iff there are null bytes in the file. SQL files shouldn't have null bytes, so I would check that first (try looking in a hex editor). I assume you do know you can force diff to treat it as text