NTLM with cURL returns 401

Goal: connecting to an Exchange server (EWS)
Method: cURL
Problem: Cannot get authenticated (NTLM), request returns 401.¹

There seems to be an old, well documented ² issue that started with cURL's move from OpenSSL to NSS. I read that the implementation of NTLM is dependant on OpenSSL, and therefore this move broke the NTLM authentication.

The issue is shown below, but the important parts seem to be the returned 401 and the gss_init_sec_context() below that.

The thing I don't understand is that my current version:

• Has an OpenSSL variant according to https://launchpad.net/ubuntu/+source/curl/7.22.0-3ubuntu4
• supports NTLM according to that same link
• Actually uses this variant (and not the NSS) according to below log (it says libcurl/7.22.0 OpenSSL)
• Should not be bothered by linked bug as per above points.
• but IS affected, as shown by the fact that I get the 401

I'm not sure how this can be fixed. I could find a lot of old (mostly 2010) references to this problem, but nothing new, and certainly nothing with a sollution. I know that provided references (see ²) show that this might work with an older version (7.19), but I am not able (nor willing) to downgrade to that version.

Several implementations of Exchange-communication (EWS) use cURL to retreive the EWS files (wsdl etc), so I'm sure there must be a working method around but I can't find it. Does anyone have a clue what I can do? Do I have another bug, am I interpreting the facts wrong and is this still the same situation as provided in the links and it will never be fixed?

¹ The error goes something like this:

curl https://*DOMAIN*/Exchange.asmx -w %{http_code} --ntlm -u *USERNAME* --verbose --show-error
Enter host password for user '*USERNAME':
* About to connect() to DOMAIN port 443 (#0)
*   Trying IP... connected
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using AES128-SHA
* Server certificate:
     *SNIP*
*        SSL certificate verify ok.
* Server auth using NTLM with user 'USERNAME'
> GET /EWS/Exchange.asmx HTTP/1.1
> Authorization: NTLM *snip*
> User-Agent: curl/7.22.0 (i686-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
> Host: DOMAIN
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Server: Microsoft-IIS/7.5
< Set-Cookie: exchangecookie=xxx; expires=Wed, 17-Jul-2013 07:45:30 GMT; path=/; HttpOnly
< WWW-Authenticate: NTLM  *SNIP*
* gss_init_sec_context() failed: : Credentials cache file '/tmp/krb5cc_1005' not foundWWW-Authenticate: Negotiate
< X-Powered-By: ASP.NET
< Date: Tue, 17 Jul 2012 07:45:30 GMT
< Content-Length: 0
<
* Connection #0 to host DOMAIN left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):

² for instance :

• https://bugs.launchpad.net/ubuntu/+source/curl/+bug/675974
• https://bugzilla.redhat.com/show_bug.cgi?id=603783
• https://stackoverflow.com/questions/4341368/curl-always-returns-401-with-ntlm
  • This is the same problem, but does not address the fact that the NSS issue should not be present in my OpenSSL libcurl.

curl info:

user@server:~$ curl -V
curl 7.22.0 (i686-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtmp rtsp smtp smtps telnet tftp
Features: GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP

Solution 1:

I am not really sure why, but while my 7.22 version shouldn't be affected by the whole NTLM issue, it seems that it is.

The only solution seems to be to use an old (<7.19, I tried with 7.15) version or to use a new version (I tried 7.26). As said, I cannot downgrade or upgrade libcurl itself just for this feature. This means we need to find a workaround....

Workaround used (warning: hack coming up)

1. Download and compile the curl you wish to use. I did NOT do this as root // sudo this because I don't want to replace the current libcurl etc!

   wget http://curl.haxx.se/download/curl-7.26.0.zip
   ./configure --prefix=/local_path/
   make
   make install
   

2. Test the new curl command we just compiled: it does give the 401, but then instead of the gss_init_sec_context() you should see it going to an (in the end) 302. This is winning.

3. The hack part: use this libcurl when calling your script. We are working with a 3rd party PHP "app" (one of the reasons I'm not happy about changing all curl libs), and instead of directly calling this, we make an ugly-but-working wrapper, that calls something like this:

   $se = shell_exec("LD_LIBRARY_PATH=/local_path/lib php /path/3rdparty.php");
   

Yes, this has drawbacks, and yes this is bit of a skeleton example (meaning you probably want to add some things like maybe the original path for instance) but the basics are kinda there.

Not really proud of this, but I think more people are restrained by the fact they cannot just go and update their libcurl to a random version and are using some sort of plugin that uses e.g. php-ews, or anything build on the NTLMSoapClient.

I hope there is another option down the line, but as this is currently the only way to get it 'working' I thought I'd share.