REST API authentication for web app and mobile app

I'm having some trouble deciding how to implement authentication for a RESTful API that will be secure for consumption by both a web app and a mobile app.

Firstly, I thought to investigate HTTP Basic Authentication over HTTPS as an option. It would work well for a mobile app, where the username and password could be stored in the OS keychain securely and couldn't be intercepted in transit since the request would be over HTTPS. It's also elegant for the API since it'll be completely stateless. The problem with this is for the web app. There won't be access to such a keychain for storing the username and password, so I would need to use a cookie or localStorage, but then I'm storing the user's private details in a readily accessible place.

After more research, I found a lot of talk about HMAC authentication. The problem I see with this approach is there needs to be a shared secret that only the client and server knows. How can I get this per-user secret to a particular user in the web app, unless I have an api/login endpoint which takes username/password and gives the secret back to store in a cookie? to use in future requests. This is introducing state to the API however.

To throw another spanner into the works, I'd like to be able to restrict the API to certain applications (or, to be able to block certain apps from using the API). I can't see how this would be possible with the web app being completely public.

I don't really want to implement OAuth. It's probably overkill for my needs.

I feel as though I might not be understanding HMAC fully, so I'd welcome an explanation and how I could implement it securely with a web app and a mobile app.

Update

I ended up using HTTP Basic Auth, however instead of providing the actual username and password every request, an endpoint was implemented to exchange the username and password for an access key which is then provided for every authenticated request. Eliminates the problem of storing the username and password in the browser, but of course you could still fish out the token if you had access to the machine and use it. In hindsight, I would probably have looked at OAuth further, but it's pretty complicated for beginners.

You should use OAuth2. Here is how:

1) Mobile App

The mobile app store client credentials as you state yourself. It then uses "Resource Owner Password Credentials Grant" (see https://www.rfc-editor.org/rfc/rfc6749#section-4.3) to send those credentials. In turn it gets a (bearer) token it can use in the following requests.

2) Web site

The website uses "Authorization Code Grant" (see https://www.rfc-editor.org/rfc/rfc6749#section-4.1):

1. Website sees unauthorized request and redirects browser to HTML-enabled autorization endpoint in the REST api.

2. User authenticates with REST service

3. REST site redirects user back to website with access token in URL.

4. Website calls REST site and swaps access token to authorization token.

Here after the website uses the authorization token for accessing the REST service (on behalf of the end-user) - usually by including the token as a "bearer" token in the HTTP Authorization header.

It is not rocket science but it does take some time to understand completely.

3) Restricting API access for certain applications

In OAuth2 each client is issued a client ID and client secret (here "client" is your mobile app or website). The client must send these credentials when authorizing. Your REST service can use this to validate the calling client

I resolved this for my own API quite easily and securely without the need to expose any client credentials.

I also split the problem into 2 parts. API authentication - is this a valid request from a recognised entity (website or native app). API authorisation, is that entity allowed to use this particular endpoint and HTTP verb.

Authorisation is coded into the API using an access control list and user permissions and settings that are set up within the API code, configuration and database as required. A simple if statement in the API can test for authorisation and return the appropriate response (not authorised or the results of processing the API call).

Authentication is now just about checking to see if the call is genuine. To do this I issue self signed certificates to clients. A call to the API is made from their server whenever they want - typically when they generate their first page (or when they are performing their own app login checks). This call uses the certificates I have previously provided. If on my side I am happy the certificate is valid I can return a nonce and a time limited generated API key. This key is used in all subsequent calls to other API endpoints, in the bearer header for example, and it can be stored quite openly in an HTML form field or javascript variable or a variable within an app.

The nonce will prevent replay attacks and the API key can be stolen if someone wants - they will not be able to continue using after it expires or if the nonce changes before they make the next call.

Each API response will contain the next nonce of if the nonce doesn't match it will return an authentication error. In fact of the nonce doesn't match I kill the API key too. This will then force a genuine API user to reauthenticate using the certificates.

As long as the end user keeps those certificates safe and doesn't expose the method they use to make the initial authentication call (like making it an ajax request that can be replayed) then the API's are nice and secure.