What should a JSON service return on failure / error

See this question for some insight into best-practices for your situation.

The topline suggestion (from said link) is to standardize a response structure (for both success and failure) that your handler looks for, catching all Exceptions at the server layer and converting them to the same structure. For example (from this answer):

{
    success:false,
    general_message:"You have reached your max number of Foos for the day",
    errors: {
        last_name:"This field is required",
        mrn:"Either SSN or MRN must be entered",
        zipcode:"996852 is not in Bernalillo county. Only Bernalillo residents are eligible"
    }
} 

This is the approach stackoverflow uses (in case you were wondering how others do this kind of thing); write operations like voting have "Success" and "Message" fields, regardless of if the vote was allowed or not:

{ Success:true, NewScore:1, Message:"", LastVoteTypeId:3 }

As @Phil.H pointed out, you should be consistent in whatever you choose. This is easier said than done (as is everything in development!).

For example, if you submit comments too quickly on SO, instead of being consistent and returning

{ Success: false, Message: "Can only comment once every blah..." }

SO will throw a server exception (HTTP 500) and catch it in their error callback.

As much as it "feels right" to use jQuery + .ashx + HTTP [status codes] IMO it will add more complexity to your client-side code base than it's worth. Realize that jQuery does not "detect" error codes but rather the lack of a success code. This is an important distinction when trying to design a client around http response codes with jQuery. You only get two choices (was it a "success" or "error"?), which you have to branch further on your own. If you have a small number of WebServices driving a small number of pages then it might be okay, but anything larger scale may get messy.

It's much more natural in a .asmx WebService (or WCF for that matter) to return a custom object than to customize the HTTP status code. Plus you get the JSON serialization for free.

The HTTP status code you return should depend on the type of error that has occurred. If an ID doesn't exist in the database, return a 404; if a user doesn't have enough privileges to make that Ajax call, return a 403; if the database times out before being able to find the record, return a 500 (server error).

jQuery automatically detects such error codes, and runs the callback function that you define in your Ajax call. Documentation: http://api.jquery.com/jQuery.ajax/

Short example of a $.ajax error callback:

$.ajax({
  type: 'POST',
  url: '/some/resource',
  success: function(data, textStatus) {
    // Handle success
  },
  error: function(xhr, textStatus, errorThrown) {
    // Handle error
  }
});