How to update multiple documents that match a query in elasticsearch

Solution 1:

You could use the update by query plugin in order to do just that. The idea is to select all document without a category and whose url matches a certain string and add the category you wish.

curl -XPOST 'localhost:9200/webproxylog/_update_by_query' -H "Content-Type: application/json" -d '
{
  "query": {
    "filtered": {
      "filter": {
        "bool": {
          "must": [
            {
              "term": {
                "url": "stackoverflow.com"
              }
            },
            {
              "missing": {
                "field": "category"
              }
            }
          ]
        }
      }
    }
  },
  "script" : "ctx._source.category = \"10\";"
}'

After running this, all your documents with url: stackoverflow.com that don't have a category, will get category: 10. You can run the same query again later to fix new stackoverflow.com documents that have been indexed in the meantime.

Also make sure to enable scripting in elasticsearch.yml and restart ES:

script.inline: on 
script.indexed: on

In the script, you're free to add as many fields as you want, e.g.

  ...
  "script" : "ctx._source.category1 = \"10\"; ctx._source.category2 = \"20\";"

UPDATE

ES 2.3 now features the update by query functionality. You can still use the above query exactly as is and it will work (except that filtered and missing are deprecated, but still working ;).

Solution 2:

That all sounds great but just to add to @Val answer, Update By Query is available form ElasticSearch 2.x but not for earlier versions. In our case we're using 1.4 for legacy reasons and there is no chance of upgrading in forseeable future so another solution is using the Update by query plugin provided here: https://github.com/yakaz/elasticsearch-action-updatebyquery