Add es v6 writer

vendor/gopkg.in/olivere/elastic.v6/CHANGELOG-3.0.md

+# Elastic 3.0
+
+Elasticsearch 2.0 comes with some [breaking changes](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/breaking-changes-2.0.html). You will probably need to upgrade your application and/or rewrite part of it due to those changes.
+
+We use that window of opportunity to also update Elastic (the Go client) from version 2.0 to 3.0. This will introduce both changes due to the Elasticsearch 2.0 update as well as changes that make Elastic cleaner by removing some old cruft.
+
+So, to summarize:
+
+1. Elastic 2.0 is compatible with Elasticsearch 1.7+ and is still actively maintained.
+2. Elastic 3.0 is compatible with Elasticsearch 2.0+ and will soon become the new master branch.
+
+The rest of the document is a list of all changes in Elastic 3.0.
+
+## Pointer types
+
+All types have changed to be pointer types, not value types. This not only is cleaner but also simplifies the API as illustrated by the following example:
+
+Example for Elastic 2.0 (old):
+
+```go
+q := elastic.NewMatchAllQuery()
+res, err := elastic.Search("one").Query(&q).Do()  // notice the & here
+```
+
+Example for Elastic 3.0 (new):
+
+```go
+q := elastic.NewMatchAllQuery()
+res, err := elastic.Search("one").Query(q).Do()   // no more &
+// ... which can be simplified as:
+res, err := elastic.Search("one").Query(elastic.NewMatchAllQuery()).Do()
+```
+
+It also helps to prevent [subtle issues](https://github.com/olivere/elastic/issues/115#issuecomment-130753046).
+
+## Query/filter merge
+
+One of the biggest changes in Elasticsearch 2.0 is the [merge of queries and filters](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/_query_dsl_changes.html#_queries_and_filters_merged). In Elasticsearch 1.x, you had a whole range of queries and filters that were basically identical (e.g. `term_query` and `term_filter`).
+
+The practical aspect of the merge is that you can now basically use queries where once you had to use filters instead. For Elastic 3.0 this means: We could remove a whole bunch of files. Yay!
+
+Notice that some methods still come by "filter", e.g. `PostFilter`. However, they accept a `Query` now when they used to accept a `Filter` before.
+
+Example for Elastic 2.0 (old):
+
+```go
+q := elastic.NewMatchAllQuery()
+f := elastic.NewTermFilter("tag", "important")
+res, err := elastic.Search().Index("one").Query(&q).PostFilter(f)
+```
+
+Example for Elastic 3.0 (new):
+
+```go
+q := elastic.NewMatchAllQuery()
+f := elastic.NewTermQuery("tag", "important") // it's a query now!
+res, err := elastic.Search().Index("one").Query(q).PostFilter(f)
+```
+
+## Facets are removed
+
+[Facets have been removed](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/_removed_features.html#_facets_have_been_removed) in Elasticsearch 2.0. You need to use aggregations now.
+
+## Errors
+
+Elasticsearch 2.0 returns more information about an error in the HTTP response body. Elastic 3.0 now reads this information and makes it accessible by the consumer.
+
+Errors and all its details are now returned in [`Error`](https://github.com/olivere/elastic/blob/release-branch.v3/errors.go#L59).
+
+### HTTP Status 404 (Not Found)
+
+When Elasticsearch does not find an entity or an index, it generally returns HTTP status code 404. In Elastic 2.0 this was a valid result and didn't raise an error from the `Do` functions. This has now changed in Elastic 3.0.
+
+Starting with Elastic 3.0, there are only two types of responses considered successful. First, responses with HTTP status codes [200..299]. Second, HEAD requests which return HTTP status 404. The latter is used by Elasticsearch to e.g. check for existence of indices or documents. All other responses will return an error.
+
+To check for HTTP Status 404 (with non-HEAD requests), e.g. when trying to get or delete a missing document, you can use the [`IsNotFound`](https://github.com/olivere/elastic/blob/release-branch.v3/errors.go#L84) helper (see below).
+
+The following example illustrates how to check for a missing document in Elastic 2.0 and what has changed in 3.0.
+
+Example for Elastic 2.0 (old):
+
+```go
+res, err = client.Get().Index("one").Type("tweet").Id("no-such-id").Do()
+if err != nil {
+  // Something else went wrong (but 404 is NOT an error in Elastic 2.0)
+}
+if !res.Found {
+	// Document has not been found
+}
+```
+
+Example for Elastic 3.0 (new):
+
+```go
+res, err = client.Get().Index("one").Type("tweet").Id("no-such-id").Do()
+if err != nil {
+  if elastic.IsNotFound(err) {
+    // Document has not been found
+  } else {
+    // Something else went wrong
+  }
+}
+```
+
+### HTTP Status 408 (Timeouts)
+
+Elasticsearch now responds with HTTP status code 408 (Timeout) when a request fails due to a timeout. E.g. if you specify a timeout with the Cluster Health API, the HTTP response status will be 408 if the timeout is raised. See [here](https://github.com/elastic/elasticsearch/commit/fe3179d9cccb569784434b2135ca9ae13d5158d3) for the specific commit to the Cluster Health API.
+
+To check for HTTP Status 408, we introduced the [`IsTimeout`](https://github.com/olivere/elastic/blob/release-branch.v3/errors.go#L101) helper.
+
+Example for Elastic 2.0 (old):
+
+```go
+health, err := client.ClusterHealth().WaitForStatus("yellow").Timeout("1s").Do()
+if err != nil {
+  // ...
+}
+if health.TimedOut {
+  // We have a timeout
+}
+```
+
+Example for Elastic 3.0 (new):
+
+```go
+health, err := client.ClusterHealth().WaitForStatus("yellow").Timeout("1s").Do()
+if elastic.IsTimeout(err) {
+  // We have a timeout
+}
+```
+
+### Bulk Errors
+
+The error response of a bulk operation used to be a simple string in Elasticsearch 1.x.
+In Elasticsearch 2.0, it returns a structured JSON object with a lot more details about the error.
+These errors are now captured in an object of type [`ErrorDetails`](https://github.com/olivere/elastic/blob/release-branch.v3/errors.go#L59) which is used in [`BulkResponseItem`](https://github.com/olivere/elastic/blob/release-branch.v3/bulk.go#L206).
+
+### Removed specific Elastic errors
+
+The specific error types `ErrMissingIndex`, `ErrMissingType`, and `ErrMissingId` have been removed. They were only used by `DeleteService` and are replaced by a generic error message.
+
+## Numeric types
+
+Elastic 3.0 has settled to use `float64` everywhere. It used to be a mix of `float32` and `float64` in Elastic 2.0. E.g. all boostable queries in Elastic 3.0 now have a boost type of `float64` where it used to be `float32`.
+
+## Pluralization
+
+Some services accept zero, one or more indices or types to operate on.
+E.g. in the `SearchService` accepts a list of zero, one, or more indices to
+search and therefor had a func called `Index(index string)` and a func
+called `Indices(indices ...string)`.
+
+Elastic 3.0 now only uses the singular form that, when applicable, accepts a
+variadic type. E.g. in the case of the `SearchService`, you now only have
+one func with the following signature: `Index(indices ...string)`.
+
+Notice this is only limited to `Index(...)` and `Type(...)`. There are other
+services with variadic functions. These have not been changed.
+
+## Multiple calls to variadic functions
+
+Some services with variadic functions have cleared the underlying slice when
+called while other services just add to the existing slice. This has now been
+normalized to always add to the underlying slice.
+
+Example for Elastic 2.0 (old):
+
+```go
+// Would only cleared scroll id "two"
+// because ScrollId cleared the values when called multiple times
+client.ClearScroll().ScrollId("one").ScrollId("two").Do()
+```
+
+Example for Elastic 3.0 (new):
+
+```go
+// Now (correctly) clears both scroll id "one" and "two"
+// because ScrollId no longer clears the values when called multiple times
+client.ClearScroll().ScrollId("one").ScrollId("two").Do()
+```
+
+## Ping service requires URL
+
+The `Ping` service raised some issues because it is different from all
+other services. If not explicitly given a URL, it always pings `127.0.0.1:9200`.
+
+Users expected to ping the cluster, but that is not possible as the cluster
+can be a set of many nodes: So which node do we ping then?
+
+To make it more clear, the `Ping` function on the client now requires users
+to explicitly set the URL of the node to ping.
+
+## Meta fields
+
+Many of the meta fields e.g. `_parent` or `_routing` are now
+[part of the top-level of a document](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/_mapping_changes.html#migration-meta-fields)
+and are no longer returned as parts of the `fields` object. We had to change
+larger parts of e.g. the `Reindexer` to get it to work seamlessly with Elasticsearch 2.0.
+
+Notice that all stored meta-fields are now [returned by default](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/_crud_and_routing_changes.html#_all_stored_meta_fields_returned_by_default).
+
+## HasParentQuery / HasChildQuery
+
+`NewHasParentQuery` and `NewHasChildQuery` must now include both parent/child type and query. It is now in line with the Java API.
+
+Example for Elastic 2.0 (old):
+
+```go
+allQ := elastic.NewMatchAllQuery()
+q := elastic.NewHasChildFilter("tweet").Query(&allQ)
+```
+
+Example for Elastic 3.0 (new):
+
+```go
+q := elastic.NewHasChildQuery("tweet", elastic.NewMatchAllQuery())
+```
+
+## SetBasicAuth client option
+
+You can now tell Elastic to pass HTTP Basic Auth credentials with each request. In previous versions of Elastic you had to set up your own `http.Transport` to do this. This should make it more convenient to use Elastic in combination with [Shield](https://www.elastic.co/products/shield) in its [basic setup](https://www.elastic.co/guide/en/shield/current/enable-basic-auth.html).
+
+Example:
+
+```go
+client, err := elastic.NewClient(elastic.SetBasicAuth("user", "secret"))
+if err != nil {
+  log.Fatal(err)
+}
+```
+
+## Delete-by-Query API
+
+The Delete-by-Query API is [a plugin now](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/_removed_features.html#_delete_by_query_is_now_a_plugin). It is no longer core part of Elasticsearch. You can [install it as a plugin as described here](https://www.elastic.co/guide/en/elasticsearch/plugins/2.0/plugins-delete-by-query.html).
+
+Elastic 3.0 still contains the `DeleteByQueryService`, but you need to install the plugin first. If you don't install it and use `DeleteByQueryService` you will most probably get a 404.
+
+An older version of this document stated the following:
+
+> Elastic 3.0 still contains the `DeleteByQueryService` but it will fail with `ErrPluginNotFound` when the plugin is not installed.
+>
+> Example for Elastic 3.0 (new):
+>
+> ```go
+> _, err := client.DeleteByQuery().Query(elastic.NewTermQuery("client", "1")).Do()
+> if err == elastic.ErrPluginNotFound {
+> 	// Delete By Query API is not available
+> }
+> ```
+
+I have decided that this is not a good way to handle the case of a missing plugin. The main reason is that with this logic, you'd always have to check if the plugin is missing in case of an error. This is not only slow, but it also puts logic into a service where it should really be just opaque and return the response of Elasticsearch.
+
+If you rely on certain plugins to be installed, you should check on startup. That's where the following two helpers come into play.
+
+## HasPlugin and SetRequiredPlugins
+
+Some of the core functionality of Elasticsearch has now been moved into plugins. E.g. the Delete-by-Query API is [a plugin now](https://www.elastic.co/guide/en/elasticsearch/plugins/2.0/plugins-delete-by-query.html).
+
+You need to make sure to add these plugins to your Elasticsearch installation to still be able to use the `DeleteByQueryService`. You can test this now with the `HasPlugin(name string)` helper in the client.
+
+Example for Elastic 3.0 (new):
+
+```go
+err, found := client.HasPlugin("delete-by-query")
+if err == nil && found {
+	// ... Delete By Query API is available
+}
+```
+
+To simplify this process, there is now a `SetRequiredPlugins` helper that can be passed as an option func when creating a new client. If the plugin is not installed, the client wouldn't be created in the first place.
+
+```go
+// Will raise an error if the "delete-by-query" plugin is NOT installed
+client, err := elastic.NewClient(elastic.SetRequiredPlugins("delete-by-query"))
+if err != nil {
+  log.Fatal(err)
+}
+```
+
+Notice that there also is a way to define [mandatory plugins](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-plugins.html#_mandatory_plugins) in the Elasticsearch configuration file.
+
+## Common Query has been renamed to Common Terms Query
+
+The `CommonQuery` has been renamed to `CommonTermsQuery` to be in line with the [Java API](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/_java_api_changes.html#_query_filter_refactoring).
+
+## Remove `MoreLikeThis` and `MoreLikeThisField`
+
+The More Like This API and the More Like This Field query [have been removed](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/_query_dsl_changes.html#_more_like_this) and replaced with the `MoreLikeThisQuery`.
+
+## Remove Filtered Query
+
+With the merge of queries and filters, the [filtered query became deprecated](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/_query_dsl_changes.html#_literal_filtered_literal_query_and_literal_query_literal_filter_deprecated). While it is only deprecated and therefore still available in Elasticsearch 2.0, we have decided to remove it from Elastic 3.0. Why? Because we think that when you're already forced to rewrite many of your application code, it might be a good chance to get rid of things that are deprecated as well. So you might simply change your filtered query with a boolean query as [described here](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/_query_dsl_changes.html#_literal_filtered_literal_query_and_literal_query_literal_filter_deprecated).
+
+## Remove FuzzyLikeThis and FuzzyLikeThisField
+
+Both have been removed from Elasticsearch 2.0 as well.
+
+## Remove LimitFilter
+
+The `limit` filter is [deprecated in Elasticsearch 2.0](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/_query_dsl_changes.html#_literal_limit_literal_filter_deprecated) and becomes a no-op. Now is a good chance to remove it from your application as well. Use the `terminate_after` parameter in your search [as described here](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/search-request-body.html) to achieve similar effects.
+
+## Remove `_cache` and `_cache_key` from filters
+
+Both have been [removed from Elasticsearch 2.0 as well](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/_query_dsl_changes.html#_filter_auto_caching).
+
+## Partial fields are gone
+
+Partial fields are [removed in Elasticsearch 2.0](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/_search_changes.html#_partial_fields) in favor of [source filtering](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/search-request-source-filtering.html).
+
+## Scripting
+
+A [`Script`](https://github.com/olivere/elastic/blob/release-branch.v3/script.go) type has been added to Elastic 3.0. In Elastic 2.0, there were various places (e.g. aggregations) where you could just add the script as a string, specify the scripting language, add parameters etc. With Elastic 3.0, you should now always use the `Script` type.
+
+Example for Elastic 2.0 (old):
+
+```go
+update, err := client.Update().Index("twitter").Type("tweet").Id("1").
+	Script("ctx._source.retweets += num").
+	ScriptParams(map[string]interface{}{"num": 1}).
+	Upsert(map[string]interface{}{"retweets": 0}).
+	Do()
+```
+
+Example for Elastic 3.0 (new):
+
+```go
+update, err := client.Update().Index("twitter").Type("tweet").Id("1").
+	Script(elastic.NewScript("ctx._source.retweets += num").Param("num", 1)).
+	Upsert(map[string]interface{}{"retweets": 0}).
+	Do()
+```
+
+## Cluster State
+
+The combination of `Metric(string)` and `Metrics(...string)` has been replaced by a single func with the signature `Metric(...string)`.
+
+## Unexported structs in response
+
+Services generally return a typed response from a `Do` func. Those structs are exported so that they can be passed around in your own application. In Elastic 3.0 however, we changed that (most) sub-structs are now unexported, meaning: You can only pass around the whole response, not sub-structures of it. This makes it easier for restructuring responses according to the Elasticsearch API. See [`ClusterStateResponse`](https://github.com/olivere/elastic/blob/release-branch.v3/cluster_state.go#L182) as an example.
+
+## Add offset to Histogram aggregation
+
+Histogram aggregations now have an [offset](https://github.com/elastic/elasticsearch/pull/9505) option.
+
+## Services
+
+### REST API specification
+
+As you might know, Elasticsearch comes with a REST API specification. The specification describes the endpoints in a JSON structure.
+
+Most services in Elastic predated the REST API specification. We are in the process of bringing all these services in line with the specification. Services can be generated by `go generate` (not 100% automatic though). This is an ongoing process.
+
+This probably doesn't mean a lot to you. However, you can now be more confident that Elastic supports all features that the REST API specification describes.
+
+At the same time, the file names of the services are renamed to match the REST API specification naming.
+
+### REST API Test Suite
+
+The REST API specification of Elasticsearch comes along with a test suite that official clients typically use to test for conformance. Up until now, Elastic didn't run this test suite. However, we are in the process of setting up infrastructure and tests to match this suite as well.
+
+This process in not completed though.
+
+

vendor/gopkg.in/olivere/elastic.v6/CHANGELOG-5.0.md

+# Changes in Elastic 5.0
+
+## Enforce context.Context in PerformRequest and Do
+
+We enforce the usage of `context.Context` everywhere you execute a request.
+You need to change all your `Do()` calls to pass a context: `Do(ctx)`.
+This enables automatic request cancelation and many other patterns.
+
+If you don't need this, simply pass `context.TODO()` or `context.Background()`.
+
+## Warmers removed
+
+Warmers are no longer necessary and have been [removed in ES 5.0](https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking_50_index_apis.html#_warmers).
+
+## Optimize removed
+
+Optimize was deprecated in ES 2.0 and has been [removed in ES 5.0](https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking_50_rest_api_changes.html#_literal__optimize_literal_endpoint_removed).
+Use [Force Merge](https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-forcemerge.html) instead.
+
+## Missing Query removed
+
+The `missing` query has been [removed](https://www.elastic.co/guide/en/elasticsearch/reference/master/query-dsl-exists-query.html#_literal_missing_literal_query).
+Use `exists` query with `must_not` in `bool` query instead.
+
+## And Query removed
+
+The `and` query has been [removed](https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking_50_search_changes.html#_deprecated_queries_removed).
+Use `must` clauses in a `bool` query instead.
+
+## Not Query removed
+
+TODO Is it removed?
+
+## Or Query removed
+
+The `or` query has been [removed](https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking_50_search_changes.html#_deprecated_queries_removed).
+Use `should` clauses in a `bool` query instead.
+
+## Filtered Query removed
+
+The `filtered` query has been [removed](https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking_50_search_changes.html#_deprecated_queries_removed).
+Use `bool` query instead, which supports `filter` clauses too.
+
+## Limit Query removed
+
+The `limit` query has been [removed](https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking_50_search_changes.html#_deprecated_queries_removed).
+Use the `terminate_after` parameter instead.
+
+# Template Query removed
+
+The `template` query has been [deprecated](https://www.elastic.co/guide/en/elasticsearch/reference/5.x/query-dsl-template-query.html). You should use
+Search Templates instead.
+
+We remove it from Elastic 5.0 as the 5.0 update is already a good opportunity
+to get rid of old stuff.
+
+## `_timestamp` and `_ttl` removed
+
+Both of these fields were deprecated and are now [removed](https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking_50_mapping_changes.html#_literal__timestamp_literal_and_literal__ttl_literal).
+
+## Search template Put/Delete API returns `acknowledged` only
+
+The response type for Put/Delete search templates has changed.
+It only returns a single `acknowledged` flag now.
+
+## Fields has been renamed to Stored Fields
+
+The `fields` parameter has been renamed to `stored_fields`.
+See [here](https://www.elastic.co/guide/en/elasticsearch/reference/5.x/breaking_50_search_changes.html#_literal_fields_literal_parameter).
+
+## Fielddatafields has been renamed to Docvaluefields
+
+The `fielddata_fields` parameter [has been renamed](https://www.elastic.co/guide/en/elasticsearch/reference/5.x/breaking_50_search_changes.html#_literal_fielddata_fields_literal_parameter)
+to `docvalue_fields`.
+
+## Type exists endpoint changed
+
+The endpoint for checking whether a type exists has been changed from
+`HEAD {index}/{type}` to `HEAD {index}/_mapping/{type}`.
+See [here](https://www.elastic.co/guide/en/elasticsearch/reference/5.0/breaking_50_rest_api_changes.html#_literal_head_index_type_literal_replaced_with_literal_head_index__mapping_type_literal).
+
+## Refresh parameter changed
+
+The `?refresh` parameter previously could be a boolean value. It indicated
+whether changes made by a request (e.g. by the Bulk API) should be immediately
+visible in search, or not. Using `refresh=true` had the positive effect of
+immediately seeing the changes when searching; the negative effect is that
+it is a rather big performance hit.
+
+With 5.0, you now have the choice between these 3 values.
+
+* `"true"` - Refresh immediately
+* `"false"` - Do not refresh (the default value)
+* `"wait_for"` - Wait until ES made the document visible in search
+
+See [?refresh](https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-refresh.html) in the documentation.
+
+Notice that `true` and `false` (the boolean values) are no longer available
+now in Elastic. You must use a string instead, with one of the above values.
+
+## ReindexerService removed
+
+The `ReindexerService` was a custom solution that was started in the ES 1.x era
+to automate reindexing data, from one index to another or even between clusters.
+
+ES 2.3 introduced its own [Reindex API](https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-reindex.html)
+so we're going to remove our custom solution and ask you to use the native reindexer.
+
+The `ReindexService` is available via `client.Reindex()` (which used to point
+to the custom reindexer).
+
+## Delete By Query back in core
+
+The [Delete By Query API](https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-delete-by-query.html)
+was moved into a plugin in 2.0. Now its back in core with a complete rewrite based on the Bulk API.
+
+It has it's own endpoint at `/_delete_by_query`.
+
+Delete By Query, Reindex, and Update By Query are very similar under the hood.
+
+## Reindex, Delete By Query, and Update By Query response changed
+
+The response from the above APIs changed a bit. E.g. the `retries` value
+used to be an `int64` and returns separate values for `bulk` and `search` now:
+
+```
+// Old
+{
+    ...
+    "retries": 123,
+    ...
+}
+```
+
+```
+// New
+{
+    ...
+    "retries": {
+        "bulk": 123,
+        "search": 0
+    },
+    ...
+}
+```
+
+## ScanService removed
+
+The `ScanService` is removed. Use the (new) `ScrollService` instead.
+
+## New ScrollService
+
+There was confusion around `ScanService` and `ScrollService` doing basically
+the same. One was returning slices and didn't support all query details, the
+other returned one document after another and wasn't safe for concurrent use.
+So we merged the two and merged it into a new `ScrollService` that
+removes all the problems with the older services.
+
+In other words:
+If you used `ScanService`, switch to `ScrollService`.
+If you used the old `ScrollService`, you might need to fix some things but
+overall it should just work.
+
+Changes:
+- We replaced `elastic.EOS` with `io.EOF` to indicate the "end of scroll".
+
+TODO Not implemented yet
+
+## Suggesters
+
+They have been [completely rewritten in ES 5.0](https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking_50_suggester.html).
+
+Some changes:
+- Suggesters no longer have an [output](https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking_50_suggester.html#_simpler_completion_indexing).
+
+TODO Fix all structural changes in suggesters
+
+## Percolator
+
+Percolator has [changed considerably](https://www.elastic.co/guide/en/elasticsearch/reference/5.x/breaking_50_percolator.html).
+
+Elastic 5.0 adds the new
+[Percolator Query](https://www.elastic.co/guide/en/elasticsearch/reference/5.x/query-dsl-percolate-query.html)
+which can be used in combination with the new
+[Percolator type](https://www.elastic.co/guide/en/elasticsearch/reference/5.x/percolator.html).
+
+The Percolate service is removed from Elastic 5.0.
+
+## Remove Consistency, add WaitForActiveShards
+
+The `consistency` parameter has been removed in a lot of places, e.g. the Bulk,
+Index, Delete, Delete-by-Query, Reindex, Update, and Update-by-Query API.
+
+It has been replaced by a somewhat similar `wait_for_active_shards` parameter.
+See https://github.com/elastic/elasticsearch/pull/19454.

vendor/gopkg.in/olivere/elastic.v6/CHANGELOG-6.0.md

+# Changes from 5.0 to 6.0
+
+See [breaking changes](https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking-changes-6.0.html).
+
+## _all removed
+
+6.0 has removed support for the `_all` field.
+
+## Boolean values coerced
+
+Only use `true` or `false` for boolean values, not `0` or `1` or `on` or `off`.
+
+## Single Type Indices
+
+Notice that 6.0 and future versions will default to single type indices, i.e. you may not use multiple types when e.g. adding an index with a mapping.
+
+See [here for details](https://www.elastic.co/guide/en/elasticsearch/reference/6.x/removal-of-types.html#_what_are_mapping_types).
+

vendor/gopkg.in/olivere/elastic.v6/CODE_OF_CONDUCT.md

+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at oliver@eilhard.net. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

vendor/gopkg.in/olivere/elastic.v6/CONTRIBUTING.md

+# How to contribute
+
+Elastic is an open-source project and we are looking forward to each
+contribution.
+
+Notice that while the [official Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html) is rather good, it is a high-level
+overview of the features of Elasticsearch. However, Elastic tries to resemble
+the Java API of Elasticsearch which you can find [on GitHub](https://github.com/elastic/elasticsearch).
+
+This explains why you might think that some options are strange or missing
+in Elastic, while often they're just different. Please check the Java API first.
+
+Having said that: Elasticsearch is moving fast and it might be very likely
+that we missed some features or changes. Feel free to change that.
+
+## Your Pull Request
+
+To make it easy to review and understand your changes, please keep the
+following things in mind before submitting your pull request:
+
+* You compared the existing implemenation with the Java API, did you?
+* Please work on the latest possible state of `olivere/elastic`.
+  Use `release-branch.v2` for targeting Elasticsearch 1.x and
+  `release-branch.v3` for targeting 2.x.
+* Create a branch dedicated to your change.
+* If possible, write a test case which confirms your change.
+* Make sure your changes and your tests work with all recent versions of
+  Elasticsearch. We currently support Elasticsearch 1.7.x in the
+  release-branch.v2 and Elasticsearch 2.x in the release-branch.v3.
+* Test your changes before creating a pull request (`go test ./...`).
+* Don't mix several features or bug fixes in one pull request.
+* Create a meaningful commit message.
+* Explain your change, e.g. provide a link to the issue you are fixing and
+  probably a link to the Elasticsearch documentation and/or source code.
+* Format your source with `go fmt`.
+
+## Additional Resources
+
+* [GitHub documentation](http://help.github.com/)
+* [GitHub pull request documentation](http://help.github.com/send-pull-requests/)

vendor/gopkg.in/olivere/elastic.v6/CONTRIBUTORS

+# This is a list of people who have contributed code
+# to the Elastic repository.
+#
+# It is just my small "thank you" to all those that helped
+# making Elastic what it is.
+#
+# Please keep this list sorted.
+
+0x6875790d0a [@huydx](https://github.com/huydx)
+Adam Alix [@adamalix](https://github.com/adamalix)
+Adam Weiner [@adamweiner](https://github.com/adamweiner)
+Adrian Lungu [@AdrianLungu](https://github.com/AdrianLungu)
+alehano [@alehano](https://github.com/alehano)
+Alex [@akotlar](https://github.com/akotlar)
+Alexander Sack [@asac](https://github.com/asac)
+Alexandre Olivier [@aliphen](https://github.com/aliphen)
+Alexey Sharov [@nizsheanez](https://github.com/nizsheanez)
+AndreKR [@AndreKR](https://github.com/AndreKR)
+André Bierlein [@ligustah](https://github.com/ligustah)
+Andrew Dunham [@andrew-d](https://github.com/andrew-d)
+Andrew Gaul [@andrewgaul](https://github.com/andrewgaul)
+Andy Walker [@alaska](https://github.com/alaska)
+Arquivei [@arquivei](https://github.com/arquivei)
+arthurgustin [@arthurgustin](https://github.com/arthurgustin)
+Benjamin Fernandes [@LotharSee](https://github.com/LotharSee)
+Benjamin Zarzycki [@kf6nux](https://github.com/kf6nux)
+Boris Popovschi [@Zyqsempai](https://github.com/Zyqsempai)
+Braden Bassingthwaite [@bbassingthwaite-va](https://github.com/bbassingthwaite-va)
+Brady Love [@bradylove](https://github.com/bradylove)
+Bryan Conklin [@bmconklin](https://github.com/bmconklin)
+Bruce Zhou [@brucez-isell](https://github.com/brucez-isell)
+Carl Dunham [@carldunham](https://github.com/carldunham)
+César Jiménez [@cesarjimenez](https://github.com/cesarjimenez)
+cforbes [@cforbes](https://github.com/cforbes)
+Chris M [@tebriel](https://github.com/tebriel)
+Chris Rice [@donutmonger](https://github.com/donutmonger)
+Claudiu Olteanu [@claudiuolteanu](https://github.com/claudiuolteanu)
+Chris Duncan [@veqryn](https://github.com/veqryn)
+Christophe Courtaut [@kri5](https://github.com/kri5)
+cmitchell [@cmitchell](https://github.com/cmitchell)
+Connor Peet [@connor4312](https://github.com/connor4312)
+Conrad Pankoff [@deoxxa](https://github.com/deoxxa)
+Corey Scott [@corsc](https://github.com/corsc)
+Daniel Barrett [@shendaras](https://github.com/shendaras)
+Daniel Heckrath [@DanielHeckrath](https://github.com/DanielHeckrath)
+Daniel Imfeld [@dimfeld](https://github.com/dimfeld)
+David Emanuel Buchmann [@wuurrd](https://github.com/wuurrd)
+Dwayne Schultz [@myshkin5](https://github.com/myshkin5)
+Ellison Leão [@ellisonleao](https://github.com/ellisonleao)
+Erik Grinaker [@erikgrinaker](https://github.com/erikgrinaker)
+Erwin [@eticzon](https://github.com/eticzon)
+Eugene Egorov [@EugeneEgorov](https://github.com/EugeneEgorov)
+Evan Shaw [@edsrzf](https://github.com/edsrzf)
+Fanfan [@wenpos](https://github.com/wenpos)
+Faolan C-P [@fcheslack](https://github.com/fcheslack)
+Filip Tepper [@filiptepper](https://github.com/filiptepper)
+Garrett Kelley [@GarrettKelley](https://github.com/GarrettKelley)
+Gaspard Douady [@plopik](https://github.com/plopik)
+Gaylord Aulke [@blafasel42](https://github.com/blafasel42)
+Gerhard Häring [@ghaering](https://github.com/ghaering)
+Guilherme Silveira [@guilherme-santos](https://github.com/guilherme-santos)
+Guillaume J. Charmes [@creack](https://github.com/creack)
+Guiseppe [@gm42](https://github.com/gm42)
+Han Yu [@MoonighT](https://github.com/MoonighT)
+Harmen [@alicebob](https://github.com/alicebob)
+Harrison Wright [@wright8191](https://github.com/wright8191)
+Henry Clifford [@hcliff](https://github.com/hcliff)
+Igor Dubinskiy [@idubinskiy](https://github.com/idubinskiy)
+initialcontext [@initialcontext](https://github.com/initialcontext)
+Isaac Saldana [@isaldana](https://github.com/isaldana)
+J Barkey Wolf [@jjhbw](https://github.com/jjhbw)
+Jack Lindamood [@cep21](https://github.com/cep21)
+Jacob [@jdelgad](https://github.com/jdelgad)
+Jayme Rotsaert [@jrots](https://github.com/jrots)
+Jeff Rand [@jeffrand](https://github.com/jeffrand)
+Jeremy Canady [@jrmycanady](https://github.com/jrmycanady)
+Jérémie Vexiau [@texvex](https://github.com/texvex)
+Jim Berlage [@jimberlage](https://github.com/jimberlage)
+Joe Buck [@four2five](https://github.com/four2five)
+John Barker [@j16r](https://github.com/j16r)
+John Goodall [@jgoodall](https://github.com/jgoodall)
+John Stanford [@jxstanford](https://github.com/jxstanford)
+Jonas Groenaas Drange [@semafor](https://github.com/semafor)
+Josef Fröhle [@Dexus](https://github.com/Dexus)
+José Martínez [@xose](https://github.com/xose)
+Josh Chorlton [@jchorl](https://github.com/jchorl)
+jun [@coseyo](https://github.com/coseyo)
+Junpei Tsuji [@jun06t](https://github.com/jun06t)
+kartlee [@kartlee](https://github.com/kartlee)
+Keith Hatton [@khatton-ft](https://github.com/khatton-ft)
+kel [@liketic](https://github.com/liketic)
+Kenta SUZUKI [@suzuken](https://github.com/suzuken)
+Kevin Mulvey [@kmulvey](https://github.com/kmulvey)
+Kyle Brandt [@kylebrandt](https://github.com/kylebrandt)
+Leandro Piccilli [@lpic10](https://github.com/lpic10)
+Lee [@leezhm](https://github.com/leezhm)
+M. Zulfa Achsani [@misterciput](https://github.com/misterciput)
+Maciej Lisiewski [@c2h5oh](https://github.com/c2h5oh)
+Mara Kim [@autochthe](https://github.com/autochthe)
+Marcy Buccellato [@marcybuccellato](https://github.com/marcybuccellato)
+Mark Costello [@mcos](https://github.com/mcos)
+Martin Häger [@protomouse](https://github.com/protomouse)
+Medhi Bechina [@mdzor](https://github.com/mdzor)
+Mike Beshai [@mbesh](https://github.com/mbesh)
+mmfrb [@mmfrb](https://github.com/mmfrb)
+mnpritula [@mnpritula](https://github.com/mnpritula)
+mosa [@mosasiru](https://github.com/mosasiru)
+Muhammet Çakır [@cakirmuha](https://github.com/cakirmuha)
+naimulhaider [@naimulhaider](https://github.com/naimulhaider)
+Naoya Yoshizawa [@azihsoyn](https://github.com/azihsoyn)
+navins [@ishare](https://github.com/ishare)
+Naoya Tsutsumi [@tutuming](https://github.com/tutuming)
+NeoCN [@NeoCN](https://github.com/NeoCN)
+Nicholas Wolff [@nwolff](https://github.com/nwolff)
+Nick K [@utrack](https://github.com/utrack)
+Nick Whyte [@nickw444](https://github.com/nickw444)
+Nicolae Vartolomei [@nvartolomei](https://github.com/nvartolomei)
+Orne Brocaar [@brocaar](https://github.com/brocaar)
+Paul [@eyeamera](https://github.com/eyeamera)
+Paul Oldenburg [@lr-paul](https://github.com/lr-paul)
+Pete C [@peteclark-ft](https://github.com/peteclark-ft)
+Paolo [@ppiccolo](https://github.com/ppiccolo)
+Radoslaw Wesolowski [r--w](https://github.com/r--w)
+rchicoli [@rchicoli](https://github.com/rchicoli)
+Roman Colohanin [@zuzmic](https://github.com/zuzmic)
+Ryan Schmukler [@rschmukler](https://github.com/rschmukler)
+Ryan Wynn [@rwynn](https://github.com/rwynn)
+Sacheendra talluri [@sacheendra](https://github.com/sacheendra)
+Sean DuBois [@Sean-Der](https://github.com/Sean-Der)
+Sagan Yaroslav [@sgnrslv](https://github.com/sgnrslv)
+Shalin LK [@shalinlk](https://github.com/shalinlk)
+singham [@zhaochenxiao90](https://github.com/zhaochenxiao90)
+Slawomir CALUCH [@slawo](https://github.com/slawo)
+Stephan Krynauw [@skrynauw](https://github.com/skrynauw)
+Stephen Kubovic [@stephenkubovic](https://github.com/stephenkubovic)
+Stuart Warren [@Woz](https://github.com/stuart-warren)
+Sulaiman [@salajlan](https://github.com/salajlan)
+Sundar [@sundarv85](https://github.com/sundarv85)
+Swarlston [@Swarlston](https://github.com/Swarlston)
+Take [ww24](https://github.com/ww24)
+Tetsuya Morimoto [@t2y](https://github.com/t2y)
+TheZeroSlave [@TheZeroSlave](https://github.com/TheZeroSlave)
+TimeEmit [@TimeEmit](https://github.com/timeemit)
+TusharM [@tusharm](https://github.com/tusharm)
+wangtuo [@wangtuo](https://github.com/wangtuo)
+Wédney Yuri [@wedneyyuri](https://github.com/wedneyyuri)
+wolfkdy [@wolfkdy](https://github.com/wolfkdy)
+Wyndham Blanton [@wyndhblb](https://github.com/wyndhblb)
+Yarden Bar [@ayashjorden](https://github.com/ayashjorden)
+zakthomas [@zakthomas](https://github.com/zakthomas)
+Yuya Kusakabe [@higebu](https://github.com/higebu)
+Zach [@snowzach](https://github.com/snowzach)
+zhangxin [@visaxin](https://github.com/visaxin)
+@林 [@zplzpl](https://github.com/zplzpl)

vendor/gopkg.in/olivere/elastic.v6/ISSUE_TEMPLATE.md

+Please use the following questions as a guideline to help me answer
+your issue/question without further inquiry. Thank you.
+
+### Which version of Elastic are you using?
+
+[ ] elastic.v6 (for Elasticsearch 6.x)
+[ ] elastic.v5 (for Elasticsearch 5.x)
+[ ] elastic.v3 (for Elasticsearch 2.x)
+[ ] elastic.v2 (for Elasticsearch 1.x)
+
+### Please describe the expected behavior
+
+
+### Please describe the actual behavior
+
+
+### Any steps to reproduce the behavior?
+

vendor/gopkg.in/olivere/elastic.v6/LICENSE

+The MIT License (MIT)
+Copyright © 2012-2015 Oliver Eilhard
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the “Software”), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included
+in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+IN THE SOFTWARE.

vendor/gopkg.in/olivere/elastic.v6/README.md

+# Elastic
+
+**This is a development branch that is actively being worked on. DO NOT USE IN PRODUCTION! If you want to use stable versions of Elastic, please use a dependency manager like [dep](https://github.com/golang/dep).**
+
+Elastic is an [Elasticsearch](http://www.elasticsearch.org/) client for the
+[Go](http://www.golang.org/) programming language.
+
+[![Build Status](https://travis-ci.org/olivere/elastic.svg?branch=release-branch.v6)](https://travis-ci.org/olivere/elastic)
+[![Godoc](http://img.shields.io/badge/godoc-reference-blue.svg?style=flat)](http://godoc.org/github.com/olivere/elastic)
+[![license](http://img.shields.io/badge/license-MIT-red.svg?style=flat)](https://raw.githubusercontent.com/olivere/elastic/master/LICENSE)
+
+See the [wiki](https://github.com/olivere/elastic/wiki) for additional information about Elastic.
+
+<a href="https://www.buymeacoffee.com/Bjd96U8fm" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Buy Me A Coffee" style="height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a>
+
+
+## Releases
+
+**The release branches (e.g. [`release-branch.v6`](https://github.com/olivere/elastic/tree/release-branch.v6))
+are actively being worked on and can break at any time.
+If you want to use stable versions of Elastic, please use a dependency manager like [dep](https://github.com/golang/dep).**
+
+Here's the version matrix:
+
+Elasticsearch version | Elastic version  | Package URL | Remarks |
+----------------------|------------------|-------------|---------|
+6.x                   | 6.0              | [`github.com/olivere/elastic`](https://github.com/olivere/elastic) ([source](https://github.com/olivere/elastic/tree/release-branch.v6) [doc](http://godoc.org/github.com/olivere/elastic)) | Use a dependency manager (see below).
+5.x                   | 5.0              | [`gopkg.in/olivere/elastic.v5`](https://gopkg.in/olivere/elastic.v5) ([source](https://github.com/olivere/elastic/tree/release-branch.v5) [doc](http://godoc.org/gopkg.in/olivere/elastic.v5)) | Actively maintained.
+2.x                   | 3.0              | [`gopkg.in/olivere/elastic.v3`](https://gopkg.in/olivere/elastic.v3) ([source](https://github.com/olivere/elastic/tree/release-branch.v3) [doc](http://godoc.org/gopkg.in/olivere/elastic.v3)) | Deprecated. Please update.
+1.x                   | 2.0              | [`gopkg.in/olivere/elastic.v2`](https://gopkg.in/olivere/elastic.v2) ([source](https://github.com/olivere/elastic/tree/release-branch.v2) [doc](http://godoc.org/gopkg.in/olivere/elastic.v2)) | Deprecated. Please update.
+0.9-1.3               | 1.0              | [`gopkg.in/olivere/elastic.v1`](https://gopkg.in/olivere/elastic.v1) ([source](https://github.com/olivere/elastic/tree/release-branch.v1) [doc](http://godoc.org/gopkg.in/olivere/elastic.v1)) | Deprecated. Please update.
+
+**Example:**
+
+You have installed Elasticsearch 6.0.0 and want to use Elastic.
+As listed above, you should use Elastic 6.0.
+
+To use the required version of Elastic in your application, it is strongly
+advised to use a tool like
+[dep](https://github.com/golang/dep)
+or
+[Go modules](https://github.com/golang/go/wiki/Modules)
+to manage dependencies. Make sure to use a version such as `^6.0.0`.
+
+To use Elastic, import:
+
+```go
+import "github.com/olivere/elastic"
+```
+
+### Elastic 6.0
+
+Elastic 6.0 targets Elasticsearch 6.x which was [released on 14th November 2017](https://www.elastic.co/blog/elasticsearch-6-0-0-released).
+
+Notice that there are a lot of [breaking changes in Elasticsearch 6.0](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/breaking-changes-6.0.html)
+and we used this as an opportunity to [clean up and refactor Elastic](https://github.com/olivere/elastic/blob/release-branch.v6/CHANGELOG-6.0.md)
+as we did in the transition from earlier versions of Elastic.
+
+### Elastic 5.0
+
+Elastic 5.0 targets Elasticsearch 5.0.0 and later. Elasticsearch 5.0.0 was
+[released on 26th October 2016](https://www.elastic.co/blog/elasticsearch-5-0-0-released).
+
+Notice that there are will be a lot of [breaking changes in Elasticsearch 5.0](https://www.elastic.co/guide/en/elasticsearch/reference/5.0/breaking-changes-5.0.html)
+and we used this as an opportunity to [clean up and refactor Elastic](https://github.com/olivere/elastic/blob/release-branch.v5/CHANGELOG-5.0.md)
+as we did in the transition from Elastic 2.0 (for Elasticsearch 1.x) to Elastic 3.0 (for Elasticsearch 2.x).
+
+Furthermore, the jump in version numbers will give us a chance to be in sync with the Elastic Stack.
+
+### Elastic 3.0
+
+Elastic 3.0 targets Elasticsearch 2.x and is published via [`gopkg.in/olivere/elastic.v3`](https://gopkg.in/olivere/elastic.v3).
+
+Elastic 3.0 will only get critical bug fixes. You should update to a recent version.
+
+### Elastic 2.0
+
+Elastic 2.0 targets Elasticsearch 1.x and is published via [`gopkg.in/olivere/elastic.v2`](https://gopkg.in/olivere/elastic.v2).
+
+Elastic 2.0 will only get critical bug fixes. You should update to a recent version.
+
+### Elastic 1.0
+
+Elastic 1.0 is deprecated. You should really update Elasticsearch and Elastic
+to a recent version.
+
+However, if you cannot update for some reason, don't worry. Version 1.0 is
+still available. All you need to do is go-get it and change your import path
+as described above.
+
+
+## Status
+
+We use Elastic in production since 2012. Elastic is stable but the API changes
+now and then. We strive for API compatibility.
+However, Elasticsearch sometimes introduces [breaking changes](https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking-changes.html)
+and we sometimes have to adapt.
+
+Having said that, there have been no big API changes that required you
+to rewrite your application big time. More often than not it's renaming APIs
+and adding/removing features so that Elastic is in sync with Elasticsearch.
+
+Elastic has been used in production starting with Elasticsearch 0.90 up to recent 6.x
+versions. Furthermore, we use [Travis CI](https://travis-ci.org/)
+to test Elastic with the most recent versions of Elasticsearch and Go.
+See the [.travis.yml](https://github.com/olivere/elastic/blob/master/.travis.yml)
+file for the exact matrix and [Travis](https://travis-ci.org/olivere/elastic)
+for the results.
+
+Elasticsearch has quite a few features. Most of them are implemented
+by Elastic. I add features and APIs as required. It's straightforward
+to implement missing pieces. I'm accepting pull requests :-)
+
+Having said that, I hope you find the project useful.
+
+
+## Getting Started
+
+The first thing you do is to create a [Client](https://github.com/olivere/elastic/blob/master/client.go).
+The client connects to Elasticsearch on `http://127.0.0.1:9200` by default.
+
+You typically create one client for your app. Here's a complete example of
+creating a client, creating an index, adding a document, executing a search etc.
+
+An example is available [here](https://olivere.github.io/elastic/).
+
+Here's a [link to a complete working example for v6](https://gist.github.com/olivere/e4a376b4783c0914e44ea4f745ce2ebf).
+
+Here are a few tips on how to get used to Elastic:
+
+1. Head over to the [Wiki](https://github.com/olivere/elastic/wiki) for detailed information and
+   topics like e.g. [how to add a middleware](/olivere/elastic/wiki/HttpTransport)
+   or how to [connect to AWS](/olivere/elastic/wiki/Using-with-AWS-Elasticsearch-Service).
+2. If you are unsure how to implement something, read the tests (all `_test.go` files).
+   They not only serve as a guard against changes, but also as a reference.
+3. The [recipes](https://github.com/olivere/elastic/tree/release-branch.v6/recipes)
+   contains small examples on how to implement something, e.g. bulk indexing, scrolling etc.
+
+
+## API Status
+
+### Document APIs
+
+- [x] Index API
+- [x] Get API
+- [x] Delete API
+- [x] Delete By Query API
+- [x] Update API
+- [x] Update By Query API
+- [x] Multi Get API
+- [x] Bulk API
+- [x] Reindex API
+- [x] Term Vectors
+- [x] Multi termvectors API
+
+### Search APIs
+
+- [x] Search
+- [x] Search Template
+- [ ] Multi Search Template
+- [x] Search Shards API
+- [x] Suggesters
+  - [x] Term Suggester
+  - [x] Phrase Suggester
+  - [x] Completion Suggester
+  - [x] Context Suggester
+- [x] Multi Search API
+- [x] Count API
+- [x] Validate API
+- [x] Explain API
+- [x] Profile API
+- [x] Field Capabilities API
+
+### Aggregations
+
+- Metrics Aggregations
+  - [x] Avg
+  - [x] Cardinality
+  - [x] Extended Stats
+  - [x] Geo Bounds
+  - [x] Geo Centroid
+  - [x] Max
+  - [x] Min
+  - [x] Percentiles
+  - [x] Percentile Ranks
+  - [ ] Scripted Metric
+  - [x] Stats
+  - [x] Sum
+  - [x] Top Hits
+  - [x] Value Count
+- Bucket Aggregations
+  - [x] Adjacency Matrix
+  - [x] Children
+  - [x] Date Histogram
+  - [x] Date Range
+  - [x] Diversified Sampler
+  - [x] Filter
+  - [x] Filters
+  - [x] Geo Distance
+  - [ ] GeoHash Grid
+  - [x] Global
+  - [x] Histogram
+  - [x] IP Range
+  - [x] Missing
+  - [x] Nested
+  - [x] Range
+  - [x] Reverse Nested
+  - [x] Sampler
+  - [x] Significant Terms
+  - [x] Significant Text
+  - [x] Terms
+  - [x] Composite
+- Pipeline Aggregations
+  - [x] Avg Bucket
+  - [x] Derivative
+  - [x] Max Bucket
+  - [x] Min Bucket
+  - [x] Sum Bucket
+  - [x] Stats Bucket
+  - [ ] Extended Stats Bucket
+  - [x] Percentiles Bucket
+  - [x] Moving Average
+  - [x] Cumulative Sum
+  - [x] Bucket Script
+  - [x] Bucket Selector
+  - [x] Bucket Sort
+  - [x] Serial Differencing
+- [x] Matrix Aggregations
+  - [x] Matrix Stats
+- [x] Aggregation Metadata
+
+### Indices APIs
+
+- [x] Create Index
+- [x] Delete Index
+- [x] Get Index
+- [x] Indices Exists
+- [x] Open / Close Index
+- [x] Shrink Index
+- [x] Rollover Index
+- [x] Put Mapping
+- [x] Get Mapping
+- [x] Get Field Mapping
+- [x] Types Exists
+- [x] Index Aliases
+- [x] Update Indices Settings
+- [x] Get Settings
+- [x] Analyze
+  - [x] Explain Analyze
+- [x] Index Templates
+- [x] Indices Stats
+- [x] Indices Segments
+- [ ] Indices Recovery
+- [ ] Indices Shard Stores
+- [ ] Clear Cache
+- [x] Flush
+  - [x] Synced Flush
+- [x] Refresh
+- [x] Force Merge
+
+### cat APIs
+
+The cat APIs are not implemented as of now. We think they are better suited for operating with Elasticsearch on the command line.
+
+- [ ] cat aliases
+- [ ] cat allocation
+- [ ] cat count
+- [ ] cat fielddata
+- [ ] cat health
+- [ ] cat indices
+- [ ] cat master
+- [ ] cat nodeattrs
+- [ ] cat nodes
+- [ ] cat pending tasks
+- [ ] cat plugins
+- [ ] cat recovery
+- [ ] cat repositories
+- [ ] cat thread pool
+- [ ] cat shards
+- [ ] cat segments
+- [ ] cat snapshots
+- [ ] cat templates
+
+### Cluster APIs
+
+- [x] Cluster Health
+- [x] Cluster State
+- [x] Cluster Stats
+- [ ] Pending Cluster Tasks
+- [x] Cluster Reroute
+- [ ] Cluster Update Settings
+- [x] Nodes Stats
+- [x] Nodes Info
+- [ ] Nodes Feature Usage
+- [ ] Remote Cluster Info
+- [x] Task Management API
+- [ ] Nodes hot_threads
+- [ ] Cluster Allocation Explain API
+
+### Query DSL
+
+- [x] Match All Query
+- [x] Inner hits
+- Full text queries
+  - [x] Match Query
+  - [x] Match Phrase Query
+  - [x] Match Phrase Prefix Query
+  - [x] Multi Match Query
+  - [x] Common Terms Query
+  - [x] Query String Query
+  - [x] Simple Query String Query
+- Term level queries
+  - [x] Term Query
+  - [x] Terms Query
+  - [x] Terms Set Query
+  - [x] Range Query
+  - [x] Exists Query
+  - [x] Prefix Query
+  - [x] Wildcard Query
+  - [x] Regexp Query
+  - [x] Fuzzy Query
+  - [x] Type Query
+  - [x] Ids Query
+- Compound queries
+  - [x] Constant Score Query
+  - [x] Bool Query
+  - [x] Dis Max Query
+  - [x] Function Score Query
+  - [x] Boosting Query
+- Joining queries
+  - [x] Nested Query
+  - [x] Has Child Query
+  - [x] Has Parent Query
+  - [x] Parent Id Query
+- Geo queries
+  - [ ] GeoShape Query
+  - [x] Geo Bounding Box Query
+  - [x] Geo Distance Query
+  - [x] Geo Polygon Query
+- Specialized queries
+  - [x] More Like This Query
+  - [x] Script Query
+  - [x] Percolate Query
+- Span queries
+  - [ ] Span Term Query
+  - [ ] Span Multi Term Query
+  - [ ] Span First Query
+  - [ ] Span Near Query
+  - [ ] Span Or Query
+  - [ ] Span Not Query
+  - [ ] Span Containing Query
+  - [ ] Span Within Query
+  - [ ] Span Field Masking Query
+- [ ] Minimum Should Match
+- [ ] Multi Term Query Rewrite
+
+### Modules
+
+- Snapshot and Restore
+  - [x] Repositories
+  - [x] Snapshot
+  - [ ] Restore
+  - [ ] Snapshot status
+  - [ ] Monitoring snapshot/restore status
+  - [ ] Stopping currently running snapshot and restore
+- Scripting
+  - [x] GetScript
+  - [x] PutScript
+  - [x] DeleteScript
+
+### Sorting
+
+- [x] Sort by score
+- [x] Sort by field
+- [x] Sort by geo distance
+- [x] Sort by script
+- [x] Sort by doc
+
+### Scrolling
+
+Scrolling is supported via a  `ScrollService`. It supports an iterator-like interface.
+The `ClearScroll` API is implemented as well.
+
+A pattern for [efficiently scrolling in parallel](https://github.com/olivere/elastic/wiki/ScrollParallel)
+is described in the [Wiki](https://github.com/olivere/elastic/wiki).
+
+## How to contribute
+
+Read [the contribution guidelines](https://github.com/olivere/elastic/blob/master/CONTRIBUTING.md).
+
+## Credits
+
+Thanks a lot for the great folks working hard on
+[Elasticsearch](https://www.elastic.co/products/elasticsearch)
+and
+[Go](https://golang.org/).
+
+Elastic uses portions of the
+[uritemplates](https://github.com/jtacoma/uritemplates) library
+by Joshua Tacoma,
+[backoff](https://github.com/cenkalti/backoff) by Cenk Altı and
+[leaktest](https://github.com/fortytw2/leaktest) by Ian Chiles.
+
+## LICENSE
+
+MIT-LICENSE. See [LICENSE](http://olivere.mit-license.org/)
+or the LICENSE file provided in the repository for details.

vendor/gopkg.in/olivere/elastic.v6/acknowledged_response.go

+// Copyright 2012-present Oliver Eilhard. All rights reserved.
+// Use of this source code is governed by a MIT-license.
+// See http://olivere.mit-license.org/license.txt for details.
+
+package elastic
+
+// AcknowledgedResponse is returned from various APIs. It simply indicates
+// whether the operation is ack'd or not.
+type AcknowledgedResponse struct {
+	Acknowledged       bool   `json:"acknowledged"`
+	ShardsAcknowledged bool   `json:"shards_acknowledged"`
+	Index              string `json:"index,omitempty"`
+}

vendor/gopkg.in/olivere/elastic.v6/backoff.go

+// Copyright 2012-present Oliver Eilhard. All rights reserved.
+// Use of this source code is governed by a MIT-license.
+// See http://olivere.mit-license.org/license.txt for details.
+
+package elastic
+
+import (
+	"math"
+	"math/rand"
+	"sync"
+	"time"
+)
+
+// BackoffFunc specifies the signature of a function that returns the
+// time to wait before the next call to a resource. To stop retrying
+// return false in the 2nd return value.
+type BackoffFunc func(retry int) (time.Duration, bool)
+
+// Backoff allows callers to implement their own Backoff strategy.
+type Backoff interface {
+	// Next implements a BackoffFunc.
+	Next(retry int) (time.Duration, bool)
+}
+
+// -- ZeroBackoff --
+
+// ZeroBackoff is a fixed backoff policy whose backoff time is always zero,
+// meaning that the operation is retried immediately without waiting,
+// indefinitely.
+type ZeroBackoff struct{}
+
+// Next implements BackoffFunc for ZeroBackoff.
+func (b ZeroBackoff) Next(retry int) (time.Duration, bool) {
+	return 0, true
+}
+
+// -- StopBackoff --
+
+// StopBackoff is a fixed backoff policy that always returns false for
+// Next(), meaning that the operation should never be retried.
+type StopBackoff struct{}
+
+// Next implements BackoffFunc for StopBackoff.
+func (b StopBackoff) Next(retry int) (time.Duration, bool) {
+	return 0, false
+}
+
+// -- ConstantBackoff --
+
+// ConstantBackoff is a backoff policy that always returns the same delay.
+type ConstantBackoff struct {
+	interval time.Duration
+}
+
+// NewConstantBackoff returns a new ConstantBackoff.
+func NewConstantBackoff(interval time.Duration) *ConstantBackoff {
+	return &ConstantBackoff{interval: interval}
+}
+
+// Next implements BackoffFunc for ConstantBackoff.
+func (b *ConstantBackoff) Next(retry int) (time.Duration, bool) {
+	return b.interval, true
+}
+
+// -- Exponential --
+
+// ExponentialBackoff implements the simple exponential backoff described by
+// Douglas Thain at http://dthain.blogspot.de/2009/02/exponential-backoff-in-distributed.html.
+type ExponentialBackoff struct {
+	t float64 // initial timeout (in msec)
+	f float64 // exponential factor (e.g. 2)
+	m float64 // maximum timeout (in msec)
+}
+
+// NewExponentialBackoff returns a ExponentialBackoff backoff policy.
+// Use initialTimeout to set the first/minimal interval
+// and maxTimeout to set the maximum wait interval.
+func NewExponentialBackoff(initialTimeout, maxTimeout time.Duration) *ExponentialBackoff {
+	return &ExponentialBackoff{
+		t: float64(int64(initialTimeout / time.Millisecond)),
+		f: 2.0,
+		m: float64(int64(maxTimeout / time.Millisecond)),
+	}
+}
+
+// Next implements BackoffFunc for ExponentialBackoff.
+func (b *ExponentialBackoff) Next(retry int) (time.Duration, bool) {
+	r := 1.0 + rand.Float64() // random number in [1..2]
+	m := math.Min(r*b.t*math.Pow(b.f, float64(retry)), b.m)
+	if m >= b.m {
+		return 0, false
+	}
+	d := time.Duration(int64(m)) * time.Millisecond
+	return d, true
+}
+
+// -- Simple Backoff --
+
+// SimpleBackoff takes a list of fixed values for backoff intervals.
+// Each call to Next returns the next value from that fixed list.
+// After each value is returned, subsequent calls to Next will only return
+// the last element. The values are optionally "jittered" (off by default).
+type SimpleBackoff struct {
+	sync.Mutex
+	ticks  []int
+	jitter bool
+}
+
+// NewSimpleBackoff creates a SimpleBackoff algorithm with the specified
+// list of fixed intervals in milliseconds.
+func NewSimpleBackoff(ticks ...int) *SimpleBackoff {
+	return &SimpleBackoff{
+		ticks:  ticks,
+		jitter: false,
+	}
+}
+
+// Jitter enables or disables jittering values.
+func (b *SimpleBackoff) Jitter(flag bool) *SimpleBackoff {
+	b.Lock()
+	b.jitter = flag
+	b.Unlock()
+	return b
+}
+
+// jitter randomizes the interval to return a value of [0.5*millis .. 1.5*millis].
+func jitter(millis int) int {
+	if millis <= 0 {
+		return 0
+	}
+	return millis/2 + rand.Intn(millis)
+}
+
+// Next implements BackoffFunc for SimpleBackoff.
+func (b *SimpleBackoff) Next(retry int) (time.Duration, bool) {
+	b.Lock()
+	defer b.Unlock()
+
+	if retry >= len(b.ticks) {
+		return 0, false
+	}
+
+	ms := b.ticks[retry]
+	if b.jitter {
+		ms = jitter(ms)
+	}
+	return time.Duration(ms) * time.Millisecond, true
+}

vendor/gopkg.in/olivere/elastic.v6/backoff_test.go

+package elastic
+
+import (
+	"math/rand"
+	"testing"
+	"time"
+)
+
+func TestZeroBackoff(t *testing.T) {
+	b := ZeroBackoff{}
+	_, ok := b.Next(0)
+	if !ok {
+		t.Fatalf("expected %v, got %v", true, ok)
+	}
+}
+
+func TestStopBackoff(t *testing.T) {
+	b := StopBackoff{}
+	_, ok := b.Next(0)
+	if ok {
+		t.Fatalf("expected %v, got %v", false, ok)
+	}
+}
+
+func TestConstantBackoff(t *testing.T) {
+	b := NewConstantBackoff(time.Second)
+	d, ok := b.Next(0)
+	if !ok {
+		t.Fatalf("expected %v, got %v", true, ok)
+	}
+	if d != time.Second {
+		t.Fatalf("expected %v, got %v", time.Second, d)
+	}
+}
+
+func TestSimpleBackoff(t *testing.T) {
+	var tests = []struct {
+		Duration time.Duration
+		Continue bool
+	}{
+		// #0
+		{
+			Duration: 1 * time.Millisecond,
+			Continue: true,
+		},
+		// #1
+		{
+			Duration: 2 * time.Millisecond,
+			Continue: true,
+		},
+		// #2
+		{
+			Duration: 7 * time.Millisecond,
+			Continue: true,
+		},
+		// #3
+		{
+			Duration: 0,
+			Continue: false,
+		},
+		// #4
+		{
+			Duration: 0,
+			Continue: false,
+		},
+	}
+
+	b := NewSimpleBackoff(1, 2, 7)
+
+	for i, tt := range tests {
+		d, ok := b.Next(i)
+		if got, want := ok, tt.Continue; got != want {
+			t.Fatalf("#%d: expected %v, got %v", i, want, got)
+		}
+		if got, want := d, tt.Duration; got != want {
+			t.Fatalf("#%d: expected %v, got %v", i, want, got)
+		}
+	}
+}
+
+func TestExponentialBackoff(t *testing.T) {
+	rand.Seed(time.Now().UnixNano())
+
+	min := time.Duration(8) * time.Millisecond
+	max := time.Duration(256) * time.Millisecond
+	b := NewExponentialBackoff(min, max)
+
+	between := func(value time.Duration, a, b int) bool {
+		x := int(value / time.Millisecond)
+		return a <= x && x <= b
+	}
+
+	got, ok := b.Next(0)
+	if !ok {
+		t.Fatalf("expected %v, got %v", true, ok)
+	}
+	if !between(got, 8, 256) {
+		t.Errorf("expected [%v..%v], got %v", 8, 256, got)
+	}
+
+	got, ok = b.Next(1)
+	if !ok {
+		t.Fatalf("expected %v, got %v", true, ok)
+	}
+	if !between(got, 8, 256) {
+		t.Errorf("expected [%v..%v], got %v", 8, 256, got)
+	}
+
+	got, ok = b.Next(2)
+	if !ok {
+		t.Fatalf("expected %v, got %v", true, ok)
+	}
+	if !between(got, 8, 256) {
+		t.Errorf("expected [%v..%v], got %v", 8, 256, got)
+	}
+
+	got, ok = b.Next(3)
+	if !ok {
+		t.Fatalf("expected %v, got %v", true, ok)
+	}
+	if !between(got, 8, 256) {
+		t.Errorf("expected [%v..%v], got %v", 8, 256, got)
+	}
+
+	got, ok = b.Next(4)
+	if !ok {
+		t.Fatalf("expected %v, got %v", true, ok)
+	}
+	if !between(got, 8, 256) {
+		t.Errorf("expected [%v..%v], got %v", 8, 256, got)
+	}
+
+	if _, ok := b.Next(5); ok {
+		t.Fatalf("expected %v, got %v", false, ok)
+	}
+
+	if _, ok = b.Next(6); ok {
+		t.Fatalf("expected %v, got %v", false, ok)
+	}
+}

vendor/gopkg.in/olivere/elastic.v6/bulk.go

+// Copyright 2012-present Oliver Eilhard. All rights reserved.
+// Use of this source code is governed by a MIT-license.
+// See http://olivere.mit-license.org/license.txt for details.
+
+package elastic
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"fmt"
+	"net/url"
+
+	"github.com/olivere/elastic/uritemplates"
+)
+
+// BulkService allows for batching bulk requests and sending them to
+// Elasticsearch in one roundtrip. Use the Add method with BulkIndexRequest,
+// BulkUpdateRequest, and BulkDeleteRequest to add bulk requests to a batch,
+// then use Do to send them to Elasticsearch.
+//
+// BulkService will be reset after each Do call. In other words, you can
+// reuse BulkService to send many batches. You do not have to create a new
+// BulkService for each batch.
+//
+// See https://www.elastic.co/guide/en/elasticsearch/reference/6.2/docs-bulk.html
+// for more details.
+type BulkService struct {
+	client  *Client
+	retrier Retrier
+
+	index               string
+	typ                 string
+	requests            []BulkableRequest
+	pipeline            string
+	timeout             string
+	refresh             string
+	routing             string
+	waitForActiveShards string
+	pretty              bool
+
+	// estimated bulk size in bytes, up to the request index sizeInBytesCursor
+	sizeInBytes       int64
+	sizeInBytesCursor int
+}
+
+// NewBulkService initializes a new BulkService.
+func NewBulkService(client *Client) *BulkService {
+	builder := &BulkService{
+		client: client,
+	}
+	return builder
+}
+
+// Reset cleans up the request queue
+func (s *BulkService) Reset() {
+	s.requests = make([]BulkableRequest, 0)
+	s.sizeInBytes = 0
+	s.sizeInBytesCursor = 0
+}
+
+// Retrier allows to set specific retry logic for this BulkService.
+// If not specified, it will use the client's default retrier.
+func (s *BulkService) Retrier(retrier Retrier) *BulkService {
+	s.retrier = retrier
+	return s
+}
+
+// Index specifies the index to use for all batches. You may also leave
+// this blank and specify the index in the individual bulk requests.
+func (s *BulkService) Index(index string) *BulkService {
+	s.index = index
+	return s
+}
+
+// Type specifies the type to use for all batches. You may also leave
+// this blank and specify the type in the individual bulk requests.
+func (s *BulkService) Type(typ string) *BulkService {
+	s.typ = typ
+	return s
+}
+
+// Timeout is a global timeout for processing bulk requests. This is a
+// server-side timeout, i.e. it tells Elasticsearch the time after which
+// it should stop processing.
+func (s *BulkService) Timeout(timeout string) *BulkService {
+	s.timeout = timeout
+	return s
+}
+
+// Refresh controls when changes made by this request are made visible
+// to search. The allowed values are: "true" (refresh the relevant
+// primary and replica shards immediately), "wait_for" (wait for the
+// changes to be made visible by a refresh before reying), or "false"
+// (no refresh related actions). The default value is "false".
+//
+// See https://www.elastic.co/guide/en/elasticsearch/reference/6.2/docs-refresh.html
+// for details.
+func (s *BulkService) Refresh(refresh string) *BulkService {
+	s.refresh = refresh
+	return s
+}
+
+// Routing specifies the routing value.
+func (s *BulkService) Routing(routing string) *BulkService {
+	s.routing = routing
+	return s
+}
+
+// Pipeline specifies the pipeline id to preprocess incoming documents with.
+func (s *BulkService) Pipeline(pipeline string) *BulkService {
+	s.pipeline = pipeline
+	return s
+}
+
+// WaitForActiveShards sets the number of shard copies that must be active
+// before proceeding with the bulk operation. Defaults to 1, meaning the
+// primary shard only. Set to `all` for all shard copies, otherwise set to
+// any non-negative value less than or equal to the total number of copies
+// for the shard (number of replicas + 1).
+func (s *BulkService) WaitForActiveShards(waitForActiveShards string) *BulkService {
+	s.waitForActiveShards = waitForActiveShards
+	return s
+}
+
+// Pretty tells Elasticsearch whether to return a formatted JSON response.
+func (s *BulkService) Pretty(pretty bool) *BulkService {
+	s.pretty = pretty
+	return s
+}
+
+// Add adds bulkable requests, i.e. BulkIndexRequest, BulkUpdateRequest,
+// and/or BulkDeleteRequest.
+func (s *BulkService) Add(requests ...BulkableRequest) *BulkService {
+	for _, r := range requests {
+		s.requests = append(s.requests, r)
+	}
+	return s
+}
+
+// EstimatedSizeInBytes returns the estimated size of all bulkable
+// requests added via Add.
+func (s *BulkService) EstimatedSizeInBytes() int64 {
+	if s.sizeInBytesCursor == len(s.requests) {
+		return s.sizeInBytes
+	}
+	for _, r := range s.requests[s.sizeInBytesCursor:] {
+		s.sizeInBytes += s.estimateSizeInBytes(r)
+		s.sizeInBytesCursor++
+	}
+	return s.sizeInBytes
+}
+
+// estimateSizeInBytes returns the estimates size of the given
+// bulkable request, i.e. BulkIndexRequest, BulkUpdateRequest, and
+// BulkDeleteRequest.
+func (s *BulkService) estimateSizeInBytes(r BulkableRequest) int64 {
+	lines, _ := r.Source()
+	size := 0
+	for _, line := range lines {
+		// +1 for the \n
+		size += len(line) + 1
+	}
+	return int64(size)
+}
+
+// NumberOfActions returns the number of bulkable requests that need to
+// be sent to Elasticsearch on the next batch.
+func (s *BulkService) NumberOfActions() int {
+	return len(s.requests)
+}
+
+func (s *BulkService) bodyAsString() (string, error) {
+	// Pre-allocate to reduce allocs
+	buf := bytes.NewBuffer(make([]byte, 0, s.EstimatedSizeInBytes()))
+
+	for _, req := range s.requests {
+		source, err := req.Source()
+		if err != nil {
+			return "", err
+		}
+		for _, line := range source {
+			buf.WriteString(line)
+			buf.WriteByte('\n')
+		}
+	}
+
+	return buf.String(), nil
+}
+
+// Do sends the batched requests to Elasticsearch. Note that, when successful,
+// you can reuse the BulkService for the next batch as the list of bulk
+// requests is cleared on success.
+func (s *BulkService) Do(ctx context.Context) (*BulkResponse, error) {
+	// No actions?
+	if s.NumberOfActions() == 0 {
+		return nil, errors.New("elastic: No bulk actions to commit")
+	}
+
+	// Get body
+	body, err := s.bodyAsString()
+	if err != nil {
+		return nil, err
+	}
+
+	// Build url
+	path := "/"
+	if len(s.index) > 0 {
+		index, err := uritemplates.Expand("{index}", map[string]string{
+			"index": s.index,
+		})
+		if err != nil {
+			return nil, err
+		}
+		path += index + "/"
+	}
+	if len(s.typ) > 0 {
+		typ, err := uritemplates.Expand("{type}", map[string]string{
+			"type": s.typ,
+		})
+		if err != nil {
+			return nil, err
+		}
+		path += typ + "/"
+	}
+	path += "_bulk"
+
+	// Parameters
+	params := make(url.Values)
+	if s.pretty {
+		params.Set("pretty", fmt.Sprintf("%v", s.pretty))
+	}
+	if s.pipeline != "" {
+		params.Set("pipeline", s.pipeline)
+	}
+	if s.refresh != "" {
+		params.Set("refresh", s.refresh)
+	}
+	if s.routing != "" {
+		params.Set("routing", s.routing)
+	}
+	if s.timeout != "" {
+		params.Set("timeout", s.timeout)
+	}
+	if s.waitForActiveShards != "" {
+		params.Set("wait_for_active_shards", s.waitForActiveShards)
+	}
+
+	// Get response
+	res, err := s.client.PerformRequest(ctx, PerformRequestOptions{
+		Method:      "POST",
+		Path:        path,
+		Params:      params,
+		Body:        body,
+		ContentType: "application/x-ndjson",
+		Retrier:     s.retrier,
+	})
+	if err != nil {
+		return nil, err
+	}
+
+	// Return results
+	ret := new(BulkResponse)
+	if err := s.client.decoder.Decode(res.Body, ret); err != nil {
+		return nil, err
+	}
+
+	// Reset so the request can be reused
+	s.Reset()
+
+	return ret, nil
+}
+
+// BulkResponse is a response to a bulk execution.
+//
+// Example:
+// {
+//   "took":3,
+//   "errors":false,
+//   "items":[{
+//     "index":{
+//       "_index":"index1",
+//       "_type":"tweet",
+//       "_id":"1",
+//       "_version":3,
+//       "status":201
+//     }
+//   },{
+//     "index":{
+//       "_index":"index2",
+//       "_type":"tweet",
+//       "_id":"2",
+//       "_version":3,
+//       "status":200
+//     }
+//   },{
+//     "delete":{
+//       "_index":"index1",
+//       "_type":"tweet",
+//       "_id":"1",
+//       "_version":4,
+//       "status":200,
+//       "found":true
+//     }
+//   },{
+//     "update":{
+//       "_index":"index2",
+//       "_type":"tweet",
+//       "_id":"2",
+//       "_version":4,
+//       "status":200
+//     }
+//   }]
+// }
+type BulkResponse struct {
+	Took   int                            `json:"took,omitempty"`
+	Errors bool                           `json:"errors,omitempty"`
+	Items  []map[string]*BulkResponseItem `json:"items,omitempty"`
+}
+
+// BulkResponseItem is the result of a single bulk request.
+type BulkResponseItem struct {
+	Index         string        `json:"_index,omitempty"`
+	Type          string        `json:"_type,omitempty"`
+	Id            string        `json:"_id,omitempty"`
+	Version       int64         `json:"_version,omitempty"`
+	Result        string        `json:"result,omitempty"`
+	Shards        *shardsInfo   `json:"_shards,omitempty"`
+	SeqNo         int64         `json:"_seq_no,omitempty"`
+	PrimaryTerm   int64         `json:"_primary_term,omitempty"`
+	Status        int           `json:"status,omitempty"`
+	ForcedRefresh bool          `json:"forced_refresh,omitempty"`
+	Error         *ErrorDetails `json:"error,omitempty"`
+	GetResult     *GetResult    `json:"get,omitempty"`
+}
+
+// Indexed returns all bulk request results of "index" actions.
+func (r *BulkResponse) Indexed() []*BulkResponseItem {
+	return r.ByAction("index")
+}
+
+// Created returns all bulk request results of "create" actions.
+func (r *BulkResponse) Created() []*BulkResponseItem {
+	return r.ByAction("create")
+}
+
+// Updated returns all bulk request results of "update" actions.
+func (r *BulkResponse) Updated() []*BulkResponseItem {
+	return r.ByAction("update")
+}
+
+// Deleted returns all bulk request results of "delete" actions.
+func (r *BulkResponse) Deleted() []*BulkResponseItem {
+	return r.ByAction("delete")
+}
+
+// ByAction returns all bulk request results of a certain action,
+// e.g. "index" or "delete".
+func (r *BulkResponse) ByAction(action string) []*BulkResponseItem {
+	if r.Items == nil {
+		return nil
+	}
+	var items []*BulkResponseItem
+	for _, item := range r.Items {
+		if result, found := item[action]; found {
+			items = append(items, result)
+		}
+	}
+	return items
+}
+
+// ById returns all bulk request results of a given document id,
+// regardless of the action ("index", "delete" etc.).
+func (r *BulkResponse) ById(id string) []*BulkResponseItem {
+	if r.Items == nil {
+		return nil
+	}
+	var items []*BulkResponseItem
+	for _, item := range r.Items {
+		for _, result := range item {
+			if result.Id == id {
+				items = append(items, result)
+			}
+		}
+	}
+	return items
+}
+
+// Failed returns those items of a bulk response that have errors,
+// i.e. those that don't have a status code between 200 and 299.
+func (r *BulkResponse) Failed() []*BulkResponseItem {
+	if r.Items == nil {
+		return nil
+	}
+	var errors []*BulkResponseItem
+	for _, item := range r.Items {
+		for _, result := range item {
+			if !(result.Status >= 200 && result.Status <= 299) {
+				errors = append(errors, result)
+			}
+		}
+	}
+	return errors
+}
+
+// Succeeded returns those items of a bulk response that have no errors,
+// i.e. those have a status code between 200 and 299.
+func (r *BulkResponse) Succeeded() []*BulkResponseItem {
+	if r.Items == nil {
+		return nil
+	}
+	var succeeded []*BulkResponseItem
+	for _, item := range r.Items {
+		for _, result := range item {
+			if result.Status >= 200 && result.Status <= 299 {
+				succeeded = append(succeeded, result)
+			}
+		}
+	}
+	return succeeded
+}

vendor/gopkg.in/olivere/elastic.v6/bulk_delete_request.go

+// Copyright 2012-present Oliver Eilhard. All rights reserved.
+// Use of this source code is governed by a MIT-license.
+// See http://olivere.mit-license.org/license.txt for details.
+
+package elastic
+
+//go:generate easyjson bulk_delete_request.go
+
+import (
+	"encoding/json"
+	"fmt"
+	"strings"
+)
+
+// -- Bulk delete request --
+
+// BulkDeleteRequest is a request to remove a document from Elasticsearch.
+//
+// See https://www.elastic.co/guide/en/elasticsearch/reference/6.2/docs-bulk.html
+// for details.
+type BulkDeleteRequest struct {
+	BulkableRequest
+	index       string
+	typ         string
+	id          string
+	parent      string
+	routing     string
+	version     int64  // default is MATCH_ANY
+	versionType string // default is "internal"
+
+	source []string
+
+	useEasyJSON bool
+}
+
+//easyjson:json
+type bulkDeleteRequestCommand map[string]bulkDeleteRequestCommandOp
+
+//easyjson:json
+type bulkDeleteRequestCommandOp struct {
+	Index       string `json:"_index,omitempty"`
+	Type        string `json:"_type,omitempty"`
+	Id          string `json:"_id,omitempty"`
+	Parent      string `json:"parent,omitempty"`
+	Routing     string `json:"routing,omitempty"`
+	Version     int64  `json:"version,omitempty"`
+	VersionType string `json:"version_type,omitempty"`
+}
+
+// NewBulkDeleteRequest returns a new BulkDeleteRequest.
+func NewBulkDeleteRequest() *BulkDeleteRequest {
+	return &BulkDeleteRequest{}
+}
+
+// UseEasyJSON is an experimental setting that enables serialization
+// with github.com/mailru/easyjson, which should in faster serialization
+// time and less allocations, but removed compatibility with encoding/json,
+// usage of unsafe etc. See https://github.com/mailru/easyjson#issues-notes-and-limitations
+// for details. This setting is disabled by default.
+func (r *BulkDeleteRequest) UseEasyJSON(enable bool) *BulkDeleteRequest {
+	r.useEasyJSON = enable
+	return r
+}
+
+// Index specifies the Elasticsearch index to use for this delete request.
+// If unspecified, the index set on the BulkService will be used.
+func (r *BulkDeleteRequest) Index(index string) *BulkDeleteRequest {
+	r.index = index
+	r.source = nil
+	return r
+}
+
+// Type specifies the Elasticsearch type to use for this delete request.
+// If unspecified, the type set on the BulkService will be used.
+func (r *BulkDeleteRequest) Type(typ string) *BulkDeleteRequest {
+	r.typ = typ
+	r.source = nil
+	return r
+}
+
+// Id specifies the identifier of the document to delete.
+func (r *BulkDeleteRequest) Id(id string) *BulkDeleteRequest {
+	r.id = id
+	r.source = nil
+	return r
+}
+
+// Parent specifies the parent of the request, which is used in parent/child
+// mappings.
+func (r *BulkDeleteRequest) Parent(parent string) *BulkDeleteRequest {
+	r.parent = parent
+	r.source = nil
+	return r
+}
+
+// Routing specifies a routing value for the request.
+func (r *BulkDeleteRequest) Routing(routing string) *BulkDeleteRequest {
+	r.routing = routing
+	r.source = nil
+	return r
+}
+
+// Version indicates the version to be deleted as part of an optimistic
+// concurrency model.
+func (r *BulkDeleteRequest) Version(version int64) *BulkDeleteRequest {
+	r.version = version
+	r.source = nil
+	return r
+}
+
+// VersionType can be "internal" (default), "external", "external_gte",
+// or "external_gt".
+func (r *BulkDeleteRequest) VersionType(versionType string) *BulkDeleteRequest {
+	r.versionType = versionType
+	r.source = nil
+	return r
+}
+
+// String returns the on-wire representation of the delete request,
+// concatenated as a single string.
+func (r *BulkDeleteRequest) String() string {
+	lines, err := r.Source()
+	if err != nil {
+		return fmt.Sprintf("error: %v", err)
+	}
+	return strings.Join(lines, "\n")
+}
+
+// Source returns the on-wire representation of the delete request,
+// split into an action-and-meta-data line and an (optional) source line.
+// See https://www.elastic.co/guide/en/elasticsearch/reference/6.2/docs-bulk.html
+// for details.
+func (r *BulkDeleteRequest) Source() ([]string, error) {
+	if r.source != nil {
+		return r.source, nil
+	}
+	command := bulkDeleteRequestCommand{
+		"delete": bulkDeleteRequestCommandOp{
+			Index:       r.index,
+			Type:        r.typ,
+			Id:          r.id,
+			Routing:     r.routing,
+			Parent:      r.parent,
+			Version:     r.version,
+			VersionType: r.versionType,
+		},
+	}
+
+	var err error
+	var body []byte
+	if r.useEasyJSON {
+		// easyjson
+		body, err = command.MarshalJSON()
+	} else {
+		// encoding/json
+		body, err = json.Marshal(command)
+	}
+	if err != nil {
+		return nil, err
+	}
+
+	lines := []string{string(body)}
+	r.source = lines
+
+	return lines, nil
+}

vendor/gopkg.in/olivere/elastic.v6/bulk_delete_request_easyjson.go

+// Code generated by easyjson for marshaling/unmarshaling. DO NOT EDIT.
+
+package elastic
+
+import (
+	json "encoding/json"
+	easyjson "github.com/mailru/easyjson"
+	jlexer "github.com/mailru/easyjson/jlexer"
+	jwriter "github.com/mailru/easyjson/jwriter"
+)
+
+// suppress unused package warning
+var (
+	_ *json.RawMessage
+	_ *jlexer.Lexer
+	_ *jwriter.Writer
+	_ easyjson.Marshaler
+)
+
+func easyjson8092efb6DecodeGithubComOlivereElastic(in *jlexer.Lexer, out *bulkDeleteRequestCommandOp) {
+	isTopLevel := in.IsStart()
+	if in.IsNull() {
+		if isTopLevel {
+			in.Consumed()
+		}
+		in.Skip()
+		return
+	}
+	in.Delim('{')
+	for !in.IsDelim('}') {
+		key := in.UnsafeString()
+		in.WantColon()
+		if in.IsNull() {
+			in.Skip()
+			in.WantComma()
+			continue
+		}
+		switch key {
+		case "_index":
+			out.Index = string(in.String())
+		case "_type":
+			out.Type = string(in.String())
+		case "_id":
+			out.Id = string(in.String())
+		case "parent":
+			out.Parent = string(in.String())
+		case "routing":
+			out.Routing = string(in.String())
+		case "version":
+			out.Version = int64(in.Int64())
+		case "version_type":
+			out.VersionType = string(in.String())
+		default:
+			in.SkipRecursive()
+		}
+		in.WantComma()
+	}
+	in.Delim('}')
+	if isTopLevel {
+		in.Consumed()
+	}
+}
+func easyjson8092efb6EncodeGithubComOlivereElastic(out *jwriter.Writer, in bulkDeleteRequestCommandOp) {
+	out.RawByte('{')
+	first := true
+	_ = first
+	if in.Index != "" {
+		const prefix string = ",\"_index\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.String(string(in.Index))
+	}
+	if in.Type != "" {
+		const prefix string = ",\"_type\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.String(string(in.Type))
+	}
+	if in.Id != "" {
+		const prefix string = ",\"_id\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.String(string(in.Id))
+	}
+	if in.Parent != "" {
+		const prefix string = ",\"parent\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.String(string(in.Parent))
+	}
+	if in.Routing != "" {
+		const prefix string = ",\"routing\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.String(string(in.Routing))
+	}
+	if in.Version != 0 {
+		const prefix string = ",\"version\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.Int64(int64(in.Version))
+	}
+	if in.VersionType != "" {
+		const prefix string = ",\"version_type\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.String(string(in.VersionType))
+	}
+	out.RawByte('}')
+}
+
+// MarshalJSON supports json.Marshaler interface
+func (v bulkDeleteRequestCommandOp) MarshalJSON() ([]byte, error) {
+	w := jwriter.Writer{}
+	easyjson8092efb6EncodeGithubComOlivereElastic(&w, v)
+	return w.Buffer.BuildBytes(), w.Error
+}
+
+// MarshalEasyJSON supports easyjson.Marshaler interface
+func (v bulkDeleteRequestCommandOp) MarshalEasyJSON(w *jwriter.Writer) {
+	easyjson8092efb6EncodeGithubComOlivereElastic(w, v)
+}
+
+// UnmarshalJSON supports json.Unmarshaler interface
+func (v *bulkDeleteRequestCommandOp) UnmarshalJSON(data []byte) error {
+	r := jlexer.Lexer{Data: data}
+	easyjson8092efb6DecodeGithubComOlivereElastic(&r, v)
+	return r.Error()
+}
+
+// UnmarshalEasyJSON supports easyjson.Unmarshaler interface
+func (v *bulkDeleteRequestCommandOp) UnmarshalEasyJSON(l *jlexer.Lexer) {
+	easyjson8092efb6DecodeGithubComOlivereElastic(l, v)
+}
+func easyjson8092efb6DecodeGithubComOlivereElastic1(in *jlexer.Lexer, out *bulkDeleteRequestCommand) {
+	isTopLevel := in.IsStart()
+	if in.IsNull() {
+		in.Skip()
+	} else {
+		in.Delim('{')
+		if !in.IsDelim('}') {
+			*out = make(bulkDeleteRequestCommand)
+		} else {
+			*out = nil
+		}
+		for !in.IsDelim('}') {
+			key := string(in.String())
+			in.WantColon()
+			var v1 bulkDeleteRequestCommandOp
+			(v1).UnmarshalEasyJSON(in)
+			(*out)[key] = v1
+			in.WantComma()
+		}
+		in.Delim('}')
+	}
+	if isTopLevel {
+		in.Consumed()
+	}
+}
+func easyjson8092efb6EncodeGithubComOlivereElastic1(out *jwriter.Writer, in bulkDeleteRequestCommand) {
+	if in == nil && (out.Flags&jwriter.NilMapAsEmpty) == 0 {
+		out.RawString(`null`)
+	} else {
+		out.RawByte('{')
+		v2First := true
+		for v2Name, v2Value := range in {
+			if v2First {
+				v2First = false
+			} else {
+				out.RawByte(',')
+			}
+			out.String(string(v2Name))
+			out.RawByte(':')
+			(v2Value).MarshalEasyJSON(out)
+		}
+		out.RawByte('}')
+	}
+}
+
+// MarshalJSON supports json.Marshaler interface
+func (v bulkDeleteRequestCommand) MarshalJSON() ([]byte, error) {
+	w := jwriter.Writer{}
+	easyjson8092efb6EncodeGithubComOlivereElastic1(&w, v)
+	return w.Buffer.BuildBytes(), w.Error
+}
+
+// MarshalEasyJSON supports easyjson.Marshaler interface
+func (v bulkDeleteRequestCommand) MarshalEasyJSON(w *jwriter.Writer) {
+	easyjson8092efb6EncodeGithubComOlivereElastic1(w, v)
+}
+
+// UnmarshalJSON supports json.Unmarshaler interface
+func (v *bulkDeleteRequestCommand) UnmarshalJSON(data []byte) error {
+	r := jlexer.Lexer{Data: data}
+	easyjson8092efb6DecodeGithubComOlivereElastic1(&r, v)
+	return r.Error()
+}
+
+// UnmarshalEasyJSON supports easyjson.Unmarshaler interface
+func (v *bulkDeleteRequestCommand) UnmarshalEasyJSON(l *jlexer.Lexer) {
+	easyjson8092efb6DecodeGithubComOlivereElastic1(l, v)
+}

vendor/gopkg.in/olivere/elastic.v6/bulk_delete_request_test.go

+// Copyright 2012-present Oliver Eilhard. All rights reserved.
+// Use of this source code is governed by a MIT-license.
+// See http://olivere.mit-license.org/license.txt for details.
+
+package elastic
+
+import (
+	"testing"
+)
+
+func TestBulkDeleteRequestSerialization(t *testing.T) {
+	tests := []struct {
+		Request  BulkableRequest
+		Expected []string
+	}{
+		// #0
+		{
+			Request: NewBulkDeleteRequest().Index("index1").Type("doc").Id("1"),
+			Expected: []string{
+				`{"delete":{"_index":"index1","_type":"doc","_id":"1"}}`,
+			},
+		},
+		// #1
+		{
+			Request: NewBulkDeleteRequest().Index("index1").Type("doc").Id("1").Parent("2"),
+			Expected: []string{
+				`{"delete":{"_index":"index1","_type":"doc","_id":"1","parent":"2"}}`,
+			},
+		},
+		// #2
+		{
+			Request: NewBulkDeleteRequest().Index("index1").Type("doc").Id("1").Routing("3"),
+			Expected: []string{
+				`{"delete":{"_index":"index1","_type":"doc","_id":"1","routing":"3"}}`,
+			},
+		},
+	}
+
+	for i, test := range tests {
+		lines, err := test.Request.Source()
+		if err != nil {
+			t.Fatalf("case #%d: expected no error, got: %v", i, err)
+		}
+		if lines == nil {
+			t.Fatalf("case #%d: expected lines, got nil", i)
+		}
+		if len(lines) != len(test.Expected) {
+			t.Fatalf("case #%d: expected %d lines, got %d", i, len(test.Expected), len(lines))
+		}
+		for j, line := range lines {
+			if line != test.Expected[j] {
+				t.Errorf("case #%d: expected line #%d to be %s, got: %s", i, j, test.Expected[j], line)
+			}
+		}
+	}
+}
+
+var bulkDeleteRequestSerializationResult string
+
+func BenchmarkBulkDeleteRequestSerialization(b *testing.B) {
+	b.Run("stdlib", func(b *testing.B) {
+		r := NewBulkDeleteRequest().Index(testIndexName).Type("doc").Id("1")
+		benchmarkBulkDeleteRequestSerialization(b, r.UseEasyJSON(false))
+	})
+	b.Run("easyjson", func(b *testing.B) {
+		r := NewBulkDeleteRequest().Index(testIndexName).Type("doc").Id("1")
+		benchmarkBulkDeleteRequestSerialization(b, r.UseEasyJSON(true))
+	})
+}
+
+func benchmarkBulkDeleteRequestSerialization(b *testing.B, r *BulkDeleteRequest) {
+	var s string
+	for n := 0; n < b.N; n++ {
+		s = r.String()
+		r.source = nil // Don't let caching spoil the benchmark
+	}
+	bulkDeleteRequestSerializationResult = s // ensure the compiler doesn't optimize
+	b.ReportAllocs()
+}

vendor/gopkg.in/olivere/elastic.v6/bulk_index_request.go

+// Copyright 2012-present Oliver Eilhard. All rights reserved.
+// Use of this source code is governed by a MIT-license.
+// See http://olivere.mit-license.org/license.txt for details.
+
+package elastic
+
+//go:generate easyjson bulk_index_request.go
+
+import (
+	"encoding/json"
+	"fmt"
+	"strings"
+)
+
+// BulkIndexRequest is a request to add a document to Elasticsearch.
+//
+// See https://www.elastic.co/guide/en/elasticsearch/reference/6.2/docs-bulk.html
+// for details.
+type BulkIndexRequest struct {
+	BulkableRequest
+	index           string
+	typ             string
+	id              string
+	opType          string
+	routing         string
+	parent          string
+	version         int64  // default is MATCH_ANY
+	versionType     string // default is "internal"
+	doc             interface{}
+	pipeline        string
+	retryOnConflict *int
+
+	source []string
+
+	useEasyJSON bool
+}
+
+//easyjson:json
+type bulkIndexRequestCommand map[string]bulkIndexRequestCommandOp
+
+//easyjson:json
+type bulkIndexRequestCommandOp struct {
+	Index  string `json:"_index,omitempty"`
+	Id     string `json:"_id,omitempty"`
+	Type   string `json:"_type,omitempty"`
+	Parent string `json:"parent,omitempty"`
+	// RetryOnConflict is "_retry_on_conflict" for 6.0 and "retry_on_conflict" for 6.1+.
+	RetryOnConflict *int   `json:"retry_on_conflict,omitempty"`
+	Routing         string `json:"routing,omitempty"`
+	Version         int64  `json:"version,omitempty"`
+	VersionType     string `json:"version_type,omitempty"`
+	Pipeline        string `json:"pipeline,omitempty"`
+}
+
+// NewBulkIndexRequest returns a new BulkIndexRequest.
+// The operation type is "index" by default.
+func NewBulkIndexRequest() *BulkIndexRequest {
+	return &BulkIndexRequest{
+		opType: "index",
+	}
+}
+
+// UseEasyJSON is an experimental setting that enables serialization
+// with github.com/mailru/easyjson, which should in faster serialization
+// time and less allocations, but removed compatibility with encoding/json,
+// usage of unsafe etc. See https://github.com/mailru/easyjson#issues-notes-and-limitations
+// for details. This setting is disabled by default.
+func (r *BulkIndexRequest) UseEasyJSON(enable bool) *BulkIndexRequest {
+	r.useEasyJSON = enable
+	return r
+}
+
+// Index specifies the Elasticsearch index to use for this index request.
+// If unspecified, the index set on the BulkService will be used.
+func (r *BulkIndexRequest) Index(index string) *BulkIndexRequest {
+	r.index = index
+	r.source = nil
+	return r
+}
+
+// Type specifies the Elasticsearch type to use for this index request.
+// If unspecified, the type set on the BulkService will be used.
+func (r *BulkIndexRequest) Type(typ string) *BulkIndexRequest {
+	r.typ = typ
+	r.source = nil
+	return r
+}
+
+// Id specifies the identifier of the document to index.
+func (r *BulkIndexRequest) Id(id string) *BulkIndexRequest {
+	r.id = id
+	r.source = nil
+	return r
+}
+
+// OpType specifies if this request should follow create-only or upsert
+// behavior. This follows the OpType of the standard document index API.
+// See https://www.elastic.co/guide/en/elasticsearch/reference/6.2/docs-index_.html#operation-type
+// for details.
+func (r *BulkIndexRequest) OpType(opType string) *BulkIndexRequest {
+	r.opType = opType
+	r.source = nil
+	return r
+}
+
+// Routing specifies a routing value for the request.
+func (r *BulkIndexRequest) Routing(routing string) *BulkIndexRequest {
+	r.routing = routing
+	r.source = nil
+	return r
+}
+
+// Parent specifies the identifier of the parent document (if available).
+func (r *BulkIndexRequest) Parent(parent string) *BulkIndexRequest {
+	r.parent = parent
+	r.source = nil
+	return r
+}
+
+// Version indicates the version of the document as part of an optimistic
+// concurrency model.
+func (r *BulkIndexRequest) Version(version int64) *BulkIndexRequest {
+	r.version = version
+	r.source = nil
+	return r
+}
+
+// VersionType specifies how versions are created. It can be e.g. internal,
+// external, external_gte, or force.
+//
+// See https://www.elastic.co/guide/en/elasticsearch/reference/6.2/docs-index_.html#index-versioning
+// for details.
+func (r *BulkIndexRequest) VersionType(versionType string) *BulkIndexRequest {
+	r.versionType = versionType
+	r.source = nil
+	return r
+}
+
+// Doc specifies the document to index.
+func (r *BulkIndexRequest) Doc(doc interface{}) *BulkIndexRequest {
+	r.doc = doc
+	r.source = nil
+	return r
+}
+
+// RetryOnConflict specifies how often to retry in case of a version conflict.
+func (r *BulkIndexRequest) RetryOnConflict(retryOnConflict int) *BulkIndexRequest {
+	r.retryOnConflict = &retryOnConflict
+	r.source = nil
+	return r
+}
+
+// Pipeline to use while processing the request.
+func (r *BulkIndexRequest) Pipeline(pipeline string) *BulkIndexRequest {
+	r.pipeline = pipeline
+	r.source = nil
+	return r
+}
+
+// String returns the on-wire representation of the index request,
+// concatenated as a single string.
+func (r *BulkIndexRequest) String() string {
+	lines, err := r.Source()
+	if err != nil {
+		return fmt.Sprintf("error: %v", err)
+	}
+	return strings.Join(lines, "\n")
+}
+
+// Source returns the on-wire representation of the index request,
+// split into an action-and-meta-data line and an (optional) source line.
+// See https://www.elastic.co/guide/en/elasticsearch/reference/6.2/docs-bulk.html
+// for details.
+func (r *BulkIndexRequest) Source() ([]string, error) {
+	// { "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } }
+	// { "field1" : "value1" }
+
+	if r.source != nil {
+		return r.source, nil
+	}
+
+	lines := make([]string, 2)
+
+	// "index" ...
+	indexCommand := bulkIndexRequestCommandOp{
+		Index:           r.index,
+		Type:            r.typ,
+		Id:              r.id,
+		Routing:         r.routing,
+		Parent:          r.parent,
+		Version:         r.version,
+		VersionType:     r.versionType,
+		RetryOnConflict: r.retryOnConflict,
+		Pipeline:        r.pipeline,
+	}
+	command := bulkIndexRequestCommand{
+		r.opType: indexCommand,
+	}
+
+	var err error
+	var body []byte
+	if r.useEasyJSON {
+		// easyjson
+		body, err = command.MarshalJSON()
+	} else {
+		// encoding/json
+		body, err = json.Marshal(command)
+	}
+	if err != nil {
+		return nil, err
+	}
+
+	lines[0] = string(body)
+
+	// "field1" ...
+	if r.doc != nil {
+		switch t := r.doc.(type) {
+		default:
+			body, err := json.Marshal(r.doc)
+			if err != nil {
+				return nil, err
+			}
+			lines[1] = string(body)
+		case json.RawMessage:
+			lines[1] = string(t)
+		case *json.RawMessage:
+			lines[1] = string(*t)
+		case string:
+			lines[1] = t
+		case *string:
+			lines[1] = *t
+		}
+	} else {
+		lines[1] = "{}"
+	}
+
+	r.source = lines
+	return lines, nil
+}

vendor/gopkg.in/olivere/elastic.v6/bulk_index_request_easyjson.go

+// Code generated by easyjson for marshaling/unmarshaling. DO NOT EDIT.
+
+package elastic
+
+import (
+	json "encoding/json"
+	easyjson "github.com/mailru/easyjson"
+	jlexer "github.com/mailru/easyjson/jlexer"
+	jwriter "github.com/mailru/easyjson/jwriter"
+)
+
+// suppress unused package warning
+var (
+	_ *json.RawMessage
+	_ *jlexer.Lexer
+	_ *jwriter.Writer
+	_ easyjson.Marshaler
+)
+
+func easyjson9de0fcbfDecodeGithubComOlivereElastic(in *jlexer.Lexer, out *bulkIndexRequestCommandOp) {
+	isTopLevel := in.IsStart()
+	if in.IsNull() {
+		if isTopLevel {
+			in.Consumed()
+		}
+		in.Skip()
+		return
+	}
+	in.Delim('{')
+	for !in.IsDelim('}') {
+		key := in.UnsafeString()
+		in.WantColon()
+		if in.IsNull() {
+			in.Skip()
+			in.WantComma()
+			continue
+		}
+		switch key {
+		case "_index":
+			out.Index = string(in.String())
+		case "_id":
+			out.Id = string(in.String())
+		case "_type":
+			out.Type = string(in.String())
+		case "parent":
+			out.Parent = string(in.String())
+		case "retry_on_conflict":
+			if in.IsNull() {
+				in.Skip()
+				out.RetryOnConflict = nil
+			} else {
+				if out.RetryOnConflict == nil {
+					out.RetryOnConflict = new(int)
+				}
+				*out.RetryOnConflict = int(in.Int())
+			}
+		case "routing":
+			out.Routing = string(in.String())
+		case "version":
+			out.Version = int64(in.Int64())
+		case "version_type":
+			out.VersionType = string(in.String())
+		case "pipeline":
+			out.Pipeline = string(in.String())
+		default:
+			in.SkipRecursive()
+		}
+		in.WantComma()
+	}
+	in.Delim('}')
+	if isTopLevel {
+		in.Consumed()
+	}
+}
+func easyjson9de0fcbfEncodeGithubComOlivereElastic(out *jwriter.Writer, in bulkIndexRequestCommandOp) {
+	out.RawByte('{')
+	first := true
+	_ = first
+	if in.Index != "" {
+		const prefix string = ",\"_index\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.String(string(in.Index))
+	}
+	if in.Id != "" {
+		const prefix string = ",\"_id\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.String(string(in.Id))
+	}
+	if in.Type != "" {
+		const prefix string = ",\"_type\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.String(string(in.Type))
+	}
+	if in.Parent != "" {
+		const prefix string = ",\"parent\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.String(string(in.Parent))
+	}
+	if in.RetryOnConflict != nil {
+		const prefix string = ",\"retry_on_conflict\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.Int(int(*in.RetryOnConflict))
+	}
+	if in.Routing != "" {
+		const prefix string = ",\"routing\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.String(string(in.Routing))
+	}
+	if in.Version != 0 {
+		const prefix string = ",\"version\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.Int64(int64(in.Version))
+	}
+	if in.VersionType != "" {
+		const prefix string = ",\"version_type\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.String(string(in.VersionType))
+	}
+	if in.Pipeline != "" {
+		const prefix string = ",\"pipeline\":"
+		if first {
+			first = false
+			out.RawString(prefix[1:])
+		} else {
+			out.RawString(prefix)
+		}
+		out.String(string(in.Pipeline))
+	}
+	out.RawByte('}')
+}
+
+// MarshalJSON supports json.Marshaler interface
+func (v bulkIndexRequestCommandOp) MarshalJSON() ([]byte, error) {
+	w := jwriter.Writer{}
+	easyjson9de0fcbfEncodeGithubComOlivereElastic(&w, v)
+	return w.Buffer.BuildBytes(), w.Error
+}
+
+// MarshalEasyJSON supports easyjson.Marshaler interface
+func (v bulkIndexRequestCommandOp) MarshalEasyJSON(w *jwriter.Writer) {
+	easyjson9de0fcbfEncodeGithubComOlivereElastic(w, v)
+}
+
+// UnmarshalJSON supports json.Unmarshaler interface
+func (v *bulkIndexRequestCommandOp) UnmarshalJSON(data []byte) error {
+	r := jlexer.Lexer{Data: data}
+	easyjson9de0fcbfDecodeGithubComOlivereElastic(&r, v)
+	return r.Error()
+}
+
+// UnmarshalEasyJSON supports easyjson.Unmarshaler interface
+func (v *bulkIndexRequestCommandOp) UnmarshalEasyJSON(l *jlexer.Lexer) {
+	easyjson9de0fcbfDecodeGithubComOlivereElastic(l, v)
+}
+func easyjson9de0fcbfDecodeGithubComOlivereElastic1(in *jlexer.Lexer, out *bulkIndexRequestCommand) {
+	isTopLevel := in.IsStart()
+	if in.IsNull() {
+		in.Skip()
+	} else {
+		in.Delim('{')
+		if !in.IsDelim('}') {
+			*out = make(bulkIndexRequestCommand)
+		} else {
+			*out = nil
+		}
+		for !in.IsDelim('}') {
+			key := string(in.String())
+			in.WantColon()
+			var v1 bulkIndexRequestCommandOp
+			(v1).UnmarshalEasyJSON(in)
+			(*out)[key] = v1
+			in.WantComma()
+		}
+		in.Delim('}')
+	}
+	if isTopLevel {
+		in.Consumed()
+	}
+}
+func easyjson9de0fcbfEncodeGithubComOlivereElastic1(out *jwriter.Writer, in bulkIndexRequestCommand) {
+	if in == nil && (out.Flags&jwriter.NilMapAsEmpty) == 0 {
+		out.RawString(`null`)
+	} else {
+		out.RawByte('{')
+		v2First := true
+		for v2Name, v2Value := range in {
+			if v2First {
+				v2First = false
+			} else {
+				out.RawByte(',')
+			}
+			out.String(string(v2Name))
+			out.RawByte(':')
+			(v2Value).MarshalEasyJSON(out)
+		}
+		out.RawByte('}')
+	}
+}
+
+// MarshalJSON supports json.Marshaler interface
+func (v bulkIndexRequestCommand) MarshalJSON() ([]byte, error) {
+	w := jwriter.Writer{}
+	easyjson9de0fcbfEncodeGithubComOlivereElastic1(&w, v)
+	return w.Buffer.BuildBytes(), w.Error
+}
+
+// MarshalEasyJSON supports easyjson.Marshaler interface
+func (v bulkIndexRequestCommand) MarshalEasyJSON(w *jwriter.Writer) {
+	easyjson9de0fcbfEncodeGithubComOlivereElastic1(w, v)
+}
+
+// UnmarshalJSON supports json.Unmarshaler interface
+func (v *bulkIndexRequestCommand) UnmarshalJSON(data []byte) error {
+	r := jlexer.Lexer{Data: data}
+	easyjson9de0fcbfDecodeGithubComOlivereElastic1(&r, v)
+	return r.Error()
+}
+
+// UnmarshalEasyJSON supports easyjson.Unmarshaler interface
+func (v *bulkIndexRequestCommand) UnmarshalEasyJSON(l *jlexer.Lexer) {
+	easyjson9de0fcbfDecodeGithubComOlivereElastic1(l, v)
+}

vendor/gopkg.in/olivere/elastic.v6/bulk_index_request_test.go

+// Copyright 2012-present Oliver Eilhard. All rights reserved.
+// Use of this source code is governed by a MIT-license.
+// See http://olivere.mit-license.org/license.txt for details.
+
+package elastic
+
+import (
+	"testing"
+	"time"
+)
+
+func TestBulkIndexRequestSerialization(t *testing.T) {
+	tests := []struct {
+		Request  BulkableRequest
+		Expected []string
+	}{
+		// #0
+		{
+			Request: NewBulkIndexRequest().Index("index1").Type("doc").Id("1").
+				Doc(tweet{User: "olivere", Created: time.Date(2014, 1, 18, 23, 59, 58, 0, time.UTC)}),
+			Expected: []string{
+				`{"index":{"_index":"index1","_id":"1","_type":"doc"}}`,
+				`{"user":"olivere","message":"","retweets":0,"created":"2014-01-18T23:59:58Z"}`,
+			},
+		},
+		// #1
+		{
+			Request: NewBulkIndexRequest().OpType("create").Index("index1").Type("doc").Id("1").
+				Doc(tweet{User: "olivere", Created: time.Date(2014, 1, 18, 23, 59, 58, 0, time.UTC)}),
+			Expected: []string{
+				`{"create":{"_index":"index1","_id":"1","_type":"doc"}}`,
+				`{"user":"olivere","message":"","retweets":0,"created":"2014-01-18T23:59:58Z"}`,
+			},
+		},
+		// #2
+		{
+			Request: NewBulkIndexRequest().OpType("index").Index("index1").Type("doc").Id("1").
+				Doc(tweet{User: "olivere", Created: time.Date(2014, 1, 18, 23, 59, 58, 0, time.UTC)}),
+			Expected: []string{
+				`{"index":{"_index":"index1","_id":"1","_type":"doc"}}`,
+				`{"user":"olivere","message":"","retweets":0,"created":"2014-01-18T23:59:58Z"}`,
+			},
+		},
+		// #3
+		{
+			Request: NewBulkIndexRequest().OpType("index").Index("index1").Type("doc").Id("1").RetryOnConflict(42).
+				Doc(tweet{User: "olivere", Created: time.Date(2014, 1, 18, 23, 59, 58, 0, time.UTC)}),
+			Expected: []string{
+				`{"index":{"_index":"index1","_id":"1","_type":"doc","retry_on_conflict":42}}`,
+				`{"user":"olivere","message":"","retweets":0,"created":"2014-01-18T23:59:58Z"}`,
+			},
+		},
+		// #4
+		{
+			Request: NewBulkIndexRequest().OpType("index").Index("index1").Type("doc").Id("1").Pipeline("my_pipeline").
+				Doc(tweet{User: "olivere", Created: time.Date(2014, 1, 18, 23, 59, 58, 0, time.UTC)}),
+			Expected: []string{
+				`{"index":{"_index":"index1","_id":"1","_type":"doc","pipeline":"my_pipeline"}}`,
+				`{"user":"olivere","message":"","retweets":0,"created":"2014-01-18T23:59:58Z"}`,
+			},
+		},
+		// #5
+		{
+			Request: NewBulkIndexRequest().OpType("index").Index("index1").Type("doc").Id("1").
+				Routing("123").
+				Doc(tweet{User: "olivere", Created: time.Date(2014, 1, 18, 23, 59, 58, 0, time.UTC)}),
+			Expected: []string{
+				`{"index":{"_index":"index1","_id":"1","_type":"doc","routing":"123"}}`,
+				`{"user":"olivere","message":"","retweets":0,"created":"2014-01-18T23:59:58Z"}`,
+			},
+		},
+	}
+
+	for i, test := range tests {
+		lines, err := test.Request.Source()
+		if err != nil {
+			t.Fatalf("case #%d: expected no error, got: %v", i, err)
+		}
+		if lines == nil {
+			t.Fatalf("case #%d: expected lines, got nil", i)
+		}
+		if len(lines) != len(test.Expected) {
+			t.Fatalf("case #%d: expected %d lines, got %d", i, len(test.Expected), len(lines))
+		}
+		for j, line := range lines {
+			if line != test.Expected[j] {
+				t.Errorf("case #%d: expected line #%d to be %s, got: %s", i, j, test.Expected[j], line)
+			}
+		}
+	}
+}
+
+var bulkIndexRequestSerializationResult string
+
+func BenchmarkBulkIndexRequestSerialization(b *testing.B) {
+	b.Run("stdlib", func(b *testing.B) {
+		r := NewBulkIndexRequest().Index(testIndexName).Type("doc").Id("1").
+			Doc(tweet{User: "olivere", Created: time.Date(2014, 1, 18, 23, 59, 58, 0, time.UTC)})
+		benchmarkBulkIndexRequestSerialization(b, r.UseEasyJSON(false))
+	})
+	b.Run("easyjson", func(b *testing.B) {
+		r := NewBulkIndexRequest().Index(testIndexName).Type("doc").Id("1").
+			Doc(tweet{User: "olivere", Created: time.Date(2014, 1, 18, 23, 59, 58, 0, time.UTC)})
+		benchmarkBulkIndexRequestSerialization(b, r.UseEasyJSON(true))
+	})
+}
+
+func benchmarkBulkIndexRequestSerialization(b *testing.B, r *BulkIndexRequest) {
+	var s string
+	for n := 0; n < b.N; n++ {
+		s = r.String()
+		r.source = nil // Don't let caching spoil the benchmark
+	}
+	bulkIndexRequestSerializationResult = s // ensure the compiler doesn't optimize
+	b.ReportAllocs()
+}

vendor/gopkg.in/olivere/elastic.v6/bulk_processor.go

+// Copyright 2012-present Oliver Eilhard. All rights reserved.
+// Use of this source code is governed by a MIT-license.
+// See http://olivere.mit-license.org/license.txt for details.
+
+package elastic
+
+import (
+	"context"
+	"errors"
+	"net"
+	"sync"
+	"sync/atomic"
+	"time"
+)
+
+var (
+	// ErrBulkItemRetry is returned in BulkProcessor from a worker when
+	// a response item needs to be retried.
+	ErrBulkItemRetry = errors.New("elastic: uncommitted bulk response items")
+
+	defaultRetryItemStatusCodes = []int{408, 429, 503, 507}
+)
+
+// BulkProcessorService allows to easily process bulk requests. It allows setting
+// policies when to flush new bulk requests, e.g. based on a number of actions,
+// on the size of the actions, and/or to flush periodically. It also allows
+// to control the number of concurrent bulk requests allowed to be executed
+// in parallel.
+//
+// BulkProcessorService, by default, commits either every 1000 requests or when the
+// (estimated) size of the bulk requests exceeds 5 MB. However, it does not
+// commit periodically. BulkProcessorService also does retry by default, using
+// an exponential backoff algorithm. It also will automatically re-enqueue items
+// returned with a status of 408, 429, 503 or 507. You can change this
+// behavior with RetryItemStatusCodes.
+//
+// The caller is responsible for setting the index and type on every
+// bulk request added to BulkProcessorService.
+//
+// BulkProcessorService takes ideas from the BulkProcessor of the
+// Elasticsearch Java API as documented in
+// https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-docs-bulk-processor.html.
+type BulkProcessorService struct {
+	c                    *Client
+	beforeFn             BulkBeforeFunc
+	afterFn              BulkAfterFunc
+	name                 string        // name of processor
+	numWorkers           int           // # of workers (>= 1)
+	bulkActions          int           // # of requests after which to commit
+	bulkSize             int           // # of bytes after which to commit
+	flushInterval        time.Duration // periodic flush interval
+	wantStats            bool          // indicates whether to gather statistics
+	backoff              Backoff       // a custom Backoff to use for errors
+	retryItemStatusCodes []int         // array of status codes for bulk response line items that may be retried
+}
+
+// NewBulkProcessorService creates a new BulkProcessorService.
+func NewBulkProcessorService(client *Client) *BulkProcessorService {
+	return &BulkProcessorService{
+		c:           client,
+		numWorkers:  1,
+		bulkActions: 1000,
+		bulkSize:    5 << 20, // 5 MB
+		backoff: NewExponentialBackoff(
+			time.Duration(200)*time.Millisecond,
+			time.Duration(10000)*time.Millisecond,
+		),
+		retryItemStatusCodes: defaultRetryItemStatusCodes,
+	}
+}
+
+// BulkBeforeFunc defines the signature of callbacks that are executed
+// before a commit to Elasticsearch.
+type BulkBeforeFunc func(executionId int64, requests []BulkableRequest)
+
+// BulkAfterFunc defines the signature of callbacks that are executed
+// after a commit to Elasticsearch. The err parameter signals an error.
+type BulkAfterFunc func(executionId int64, requests []BulkableRequest, response *BulkResponse, err error)
+
+// Before specifies a function to be executed before bulk requests get comitted
+// to Elasticsearch.
+func (s *BulkProcessorService) Before(fn BulkBeforeFunc) *BulkProcessorService {
+	s.beforeFn = fn
+	return s
+}
+
+// After specifies a function to be executed when bulk requests have been
+// comitted to Elasticsearch. The After callback executes both when the
+// commit was successful as well as on failures.
+func (s *BulkProcessorService) After(fn BulkAfterFunc) *BulkProcessorService {
+	s.afterFn = fn
+	return s
+}
+
+// Name is an optional name to identify this bulk processor.
+func (s *BulkProcessorService) Name(name string) *BulkProcessorService {
+	s.name = name
+	return s
+}
+
+// Workers is the number of concurrent workers allowed to be
+// executed. Defaults to 1 and must be greater or equal to 1.
+func (s *BulkProcessorService) Workers(num int) *BulkProcessorService {
+	s.numWorkers = num
+	return s
+}
+
+// BulkActions specifies when to flush based on the number of actions
+// currently added. Defaults to 1000 and can be set to -1 to be disabled.
+func (s *BulkProcessorService) BulkActions(bulkActions int) *BulkProcessorService {
+	s.bulkActions = bulkActions
+	return s
+}
+
+// BulkSize specifies when to flush based on the size (in bytes) of the actions
+// currently added. Defaults to 5 MB and can be set to -1 to be disabled.
+func (s *BulkProcessorService) BulkSize(bulkSize int) *BulkProcessorService {
+	s.bulkSize = bulkSize
+	return s
+}
+
+// FlushInterval specifies when to flush at the end of the given interval.
+// This is disabled by default. If you want the bulk processor to
+// operate completely asynchronously, set both BulkActions and BulkSize to
+// -1 and set the FlushInterval to a meaningful interval.
+func (s *BulkProcessorService) FlushInterval(interval time.Duration) *BulkProcessorService {
+	s.flushInterval = interval
+	return s
+}
+
+// Stats tells bulk processor to gather stats while running.
+// Use Stats to return the stats. This is disabled by default.
+func (s *BulkProcessorService) Stats(wantStats bool) *BulkProcessorService {
+	s.wantStats = wantStats
+	return s
+}
+
+// Backoff sets the backoff strategy to use for errors.
+func (s *BulkProcessorService) Backoff(backoff Backoff) *BulkProcessorService {
+	s.backoff = backoff
+	return s
+}
+
+// RetryItemStatusCodes sets an array of status codes that indicate that a bulk
+// response line item should be retried.
+func (s *BulkProcessorService) RetryItemStatusCodes(retryItemStatusCodes ...int) *BulkProcessorService {
+	s.retryItemStatusCodes = retryItemStatusCodes
+	return s
+}
+
+// Do creates a new BulkProcessor and starts it.
+// Consider the BulkProcessor as a running instance that accepts bulk requests
+// and commits them to Elasticsearch, spreading the work across one or more
+// workers.
+//
+// You can interoperate with the BulkProcessor returned by Do, e.g. Start and
+// Stop (or Close) it.
+//
+// Context is an optional context that is passed into the bulk request
+// service calls. In contrast to other operations, this context is used in
+// a long running process. You could use it to pass e.g. loggers, but you
+// shouldn't use it for cancellation.
+//
+// Calling Do several times returns new BulkProcessors. You probably don't
+// want to do this. BulkProcessorService implements just a builder pattern.
+func (s *BulkProcessorService) Do(ctx context.Context) (*BulkProcessor, error) {
+
+	retryItemStatusCodes := make(map[int]struct{})
+	for _, code := range s.retryItemStatusCodes {
+		retryItemStatusCodes[code] = struct{}{}
+	}
+
+	p := newBulkProcessor(
+		s.c,
+		s.beforeFn,
+		s.afterFn,
+		s.name,
+		s.numWorkers,
+		s.bulkActions,
+		s.bulkSize,
+		s.flushInterval,
+		s.wantStats,
+		s.backoff,
+		retryItemStatusCodes)
+
+	err := p.Start(ctx)
+	if err != nil {
+		return nil, err
+	}
+	return p, nil
+}
+
+// -- Bulk Processor Statistics --
+
+// BulkProcessorStats contains various statistics of a bulk processor
+// while it is running. Use the Stats func to return it while running.
+type BulkProcessorStats struct {
+	Flushed   int64 // number of times the flush interval has been invoked
+	Committed int64 // # of times workers committed bulk requests
+	Indexed   int64 // # of requests indexed
+	Created   int64 // # of requests that ES reported as creates (201)
+	Updated   int64 // # of requests that ES reported as updates
+	Deleted   int64 // # of requests that ES reported as deletes
+	Succeeded int64 // # of requests that ES reported as successful
+	Failed    int64 // # of requests that ES reported as failed
+
+	Workers []*BulkProcessorWorkerStats // stats for each worker
+}
+
+// BulkProcessorWorkerStats represents per-worker statistics.
+type BulkProcessorWorkerStats struct {
+	Queued       int64         // # of requests queued in this worker
+	LastDuration time.Duration // duration of last commit
+}
+
+// newBulkProcessorStats initializes and returns a BulkProcessorStats struct.
+func newBulkProcessorStats(workers int) *BulkProcessorStats {
+	stats := &BulkProcessorStats{
+		Workers: make([]*BulkProcessorWorkerStats, workers),
+	}
+	for i := 0; i < workers; i++ {
+		stats.Workers[i] = &BulkProcessorWorkerStats{}
+	}
+	return stats
+}
+
+func (st *BulkProcessorStats) dup() *BulkProcessorStats {
+	dst := new(BulkProcessorStats)
+	dst.Flushed = st.Flushed
+	dst.Committed = st.Committed
+	dst.Indexed = st.Indexed
+	dst.Created = st.Created
+	dst.Updated = st.Updated
+	dst.Deleted = st.Deleted
+	dst.Succeeded = st.Succeeded
+	dst.Failed = st.Failed
+	for _, src := range st.Workers {
+		dst.Workers = append(dst.Workers, src.dup())
+	}
+	return dst
+}
+
+func (st *BulkProcessorWorkerStats) dup() *BulkProcessorWorkerStats {
+	dst := new(BulkProcessorWorkerStats)
+	dst.Queued = st.Queued
+	dst.LastDuration = st.LastDuration
+	return dst
+}
+
+// -- Bulk Processor --
+
+// BulkProcessor encapsulates a task that accepts bulk requests and
+// orchestrates committing them to Elasticsearch via one or more workers.
+//
+// BulkProcessor is returned by setting up a BulkProcessorService and
+// calling the Do method.
+type BulkProcessor struct {
+	c                    *Client
+	beforeFn             BulkBeforeFunc
+	afterFn              BulkAfterFunc
+	name                 string
+	bulkActions          int
+	bulkSize             int
+	numWorkers           int
+	executionId          int64
+	requestsC            chan BulkableRequest
+	workerWg             sync.WaitGroup
+	workers              []*bulkWorker
+	flushInterval        time.Duration
+	flusherStopC         chan struct{}
+	wantStats            bool
+	retryItemStatusCodes map[int]struct{}
+	backoff              Backoff
+
+	startedMu sync.Mutex // guards the following block
+	started   bool
+
+	statsMu sync.Mutex // guards the following block
+	stats   *BulkProcessorStats
+
+	stopReconnC chan struct{} // channel to signal stop reconnection attempts
+}
+
+func newBulkProcessor(
+	client *Client,
+	beforeFn BulkBeforeFunc,
+	afterFn BulkAfterFunc,
+	name string,
+	numWorkers int,
+	bulkActions int,
+	bulkSize int,
+	flushInterval time.Duration,
+	wantStats bool,
+	backoff Backoff,
+	retryItemStatusCodes map[int]struct{}) *BulkProcessor {
+	return &BulkProcessor{
+		c:                    client,
+		beforeFn:             beforeFn,
+		afterFn:              afterFn,
+		name:                 name,
+		numWorkers:           numWorkers,
+		bulkActions:          bulkActions,
+		bulkSize:             bulkSize,
+		flushInterval:        flushInterval,
+		wantStats:            wantStats,
+		retryItemStatusCodes: retryItemStatusCodes,
+		backoff:              backoff,
+	}
+}
+
+// Start starts the bulk processor. If the processor is already started,
+// nil is returned.
+func (p *BulkProcessor) Start(ctx context.Context) error {
+	p.startedMu.Lock()
+	defer p.startedMu.Unlock()
+
+	if p.started {
+		return nil
+	}
+
+	// We must have at least one worker.
+	if p.numWorkers < 1 {
+		p.numWorkers = 1
+	}
+
+	p.requestsC = make(chan BulkableRequest)
+	p.executionId = 0
+	p.stats = newBulkProcessorStats(p.numWorkers)
+	p.stopReconnC = make(chan struct{})
+
+	// Create and start up workers.
+	p.workers = make([]*bulkWorker, p.numWorkers)
+	for i := 0; i < p.numWorkers; i++ {
+		p.workerWg.Add(1)
+		p.workers[i] = newBulkWorker(p, i)
+		go p.workers[i].work(ctx)
+	}
+
+	// Start the ticker for flush (if enabled)
+	if int64(p.flushInterval) > 0 {
+		p.flusherStopC = make(chan struct{})
+		go p.flusher(p.flushInterval)
+	}
+
+	p.started = true
+
+	return nil
+}
+
+// Stop is an alias for Close.
+func (p *BulkProcessor) Stop() error {
+	return p.Close()
+}
+
+// Close stops the bulk processor previously started with Do.
+// If it is already stopped, this is a no-op and nil is returned.
+//
+// By implementing Close, BulkProcessor implements the io.Closer interface.
+func (p *BulkProcessor) Close() error {
+	p.startedMu.Lock()
+	defer p.startedMu.Unlock()
+
+	// Already stopped? Do nothing.
+	if !p.started {
+		return nil
+	}
+
+	// Tell connection checkers to stop
+	if p.stopReconnC != nil {
+		close(p.stopReconnC)
+		p.stopReconnC = nil
+	}
+
+	// Stop flusher (if enabled)
+	if p.flusherStopC != nil {
+		p.flusherStopC <- struct{}{}
+		<-p.flusherStopC
+		close(p.flusherStopC)
+		p.flusherStopC = nil
+	}
+
+	// Stop all workers.
+	close(p.requestsC)
+	p.workerWg.Wait()
+
+	p.started = false
+
+	return nil
+}
+
+// Stats returns the latest bulk processor statistics.
+// Collecting stats must be enabled first by calling Stats(true) on
+// the service that created this processor.
+func (p *BulkProcessor) Stats() BulkProcessorStats {
+	p.statsMu.Lock()
+	defer p.statsMu.Unlock()
+	return *p.stats.dup()
+}
+
+// Add adds a single request to commit by the BulkProcessorService.
+//
+// The caller is responsible for setting the index and type on the request.
+func (p *BulkProcessor) Add(request BulkableRequest) {
+	p.requestsC <- request
+}
+
+// Flush manually asks all workers to commit their outstanding requests.
+// It returns only when all workers acknowledge completion.
+func (p *BulkProcessor) Flush() error {
+	p.statsMu.Lock()
+	p.stats.Flushed++
+	p.statsMu.Unlock()
+
+	for _, w := range p.workers {
+		w.flushC <- struct{}{}
+		<-w.flushAckC // wait for completion
+	}
+	return nil
+}
+
+// flusher is a single goroutine that periodically asks all workers to
+// commit their outstanding bulk requests. It is only started if
+// FlushInterval is greater than 0.
+func (p *BulkProcessor) flusher(interval time.Duration) {
+	ticker := time.NewTicker(interval)
+	defer ticker.Stop()
+
+	for {
+		select {
+		case <-ticker.C: // Periodic flush
+			p.Flush() // TODO swallow errors here?
+
+		case <-p.flusherStopC:
+			p.flusherStopC <- struct{}{}
+			return
+		}
+	}
+}
+
+// -- Bulk Worker --
+
+// bulkWorker encapsulates a single worker, running in a goroutine,
+// receiving bulk requests and eventually committing them to Elasticsearch.
+// It is strongly bound to a BulkProcessor.
+type bulkWorker struct {
+	p           *BulkProcessor
+	i           int
+	bulkActions int
+	bulkSize    int
+	service     *BulkService
+	flushC      chan struct{}
+	flushAckC   chan struct{}
+}
+
+// newBulkWorker creates a new bulkWorker instance.
+func newBulkWorker(p *BulkProcessor, i int) *bulkWorker {
+	return &bulkWorker{
+		p:           p,
+		i:           i,
+		bulkActions: p.bulkActions,
+		bulkSize:    p.bulkSize,
+		service:     NewBulkService(p.c),
+		flushC:      make(chan struct{}),
+		flushAckC:   make(chan struct{}),
+	}
+}
+
+// work waits for bulk requests and manual flush calls on the respective
+// channels and is invoked as a goroutine when the bulk processor is started.
+func (w *bulkWorker) work(ctx context.Context) {
+	defer func() {
+		w.p.workerWg.Done()
+		close(w.flushAckC)
+		close(w.flushC)
+	}()
+
+	var stop bool
+	for !stop {
+		var err error
+		select {
+		case req, open := <-w.p.requestsC:
+			if open {
+				// Received a new request
+				if _, err = req.Source(); err == nil {
+					w.service.Add(req)
+					if w.commitRequired() {
+						err = w.commit(ctx)
+					}
+				}
+			} else {
+				// Channel closed: Stop.
+				stop = true
+				if w.service.NumberOfActions() > 0 {
+					err = w.commit(ctx)
+				}
+			}
+
+		case <-w.flushC:
+			// Commit outstanding requests
+			if w.service.NumberOfActions() > 0 {
+				err = w.commit(ctx)
+			}
+			w.flushAckC <- struct{}{}
+		}
+		if err != nil {
+			w.p.c.errorf("elastic: bulk processor %q was unable to perform work: %v", w.p.name, err)
+			if !stop {
+				waitForActive := func() {
+					// Add back pressure to prevent Add calls from filling up the request queue
+					ready := make(chan struct{})
+					go w.waitForActiveConnection(ready)
+					<-ready
+				}
+				if _, ok := err.(net.Error); ok {
+					waitForActive()
+				} else if IsConnErr(err) {
+					waitForActive()
+				}
+			}
+		}
+	}
+}
+
+// commit commits the bulk requests in the given service,
+// invoking callbacks as specified.
+func (w *bulkWorker) commit(ctx context.Context) error {
+	var res *BulkResponse
+
+	// commitFunc will commit bulk requests and, on failure, be retried
+	// via exponential backoff
+	commitFunc := func() error {
+		var err error
+		// Save requests because they will be reset in service.Do
+		reqs := w.service.requests
+		res, err = w.service.Do(ctx)
+		if err == nil {
+			// Overall bulk request was OK.  But each bulk response item also has a status
+			if w.p.retryItemStatusCodes != nil && len(w.p.retryItemStatusCodes) > 0 {
+				// Check res.Items since some might be soft failures
+				if res.Items != nil && res.Errors {
+					// res.Items will be 1 to 1 with reqs in same order
+					for i, item := range res.Items {
+						for _, result := range item {
+							if _, found := w.p.retryItemStatusCodes[result.Status]; found {
+								w.service.Add(reqs[i])
+								if err == nil {
+									err = ErrBulkItemRetry
+								}
+							}
+						}
+					}
+				}
+			}
+		}
+		return err
+	}
+	// notifyFunc will be called if retry fails
+	notifyFunc := func(err error) {
+		w.p.c.errorf("elastic: bulk processor %q failed but may retry: %v", w.p.name, err)
+	}
+
+	id := atomic.AddInt64(&w.p.executionId, 1)
+
+	// Update # documents in queue before eventual retries
+	w.p.statsMu.Lock()
+	if w.p.wantStats {
+		w.p.stats.Workers[w.i].Queued = int64(len(w.service.requests))
+	}
+	w.p.statsMu.Unlock()
+
+	// Save requests because they will be reset in commitFunc
+	reqs := w.service.requests
+
+	// Invoke before callback
+	if w.p.beforeFn != nil {
+		w.p.beforeFn(id, reqs)
+	}
+
+	// Commit bulk requests
+	err := RetryNotify(commitFunc, w.p.backoff, notifyFunc)
+	w.updateStats(res)
+	if err != nil {
+		w.p.c.errorf("elastic: bulk processor %q failed: %v", w.p.name, err)
+	}
+
+	// Invoke after callback
+	if w.p.afterFn != nil {
+		w.p.afterFn(id, reqs, res, err)
+	}
+
+	return err
+}
+
+func (w *bulkWorker) waitForActiveConnection(ready chan<- struct{}) {
+	defer close(ready)
+
+	t := time.NewTicker(5 * time.Second)
+	defer t.Stop()
+
+	client := w.p.c
+	stopReconnC := w.p.stopReconnC
+	w.p.c.errorf("elastic: bulk processor %q is waiting for an active connection", w.p.name)
+
+	// loop until a health check finds at least 1 active connection or the reconnection channel is closed
+	for {
+		select {
+		case _, ok := <-stopReconnC:
+			if !ok {
+				w.p.c.errorf("elastic: bulk processor %q active connection check interrupted", w.p.name)
+				return
+			}
+		case <-t.C:
+			client.healthcheck(context.Background(), 3*time.Second, true)
+			if client.mustActiveConn() == nil {
+				// found an active connection
+				// exit and signal done to the WaitGroup
+				return
+			}
+		}
+	}
+}
+
+func (w *bulkWorker) updateStats(res *BulkResponse) {
+	// Update stats
+	if res != nil {
+		w.p.statsMu.Lock()
+		if w.p.wantStats {
+			w.p.stats.Committed++
+			if res != nil {
+				w.p.stats.Indexed += int64(len(res.Indexed()))
+				w.p.stats.Created += int64(len(res.Created()))
+				w.p.stats.Updated += int64(len(res.Updated()))
+				w.p.stats.Deleted += int64(len(res.Deleted()))
+				w.p.stats.Succeeded += int64(len(res.Succeeded()))
+				w.p.stats.Failed += int64(len(res.Failed()))
+			}
+			w.p.stats.Workers[w.i].Queued = int64(len(w.service.requests))
+			w.p.stats.Workers[w.i].LastDuration = time.Duration(int64(res.Took)) * time.Millisecond
+		}
+		w.p.statsMu.Unlock()
+	}
+}
+
+// commitRequired returns true if the service has to commit its
+// bulk requests. This can be either because the number of actions
+// or the estimated size in bytes is larger than specified in the
+// BulkProcessorService.
+func (w *bulkWorker) commitRequired() bool {
+	if w.bulkActions >= 0 && w.service.NumberOfActions() >= w.bulkActions {
+		return true
+	}
+	if w.bulkSize >= 0 && w.service.EstimatedSizeInBytes() >= int64(w.bulkSize) {
+		return true
+	}
+	return false
+}

vendor/gopkg.in/olivere/elastic.v6/bulk_processor_test.go

+// Copyright 2012-present Oliver Eilhard. All rights reserved.
+// Use of this source code is governed by a MIT-license.
+// See http://olivere.mit-license.org/license.txt for details.
+
+package elastic
+
+import (
+	"context"
+	"fmt"
+	"math/rand"
+	"reflect"
+	"sync/atomic"
+	"testing"
+	"time"
+)
+
+func TestBulkProcessorDefaults(t *testing.T) {
+	client := setupTestClientAndCreateIndex(t)
+
+	p := client.BulkProcessor()
+	if p == nil {
+		t.Fatalf("expected BulkProcessorService; got: %v", p)
+	}
+	if got, want := p.name, ""; got != want {
+		t.Errorf("expected %q; got: %q", want, got)
+	}
+	if got, want := p.numWorkers, 1; got != want {
+		t.Errorf("expected %d; got: %d", want, got)
+	}
+	if got, want := p.bulkActions, 1000; got != want {
+		t.Errorf("expected %d; got: %d", want, got)
+	}
+	if got, want := p.bulkSize, 5*1024*1024; got != want {
+		t.Errorf("expected %d; got: %d", want, got)
+	}
+	if got, want := p.flushInterval, time.Duration(0); got != want {
+		t.Errorf("expected %v; got: %v", want, got)
+	}
+	if got, want := p.wantStats, false; got != want {
+		t.Errorf("expected %v; got: %v", want, got)
+	}
+	if got, want := p.retryItemStatusCodes, defaultRetryItemStatusCodes; !reflect.DeepEqual(got, want) {
+		t.Errorf("expected %v; got: %v", want, got)
+	}
+	if p.backoff == nil {
+		t.Fatalf("expected non-nill backoff; got: %v", p.backoff)
+	}
+}
+
+func TestBulkProcessorCommitOnBulkActions(t *testing.T) {
+	//client := setupTestClientAndCreateIndexAndLog(t, SetTraceLog(log.New(os.Stdout, "", 0)))
+	client := setupTestClientAndCreateIndex(t)
+
+	testBulkProcessor(t,
+		10000,
+		client.BulkProcessor().
+			Name("Actions-1").
+			Workers(1).
+			BulkActions(100).
+			BulkSize(-1),
+	)
+
+	testBulkProcessor(t,
+		10000,
+		client.BulkProcessor().
+			Name("Actions-2").
+			Workers(2).
+			BulkActions(100).
+			BulkSize(-1),
+	)
+}
+
+func TestBulkProcessorCommitOnBulkSize(t *testing.T) {
+	//client := setupTestClientAndCreateIndexAndLog(t, SetTraceLog(log.New(os.Stdout, "", 0)))
+	client := setupTestClientAndCreateIndex(t)
+
+	testBulkProcessor(t,
+		10000,
+		client.BulkProcessor().
+			Name("Size-1").
+			Workers(1).
+			BulkActions(-1).
+			BulkSize(64*1024),
+	)
+
+	testBulkProcessor(t,
+		10000,
+		client.BulkProcessor().
+			Name("Size-2").
+			Workers(2).
+			BulkActions(-1).
+			BulkSize(64*1024),
+	)
+}
+
+func TestBulkProcessorBasedOnFlushInterval(t *testing.T) {
+	//client := setupTestClientAndCreateIndexAndLog(t, SetTraceLog(log.New(os.Stdout, "", 0)))
+	client := setupTestClientAndCreateIndex(t)
+
+	var beforeRequests int64
+	var befores int64
+	var afters int64
+	var failures int64
+	var afterRequests int64
+
+	beforeFn := func(executionId int64, requests []BulkableRequest) {
+		atomic.AddInt64(&beforeRequests, int64(len(requests)))
+		atomic.AddInt64(&befores, 1)
+	}
+	afterFn := func(executionId int64, requests []BulkableRequest, response *BulkResponse, err error) {
+		atomic.AddInt64(&afters, 1)
+		if err != nil {
+			atomic.AddInt64(&failures, 1)
+		}
+		atomic.AddInt64(&afterRequests, int64(len(requests)))
+	}
+
+	svc := client.BulkProcessor().
+		Name("FlushInterval-1").
+		Workers(2).
+		BulkActions(-1).
+		BulkSize(-1).
+		FlushInterval(1 * time.Second).
+		Before(beforeFn).
+		After(afterFn)
+
+	p, err := svc.Do(context.Background())
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	const numDocs = 1000 // low-enough number that flush should be invoked
+
+	for i := 1; i <= numDocs; i++ {
+		tweet := tweet{User: "olivere", Message: fmt.Sprintf("%d. %s", i, randomString(rand.Intn(64)))}
+		request := NewBulkIndexRequest().Index(testIndexName).Type("doc").Id(fmt.Sprintf("%d", i)).Doc(tweet)
+		p.Add(request)
+	}
+
+	// Should flush at least once
+	time.Sleep(2 * time.Second)
+
+	err = p.Close()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if p.stats.Flushed == 0 {
+		t.Errorf("expected at least 1 flush; got: %d", p.stats.Flushed)
+	}
+	if got, want := beforeRequests, int64(numDocs); got != want {
+		t.Errorf("expected %d requests to before callback; got: %d", want, got)
+	}
+	if got, want := afterRequests, int64(numDocs); got != want {
+		t.Errorf("expected %d requests to after callback; got: %d", want, got)
+	}
+	if befores == 0 {
+		t.Error("expected at least 1 call to before callback")
+	}
+	if afters == 0 {
+		t.Error("expected at least 1 call to after callback")
+	}
+	if failures != 0 {
+		t.Errorf("expected 0 calls to failure callback; got: %d", failures)
+	}
+
+	// Check number of documents that were bulk indexed
+	_, err = p.c.Flush(testIndexName).Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	count, err := p.c.Count(testIndexName).Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if count != int64(numDocs) {
+		t.Fatalf("expected %d documents; got: %d", numDocs, count)
+	}
+}
+
+func TestBulkProcessorClose(t *testing.T) {
+	//client := setupTestClientAndCreateIndexAndLog(t, SetTraceLog(log.New(os.Stdout, "", 0)))
+	client := setupTestClientAndCreateIndex(t)
+
+	var beforeRequests int64
+	var befores int64
+	var afters int64
+	var failures int64
+	var afterRequests int64
+
+	beforeFn := func(executionId int64, requests []BulkableRequest) {
+		atomic.AddInt64(&beforeRequests, int64(len(requests)))
+		atomic.AddInt64(&befores, 1)
+	}
+	afterFn := func(executionId int64, requests []BulkableRequest, response *BulkResponse, err error) {
+		atomic.AddInt64(&afters, 1)
+		if err != nil {
+			atomic.AddInt64(&failures, 1)
+		}
+		atomic.AddInt64(&afterRequests, int64(len(requests)))
+	}
+
+	p, err := client.BulkProcessor().
+		Name("FlushInterval-1").
+		Workers(2).
+		BulkActions(-1).
+		BulkSize(-1).
+		FlushInterval(30 * time.Second). // 30 seconds to flush
+		Before(beforeFn).After(afterFn).
+		Do(context.Background())
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	const numDocs = 1000 // low-enough number that flush should be invoked
+
+	for i := 1; i <= numDocs; i++ {
+		tweet := tweet{User: "olivere", Message: fmt.Sprintf("%d. %s", i, randomString(rand.Intn(64)))}
+		request := NewBulkIndexRequest().Index(testIndexName).Type("doc").Id(fmt.Sprintf("%d", i)).Doc(tweet)
+		p.Add(request)
+	}
+
+	// Should not flush because 30s > 1s
+	time.Sleep(1 * time.Second)
+
+	// Close should flush
+	err = p.Close()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if p.stats.Flushed != 0 {
+		t.Errorf("expected no flush; got: %d", p.stats.Flushed)
+	}
+	if got, want := beforeRequests, int64(numDocs); got != want {
+		t.Errorf("expected %d requests to before callback; got: %d", want, got)
+	}
+	if got, want := afterRequests, int64(numDocs); got != want {
+		t.Errorf("expected %d requests to after callback; got: %d", want, got)
+	}
+	if befores == 0 {
+		t.Error("expected at least 1 call to before callback")
+	}
+	if afters == 0 {
+		t.Error("expected at least 1 call to after callback")
+	}
+	if failures != 0 {
+		t.Errorf("expected 0 calls to failure callback; got: %d", failures)
+	}
+
+	// Check number of documents that were bulk indexed
+	_, err = p.c.Flush(testIndexName).Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	count, err := p.c.Count(testIndexName).Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if count != int64(numDocs) {
+		t.Fatalf("expected %d documents; got: %d", numDocs, count)
+	}
+}
+
+func TestBulkProcessorFlush(t *testing.T) {
+	//client := setupTestClientAndCreateIndexAndLog(t, SetTraceLog(log.New(os.Stdout, "", 0)))
+	client := setupTestClientAndCreateIndex(t)
+
+	p, err := client.BulkProcessor().
+		Name("ManualFlush").
+		Workers(10).
+		BulkActions(-1).
+		BulkSize(-1).
+		FlushInterval(30 * time.Second). // 30 seconds to flush
+		Stats(true).
+		Do(context.Background())
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	const numDocs = 100
+
+	for i := 1; i <= numDocs; i++ {
+		tweet := tweet{User: "olivere", Message: fmt.Sprintf("%d. %s", i, randomString(rand.Intn(64)))}
+		request := NewBulkIndexRequest().Index(testIndexName).Type("doc").Id(fmt.Sprintf("%d", i)).Doc(tweet)
+		p.Add(request)
+	}
+
+	// Should not flush because 30s > 1s
+	time.Sleep(1 * time.Second)
+
+	// No flush yet
+	stats := p.Stats()
+	if stats.Flushed != 0 {
+		t.Errorf("expected no flush; got: %d", p.stats.Flushed)
+	}
+
+	// Manual flush
+	err = p.Flush()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	time.Sleep(1 * time.Second)
+
+	// Now flushed
+	stats = p.Stats()
+	if got, want := p.stats.Flushed, int64(1); got != want {
+		t.Errorf("expected %d flush; got: %d", want, got)
+	}
+
+	// Close should not start another flush
+	err = p.Close()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// Still 1 flush
+	stats = p.Stats()
+	if got, want := p.stats.Flushed, int64(1); got != want {
+		t.Errorf("expected %d flush; got: %d", want, got)
+	}
+
+	// Check number of documents that were bulk indexed
+	_, err = p.c.Flush(testIndexName).Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	count, err := p.c.Count(testIndexName).Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if count != int64(numDocs) {
+		t.Fatalf("expected %d documents; got: %d", numDocs, count)
+	}
+}
+
+// -- Helper --
+
+func testBulkProcessor(t *testing.T, numDocs int, svc *BulkProcessorService) {
+	var beforeRequests int64
+	var befores int64
+	var afters int64
+	var failures int64
+	var afterRequests int64
+
+	beforeFn := func(executionId int64, requests []BulkableRequest) {
+		atomic.AddInt64(&beforeRequests, int64(len(requests)))
+		atomic.AddInt64(&befores, 1)
+	}
+	afterFn := func(executionId int64, requests []BulkableRequest, response *BulkResponse, err error) {
+		atomic.AddInt64(&afters, 1)
+		if err != nil {
+			atomic.AddInt64(&failures, 1)
+		}
+		atomic.AddInt64(&afterRequests, int64(len(requests)))
+	}
+
+	p, err := svc.Before(beforeFn).After(afterFn).Stats(true).Do(context.Background())
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	for i := 1; i <= numDocs; i++ {
+		tweet := tweet{User: "olivere", Message: fmt.Sprintf("%07d. %s", i, randomString(1+rand.Intn(63)))}
+		request := NewBulkIndexRequest().Index(testIndexName).Type("doc").Id(fmt.Sprintf("%d", i)).Doc(tweet)
+		p.Add(request)
+	}
+
+	err = p.Close()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	stats := p.Stats()
+
+	if stats.Flushed != 0 {
+		t.Errorf("expected no flush; got: %d", stats.Flushed)
+	}
+	if stats.Committed <= 0 {
+		t.Errorf("expected committed > %d; got: %d", 0, stats.Committed)
+	}
+	if got, want := stats.Indexed, int64(numDocs); got != want {
+		t.Errorf("expected indexed = %d; got: %d", want, got)
+	}
+	if got, want := stats.Created, int64(0); got != want {
+		t.Errorf("expected created = %d; got: %d", want, got)
+	}
+	if got, want := stats.Updated, int64(0); got != want {
+		t.Errorf("expected updated = %d; got: %d", want, got)
+	}
+	if got, want := stats.Deleted, int64(0); got != want {
+		t.Errorf("expected deleted = %d; got: %d", want, got)
+	}
+	if got, want := stats.Succeeded, int64(numDocs); got != want {
+		t.Errorf("expected succeeded = %d; got: %d", want, got)
+	}
+	if got, want := stats.Failed, int64(0); got != want {
+		t.Errorf("expected failed = %d; got: %d", want, got)
+	}
+	if got, want := beforeRequests, int64(numDocs); got != want {
+		t.Errorf("expected %d requests to before callback; got: %d", want, got)
+	}
+	if got, want := afterRequests, int64(numDocs); got != want {
+		t.Errorf("expected %d requests to after callback; got: %d", want, got)
+	}
+	if befores == 0 {
+		t.Error("expected at least 1 call to before callback")
+	}
+	if afters == 0 {
+		t.Error("expected at least 1 call to after callback")
+	}
+	if failures != 0 {
+		t.Errorf("expected 0 calls to failure callback; got: %d", failures)
+	}
+
+	// Check number of documents that were bulk indexed
+	_, err = p.c.Flush(testIndexName).Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	count, err := p.c.Count(testIndexName).Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if count != int64(numDocs) {
+		t.Fatalf("expected %d documents; got: %d", numDocs, count)
+	}
+}

vendor/gopkg.in/olivere/elastic.v6/bulk_request.go

+// Copyright 2012-present Oliver Eilhard. All rights reserved.
+// Use of this source code is governed by a MIT-license.
+// See http://olivere.mit-license.org/license.txt for details.
+
+package elastic
+
+import (
+	"fmt"
+)
+
+// -- Bulkable request (index/update/delete) --
+
+// BulkableRequest is a generic interface to bulkable requests.
+type BulkableRequest interface {
+	fmt.Stringer
+	Source() ([]string, error)
+}

vendor/gopkg.in/olivere/elastic.v6/bulk_test.go

+// Copyright 2012-present Oliver Eilhard. All rights reserved.
+// Use of this source code is governed by a MIT-license.
+// See http://olivere.mit-license.org/license.txt for details.
+
+package elastic
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"math/rand"
+	"net/http"
+	"net/http/httptest"
+	"testing"
+)
+
+func TestBulk(t *testing.T) {
+	client := setupTestClientAndCreateIndex(t) //, SetTraceLog(log.New(os.Stdout, "", 0)))
+
+	tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
+	tweet2 := tweet{User: "sandrae", Message: "Dancing all night long. Yeah."}
+
+	index1Req := NewBulkIndexRequest().Index(testIndexName).Type("doc").Id("1").Doc(tweet1)
+	index2Req := NewBulkIndexRequest().Index(testIndexName).Type("doc").Id("2").Doc(tweet2)
+	delete1Req := NewBulkDeleteRequest().Index(testIndexName).Type("doc").Id("1")
+
+	bulkRequest := client.Bulk()
+	bulkRequest = bulkRequest.Add(index1Req)
+	bulkRequest = bulkRequest.Add(index2Req)
+	bulkRequest = bulkRequest.Add(delete1Req)
+
+	if bulkRequest.NumberOfActions() != 3 {
+		t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 3, bulkRequest.NumberOfActions())
+	}
+
+	bulkResponse, err := bulkRequest.Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if bulkResponse == nil {
+		t.Errorf("expected bulkResponse to be != nil; got nil")
+	}
+
+	if bulkRequest.NumberOfActions() != 0 {
+		t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 0, bulkRequest.NumberOfActions())
+	}
+
+	// Document with Id="1" should not exist
+	exists, err := client.Exists().Index(testIndexName).Type("doc").Id("1").Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if exists {
+		t.Errorf("expected exists %v; got %v", false, exists)
+	}
+
+	// Document with Id="2" should exist
+	exists, err = client.Exists().Index(testIndexName).Type("doc").Id("2").Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !exists {
+		t.Errorf("expected exists %v; got %v", true, exists)
+	}
+
+	// Update
+	updateDoc := struct {
+		Retweets int `json:"retweets"`
+	}{
+		42,
+	}
+	update1Req := NewBulkUpdateRequest().Index(testIndexName).Type("doc").Id("2").Doc(&updateDoc)
+	bulkRequest = client.Bulk()
+	bulkRequest = bulkRequest.Add(update1Req)
+
+	if bulkRequest.NumberOfActions() != 1 {
+		t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 1, bulkRequest.NumberOfActions())
+	}
+
+	bulkResponse, err = bulkRequest.Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if bulkResponse == nil {
+		t.Errorf("expected bulkResponse to be != nil; got nil")
+	}
+
+	if bulkRequest.NumberOfActions() != 0 {
+		t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 0, bulkRequest.NumberOfActions())
+	}
+
+	// Document with Id="1" should have a retweets count of 42
+	doc, err := client.Get().Index(testIndexName).Type("doc").Id("2").Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if doc == nil {
+		t.Fatal("expected doc to be != nil; got nil")
+	}
+	if !doc.Found {
+		t.Fatalf("expected doc to be found; got found = %v", doc.Found)
+	}
+	if doc.Source == nil {
+		t.Fatal("expected doc source to be != nil; got nil")
+	}
+	var updatedTweet tweet
+	err = json.Unmarshal(*doc.Source, &updatedTweet)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if updatedTweet.Retweets != 42 {
+		t.Errorf("expected updated tweet retweets = %v; got %v", 42, updatedTweet.Retweets)
+	}
+
+	// Update with script
+	update2Req := NewBulkUpdateRequest().Index(testIndexName).Type("doc").Id("2").
+		RetryOnConflict(3).
+		Script(NewScript("ctx._source.retweets += params.v").Param("v", 1))
+	bulkRequest = client.Bulk()
+	bulkRequest = bulkRequest.Add(update2Req)
+	if bulkRequest.NumberOfActions() != 1 {
+		t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 1, bulkRequest.NumberOfActions())
+	}
+	bulkResponse, err = bulkRequest.Refresh("wait_for").Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if bulkResponse == nil {
+		t.Errorf("expected bulkResponse to be != nil; got nil")
+	}
+
+	if bulkRequest.NumberOfActions() != 0 {
+		t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 0, bulkRequest.NumberOfActions())
+	}
+
+	// Document with Id="1" should have a retweets count of 43
+	doc, err = client.Get().Index(testIndexName).Type("doc").Id("2").Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if doc == nil {
+		t.Fatal("expected doc to be != nil; got nil")
+	}
+	if !doc.Found {
+		t.Fatalf("expected doc to be found; got found = %v", doc.Found)
+	}
+	if doc.Source == nil {
+		t.Fatal("expected doc source to be != nil; got nil")
+	}
+	err = json.Unmarshal(*doc.Source, &updatedTweet)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if updatedTweet.Retweets != 43 {
+		t.Errorf("expected updated tweet retweets = %v; got %v", 43, updatedTweet.Retweets)
+	}
+}
+
+func TestBulkWithIndexSetOnClient(t *testing.T) {
+	client := setupTestClientAndCreateIndex(t)
+
+	tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
+	tweet2 := tweet{User: "sandrae", Message: "Dancing all night long. Yeah."}
+
+	index1Req := NewBulkIndexRequest().Index(testIndexName).Type("doc").Id("1").Doc(tweet1).Routing("1")
+	index2Req := NewBulkIndexRequest().Index(testIndexName).Type("doc").Id("2").Doc(tweet2)
+	delete1Req := NewBulkDeleteRequest().Index(testIndexName).Type("doc").Id("1")
+
+	bulkRequest := client.Bulk().Index(testIndexName).Type("doc")
+	bulkRequest = bulkRequest.Add(index1Req)
+	bulkRequest = bulkRequest.Add(index2Req)
+	bulkRequest = bulkRequest.Add(delete1Req)
+
+	if bulkRequest.NumberOfActions() != 3 {
+		t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 3, bulkRequest.NumberOfActions())
+	}
+
+	bulkResponse, err := bulkRequest.Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if bulkResponse == nil {
+		t.Errorf("expected bulkResponse to be != nil; got nil")
+	}
+
+	// Document with Id="1" should not exist
+	exists, err := client.Exists().Index(testIndexName).Type("doc").Id("1").Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if exists {
+		t.Errorf("expected exists %v; got %v", false, exists)
+	}
+
+	// Document with Id="2" should exist
+	exists, err = client.Exists().Index(testIndexName).Type("doc").Id("2").Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !exists {
+		t.Errorf("expected exists %v; got %v", true, exists)
+	}
+}
+
+func TestBulkIndexDeleteUpdate(t *testing.T) {
+	client := setupTestClientAndCreateIndex(t)
+	//client := setupTestClientAndCreateIndexAndLog(t)
+
+	tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
+	tweet2 := tweet{User: "sandrae", Message: "Dancing all night long. Yeah."}
+
+	index1Req := NewBulkIndexRequest().Index(testIndexName).Type("doc").Id("1").Doc(tweet1)
+	index2Req := NewBulkIndexRequest().OpType("create").Index(testIndexName).Type("doc").Id("2").Doc(tweet2)
+	delete1Req := NewBulkDeleteRequest().Index(testIndexName).Type("doc").Id("1")
+	update2Req := NewBulkUpdateRequest().Index(testIndexName).Type("doc").Id("2").
+		ReturnSource(true).
+		Doc(struct {
+			Retweets int `json:"retweets"`
+		}{
+			Retweets: 42,
+		})
+
+	bulkRequest := client.Bulk()
+	bulkRequest = bulkRequest.Add(index1Req)
+	bulkRequest = bulkRequest.Add(index2Req)
+	bulkRequest = bulkRequest.Add(delete1Req)
+	bulkRequest = bulkRequest.Add(update2Req)
+
+	if bulkRequest.NumberOfActions() != 4 {
+		t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 4, bulkRequest.NumberOfActions())
+	}
+
+	expected := `{"index":{"_index":"` + testIndexName + `","_id":"1","_type":"doc"}}
+{"user":"olivere","message":"Welcome to Golang and Elasticsearch.","retweets":0,"created":"0001-01-01T00:00:00Z"}
+{"create":{"_index":"` + testIndexName + `","_id":"2","_type":"doc"}}
+{"user":"sandrae","message":"Dancing all night long. Yeah.","retweets":0,"created":"0001-01-01T00:00:00Z"}
+{"delete":{"_index":"` + testIndexName + `","_type":"doc","_id":"1"}}
+{"update":{"_index":"` + testIndexName + `","_type":"doc","_id":"2"}}
+{"doc":{"retweets":42},"_source":true}
+`
+	got, err := bulkRequest.bodyAsString()
+	if err != nil {
+		t.Fatalf("expected no error, got: %v", err)
+	}
+	if got != expected {
+		t.Errorf("expected\n%s\ngot:\n%s", expected, got)
+	}
+
+	// Run the bulk request
+	bulkResponse, err := bulkRequest.Pretty(true).Do(context.TODO())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if bulkResponse == nil {
+		t.Errorf("expected bulkResponse to be != nil; got nil")
+	}
+	if bulkResponse.Took == 0 {
+		t.Errorf("expected took to be > 0; got %d", bulkResponse.Took)
+	}
+	if bulkResponse.Errors {
+		t.Errorf("expected errors to be %v; got %v", false, bulkResponse.Errors)
+	}
+	if len(bulkResponse.Items) != 4 {
+		t.Fatalf("expected 4 result items; got %d", len(bulkResponse.Items))
+	}
+
+	// Indexed actions
+	indexed := bulkResponse.Indexed()
+	if indexed == nil {
+		t.Fatal("expected indexed to be != nil; got nil")
+	}
+	if len(indexed) != 1 {
+		t.Fatalf("expected len(indexed) == %d; got %d", 1, len(indexed))
+	}
+	if indexed[0].Id != "1" {
+		t.Errorf("expected indexed[0].Id == %s; got %s", "1", indexed[0].Id)
+	}
+	if indexed[0].Status != 201 {
+		t.Errorf("expected indexed[0].Status == %d; got %d", 201, indexed[0].Status)
+	}
+
+	// Created actions
+	created := bulkResponse.Created()
+	if created == nil {
+		t.Fatal("expected created to be != nil; got nil")
+	}
+	if len(created) != 1 {
+		t.Fatalf("expected len(created) == %d; got %d", 1, len(created))
+	}
+	if created[0].Id != "2" {
+		t.Errorf("expected created[0].Id == %s; got %s", "2", created[0].Id)
+	}
+	if created[0].Status != 201 {
+		t.Errorf("expected created[0].Status == %d; got %d", 201, created[0].Status)
+	}
+	if want, have := "created", created[0].Result; want != have {
+		t.Errorf("expected created[0].Result == %q; got %q", want, have)
+	}
+
+	// Deleted actions
+	deleted := bulkResponse.Deleted()
+	if deleted == nil {
+		t.Fatal("expected deleted to be != nil; got nil")
+	}
+	if len(deleted) != 1 {
+		t.Fatalf("expected len(deleted) == %d; got %d", 1, len(deleted))
+	}
+	if deleted[0].Id != "1" {
+		t.Errorf("expected deleted[0].Id == %s; got %s", "1", deleted[0].Id)
+	}
+	if deleted[0].Status != 200 {
+		t.Errorf("expected deleted[0].Status == %d; got %d", 200, deleted[0].Status)
+	}
+	if want, have := "deleted", deleted[0].Result; want != have {
+		t.Errorf("expected deleted[0].Result == %q; got %q", want, have)
+	}
+
+	// Updated actions
+	updated := bulkResponse.Updated()
+	if updated == nil {
+		t.Fatal("expected updated to be != nil; got nil")
+	}
+	if len(updated) != 1 {
+		t.Fatalf("expected len(updated) == %d; got %d", 1, len(updated))
+	}
+	if updated[0].Id != "2" {
+		t.Errorf("expected updated[0].Id == %s; got %s", "2", updated[0].Id)
+	}
+	if updated[0].Status != 200 {
+		t.Errorf("expected updated[0].Status == %d; got %d", 200, updated[0].Status)
+	}
+	if updated[0].Version != 2 {
+		t.Errorf("expected updated[0].Version == %d; got %d", 2, updated[0].Version)
+	}
+	if want, have := "updated", updated[0].Result; want != have {
+		t.Errorf("expected updated[0].Result == %q; got %q", want, have)
+	}
+	if updated[0].GetResult == nil {
+		t.Fatalf("expected updated[0].GetResult to be != nil; got nil")
+	}
+	if updated[0].GetResult.Source == nil {
+		t.Fatalf("expected updated[0].GetResult.Source to be != nil; got nil")
+	}
+	if want, have := true, updated[0].GetResult.Found; want != have {
+		t.Fatalf("expected updated[0].GetResult.Found to be != %v; got %v", want, have)
+	}
+	var doc tweet
+	if err := json.Unmarshal(*updated[0].GetResult.Source, &doc); err != nil {
+		t.Fatalf("expected to unmarshal updated[0].GetResult.Source; got %v", err)
+	}
+	if want, have := 42, doc.Retweets; want != have {
+		t.Fatalf("expected updated tweet to have Retweets = %v; got %v", want, have)
+	}
+
+	// Succeeded actions
+	succeeded := bulkResponse.Succeeded()
+	if succeeded == nil {
+		t.Fatal("expected succeeded to be != nil; got nil")
+	}
+	if len(succeeded) != 4 {
+		t.Fatalf("expected len(succeeded) == %d; got %d", 4, len(succeeded))
+	}
+
+	// ById
+	id1Results := bulkResponse.ById("1")
+	if id1Results == nil {
+		t.Fatal("expected id1Results to be != nil; got nil")
+	}
+	if len(id1Results) != 2 {
+		t.Fatalf("expected len(id1Results) == %d; got %d", 2, len(id1Results))
+	}
+	if id1Results[0].Id != "1" {
+		t.Errorf("expected id1Results[0].Id == %s; got %s", "1", id1Results[0].Id)
+	}
+	if id1Results[0].Status != 201 {
+		t.Errorf("expected id1Results[0].Status == %d; got %d", 201, id1Results[0].Status)
+	}
+	if id1Results[0].Version != 1 {
+		t.Errorf("expected id1Results[0].Version == %d; got %d", 1, id1Results[0].Version)
+	}
+	if id1Results[1].Id != "1" {
+		t.Errorf("expected id1Results[1].Id == %s; got %s", "1", id1Results[1].Id)
+	}
+	if id1Results[1].Status != 200 {
+		t.Errorf("expected id1Results[1].Status == %d; got %d", 200, id1Results[1].Status)
+	}
+	if id1Results[1].Version != 2 {
+		t.Errorf("expected id1Results[1].Version == %d; got %d", 2, id1Results[1].Version)
+	}
+}
+
+func TestFailedBulkRequests(t *testing.T) {
+	js := `{
+  "took" : 2,
+  "errors" : true,
+  "items" : [ {
+    "index" : {
+      "_index" : "elastic-test",
+      "_type" : "doc",
+      "_id" : "1",
+      "_version" : 1,
+      "status" : 201
+    }
+  }, {
+    "create" : {
+      "_index" : "elastic-test",
+      "_type" : "doc",
+      "_id" : "2",
+      "_version" : 1,
+      "status" : 423,
+      "error" : {
+      	"type":"routing_missing_exception",
+      	"reason":"routing is required for [elastic-test2]/[comment]/[1]"
+      }
+    }
+  }, {
+    "delete" : {
+      "_index" : "elastic-test",
+      "_type" : "doc",
+      "_id" : "1",
+      "_version" : 2,
+      "status" : 404,
+      "found" : false
+    }
+  }, {
+    "update" : {
+      "_index" : "elastic-test",
+      "_type" : "doc",
+      "_id" : "2",
+      "_version" : 2,
+      "status" : 200
+    }
+  } ]
+}`
+
+	var resp BulkResponse
+	err := json.Unmarshal([]byte(js), &resp)
+	if err != nil {
+		t.Fatal(err)
+	}
+	failed := resp.Failed()
+	if len(failed) != 2 {
+		t.Errorf("expected %d failed items; got: %d", 2, len(failed))
+	}
+}
+
+func TestBulkEstimatedSizeInBytes(t *testing.T) {
+	client := setupTestClientAndCreateIndex(t)
+
+	tweet1 := tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."}
+	tweet2 := tweet{User: "sandrae", Message: "Dancing all night long. Yeah."}
+
+	index1Req := NewBulkIndexRequest().Index(testIndexName).Type("doc").Id("1").Doc(tweet1)
+	index2Req := NewBulkIndexRequest().OpType("create").Index(testIndexName).Type("doc").Id("2").Doc(tweet2)
+	delete1Req := NewBulkDeleteRequest().Index(testIndexName).Type("doc").Id("1")
+	update2Req := NewBulkUpdateRequest().Index(testIndexName).Type("doc").Id("2").
+		Doc(struct {
+			Retweets int `json:"retweets"`
+		}{
+			Retweets: 42,
+		})
+
+	bulkRequest := client.Bulk()
+	bulkRequest = bulkRequest.Add(index1Req)
+	bulkRequest = bulkRequest.Add(index2Req)
+	bulkRequest = bulkRequest.Add(delete1Req)
+	bulkRequest = bulkRequest.Add(update2Req)
+
+	if bulkRequest.NumberOfActions() != 4 {
+		t.Errorf("expected bulkRequest.NumberOfActions %d; got %d", 4, bulkRequest.NumberOfActions())
+	}
+
+	// The estimated size of the bulk request in bytes must be at least
+	// the length of the body request.
+	raw, err := bulkRequest.bodyAsString()
+	if err != nil {
+		t.Fatal(err)
+	}
+	rawlen := int64(len([]byte(raw)))
+
+	if got, want := bulkRequest.EstimatedSizeInBytes(), rawlen; got < want {
+		t.Errorf("expected an EstimatedSizeInBytes = %d; got: %v", want, got)
+	}
+
+	// Reset should also reset the calculated estimated byte size
+	bulkRequest.Reset()
+
+	if got, want := bulkRequest.EstimatedSizeInBytes(), int64(0); got != want {
+		t.Errorf("expected an EstimatedSizeInBytes = %d; got: %v", want, got)
+	}
+}
+
+func TestBulkEstimateSizeInBytesLength(t *testing.T) {
+	client := setupTestClientAndCreateIndex(t)
+	s := client.Bulk()
+	r := NewBulkDeleteRequest().Index(testIndexName).Type("doc").Id("1")
+	s = s.Add(r)
+	if got, want := s.estimateSizeInBytes(r), int64(1+len(r.String())); got != want {
+		t.Fatalf("expected %d; got: %d", want, got)
+	}
+}
+
+func TestBulkContentType(t *testing.T) {
+	var header http.Header
+	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		header = r.Header
+		fmt.Fprintln(w, `{}`)
+	}))
+	defer ts.Close()
+
+	client, err := NewSimpleClient(SetURL(ts.URL))
+	if err != nil {
+		t.Fatal(err)
+	}
+	indexReq := NewBulkIndexRequest().Index(testIndexName).Type("doc").Id("1").Doc(tweet{User: "olivere", Message: "Welcome to Golang and Elasticsearch."})
+	if _, err := client.Bulk().Add(indexReq).Do(context.Background()); err != nil {
+		t.Fatal(err)
+	}
+	if header == nil {
+		t.Fatalf("expected header, got %v", header)
+	}
+	if want, have := "application/x-ndjson", header.Get("Content-Type"); want != have {
+		t.Fatalf("Content-Type: want %q, have %q", want, have)
+	}
+}
+
+// -- Benchmarks --
+
+var benchmarkBulkEstimatedSizeInBytes int64
+
+func BenchmarkBulkEstimatedSizeInBytesWith1Request(b *testing.B) {
+	client := setupTestClientAndCreateIndex(b)
+	s := client.Bulk()
+	var result int64
+	for n := 0; n < b.N; n++ {
+		s = s.Add(NewBulkIndexRequest().Index(testIndexName).Type("doc").Id("1").Doc(struct{ A string }{"1"}))
+		s = s.Add(NewBulkUpdateRequest().Index(testIndexName).Type("doc").Id("1").Doc(struct{ A string }{"2"}))
+		s = s.Add(NewBulkDeleteRequest().Index(testIndexName).Type("doc").Id("1"))
+		result = s.EstimatedSizeInBytes()
+		s.Reset()
+	}
+	b.ReportAllocs()
+	benchmarkBulkEstimatedSizeInBytes = result // ensure the compiler doesn't optimize
+}
+
+func BenchmarkBulkEstimatedSizeInBytesWith100Requests(b *testing.B) {
+	client := setupTestClientAndCreateIndex(b)
+	s := client.Bulk()
+	var result int64
+	for n := 0; n < b.N; n++ {
+		for i := 0; i < 100; i++ {
+			s = s.Add(NewBulkIndexRequest().Index(testIndexName).Type("doc").Id("1").Doc(struct{ A string }{"1"}))
+			s = s.Add(NewBulkUpdateRequest().Index(testIndexName).Type("doc").Id("1").Doc(struct{ A string }{"2"}))
+			s = s.Add(NewBulkDeleteRequest().Index(testIndexName).Type("doc").Id("1"))
+		}
+		result = s.EstimatedSizeInBytes()
+		s.Reset()
+	}
+	b.ReportAllocs()
+	benchmarkBulkEstimatedSizeInBytes = result // ensure the compiler doesn't optimize
+}
+
+func BenchmarkBulkAllocs(b *testing.B) {
+	b.Run("1000 docs with 64 byte", func(b *testing.B) { benchmarkBulkAllocs(b, 64, 1000) })
+	b.Run("1000 docs with 1 KiB", func(b *testing.B) { benchmarkBulkAllocs(b, 1024, 1000) })
+	b.Run("1000 docs with 4 KiB", func(b *testing.B) { benchmarkBulkAllocs(b, 4096, 1000) })
+	b.Run("1000 docs with 16 KiB", func(b *testing.B) { benchmarkBulkAllocs(b, 16*1024, 1000) })
+	b.Run("1000 docs with 64 KiB", func(b *testing.B) { benchmarkBulkAllocs(b, 64*1024, 1000) })
+	b.Run("1000 docs with 256 KiB", func(b *testing.B) { benchmarkBulkAllocs(b, 256*1024, 1000) })
+	b.Run("1000 docs with 1 MiB", func(b *testing.B) { benchmarkBulkAllocs(b, 1024*1024, 1000) })
+}
+
+const (
+	charset = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-"
+)
+
+func benchmarkBulkAllocs(b *testing.B, size, num int) {
+	buf := make([]byte, size)
+	for i := range buf {
+		buf[i] = charset[rand.Intn(len(charset))]
+	}
+
+	s := &BulkService{}
+	n := 0
+	for {
+		n++
+		s = s.Add(NewBulkIndexRequest().Index("test").Type("doc").Id("1").Doc(struct {
+			S string `json:"s"`
+		}{
+			S: string(buf),
+		}))
+		if n >= num {
+			break
+		}
+	}
+	for i := 0; i < b.N; i++ {
+		s.bodyAsString()
+	}
+	b.ReportAllocs()
+}

vendor/gopkg.in/olivere/elastic.v6/bulk_update_request_test.go

+// Copyright 2012-present Oliver Eilhard. All rights reserved.
+// Use of this source code is governed by a MIT-license.
+// See http://olivere.mit-license.org/license.txt for details.
+
+package elastic
+
+import (
+	"encoding/json"
+	"testing"
+)
+
+func TestBulkUpdateRequestSerialization(t *testing.T) {
+	rawJson := json.RawMessage(`{"counter":42}`)
+	rawString := `{"counter":42}`
+	tests := []struct {
+		Request  BulkableRequest
+		Expected []string
+	}{
+		// #0
+		{
+			Request: NewBulkUpdateRequest().Index("index1").Type("doc").Id("1").Doc(struct {
+				Counter int64 `json:"counter"`
+			}{
+				Counter: 42,
+			}),
+			Expected: []string{
+				`{"update":{"_index":"index1","_type":"doc","_id":"1"}}`,
+				`{"doc":{"counter":42}}`,
+			},
+		},
+		// #1
+		{
+			Request: NewBulkUpdateRequest().Index("index1").Type("doc").Id("1").
+				Routing("123").
+				RetryOnConflict(3).
+				DocAsUpsert(true).
+				Doc(struct {
+					Counter int64 `json:"counter"`
+				}{
+					Counter: 42,
+				}),
+			Expected: []string{
+				`{"update":{"_index":"index1","_type":"doc","_id":"1","retry_on_conflict":3,"routing":"123"}}`,
+				`{"doc":{"counter":42},"doc_as_upsert":true}`,
+			},
+		},
+		// #2
+		{
+			Request: NewBulkUpdateRequest().Index("index1").Type("doc").Id("1").
+				RetryOnConflict(3).
+				Script(NewScript(`ctx._source.retweets += param1`).Lang("javascript").Param("param1", 42)).
+				Upsert(struct {
+					Counter int64 `json:"counter"`
+				}{
+					Counter: 42,
+				}),
+			Expected: []string{
+				`{"update":{"_index":"index1","_type":"doc","_id":"1","retry_on_conflict":3}}`,
+				`{"script":{"lang":"javascript","params":{"param1":42},"source":"ctx._source.retweets += param1"},"upsert":{"counter":42}}`,
+			},
+		},
+		// #3
+		{
+			Request: NewBulkUpdateRequest().Index("index1").Type("doc").Id("1").DetectNoop(true).Doc(struct {
+				Counter int64 `json:"counter"`
+			}{
+				Counter: 42,
+			}),
+			Expected: []string{
+				`{"update":{"_index":"index1","_type":"doc","_id":"1"}}`,
+				`{"detect_noop":true,"doc":{"counter":42}}`,
+			},
+		},
+		// #4
+		{
+			Request: NewBulkUpdateRequest().Index("index1").Type("doc").Id("1").
+				RetryOnConflict(3).
+				ScriptedUpsert(true).
+				Script(NewScript(`ctx._source.retweets += param1`).Lang("javascript").Param("param1", 42)).
+				Upsert(struct {
+					Counter int64 `json:"counter"`
+				}{
+					Counter: 42,
+				}),
+			Expected: []string{
+				`{"update":{"_index":"index1","_type":"doc","_id":"1","retry_on_conflict":3}}`,
+				`{"script":{"lang":"javascript","params":{"param1":42},"source":"ctx._source.retweets += param1"},"scripted_upsert":true,"upsert":{"counter":42}}`,
+			},
+		},
+		// #5
+		{
+			Request: NewBulkUpdateRequest().Index("index1").Type("doc").Id("4").ReturnSource(true).Doc(struct {
+				Counter int64 `json:"counter"`
+			}{
+				Counter: 42,
+			}),
+			Expected: []string{
+				`{"update":{"_index":"index1","_type":"doc","_id":"4"}}`,
+				`{"doc":{"counter":42},"_source":true}`,
+			},
+		},
+		// #6
+		{
+			Request: NewBulkUpdateRequest().Index("index1").Type("doc").Id("4").
+				ReturnSource(true).
+				Doc(`{"counter":42}`),
+			Expected: []string{
+				`{"update":{"_index":"index1","_type":"doc","_id":"4"}}`,
+				`{"doc":{"counter":42},"_source":true}`,
+			},
+		},
+		// #7
+		{
+			Request: NewBulkUpdateRequest().Index("index1").Type("doc").Id("4").
+				ReturnSource(true).
+				Doc(json.RawMessage(`{"counter":42}`)),
+			Expected: []string{
+				`{"update":{"_index":"index1","_type":"doc","_id":"4"}}`,
+				`{"doc":{"counter":42},"_source":true}`,
+			},
+		},
+		// #8
+		{
+			Request: NewBulkUpdateRequest().Index("index1").Type("doc").Id("4").
+				ReturnSource(true).
+				Doc(&rawJson),
+			Expected: []string{
+				`{"update":{"_index":"index1","_type":"doc","_id":"4"}}`,
+				`{"doc":{"counter":42},"_source":true}`,
+			},
+		},
+		// #9
+		{
+			Request: NewBulkUpdateRequest().Index("index1").Type("doc").Id("4").
+				ReturnSource(true).
+				Doc(&rawString),
+			Expected: []string{
+				`{"update":{"_index":"index1","_type":"doc","_id":"4"}}`,
+				`{"doc":{"counter":42},"_source":true}`,
+			},
+		},
+	}
+
+	for i, test := range tests {
+		lines, err := test.Request.Source()
+		if err != nil {
+			t.Fatalf("#%d: expected no error, got: %v", i, err)
+		}
+		if lines == nil {
+			t.Fatalf("#%d: expected lines, got nil", i)
+		}
+		if len(lines) != len(test.Expected) {
+			t.Fatalf("#%d: expected %d lines, got %d", i, len(test.Expected), len(lines))
+		}
+		for j, line := range lines {
+			if line != test.Expected[j] {
+				t.Errorf("#%d: expected line #%d to be\n%s\nbut got:\n%s", i, j, test.Expected[j], line)
+			}
+		}
+	}
+}
+
+var bulkUpdateRequestSerializationResult string
+
+func BenchmarkBulkUpdateRequestSerialization(b *testing.B) {
+	b.Run("stdlib", func(b *testing.B) {
+		r := NewBulkUpdateRequest().Index("index1").Type("doc").Id("1").Doc(struct {
+			Counter int64 `json:"counter"`
+		}{
+			Counter: 42,
+		})
+		benchmarkBulkUpdateRequestSerialization(b, r.UseEasyJSON(false))
+	})
+	b.Run("easyjson", func(b *testing.B) {
+		r := NewBulkUpdateRequest().Index("index1").Type("doc").Id("1").Doc(struct {
+			Counter int64 `json:"counter"`
+		}{
+			Counter: 42,
+		}).UseEasyJSON(false)
+		benchmarkBulkUpdateRequestSerialization(b, r.UseEasyJSON(true))
+	})
+}
+
+func benchmarkBulkUpdateRequestSerialization(b *testing.B, r *BulkUpdateRequest) {
+	var s string
+	for n := 0; n < b.N; n++ {
+		s = r.String()
+		r.source = nil // Don't let caching spoil the benchmark
+	}
+	bulkUpdateRequestSerializationResult = s // ensure the compiler doesn't optimize
+	b.ReportAllocs()
+}

vendor/gopkg.in/olivere/elastic.v6/cat_allocation_test.go

+// Copyright 2012-present Oliver Eilhard. All rights reserved.
+// Use of this source code is governed by a MIT-license.
+// See http://olivere.mit-license.org/license.txt for details.
+
+package elastic
+
+import (
+	"context"
+	"testing"
+)
+
+func TestCatAllocation(t *testing.T) {
+	client := setupTestClientAndCreateIndexAndAddDocs(t) // , SetTraceLog(log.New(os.Stdout, "", 0)))
+	ctx := context.Background()
+	res, err := client.CatAllocation().Do(ctx)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if res == nil {
+		t.Fatal("want response, have nil")
+	}
+	if len(res) == 0 {
+		t.Fatalf("want response, have: %v", res)
+	}
+	if have := res[0].IP; have == "" {
+		t.Fatalf("IP[0]: want != %q, have %q", "", have)
+	}
+}