Elasticsearch Bulk Index JSON Data

by

What you need to do is to read that JSON file and then build a bulk request with the format expected by the _bulk endpoint, i.e. one line for the command and one line for the document, separated by a newline character… rinse and repeat for each document:

curl -XPOST localhost:9200/your_index/_bulk -d '
{"index": {"_index": "your_index", "_type": "your_type", "_id": "975463711"}}
{"Amount": "480", "Quantity": "2", "Id": "975463711", "Client_Store_sk": "1109"}
{"index": {"_index": "your_index", "_type": "your_type", "_id": "975463943"}}
{"Amount": "2105", "Quantity": "2", "Id": "975463943", "Client_Store_sk": "1109"}
... etc for all your documents
'

Just make sure to replace your_index and your_type with the actual index and type names you’re using.

UPDATE

Note that the command-line can be shortened, by removing _index and _type if those are specified in your URL. It is also possible to remove _id if you specify the path to your id field in your mapping (note that this feature will be deprecated in ES 2.0, though). At the very least, your command line can look like {"index":{}} for all documents but it will always be mandatory in order to specify which kind of operation you want to perform (in this case index the document)

UPDATE 2

curl -XPOST localhost:9200/index_local/my_doc_type/_bulk --data-binary  @/home/data1.json

/home/data1.json should look like this:

{"index":{}}
{"Amount": "480", "Quantity": "2", "Id": "975463711", "Client_Store_sk": "1109"}
{"index":{}}
{"Amount": "2105", "Quantity": "2", "Id": "975463943", "Client_Store_sk": "1109"}
{"index":{}}
{"Amount": "2107", "Quantity": "3", "Id": "974920111", "Client_Store_sk": "1109"}

UPDATE 3

You can refer to this answer to see how to generate the new json style file mentioned in UPDATE 2.

UPDATE 4

As of ES 7.x, the doc_type is not necessary anymore and should simply be _doc instead of my_doc_type. As of ES 8.x, the doc type will be removed completely. You can read more about this here

More Related Contents:

  1. Import/Index a JSON file into Elasticsearch
  2. Elasticsearch Bulk JSON Data
  3. Can JSON start with “[“?
  4. Parse JSON with R
  5. Use of PUT vs PATCH methods in REST API real life scenarios
  6. Representing null in JSON
  7. How to send a POST request with BODY in swift
  8. Reading in a JSON File Using Swift
  9. (un)marshalling json golang not working
  10. Angular 2: Get Values of Multiple Checked Checkboxes
  11. Swift JSONEncoder number rounding
  12. Return JSON from ASMX web service, without XML wrapper?
  13. JSON or XML? Which is better? [closed]
  14. Laravel: How do I parse this json data in view blade?
  15. work with json in oracle
  16. How to convert a JSON object stream into an array with jq
  17. Json.NET (Newtonsoft.Json) – Two ‘properties’ with same name?
  18. Formatting DynamoDB data to normal JSON in AWS Lambda
  19. Error in searching data using listview in flutter
  20. Parse JSON to array in a shell script
  21. Accessing a request’s body using classic ASP?
  22. In Excel VBA on Windows, how to get stringified JSON respresentation instead of “[object Object]” for parsed JSON variables?
  23. How to parse JSON from the Invoke-WebRequest in PowerShell?
  24. AngularJS Intercept all $http JSON responses
  25. How to escape single quotes into double quotes into single quotes
  26. How to convert a JSON file to an SQLite database
  27. Any when decoding JSON with Codable?
  28. How to access remote data and write it into a file during Nuxt build?
  29. Using VBA and VBA-JSON to access JSON data from WordPress API
  30. Did the publication of ECMA-404 affect the validity of JSON texts such as ‘2’ or ‘”hello”‘?

Leave a Comment