The answer for this one was tricky, but thanks to samklr, I have managed to figure about what the problem was. The solution isn’t straightforward nevertheless and might consider some “unnecessary” transformations. First let’s talk about Serialization. There are two aspects of serialization to consider in Spark serialization of data and serialization of functions. In … Read more
Use a terms aggregation on the color field. And you need to pay attention to how that field you want to get distinct values on is analyzed, meaning you need to make sure you’re not tokenizing it while indexing, otherwise every entry in the aggregation will be a different term that is part of the … Read more
“my_field2”: { “properties”: { “title”: { “type”: “string”, “index”: “not_analyzed” } } } Check you here, https://www.elastic.co/guide/en/elasticsearch/reference/1.4/mapping-core-types.html, for further info.
Just create a template. run curl -XPUT localhost:9200/_template/template_1 -d ‘{ “template”: “*”, “settings”: { “index.refresh_interval”: “5s” }, “mappings”: { “_default_”: { “_all”: { “enabled”: true }, “dynamic_templates”: [ { “string_fields”: { “match”: “*”, “match_mapping_type”: “string”, “mapping”: { “index”: “not_analyzed”, “omit_norms”: true, “type”: “string” } } } ], “properties”: { “@version”: { “type”: “string”, “index”: “not_analyzed” … Read more
I thought that if “store” is set to “no” it would mean I could not retrieve the specific field, but had to get the whole _source and parse it on the client side. That’s exactly what elasticsearch does for you when a field is not stored (default) and the _source field is enabled (default too). … Read more
The difference is simple: filters are cached and don’t influence the score, therefore faster than queries. Have a look here too. Let’s say a query is usually something that the users type and pretty much unpredictable, while filters help users narrowing down the search results , for example using facets.
There are 2 steps to this — a dynamic_mapping and an ingest_pipeline. I’m assuming your field c is non-trivial so you may want to match that field in a dynamic template using a match and assign the keyword mapping to it: PUT synthetic { “mappings”: { “dynamic_templates”: [ { “c_like_field”: { “match_mapping_type”: “string”, “match”: “c*”, … Read more
You could use the update by query plugin in order to do just that. The idea is to select all document without a category and whose url matches a certain string and add the category you wish. curl -XPOST ‘localhost:9200/webproxylog/_update_by_query’ -H “Content-Type: application/json” -d ‘ { “query”: { “filtered”: { “filter”: { “bool”: { “must”: … Read more
So you are running into an issue with the maximum size for a single term. When you set a field to not_analyzed it will treat it as one single term. The maximum size for a single term in the underlying Lucene index is 32766 bytes, which is I believe hard coded. Your two primary options … Read more
I don’t think there is any limit set by Elaticsearch or Lucene explicitly. The limit you might hit, though, is the one set in place by the JDK. To prove my statement above, I looked at the source code of Elasticsearch: when the request comes in there is a parser that parses the array of … Read more