{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/10217","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/10217/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/10217/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/10217/events","html_url":"https://github.com/elastic/elasticsearch/issues/10217","id":63624736,"number":10217,"title":"Refactor parsing of queries/filters, aggs, suggester APIs","user":{"login":"clintongormley","id":56599,"avatar_url":"https://avatars.githubusercontent.com/u/56599?v=3","gravatar_id":"","url":"https://api.github.com/users/clintongormley","html_url":"https://github.com/clintongormley","followers_url":"https://api.github.com/users/clintongormley/followers","following_url":"https://api.github.com/users/clintongormley/following{/other_user}","gists_url":"https://api.github.com/users/clintongormley/gists{/gist_id}","starred_url":"https://api.github.com/users/clintongormley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/clintongormley/subscriptions","organizations_url":"https://api.github.com/users/clintongormley/orgs","repos_url":"https://api.github.com/users/clintongormley/repos","events_url":"https://api.github.com/users/clintongormley/events{/privacy}","received_events_url":"https://api.github.com/users/clintongormley/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/:Java%20API","name":":Java API","color":"009800"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/breaking","name":"breaking","color":"ff0000"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/enhancement","name":"enhancement","color":"4a4ea8"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/Meta","name":"Meta","color":"e11d21"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-03-23T04:43:40Z","updated_at":"2015-09-03T09:49:22Z","closed_at":null,"body":"Copied from #9901:\r\n\r\nToday we have a massive infrastructure to parse all our requests. We have client side builders and server side parsers but no real representation of the query, filter, aggregation etc until it's executed. What is produced from a XContent binary is a Lucene query directly which causes huge parse methods in separate classes etc. that hare hard to test and don't allow decoupled modifications or actions on the query itself between parsing and executing. \r\n\r\nThis PR is a small prototype how things could look in the future that would allow for more flexibility and cleaner code IMO.\r\n\r\nThis refactoring splits the parsing and the creation of the lucene query, this has a couple of advantages\r\n * XContent parsing creation are in one file and can be tested more easily\r\n * the class allows a typed in-memory representation of the query that can be modified before a lucene query is build\r\n * the query can be normalized and serialized via Streamable to be used as a normalized cache key (not depending on the order of the keys in the XContent)\r\n * the query can be parsed on the coordinating node to allow document prefetching etc. forwarding to the executing nodes would work via Streamable binary representation --> https://github.com/elasticsearch/elasticsearch/issues/8150\r\n * for the query cache a query tree can be \"walked\" to rewrite range queries into match all queries with MIN/MAX terms to get cache hits for sliding windows --> https://github.com/elasticsearch/elasticsearch/issues/9526\r\n * code wise two classes are merged into one which is nice\r\n * filter and query can maybe share once class and we add a `toFilter(QueryParserContenxt ctx)` method that returns a filter and by default return a `new QueryWrapperFilter(toQuery(context));`\r\n\r\nQueries\r\n------------\r\n* [x] AndQueryBuilder  (https://github.com/elastic/elasticsearch/pull/11628) \r\n* [x] BoolQueryBuilder (https://github.com/elastic/elasticsearch/pull/11121)\r\n* [x] BoostingQueryBuilder (https://github.com/elastic/elasticsearch/pull/11621)\r\n* [x] CommonTermsQueryBuilder (https://github.com/elastic/elasticsearch/pull/11345)\r\n* [x] ConstantScoreQueryBuilder (https://github.com/elastic/elasticsearch/pull/11629)\r\n* [x] DisMaxQueryBuilder (https://github.com/elastic/elasticsearch/pull/11703)\r\n* [x] ExistsQueryBuilder (https://github.com/elastic/elasticsearch/pull/11427)\r\n* [x] FieldMaskingSpanQueryBuilder (https://github.com/elastic/elasticsearch/pull/11717)\r\n* [x] FilteredQueryBuilder (https://github.com/elastic/elasticsearch/pull/11885)\r\n* [x] FQueryFilterBuilder (https://github.com/elastic/elasticsearch/pull/11729)\r\n* [ ] FunctionScoreQueryBuilder (https://github.com/elastic/elasticsearch/pull/12608)\r\n* [x] FuzzyQueryBuilder (https://github.com/elastic/elasticsearch/pull/11865)\r\n* [ ] GeoBoundingBoxQueryBuilder (https://github.com/elastic/elasticsearch/pull/11969)\r\n* [ ] GeoDistanceQueryBuilder(https://github.com/elastic/elasticsearch/pull/12283)\r\n* [ ] GeoDistanceRangeQueryBuilder\r\n* [ ] GeohashCellQuery\r\n* [ ] GeoPolygonQueryBuilder\r\n* [ ] GeoShapeQueryBuilder\r\n* [ ] HasChildQueryBuilder\r\n* [ ] HasParentQueryBuilder\r\n* [x] IdsQueryBuilder\r\n* [x] IndicesQueryBuilder (https://github.com/elastic/elasticsearch/pull/12031)\r\n* [x] LimitQueryBuilder\r\n* [x] MatchAllQueryBuilder\r\n* [ ] MatchQueryBuilder\r\n* [x] MissingQueryBuilder (https://github.com/elastic/elasticsearch/pull/12030)\r\n* [ ] MoreLikeThisQueryBuilder\r\n* [ ] MultiMatchQueryBuilder\r\n* [ ] NestedQueryBuilder\r\n* [x] NotQueryBuilder (https://github.com/elastic/elasticsearch/pull/11823)\r\n* [x] OrQueryBuilder (https://github.com/elastic/elasticsearch/pull/11817)\r\n* [x] PrefixQueryBuilder (https://github.com/elastic/elasticsearch/pull/12032)\r\n* [x] QueryFilterBuilder (https://github.com/elastic/elasticsearch/pull/11729)\r\n* [ ] QueryStringQueryBuilder\r\n* [x] QueryWrappingQueryBuilder (will be removed once all queries refactored)\r\n* [x] RangeQueryBuilder\r\n* [x] RegexpQueryBuilder (https://github.com/elastic/elasticsearch/pull/11896)\r\n* [x] ScriptQueryBuilder (https://github.com/elastic/elasticsearch/pull/12115)\r\n* [x] SimpleQueryStringBuilder (https://github.com/elastic/elasticsearch/pull/11274)\r\n* [x] SpanContainingQueryBuilder (https://github.com/elastic/elasticsearch/pull/12057)\r\n* [x] SpanFirstQueryBuilder (https://github.com/elastic/elasticsearch/pull/12073)\r\n* [x] SpanMultiTermQueryBuilder (https://github.com/elastic/elasticsearch/pull/12182)\r\n* [x] SpanNearQueryBuilder (https://github.com/elastic/elasticsearch/pull/12156)\r\n* [x] SpanNotQueryBuilder (https://github.com/elastic/elasticsearch/pull/12365)\r\n* [x] SpanOrQueryBuilder (https://github.com/elastic/elasticsearch/pull/12342)\r\n* [x] SpanTermQueryBuilder\r\n* [x] SpanWithinQueryBuilder\r\n* [ ] TemplateQueryBuilder (https://github.com/elastic/elasticsearch/pull/13253)\r\n* [x] TermQueryBuilder\r\n* [x] TermsLookupQueryBuilder (https://github.com/elastic/elasticsearch/pull/12042)\r\n* [x] TermsQueryBuilder (https://github.com/elastic/elasticsearch/pull/12042)\r\n* [x] TypeQueryBuilder (https://github.com/elastic/elasticsearch/pull/12035)\r\n* [x] WildcardQueryBuilder (https://github.com/elastic/elasticsearch/pull/12110)\r\n* [x] WrapperQueryBuilder (https://github.com/elastic/elasticsearch/pull/12037)\r\n\r\n* Total of 54 Queries\r\n* 39 done\r\n\r\nFormer filters were mostly merged or converted to queries and are inclused in this list.\r\n\r\nAggregations\r\n-----------------\r\n* [ ] Min Aggregation\r\n* [ ] Max Aggregation\r\n* [ ] Sum Aggregation\r\n* [ ] Avg Aggregation\r\n* [ ] Stats Aggregation\r\n* [ ] Extended Stats Aggregation\r\n* [ ] Value Count Aggregation\r\n* [ ] Percentiles Aggregation\r\n* [ ] Percentile Ranks Aggregation\r\n* [ ] Cardinality Aggregation\r\n* [ ] Geo Bounds Aggregation\r\n* [ ] Top hits Aggregation\r\n* [ ] Scripted Metric Aggregation\r\n* [ ] Global Aggregation\r\n* [ ] Filter Aggregation\r\n* [ ] Filters Aggregation\r\n* [ ] Missing Aggregation\r\n* [ ] Nested Aggregation\r\n* [ ] Reverse nested Aggregation\r\n* [ ] Children Aggregation\r\n* [ ] Terms Aggregation\r\n* [ ] Significant Terms Aggregation\r\n* [ ] Sampler Aggregation\r\n* [ ] Range Aggregation\r\n* [ ] Date Range Aggregation\r\n* [ ] IPv4 Range Aggregation\r\n* [ ] Histogram Aggregation\r\n* [ ] Date Histogram Aggregation\r\n* [ ] Geo Distance Aggregation\r\n* [ ] GeoHash grid Aggregation\r\n* [ ] Min Bucket Aggregation\r\n* [ ] Max Bucket Aggregation\r\n* [ ] Avg Bucket Aggregation\r\n* [ ] Sum Bucket Aggregation\r\n* [ ] Bucket Script Aggregation\r\n* [ ] Cumulative Sum Aggregation\r\n* [ ] Derivative Aggregation\r\n* [ ] Bucket Selector Aggregation\r\n* [ ] Moving Average Aggregation\r\n* [ ] Serial Differencing Aggregation\r\n\r\nTotal of 40 Aggregations\r\n0 done, 0 open PRs\r\n\r\nSuggesters\r\n----------------\r\n* [ ] Term suggester\r\n* [ ] Phrase Suggester\r\n* [ ] Completion Suggester\r\n* [ ] Context Suggester\r\n\r\nHighlighters\r\n----------------\r\n* [ ] plain\r\n* [ ] fvh\r\n* [ ] postings\r\n\r\nOthers\r\n----------------\r\n* [ ] from\r\n* [ ] size\r\n* [ ] min_score\r\n* [ ] fields\r\n* [ ] script_fields\r\n* [ ] fielddata_fields\r\n* [ ] _source\r\n* [ ] sort\r\n* [ ] timeout\r\n* [ ] explain\r\n* [ ] terminate_after\r\n* [ ] version\r\n* [ ] stats\r\n* [ ] indices_boost\r\n* [ ] track_scores\r\n* [ ] inner_hits\r\n* [ ] query_binary\r\n* [ ] filter_binary\r\n* [ ] aggregations_binary\r\n* [ ] rescore\r\n* [ ] post_filter"},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/137394610","html_url":"https://github.com/elastic/elasticsearch/issues/10217#issuecomment-137394610","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/10217","id":137394610,"user":{"login":"javanna","id":832460,"avatar_url":"https://avatars.githubusercontent.com/u/832460?v=3","gravatar_id":"","url":"https://api.github.com/users/javanna","html_url":"https://github.com/javanna","followers_url":"https://api.github.com/users/javanna/followers","following_url":"https://api.github.com/users/javanna/following{/other_user}","gists_url":"https://api.github.com/users/javanna/gists{/gist_id}","starred_url":"https://api.github.com/users/javanna/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/javanna/subscriptions","organizations_url":"https://api.github.com/users/javanna/orgs","repos_url":"https://api.github.com/users/javanna/repos","events_url":"https://api.github.com/users/javanna/events{/privacy}","received_events_url":"https://api.github.com/users/javanna/received_events","type":"User","site_admin":false},"created_at":"2015-09-03T09:49:22Z","updated_at":"2015-09-03T09:49:22Z","body":"I've just updated the description of this issue and added all the other search sections that will need to be refactored, along with queries and aggregations."}}