{"action":"closed","issue":{"url":"https://api.github.com/repos/calrissian/flowmix/issues/45","labels_url":"https://api.github.com/repos/calrissian/flowmix/issues/45/labels{/name}","comments_url":"https://api.github.com/repos/calrissian/flowmix/issues/45/comments","events_url":"https://api.github.com/repos/calrissian/flowmix/issues/45/events","html_url":"https://github.com/calrissian/flowmix/issues/45","id":54722242,"number":45,"title":"Light Aggregator (not keeping the whole field group)","user":{"login":"miguelantonio","id":1669317,"avatar_url":"https://avatars.githubusercontent.com/u/1669317?v=3","gravatar_id":"","url":"https://api.github.com/users/miguelantonio","html_url":"https://github.com/miguelantonio","followers_url":"https://api.github.com/users/miguelantonio/followers","following_url":"https://api.github.com/users/miguelantonio/following{/other_user}","gists_url":"https://api.github.com/users/miguelantonio/gists{/gist_id}","starred_url":"https://api.github.com/users/miguelantonio/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/miguelantonio/subscriptions","organizations_url":"https://api.github.com/users/miguelantonio/orgs","repos_url":"https://api.github.com/users/miguelantonio/repos","events_url":"https://api.github.com/users/miguelantonio/events{/privacy}","received_events_url":"https://api.github.com/users/miguelantonio/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-01-19T03:09:48Z","updated_at":"2015-01-23T02:29:26Z","closed_at":"2015-01-23T02:29:26Z","body":"Right now the aggregators keep the field group and do the math on the fly, I've seen the following approach in Tibco Streambase to process aggregation:\r\n1- the tuple (as group of attributes/columns, basically a map) gets into the aggregator and only the aggregator input fields are considered and the rest is forgotten (the rest of the tuple)\r\n2- the value gets processed into simple math: add, subtract. This is in a deconstructed manner, ej. avg is just one Long/BigInteger count and one Long/BigInteger sum. The actual field gets forgotten at this time.\r\n3- when the aggregation window closes (emits, etc.) the 'heavy' math is done: multiplication, division, etc.\r\n\r\nI've seen millions of tuples get processed into this type of aggregator with very low CPU and memory consumption and attack ships on fire off the shoulder of Orion"}}