{"action":"created","issue":{"url":"https://api.github.com/repos/mbraceproject/MBrace.Core/issues/149","labels_url":"https://api.github.com/repos/mbraceproject/MBrace.Core/issues/149/labels{/name}","comments_url":"https://api.github.com/repos/mbraceproject/MBrace.Core/issues/149/comments","events_url":"https://api.github.com/repos/mbraceproject/MBrace.Core/issues/149/events","html_url":"https://github.com/mbraceproject/MBrace.Core/issues/149","id":114468608,"number":149,"title":"Predicate across averageByKey, percentageByKey etc.","user":{"login":"isaacabraham","id":1781813,"avatar_url":"https://avatars.githubusercontent.com/u/1781813?v=3","gravatar_id":"","url":"https://api.github.com/users/isaacabraham","html_url":"https://github.com/isaacabraham","followers_url":"https://api.github.com/users/isaacabraham/followers","following_url":"https://api.github.com/users/isaacabraham/following{/other_user}","gists_url":"https://api.github.com/users/isaacabraham/gists{/gist_id}","starred_url":"https://api.github.com/users/isaacabraham/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/isaacabraham/subscriptions","organizations_url":"https://api.github.com/users/isaacabraham/orgs","repos_url":"https://api.github.com/users/isaacabraham/repos","events_url":"https://api.github.com/users/isaacabraham/events{/privacy}","received_events_url":"https://api.github.com/users/isaacabraham/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-11-01T13:06:10Z","updated_at":"2015-11-02T16:50:34Z","closed_at":null,"body":"Something I've noticed from the House Price example in the starter kit is that it would be useful to be able to discard groups based on an arbitrary predicate. For example, look at the following sample: -\r\n\r\n```fsharp\r\nlet averagePricesTask =\r\n    persistedHousePrices\r\n    |> CloudFlow.averageByKey\r\n          (fun row -> (row.TownCity, row.Street))\r\n          (fun row -> float row.Price)\r\n    |> CloudFlow.cache\r\n    |> cluster.CreateProcess\r\n```\r\n\r\nThe problem here is that it brings in all groupings, even ones of 1 item. If you want more control, you need to resort to falling back to the less efficient groupBy: -\r\n\r\n```fsharp\r\nlet averagePricesTask =\r\n    persistedHousePrices\r\n    |> CloudFlow.groupBy(fun row -> row.TownCity, row.Street)\r\n    |> CloudFlow.filter(fun (_, items) -> items |> Seq.length > 5) // only bring back streets with at least 5 houses in them.\r\n    |> CloudFlow.map(fun (key, items) -> key, items |> Seq.averageBy(fun row -> float row.Price))\r\n    |> CloudFlow.cache\r\n    |> cluster.CreateProcess\r\n```\r\n\r\nIt would be good to have a generalised filter for each grouping that would be run to limit what sort of groupings are returned: -\r\n\r\n```fsharp\r\nlet averagePricesTask =\r\n    persistedHousePrices\r\n    |> CloudFlow.averageByKey\r\n          (fun row -> (row.TownCity, row.Street))\r\n          (fun row -> float row.Price)\r\n          (fun group -> group |> Seq.length > 5) // grouping filter predicate\r\n    |> CloudFlow.cache\r\n    |> cluster.CreateProcess\r\n```\r\n\r\nI have no idea if this is possible whilst retaining the efficiency of averageByKey rather than groupBy though, so happy to hear your thoughts on this @palladin @dsyme."},"comment":{"url":"https://api.github.com/repos/mbraceproject/MBrace.Core/issues/comments/153080245","html_url":"https://github.com/mbraceproject/MBrace.Core/issues/149#issuecomment-153080245","issue_url":"https://api.github.com/repos/mbraceproject/MBrace.Core/issues/149","id":153080245,"user":{"login":"isaacabraham","id":1781813,"avatar_url":"https://avatars.githubusercontent.com/u/1781813?v=3","gravatar_id":"","url":"https://api.github.com/users/isaacabraham","html_url":"https://github.com/isaacabraham","followers_url":"https://api.github.com/users/isaacabraham/followers","following_url":"https://api.github.com/users/isaacabraham/following{/other_user}","gists_url":"https://api.github.com/users/isaacabraham/gists{/gist_id}","starred_url":"https://api.github.com/users/isaacabraham/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/isaacabraham/subscriptions","organizations_url":"https://api.github.com/users/isaacabraham/orgs","repos_url":"https://api.github.com/users/isaacabraham/repos","events_url":"https://api.github.com/users/isaacabraham/events{/privacy}","received_events_url":"https://api.github.com/users/isaacabraham/received_events","type":"User","site_admin":false},"created_at":"2015-11-02T16:50:34Z","updated_at":"2015-11-02T16:50:34Z","body":"Makes sense. But there must be, somewhere, some aggregation of the partial results being done in order to calculate the average?"}}