{"action":"created","issue":{"url":"https://api.github.com/repos/andredumas/techan.js/issues/38","labels_url":"https://api.github.com/repos/andredumas/techan.js/issues/38/labels{/name}","comments_url":"https://api.github.com/repos/andredumas/techan.js/issues/38/comments","events_url":"https://api.github.com/repos/andredumas/techan.js/issues/38/events","html_url":"https://github.com/andredumas/techan.js/issues/38","id":52824259,"number":38,"title":"Speed up techan.js for large (>10k) datasets","user":{"login":"GodRtio","id":10273455,"avatar_url":"https://avatars.githubusercontent.com/u/10273455?v=3","gravatar_id":"","url":"https://api.github.com/users/GodRtio","html_url":"https://github.com/GodRtio","followers_url":"https://api.github.com/users/GodRtio/followers","following_url":"https://api.github.com/users/GodRtio/following{/other_user}","gists_url":"https://api.github.com/users/GodRtio/gists{/gist_id}","starred_url":"https://api.github.com/users/GodRtio/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GodRtio/subscriptions","organizations_url":"https://api.github.com/users/GodRtio/orgs","repos_url":"https://api.github.com/users/GodRtio/repos","events_url":"https://api.github.com/users/GodRtio/events{/privacy}","received_events_url":"https://api.github.com/users/GodRtio/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/andredumas/techan.js/labels/api_change","name":"api_change","color":"e11d21"},{"url":"https://api.github.com/repos/andredumas/techan.js/labels/css_change","name":"css_change","color":"e11d21"},{"url":"https://api.github.com/repos/andredumas/techan.js/labels/feature","name":"feature","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2014-12-24T15:25:25Z","updated_at":"2015-01-06T13:20:16Z","closed_at":null,"body":"Hi @andredumas \r\n\r\ntechan.js can't handle big dataset. techan.js can load ok with < 10K data.\r\nBut my browser freezed when load more than that. Is is because of D3 or SVG?\r\n\r\nI do some research and find http://square.github.io/crossfilter/ can load  231,083 dataset ok.\r\nand here proposed an solution that use css transforms: https://github.com/masayuki0812/c3/issues/774\r\n\r\nPlease let me know where i should look at the code to help?\r\n\r\nMerry Christmas @andredumas \r\n\r\nGodRtio\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/andredumas/techan.js/issues/comments/68863921","html_url":"https://github.com/andredumas/techan.js/issues/38#issuecomment-68863921","issue_url":"https://api.github.com/repos/andredumas/techan.js/issues/38","id":68863921,"user":{"login":"GodRtio","id":10273455,"avatar_url":"https://avatars.githubusercontent.com/u/10273455?v=3","gravatar_id":"","url":"https://api.github.com/users/GodRtio","html_url":"https://github.com/GodRtio","followers_url":"https://api.github.com/users/GodRtio/followers","following_url":"https://api.github.com/users/GodRtio/following{/other_user}","gists_url":"https://api.github.com/users/GodRtio/gists{/gist_id}","starred_url":"https://api.github.com/users/GodRtio/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GodRtio/subscriptions","organizations_url":"https://api.github.com/users/GodRtio/orgs","repos_url":"https://api.github.com/users/GodRtio/repos","events_url":"https://api.github.com/users/GodRtio/events{/privacy}","received_events_url":"https://api.github.com/users/GodRtio/received_events","type":"User","site_admin":false},"created_at":"2015-01-06T13:20:16Z","updated_at":"2015-01-06T13:20:16Z","body":"Hi @ andredumas \r\n\r\nI can't wait to see the next version.\r\n\r\nI have coded a chart library using canvas, it is really fast.\r\nBut when add more layer, the more complicated the code and harder to debug. :(\r\nSo i will wait for you.\r\n\r\nBTW, i have coded a convert timeframe function, i can group small timeframe to bigger.\r\nHere it is:\r\n\r\nTime = []\r\nOpen = []\r\nHigh = []\r\nLow = []\r\nClose = []\r\nVolume = []\r\n\r\n//\r\n// base TF of current Data,\r\n// 5 = M5\r\n// tf = number of minute\r\n// 15 = 15\r\n// 1 hour = 60\r\n// 4 hour = 240\r\n// ...\r\n// convertTF(5, 60) will group 5 mins into 1 hour data\r\nfunction convertTF(base, tf) {\r\n    var tfOpen = [],\r\n        tfClose = [],\r\n        tfHigh = [],\r\n        tfLow = [],\r\n        tfVol = [],\r\n        tfTime = [];\r\n\r\n    var div = tf / base;\r\n    var vol;\r\n    var min, max;\r\n    var i, j;\r\n    var end = Time.length;\r\n\r\n    // process lastest unfinished? data\r\n    for (i = end - div; i < end; i++) {\r\n        if (Time[i].getMinutes() * Time[i].getHours() % tf == 0) {\r\n            tfTime.push(Time[i]);\r\n            tfOpen.push(Open[i]);\r\n            tfClose.push(Close[end - 1])\r\n\r\n            vol = Volume[i];\r\n            min = Low[i];\r\n            max = High[i];\r\n\r\n            for (j = i + 1; j < end; j++) {\r\n                if (Low[j] < min) {\r\n                    min = Low[j];\r\n                }\r\n                if (High[j] > max) {\r\n                    max = High[j];\r\n                }\r\n\r\n                vol += Volume[j];\r\n            }\r\n\r\n            tfHigh.push(max);\r\n            tfLow.push(min);\r\n            tfVol.push(vol);\r\n\r\n            break;\r\n        }\r\n    }\r\n\r\n    // process data\r\n    while (i - div > 0) {\r\n        i -= div;\r\n\r\n        tfTime.push(Time[i]);\r\n        tfOpen.push(Open[i]);\r\n        tfClose.push(Close[i + div - 1]);\r\n\r\n        vol = Volume[i];\r\n        min = Low[i];\r\n        max = High[i];\r\n\r\n        for (j = 1; j < div; j++) {\r\n\r\n            if (Low[i + j] < min) {\r\n                min = Low[i + j];\r\n            }\r\n            if (High[i + j] > max) {\r\n                max = High[i + j];\r\n            }\r\n\r\n            vol += Volume[i + j];\r\n        }\r\n\r\n        tfHigh.push(max);\r\n        tfLow.push(min);\r\n        tfVol.push(vol);\r\n    }\r\n\r\n    // reverse data since we loop from end to start\r\n    tfTime.reverse();\r\n    tfOpen.reverse();\r\n    tfHigh.reverse();\r\n    tfLow.reverse();\r\n    tfClose.reverse();\r\n    tfVol.reverse();\r\n\r\n    return {\r\n        Time: tfTime,\r\n        Open: tfOpen,\r\n        High: tfHigh,\r\n        Low: tfLow,\r\n        Close: tfClose,\r\n        Volume: tfVol\r\n    };\r\n}\r\n\r\nHave a nice day, @andredumas \r\n\r\nGodRtio"}}