{"action":"created","issue":{"url":"https://api.github.com/repos/moovweb/gokogiri/issues/36","labels_url":"https://api.github.com/repos/moovweb/gokogiri/issues/36/labels{/name}","comments_url":"https://api.github.com/repos/moovweb/gokogiri/issues/36/comments","events_url":"https://api.github.com/repos/moovweb/gokogiri/issues/36/events","html_url":"https://github.com/moovweb/gokogiri/issues/36","id":17072276,"number":36,"title":"Memory Leak","user":{"login":"JakeAustwick","id":1095574,"avatar_url":"https://avatars.githubusercontent.com/u/1095574?v=3","gravatar_id":"","url":"https://api.github.com/users/JakeAustwick","html_url":"https://github.com/JakeAustwick","followers_url":"https://api.github.com/users/JakeAustwick/followers","following_url":"https://api.github.com/users/JakeAustwick/following{/other_user}","gists_url":"https://api.github.com/users/JakeAustwick/gists{/gist_id}","starred_url":"https://api.github.com/users/JakeAustwick/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JakeAustwick/subscriptions","organizations_url":"https://api.github.com/users/JakeAustwick/orgs","repos_url":"https://api.github.com/users/JakeAustwick/repos","events_url":"https://api.github.com/users/JakeAustwick/events{/privacy}","received_events_url":"https://api.github.com/users/JakeAustwick/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":23,"created_at":"2013-07-22T21:41:24Z","updated_at":"2015-01-19T20:46:53Z","closed_at":null,"body":"Hey, I'm new to Go so please excuse me if the problem is with my code.\r\n\r\nWhen I run the following code, the memory just keeps growing and growing, as if it is being leaked somewhere. It leaks slow though, if I run it with a list of 42k urls, it slowly just keeps on climbing and climbing. You should be able to spot it with this url list: https://gist.github.com/JakeAustwick/82c9d4ce300639a4d275/raw/368c41ce6ba95f03cbc25a188dd3c07646a068b0/gistfile1.txt\r\n\r\nCan you spot what I'm doing wrong, or have I found a bug?\r\n\r\n```go\r\npackage main\r\n\r\n// import \"github.com/hoisie/redis\"\r\n// import \"code.google.com/p/go.net/html\"\r\n// import \"github.com/davecheney/profile\"\r\nimport \"github.com/moovweb/gokogiri\"\r\nimport \"github.com/moovweb/gokogiri/xml\"\r\nimport \"github.com/moovweb/gokogiri/html\"\r\nimport \"github.com/mreiferson/go-httpclient\"\r\n\r\n// import \"github.com/PuerkitoBio/goquery\"\r\nimport \"log\"\r\nimport \"time\"\r\n\r\n// import \"strconv\"\r\nimport \"errors\"\r\n\r\n// import \"net/url\"\r\n\r\nimport \"strings\"\r\n\r\n// import \"io\"\r\nimport \"io/ioutil\"\r\n\r\n// import \"bytes\"\r\n\r\nimport \"net/http\"\r\nimport \"runtime\"\r\n\r\nfunc main() {\r\n\truntime.GOMAXPROCS(runtime.NumCPU())\r\n\tWORKER_COUNT := 25\r\n\r\n\ttransport := &httpclient.Transport{\r\n\t\tConnectTimeout:        3 * time.Second,\r\n\t\tRequestTimeout:        8 * time.Second,\r\n\t\tResponseHeaderTimeout: 3 * time.Second,\r\n\t}\r\n\thttp.DefaultClient = &http.Client{Transport: transport}\r\n\t// QUEUES := []string{\"feed\"}\r\n\r\n\tjobChannel := make(chan string)\r\n\tfor id := 0; id < WORKER_COUNT; id++ {\r\n\t\tgo fetchPage(id, jobChannel)\r\n\t}\r\n\t// Not most efficient way to read file, but isn't gonna use up all\r\n\t// available RAM. This isn't the problem.\r\n\tcontent, err := ioutil.ReadFile(\"urls.txt\")\r\n\tif err != nil {\r\n\t\tlog.Panicln(\"FILE READ ERROR\")\r\n\t}\r\n\turls := strings.Split(string(content), \"\\n\")\r\n\r\n\tfor _, url := range urls {\r\n\t\tjobChannel <- url\r\n\t}\r\n\r\n\t// Stop program from ending\r\n\tc := make(chan string)\r\n\t<-c\r\n\r\n}\r\n\r\nfunc fetchPage(id int, jobChannel chan string) {\r\n\tfor url := range jobChannel {\r\n\t\tresp, err := http.Get(url)\r\n\t\tif err != nil {\r\n\t\t\tlog.Println(\"http error\")\r\n\t\t\tcontinue\r\n\t\t}\r\n\t\tbody, err := ioutil.ReadAll(resp.Body)\r\n\t\tif err != nil {\r\n\t\t\tlog.Println(\"read error\")\r\n\t\t\tcontinue\r\n\t\t}\r\n\t\tresp.Body.Close()\r\n\t\tdoc, err := gokogiri.ParseHtml(body)\r\n\t\tif err != nil {\r\n\t\t\tlog.Println(\"parse error\")\r\n\t\t}\r\n\r\n\t\tform, err := BiggestForm(doc)\r\n\t\tif err != nil {\r\n\t\t\tlog.Println(err)\r\n\t\t} else {\r\n\t\t\tlog.Println(form.Name())\r\n\t\t}\r\n\t\tdoc.Free()\r\n\t}\r\n}\r\n\r\nfunc PostForms(doc *html.HtmlDocument) []xml.Node {\r\n\tvar forms []xml.Node\r\n\tfoundForms, err := doc.Search(\"//form\")\r\n\tif err != nil {\r\n\t\treturn forms\r\n\t}\r\n\treturn foundForms\r\n}\r\n\r\nfunc BiggestForm(doc *html.HtmlDocument) (xml.Node, error) {\r\n\tforms := PostForms(doc)\r\n\tif len(forms) > 0 {\r\n\t\thigh_input_count := 0\r\n\t\thigh_index := 0\r\n\t\tfor i, f := range forms {\r\n\t\t\tinputs, _ := f.Search(\".//descendant::input\")\r\n\t\t\ttextareas, _ := f.Search(\".//descendant::textarea\")\r\n\t\t\tfieldCount := len(inputs) + len(textareas)\r\n\t\t\tif fieldCount > high_input_count {\r\n\t\t\t\thigh_input_count = fieldCount\r\n\t\t\t\thigh_index = i\r\n\t\t\t}\r\n\t\t}\r\n\t\treturn forms[high_index], nil\r\n\t} else {\r\n\t\tvar f xml.Node\r\n\t\treturn f, errors.New(\"No form on the Page\")\r\n\t}\r\n}\r\n```\r\n\r\nOnce again I apologise if it is me doing something wrong."},"comment":{"url":"https://api.github.com/repos/moovweb/gokogiri/issues/comments/70557776","html_url":"https://github.com/moovweb/gokogiri/issues/36#issuecomment-70557776","issue_url":"https://api.github.com/repos/moovweb/gokogiri/issues/36","id":70557776,"user":{"login":"weil","id":1095580,"avatar_url":"https://avatars.githubusercontent.com/u/1095580?v=3","gravatar_id":"","url":"https://api.github.com/users/weil","html_url":"https://github.com/weil","followers_url":"https://api.github.com/users/weil/followers","following_url":"https://api.github.com/users/weil/following{/other_user}","gists_url":"https://api.github.com/users/weil/gists{/gist_id}","starred_url":"https://api.github.com/users/weil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/weil/subscriptions","organizations_url":"https://api.github.com/users/weil/orgs","repos_url":"https://api.github.com/users/weil/repos","events_url":"https://api.github.com/users/weil/events{/privacy}","received_events_url":"https://api.github.com/users/weil/received_events","type":"User","site_admin":false},"created_at":"2015-01-19T20:46:53Z","updated_at":"2015-01-19T20:46:53Z","body":"@akhleung  @JakeAustwick \r\nI just tried to following code, parsing the same url again and again.  Memory usage keep going up to a few GBs within a minute.\r\n\r\n\r\n\turl = \"http://www.bestbuy.com/site/withings-pulse-o2-tracker-black/5686019.p?id=1219148342760&skuId=5686019\" \r\n\t\r\n\tresp, _ := http.Get(url)\r\n\tpage, _ := ioutil.ReadAll(resp.Body)\r\n\r\n\tfor {\r\n\t\tgokogiri.ParseHtml(page)\r\n\t\truntime.GC()\r\n\t}"}}