{"action":"created","issue":{"url":"https://api.github.com/repos/fsharp/FSharp.Data/issues/779","labels_url":"https://api.github.com/repos/fsharp/FSharp.Data/issues/779/labels{/name}","comments_url":"https://api.github.com/repos/fsharp/FSharp.Data/issues/779/comments","events_url":"https://api.github.com/repos/fsharp/FSharp.Data/issues/779/events","html_url":"https://github.com/fsharp/FSharp.Data/issues/779","id":57242184,"number":779,"title":"Csv type provider fails with OutOfMemory with large sample file","user":{"login":"evelinag","id":5541162,"avatar_url":"https://avatars.githubusercontent.com/u/5541162?v=3","gravatar_id":"","url":"https://api.github.com/users/evelinag","html_url":"https://github.com/evelinag","followers_url":"https://api.github.com/users/evelinag/followers","following_url":"https://api.github.com/users/evelinag/following{/other_user}","gists_url":"https://api.github.com/users/evelinag/gists{/gist_id}","starred_url":"https://api.github.com/users/evelinag/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/evelinag/subscriptions","organizations_url":"https://api.github.com/users/evelinag/orgs","repos_url":"https://api.github.com/users/evelinag/repos","events_url":"https://api.github.com/users/evelinag/events{/privacy}","received_events_url":"https://api.github.com/users/evelinag/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-02-10T21:44:58Z","updated_at":"2015-02-10T22:41:44Z","closed_at":null,"body":"I tried to pass a large sample file into csv type provider:\r\n```\r\ntype BigData = CsvProvider<\"MyLargeFile.csv\",CacheRows=false,InferRows=10>\r\n```\r\nand it fails with OutOfMemoryException. Even thou it shouldn't cache all the rows, it tries to read the entire file. \r\n"},"comment":{"url":"https://api.github.com/repos/fsharp/FSharp.Data/issues/comments/73800114","html_url":"https://github.com/fsharp/FSharp.Data/issues/779#issuecomment-73800114","issue_url":"https://api.github.com/repos/fsharp/FSharp.Data/issues/779","id":73800114,"user":{"login":"tpetricek","id":485413,"avatar_url":"https://avatars.githubusercontent.com/u/485413?v=3","gravatar_id":"","url":"https://api.github.com/users/tpetricek","html_url":"https://github.com/tpetricek","followers_url":"https://api.github.com/users/tpetricek/followers","following_url":"https://api.github.com/users/tpetricek/following{/other_user}","gists_url":"https://api.github.com/users/tpetricek/gists{/gist_id}","starred_url":"https://api.github.com/users/tpetricek/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tpetricek/subscriptions","organizations_url":"https://api.github.com/users/tpetricek/orgs","repos_url":"https://api.github.com/users/tpetricek/repos","events_url":"https://api.github.com/users/tpetricek/events{/privacy}","received_events_url":"https://api.github.com/users/tpetricek/received_events","type":"User","site_admin":false},"created_at":"2015-02-10T22:41:44Z","updated_at":"2015-02-10T22:41:44Z","body":"I had a quick look into this and it seems that we are calling `ReadToEnd` on a stream here:\r\n\r\n * https://github.com/fsharp/FSharp.Data/blob/363cca3eeb291931294efe3ec2c30680a10f16bf/src/CommonRuntime/IO.fs#L226\r\n\r\nI guess this file is shared with XML and JSON, where we need to read the whole thing into memory anyway, but for CSV files, we should be able to just read first X lines..."}}