{"action":"created","issue":{"url":"https://api.github.com/repos/solomonxie/solomonxie.github.io/issues/8","labels_url":"https://api.github.com/repos/solomonxie/solomonxie.github.io/issues/8/labels{/name}","comments_url":"https://api.github.com/repos/solomonxie/solomonxie.github.io/issues/8/comments","events_url":"https://api.github.com/repos/solomonxie/solomonxie.github.io/issues/8/events","html_url":"https://github.com/solomonxie/solomonxie.github.io/issues/8","id":120586965,"number":8,"title":"Python 文件读取的练习","user":{"login":"solomonxie","id":14041622,"avatar_url":"https://avatars.githubusercontent.com/u/14041622?v=3","gravatar_id":"","url":"https://api.github.com/users/solomonxie","html_url":"https://github.com/solomonxie","followers_url":"https://api.github.com/users/solomonxie/followers","following_url":"https://api.github.com/users/solomonxie/following{/other_user}","gists_url":"https://api.github.com/users/solomonxie/gists{/gist_id}","starred_url":"https://api.github.com/users/solomonxie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/solomonxie/subscriptions","organizations_url":"https://api.github.com/users/solomonxie/orgs","repos_url":"https://api.github.com/users/solomonxie/repos","events_url":"https://api.github.com/users/solomonxie/events{/privacy}","received_events_url":"https://api.github.com/users/solomonxie/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-12-05T20:59:32Z","updated_at":"2015-12-05T21:00:47Z","closed_at":null,"body":"> 为了了解txt的读取速度，每一段都加上了时间计算\r\n\r\n# 先练习了下基本的txt文件读写\r\n```python\r\nstart = time.clock()\r\nf = open('hello.txt', 'r+')\r\nf2 = open('hello2.txt', 'w')\r\nf2.write(f.read())\r\nf2.close\r\nf.close()\r\nend = time.clock()\r\nprint str(end - start)\r\n```\r\n# 然后是将14M的txt文件逐行复制到新文件的速度\r\n```python\r\nstart = time.clock()\r\nf = open('hello.txt', 'r+') # 14M的txt文件\r\nf2 = open('hello2.txt', 'w') # 逐行复制到新文件里\r\nn = 0\r\nwhile True:\r\n    line = f.readline()\r\n    n += 1\r\n    if line:\r\n        f2.write(line)\r\n        #print '现在正在写入第 %s 行数据。' % n\r\n    else:\r\n        break\r\nf2.close\r\nf.close()\r\nend = time.clock()\r\nprint str(end - start)\r\n```\r\n\r\n# 接着是生成1.4G的txt文件\r\n```python\r\nstart = time.clock()\r\nf = open('hello.txt', 'r') # 原文件14M\r\nf2 = open('hello-1g.txt','a')\r\ncontent = f.read()\r\nf.close()\r\n# 想要生成1.4G的txt文件\r\n# 用来测试各种方式读写大文件的速度\r\nfor i in range(1000):\r\n    f2.write(content)\r\nf2.close()\r\nend = time.clock()\r\nprint str(end-start) # 用了446秒，无所谓了，主要是弄出一个1G的文件\r\n```\r\n\r\n# 用readline()来逐行读取1G文件的速度\r\n```python\r\nstart = time.clock()\r\nf = open('hello-1g.txt', 'r')\r\nprint f.readline().decode('gb2312')\r\nprint f.readline().decode('gb2312')\r\nprint f.readline().decode('gb2312')\r\nprint f.readline().decode('gb2312')\r\nprint f.readline().decode('gb2312')\r\nf.close()\r\nend = time.clock()\r\nprint str(end-start) #用了0.08秒\r\n```\r\n时间是一瞬间吧，这样就看出来，其实不管txt文件多大，\r\n只要用readline()一行一行读，就不占内存，也就是不影响速度。\r\n\r\n# 用read()和来读取1G文件的速度\r\n```python\r\nstart = time.clock()\r\nf = open('hello-1g.txt', 'r')\r\ncontent = f.read() # print出1G是不可能的，直接赋予变量吧\r\nf.close()\r\nend = time.clock()\r\nprint str(end-start) \r\n```\r\n呃，把1G的内容赋予变量——也就是一个变量在内存中占据1G以上的地盘，直接报错，如下图：\r\n![baidushurufa_2015-12-6_4-55-26](https://cloud.githubusercontent.com/assets/14041622/11610187/fcf0cde6-9bd5-11e5-89d2-a4a2a09fe849.png)\r\n\r\n哈哈 ，这样一来，大概就明白了。\r\n几十M的小文件的话，read()和readline()哪个都无所谓了，方便就行。\r\n大文件的话，绝对不能read()直接读取。"},"comment":{"url":"https://api.github.com/repos/solomonxie/solomonxie.github.io/issues/comments/162245386","html_url":"https://github.com/solomonxie/solomonxie.github.io/issues/8#issuecomment-162245386","issue_url":"https://api.github.com/repos/solomonxie/solomonxie.github.io/issues/8","id":162245386,"user":{"login":"solomonxie","id":14041622,"avatar_url":"https://avatars.githubusercontent.com/u/14041622?v=3","gravatar_id":"","url":"https://api.github.com/users/solomonxie","html_url":"https://github.com/solomonxie","followers_url":"https://api.github.com/users/solomonxie/followers","following_url":"https://api.github.com/users/solomonxie/following{/other_user}","gists_url":"https://api.github.com/users/solomonxie/gists{/gist_id}","starred_url":"https://api.github.com/users/solomonxie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/solomonxie/subscriptions","organizations_url":"https://api.github.com/users/solomonxie/orgs","repos_url":"https://api.github.com/users/solomonxie/repos","events_url":"https://api.github.com/users/solomonxie/events{/privacy}","received_events_url":"https://api.github.com/users/solomonxie/received_events","type":"User","site_admin":false},"created_at":"2015-12-05T21:00:47Z","updated_at":"2015-12-05T21:00:47Z","body":"# 源代码如下：\r\n\r\n```python\r\n# coding:utf-8\r\nimport time\r\n##################################\r\n\"\"\"\r\n# 用read()和来读取1G文件的速度\r\nstart = time.clock()\r\nf = open('hello-1g.txt', 'r')\r\n#因为内容太长了，print是看不出处理速度的\r\n#所以直接赋予变量了\r\ncontent = f.read()\r\nf.close()\r\nend = time.clock()\r\nprint str(end-start)\r\n\"\"\"\r\n##################################\r\n#\"\"\"\r\n# 用readline()和来读取1G文件的速度\r\nstart = time.clock()\r\nf = open('hello-1g.txt', 'r')\r\nprint f.readline().decode('gb2312')\r\nprint f.readline().decode('gb2312')\r\nprint f.readline().decode('gb2312')\r\nprint f.readline().decode('gb2312')\r\nprint f.readline().decode('gb2312')\r\nf.close()\r\nend = time.clock()\r\nprint str(end-start) #用了0.08秒\r\n#\"\"\"\r\n##################################\r\n\"\"\"\r\nstart = time.clock()\r\nf = open('hello.txt', 'r') # 原文件14M\r\nf2 = open('hello-1g.txt','a')\r\ncontent = f.read()\r\nf.close()\r\n# 想要生成1.4G的txt文件\r\n# 用来测试各种方式读写大文件的速度\r\nfor i in range(1000):\r\n    f2.write(content)\r\nf2.close()\r\nend = time.clock()\r\nprint str(end-start) # 用了446秒\r\n\"\"\"\r\n##################################\r\n\"\"\"\r\nstart = time.clock()\r\nf = open('hello.txt', 'r+')\r\nf2 = open('hello2.txt', 'w')\r\nn = 0\r\nwhile True:\r\n    line = f.readline()\r\n    n += 1\r\n    if line:\r\n        f2.write(line)\r\n        #print '现在正在写入第 %s 行数据。' % n\r\n    else:\r\n        break\r\nf2.close\r\nf.close()\r\nend = time.clock()\r\nprint str(end - start)\r\n\"\"\"\r\n##################################\r\n\"\"\"\r\nstart = time.clock()\r\nf = open('hello.txt', 'r+')\r\nf2 = open('hello2.txt', 'w')\r\nf2.write(f.read())\r\nf2.close\r\nf.close()\r\nend = time.clock()\r\nprint str(end - start)\r\n\"\"\"\r\n\r\n```"}}