{"action":"opened","issue":{"url":"https://api.github.com/repos/axboe/fio/issues/50","labels_url":"https://api.github.com/repos/axboe/fio/issues/50/labels{/name}","comments_url":"https://api.github.com/repos/axboe/fio/issues/50/comments","events_url":"https://api.github.com/repos/axboe/fio/issues/50/events","html_url":"https://github.com/axboe/fio/issues/50","id":60598681,"number":50,"title":"Strong evidence suggesting there is a memory leak while using the --verify parameter.","user":{"login":"rustyscottweber","id":4672283,"avatar_url":"https://avatars.githubusercontent.com/u/4672283?v=3","gravatar_id":"","url":"https://api.github.com/users/rustyscottweber","html_url":"https://github.com/rustyscottweber","followers_url":"https://api.github.com/users/rustyscottweber/followers","following_url":"https://api.github.com/users/rustyscottweber/following{/other_user}","gists_url":"https://api.github.com/users/rustyscottweber/gists{/gist_id}","starred_url":"https://api.github.com/users/rustyscottweber/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rustyscottweber/subscriptions","organizations_url":"https://api.github.com/users/rustyscottweber/orgs","repos_url":"https://api.github.com/users/rustyscottweber/repos","events_url":"https://api.github.com/users/rustyscottweber/events{/privacy}","received_events_url":"https://api.github.com/users/rustyscottweber/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-03-11T01:55:58Z","updated_at":"2015-03-11T01:55:58Z","closed_at":null,"body":"While comparing the memory consumption of the following command on a stock RHEL 6.5 machine over time to the equivalent without the --verify parameter, there is a considerable difference in memory consumption over time. Attempting different verification methods did not seem to  change the rate of memory leakage. crc32c crc64 md5 sha1... etc all had the same effect. while lowering the iodepth slowed the effect which also lowered the maximum number of iops being processed.\r\n<pre><code>\r\n./fio --ioengine=libaio --direct=1 --rw=randrw --iodepth=256 --time_based --bs=4k --runtime 24h --verify md5 --verify_fatal=1 --name=output0 --filename=/dev/mapper/mpathdof --name=output1 --filename=/dev/mapper/mpathdog --name=output2 --filename=/dev/mapper/mpathdoh --name=output3 --filename=/dev/mapper/mpathdod --name=output4 --filename=/dev/mapper/mpathdoe --name=output5 --filename=/dev/mapper/mpathdoa --name=output6 --filename=/dev/mapper/mpathdoc --name=output7 --filename=/dev/mapper/mpathdnw --name=output8 --filename=/dev/mapper/mpathdob --name=output9 --filename=/dev/mapper/mpathdnz --name=output10 --filename=/dev/mapper/mpathdny --name=output11 --filename=/dev/mapper/mpathdnx --name=output12 --filename=/dev/mapper/mpathdnv --name=output13 --filename=/dev/mapper/mpathdnt --name=output14 --filename=/dev/mapper/mpathdnu --name=output15 --filename=/dev/mapper/mpathdnq --name=output16 --filename=/dev/mapper/mpathdns --name=output17 --filename=/dev/mapper/mpathdnr --name=output18 --filename=/dev/mapper/mpathdnp --name=output19 --filename=/dev/mapper/mpathdno --name=output20 --filename=/dev/mapper/mpathdnk --name=output21 --filename=/dev/mapper/mpathdnl --name=output22 --filename=/dev/mapper/mpathdnn --name=output23 --filename=/dev/mapper/mpathdnm --name=output24 --filename=/dev/mapper/mpathdnj --name=output25 --filename=/dev/mapper/mpathdnf --name=output26 --filename=/dev/mapper/mpathdne --name=output27 --filename=/dev/mapper/mpathdni --name=output28 --filename=/dev/mapper/mpathdnh --name=output29 --filename=/dev/mapper/mpathdnd --name=output30 --filename=/dev/mapper/mpathdnc --name=output31 --filename=/dev/mapper/mpathdng\r\n</code></pre>\r\n\r\nAfter only 5 minutes of running with the --verify parameter, a considerable amount of memory has been used.  \r\n\r\n<pre><code>top - 18:49:11 up 45 min,  2 users,  load average: 36.92, 33.17, 20.32\r\nTasks: 454 total,  19 running, 435 sleeping,   0 stopped,   0 zombie\r\nCpu(s): 22.9%us, 32.8%sy,  0.0%ni,  0.2%id, 28.0%wa,  0.0%hi, 16.1%si,  0.0%st\r\nMem:  24599096k total, 12130100k used, 12468996k free,   162512k buffers\r\nSwap: 12369912k total,        0k used, 12369912k free,   922372k cached\r\n\r\n  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND\r\n25540 root      20   0  753m 334m  552 D 21.6  1.4   2:13.45 fio\r\n25555 root      20   0  751m 333m  552 R 21.2  1.4   2:13.75 fio\r\n25535 root      20   0  751m 333m  552 D 21.2  1.4   2:12.22 fio\r\n25537 root      20   0  751m 333m  548 R 20.9  1.4   2:13.57 fio\r\n25559 root      20   0  751m 333m  552 D 20.9  1.4   2:13.15 fio\r\n25557 root      20   0  751m 333m  548 D 21.6  1.4   2:13.48 fio\r\n25552 root      20   0  751m 332m  552 D 21.6  1.4   2:13.43 fio\r\n25538 root      20   0  751m 332m  544 D 21.2  1.4   2:12.90 fio\r\n25545 root      20   0  750m 332m  552 R 21.9  1.4   2:12.86 fio\r\n25520 root      20   0  750m 332m  552 R 20.6  1.4   2:11.30 fio\r\n25539 root      20   0  749m 331m  552 R 21.2  1.4   2:12.60 fio\r\n25546 root      20   0  748m 330m  544 D 21.9  1.4   2:12.76 fio\r\n25550 root      20   0  748m 330m  548 D 21.6  1.4   2:11.86 fio\r\n25548 root      20   0  742m 324m  552 D 21.6  1.4   2:09.51 fio\r\n25541 root      20   0  740m 322m  548 D 22.2  1.3   2:09.55 fio\r\n25554 root      20   0  740m 322m  544 D 21.6  1.3   2:08.30 fio\r\n25549 root      20   0  740m 321m  544 D 21.2  1.3   2:07.98 fio\r\n25556 root      20   0  739m 321m  548 R 21.2  1.3   2:07.79 fio\r\n25532 root      20   0  739m 321m  552 D 20.2  1.3   2:07.09 fio\r\n25543 root      20   0  739m 321m  556 R 20.9  1.3   2:08.99 fio\r\n25544 root      20   0  739m 320m  552 D 20.6  1.3   2:08.26 fio\r\n25558 root      20   0  738m 320m  544 D 21.6  1.3   2:08.07 fio\r\n25534 root      20   0  737m 319m  548 R 20.9  1.3   2:06.16 fio\r\n25562 root      20   0  737m 319m  548 D 21.9  1.3   2:07.70 fio\r\n25547 root      20   0  722m 303m  560 D 19.6  1.3   2:00.81 fio\r\n25560 root      20   0  721m 303m  548 D 19.9  1.3   1:59.64 fio\r\n25553 root      20   0  718m 300m  548 D 19.2  1.3   1:58.87 fio\r\n25533 root      20   0  711m 293m  544 D 19.6  1.2   1:54.97 fio\r\n25561 root      20   0  711m 293m  548 D 19.2  1.2   1:56.98 fio\r\n25536 root      20   0  711m 292m  552 D 19.6  1.2   1:55.96 fio\r\n25551 root      20   0  710m 292m  556 D 18.9  1.2   1:56.46 fio\r\n25542 root      20   0  708m 290m  548 R 18.9  1.2   1:55.94 fio\r\n24307 root      20   0  377m 172m 171m S  0.7  0.7   0:04.95 fio</code></pre>   If left unchecked, the memory leak will consume all of the system's ram until the kernel starts killing the fio job because there is no memory left on the system."}}