{"action":"created","issue":{"url":"https://api.github.com/repos/vstakhov/rspamd/issues/267","labels_url":"https://api.github.com/repos/vstakhov/rspamd/issues/267/labels{/name}","comments_url":"https://api.github.com/repos/vstakhov/rspamd/issues/267/comments","events_url":"https://api.github.com/repos/vstakhov/rspamd/issues/267/events","html_url":"https://github.com/vstakhov/rspamd/issues/267","id":69784794,"number":267,"title":"fuzzy_add does not complete SQLite transactions","user":{"login":"moisseev","id":2275981,"avatar_url":"https://avatars.githubusercontent.com/u/2275981?v=3","gravatar_id":"","url":"https://api.github.com/users/moisseev","html_url":"https://github.com/moisseev","followers_url":"https://api.github.com/users/moisseev/followers","following_url":"https://api.github.com/users/moisseev/following{/other_user}","gists_url":"https://api.github.com/users/moisseev/gists{/gist_id}","starred_url":"https://api.github.com/users/moisseev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/moisseev/subscriptions","organizations_url":"https://api.github.com/users/moisseev/orgs","repos_url":"https://api.github.com/users/moisseev/repos","events_url":"https://api.github.com/users/moisseev/events{/privacy}","received_events_url":"https://api.github.com/users/moisseev/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":25,"created_at":"2015-04-21T08:44:15Z","updated_at":"2015-04-25T09:31:52Z","closed_at":null,"body":"It seems that fuzzy_add does not complete SQLite transactions.\r\n\r\nAfter executing the command:\r\n```\r\n# rspamc -f 1 -w 20 fuzzy_add fuzzy.eml\r\nResults for file: fuzzy.eml\r\nsuccess = true;\r\n```\r\nfuzzy.db-journal file appears in /var/db/rspamd directory and then disappears in about 90 seconds.\r\nChanges not commited to fuzzy.db file.\r\n```\r\n# pkg info -E rspamd sqlite3\r\nrspamd-0.8.3\r\nsqlite3-3.8.9.0\r\n# uname -v\r\nFreeBSD 10.1-RELEASE #0 r274401: Tue Nov 11 22:51:51 UTC 2014     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC\r\n```\r\nrspamd.conf:\r\n```\r\nworker {\r\n    type = \"fuzzy\";\r\n    bind_socket = \"*:11335\";\r\n    hash_file = \"${DBDIR}/fuzzy.db\"\r\n    expire = 90d;\r\n    allow_update = \"127.0.0.1\";\r\n}\r\n```\r\nmodules.conf:\r\n```\r\nfuzzy_check {\r\n    min_bytes = 300;\r\n        rule {\r\n                servers = \"localhost:11335\";\r\n                servers = \"highsecure.ru:11335\";\r\n                symbol = \"FUZZY_UNKNOWN\";\r\n                mime_types = \"application/pdf\";\r\n                max_score = 20.0;\r\n                skip_unknown = yes;\r\n                fuzzy_map = {\r\n                        FUZZY_DENIED {\r\n                                max_score = 20.0;\r\n                                flag = 1\r\n                        }\r\n                        FUZZY_PROB {\r\n                                max_score = 10.0;\r\n                                flag = 2\r\n                        }\r\n                        FUZZY_WHITE {\r\n                                max_score = 2.0;\r\n                                flag = 3\r\n                        }\r\n                }\r\n        }\r\n}\r\n```\r\n\r\n<bountysource-plugin>\r\n---\r\nWant to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/11672720-fuzzy_add-does-not-complete-sqlite-transactions?utm_campaign=plugin&utm_content=tracker%2F320726&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F320726&utm_medium=issues&utm_source=github).\r\n</bountysource-plugin>"},"comment":{"url":"https://api.github.com/repos/vstakhov/rspamd/issues/comments/96168724","html_url":"https://github.com/vstakhov/rspamd/issues/267#issuecomment-96168724","issue_url":"https://api.github.com/repos/vstakhov/rspamd/issues/267","id":96168724,"user":{"login":"moisseev","id":2275981,"avatar_url":"https://avatars.githubusercontent.com/u/2275981?v=3","gravatar_id":"","url":"https://api.github.com/users/moisseev","html_url":"https://github.com/moisseev","followers_url":"https://api.github.com/users/moisseev/followers","following_url":"https://api.github.com/users/moisseev/following{/other_user}","gists_url":"https://api.github.com/users/moisseev/gists{/gist_id}","starred_url":"https://api.github.com/users/moisseev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/moisseev/subscriptions","organizations_url":"https://api.github.com/users/moisseev/orgs","repos_url":"https://api.github.com/users/moisseev/repos","events_url":"https://api.github.com/users/moisseev/events{/privacy}","received_events_url":"https://api.github.com/users/moisseev/received_events","type":"User","site_admin":false},"created_at":"2015-04-25T09:31:52Z","updated_at":"2015-04-25T09:31:52Z","body":"I tried to change the expire to different values in the worker configuration below but no luck.\r\n```\r\nworker {\r\n    type = \"fuzzy\";\r\n    bind_socket = \"*:11335\";\r\n    hash_file = \"${DBDIR}/fuzzy.db\"\r\n    expire = 90d;\r\n    allow_update = \"127.0.0.1\";\r\n    sync = 10s;\r\n}\r\n```\r\nAlso I have installed rspamd from the master branch. It doesn't write to the storage at all.\r\n```\r\n2015-04-25 12:25:24 #36740(controller) process_image: detected GIF image of size 735 x 936 in message <1E1BADFEF52102815D5811ACA832053E@galerie-neuhaus.ru>\r\n2015-04-25 12:25:24 #36740(controller) fuzzy_generate_commands: <1E1BADFEF52102815D5811ACA832053E@galerie-neuhaus.ru>, part is shorter than 300 symbols, skip fuzzy check\r\n2015-04-25 12:25:24 #36739(fuzzy) radix_find_compressed: trying to find key\r\n2015-04-25 12:25:24 #36739(fuzzy) rspamd_fuzzy_backend_run_stmt: executing `SELECT value, time, flag FROM digests WHERE digest==?1;`\r\n2015-04-25 12:25:24 #36739(fuzzy) rspamd_fuzzy_backend_run_stmt: executing `INSERT INTO digests(flag, digest, value, time) VALUES(?1, ?2, ?3, ?4);`\r\n2015-04-25 12:25:24 #36739(fuzzy) radix_find_compressed: trying to find key\r\n2015-04-25 12:25:24 #36739(fuzzy) rspamd_fuzzy_backend_run_stmt: executing `SELECT value, time, flag FROM digests WHERE digest==?1;`\r\n2015-04-25 12:25:24 #36739(fuzzy) rspamd_fuzzy_backend_run_stmt: executing `UPDATE digests SET value = value + ?1 WHERE digest==?2;`\r\n2015-04-25 12:25:24 #36740(controller) fuzzy_learn_callback: processed fuzzy hash, list: FUZZY_DENIED:1 for message <1E1BADFEF52102815D5811ACA832053E@galerie-neuhaus.ru>\r\n2015-04-25 12:25:24 #36740(controller) rspamd_task_free: free pointer 29025108\r\n2015-04-25 12:25:24 #36740(controller) destroy_message: freeing pointer 29055900\r\n2015-04-25 12:25:34 #36739(fuzzy) rspamd_fuzzy_backend_run_stmt: executing `DELETE FROM digests WHERE time < ?1;`\r\n2015-04-25 12:25:34 #36739(fuzzy) rspamd_fuzzy_backend_run_stmt: executing `COMMIT;`\r\n2015-04-25 12:25:34 #36739(fuzzy) rspamd_fuzzy_backend_run_stmt: executing `BEGIN TRANSACTION;`\r\n2015-04-25 12:25:46 #36739(fuzzy) rspamd_fuzzy_backend_run_stmt: executing `DELETE FROM digests WHERE time < ?1;`\r\n2015-04-25 12:25:46 #36739(fuzzy) rspamd_fuzzy_backend_run_stmt: executing `COMMIT;`\r\n2015-04-25 12:25:46 #36739(fuzzy) rspamd_fuzzy_backend_run_stmt: executing `BEGIN TRANSACTION;`\r\n2015-04-25 12:26:02 #36739(fuzzy) rspamd_fuzzy_backend_run_stmt: executing `DELETE FROM digests WHERE time < ?1;`\r\n2015-04-25 12:26:02 #36739(fuzzy) rspamd_fuzzy_backend_run_stmt: executing `COMMIT;`\r\n\r\n```"}}