{"action":"created","issue":{"url":"https://api.github.com/repos/SpiderLabs/owasp-modsecurity-crs/issues/220","labels_url":"https://api.github.com/repos/SpiderLabs/owasp-modsecurity-crs/issues/220/labels{/name}","comments_url":"https://api.github.com/repos/SpiderLabs/owasp-modsecurity-crs/issues/220/comments","events_url":"https://api.github.com/repos/SpiderLabs/owasp-modsecurity-crs/issues/220/events","html_url":"https://github.com/SpiderLabs/owasp-modsecurity-crs/issues/220","id":64490951,"number":220,"title":"And again about the non-English letters","user":{"login":"shrv","id":11438507,"avatar_url":"https://avatars.githubusercontent.com/u/11438507?v=3","gravatar_id":"","url":"https://api.github.com/users/shrv","html_url":"https://github.com/shrv","followers_url":"https://api.github.com/users/shrv/followers","following_url":"https://api.github.com/users/shrv/following{/other_user}","gists_url":"https://api.github.com/users/shrv/gists{/gist_id}","starred_url":"https://api.github.com/users/shrv/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shrv/subscriptions","organizations_url":"https://api.github.com/users/shrv/orgs","repos_url":"https://api.github.com/users/shrv/repos","events_url":"https://api.github.com/users/shrv/events{/privacy}","received_events_url":"https://api.github.com/users/shrv/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-03-26T10:28:45Z","updated_at":"2015-03-26T12:59:41Z","closed_at":null,"body":"Suffered from a branch modseсurity. Question closer to this thread.\r\n\r\nGood afternoon.\r\nI apologize in advance for my bad English.\r\nWe use ModSecurity as WAF to our web resources. The version currently 2.7.0.\r\n\r\nCharset of our pages utf-8. Since Russian resources, then the characters used in the URL or in the body of the query, refers to both the English and to the Russian alphabet.\r\n\r\nWhat would be clear, with a real example of a query.\r\nConsider for example the rules with ID 981173\r\nSecRule ARGS_NAMES|ARGS|XML:/* \"([~!\\@#\\$\\%^&*()-+\\={}[]|:\\;\\\"\\'\\´\\’\\‘`<>].*?){4,}\" \"phase:2,t:none,t:urlDecodeUni,block,id:'981173',rev:'2',ver:'OWASP_CRS/2.2.9',maturity:'9',accuracy:'8',msg:'Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded',capture,logdata:'Matched Data: %{TX.1} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',tag:'OWASP_CRS/WEB_ATTACK/SQL_INJECTION',setvar:tx.anomaly_score=+%{tx.warning_anomaly_score},setvar:tx.sql_injection_score=+1,setvar:'tx.msg=%{rule.msg}',setvar:tx.%{rule.id}-OWASP_CRS/WEB_ATTACK/RESTRICTED_SQLI_CHARS-%{matched_var_name}=%{tx.0}\"\r\n\r\nDuring experimentation, I broke it into three separate rules\r\n\r\nSecRule ARGS_NAMES|ARGS|XML:/* \"([~!\\@#\\$\\%^&*()-+\\={}[]|:\\;\\\"\\'`<>].*?){4,}\" \"phase:2,t:none,t:urlDecodeUni,block,id:'901201',rev:'2',ver:'OWASP_CRS/2.2.9',maturity:'9',accuracy:'8',msg:'Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded',capture,logdata:'Matched Data: %{TX.1} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',tag:'OWASP_CRS/WEB_ATTACK/SQL_INJECTION',setvar:tx.anomaly_score=+%{tx.warning_anomaly_score},setvar:tx.sql_injection_score=+1,setvar:'tx.msg=%{rule.msg}',setvar:tx.%{rule.id}-OWASP_CRS/WEB_ATTACK/RESTRICTED_SQLI_CHARS-%{matched_var_name}=%{tx.0}\"\r\n\r\nSecRule ARGS_NAMES|ARGS|XML:/* \"(\\´.*?){2,}\" \"phase:2,t:none,t:urlDecodeUni,block,id:'901202',rev:'2',ver:'OWASP_CRS/2.2.9',maturity:'9',accuracy:'8',msg:'Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded',capture,logdata:'Matched Data: %{TX.1} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',tag:'OWASP_CRS/WEB_ATTACK/SQL_INJECTION',setvar:tx.anomaly_score=+%{tx.warning_anomaly_score},setvar:tx.sql_injection_score=+1,setvar:'tx.msg=%{rule.msg}',setvar:tx.%{rule.id}-OWASP_CRS/WEB_ATTACK/RESTRICTED_SQLI_CHARS-%{matched_var_name}=%{tx.0}\"\r\n\r\nSecRule ARGS_NAMES|ARGS|XML:/* \"([\\’\\‘].*?){2,}\" \"phase:2,t:none,t:urlDecodeUni,block,id:'901203',rev:'2',ver:'OWASP_CRS/2.2.9',maturity:'9',accuracy:'8',msg:'Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded',capture,logdata:'Matched Data: %{TX.1} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',tag:'OWASP_CRS/WEB_ATTACK/SQL_INJECTION',setvar:tx.anomaly_score=+%{tx.warning_anomaly_score},setvar:tx.sql_injection_score=+1,setvar:'tx.msg=%{rule.msg}',setvar:tx.%{rule.id}-OWASP_CRS/WEB_ATTACK/RESTRICTED_SQLI_CHARS-%{matched_var_name}=%{tx.0}\"\r\n\r\nDivided into three rules based on the simple principle. The first rule in decoding all symbols are of the form \\ххх, for example ~ is \\x7E. In the second rule \\xxx\\xxx , for example ´ is \\xC2\\xB4. In a third \\xxx\\xxx\\xxx, for example ’ is \\xE2\\x80\\x99.\r\n\r\nNow take the query, we are interested only body \r\n2_1=0&2_2=0&5_1_r=single&2_3=предприятиях по добыче и переработке.\r\nпредприятиях по добыче и переработке - this text contains only Russian letters. Any letter of the Russian alphabet has the form \\xxx\\xxx. Example р is \\xD1\\x80.\r\n\r\nAnd see the following logs (worked the third rule)\r\nID Rule 901203\r\nRule Message: Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded\r\nEvent: Pattern match \"([\\\\xe2\\x80\\x99\\\\xe2\\x80\\x98].*?){2,}\" at ARGS:2_3 \r\nData: Matched Data: \\x80 found within ARGS:2_3: \\xd0\\xbf\\xd1\\x80\\xd0\\xb5\\xd0\\xb4\\xd0\\xbf\\xd1\\x80\\xd0\\xb8\\xd1\\x8f\\xd1\\x82\\xd0\\xb8\\xd1\\x8f\\xd1\\x85 \\xd0\\xbf\\xd0\\xbe \\xd0\\xb4\\xd0\\xbe\\xd0\\xb1\\xd1\\x8b\\xd1\\x87\\xd0\\xb5 \\xd0\\xb8 \\xd0\\xbf\\xd0\\xb5\\xd1\\x80\\xd0\\xb5\\xd1\\x8\r\n\r\nReceived a false alarm. We see that the rule, instead of what would as a pattern for comparison use \\xE2\\x80\\x99, uses \\xE2, \\x80 as individual characters. It seems to me, is not correct.\r\n\r\nMaybe it makes sense to consider the following scenario, I think it will reduce the number of false positives. Similar rules are divided into several. As described above. Introduces a new transformation functions t:length1, length2, etc. describes the length of the pattern with which to compare the values of the arguments, arguments? We also need to find ', which is exactly \\ xE2 \\ x80 \\ x99, but not \\ x80.\r\n\r\nFurther, the text of which seek coincidence divided into tighter length as the pattern, and each piece is compared with it so long as the length of the pattern pieces of equal length. For those from the original text rejected the first \\ xxx element and then divided into pieces of equal length of the pattern. Repeat the comparison. Again reject \\ xxx element. And so long as the length of the remaining piece of the original text will not be less than the length of the pattern.\r\n\r\nOr maybe easier and the problem is not that it is not right to compare but not in the correct settings do I have?"},"comment":{"url":"https://api.github.com/repos/SpiderLabs/owasp-modsecurity-crs/issues/comments/86504025","html_url":"https://github.com/SpiderLabs/owasp-modsecurity-crs/issues/220#issuecomment-86504025","issue_url":"https://api.github.com/repos/SpiderLabs/owasp-modsecurity-crs/issues/220","id":86504025,"user":{"login":"shrv","id":11438507,"avatar_url":"https://avatars.githubusercontent.com/u/11438507?v=3","gravatar_id":"","url":"https://api.github.com/users/shrv","html_url":"https://github.com/shrv","followers_url":"https://api.github.com/users/shrv/followers","following_url":"https://api.github.com/users/shrv/following{/other_user}","gists_url":"https://api.github.com/users/shrv/gists{/gist_id}","starred_url":"https://api.github.com/users/shrv/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shrv/subscriptions","organizations_url":"https://api.github.com/users/shrv/orgs","repos_url":"https://api.github.com/users/shrv/repos","events_url":"https://api.github.com/users/shrv/events{/privacy}","received_events_url":"https://api.github.com/users/shrv/received_events","type":"User","site_admin":false},"created_at":"2015-03-26T12:59:41Z","updated_at":"2015-03-26T12:59:41Z","body":"Yes, the month of April will be more accurate to use the current version 2.7.7. And maybe longer. In the near future I translate protection on two resources in the lock mode. And be updated to the latest version will be only after checking it on a test server.\r\n\r\nThank you for the correction of information in the rules. when I read the summary of the changes in the latest version, see this"}}