{"action":"created","issue":{"url":"https://api.github.com/repos/auraphp/Aura.Filter/issues/87","labels_url":"https://api.github.com/repos/auraphp/Aura.Filter/issues/87/labels{/name}","comments_url":"https://api.github.com/repos/auraphp/Aura.Filter/issues/87/comments","events_url":"https://api.github.com/repos/auraphp/Aura.Filter/issues/87/events","html_url":"https://github.com/auraphp/Aura.Filter/issues/87","id":95551200,"number":87,"title":"Multibyte strings, various problems with the lack thereof","user":{"login":"mivanov93","id":12671643,"avatar_url":"https://avatars.githubusercontent.com/u/12671643?v=3","gravatar_id":"","url":"https://api.github.com/users/mivanov93","html_url":"https://github.com/mivanov93","followers_url":"https://api.github.com/users/mivanov93/followers","following_url":"https://api.github.com/users/mivanov93/following{/other_user}","gists_url":"https://api.github.com/users/mivanov93/gists{/gist_id}","starred_url":"https://api.github.com/users/mivanov93/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mivanov93/subscriptions","organizations_url":"https://api.github.com/users/mivanov93/orgs","repos_url":"https://api.github.com/users/mivanov93/repos","events_url":"https://api.github.com/users/mivanov93/events{/privacy}","received_events_url":"https://api.github.com/users/mivanov93/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":32,"created_at":"2015-07-16T23:18:27Z","updated_at":"2015-07-21T17:31:11Z","closed_at":null,"body":"I was looking at the code and found that currently the non-mb string functions are being used:\r\nex:\r\n```\r\nAura\\Filter\\Rule\\Sanitize\\Strlen\r\n```\r\n\r\n\r\n```\r\nif (strlen($value) < $len) {\r\n     $subject->$field = str_pad($value, $len, $pad_string, $pad_type);\r\n}\r\nif (strlen($value) > $len) {\r\n     $subject->$field = substr($value, 0, $len);\r\n}\r\n```\r\n\r\nHere we have strlen, substr and str_pad which are non-mb safe. Which will result in various problems:\r\n\r\nEx.  take a cyrillic script string's length - strlen(\"тесттест\") = 16 instead of 8, so our validation will fail if we ask for strings of less than 15 characters.\r\n\r\nAn even bigger issue will ensue with the substr which may cut a unicode character in half(offtopic: slashing symbols sounds funny though) - we will get a random ascii character.\r\n\r\nShouldn't we instead switch to the mb functions?"},"comment":{"url":"https://api.github.com/repos/auraphp/Aura.Filter/issues/comments/123410194","html_url":"https://github.com/auraphp/Aura.Filter/issues/87#issuecomment-123410194","issue_url":"https://api.github.com/repos/auraphp/Aura.Filter/issues/87","id":123410194,"user":{"login":"mivanov93","id":12671643,"avatar_url":"https://avatars.githubusercontent.com/u/12671643?v=3","gravatar_id":"","url":"https://api.github.com/users/mivanov93","html_url":"https://github.com/mivanov93","followers_url":"https://api.github.com/users/mivanov93/followers","following_url":"https://api.github.com/users/mivanov93/following{/other_user}","gists_url":"https://api.github.com/users/mivanov93/gists{/gist_id}","starred_url":"https://api.github.com/users/mivanov93/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mivanov93/subscriptions","organizations_url":"https://api.github.com/users/mivanov93/orgs","repos_url":"https://api.github.com/users/mivanov93/repos","events_url":"https://api.github.com/users/mivanov93/events{/privacy}","received_events_url":"https://api.github.com/users/mivanov93/received_events","type":"User","site_admin":false},"created_at":"2015-07-21T17:31:11Z","updated_at":"2015-07-21T17:31:11Z","body":"Agreed. For now we should default on UTF-8 and ASCII which is a valid subset of UTF-8, hence all ASCII strings will be valid as UTF-8 and we will have compatibility with old versions of the Rules. \r\n\r\nAs for the conversion, agreed too, later it may be implemented, but only from known encodings.\r\n\r\nSo for now all need should be preg_match, preg_replace(both with the u flag), mb_strlen and a mb version of str_pad, which I think only depends of mb_strlen. The rest should be fine."}}