{"action":"created","issue":{"url":"https://api.github.com/repos/dhh1128/inox-java-experiments/issues/8","labels_url":"https://api.github.com/repos/dhh1128/inox-java-experiments/issues/8/labels{/name}","comments_url":"https://api.github.com/repos/dhh1128/inox-java-experiments/issues/8/comments","events_url":"https://api.github.com/repos/dhh1128/inox-java-experiments/issues/8/events","html_url":"https://github.com/dhh1128/inox-java-experiments/issues/8","id":114197047,"number":8,"title":"Create a high-precision timer service","user":{"login":"dhh1128","id":2208904,"avatar_url":"https://avatars.githubusercontent.com/u/2208904?v=3","gravatar_id":"","url":"https://api.github.com/users/dhh1128","html_url":"https://github.com/dhh1128","followers_url":"https://api.github.com/users/dhh1128/followers","following_url":"https://api.github.com/users/dhh1128/following{/other_user}","gists_url":"https://api.github.com/users/dhh1128/gists{/gist_id}","starred_url":"https://api.github.com/users/dhh1128/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dhh1128/subscriptions","organizations_url":"https://api.github.com/users/dhh1128/orgs","repos_url":"https://api.github.com/users/dhh1128/repos","events_url":"https://api.github.com/users/dhh1128/events{/privacy}","received_events_url":"https://api.github.com/users/dhh1128/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-10-30T04:21:54Z","updated_at":"2015-11-14T00:06:07Z","closed_at":"2015-11-14T00:06:07Z","body":"(This issue assumes that creating one-off timeouts whenever we need them is going to be clunky and inefficient. If that assumption can be proved false, most or all of this work might go away.)\r\n\r\nMany features in inox operate under important time constraints. At a high level, we make a promise to our customers that we will give them an answer from our rating/risk scoring system within about 3 seconds, so they can depend on their browsing experience being zippy. Within the time that we have to give them an answer, we have to accomplish many sub-tasks, and sometimes earlier tasks should be abandoned if they take too long--without abandoning the task as a whole. For example, we may attempt to download a file to help us decide how risky a web page is; if we get the file in time, we take it into account as we do our final analysis. If we don't get the file in time, we still have to give an answer, so we do the best with what we have. Sometimes, there are even multiple things happening in parallel, with their own timeouts, plus a larger timeout that constrains the parent task for all of them.\r\n\r\nSome of the timeouts in our system can be handled without us doing much. For example, we can configure tcp/ip to communicate with a remote client in such a way that unusually slow communications cause timeouts, which ultimately turn into java IOExceptions or similar.\r\n\r\nFor most other things, however, we need a generic and standard answer: how will we time out all these tasks? We need an answer that's:\r\n\r\n* *scalable* (handling hundreds or thousands of timeouts at once)\r\n* *simple* (just configure the timeout with a callback to take action, and a timeframe, and then set it and forget it)\r\n* *reasonably accurate* (we can't have a backlog of timeouts causing other timeouts not to fire; if we say to timeout in 250 millis, it might be acceptable to timeout in 262 millis, but not in 5000 millis when the system gets around to it)\r\n* *efficient* (can't use hundreds of threads, can't require lots of mutexing)\r\n\r\nThe interface for this class includes:\r\n\r\n```\r\nlong add(Callback, MonotonicTicks [long] when); // insert a timeout and return the ID of the added item\r\nlong remove(long id); // remove a timeout, using the id returned by add()\r\nvoid shutdown(); // inform the manager that its internal thread should be stopped gracefully, because the app is terminating\r\n```\r\n\r\nThe foundation of our solution will be a monotonic clock. We can use System.nanoTime() for this; it provides a high-resolution timestamp (typically, better than a microsecond) that always increases as long as the system is running, even if the system's \"current time\" gets reset. (System.nanoTime() didn't originally offer this guarantee, but now it does on modern linux kernels). This monotonic clock is the equivalent of GetTickCount() on Windows.\r\n\r\nThe plan is to have the timeout manager run an internal thread that wakes up reasonably often (say, every 25 ms), but not constantly. Each time the thread wakes up, it builds a list of callbacks to invoke because their associated timeout has passed, then releases its mutex, then invokes a callback for every item that has timed out, removes the items from its list of timeouts, and goes back to sleep. Callers can add new timeouts at any time; there will almost never be contention for the mutex protecting the list, since the dispatch thread will mostly be sleeping.\r\n\r\nThis will only work if callbacks return almost instantly; if we ever have a callback that takes, say, 500 millis to run, it will get us into trouble. We could add a bunch of dispatch threads inside the timeout mgr to compensate for this, but that increases complexity a lot, and it really puts responsibility where it doesn't belong. The timeout mgr should be responsible for deciding a timeout has occurred, and telling somebody about it promptly so they can react--nothing else. To help people write code that consumes timeout functionality corrrectly, then, I suggest that we have the timeout mgr keep track of how long each callback takes, and log warnings any time an individual callback doesn't execute within, say, 1 millisec. Hopefully this would get the attention of developers. It shouldn't be hard for a developer to react to a timeout by puttting long-running work on a new thread instead of handling it directly on the notifying thread, right?\r\n\r\nIn order to make the add/remove methods and the calculation of expired timeouts as fast as possible, internally we'll maintain a list or array of timeouts, sorted by when they should fire. Adding will thus require a binary search, and so will removing. However, calculating what's timed out since the last iteration of our loop will be super fast; just start at the top of the array and select items until the timeout value is greater than now."},"comment":{"url":"https://api.github.com/repos/dhh1128/inox-java-experiments/issues/comments/156594410","html_url":"https://github.com/dhh1128/inox-java-experiments/issues/8#issuecomment-156594410","issue_url":"https://api.github.com/repos/dhh1128/inox-java-experiments/issues/8","id":156594410,"user":{"login":"dhh1128","id":2208904,"avatar_url":"https://avatars.githubusercontent.com/u/2208904?v=3","gravatar_id":"","url":"https://api.github.com/users/dhh1128","html_url":"https://github.com/dhh1128","followers_url":"https://api.github.com/users/dhh1128/followers","following_url":"https://api.github.com/users/dhh1128/following{/other_user}","gists_url":"https://api.github.com/users/dhh1128/gists{/gist_id}","starred_url":"https://api.github.com/users/dhh1128/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dhh1128/subscriptions","organizations_url":"https://api.github.com/users/dhh1128/orgs","repos_url":"https://api.github.com/users/dhh1128/repos","events_url":"https://api.github.com/users/dhh1128/events{/privacy}","received_events_url":"https://api.github.com/users/dhh1128/received_events","type":"User","site_admin":false},"created_at":"2015-11-14T00:06:07Z","updated_at":"2015-11-14T00:06:07Z","body":"This functionality is all provided by Java's Timer class."}}