{"action":"created","issue":{"url":"https://api.github.com/repos/ARCANEDEV/LogViewer/issues/21","labels_url":"https://api.github.com/repos/ARCANEDEV/LogViewer/issues/21/labels{/name}","comments_url":"https://api.github.com/repos/ARCANEDEV/LogViewer/issues/21/comments","events_url":"https://api.github.com/repos/ARCANEDEV/LogViewer/issues/21/events","html_url":"https://github.com/ARCANEDEV/LogViewer/issues/21","id":113567143,"number":21,"title":"Avoid loading whole log files into memory","user":{"login":"judgej","id":395934,"avatar_url":"https://avatars.githubusercontent.com/u/395934?v=3","gravatar_id":"","url":"https://api.github.com/users/judgej","html_url":"https://github.com/judgej","followers_url":"https://api.github.com/users/judgej/followers","following_url":"https://api.github.com/users/judgej/following{/other_user}","gists_url":"https://api.github.com/users/judgej/gists{/gist_id}","starred_url":"https://api.github.com/users/judgej/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/judgej/subscriptions","organizations_url":"https://api.github.com/users/judgej/orgs","repos_url":"https://api.github.com/users/judgej/repos","events_url":"https://api.github.com/users/judgej/events{/privacy}","received_events_url":"https://api.github.com/users/judgej/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ARCANEDEV/LogViewer/labels/question","name":"question","color":"cc317c"}],"state":"closed","locked":false,"assignee":{"login":"arcanedev-maroc","id":3282340,"avatar_url":"https://avatars.githubusercontent.com/u/3282340?v=3","gravatar_id":"","url":"https://api.github.com/users/arcanedev-maroc","html_url":"https://github.com/arcanedev-maroc","followers_url":"https://api.github.com/users/arcanedev-maroc/followers","following_url":"https://api.github.com/users/arcanedev-maroc/following{/other_user}","gists_url":"https://api.github.com/users/arcanedev-maroc/gists{/gist_id}","starred_url":"https://api.github.com/users/arcanedev-maroc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arcanedev-maroc/subscriptions","organizations_url":"https://api.github.com/users/arcanedev-maroc/orgs","repos_url":"https://api.github.com/users/arcanedev-maroc/repos","events_url":"https://api.github.com/users/arcanedev-maroc/events{/privacy}","received_events_url":"https://api.github.com/users/arcanedev-maroc/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/ARCANEDEV/LogViewer/milestones/1","html_url":"https://github.com/ARCANEDEV/LogViewer/milestones/3.6.0","labels_url":"https://api.github.com/repos/ARCANEDEV/LogViewer/milestones/1/labels","id":1381562,"number":1,"title":"3.6.0","description":"","creator":{"login":"arcanedev-maroc","id":3282340,"avatar_url":"https://avatars.githubusercontent.com/u/3282340?v=3","gravatar_id":"","url":"https://api.github.com/users/arcanedev-maroc","html_url":"https://github.com/arcanedev-maroc","followers_url":"https://api.github.com/users/arcanedev-maroc/followers","following_url":"https://api.github.com/users/arcanedev-maroc/following{/other_user}","gists_url":"https://api.github.com/users/arcanedev-maroc/gists{/gist_id}","starred_url":"https://api.github.com/users/arcanedev-maroc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arcanedev-maroc/subscriptions","organizations_url":"https://api.github.com/users/arcanedev-maroc/orgs","repos_url":"https://api.github.com/users/arcanedev-maroc/repos","events_url":"https://api.github.com/users/arcanedev-maroc/events{/privacy}","received_events_url":"https://api.github.com/users/arcanedev-maroc/received_events","type":"User","site_admin":false},"open_issues":1,"closed_issues":1,"state":"open","created_at":"2015-10-28T14:44:36Z","updated_at":"2015-10-28T14:45:11Z","due_on":null,"closed_at":null},"comments":3,"created_at":"2015-10-27T12:00:42Z","updated_at":"2015-10-28T16:11:50Z","closed_at":"2015-10-28T14:43:43Z","body":"This package looks well thought-through and organised - looking forward to trying it out.\r\n\r\nNow, every single other log viewer package I've seen starts by loading an entire log file into memory, then parsing it from there. When the log files get big, then it blows the memory limits and fails. This can be a big problem at those times when you really need to be able to view the logs.\r\n\r\nSo, does this package load whole log files into memory (in parsed or raw form)? Or can it extract a portion of a log file direct from disk based on the page options (e.g. retrieve just log entries N to N+m from a file)?\r\n\r\nIt strikes me also that the start of each log entry could be indexed by byte position, so a position in a file can be seeked very quickly. Even if only every 10th or 100th log entry were indexed, it would provide some starting points.\r\n\r\nAnyway - this is mainly a question (and I have not found the bit where you actually open the file for reading yet, so I don't know the answer)."},"comment":{"url":"https://api.github.com/repos/ARCANEDEV/LogViewer/issues/comments/151896055","html_url":"https://github.com/ARCANEDEV/LogViewer/issues/21#issuecomment-151896055","issue_url":"https://api.github.com/repos/ARCANEDEV/LogViewer/issues/21","id":151896055,"user":{"login":"arcanedev-maroc","id":3282340,"avatar_url":"https://avatars.githubusercontent.com/u/3282340?v=3","gravatar_id":"","url":"https://api.github.com/users/arcanedev-maroc","html_url":"https://github.com/arcanedev-maroc","followers_url":"https://api.github.com/users/arcanedev-maroc/followers","following_url":"https://api.github.com/users/arcanedev-maroc/following{/other_user}","gists_url":"https://api.github.com/users/arcanedev-maroc/gists{/gist_id}","starred_url":"https://api.github.com/users/arcanedev-maroc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arcanedev-maroc/subscriptions","organizations_url":"https://api.github.com/users/arcanedev-maroc/orgs","repos_url":"https://api.github.com/users/arcanedev-maroc/repos","events_url":"https://api.github.com/users/arcanedev-maroc/events{/privacy}","received_events_url":"https://api.github.com/users/arcanedev-maroc/received_events","type":"User","site_admin":false},"created_at":"2015-10-28T16:11:50Z","updated_at":"2015-10-28T16:11:50Z","body":"There will be an action. I've added this issue to the milestone, to think about it for the next release.\r\n\r\nWhat do you think about this : Limiting the loading log files by year (current year / chosen year) ?\r\n\r\nI'm optimistic that log files will not exceed the 200~400Mo in total (daily log system of course) in one year."}}