{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/51","labels_url":"https://api.github.com/repos/fchollet/keras/issues/51/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/51/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/51/events","html_url":"https://github.com/fchollet/keras/issues/51","id":68995985,"number":51,"title":"Add interrupt handlers","user":{"login":"jfsantos","id":5733,"avatar_url":"https://avatars.githubusercontent.com/u/5733?v=3","gravatar_id":"","url":"https://api.github.com/users/jfsantos","html_url":"https://github.com/jfsantos","followers_url":"https://api.github.com/users/jfsantos/followers","following_url":"https://api.github.com/users/jfsantos/following{/other_user}","gists_url":"https://api.github.com/users/jfsantos/gists{/gist_id}","starred_url":"https://api.github.com/users/jfsantos/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jfsantos/subscriptions","organizations_url":"https://api.github.com/users/jfsantos/orgs","repos_url":"https://api.github.com/users/jfsantos/repos","events_url":"https://api.github.com/users/jfsantos/events{/privacy}","received_events_url":"https://api.github.com/users/jfsantos/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":10,"created_at":"2015-04-16T19:42:30Z","updated_at":"2015-04-21T21:47:11Z","closed_at":null,"body":"Since many experiments can take a while and will be running on an environment where the user does not have a lot of control (e.g., a shared cluster), it would be interesting to have interrupt handlers to do something in case the operating system sends a signal to kill the process during the execution of the `fit` method. [Blocks](https://github.com/bartvm/blocks/blob/master/blocks/main_loop.py) does this by using the `signal` module (which is part of the standard library). That way, you can save the current model state (using pickle, for example) before letting the OS kill your process. Would you be interested in adding something similar to the `fit` method in model.py? "},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/94953043","html_url":"https://github.com/fchollet/keras/issues/51#issuecomment-94953043","issue_url":"https://api.github.com/repos/fchollet/keras/issues/51","id":94953043,"user":{"login":"fchollet","id":710255,"avatar_url":"https://avatars.githubusercontent.com/u/710255?v=3","gravatar_id":"","url":"https://api.github.com/users/fchollet","html_url":"https://github.com/fchollet","followers_url":"https://api.github.com/users/fchollet/followers","following_url":"https://api.github.com/users/fchollet/following{/other_user}","gists_url":"https://api.github.com/users/fchollet/gists{/gist_id}","starred_url":"https://api.github.com/users/fchollet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fchollet/subscriptions","organizations_url":"https://api.github.com/users/fchollet/orgs","repos_url":"https://api.github.com/users/fchollet/repos","events_url":"https://api.github.com/users/fchollet/events{/privacy}","received_events_url":"https://api.github.com/users/fchollet/received_events","type":"User","site_admin":false},"created_at":"2015-04-21T21:47:11Z","updated_at":"2015-04-21T21:47:11Z","body":"One clean solution to \"save\" optimizers would be heavily modify them to store all moving parts as class attributes, and allow for state snapshotting (as a dictionary of these attributes) and instantiation from a previous state. We would lose in agility by doing so, but anything else would be pretty hacky and possibly unsafe.\r\n\r\nFor performance reasons if would definitely be preferable to incorporate the monitoring / data dumping to the Progbar (abstracted as a Monitor class), rather than having to pass updates to both the progbar for logging and the monitor for data dumping... in fact we can get there with only a slight modification of the existing progbar.\r\n\r\nAs we add more features, the challenge will be the interface: we want to design a way to configure the monitor that is sufficiently powerful, but that isn't heavy or complicated. Large configuration dictionaries have a tendency to quickly get out of hand. \r\n "}}