{"action":"created","issue":{"url":"https://api.github.com/repos/airbnb/airflow/issues/57","labels_url":"https://api.github.com/repos/airbnb/airflow/issues/57/labels{/name}","comments_url":"https://api.github.com/repos/airbnb/airflow/issues/57/comments","events_url":"https://api.github.com/repos/airbnb/airflow/issues/57/events","html_url":"https://github.com/airbnb/airflow/issues/57","id":90004785,"number":57,"title":"I need a flow to start with a sensor","user":{"login":"r39132","id":581734,"avatar_url":"https://avatars.githubusercontent.com/u/581734?v=3","gravatar_id":"","url":"https://api.github.com/users/r39132","html_url":"https://github.com/r39132","followers_url":"https://api.github.com/users/r39132/followers","following_url":"https://api.github.com/users/r39132/following{/other_user}","gists_url":"https://api.github.com/users/r39132/gists{/gist_id}","starred_url":"https://api.github.com/users/r39132/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/r39132/subscriptions","organizations_url":"https://api.github.com/users/r39132/orgs","repos_url":"https://api.github.com/users/r39132/repos","events_url":"https://api.github.com/users/r39132/events{/privacy}","received_events_url":"https://api.github.com/users/r39132/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-06-22T04:10:10Z","updated_at":"2015-06-22T04:41:31Z","closed_at":null,"body":"I have a flow that is linear and sequential as shown below : \r\n\r\n1. new data arrives in SQS\r\n2. wait for that data to start being written to postgres\r\n3. wait for the SQS queue to drain\r\n4. send an email that the flow completed\r\n\r\nEventually, we will have a pre-cursor step (e.g. step 0.) that kicks off a Spark job before waiting for new data in the SQS queue. For now, I do not have that, so I would like my flow to simply reset to the first step. In some ways, I am asking for a circular flow that never ends. This breaks the entire workflow abstraction. On another note, where is the \"cron\" functionality? Do I schedule airflow commands in cron outside of airflow?"},"comment":{"url":"https://api.github.com/repos/airbnb/airflow/issues/comments/113998635","html_url":"https://github.com/airbnb/airflow/issues/57#issuecomment-113998635","issue_url":"https://api.github.com/repos/airbnb/airflow/issues/57","id":113998635,"user":{"login":"mistercrunch","id":487433,"avatar_url":"https://avatars.githubusercontent.com/u/487433?v=3","gravatar_id":"","url":"https://api.github.com/users/mistercrunch","html_url":"https://github.com/mistercrunch","followers_url":"https://api.github.com/users/mistercrunch/followers","following_url":"https://api.github.com/users/mistercrunch/following{/other_user}","gists_url":"https://api.github.com/users/mistercrunch/gists{/gist_id}","starred_url":"https://api.github.com/users/mistercrunch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mistercrunch/subscriptions","organizations_url":"https://api.github.com/users/mistercrunch/orgs","repos_url":"https://api.github.com/users/mistercrunch/repos","events_url":"https://api.github.com/users/mistercrunch/events{/privacy}","received_events_url":"https://api.github.com/users/mistercrunch/received_events","type":"User","site_admin":false},"created_at":"2015-06-22T04:41:31Z","updated_at":"2015-06-22T04:41:31Z","body":"Airflow runs its own scheduler, it's hard to miss it in the docs:\r\nhttp://pythonhosted.org/airflow/scheduler.html\r\n\r\nIs your process a batch job that runs on a fixed scheduled interval? If not Airflow is probably not the right platform. DAG stands for Directed Acyclic Graph. Acyclic means there's no loop. Though its assumed we run the same set of jobs (the DAG) at a scheduled interval, which kind of makes for a large loop around the DAG execution."}}