{"action":"created","issue":{"url":"https://api.github.com/repos/apache/cloudstack/issues/829","labels_url":"https://api.github.com/repos/apache/cloudstack/issues/829/labels{/name}","comments_url":"https://api.github.com/repos/apache/cloudstack/issues/829/comments","events_url":"https://api.github.com/repos/apache/cloudstack/issues/829/events","html_url":"https://github.com/apache/cloudstack/pull/829","id":106530550,"number":829,"title":"CLOUDSTACK-8848: extra state to handle; no power report from host","user":{"login":"DaanHoogland","id":2486961,"avatar_url":"https://avatars.githubusercontent.com/u/2486961?v=3","gravatar_id":"","url":"https://api.github.com/users/DaanHoogland","html_url":"https://github.com/DaanHoogland","followers_url":"https://api.github.com/users/DaanHoogland/followers","following_url":"https://api.github.com/users/DaanHoogland/following{/other_user}","gists_url":"https://api.github.com/users/DaanHoogland/gists{/gist_id}","starred_url":"https://api.github.com/users/DaanHoogland/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DaanHoogland/subscriptions","organizations_url":"https://api.github.com/users/DaanHoogland/orgs","repos_url":"https://api.github.com/users/DaanHoogland/repos","events_url":"https://api.github.com/users/DaanHoogland/events{/privacy}","received_events_url":"https://api.github.com/users/DaanHoogland/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":16,"created_at":"2015-09-15T10:50:45Z","updated_at":"2015-09-24T10:18:24Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/cloudstack/pulls/829","html_url":"https://github.com/apache/cloudstack/pull/829","diff_url":"https://github.com/apache/cloudstack/pull/829.diff","patch_url":"https://github.com/apache/cloudstack/pull/829.patch"},"body":"there no no real handling code now so this can be regarded as a DISCUSS item. It may well be an improvement over existing code as is, however."},"comment":{"url":"https://api.github.com/repos/apache/cloudstack/issues/comments/142882541","html_url":"https://github.com/apache/cloudstack/pull/829#issuecomment-142882541","issue_url":"https://api.github.com/repos/apache/cloudstack/issues/829","id":142882541,"user":{"login":"resmo","id":23809,"avatar_url":"https://avatars.githubusercontent.com/u/23809?v=3","gravatar_id":"","url":"https://api.github.com/users/resmo","html_url":"https://github.com/resmo","followers_url":"https://api.github.com/users/resmo/followers","following_url":"https://api.github.com/users/resmo/following{/other_user}","gists_url":"https://api.github.com/users/resmo/gists{/gist_id}","starred_url":"https://api.github.com/users/resmo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/resmo/subscriptions","organizations_url":"https://api.github.com/users/resmo/orgs","repos_url":"https://api.github.com/users/resmo/repos","events_url":"https://api.github.com/users/resmo/events{/privacy}","received_events_url":"https://api.github.com/users/resmo/received_events","type":"User","site_admin":false},"created_at":"2015-09-24T10:18:24Z","updated_at":"2015-09-24T10:18:24Z","body":"@anshul1886 @koushik-das \r\n@DaanHoogland  and I  had a debug session last friday, and since he is off for the next couple of days I can give you more details about we analysed. \r\n\r\nThe powerReportMissing is not the problem, it is only the trigger. The graceful period is the problem. The calculation of this period is relaying (see https://github.com/apache/cloudstack/blob/4.5.2/engine/orchestration/src/com/cloud/vm/VirtualMachinePowerStateSyncImpl.java#L114) on the field `update_time` in table `vm_instance`. But if I look at the value  it seems it doesn't get updated. So the grace period has most likely always passed. \r\n\r\nI tried to do a workaround doing the following, I ran an update sql for every 5 seconds which updated the `update_time` for my router r-342 which I was migrating around esx cluster nodes:\r\n~~~\r\n mysql -e 'update cloud.vm_instance set update_time=NOW() where id=342;'\r\n~~~\r\nAnd the router didn't get rebooted:\r\n~~~\r\n2015-09-24 11:47:07,685 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-218:ctx-5849bd19) VM state report. host: 25, vm id: 342, power state: PowerOn\r\n2015-09-24 11:47:07,696 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-218:ctx-5849bd19) VM state report is updated. host: 25, vm id: 342, power state: PowerOn\r\n2015-09-24 11:48:06,462 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-55:ctx-84cd4323) VM state report. host: 19, vm id: 342, power state: PowerOn\r\n2015-09-24 11:48:06,471 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-55:ctx-84cd4323) VM state report is updated. host: 19, vm id: 342, power state: PowerOn\r\n2015-09-24 11:48:06,493 WARN  [o.a.c.alerts] (DirectAgentCronJob-55:ctx-84cd4323)  alertType:: 9 // dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Router has been migrated out of band: r-342-VM\r\n2015-09-24 11:49:06,539 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-29:ctx-2a57d676) Detected missing VM. host: 19, vm id: 342, power state: PowerReportMissing, last state update: 1443095344000\r\n2015-09-24 11:49:06,539 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-29:ctx-2a57d676) vm id: 342 - time since last state update(-7197461ms) has not passed graceful period yet\r\n2015-09-24 11:49:07,719 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-444:ctx-fdd4c055) VM state report. host: 20, vm id: 342, power state: PowerOn\r\n~~~\r\n\r\nWhich means this patch is not fix the root cause. To me the root cause is that `update_time` is not updated or the gracePeriod calculation is wrong.\r\n\r\nAny thoughts?"}}