{"action":"closed","issue":{"url":"https://api.github.com/repos/rancherio/rancher/issues/860","labels_url":"https://api.github.com/repos/rancherio/rancher/issues/860/labels{/name}","comments_url":"https://api.github.com/repos/rancherio/rancher/issues/860/comments","events_url":"https://api.github.com/repos/rancherio/rancher/issues/860/events","html_url":"https://github.com/rancherio/rancher/issues/860","id":78721746,"number":860,"title":"When service that is volume mounted on other service is deactivated and activated back , services are stuck in \"activating\" state","user":{"login":"sangeethah","id":4266958,"avatar_url":"https://avatars.githubusercontent.com/u/4266958?v=3","gravatar_id":"","url":"https://api.github.com/users/sangeethah","html_url":"https://github.com/sangeethah","followers_url":"https://api.github.com/users/sangeethah/followers","following_url":"https://api.github.com/users/sangeethah/following{/other_user}","gists_url":"https://api.github.com/users/sangeethah/gists{/gist_id}","starred_url":"https://api.github.com/users/sangeethah/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sangeethah/subscriptions","organizations_url":"https://api.github.com/users/sangeethah/orgs","repos_url":"https://api.github.com/users/sangeethah/repos","events_url":"https://api.github.com/users/sangeethah/events{/privacy}","received_events_url":"https://api.github.com/users/sangeethah/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/rancherio/rancher/labels/area/service","name":"area/service","color":"FBCA04"},{"url":"https://api.github.com/repos/rancherio/rancher/labels/bug","name":"bug","color":"fc2929"},{"url":"https://api.github.com/repos/rancherio/rancher/labels/status/to-test","name":"status/to-test","color":"66aa66"}],"state":"closed","locked":false,"assignee":{"login":"sangeethah","id":4266958,"avatar_url":"https://avatars.githubusercontent.com/u/4266958?v=3","gravatar_id":"","url":"https://api.github.com/users/sangeethah","html_url":"https://github.com/sangeethah","followers_url":"https://api.github.com/users/sangeethah/followers","following_url":"https://api.github.com/users/sangeethah/following{/other_user}","gists_url":"https://api.github.com/users/sangeethah/gists{/gist_id}","starred_url":"https://api.github.com/users/sangeethah/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sangeethah/subscriptions","organizations_url":"https://api.github.com/users/sangeethah/orgs","repos_url":"https://api.github.com/users/sangeethah/repos","events_url":"https://api.github.com/users/sangeethah/events{/privacy}","received_events_url":"https://api.github.com/users/sangeethah/received_events","type":"User","site_admin":false},"milestone":null,"comments":2,"created_at":"2015-05-20T22:17:57Z","updated_at":"2015-05-22T21:33:22Z","closed_at":"2015-05-22T21:33:22Z","body":"Server Version - V0.21.0\r\n\r\nSteps to reproduce the problem:\r\nCreated environment - e1.\r\nCreated a service S1 in e1 and activated it.\r\nCreated a service S2  with dataVolumesFromService pointing to S1.\r\n\r\nDeactivate S1.\r\nWait for 1 minute.\r\nActivate S1.\r\n\r\nService S2 is stuck in \"updating-active\" state and S1 is stuck in \"activating\"\r\n\r\n\r\nmysql> select * from service where environment_id=26;\r\n+----+------------+------------+---------+--------------------------------------+-------------+-----------------+---------------------+---------+-------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+\r\n| id | name       | account_id | kind    | uuid                                 | description | state           | created             | removed | remove_time | data                                                                                                                                                                                                                                                                  | environment_id |\r\n+----+------------+------------+---------+--------------------------------------+-------------+-----------------+---------------------+---------+-------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+\r\n| 61 | test726151 |          5 | service | 4bbd8b7a-271b-4ac4-bb3f-b365d9cb88d9 | NULL        | activating      | 2015-05-20 20:37:43 | NULL    | NULL        | {\"fields\":{\"scale\":1,\"launchConfig\":{\"imageUuid\":\"docker:sangeetha/testclient:latest\",\"labels\":{\"io.rancher.service.sidekick\":\"test2\"},\"startOnCreate\":true,\"publishAllPorts\":false,\"privileged\":false,\"stdinOpen\":false,\"tty\":false}}}                               |             26 |\r\n| 62 | test553753 |          5 | service | fd4d6721-8bc4-4bbb-987e-d5aa17e354cd | NULL        | updating-active | 2015-05-20 20:37:43 | NULL    | NULL        | {\"fields\":{\"scale\":1,\"launchConfig\":{\"imageUuid\":\"docker:sangeetha/testclient:latest\",\"labels\":{\"io.rancher.service.sidekick\":\"test2\"},\"startOnCreate\":true,\"publishAllPorts\":false,\"privileged\":false,\"stdinOpen\":false,\"tty\":false},\"dataVolumesFromService\":[61]}} |             26 |\r\n+----+------------+------------+---------+--------------------------------------+-------------+-----------------+---------------------+---------+-------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+\r\n2 rows in set (0.01 sec)\r\n\r\nError in server logs:\r\n\r\n2015-05-20 21:37:19,555 INFO  [06927096-c8ce-45ce-ab1a-c1f3da6f74a2:3025] [service:61] [service.activate] [] [cutorService-65] [c.p.e.p.i.DefaultProcessInstanceImpl] Exiting with code [RESOURCE_BUSY] : RESOURCE_BUSY io.cattle.platform.lock.exception.FailedToAcquireLockException: Failed to acquire lock [SERVICE.61]\r\n\tat io.cattle.platform.lock.provider.impl.StandardLock.lock(StandardLock.java:51) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.MultiLock.lock(MultiLock.java:51) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.AbstractLockManagerImpl$1.withLock(AbstractLockManagerImpl.java:17) ~[cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:30) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.servicediscovery.deployment.impl.DeploymentManagerImpl.activate(DeploymentManagerImpl.java:71) ~[cattle-iaas-service-discovery-server-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.servicediscovery.process.ServiceUpdateActivate.handle(ServiceUpdateActivate.java:42) ~[cattle-iaas-service-discovery-server-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runHandler(DefaultProcessInstanceImpl.java:432) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$4.execute(DefaultProcessInstanceImpl.java:372) ~[cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$4.execute(DefaultProcessInstanceImpl.java:369) ~[cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.idempotent.Idempotent.execute(Idempotent.java:42) ~[cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runHandlers(DefaultProcessInstanceImpl.java:369) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runLogic(DefaultProcessInstanceImpl.java:496) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runWithProcessLock(DefaultProcessInstanceImpl.java:313) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$2.doWithLockNoResult(DefaultProcessInstanceImpl.java:257) ~[cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:7) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:3) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.acquireLockAndRun(DefaultProcessInstanceImpl.java:254) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.runDelegateLoop(DefaultProcessInstanceImpl.java:187) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.executeWithProcessInstanceLock(DefaultProcessInstanceImpl.java:160) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$1.doWithLock(DefaultProcessInstanceImpl.java:109) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl$1.doWithLock(DefaultProcessInstanceImpl.java:106) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.process.impl.DefaultProcessInstanceImpl.execute(DefaultProcessInstanceImpl.java:106) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.engine.eventing.impl.ProcessEventListenerImpl.processExecute(ProcessEventListenerImpl.java:53) [cattle-framework-engine-0.5.0-SNAPSHOT.jar:na]\r\n\tat sun.reflect.GeneratedMethodAccessor568.invoke(Unknown Source) ~[na:na]\r\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_79]\r\n\tat java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_79]\r\n\tat io.cattle.platform.eventing.annotation.MethodInvokingListener$1.doWithLockNoResult(MethodInvokingListener.java:69) [cattle-framework-eventing-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:7) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.LockCallbackNoReturn.doWithLock(LockCallbackNoReturn.java:3) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.AbstractLockManagerImpl$3.doWithLock(AbstractLockManagerImpl.java:40) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.LockManagerImpl.doLock(LockManagerImpl.java:33) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:13) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.lock.impl.AbstractLockManagerImpl.lock(AbstractLockManagerImpl.java:37) [cattle-framework-lock-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.eventing.annotation.MethodInvokingListener.onEvent(MethodInvokingListener.java:65) [cattle-framework-eventing-0.5.0-SNAPSHOT.jar:na]\r\n\tat io.cattle.platform.eventing.impl.AbstractThreadPoolingEventService$2.doRun(AbstractThreadPoolingEventService.java:135) [cattle-framework-eventing-0.5.0-SNAPSHOT.jar:na]\r\n\tat org.apache.cloudstack.managed.context.NoExceptionRunnable.runInContext(NoExceptionRunnable.java:13) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]\r\n\tat org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]\r\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]\r\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:108) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]\r\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]\r\n\tat org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]\r\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79]\r\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79]\r\n\tat java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]\r\n"}}