{"action":"created","issue":{"url":"https://api.github.com/repos/cooperative-computing-lab/cctools/issues/981","labels_url":"https://api.github.com/repos/cooperative-computing-lab/cctools/issues/981/labels{/name}","comments_url":"https://api.github.com/repos/cooperative-computing-lab/cctools/issues/981/comments","events_url":"https://api.github.com/repos/cooperative-computing-lab/cctools/issues/981/events","html_url":"https://github.com/cooperative-computing-lab/cctools/issues/981","id":112802741,"number":981,"title":"docker+ucernvm+parrot+condor","user":{"login":"tommasoboccali","id":4093537,"avatar_url":"https://avatars.githubusercontent.com/u/4093537?v=3","gravatar_id":"","url":"https://api.github.com/users/tommasoboccali","html_url":"https://github.com/tommasoboccali","followers_url":"https://api.github.com/users/tommasoboccali/followers","following_url":"https://api.github.com/users/tommasoboccali/following{/other_user}","gists_url":"https://api.github.com/users/tommasoboccali/gists{/gist_id}","starred_url":"https://api.github.com/users/tommasoboccali/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tommasoboccali/subscriptions","organizations_url":"https://api.github.com/users/tommasoboccali/orgs","repos_url":"https://api.github.com/users/tommasoboccali/repos","events_url":"https://api.github.com/users/tommasoboccali/events{/privacy}","received_events_url":"https://api.github.com/users/tommasoboccali/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-10-22T13:25:11Z","updated_at":"2015-10-22T13:56:39Z","closed_at":null,"body":"Ciao, referring here to a htcondor thread (where I was adviced about opening an issue here: https://www-auth.cs.wisc.edu/lists/htcondor-users/2015-October/msg00067.shtml )\r\n\r\nSo this is the problem: I an trying to run condor inside a uCERNVM under docker, which gets /usr, /bin etc from CVMFS (the usual uno fro @jblomer ). Also condor is not in a local FS, but taken from cvmfs. There is nothing outisde parrot, since before parrot the system is not functional (no /usr, no /bin).\r\n\r\ni can start condor, connect to a pool, be visible to external users, and accept jobs.\r\nThe problem (if we debugged the logs to a good level), is that at that point condor does not allow to run the payload as root, but wants to move to the user\"nobody\" (see logs below), using setuid().\r\n\r\nIn my understanding, that is not allowed by parrot, which traps it and the user remains root. at the next second, condor screams it is still root and closes the job.\r\n\r\nI can reproduce the thing in a simplified way:\r\n\r\nbash-4.1# su nobody\r\n2015/10/22 13:10:45.68 parrot_run[1] <child:10616> notice: cannot execute the program /bin/su because it is setuid.\r\nbash: /bin/su: Permission denied\r\n\r\neven if I copy the /bin/su command to a non cvmfs disk (local), it still fails. I could not really test via the -M parrot option since /bin/su does not exist before parrot (and not even /bin), but I understood the effect is the same as my local test:\r\n\r\nbash-4.1# cp /bin/su /root\r\nbash-4.1# /root/su nobody\r\n2015/10/22 13:21:51.96 parrot_run[1] <child:10664> notice: warning: system call 250 (keyctl) not supported for program /root/su\r\n\r\nbash-4.1# chmod +s /root/su \r\nbash-4.1# /root/su nobody\r\n2015/10/22 13:22:03.67 parrot_run[1] <child:10668> notice: cannot execute the program /root/su because it is setuid.\r\nbash: /root/su: Permission denied\r\n\r\nI think the basic question is: is there a way to have setuid () (and hence all the command using it, like su and sudo) working inside parrot?\r\n\r\nHere is the log which shows how Condor gets the payload and tries to change user:\r\n\r\n10/22/15 13:11:26 (fd:15) (pid:10651) (D_ALWAYS) Using wrapper /root/condor_job_wrapper.sh to exec /home/glidein_pilot/dir_10651/condor_exec.exe -a sandbox.tar.gz --sourceURL=https://cmsweb.cern.ch/crabcache --jobNumber=15 --cmsswVersion=CMSSW_7_3_5_patch2 --scramArch=slc6_amd64_gcc491 --inputFile=job_input_file_list_15.txt --runAndLumis=job_lumis_15.json --lheInputFiles=False --firstEvent=None --firstLumi=None --lastEvent=None --firstRun=None --seeding=AutomaticSeeding --scriptExe=None --eventsPerLumi=None --scriptArgs=[] -o {}\r\n10/22/15 13:11:26 (fd:15) (pid:10651) (D_ALWAYS) Setting job's virtual memory rlimit to 0 megabytes\r\n10/22/15 13:11:26 (fd:15) (pid:10651) (D_CONFIG) ENFORCE_CPU_AFFINITY is undefined, using default value of False\r\n10/22/15 13:11:26 (fd:15) (pid:10651) (D_ALWAYS) Running job as user nobody\r\n10/22/15 13:11:26 (fd:15) (pid:10651) (D_PRIV) PRIV_USER --> PRIV_CONDOR at /slots/01/dir_57518/userdir/src/condor_starter.V6.1/os_proc.cpp:529\r\n10/22/15 13:11:26 (fd:15) (pid:10651) (D_DAEMONCORE) In DaemonCore::Create_Process(/root/condor_job_wrapper.sh,...)\r\n10/22/15 13:11:26 (fd:17) (pid:10651) (D_PRIV) PRIV_CONDOR --> PRIV_USER at /slots/01/dir_57518/userdir/src/condor_daemon_core.V6/daemon_core.cpp:7260\r\n10/22/15 13:11:26 (fd:17) (pid:10651) (D_PRIV) PRIV_USER --> PRIV_CONDOR at /slots/01/dir_57518/userdir/src/condor_daemon_core.V6/daemon_core.cpp:7282\r\n10/22/15 13:11:26 (fd:17) (pid:10660) (D_DAEMONCORE) Create_Process: Arg: condor_exec.exe /home/glidein_pilot/dir_10651/condor_exec.exe -a sandbox.tar.gz --sourceURL=https://cmsweb.cern.ch/crabcache --jobNumber=15 --cmsswVersion=CMSSW_7_3_5_patch2 --scramArch=slc6_amd64_gcc491 --inputFile=job_input_file_list_15.txt --runAndLumis=job_lumis_15.json --lheInputFiles=False --firstEvent=None --firstLumi=None --lastEvent=None --firstRun=None --seeding=AutomaticSeeding --scriptExe=None --eventsPerLumi=None --scriptArgs=[] -o {}\r\n10/22/15 13:11:26 (fd:17) (pid:10660) (D_PROCFAMILY) About to register family for PID 10660 with the ProcD\r\n10/22/15 13:11:26 (fd:17) (pid:10660) (D_PROCFAMILY) Result of \"register_subfamily\" operation from ProcD: SUCCESS\r\n10/22/15 13:11:26 (fd:17) (pid:10660) (D_PROCFAMILY) About to tell ProcD to track family with root 10660 via environment\r\n10/22/15 13:11:26 (fd:17) (pid:10660) (D_PROCFAMILY) Result of \"track_family_via_environment\" operation from ProcD: SUCCESS\r\n10/22/15 13:11:26 (fd:17) (pid:10660) (D_DAEMONCORE) Re-mapping std(in|out|err) in child.\r\n10/22/15 13:11:26 (fd:17) (pid:10660) (D_DAEMONCORE) calling nice(10)\r\n10/22/15 13:11:26 (fd:17) (pid:10660) (D_DAEMONCORE) Printing fds to inherit: \r\n10/22/15 13:11:26 (fd:17) (pid:10660) (D_DAEMONCORE) About to exec \"/root/condor_job_wrapper.sh\"\r\n10/22/15 13:11:27 (fd:20) (pid:10651) (D_ALWAYS) Create_Process(/root/condor_job_wrapper.sh): child failed because PRIV_USER_FINAL process was still root before exec()\r\n10/22/15 13:11:27 (fd:17) (pid:10651) (D_DAEMONCORE) Warning: cleanup from Create_Process took 0.693 sec, Create_Process took 0.693 sec overall\r\n10/22/15 13:11:27 (fd:12) (pid:10651) (D_SYSCALLS) Doing CONDOR_ulog\r\n10/22/15 13:11:27 (fd:12) (pid:10651) (D_NETWORK) condor_write(fd=9 <128.142.242.250:4080>,,size=755,timeout=300,flags=0)\r\n10/22/15 13:11:27 (fd:12) (pid:10651) (D_ALWAYS) Create_Process(/root/condor_job_wrapper.sh,/home/glidein_pilot/dir_10651/condor_exec.exe -a sandbox.tar.gz --sourceURL=https://cmsweb.cern.ch/crabcache --jobNumber=15 --cmsswVersion=CMSSW_7_3_5_patch2 --scramArch=slc6_amd64_gcc491 --inputFile=job_input_file_list_15.txt --runAndLumis=job_lumis_15.json --lheInputFiles=False --firstEvent=None --firstLumi=None --lastEvent=None --firstRun=None --seeding=AutomaticSeeding --scriptExe=None --eventsPerLumi=None --scriptArgs=[] -o {}, ...) failed: (errno=666666: 'Unknown error 666666')\r\n10/22/15 13:11:27 (fd:12) (pid:10651) (D_ALWAYS) Failed to start job, exiting\r\n\r\nthanks a lot\r\n\r\ntommaso"},"comment":{"url":"https://api.github.com/repos/cooperative-computing-lab/cctools/issues/comments/150230920","html_url":"https://github.com/cooperative-computing-lab/cctools/issues/981#issuecomment-150230920","issue_url":"https://api.github.com/repos/cooperative-computing-lab/cctools/issues/981","id":150230920,"user":{"login":"batrick","id":548304,"avatar_url":"https://avatars.githubusercontent.com/u/548304?v=3","gravatar_id":"","url":"https://api.github.com/users/batrick","html_url":"https://github.com/batrick","followers_url":"https://api.github.com/users/batrick/followers","following_url":"https://api.github.com/users/batrick/following{/other_user}","gists_url":"https://api.github.com/users/batrick/gists{/gist_id}","starred_url":"https://api.github.com/users/batrick/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/batrick/subscriptions","organizations_url":"https://api.github.com/users/batrick/orgs","repos_url":"https://api.github.com/users/batrick/repos","events_url":"https://api.github.com/users/batrick/events{/privacy}","received_events_url":"https://api.github.com/users/batrick/received_events","type":"User","site_admin":false},"created_at":"2015-10-22T13:56:39Z","updated_at":"2015-10-22T13:56:39Z","body":"Parrot has always turned `setuid` into a no-op. So when HTCondor checks that `setuid` actually changed the uid, it fails.\r\n\r\nOne of the issues with allowing `setuid` is that it complicates signal handling and access control in Parrot (the tracee may no longer be the same uid as Parrot). I don't think the fix is as simple as turning the system call back on. What do you think @dthain?"}}