{"action":"created","issue":{"url":"https://api.github.com/repos/owncloud/core/issues/14873","labels_url":"https://api.github.com/repos/owncloud/core/issues/14873/labels{/name}","comments_url":"https://api.github.com/repos/owncloud/core/issues/14873/comments","events_url":"https://api.github.com/repos/owncloud/core/issues/14873/events","html_url":"https://github.com/owncloud/core/issues/14873","id":61052189,"number":14873,"title":"external storage or encryption plugin causes endless re-sync in client","user":{"login":"qnxor","id":5108876,"avatar_url":"https://avatars.githubusercontent.com/u/5108876?v=3","gravatar_id":"","url":"https://api.github.com/users/qnxor","html_url":"https://github.com/qnxor","followers_url":"https://api.github.com/users/qnxor/followers","following_url":"https://api.github.com/users/qnxor/following{/other_user}","gists_url":"https://api.github.com/users/qnxor/gists{/gist_id}","starred_url":"https://api.github.com/users/qnxor/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/qnxor/subscriptions","organizations_url":"https://api.github.com/users/qnxor/orgs","repos_url":"https://api.github.com/users/qnxor/repos","events_url":"https://api.github.com/users/qnxor/events{/privacy}","received_events_url":"https://api.github.com/users/qnxor/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/owncloud/core/labels/app:files_external","name":"app:files_external","color":"02d7e1"},{"url":"https://api.github.com/repos/owncloud/core/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2015-03-13T12:20:13Z","updated_at":"2015-03-17T09:42:06Z","closed_at":null,"body":"As asked by @guruz in #2479 I'm copying here the info initially reported there which causes my owncloud client to sync every 60 seconds and also cause cpu spikes. This is on MacOSX 10.10.2 but it seems server related.\r\n\r\n---\r\n\r\n@mcnesium I installed 1.8 beta2 and little changed in my case. I still see cpu spikes and high usages ***every minute*** or so. CPU time was 02m04s for owncloud,  compared to 00m19s for dropbox ... both idle, started at the same time, after about 2h. Dropbox handles over 20,000 files, while owncloud only 4 files.\r\n\r\nI'll run the `ps` test and terminal logging and post again.\r\n\r\n**EDIT:** here's the `ps` graph, notice the multiple spikes that happen every minute\r\n![cpu usage vs time](http://cl.ly/image/18033v1g2Q3Q/oc.png)\r\n\r\n**EDIT 2:** Here is the relevant part of the log (sanitized) from running `owncloud --logfile oc1.log` ... everything you see is being repeated every 60 seconds: https://dl.dropbox.com/s/ovw1czxlk3pe2k4/oc1.log\r\n\r\n---\r\n\r\n> Can you look at the complete log output and say which folder is always re-scheduled and synced? Is it all three folders or only that one?\r\n\r\nThe complete log consists of repeated blocks of the text I posted here: https://dl.dropbox.com/s/ovw1czxlk3pe2k4/oc1.log ... all of this repeats every 60 seconds. \r\n\r\nNote how is it says `slotEtagPollTimeTimeout` and `slotRunEtagJob: * Trying to check \"ownCloud\" for changes via Etag check. (time since last sync: 53 s)`. It looks there is a timeout of 53 s which triggers regular checks.\r\n\r\nBut that aside, the fact that the etags are different doesn't look right (see log): `03-05 04:35:32:250 0x7f91d0614440 OCC::Folder::etagRetreived: * Compare etag with previous etag: last: \"\"54f7dca18ee8f\"\" , received: \"\"54f7dcdd84d36\"\"`\r\n\r\n> Just check for the message `OCC::FolderMan::slotScheduleSync: Schedule folder \"???\" to sync!`\r\n\r\nSee the log I posted, line 11: `OCC::FolderMan::slotRunOneEtagJob: Scheduling \"ownCloud\" to check remote ETag` (that's the main and only folder in MacOSX that the desktop client syncs)\r\n\r\n> Is there anything special about the sync folder you have configured (shared folder, external storage, ...) ?\r\n\r\nLike I said previously, one of the folders (the one called `Dropbox-Enc` you see in the log) is on Dropbox, via the external storage plugin and is also encrypted by ownCloud via the encryption plugin. Inside that folder there is only one file (ArcSin.pdf). All other folders are the default ones that owncloud created itself. All this is in the log.\r\n\r\n> Can you check the owncloud.log on the server for possible errors/warnings?\r\n\r\nGood call! There is an error related to external_storage: `{\"reqId\":\"9f70682ab4c7c24aa2c687dd5d288529\",\"remoteAddr\":\"CLIENT_IP_ADDRESS\",\"app\":\"files_external\",\"message\":\"Error: (401) Unauthorized\\n\",\"level\":3,\"time\":\"2015-03-05T04:35:37+00:00\"}` ... where CLIENT_IP_ADDRESS is my home IP, the one on which the ownCloud MacOSX client runs.\r\n\r\nBut that can't be true ... the external storage sync with the Dropbox-Enc folder works just fine. I just copied another file in \"Dropbox-Enc\" and it was sync'ed just fine to Dropbox, and I can also see it in the web interface. Something else must be going on.\r\n\r\n> Which server version do you use?\r\n\r\nI'm using Ubuntu 14.04.2 with Apache 2.4.12 and PHP 5.5 (both from this PPA: https://launchpad.net/~ondrej/+archive/ubuntu/apache2).\r\n\r\nThe enabled apache modules are: `access_compat actions alias auth_basic auth_digest authn_core authn_file authz_core authz_groupfile authz_host authz_user autoindex cgi cgid dav dav_fs deflate dir env expires filter headers mime mpm_prefork negotiation php5 reqtimeout rewrite setenvif socache_shmcb ssl status`\r\n\r\nThe enabled php5 modules are (note I have opcode caching enabled via opcache):\r\n```\r\n# grep extension= /etc/php5/apache2/conf.d/*\r\n/etc/php5/apache2/conf.d/05-opcache.ini:zend_extension=opcache.so\r\n/etc/php5/apache2/conf.d/10-pdo.ini:extension=pdo.so\r\n/etc/php5/apache2/conf.d/20-curl.ini:extension=curl.so\r\n/etc/php5/apache2/conf.d/20-gd.ini:extension=gd.so\r\n/etc/php5/apache2/conf.d/20-intl.ini:extension=intl.so\r\n/etc/php5/apache2/conf.d/20-json.ini:extension=json.so\r\n/etc/php5/apache2/conf.d/20-mcrypt.ini:extension=mcrypt.so\r\n/etc/php5/apache2/conf.d/20-mysqli.ini:extension=mysqli.so\r\n/etc/php5/apache2/conf.d/20-mysql.ini:extension=mysql.so\r\n/etc/php5/apache2/conf.d/20-pdo_mysql.ini:extension=pdo_mysql.so\r\n/etc/php5/apache2/conf.d/20-pdo_pgsql.ini:extension=pdo_pgsql.so\r\n/etc/php5/apache2/conf.d/20-pdo_sqlite.ini:extension=pdo_sqlite.so\r\n/etc/php5/apache2/conf.d/20-pgsql.ini:extension=pgsql.so\r\n/etc/php5/apache2/conf.d/20-readline.ini:extension=readline.so\r\n/etc/php5/apache2/conf.d/20-sqlite3.ini:extension=sqlite3.so\r\n```\r\n\r\nThe apache conf and htaccess are the default ones. The host conf for the site that owncloud is running under is:\r\n```\r\n<IfModule mod_ssl.c>\r\n<VirtualHost *:443>\r\n\r\n    ServerAdmin admins@DOMAIN.COM\r\n    ServerName  DOMAIN.COM\r\n    ServerAlias www.DOMAIN.COM\r\n\r\n    DocumentRoot        /var/www/DOMAIN.COM\r\n    UseCanonicalName    Off\r\n\r\n    <Directory /var/www/DOMAIN.COM>\r\n        Options FollowSymLinks\r\n        AllowOverride All\r\n        Order allow,deny\r\n        Allow from all\r\n    </Directory>\r\n\r\n    ErrorLog ${APACHE_LOG_DIR}/ssl_error.log\r\n    LogLevel warn\r\n    CustomLog ${APACHE_LOG_DIR}/access.log combined\r\n\r\n    SSLEngine on\r\n    SSLCertificateKeyFile    /etc/ssl/private/DOMAIN.COM.key\r\n    ## Apache 2.4 complains about using two directives for this\r\n    #SSLCertificateFile       /etc/ssl/certs/DOMAIN.COM.pem\r\n    #SSLCertificateChainFile  /etc/ssl/certs/DOMAIN.COM.ca-bundle\r\n    ## Just cat the above two files into one file, the one below\r\n    SSLCertificateFile       /etc/ssl/certs/DOMAIN.COM.full.crt\r\n\r\n</VirtualHost>\r\n</IfModule>\r\n```\r\n\r\n---\r\n\r\n@guruz Here's a full (sanitized) log of 5 minutes from the moment the desktop client starts: https://dl.dropbox.com/s/ntla2jgkp7p3mck/oc3.log\r\n\r\nI now have bigger problems. I added a file to the \"Dropbox-Enc\" subfolder to test if it syncs ... and it does. But if I delete it using Finder, the owncloud client puts it back after 5-10 seconds! Here's a full log of me starting owncloud, then deleting the file `fntoggle` from the \"Dropbox-Enc\" subfolder : https://dl.dropbox.com/s/h6i17n2810uti0c/oc4.log (the deletion happens at 03-06 12:27:00:048, and then the file is restored at 03-06 12:27:06:977)\r\n\r\nDeleting a file from another subfolder, or from the main folder, does not bring it back. It feels like one or more components are flaky (encryption plugin, external storage plugin, desktop client). I don't feel safe using it for my regular work. Maybe this should be a different bug report.\r\n\r\n> Do I understand it well that the steps to reproduce are as follows:\r\n> 1) Upload a file \"test.txt\" to \"Dropbox-Enc\" using the Web UI\r\n> 2) Wait for the sync client to sync it down\r\n> 3) Edit the file using the web UI (or upload+overwrite using the web UI)\r\n> 4) Wait for the sync client to sync it down\r\n>\r\n> But it never syncs it down because the file or folder etag did not change ?\r\n\r\nNot quite. Everything is sync'ed correctly (except deletions, see above). The problem is that the desktop client appears to continuously attempt to sync the folder when it should not.\r\n\r\nWhat I did was as follows\r\n- fresh install and created another admin user\r\n- activated the external storage and encryption plugins\r\n- at dropbox.com I created an app restricted to a single folder and got the API key and passcode\r\n- in the owncloud interface I created a Dropbox external storage using the API key and passcode provided by Dropbox ... this is the sub-folder called \"Dropbox-Enc\" we were speaking of\r\n- added a file to the Dropbox-Enc folder\r\n- the desktop client (at least the MacOSX one) constantly syncs every 60 seconds, causing CPU spikes, see log and my reports in my previous posts here.\r\n\r\nHowever now I have bigger problems (see above).\r\n"},"comment":{"url":"https://api.github.com/repos/owncloud/core/issues/comments/82246768","html_url":"https://github.com/owncloud/core/issues/14873#issuecomment-82246768","issue_url":"https://api.github.com/repos/owncloud/core/issues/14873","id":82246768,"user":{"login":"PVince81","id":277525,"avatar_url":"https://avatars.githubusercontent.com/u/277525?v=3","gravatar_id":"","url":"https://api.github.com/users/PVince81","html_url":"https://github.com/PVince81","followers_url":"https://api.github.com/users/PVince81/followers","following_url":"https://api.github.com/users/PVince81/following{/other_user}","gists_url":"https://api.github.com/users/PVince81/gists{/gist_id}","starred_url":"https://api.github.com/users/PVince81/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/PVince81/subscriptions","organizations_url":"https://api.github.com/users/PVince81/orgs","repos_url":"https://api.github.com/users/PVince81/repos","events_url":"https://api.github.com/users/PVince81/events{/privacy}","received_events_url":"https://api.github.com/users/PVince81/received_events","type":"User","site_admin":false},"created_at":"2015-03-17T09:42:06Z","updated_at":"2015-03-17T09:42:06Z","body":"Okay, I see it here too:\r\n```\r\n{\"reqId\":\"62e8a0864d4d2ef202d714bb16bdbd1a\",\"remoteAddr\":\"127.0.0.1\",\"app\":\"files_external\",\"message\":\"Error: (401) Unauthorized\\n\",\"level\":3,\"time\":\"2015-03-17T09:41:12+00:00\",\"method\":\"PROPFIND\",\"url\":\"\\/owncloud\\/remote.php\\/webdav\\/dropbox\\/Photos\"}\r\n{\"reqId\":\"51216d03e3702c9bdeace3de47b9e32e\",\"remoteAddr\":\"127.0.0.1\",\"app\":\"files_external\",\"message\":\"Error: (401) Unauthorized\\n\",\"level\":3,\"time\":\"2015-03-17T09:41:17+00:00\",\"method\":\"PROPFIND\",\"url\":\"\\/owncloud\\/remote.php\\/webdav\\/dropbox\\/Public\"}\r\n```\r\n\r\nfor every PROPFIND. Maybe Dropbox has an API limit and doesn't let us do more than X calls an hour or something.\r\n"}}