{"action":"created","issue":{"url":"https://api.github.com/repos/OpenGrok/OpenGrok/issues/527","labels_url":"https://api.github.com/repos/OpenGrok/OpenGrok/issues/527/labels{/name}","comments_url":"https://api.github.com/repos/OpenGrok/OpenGrok/issues/527/comments","events_url":"https://api.github.com/repos/OpenGrok/OpenGrok/issues/527/events","html_url":"https://github.com/OpenGrok/OpenGrok/issues/527","id":14607246,"number":527,"title":"If a git repository does not have a valid HEAD indexing breaks (Bugzilla #19163)","user":{"login":"vladak","id":2934284,"avatar_url":"https://avatars.githubusercontent.com/u/2934284?v=3","gravatar_id":"","url":"https://api.github.com/users/vladak","html_url":"https://github.com/vladak","followers_url":"https://api.github.com/users/vladak/followers","following_url":"https://api.github.com/users/vladak/following{/other_user}","gists_url":"https://api.github.com/users/vladak/gists{/gist_id}","starred_url":"https://api.github.com/users/vladak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vladak/subscriptions","organizations_url":"https://api.github.com/users/vladak/orgs","repos_url":"https://api.github.com/users/vladak/repos","events_url":"https://api.github.com/users/vladak/events{/privacy}","received_events_url":"https://api.github.com/users/vladak/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/OpenGrok/OpenGrok/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":27,"created_at":"2013-05-22T07:06:55Z","updated_at":"2015-10-27T17:03:50Z","closed_at":null,"body":"status NEW severity *minor* in component *scm* for *---*\nReported in version *unspecified* on platform *ANY/Generic*\nAssigned to: Trond Norbye\n\nOn 2012-02-05 19:09:36 +0000, Johan wrote:\n\n> phb@opengrok:/var/lib/opengrok/src/opstools-misc$ git log\n> fatal: bad default revision 'HEAD'\n> \n> \n> java.lang.RuntimeException: Failed to acquire random test lock; please verify filesystem for lock directory '/var/opengrok/data/spellIndex/opstools-misc' supports locking\n> \tat org.apache.lucene.store.NativeFSLockFactory.acquireTestLock(NativeFSLockFactory.java:103)\n> \tat org.apache.lucene.store.NativeFSLockFactory.makeLock(NativeFSLockFactory.java:142)\n> \tat org.apache.lucene.store.Directory.makeLock(Directory.java:106)\n> \tat org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1058)\n> \tat org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:882)\n> \tat org.apache.lucene.search.spell.SpellChecker.setSpellIndex(SpellChecker.java:142)\n> \tat org.apache.lucene.search.spell.SpellChecker.<init>(SpellChecker.java:110)\n> \tat org.apache.lucene.search.spell.SpellChecker.<init>(SpellChecker.java:124)\n> \tat org.apache.jsp.search_jsp._jspService(search_jsp.java:689)\n> \tat org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)\n> \tat javax.servlet.http.HttpServlet.service(HttpServlet.java:717)\n> \tat org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:377)\n> \tat org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)\n> \tat org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)\n> \tat javax.servlet.http.HttpServlet.service(HttpServlet.java:717)\n> \tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)\n> \tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)\n> \tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)\n> \tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)\n> \tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)\n> \tat org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)\n> \tat org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)\n> \tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)\n> \tat org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)\n> \tat org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)\n> \tat org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)\n> \tat java.lang.Thread.run(Thread.java:662)\n> Caused by: org.apache.lucene.store.LockReleaseFailedException: Cannot forcefully unlock a NativeFSLock which is held by another indexer component: /var/opengrok/data/spellIndex/opstools-misc/lucene-....--3y5zar-test.lock\n> \tat org.apache.lucene.store.NativeFSLock.release(NativeFSLockFactory.java:329)\n> \tat org.apache.lucene.store.NativeFSLockFactory.acquireTestLock(NativeFSLockFactory.java:95)\n> \t... 26 more\n> \n> phb@opengrok:~$ ls -al /var/opengrok/data/spellIndex/opstools-misc\n> total 24\n> drwxr-xr-x   2 opengrok opengrok  4096 Jan 31 02:33 .\n> drwxr-xr-x 548 opengrok opengrok 20480 Feb  4 02:34 ..\n> \n> Workaround is simple, ensure no repositories like that exist in the directory that is being indexed. (We use git show-ref -q --verify refs/remotes/origin/master )\n\nOn 2012-02-06 08:28:43 +0000, Lubos Kosco wrote:\n\n> this is a bit weird\n> I think it shouldn't happen with latest binaries\n> \n> afaik our executors have a timeout on forked commands\n> \n> below stack shows this: an indexer is stuck indexing \"opstools-misc\"\n> and another indexer was started in between (within maybe the timeout of above)\n> \n> so basically you have 2 indexers running on the same box and they seem to clash, check ps axf (or ptree) and let us know\n> \n> + locking on lucene is supposed to be automatic, I removed forced options on lock clean ups because of that (and no one complained, so you're the first one)\n> \n> the other Q can be, whether you use latest opengrok ;)\n> \n> hmm?\n> L\n> \n> note that current 0.11rc and 0.10 didn't have big changes in this area, so the locking should be the same, however older opengrok versions had old locking, which was problematic sometimes\n\nOn 2012-02-06 09:43:10 +0000, Johan wrote:\n\n> \n> (In reply to comment # 1)\n> > this is a bit weird\n> > I think it shouldn't happen with latest binaries\n> > \n> It is using opengrok 0.11, or at the very least 0.10 if I failed the upgrade somehow (There seems to be no UI-visible version # to verify?)\n> \n> > afaik our executors have a timeout on forked commands\n> > \n> > below stack shows this: an indexer is stuck indexing \"opstools-misc\"\n> > and another indexer was started in between (within maybe the timeout of above)\n> > \n> > so basically you have 2 indexers running on the same box and they seem to\n> > clash, check ps axf (or ptree) and let us know\n> > \n> No indexers running at all. It happened for us on 5 repositories, all exhibiting the above problem, so I'm really suspecting something w.r.t the git scm history.\n> \n> All the other directories under /var/opengrok/data/spellIndex has a couple of files (\n> 672 -rw-r--r--   1 opengrok opengrok 684629 Nov 28 23:07 _0.cfs\n> 128 -rw-r--r--   1 opengrok opengrok 130345 Nov 28 23:07 _0.cfx\n>   4 -rw-r--r--   1 opengrok opengrok     20 Nov 28 23:07 segments.gen\n>   4 -rw-r--r--   1 opengrok opengrok    246 Nov 28 23:07 segment)\n> > + locking on lucene is supposed to be automatic, I removed forced options on\n> > lock clean ups because of that (and no one complained, so you're the first one)\n> > \n> > the other Q can be, whether you use latest opengrok ;)\n> Well, most probably 0.11rc2 unless my installation failed and then it would be running 0.10.\n> \n> > \n> > hmm?\n> > L\n> > \n> > note that current 0.11rc and 0.10 didn't have big changes in this area, so the\n> > locking should be the same, however older opengrok versions had old locking,\n> > which was problematic sometimes\n\n"},"comment":{"url":"https://api.github.com/repos/OpenGrok/OpenGrok/issues/comments/151570702","html_url":"https://github.com/OpenGrok/OpenGrok/issues/527#issuecomment-151570702","issue_url":"https://api.github.com/repos/OpenGrok/OpenGrok/issues/527","id":151570702,"user":{"login":"tarzanek","id":504773,"avatar_url":"https://avatars.githubusercontent.com/u/504773?v=3","gravatar_id":"","url":"https://api.github.com/users/tarzanek","html_url":"https://github.com/tarzanek","followers_url":"https://api.github.com/users/tarzanek/followers","following_url":"https://api.github.com/users/tarzanek/following{/other_user}","gists_url":"https://api.github.com/users/tarzanek/gists{/gist_id}","starred_url":"https://api.github.com/users/tarzanek/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tarzanek/subscriptions","organizations_url":"https://api.github.com/users/tarzanek/orgs","repos_url":"https://api.github.com/users/tarzanek/repos","events_url":"https://api.github.com/users/tarzanek/events{/privacy}","received_events_url":"https://api.github.com/users/tarzanek/received_events","type":"User","site_admin":false},"created_at":"2015-10-27T17:03:50Z","updated_at":"2015-10-27T17:03:50Z","body":"even with latest?\r\nok, so you must have some file that triggers this behaviour, can you check which file gets indexed when you get OOM ?"}}