{"action":"created","issue":{"url":"https://api.github.com/repos/geonetwork/core-geonetwork/issues/731","labels_url":"https://api.github.com/repos/geonetwork/core-geonetwork/issues/731/labels{/name}","comments_url":"https://api.github.com/repos/geonetwork/core-geonetwork/issues/731/comments","events_url":"https://api.github.com/repos/geonetwork/core-geonetwork/issues/731/events","html_url":"https://github.com/geonetwork/core-geonetwork/issues/731","id":55411642,"number":731,"title":"Too many open files","user":{"login":"fxprunayre","id":1701393,"avatar_url":"https://avatars.githubusercontent.com/u/1701393?v=3","gravatar_id":"","url":"https://api.github.com/users/fxprunayre","html_url":"https://github.com/fxprunayre","followers_url":"https://api.github.com/users/fxprunayre/followers","following_url":"https://api.github.com/users/fxprunayre/following{/other_user}","gists_url":"https://api.github.com/users/fxprunayre/gists{/gist_id}","starred_url":"https://api.github.com/users/fxprunayre/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fxprunayre/subscriptions","organizations_url":"https://api.github.com/users/fxprunayre/orgs","repos_url":"https://api.github.com/users/fxprunayre/repos","events_url":"https://api.github.com/users/fxprunayre/events{/privacy}","received_events_url":"https://api.github.com/users/fxprunayre/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/geonetwork/core-geonetwork/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/geonetwork/core-geonetwork/milestones/12","labels_url":"https://api.github.com/repos/geonetwork/core-geonetwork/milestones/12/labels","id":679090,"number":12,"title":"3.0.0","description":"","creator":{"login":"fxprunayre","id":1701393,"avatar_url":"https://avatars.githubusercontent.com/u/1701393?v=3","gravatar_id":"","url":"https://api.github.com/users/fxprunayre","html_url":"https://github.com/fxprunayre","followers_url":"https://api.github.com/users/fxprunayre/followers","following_url":"https://api.github.com/users/fxprunayre/following{/other_user}","gists_url":"https://api.github.com/users/fxprunayre/gists{/gist_id}","starred_url":"https://api.github.com/users/fxprunayre/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fxprunayre/subscriptions","organizations_url":"https://api.github.com/users/fxprunayre/orgs","repos_url":"https://api.github.com/users/fxprunayre/repos","events_url":"https://api.github.com/users/fxprunayre/events{/privacy}","received_events_url":"https://api.github.com/users/fxprunayre/received_events","type":"User","site_admin":false},"open_issues":8,"closed_issues":22,"state":"open","created_at":"2014-06-03T12:57:24Z","updated_at":"2015-01-25T11:37:18Z","due_on":"2015-03-15T23:00:00Z","closed_at":null},"comments":5,"created_at":"2015-01-25T11:37:18Z","updated_at":"2015-01-25T20:35:38Z","closed_at":null,"body":"I get quite often too many open files exceptions. \r\n\r\neg. while indexing catalog with 700 records\r\n```\r\n\r\n\t Marking the metadata as _indexingError=1 in index\r\n2015-01-25 12:30:13,138 ERROR [geonetwork.index] - Failed to properly index geometry of metadata 400. Error: /home/francois/dev/eea-geonetwork/web/src/main/webapp/WEB-INF/data/index/spatialindex.dbf (Too many open files)\r\njava.io.FileNotFoundException: /home/francois/dev/eea-geonetwork/web/src/main/webapp/WEB-INF/data/index/spatialindex.dbf (Too many open files)\r\n\tat java.io.RandomAccessFile.open(Native Method)\r\n\tat java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)\r\n\tat org.geotools.data.shapefile.ShpFiles.getReadChannel(ShpFiles.java:831)\r\n\tat org.geotools.data.shapefile.dbf.DbaseFileReader.<init>(DbaseFileReader.java:139)\r\n\tat org.geotools.data.shapefile.dbf.IndexedDbaseFileReader.<init>(IndexedDbaseFileReader.java:149)\r\n\tat org.geotools.data.shapefile.indexed.IndexedShapefileDataStore.openDbfReader(IndexedShapefileDataStore.java:859)\r\n\tat org.geotools.data.shapefile.indexed.IndexedShapefileDataStore.getAttributesReader(IndexedShapefileDataStore.java:561)\r\n\tat org.geotools.data.shapefile.indexed.IndexedShapefileDataStore.createFeatureWriter(IndexedShapefileDataStore.java:965)\r\n\tat org.geotools.data.AbstractDataStore.getFeatureWriter(AbstractDataStore.java:470)\r\n\tat org.geotools.data.AbstractFeatureStore.removeFeatures(AbstractFeatureStore.java:361)\r\n\tat org.fao.geonet.ker\r\n\r\n```\r\n\r\nIt also happens when using the catalog - viewing records.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/geonetwork/core-geonetwork/issues/comments/71391873","html_url":"https://github.com/geonetwork/core-geonetwork/issues/731#issuecomment-71391873","issue_url":"https://api.github.com/repos/geonetwork/core-geonetwork/issues/731","id":71391873,"user":{"login":"jesseeichar","id":54207,"avatar_url":"https://avatars.githubusercontent.com/u/54207?v=3","gravatar_id":"","url":"https://api.github.com/users/jesseeichar","html_url":"https://github.com/jesseeichar","followers_url":"https://api.github.com/users/jesseeichar/followers","following_url":"https://api.github.com/users/jesseeichar/following{/other_user}","gists_url":"https://api.github.com/users/jesseeichar/gists{/gist_id}","starred_url":"https://api.github.com/users/jesseeichar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jesseeichar/subscriptions","organizations_url":"https://api.github.com/users/jesseeichar/orgs","repos_url":"https://api.github.com/users/jesseeichar/repos","events_url":"https://api.github.com/users/jesseeichar/events{/privacy}","received_events_url":"https://api.github.com/users/jesseeichar/received_events","type":"User","site_admin":false},"created_at":"2015-01-25T20:35:38Z","updated_at":"2015-01-25T20:35:38Z","body":"I have added 2 methods to IO (newInputStream and newBufferedReader) and made all calls of Files.newInputStream or newBufferedReader be changed to use those methods.  In those methods I wrap the inputstream/reader with the Debugging equivalents (DebuggingReader and DebuggingInputStream).  \r\n\r\nThese classes register an exception with the OpenResourceTracker class on creation and remove the exception on closing.  Thus the OpenResourceTracker maintains a list of all the open resources (along with the file name) and when there are 1000 open files it will write to the log all of the open files and the stacktrace at the point when they were opened..\r\n\r\n\r\nNow that I think about it, it will kill the system at that point because each time an resource is opened after 1000 then all open files will be logged repeatedly.  essentially crashing the system.  Maybe it should be changed to only print once.\r\n\r\nIf you get a linux error you can still:\r\n\r\n* put a break point in ... ServiceManager for example.  \r\n* open the expression evaluator dialog\r\n* execute: `System.out.println(org.fao.geonet.utils.debug.OpenResourceTracker.openExceptions.size())`\r\n* execute: `org.fao.geonet.utils.debug.OpenResourceTracker.printExceptions()`\r\n\r\n"}}