{"action":"created","issue":{"url":"https://api.github.com/repos/chrismattmann/drat/issues/39","labels_url":"https://api.github.com/repos/chrismattmann/drat/issues/39/labels{/name}","comments_url":"https://api.github.com/repos/chrismattmann/drat/issues/39/comments","events_url":"https://api.github.com/repos/chrismattmann/drat/issues/39/events","html_url":"https://github.com/chrismattmann/drat/issues/39","id":84130106,"number":39,"title":"java.lang.RuntimeException: Pge didn't finish successfully","user":{"login":"lewismc","id":1165719,"avatar_url":"https://avatars.githubusercontent.com/u/1165719?v=3","gravatar_id":"","url":"https://api.github.com/users/lewismc","html_url":"https://github.com/lewismc","followers_url":"https://api.github.com/users/lewismc/followers","following_url":"https://api.github.com/users/lewismc/following{/other_user}","gists_url":"https://api.github.com/users/lewismc/gists{/gist_id}","starred_url":"https://api.github.com/users/lewismc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lewismc/subscriptions","organizations_url":"https://api.github.com/users/lewismc/orgs","repos_url":"https://api.github.com/users/lewismc/repos","events_url":"https://api.github.com/users/lewismc/events{/privacy}","received_events_url":"https://api.github.com/users/lewismc/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-06-02T18:00:11Z","updated_at":"2015-06-03T22:36:06Z","closed_at":null,"body":"I am getting a runtime exception and subsequent failure of the entire **go** task using master branch as of today.\r\n@jihyunoh can you please review and see if it is/was the same or similar issue to that you were getting a few weeks ago...\r\n```\r\nINFO: Loading PgeConfig file '/usr/local/drat/deploy//pge/policy/PgeConfig_MimePartitioner.xml'\r\nJun 02, 2015 10:44:30 AM org.apache.oodt.cas.pge.PGETaskInstance createExeDir\r\nINFO: Creating PGE execution working directory: [/usr/local/drat/deploy//data/jobs/mimepartitioner/1433267070660]\r\nJun 02, 2015 10:44:30 AM org.apache.oodt.cas.pge.PGETaskInstance updateStatus\r\nINFO: Updating status to workflow as BUILDING CONFIG FILE\r\nJun 02, 2015 10:44:30 AM org.apache.oodt.cas.pge.PGETaskInstance updateStatus\r\nINFO: Updating status to workflow as PGE EXEC\r\nJun 02, 2015 10:44:30 AM org.apache.oodt.commons.io.LoggerOutputStream flush\r\nINFO: OUTPUT: Creating working dirs\r\nOUTPUT: Running MIME partitioner\r\n\r\nJun 02, 2015 10:44:30 AM org.apache.oodt.commons.io.LoggerOutputStream flush\r\nSEVERE: ERROR:   File \"/usr/local/drat/deploy//pge/bin/mime_partitioner/mime_rat_partitioner.py\", line 111\r\nERROR: SyntaxError: Non-ASCII character '\\xe2' in file /usr/local/drat/deploy//pge/bin/mime_partitioner/mime_rat_partitioner.py on line 111, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details\r\n\r\nJun 02, 2015 10:44:30 AM org.apache.oodt.commons.io.LoggerOutputStream flush\r\nINFO: OUTPUT: Creating working dirs\r\nOUTPUT: Running MIME partitioner\r\n\r\njava.lang.RuntimeException: Pge didn't finish successfully\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.runPge(PGETaskInstance.java:322)\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:451)\r\n\tat org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.executeTaskLocally(IterativeWorkflowProcessorThread.java:574)\r\n\tat org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.run(IterativeWorkflowProcessorThread.java:321)\r\n\tat EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\norg.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceException: PGETask failed : Exception when executing PGE commands '[export PATH=$HOME/bin/:${PATH}, shopt -s expand_aliases, echo \"Creating working dirs\", mkdir /usr/local/drat/deploy//data/jobs/mimepartitioner/1433267070660/input ; mkdir /usr/local/drat/deploy//data/jobs/mimepartitioner/1433267070660/output; mkdir /usr/local/drat/deploy//data/jobs/mimepartitioner/1433267070660/logs, echo \"Running MIME partitioner\", /usr/local/drat/deploy//pge/bin/mime_partitioner/mime_rat_partitioner.py -u http://localhost:8080/solr/drat -c 100 -w http://localhost:9001 -t urn:drat:RatCodeAudit > /usr/local/drat/deploy//data/jobs/mimepartitioner/1433267070660/logs/mime-partitioner-1433267070660.log]' : Pge didn't finish successfully\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:456)\r\n\tat org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.executeTaskLocally(IterativeWorkflowProcessorThread.java:574)\r\n\tat org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.run(IterativeWorkflowProcessorThread.java:321)\r\n\tat EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\nCaused by: java.lang.Exception: Exception when executing PGE commands '[export PATH=$HOME/bin/:${PATH}, shopt -s expand_aliases, echo \"Creating working dirs\", mkdir /usr/local/drat/deploy//data/jobs/mimepartitioner/1433267070660/input ; mkdir /usr/local/drat/deploy//data/jobs/mimepartitioner/1433267070660/output; mkdir /usr/local/drat/deploy//data/jobs/mimepartitioner/1433267070660/logs, echo \"Running MIME partitioner\", /usr/local/drat/deploy//pge/bin/mime_partitioner/mime_rat_partitioner.py -u http://localhost:8080/solr/drat -c 100 -w http://localhost:9001 -t urn:drat:RatCodeAudit > /usr/local/drat/deploy//data/jobs/mimepartitioner/1433267070660/logs/mime-partitioner-1433267070660.log]' : Pge didn't finish successfully\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.runPge(PGETaskInstance.java:334)\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:451)\r\n\t... 4 more\r\nJun 02, 2015 10:44:30 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally\r\nWARNING: Exception executing task: [MimePartitioner] locally: Message: PGETask failed : Exception when executing PGE commands '[export PATH=$HOME/bin/:${PATH}, shopt -s expand_aliases, echo \"Creating working dirs\", mkdir /usr/local/drat/deploy//data/jobs/mimepartitioner/1433267070660/input ; mkdir /usr/local/drat/deploy//data/jobs/mimepartitioner/1433267070660/output; mkdir /usr/local/drat/deploy//data/jobs/mimepartitioner/1433267070660/logs, echo \"Running MIME partitioner\", /usr/local/drat/deploy//pge/bin/mime_partitioner/mime_rat_partitioner.py -u http://localhost:8080/solr/drat -c 100 -w http://localhost:9001 -t urn:drat:RatCodeAudit > /usr/local/drat/deploy//data/jobs/mimepartitioner/1433267070660/logs/mime-partitioner-1433267070660.log]' : Pge didn't finish successfully\r\nJun 02, 2015 10:44:36 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata\r\nINFO: Task: [RatAggregator] has no required metadata fields\r\nJun 02, 2015 10:44:36 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally\r\nINFO: Executing task: [RatAggregator] locally\r\nJun 02, 2015 10:44:36 AM org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder buildImports\r\nINFO: Loading PgeConfig file '/usr/local/drat/deploy//pge/policy/PgeConfig_RatAggregator.xml'\r\nJun 02, 2015 10:44:37 AM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog paginateQuery\r\nWARNING: Query: [q=] for Product Type: [urn:drat:RatLog] returned no results\r\njava.lang.NullPointerException\r\n\tat org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.complexQuery(XmlRpcFileManager.java:607)\r\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\r\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\r\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\r\n\tat java.lang.reflect.Method.invoke(Method.java:606)\r\n\tat org.apache.xmlrpc.Invoker.execute(Invoker.java:130)\r\n\tat org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)\r\n\tat org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)\r\n\tat org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)\r\n\tat org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)\r\n\tat org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)\r\n\tat org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\norg.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed to perform complex query : null\r\n\tat org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)\r\n\tat org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)\r\n\tat org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)\r\n\tat org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)\r\n\tat org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)\r\n\tat org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)\r\n\tat org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery(XmlRpcFileManagerClient.java:1003)\r\n\tat org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.fillIn(XmlFilePgeConfigBuilder.java:322)\r\n\tat org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.fillIn(XmlFilePgeConfigBuilder.java:313)\r\n\tat org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.getCustomMetadata(XmlFilePgeConfigBuilder.java:180)\r\n\tat org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:109)\r\n\tat org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.buildImports(XmlFilePgeConfigBuilder.java:101)\r\n\tat org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:69)\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.java:133)\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.initialize(PGETaskInstance.java:98)\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:449)\r\n\tat org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.executeTaskLocally(IterativeWorkflowProcessorThread.java:574)\r\n\tat org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.run(IterativeWorkflowProcessorThread.java:321)\r\n\tat EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\njava.lang.Exception: Failed to parse value: SQL(FORMAT='$FileLocation/$Filename',SORT_BY='CAS.ProductReceivedTime'){SELECT FileLocation,Filename,CAS.ProductReceivedTime FROM RatLog}\r\n\tat org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.fillIn(XmlFilePgeConfigBuilder.java:326)\r\n\tat org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.fillIn(XmlFilePgeConfigBuilder.java:313)\r\n\tat org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.getCustomMetadata(XmlFilePgeConfigBuilder.java:180)\r\n\tat org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:109)\r\n\tat org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.buildImports(XmlFilePgeConfigBuilder.java:101)\r\n\tat org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:69)\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.java:133)\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.initialize(PGETaskInstance.java:98)\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:449)\r\n\tat org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.executeTaskLocally(IterativeWorkflowProcessorThread.java:574)\r\n\tat org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.run(IterativeWorkflowProcessorThread.java:321)\r\n\tat EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\nCaused by: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed to perform complex query : null\r\n\tat org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery(XmlRpcFileManagerClient.java:1009)\r\n\tat org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.fillIn(XmlFilePgeConfigBuilder.java:322)\r\n\t... 12 more\r\njava.io.IOException: Failed to build PgeConfig : Failed to parse value: SQL(FORMAT='$FileLocation/$Filename',SORT_BY='CAS.ProductReceivedTime'){SELECT FileLocation,Filename,CAS.ProductReceivedTime FROM RatLog}\r\n\tat org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:75)\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.java:133)\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.initialize(PGETaskInstance.java:98)\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:449)\r\n\tat org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.executeTaskLocally(IterativeWorkflowProcessorThread.java:574)\r\n\tat org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.run(IterativeWorkflowProcessorThread.java:321)\r\n\tat EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\norg.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceException: PGETask failed : Failed to instanciate PGETaskInstance : Failed to build PgeConfig : Failed to parse value: SQL(FORMAT='$FileLocation/$Filename',SORT_BY='CAS.ProductReceivedTime'){SELECT FileLocation,Filename,CAS.ProductReceivedTime FROM RatLog}\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:456)\r\n\tat org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.executeTaskLocally(IterativeWorkflowProcessorThread.java:574)\r\n\tat org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.run(IterativeWorkflowProcessorThread.java:321)\r\n\tat EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\nCaused by: java.lang.InstantiationException: Failed to instanciate PGETaskInstance : Failed to build PgeConfig : Failed to parse value: SQL(FORMAT='$FileLocation/$Filename',SORT_BY='CAS.ProductReceivedTime'){SELECT FileLocation,Filename,CAS.ProductReceivedTime FROM RatLog}\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.initialize(PGETaskInstance.java:116)\r\n\tat org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:449)\r\n\t... 4 more\r\nJun 02, 2015 10:44:37 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally\r\nWARNING: Exception executing task: [RatAggregator] locally: Message: PGETask failed : Failed to instanciate PGETaskInstance : Failed to build PgeConfig : Failed to parse value: SQL(FORMAT='$FileLocation/$Filename',SORT_BY='CAS.ProductReceivedTime'){SELECT FileLocation,Filename,CAS.ProductReceivedTime FROM RatLog}\r\nJun 02, 2015 10:45:25 AM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog paginateQuery\r\nWARNING: Query: [q=] for Product Type: [urn:drat:RatAggregateLog] returned no results\r\n```"},"comment":{"url":"https://api.github.com/repos/chrismattmann/drat/issues/comments/108633499","html_url":"https://github.com/chrismattmann/drat/issues/39#issuecomment-108633499","issue_url":"https://api.github.com/repos/chrismattmann/drat/issues/39","id":108633499,"user":{"login":"jihyunoh","id":11444103,"avatar_url":"https://avatars.githubusercontent.com/u/11444103?v=3","gravatar_id":"","url":"https://api.github.com/users/jihyunoh","html_url":"https://github.com/jihyunoh","followers_url":"https://api.github.com/users/jihyunoh/followers","following_url":"https://api.github.com/users/jihyunoh/following{/other_user}","gists_url":"https://api.github.com/users/jihyunoh/gists{/gist_id}","starred_url":"https://api.github.com/users/jihyunoh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jihyunoh/subscriptions","organizations_url":"https://api.github.com/users/jihyunoh/orgs","repos_url":"https://api.github.com/users/jihyunoh/repos","events_url":"https://api.github.com/users/jihyunoh/events{/privacy}","received_events_url":"https://api.github.com/users/jihyunoh/received_events","type":"User","site_admin":false},"created_at":"2015-06-03T22:36:06Z","updated_at":"2015-06-03T22:36:06Z","body":"@lewismc Unfortunately, I reinstalled DRAT, so I don't keep the previous log file... But if DRAT runs successfully, it shows this message:\r\n\r\nMay 27, 2015 1:01:12 PM org.apache.oodt.commons.io.LoggerOutputStream flush\r\nINFO: OUTPUT: Creating working dirs\r\nOUTPUT: Running MIME partitioner\r\n\r\nMay 27, 2015 1:01:12 PM org.apache.oodt.cas.pge.PGETaskInstance runPge\r\nINFO: Successfully completed running: '[export PATH=$HOME/bin/:${PATH}, shopt -s expand_aliases, echo \"Creating working dirs\", mkdir /Users/ohjihyun/drat/deploy/data/jobs/mimepartitioner/1432746072346/input ; mkdir /Users/ohjihyun/drat/deploy/data/jobs/mimepartitioner/1432746072346/output; mkdir /Users/ohjihyun/drat/deploy/data/jobs/mimepartitioner/1432746072346/logs, echo \"Running MIME partitioner\", /Users/ohjihyun/drat/deploy/pge/bin/mime_partitioner/mime_rat_partitioner.py -u http://localhost:8080/solr/drat -c 100 -w http://localhost:9001 -t urn:drat:RatCodeAudit > /Users/ohjihyun/drat/deploy/data/jobs/mimepartitioner/1432746072346/logs/mime-partitioner-1432746072346.log]'\r\nMay 27, 2015 1:01:12 PM org.apache.oodt.cas.pge.PGETaskInstance setCrawlerConfigurations\r\nINFO: Passing Workflow Metadata to CAS-Crawler as global metadata . . ."}}