{"action":"created","issue":{"url":"https://api.github.com/repos/KylinOLAP/Kylin/issues/101","labels_url":"https://api.github.com/repos/KylinOLAP/Kylin/issues/101/labels{/name}","comments_url":"https://api.github.com/repos/KylinOLAP/Kylin/issues/101/comments","events_url":"https://api.github.com/repos/KylinOLAP/Kylin/issues/101/events","html_url":"https://github.com/KylinOLAP/Kylin/issues/101","id":49980836,"number":101,"title":"Can't get cube source record size.","user":{"login":"Yancey1989","id":1426912,"avatar_url":"https://avatars.githubusercontent.com/u/1426912?v=3","gravatar_id":"","url":"https://api.github.com/users/Yancey1989","html_url":"https://github.com/Yancey1989","followers_url":"https://api.github.com/users/Yancey1989/followers","following_url":"https://api.github.com/users/Yancey1989/following{/other_user}","gists_url":"https://api.github.com/users/Yancey1989/gists{/gist_id}","starred_url":"https://api.github.com/users/Yancey1989/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Yancey1989/subscriptions","organizations_url":"https://api.github.com/users/Yancey1989/orgs","repos_url":"https://api.github.com/users/Yancey1989/repos","events_url":"https://api.github.com/users/Yancey1989/events{/privacy}","received_events_url":"https://api.github.com/users/Yancey1989/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/KylinOLAP/Kylin/labels/help%20wanted","name":"help wanted","color":"159818"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2014-11-25T06:22:13Z","updated_at":"2015-05-27T10:54:46Z","closed_at":"2014-11-28T01:18:52Z","body":"Dear All!\r\nI deploy kylin on Apache Hadoop, when i build the job ,  it failed on the last step. here is the some log message:\r\n```java\r\n[QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,280][INFO][com.kylinolap.job.flow.JobFlowListener.jobWasExecuted(JobFlowListener.java:93)] - cube_job_group.test_kylin_cube_without_slr_empty.30bbd69c-33ae-4983-aaf7-f46ab42b88dd.14 status: FINISHED\r\n[QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,280][DEBUG][com.kylinolap.cube.CubeManager.loadCubeInstance(CubeManager.java:604)] - Loading CubeInstance kylin_metadata_qa(key='/cube/test_kylin_cube_without_slr_empty.json')@kylin_metadata_qa@hbase:192.168.44.21:2181:/hbase9827\r\n[QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,295][DEBUG][com.kylinolap.common.persistence.ResourceStore.putResource(ResourceStore.java:166)] - Saving resource /job/30bbd69c-33ae-4983-aaf7-f46ab42b88dd (Store kylin_metadata_qa@hbase:192.168.44.21:2181:/hbase9827)\r\n[QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,300][INFO][com.kylinolap.job.flow.JobFlowListener.updateCubeSegmentInfoOnSucceed(JobFlowListener.java:238)] - Updating cube segment FULL_BUILD for cube test_kylin_cube_without_slr_empty\r\n[QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,301][ERROR][com.kylinolap.job.flow.JobFlowListener.jobWasExecuted(JobFlowListener.java:117)] - Can't get cube source record size.\r\njava.lang.RuntimeException: Can't get cube source record size.\r\n        at com.kylinolap.job.flow.JobFlowListener.updateCubeSegmentInfoOnSucceed(JobFlowListener.java:274)\r\n        at com.kylinolap.job.flow.JobFlowListener.jobWasExecuted(JobFlowListener.java:99)\r\n        at org.quartz.core.QuartzScheduler.notifyJobListenersWasExecuted(QuartzScheduler.java:1985)\r\n        at org.quartz.core.JobRunShell.notifyJobListenersComplete(JobRunShell.java:340)\r\n        at org.quartz.core.JobRunShell.run(JobRunShell.java:224)\r\n        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)\r\n```\r\nhere is some version info:\r\nKylin 0.6.2\r\nhadoop 2.4.1\r\nhbase 0.98.4-hadoop2\r\nzookeeper 3.4.5\r\nhive 0.13.0\r\n\r\nI find some config info on hbase\r\n```\r\nget 'kylin_metadata_qa_job', '/job/1eaf05f8-e9f5-4dd7-9748-ab9b3098a512'\r\n{\r\n    \"uuid\": \"1eaf05f8-e9f5-4dd7-9748-ab9b3098a512\",\r\n    \"name\": \"test_kylin_cube_without_slr_empty - FULL_BUILD - BUILD - PST 2014-11-24 23:00:15\",\r\n    \"type\": \"BUILD\",\r\n    \"duration\": 461,\r\n    \"steps\": [\r\n        {\r\n            \"name\": \"Create Intermediate Flat Hive Table\",\r\n            \"info\": null,\r\n            \"interruptCmd\": null,\r\n            \"sequence_id\": 0,\r\n            \"exec_cmd\": \"hive -e \"DROPTABLEIFEXISTSkylin_intermediate_test_kylin_cube_without_slr_desc_FULL_BUILD_1eaf05f8_e9f5_4dd7_9748_ab9b3098a512;nCREATEEXTERNALTABLEIFNOTEXISTSkylin_intermediate_test_kylin_cube_without_slr_desc_FULL_BUILD_1eaf05f8_e9f5_4dd7_9748_ab9b3098a512n(nCAL_DTdaten,\r\n            LEAF_CATEG_IDintn,\r\n            LSTG_SITE_IDintn,\r\n            META_CATEG_NAMEstringn,\r\n            CATEG_LVL2_NAMEstringn,\r\n            CATEG_LVL3_NAMEstringn,\r\n            LSTG_FORMAT_NAMEstringn,\r\n            SLR_SEGMENT_CDsmallintn,\r\n            PRICEdecimal(38,\r\n            16)n,\r\n            SELLER_IDbigintn)\r\n...}\r\n]\r\n```\r\nin source code: com.kylinolap.job.flow.JobFlowListener.updateCubeSegmentInfoOnSucceed(JobFlowListener.java:272)\r\n```\r\n               JobStep createFlatTableStep = jobInstance.findStep(JobConstants.STEP_NAME_CREATE_FLAT_HIVE_TABLE);\r\n                if (null != createFlatTableStep) {\r\n                    String sourceRecordsSize = createFlatTableStep.getInfo(JobInstance.SOURCE_RECORDS_SIZE);\r\n                    if (sourceRecordsSize == null || sourceRecordsSize.equals(\"\")) {\r\n                        throw new RuntimeException(\"Can't get cube source record size.\");\r\n                    }\r\n                    sourceSize = Long.parseLong(sourceRecordsSize);\r\n                } else {\r\n                    log.info(\"No step with name '\" + JobConstants.STEP_NAME_CREATE_FLAT_HIVE_TABLE + \"' is found\");\r\n                }\r\n```\r\n\r\nthe \"info\" is null, cased throw a RuntimeException,  some one have idea with it??"},"comment":{"url":"https://api.github.com/repos/KylinOLAP/Kylin/issues/comments/105867254","html_url":"https://github.com/KylinOLAP/Kylin/issues/101#issuecomment-105867254","issue_url":"https://api.github.com/repos/KylinOLAP/Kylin/issues/101","id":105867254,"user":{"login":"vonsuu","id":2961656,"avatar_url":"https://avatars.githubusercontent.com/u/2961656?v=3","gravatar_id":"","url":"https://api.github.com/users/vonsuu","html_url":"https://github.com/vonsuu","followers_url":"https://api.github.com/users/vonsuu/followers","following_url":"https://api.github.com/users/vonsuu/following{/other_user}","gists_url":"https://api.github.com/users/vonsuu/gists{/gist_id}","starred_url":"https://api.github.com/users/vonsuu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vonsuu/subscriptions","organizations_url":"https://api.github.com/users/vonsuu/orgs","repos_url":"https://api.github.com/users/vonsuu/repos","events_url":"https://api.github.com/users/vonsuu/events{/privacy}","received_events_url":"https://api.github.com/users/vonsuu/received_events","type":"User","site_admin":false},"created_at":"2015-05-27T10:54:46Z","updated_at":"2015-05-27T10:54:46Z","body":"Hi, \r\n\r\nI'am trying to build my first cube using Apache Kylin, everything goes fine until last step where I'm getting error:\r\n\r\n    java.lang.IllegalStateException: Can't get cube source record count.\r\n\tat com.google.common.base.Preconditions.checkState(Preconditions.java:149)\r\n\tat org.apache.kylin.job.cube.UpdateCubeInfoAfterBuildStep.doWork(UpdateCubeInfoAfterBuildStep.java:104)\r\n\tat org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)\r\n\tat org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)\r\n\tat org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)\r\n\tat org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132)\r\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\r\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\n\r\nAccording to this issue above error occurs because Kylin attempt to find this pattern in hive's output: \"HDFS Read: (\\d+) HDFS Write: (\\d+) SUCCESS\".\r\n\r\nCorrect output from cube building tutorial: github.com/KylinOLAP/Kylin/wiki/Kylin-Cube-Build-and-Job-Monitoring-Tutorial:\r\n\r\nhttp://i.imgur.com/71tEeIy.png\r\n\r\nMy output in Kylin looks corrupted:\r\n\r\nhttp://i.imgur.com/GCbrN03.png\r\n\r\nHowever when I check Hive log it looks ok:\r\n\r\n    2015-05-27 08:40:13,419 INFO  [main]: ql.Driver (Driver.java:execute(1285)) - Starting command: \r\n    INSERT OVERWRITE TABLE kylin_intermediate_Kubek_19700101000000_2922789940817071255_f23ac1b1_10fe_4112_ac9e_b4e6baf07654 SELECT\r\n    FACT_TABLE.DATE\r\n    ,FACT_TABLE.MONEY_ADVERTISER\r\n    ,FACT_TABLE.MONEY_PUBLISHER\r\n    FROM DEFAULT.ADVSTATS as FACT_TABLE \r\n    ...\r\n    2015-05-27 08:45:05,132 INFO  [main]: ql.Driver (SessionState.java:printInfo(824)) - MapReduce Jobs Launched: \r\n    2015-05-27 08:45:05,148 INFO  [main]: ql.Driver (SessionState.java:printInfo(824)) - Stage-Stage-1: Map: 1   Cumulative CPU: 17.32 sec   HDFS Read: 44644035 HDFS Write: 2347008 SUCCESS\r\n    2015-05-27 08:45:05,153 INFO  [main]: ql.Driver (SessionState.java:printInfo(824)) - Total MapReduce CPU Time Spent: 17 seconds 320 msec\r\n    2015-05-27 08:45:05,167 INFO  [main]: ql.Driver (SessionState.java:printInfo(824)) - OK\r\n\r\nI'm using Hortonworks Sandbox 2.2:\r\n\r\n    hadoop-2.6.0\r\n    hbase-0.98.12\r\n    hive-0.14.0\r\n    zookeeper-3.4.6\r\n\r\nCan someone tell me why my kylin logs preview looks like that and first of all is that could be a reason of last step error?\r\n\r\n\r\n\r\n"}}