{"push_id":612262517,"size":3,"distinct_size":3,"ref":"refs/heads/master","head":"d8ccf655f344eed65cdaf5d9252f1b565b8406ca","before":"7215aa745590a3eec9c1ff35d28194235a550db7","commits":[{"sha":"73348012d4ce6c9db85dfb48d51026efe5051c73","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-6428][SQL] Added explicit types for all public methods in catalyst\n\nI think after this PR, we can finally turn the rule on. There are still some smaller ones that need to be fixed, but those are easier.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #5162 from rxin/catalyst-explicit-types and squashes the following commits:\n\ne7eac03 [Reynold Xin] [SPARK-6428][SQL] Added explicit types for all public methods in catalyst.","distinct":true,"url":"https://api.github.com/repos/andrewor14/spark/commits/73348012d4ce6c9db85dfb48d51026efe5051c73"},{"sha":"6948ab6f8ba836446b005f2cf1cc4abc944c5053","author":{"email":"284a00ea9901227c42561196ff8576aed3ac5631@gmail.com","name":"Kay Ousterhout"},"message":"[SPARK-6088] Correct how tasks that get remote results are shown in UI.\n\nIt would be great to fix this for 1.3. since the fix is surgical and it helps understandability for users.\n\ncc shivaram pwendell\n\nAuthor: Kay Ousterhout <kayousterhout@gmail.com>\n\nCloses #4839 from kayousterhout/SPARK-6088 and squashes the following commits:\n\n3ab012c [Kay Ousterhout] Update getting result time incrementally, correctly set GET_RESULT status\nf346b49 [Kay Ousterhout] Typos\n748ea6b [Kay Ousterhout] Fixed build failure\n84d617c [Kay Ousterhout] [SPARK-6088] Correct how tasks that get remote results are shown in the UI.","distinct":true,"url":"https://api.github.com/repos/andrewor14/spark/commits/6948ab6f8ba836446b005f2cf1cc4abc944c5053"},{"sha":"d8ccf655f344eed65cdaf5d9252f1b565b8406ca","author":{"email":"284a00ea9901227c42561196ff8576aed3ac5631@gmail.com","name":"Kay Ousterhout"},"message":"[SPARK-3570] Include time to open files in shuffle write time.\n\nOpening shuffle files can be very significant when the disk is\ncontended, especially when using ext3. While writing data to\na file can avoid hitting disk (and instead hit the buffer\ncache), opening a file always involves writing some metadata\nabout the file to disk, so the open time can be a very significant\nportion of the shuffle write time. In one job I ran recently, the time to\nwrite shuffle data to the file was only 4ms for each task, but\nthe time to open the file was about 100x as long (~400ms).\n\nWhen we add metrics about spilled data (#2504), we should ensure\nthat the file open time is also included there.\n\nAuthor: Kay Ousterhout <kayousterhout@gmail.com>\n\nCloses #4550 from kayousterhout/SPARK-3570 and squashes the following commits:\n\nea3a4ae [Kay Ousterhout] Added comment about excluded open time\nfdc5185 [Kay Ousterhout] Improved comment\n42b7e43 [Kay Ousterhout] Fixed parens for nanotime\n2423555 [Kay Ousterhout] [SPARK-3570] Include time to open files in shuffle write time.","distinct":true,"url":"https://api.github.com/repos/andrewor14/spark/commits/d8ccf655f344eed65cdaf5d9252f1b565b8406ca"}]}