Hi all, good morning. Thanks for coming and attending this talk. Today, we will explain you how program and tester can be productive. So in this, we will share our experience. Like me and Nancy, we are working from last nine to 10 year on multiple projects. So we face a couple of challenges. What are the things we are doing? What are the principle we follow so that everything will be on the time and we deliver the good product? So mostly, we will share our experience in this, and let me introduce myself. I'm Anu Singhla, working as a principal software engineer in Red Hat. So mostly, I work on the front-end technologies, and apart from my work, I also share my knowledge and educate to the people on the YouTube. Good morning, everyone. I'm Nancy Chan. I'm Quality Engineering Manager at Red Hat, and I have over 11 years of experience in quality, software quality, and as a QE manager, so me and my team, we both are responsible for overseeing the development and implementation of quality control processes, and we make sure that our products reach highest quality and standards. So that's my quick introduction. Now, Anu, we're going to talk about what we're going to discuss today along with why we choose this topic. Okay. Thanks, Nancy. So today, our agenda is why did we choose this topic? What are the top five conflicts, and how do we resolve these problems, and collaboration between Dvalpa and Tester, when the Dvalpa and Tester collaborate with each other, and the couple of case study like we follow in our projects, and the couple of suggestions from our side, and then Q&A. Okay. So now, why did we choose this topic? Like the Dvalpa and Tester collaboration is very important. Okay. So we need to work together, and if they are working together, definitely the productivity will increase, and the customer satisfaction will also increase. Like if we say in 1990 or before the agile, the waterfall model was used. So in that like Dvalpa and Tester has very less collaborations. So like I didn't get a chance to work on the waterfall model, but one of my friend, Deepak Cole, he has shared some experience with me because he has worked on that. So at that time, like the Tester and Dvalpa, like Tester is working in the end, and the Dvalpa working in the starting. So in that, like once a garment comes, so there is one SRS document is prepared on that time. So one is here with the Dvalpa, one is here with the Tester. So there was very less communication between them, and the Tester is working on, like as per the SRS document, they're working on their use cases. But take example, like if in the end, the Tester is testing the bug, testing all the product, and some critical bugs is found, then maybe it requires some infrastructure change. Okay. So at that time, it was very difficult to make the changes because Tester is already testing in the end, and it will delay the productivity or delay the production, okay? So that's why like the agile process has come, and in the agile, like everything is on the collaborating, and we are working on the sprint, like we follow every fourth week, we deploy our changes to the production, and there are a lot of meetings is going on retrospective, stand up or grooming meetings. So it helps to grow and deliver our product on time. So before I jump to the top five conflicts, I would like to ask quick questions to our audience. You guys might work with the different development teams, or might be like interacted with the QE team and program teams. So as per your experience, what do you think, what are the reasons which creates conflicts between programmer and Tester, anybody in the audience? Yes? Yeah, right, I totally agree with your answer. Miscommunication, some like pressure on the releasing because like we always have some deadlines where we need to like release our changes to the production. So if that's the reason like miscommunication, time constraint, pressure, so what do you think like, how does this impact teams' efficiency and productivity? Anyone? Yeah, please? True, you are right. So I can, yeah, so the question is how does this impact the teams' efficiency and productivity? And she said like, yeah, so we sometimes happens where we fix, we found some bug, then we tested that again, it's kind of a loop, right? And I do agree like it will create some kind of empowerment where like we feel like it increased a lot of errors, right? And along with like it also impact the overall job satisfaction as to sit moral. Why? Because we have that kind of like because these two teams have conflicts. So these are the five top causes which like conflicts, which I feel as per my experience working with different teams. So it's not accurate to say like testers and programmers can never become productive of friends, but yeah, there are few challenges, there are few scenarios or situations where the relationship impacted because of the challenges. So the first one is competing goals. So you know like when it comes to the programmer. So programmers and testers, both have different goals and priorities and as per their job, right? And programmers are mostly focused on delivering and developing the software. And on the other side, tester are mostly work on like testing the finding the bugs and make sure that like the software is the highest quality. So I will take a quick example of like how this works like quality versus delivery, bug free versus functionality, functional software. So let's take an example. Testers usually test, they try to find bugs as much as possible. And if they're aiming that like, we will going to have a bug free software. And but on the other side, programmer really wanted to have like release the things, release the changes on production as quickly as possible. If that's the case, if they found some bug, the tester said like, okay, we have one bug which is related to UI and UX, like some font or some alignment is not good because they're prioritizing the customer satisfaction priority, but when it comes to the programmer, they say like, we can take it that later. Let's work on the quickly release. So here the conflicts arise. So next one is blame game. So that is very common blame game. So what it means, let's here I will also give you a real time example about new feature testing. So when it comes to the new work, the new work, like we already have existing system and we are just adding a new feature in the system. So what tests do you usually do? They pick that chunk and test. So when it comes to the new feature testing, we also need to test the regression part. So the regression is nothing, but we're like you're checking that with this new changes, there is no impact, the existing functionality, which is very, very important. And if they found some regression and they reach out to the programmer, saying that that's a regression, they might argue with, no, that's not, it's not the part of, you need to focus on the new feature. Why you are going to the regression? And I'm sure this is not because of my change. So that's kind of arguing, it's very common. So the next, the third one is communication gap. So I like that's a very major point. Communication gaps happen like tester found bug, they write the bug, they lock the bugs in the different machines, we have Jira, bugzilla, and sometimes happen like, they not explain each and every details and information. So how this will impact? Programmer pick that bug and they're unable to address what's a problem because they're not able to, they're saying like it's not working on my machine, right? That's again a very common. So conflict arise because they say, tester haven't shared the exact steps, steps to reproduce so that they can work on the solution. So here the again conflict arise when like, they know no and due to the lack of the information, it delays to the resolution. So the last one is the another one is limited interaction. So limited interaction is basically depending on team structure, depending on the project. So there may be the high chances both programmer and tester teams got let very minimum chances to interact. And if they are not interacting, how they will going to understand the end to end business strategy behind the work they are working on. So they might work on a silo basis if they are working on a silo mode. So they're high chance they have a minimum opportunity to connect. So again, limited interaction also create conflicts. Time conflicts, time conflict, both roles have a totally different priorities and goals which we need to think. And when I told, I already explained like, when it comes to the programmer, they mostly focus on delivering the things beyond time. And when it comes to quality, there may be a chance for, they ask for like, okay, we are not ready and we are not confident to give the acknowledgement to release the things. And they need extra time. So they again conflict arise because tester needs more time to release. So these are the top five causes like which creates conflict as per my experience. And this is very basic scenario. You guys can like relate like we are developer and tester is saying, it's working on my machine and develop, no, no, no, no, no. It's not working, you are wrong somewhere. And the another image is saying like, okay, that's why I'm giving a bug to the developer and developer is saying, no, it's not a bug. It's a feature, it's an improvement. So yeah, so this is how it works. Now, how we can solve coming through these point, a developer, I think it will create a more impact. So Anish. Yeah, so Nancy has shared like a couple of problems. There are a lot of problems between developer and tester. Now we will see how we will solve these types of problem. And like as per me, like developer testers should be friends. They sit together like after the COVID is not possible. Maybe we are not going office, but at least we should go office, contribute, collaborate with each other, okay? Yeah, so like the first step is supportive language. Always support your tester or developer, okay? So like take example, if tester is find some last mint bugs, okay? So don't blame them. What are you doing, man? Why you didn't find in the starting? Why you are coming in then? At least thanks them. At least we find that bug before the production. Or if the customer come up with that bug, it will create a negative impact. So always support them. Second way, okay? And one of James's best testing guru who tell always like tester don't hate developers. And it's like a means the wife tell to her husband before going out. If there is some stain on his shirt, so he will not feel embarrassed. Something like that. So developer and tester roles should be productive, should be friends. They know each other. They know their strength and weaknesses. Or take example one more like if the tester is not able to reproduce an issue. But help them to reproduce the issue. Give some system logs. Because as a developer we know what are the input. If we provide that input maybe it reproduce the issue. So provide that information so that they find the issue and they can look the issue, okay? So second one is defect triage meeting. So we follow this like every week the test, the bugs will come from any places. Like that the test are from developer, from multiple players, from customer also. So every week we do one meeting. In that meeting we discuss what is the priority? Either it's a bug or not. Or what are the solution? The basic things we just provide that in the, in the ticket also and the story points also. Like the defect difficulty level update bugs. And the code freeze. It's very important guys. Like means if, like tester is, like tester is a human. He don't test each and every part of the app, okay? So if we are pushing our code till the end, like we have the production which tomorrow and we are pushing our code till now. Then how the tester will test each and every part? Because they need to test a regression like each and every part. Like right now we have the automation we can do it. But still we need to do some manual testing also. And like what we follow is we have the three, like every four week we do our production push. So three week we are pushing our changes to the, pushing our changes in the testing environment. Last week we give full week to the tester to test each and every part. But we give like if they find any critical issue, just we fix that issue and push on the testing environment. Apart from this full week we give to the tester to do it. And next one is the resource alignment. Like in IT word resource, resource currency is very normal. Like somebody is joining the company, somebody is leaving the company. And the most standard ratio for developer and tester is like three ratio one or as per the product, as per their experience. But mostly this is the standard ratio. So sometime if there's some resource crunch, so always like as a tester push, as a developer, push less code in the production or in the testing environment so that they will get a time to test the things or like help them like as a developer we can test other developer code or the second one is also test our code. So we are also follow these things. Sometime is the, we have the little bit crunch in the resource. So the part of resource alignment, I would also like to add the resource balance in the both developer and QE ratio is very important. Why? Because of like if we don't have a balance team like for example we have five QE, five QE is on one developer. No, not good, right? And on the like vice versa, five developers, one QE, a lot of pressure, a lot of pressure to work on the things. So being like as a QE manager, I always sit together with the project team and decide like how the ratio should be. So when we are talking about the industry ratio about the dev versus QE, it should be like either three ratio one and two ratio one depending on the project, the complexity of the project, the timelines and the like what we committed to our stakeholders. So always like don't go to your manager, we have a crunch and just we need to help, like manager can't do anything, right? We have the crunch, so we need to manage ourselves, okay? And the pair program and testing, like I follow this one, like I always try to share my knowledge, architecture knowledge with the tester because if we share our knowledge, definitely they will not come with the small, small bugs like the data issue or some like the, if some field is not showing in the UI, then they will not come to us. They analyze from their own self if we share our knowledge with them or like they can directly ping to the backend team, I'm not able to see this field, please take a look. So it save a lot of time between developer and testers, yeah? So now we have seen like how we will solve these problems. Now at what time the developer and tester collaborate more each other, okay? So like the shift-left testing and in the shift-left testing, like always try to involve that tester in the initial phase, in the initial phase, so that like once the requirement comes, so we analyze, we figure out, okay? How we will deliver that product? So at that time, if we involve our tester, then like tester is feel like a customer, okay? They think like a customer, how the customer will use this product? So in this way, if they provide their suggestion, then as a developer when we are designing our architecture, then they share our knowledge, okay? And we use their knowledge, we use their use cases while designing the product or while designing any feature. So it helps like and if we are thinking like all the scenario in the starting, then it will definitely will help us to grow in the future also, if the new things come. And defect reporting and regulation, it's a normal thing like always the helper say, I'm not able to reproduce this issue. So like when the tester is reporting the bug, then definitely provide all the things like attach the video, attach the screen, so provide the proper steps. So that like when the developer start working on this, he has some information, okay? So I need to do these things and I'm able to reproduce this issue. At least he will not come to the tester, oh man, I'm not able to reproduce this issue. What can I do with that? So it save a lot of time between developer and tester, okay? So next one is regulation, recognition and appreciation. So like in Red Hat, we always recognize anybody if somebody help us. Like in Red Hat, we give some reward points, some and thank you note, so that if like the next person will feel satisfied, like he help us and we recognize him, something like that. Yeah, so now we're going to discuss about the different case studies. So before I will discuss the case study, I would just like to call like these case studies are the result, we are actually actively, these are the real time scenarios, which we are actually using in our teams with the helper and collaboration from like a development team along with the program team. So the first one is, okay, the first one is implementing a CI CD pipeline, which result improving the software quality. So I will go into, as a part of this case study, I will going to discuss like how these two roles actually helps like how the tester can help the development team during the development phase. So the first one is like, you can see like how usually developers works. So developer build the work, they usually work on the requirements they build and before, so how the team testers can contribute during the building or during the development phase with programmers, they can like add a automation suit. So automation suit is nothing, but it's a smoke suit, which give us a confidence like the basic functionality or basic critical functionality of the software is working fine. So what we will do, we will hear like when the developer is building their code, before like merging into the master branch, at that time, we will integrate our automation suit with the unit test. So along with the unit test, our automation suit will going to run. If it's working fine like the basic functionality, which we added in a suit is working fine, it will ready to go with the merge request. Otherwise, it will detect the problem and share the report with the developer. So that in the early phase, developer can take a look, what are the problem or you can consider, what are the regression with this new change? And it will also solve the time tested. They don't need to like much worry about the basic scenario because we have already tested as a part of the building process. So the next one, which we have is like cross-functional collaboration for a comprehensive testing. So comprehensive testing is very important. Why? Because we in agile module, like we do like quick changes, move that changes to the production. So how we can achieve cross-functional collaboration? So we usually conduct the bug bash. So bug bash is an event where like we invite different teams, different representatives from the development team, program team, as well as different QE teams. So they keep their normal, they set aside their normal jobs for some time and they start testing on a piece of the thing which we have planned to release. So for that, we can take an example of a e-commerce site which they are planning to release the major changes in the production. So we will conduct a bug bash event where we are like representative from the development team, QE team come together and doing random testing to make sure that like to find the bugs or testing different scenario. For example, shopping cart workflow, payment workflow to make sure that like we like give a, cover all the scenarios related to the UIT. So at the end of this event, what we will do, we will call out few winners on the basis of the defect quality, their severity and the details which they have shared. So it could be like you can, every organization have some mechanism to appreciate that it could be rewards points, it could be some gifts. So this is how you can encourage the cross-functional collaboration. So the next case study we have is about the actively participation of feedback loop, like how you can involve tester in during the feedback loop. So customer satisfaction is very, very important when it comes to the software development. And if we act on the customer feedback on timely basis, we can increase the overall customer satisfaction as well as the product quality. And as you know, testers, QEs are known as a customer advocate. And if we are involving them, it will definitely we're going to have a very good impact on the, as a feedback. So let's say here, let's take example, every team, every software had their different mechanism to gather the customer feedback. It could be an email directly, it could be a form attached on the system, it could be a account time who helping, who connects with the customer and customer directly share the details. So we gather all the feedback in a one space and we usually have a monthly base meetings where we discuss the overall the feedback which we receive. Now on the basis of like then developers and testers both contribute and share like, what are the things which you need to prioritize? So we log the tickets in our ticketing mechanism and testers start working on it on the parallel side, development will going to work on the fixing the things and testers will start working on the test plan. And then they will deploy a test and inform the customer like, okay, these are the feedback we receive and we fix this and now it's ready to use. So this is how testers can also contribute as a part of feedback loop. So I'll hand over to Anuj to talk about the overall development and feedback. So Nancy has shared like all the three principle which we follow. Guys, if you are not followed, try to follow these things. Like the CI CD it's very important like means before pushing our code changes we try to like test the things so it will not create a regression. So we follow the same feedback loop is very important. We follow these things, okay. So now we will discuss like how we deploy, build and deploy the features, okay. So this is a full diagram I try to make and try to explain you guys. So how we are doing, so how we are involving different people like the tester, developer, UX engineer or the PM level, okay. So like the first one is requirements. So requirement come from many places. PM level, analytics level or the customer, okay. So after that like we analyze the requirements. So in that like the all the team members like who are tester, developer, UX engineer or the PM level managers, everybody is there. So in that we decide, okay. Either we need to do these things or not, okay. We are doing these things. Then what are the priority of that or how the customer will use these features. So the tester is giving their suggestion. They think like a customer and then like as a developer we think, okay. So either it's possible or not or the backend team also decide, okay. So how we will deliver or how we will give to the UI this some if we need some backend support or some data from the backend. So in this way like we analyze each and every part. So once we figure out, okay. Then we go for the build, build the feature. The UX will give us the UX design and then we build the feature. And after that we like the tester things and like in the testing phase we are also helping the as a developer we also help the tester to test the things share our knowledge. And then after that like the customer demo. So after that like before going to the production we give the customer demo and like if it's a critical feature or the new feature we give some UAT like some give some pre-prod environments and give some URL so that they can do some UAT testing. And then after that like deploy the features and we continuously take a feedback from the customer. It's very important. So like in this way like we try to build our product and try to build the quality and improve the quality and all things. We follow this role, okay. So guys like these are the couple of suggestions from our side to all, okay. So try to build a network with all the tester and developer means try to go for offline meetup try to go for conferences like I'm here. So I meet a lot of people like in this way like we get to know okay how the people other people are using the technology different products and all things. So continuous learning like it's a very difficult like as a I'm a front end developer every day coming new things the new library come like there are a lot of framework in the UI. So it's very difficult to learn the new things or continuously learning but try to update whatever tool you are using try to things okay maybe means try to the new feature is come try to think about okay the new feature come learn the new things. So it's always like always try to update yourself and be kind in stress so like in IT the stress out is a normal thing right. So we have a lot of work to do and but try to come out take a break it's always a good thing don't starting blaming just take a break it's a bad thing or take a feedback in a positive way I always follow these things. So I try to take a feedback from multiple people if somebody give you the negative feedback take is a positive because thanks them at least he give you the negative feedback and he finds something bad in you try to improve that and definitely in the end you will feel yourself as a positive way. Make personal brand like means I follow this rule means try to write a blog like I try to educate try to educate via YouTube. So let it web development but means try to do these things and means it help you if it helps a lot means definitely. So with that this code we will wrap up our talk. So the code means that testers and programmers should work together rather than or instead of like competing with each other by sharing and collaborating there's knowledge the skills they can create a better and productive or good product. So thank you thank you any questions. Okay so the question is like how you are maintaining the quality while pushing the changes right on the QA. Okay so like this is diagram if you see here what we are doing is when we are merging our changes okay when we are create our MR okay. So in that time once build the code everything is fine then we run this unit test and the automation test. So tester give us one URL like tester also doing like running their automation test right. So in the GitLab we are also running their unit test on our GitLab. So in this way like suppose you have make one button change okay. So we are 100% sure it this button will not create any regression at least in the existing or the critical functionality or the unit test we are also follow this rule like every time we try to write the unit test otherwise we don't merge any code. So this is like we follow this rule. And in this way like once everything like at least the existing functionality is working fine then we go for the merge request and then deploy it to the testing environment. And then tester will also do their own testing manual or whatever they want to do and after that we go for the production. So this is way like we follow. And if you see here in the red sign problem detected in the unit test like if there's some issue in while running our changes then we go to the developer will take a look it's either because of me or there is some issue in the other unit test or the automation test. So we connect with the tester and in this way like we follow this rule. So as of like in this section like you know like when we execute our automation run right we do have some report which explain like what are the things is not going well. So developers can take a look because we have already shared all the workflow and all the framework with them along with their like integration with the unit test report. Okay the question here is like how we spread this collaboration across time because they have high chances they are teams which not following this right. So you know like we usually have a flow where we have a QE team or development team and program team. So we usually have one monthly or like monthly or like twice in a quarterly meetings with the leads. So over there we usually have a bidirectional discussion and feedback on the QE team as well as development. So like as a QE manager I have one on one with my team members like to see if they do have any feedback for the developers because they're open to share. Like they can share the feedback if they are facing any challenges. So they discuss with me and because I don't want like their developers and Qs directly have some some relationship conflicts. So I act as a manager act as a moderator and they connect with the development manager as well as program manage to make sure like this is what we are facing like how we can fix how we can fill this gap and vice versa if development manager have any feedback like okay we have a resource crunch or last release we got a critical bug on production so how we can like improve this in a better way. So being a leader so we discuss internally and make sure that like it should be a bidirectional call. So we usually keep like I usually follow the monthly connect why because we usually have you follow the agile and or Kanban which has like three weeks spent or two weeks spent so this is how we actually work. And second way like we have the retrospective meeting. Okay so if something is not going good so we discuss in the in that retrospective and discuss like provide some points and action item also we followed that. Okay so we will take care in the next time. Okay so like first time mistake is a normal thing but if you are repeating the things that's a bad thing. So try so the retrospective is a good thing and we have the weekly grooming meeting daily stand up. So we follow these things and at least like we are maintaining the quality and all things.  Okay anyone in the back. Anyone any other question. I think like how do you also expect. Okay so I'll give my I will share my experience maybe Anuj will share. So the question here is like how we can encourage peer program your peer testing. So I'll give you few years back me and Anuj were in one team and just like before the release I found very critical bug. And when I found like it's like two or I think next within a few hours we were supposed to really I was just doing the smoke testing. And when I informed Anuj said okay Nancy come at my desk I'll show you what is going on.  I was like how I can fix it that's your job. So he said like it's just a mistake of full stop. You just we miss that that is why it's a blocker is coming. So I sit on his desk. He explained me the whole code and I fix that bug. So this is how you can encourage. So if it's coming like if developers should like explain like how they're very small small tips like we also encourage that culture in a team like they explain like how the console and how the network logs tester can understand right. So because if there's something is failing without directly logging up sorry bug but we usually do which firstly check console is there any errors going on? Is there any what's the network locks? Because there may be a chance. It could be a internet intimate issue right. It's you have a like network connection problem that is why it's failing. So just like learning these things and collaborating with developers it helps a lot. So yeah. Okay. So I follow this thing like I always share my knowledge. Okay. So somebody tester come up. I'm not able to see this field. I ask him have you think about this? Why you are not able to see these things? So have you checked the network call or are you getting any console error? So he think like this and then I explain him. Okay. We are doing some these things and that's why you are getting this issue. So at least in the future he will not come up with the same question. And the second way we follow we like every month we have the one guilt meeting. So in that meeting like everybody share their own experience like something he if somebody is did something good. So like in that meeting he explained okay. I did these things and everybody is in that meeting like the developer tester. So this is also the second way we follow these things. And the third way we try to like automation test right. So we help them maybe I have some bandwidth one day or five hour, 10 hour then we told them okay could you please give us one automation test maybe I help you or we review their code also. They review our code but sometimes they are not able to do but we ask them okay is there any scenario is pending from our side? So we can write the unit test for that. So that it will not come up with the because they think like a customer they have a lot of scenario we don't think like that right. And in this way like always help them to decide the framework which framework is good and what are the benefits if you are doing these things best practices there are a lot of things like we follow. This is just my suggestion guys. Thank you.


What's going on everybody welcome back to another draw preview and we have a big event this week the Miami open It's a WTA and ATP event going on at the same time in the one venue It's a big big event for the women, but a lot of players have pulled out Let's go check it out So the players that have pulled out so far before the tournament started we have her cog Shay Kazakina's pulled out which is unfortunate because she's been a little bit of form lately Lee Mokova Pavlochenkova Strick of his pulled out Teague van Udvank Vekage is also pulled out and the biggest thing to pull out Serena Williams She is not playing the Miami open and also your strength is still on a provisional suspension. All right starting at the top of the draw There are 96 players playing this event 32 seeds and all 32 seeds get a buy in the first round So this is like a mini grand slam with the number of players that are playing top seed ash party and the defending champion Of course from 2019 she gets a buy in the first round to play a qualifier in the second round No matter what because we've got two qualities going at it in the first round So good start for body. Let me have Oster Penko versus the wild card Wang winner of that match It's going to take a number 25 seed risk in the second round Then we have the number 24 seed Kerber already into the second round should play the winner of Hibino or a qualifier in the second round Then we have Mikhail versus Sigmund in that first round match winner of that gets to take a number 14 seed Azarenka in the second round and straight away you can see we have four grand slam champions in this part of the draw We have Bardi, Oster Penko, Kerber and Azarenka so a stacked part of the draw here for Bardi She's trying to defend all those points and defend the title. She might have to take on Azarenka in the fourth round That's a tough matchup second part of the draw now We have the number 11 seed Benchich getting a buy in the first round She'll take the winner of Diaz versus Venus Williams in the second round so Venus versus Benchich That'll be a fun matchup. Then we have a qualifier versus Wang winner of that's gonna play against number 19 seed Vondra Sova in the second round then we have the number 32 seed Kudamatova getting a buy in the first round She'll take on the winner of Marjanovic or Collins who are playing in the first round Then we have a qualifier versus Kostjak the winner of that's gonna play against number seven seed Sabalenka so this part of the draw isn't as stacked as you probably expect Benchich has been okay this year Sabalenka's been good But other than that there's not too many players here that stand out especially on a hard court Don't forget the winner of this part of the draw in the quarterfinals will take on the top section of the draw So we could get Sabalenka versus Azarenka Sabalenka versus Bardi so remember that when we go forward third part of the draw now We've got Simone Halop the number three seed buying the first round She'll take on the winner of a qualifier versus Garcia in the second round Then we got another qualifier versus Sever Stover in the first round winner That's gonna take on number 31 seed Cocoa golf So straight away we could be getting a third round meeting between golf and Halop, which is gonna be a great matchup I'm very excited to see that one We've got number 18 seed Keys buying the first round she'll take on either Sinecova or the wild card Conju who are playing in the first round then we have Krajiekova versus Blinkova winner of that it's gonna take on number 15 seed Sveontech so again straight away you can see fourth round potential meeting between Halop and Sveontech It's been a fun match over the last six months that I've been looking forward to watching every single tournament They've both played and we could be getting that again in the Miami tournament maybe in the fourth round All right rounding out the top half of the draw now We have the number nine seed Caviteva she gets to buy in the first round She'll play the winner of Cornet versus Cousnets over in the first round So not an easy start for Caviteva who isn't a little bit of form She won a title a couple of weeks ago. Then we have Lynette versus the wild card Montgomery winner of that will take on number 17 seed Conta in the second round then we have the number 30 seed Alexandrova getting a buy in the first round she'll take on the winner of Potterosca or the wild card Sharif who are playing in the first round then we got a battle of the Americans in the first round Between Rogers and Bregel winner of that will take on number five seed Svetolina who hasn't been in the best form in 2021 she's lost matches that she should have won but maybe being the number five seed She might be able to sneak through the draw in this one. All right, let's go to the top of the bottom half now We've got the number eight seed Andrescu. She gets a buy in the first round She'll take on the winner of Svedova or a qualifier in the second round Then we have Stevens versus a qualifier in the first round winner That's going to take on number 28 seed Anissa Mova in the second round Then we have the number 20 seed Martich buy in the first round She'll take on the winner of the wild card Callanskaya or Begu in the second round then in the first round We've got Pedersen versus a wild card Wang winner of that will take on number 12 seed and probably the best player of 2021 Moogarutha and again you can see straight away fourth round Moogarutha Andrescu What a match that'll be if we can get it But don't sleep on Anissa Mova because she can produce some goods especially when she's playing in front of her home crowd So watch out for Anissa Mova, but I want to see Andrescu versus Moogarutha That'd be epic in the fourth round. All right going to the next part of the draw We have the number 13 seed Brady and of course the Australian Open finalist She gets a buy in the first round She'll take on the winner of Perra or Tormo in the second round and Tormo has been in some form So watch out for her then we have Kineppi versus Davis winner That's going to play number 21 seed Rybakina in the second round Then we have the number 27 seed Jabor getting a buy in the first round She'll take on the winner of Taikman or Bodosa in the second round Then we have Petkovich versus Zhang in the first round Winner of that's going to take on number four seed Sophia Kennan Who we haven't seen for a while So this is the first time we're going to see Kennan for a couple of months And I'm excited to see how she does here Especially in Miami being back in front of her home crowd You could say in the USA Going to the next part of the draw We have the number 6 seed Plishkova buy in the first round She'll take on the winner of Zheng or Ferro in the second round Then we have a qualifier versus the wildcard Sanders Winner of that's going to take on number 29 seed Pagula And straight away you can see that Plishkova might have to take on Pagula again In 2021 in the third round And Pagula's got the wins over Plishkova So Plishkova absolutely nightmare Third round matchup potentially against Pagula Who has beaten her a lot this year Then we have number 23 seed Sakari in the second round She'll be playing the winner of Baskova or Rus Who are playing in the first round Then we have Georgie versus a qualifier Winner of that will take on number 10 seed Bertens in the second round And this part of the draw wide open Pagula being informed Plishkova the highest seed in this part of the draw And Sakari, she looks like she's ready to do something good as well And Kiki Bertens still on the comeback So this part of the draw super wide open for me And rounding out the draw, the very bottom part of the draw We have Mertens The number 16 seed buy in the first round She'll take on the winner of Plishkova or Bolta in the second round Now of course that is not the main Plishkova It is the sister of Plishkova Then we have Kastea versus the wildcard Scott Winner of that's going to take on number 22 seed Contavit in the second round Then we have the number 26 seed Puttenseba Buy in the first round She'll take on the winner of Watson or a qualifier Who are playing in the first round Then we have Potapova versus Tom Mianovich In the first round, winner of that gets to take on the Australian Open Champion Number two seed Naomi Osaka Who we haven't seen since lifting the trophy in Australia So let's see how she does But there's some dangerous players here Tom Mianovich She almost beat Halep at the Australian Open Puttenseba's beaten Osaka in the past And also Mertens, she's been in some form So watch out Osaka Not an easy part of the draw for her So there you have it That is the draw for Miami And it is absolutely stacked It feels like a mini grand slam Except for all the buys in the first round Let me know down in the comments below Who do you think is going to win this event It's very wide open As the WTA always is But there are some names there that I like Especially because they've got nice draws as well So let me know down in the comments below Who is winning it for you So this was a very tough one to pick There are some players that have some good draws And I'm going to back Sabalenka To do something crazy here Because she has got a good draw She's probably in the easiest part of the draw for her And if she can get through players like Bardi or Azarenka Hela or Sveontek In the quarters and semifinals She can make it to the final quite easily So I'm going to back Sabalenka Who has been pretty good this year So again, let me know down in the comments below Who are you picking for this one So the Miami Open for the WTA It's absolutely stacked Can't wait to watch it next week


Welcome. This talk is called Descent into Darkness. Understanding your system's binary interface is the only way out. So we're going to talk about what that means shortly. So who am I? I'm Joe D'Amato. I live in San Francisco. Used to work at VMware, went to CMU, some stuff that I work on. I wrote the first version of Memproff that you saw the web demo of yesterday. I wrote some L-trace patches, some MRI thread improvement patches and stuff. I maintain a blog, TimeToBleed.com. That's the actual URL if you don't believe me. And on Twitter I'm just Joe D'Amato. Alright, so enough of that. So we've only got 30 minutes right now. So probably won't have enough time to get through everything, but I'm just going to try. So I just got to say welcome to flight school. We just got to roll right now. So I have no clue why this talk was actually accepted. There's only about five lines of Ruby code in it. But before we actually get started, I need to introduce you to a really, really good friend of mine, Satan. And the reason why I need to meet my good buddy Satan is because this talk is basically about how being evil is totally awesome. So in other words, you shouldn't actually do any of the stuff that I'm talking about in this talk ever unless you do it in a VM because you could seriously destroy your system. But if you get it working right, like amazing things are possible, which is what we're going to show. Okay, so what's the problem? Basically the problem is my reprocess is 700 megabytes and I want to know why. So I don't know if you guys saw it, you who just saw it yesterday when you were saying object allocations were free. I was like having a seizure in the corner over there. Alright, so there's Ruby. It's large and slow and we want it to be small and fast like that guy. So what's the problem? The problem is it's really easy to leak references in your Ruby code. So if you leak a reference to an object, that causes that object, any object that that object references to stick around in memory forever. So let me show you a picture of what that means. So as long as somebody somewhere is holding a reference to the object on the top there in gray, this entire tree of objects cannot be freed and that can actually add up to a lot of memory really fast. But sort of also really important too is that every GC cycle, GC is going to scan this tree of objects and it's going to burn CPU cycles looking at it saying like, hey are these free yet? No, are these free yet? No, over and over and over again. And that can actually add up to quite a bit of CPU that's being burned every GC cycle. But you might say, hey, memory is cheap, CPUs are cheap, I don't really care, I'm not scared. But I would say, hey, Ruby's GC is a naive stop the world market sweep GC, which basically means that the more objects that stick around in memory that you don't need, the longer your GC runs are going to take. And the longer GC runs take, the less time your app actually has to run Ruby code. And that's bad, obviously. So we want to eliminate, if we eliminate leak references, we reduce the length of GC runs, run more Ruby app code, everybody's happy. And that's cool. We want to do that but how? How can we track down these reference leaks? So the problem requirements, right?  So I don't want to apply patches to Ruby and I don't want to rebuild Ruby because doing that's hard. I don't want to break binary compatibility with my extensions. I just want to do this once, just have it work. I really wanted to just be a gem. I just want to require Ruby gem, do memory profiling and then just be done. And then anything other than that is way too much work. Luckily my boy Sane has my back. We can do some really, really nasty things to actually get this going right. I'm actually going to skip these two slides real quick because I don't think we're going to have time to get through everything if I don't. But all you really need to know is that AMD64 is just a processor spec and Intel and AMD both, so AMD invented the AMD64 processor spec but Intel implemented it as well and they named it Intel64. So if you see AMD64 on here anywhere, it's actually applies to Intel CPUs as well. It's just the name of the spec. So I'm going to skip this too. This is basically just saying they both try to implement it and whatever. Anyway so what's an ABI? Application binary interface what does that mean? According to Wikipedia it describes the low level interface between a program and the operating system or another application.  So it'll specify the alignment of data types, calling conventions, object file and library formats and sometimes two syscalls, how they live, how they work, whatever. So in terms of ABI's like how do you actually find them? What's the deal? So there's a system 5 ABI which is like the big one above all of them. It's 271 pages and then underneath it there's architecture specific supplements for each of them. And the awesome thing about the AMD64 supplement is that it says yo include everything in the 3D6 supplement except with these changes. So if you actually want to do any like low level work on the 64 bit X86 you have to read all three of those. And if you know there's other specs for other architectures too. But luckily I brought copies of all three of these and we can just chill and read them like for the rest of the day. No but like actually let's blaze through some of this stuff right now. So we got to use some evil devices to get some of this stuff going. You have NM. NM comes default on OSX and Linux. It just dumps a symbol table. ObjDump will disassemble lots of different object formats. It'll disassemble elf, Mako, lots of different things you can do with it. ReadElf is Linux specific and it'll dump elf specific information about a binary which we'll talk about very shortly. And then there's also DorfDump which will Dorf debugging information. So your sample usage of NM, I'm using NM on the Ruby binary. On the left right there you see symbol value. It's value in quotes depending on the object. The value could be different. It could be an absolute address or just an offset. On the right you have a bunch of symbol names. And then in the middle there's some stuff. You can read about it on the main page. ObjDump. This is sorry I'm going so fast. I just got to get through some of this initial stuff. This boilerplate stuff. Alright so ObjDump-D. All I'm saying is yo, ObjDump disassemble all of Ruby and give me some data. On the left there you have the offsets. To the right of that you have the op codes. So those are the actual machine codes. So that's like the actual bytes the CPU is reading and executing. And then next to that are the actual instructions. So those are those bytes translated into assembly instructions. And then on the far right is this helpful metadata that LibOp code will output and tell you hey you know this line is referring to a constant called DoringGC or a global DoringGC and this thing is referring to malloc limit whatever. Just to make it easier to understand what's going on. You also have readElf which will dump out elf information. There's a lot more output you can get than just this. There's also DorfDump you know same idea. Okay cool. So we need to meet some new friends before we can really get down and dirty. So registers they're important. They're small, fast pieces of memory that live on a CPU. And some registers have specific jobs. So two examples of registers that have jobs are RAX or RACs. It holds the return value from a function and there's also RIP which is the instruction pointer. Keep track of which instruction you're executing. You can also refer to registers in pieces. So let's take a look at what that means. So here's RACs uncensored. On the bottom that's the full 64-bit RACs register. That guy takes up 8 bytes and in IntelSpeak that's one quad word. There's also a register called EAX which is just the lower 32 bits of RACs. So it's referring to the same piece of memory but it's just the lower 32 bits. And this guy is 4 bytes wide or 1D word, 1 double word. There's a lower 16 bits of that called AX and then AX is split into two pieces, the upper 8 and the lower 8. So you can sort of see by looking at this diagram, this is sort of like you're like digging up like remains. This is like 16-bit CPUs, 32-bit CPUs, and then 64-bit CPUs. They just kept gluing stuff on to the side. So just some other simple notes we need to know. There's two different syntaxes for assembly just to make your life that much more painful. There's AT&T slash gas syntax and Intel syntax. By default all the GNU tools display stuff in gas AT&T syntax but you can set it to Intel if you want by using those commands. I personally like gas, that's just what I learned on. I don't know, Intel is hard to read I think but you know whatever floats your boat. And so unless otherwise noted all the assembly in the slides coming up will be in AT&T gas syntax. And yes there is a lot of assembly coming up. So moving stuff. So everybody else took a poll and I feel left out if I don't take a poll. So show of hands if you thought you were going to learn assembly today. Awesome. Alright so moving stuff. How does this work? So this is only AT&T syntax. Keep that in mind. So you have source, comma destination. You can move immediate values like that second line right there. I'm moving zero into the register RBX or I can move a register to a register. So I'm moving EAX into RAX. And if you remember from the last slide we just looked at EAX is a lower 32 so I'm basically truncating whatever is in racks to be only 32 bits wide. Important note, source, destination cannot both be memory. Calling functions. Lots and lots of different ways to call functions on the 64 bit CPU. There's only two ways that we really care about for this talk. There's the indirect absolute way where you can think of essentially RBX as a pointer. And the value of that pointer is the absolute address of a function. So that's quite literally dereferencing RBX and executing the code that's found there. The next instruction is just it shows an absolute address. Dead beef is the absolute address but that calculation is done relative to the next instruction pointer with a 32 bit displacement. I know that makes absolutely no sense. I'm going to show a diagram of that shortly. So calling conventions. How do you call functions in assembly? Well, arguments live in registers from left to right. So the first argument lives in RDI, second in RSI, so on and so forth. And that's for integer class items. There's lots of different classes. And stuff gets passed on the stack too just like on I3, RBX. And the important thing this is going to come back to bytes in the ass later is that the end of the argument area must be aligned on a 16 byte boundary to comply with the ABI. And also there's these huge contracts about whether or not you're supposed to save function, whether you're supposed to save registers yourself or the guy you call supposed to save registers. So each person has a specific job. If you're the callee, you have to save certain things.  Alright, so I'm going to very quickly walk through some assembly code and the matching C code that goes along with it. On the left it goes over, yeah, on the right, whatever. On the one side is Intel and on the other side is AT&T. And then on the bottom is C code. So on the top what we're doing, these two guys are just saving the old stack frame base pointer and they're setting the base pointer to the current stack pointer. So they're just basically carving out a slot saying, yo, we're going to start referencing things from here. These two instructions right here are doing the initial setup. So that sort of move EDI you see on the top, that first instruction. So if you remember from the slide before, RDI is the first argument to a function. And as we looked at in RACS Uncensored you can refer to pieces of registers. So we're passing an amount down here in the C function and amount is only 4 bytes wide. So GCC is outputting an instruction that's saying, hey, it's only 4 bytes wide. So just move the lower 4 bytes EDI into this random spot on the stack RBP minus 14. And then it's setting another spot on the stack equal to 0. Which matches up with that ret equals 0 line right there. So next what's going down is we're moving whatever we just moved into RBP minus 14. So RBP minus 14 just had those lower 4 bytes which is amount. So that first instruction up top matches up with that first line right there which is just moving amount into the register EAX. The second instruction is just an add. It's saying, yo, take whatever's in EAX, add 150 to it and then the result of that lives in EAX. So that matches up with that line of C code. And then these two instructions are actually totally useless. They're just generated by GCC and they would be removed on higher optimization levels. And then this just returns from the function and the result lives in RAX and you can plug the ABI and everybody's happy. Cool. Okay, so L-objects. What are L-objects? What's the deal? They can be executables, they can be libraries. This is what they look like if you were to diagram them. So each shared library you load into the process has its own set of that that goes along with it. And each set is independent and they all kind of work together using the runtime dynamic linker. So they have headers. Headers help describe the object. There's different types of headers to describe segments or sections. Segments are collections of sections. And so Memproff actually uses libelf to wander through an elf object and extract useful information. But like I just said, the executable in each shared object just has its own set of data. And sections that matter to this Memproff adventure we're going on right now is the text segment, that's where code lives. There's the PLT which we'll talk about soon which helps resolve absolute function addresses because shared objects can be mapped anywhere in your address space. And you don't know until runtime the actual address that a function will be at. So you need this PLT thing to help you figure out where it lives. And then the PLT needs this other table called the got PLT for storage. Okay, so PLT, PLT, what's the deal with the PLT? Well it's the procedure linkage table. It's used to find functions in shared libraries at runtime. Shared libraries are precision independent, they can be mapped anywhere in the address space. So by now you should be thinking, what does this have anything to do with Ruby? And well it has a lot of things to do with Ruby actually. These are actually the ingredients we need to do serious, serious evil. We know the AMD64 ABI or we can at least look up whatever we need to know. We know how elf objects work and we know why I didn't actually go over it so I just don't have time but just take my word for it that in the RubyVM when you allocate an object you know with like string.new or whatever it calls in the RubyVM rb new object and when it frees it it calls a function let's just say it's called add free list. So you won't. This is pretty much what this guy Brian Mitchell said to me a long time ago when I proposed this idea. He said you won't combine all the stuff you just talked about and rewrite the RubyVM in memory while it's executing code. You should be scared like her. Sorry, I'm going to rewrite Ruby while it's running. I apologize for destroying your VM. So we want to track allocations that freeze. We want to do it as a Ruby gem so what's the deal? We know that the RubyVM calls rb new object to allocate a new object. This happens so we can track objects. So we know what call instructions look like. We just talked about calling functions so why don't we just scan the Ruby binary in memory and just rewrite all the function calls to rb new object to call a handler of ours instead. I'm not that scared so let's just do it. So you disassemble Ruby and this is what you get. There's just some function somewhere calling rb new object. That guy there on the left in blue that's the address of this instruction. That byte right there that's actually the call op code and then the next four this is sort of the four byte displacement. So it's trying to call rb new object so over there in orange you see rb new object and you see that sort of hex strings to the left. That hex strings to the left is the absolute address of rb new object. That's where rb new object lives in memory and it's calculating that by adding those two things together. That's how relative call instructions work. So let's just overwrite that part in red, the displacement, so that all calls to rb new object actually call a different function instead. And the other function that we're going to call instead might look something like this, like call rb new object ourselves, do something to track this object and then return the object. If we don't actually call rb new object and return the object then all hell will break loose because the VM was expecting you to return memory for an object so we need to actually do what rb new object was doing but do whatever else we want as well. It doesn't work for everybody though so this only works for rubies that are built with disabled shared that don't have a lib ruby.so but if you build ruby with enable shared and so this is how most Debian, Ubuntu, et cetera boxes build ruby with a lib ruby.so then code in lib ruby so calls rb new object via the plt so we need to do something else to handle that case so let's look at how the plt works really quick. So you have this assembly instruction in a ruby library that was, in a ruby binary that was built for shared you'll have this instruction right. So over there in blue on the right you see it's calling rb new object at plt so if we go and we just assemble what lives there wow these colors are really bad I'm sorry guys these slides are on time to bleed if you want to fall on your iPhone or whatever. So anyway at that instruction if we disassemble it we see these three instructions a jump a push and a jump so what's actually going on here well over there on the right that's just sort of metadata saying hey this jump is referring to this absolute address and this is an entry in the got plt so initially when a shared library is loaded into memory it has no idea where the function you're trying to call actually lives. So what it does at first is the entry in the table just points to the instruction following the jump which seems like it makes no sense but it actually will make sense very very shortly. So initially it's set up this way because it has no idea where it's living so that instruction is actually pushing an ID it's saving an ID out for state and then that second jump is invoking the runtime dynamic linker so the runtime dynamic linker runs it looks at this ID that was saved and it says okay word I know where this thing lives I'm going to fill in the address in the table and then transfer execution to the function rb new object and then this acts like a cache so the next time when you call rb new object at plt it just jumps directly to rb new object without re-invoking the runtime dynamic linker cool all right I know I just meant like is everybody like we good right now everybody have questions all right word okay so basically we hook so this is a global offset we hook the global offset table by we redirect execution by overwriting that entry so instead of rb new object we call our handler function that's sort of like so this whole wow I just killed that so like this whole setup is the initial setup right and then the linker runs and it like writes this thing in there so that next time you hit rb new object correctly so we want to call our function instead of rb new object so we can just pretend where that we're the the runtime dynamic linker so we'll just say yo I am the linker I'm going to go in find the table and then write the address for my function there instead completely bypass the runtime dynamic linker so what that looks like in the sort of using that same diagram is you have the plt entry and then that entry in the table actually just directly refers to my other function so we just completely sidestep that push and that other jump to just go and call our other handler which that's actually a copy paste oh no that's right that's actually right yeah that's right so our handler calls rb new object does some tracking stuff and then returns it so you should be thinking yo like your other function is calling rb new object like is that infinite loop like want to just end up calling itself over and over again no keep in mind that thing I said before about each shared object has its own state other function happens to live in the mem prof shared library as long as we don't modify mem prof but modify everybody else then we're good right because mem prof is still going to resolve symbols in normal way but everybody else is screwed because I just wrote over their tables so we're now tracking object allocation right but we need to find links we need to track when objects get freed add free list is called in the vm when an object is freed want to just overwrite the call instruction for that or hook the god or you know whatever what we just did before just do that again actually we can't do that because this add free list function is inlined that right there so that basically suggests to gcc like yo you can build this thing output a set of instructions and then I'll take those instructions insert them into whoever called me and then optimize the function from there so if the thing gets inlined you won't actually see any calls to add free list so what now you know are we jacked can we do anything what's the deal well if we carefully study this small snippet of code we notice that free list is getting updated right and free list has file level scope so this thing just lives at some static address somewhere and we're just overwriting it every time we update the free list so why don't we do something really stupid we know there's some static address we have free list updates to free list so why not search the binary for move instructions that have free list as the target and just overwrite all of those so let's do that but we have a problem we have a problem the problem is that the system isn't ready for a call instruction yet and what does that mean well the 64 bit a bi says that the stack has to be aligned to a 16 byte boundary after any and all arguments have been arranged okay since we're just overwriting some random move we have no idea what the state of the system is and we can't guarantee that we won't completely fuck something up by just doing a call so we can't just plop a call instruction in there we have to do something different we need to obey the a bi we need to move things out of the way we need to align the stack we need to save the register we're supposed to save but you know what can we do to do that right well we can do a jump instruction jumps have no restrictions we can just jump anywhere the difference between a jump and a call is that call saves a return address so you can do a call and you can go back to where you were before a jump leaves no trace of where you came from so we can transfer execution to an assembly stub that stub will set up the system in accordance with the a bi and then that stub will transfer to our c level handler which will do stuff and then we have to make sure to jump back to where we overwrote that instruction when we're done because as we said before jump saves no state okay so let's do it there's an instruction that's updating the free list we can't overwrite it with a call instruction because the state of system is not ready for a function call as I just said so we inserted jump instead now we can't change the size of the binary so the move instruction up there is pretty wide it's seven bytes wide the jump is not quite as wide it's only five bytes wide but you can't change the size of things so we're going to insert two no ops just to pad it out and then that address right there that's just the address of the assembly stub and then when we're done with the assembly stub we have to jump back here fall through the no ops then continue executing code in the VM as if nothing happened so don't worry I'm not going to go through all of this but this is basically the slightly abbreviated version of the assembly stub that's needed to make hooking free list work so you overwrite I wonder if I can get my pointer over there so I can show you something probably not anyway so that first instruction at the top right there so we overwrite the move that overwrites the free list we're basically saying okay this thing's updating the free list we're going to completely stomp on it and insert a jump but we have to actually update the free list otherwise the state in the VM is completely inconsistent so we need to figure out like how the filter is being updated regenerate that instruction to actually do it save all the stuff we need to save set the world up to align the stack like if you see right there two spots up from that green box there's an and instruction that's actually aligning the stack to a 16 byte boundary and that's sort of complying with the ABI and then that call instruction is transferring to our handler our handler says yo this object was just freed let's mark that in our internal state we return from this handler back to that leave instruction we restore a whole bunch of state and then we continue executing the Ruby VM as if nothing happened and surprisingly this actually works so here's a sample of how you might use the gem in a really simple way you just have a you require memproff you call memproff start you require a bunch of other stuff and then I'll give you output like that and the important thing to notice about this output is that it's not just telling you like yo you're you're you're just taking up a lot of memory it's telling you like yo like these are the actual numbers of objects being allocated on this specific like in this specific file on this line okay so it's telling you exactly where you're leaking memory and it even includes the type too so you can see what objects you're leaking but the really really important thing about this slide that you should all be like completely out of your mind about is that this slide actually contains Ruby code oh my god so you know that's cool but what else can we do with memproff well we can also just track a block we can call memproff.start track pass it a file do stuff and then track like memory allocations and memory leaks of a specific block or we can dump the entire heap as JSON and if you dump the entire heap as JSON you can load it into the beautiful web UI that Amon demoed yesterday at the lightning talk so anyway there's there's also a middleware component too like I was thinking like you know we were thinking like oh wouldn't this be cool if you could insert memproff as middleware and just get per request object count information that way you can see like per request I'm leaking like a hundred controllers or whatever so you can do that you install the memproff gem you can require memproff middleware set up some middlewares and then each request to get an output like this telling you the number of objects the file and the line that these things are being leaked from so you can easily track down what's wrong with your applications and fix your memory bloat so as Amon demoed yesterday there's memproff.com you can go there check it out play around with it that's sort of the web UI that's built on top of the output from the gem so the gem outputs data the web UI parses it and makes it really easy to read so there's a couple limitations from memproff it only works on AMD64 linux and snow leopard there is some 32-bit support but it doesn't really work working on it soon hopefully it only works on MRI and RE is 1.8 right now it only works on binaries that are not stripped and OSX system Ruby is not supported because it is really stripped. Support for EY rubies those are stripped but support for those is forthcoming as is support for stripped rubies in like Dabid and Ubuntu there's a little more work though for the user end the user will have to install a debug package that comes with their package manager so nothing like too crazy just an app to get installed or merge or whatever and to get debugging symbols and I'm working on support for that I was working on it this weekend actually and don't worry too there's more evil brewing and lots of really crazy stupid ideas that we hope you'll love so stay tuned to find out what they are and just as a hint 1.9 support for memproff is right around the corner that's one of the ideas we're playing around with we're pretty close but anyway we're getting close to the end of the talk I just need to say thanks to a bunch of people who without their contributions this would have been impossible so use RVM without RVM without Wayne's project this would have honestly been impossible for us in all different rubies so use this project give him money he's awesome what's up hope you were clapping for Wayne because I'm not done yet so you can get memproff just to be completely clear this talk was about the memproff ruby gem the gem is completely free open source et cetera and provides text based output you can get it at github so my github account right there so you don't want to do this yet because I haven't cut a new release and the current release on gemcutter is pretty old so you can just do that if you want to come hang out on RC and talk about how this works and if you want to help out too jump in there and then memproff.com is a separate thing they're working on on the side that just sort of generalizes it loads in data from the full heap dump excuse me and makes it nice and easy to read and that's sort of an alpha motor right now special thanks to a couple people special thanks to Aman for making the beautiful web UI the JSON output and much much more thanks to Jake Douglas and Jake up there none of the Apple people would be able to run this on your systems I kept telling Jake when I was working on this I don't care about Apple I just want to make this work on Linux and he was like well I care so he wrote it so he's awesome Brian Mario or Brian Lopez because he's cool and this guy over here with the camera Brian Mitchell because if he didn't tell me not to do it then I wouldn't have done it or something so questions and yeah that's it thank you what's up man I don't know you asked if there was any chance that this would work with a readable rightable memory module oh yeah yeah yeah so the NX bit yeah this works with NX because I just use N protect and I just say I don't care I have additional materials about that level of engineering I was just wondering is there something you recommend low level programming for high level programmers people ask this all the time and I really don't know of an existing resource that type of information you just got to read lots of code just disassemble stuff you can read the Intel instruction set architecture there's like some amount of knowledge that you just got to go out there and just try to do it usually on my blog I blog about this type of stuff and try to lay it out and explain it in a way that I hope people can understand like people who aren't really familiar with assembly and really low level stuff so if that's your thing you can check out Time to Bleed in Fiverr find any good reading materials there I'll definitely post them and link to them what's up you have a question works on RE MRI anybody else how does this compare to the state of either JRuby or Python I haven't looked in JRuby but I can so all this evil stuff I just did I can do the same thing in Python because I'm not scared I mean it has to be rewritten so it'll work with the Python VM obviously but the same tricks will work because Python you can load C extensions so as long as you can load a C extension then you can do whatever you want I actually answered the previous question about the materials on that stuff so there's a book it's called Programming from the Ground Up it's a free book so you can read it online and it has all this stuff on Linus assembly and it's very easy to read and the memproff source is actually really well documented we've been working hard on it like the last couple weekends Jake and I so you can definitely check out the memproff source to get some information as well what's up man so dynamically insert details so the method yeah you can inject you can so like using like sort of this base framework like the way like memproff actually works you can do whatever you can do literally whatever you want you can rewrite Ruby to do it like if you just say like hey I don't like the threading implementation you can just like remove the one that's in there and insert your own and do whatever you want yeah you can do anything dude alright cool thanks a lot guys


So, I think it was a bigger year than perhaps we all remember. It's different, like, when there's this storm of react news all the time, it's easy to forget what's going on in Emberland, but as I was going through it, putting these slides together, it wore me out. There's so much happened. So, in March, Embercon's 2015 was some pretty enormous announcements, glimmer among them, an amazing lineup of speakers, apart from one Englishman who couldn't really keep it together. I don't know how many people went. I attended and absolutely loved it and attended previously and loved it too. This was an awesome, awesome conference and we managed to pull off one in London too in November and I think this turned out really, really well and it's something I hope we can repeat this year if circumstances allow. The RFC's process appeared last year. I feel like we've always had this and it's so important to the way the community works and the way new features land that I can't remember living without it, but this actually landed at the beginning of 2015 and I think it's now really fundamental to the way new stuff happens in Ember. Also Ember twiddler pit, you know, I think for a long time JS Bin was, you know, not to knock JS Bin. It's fantastic, but it was kind of good enough for the purposes of demonstrating Ember things, but as Ember CLI took off, it started to drift further and further away from the experience there and Ember twiddle has course corrected that dramatically. Now it's the thing I think we all reach for when you want to quickly sketch up an Ember app, share that code with someone else, collaborate on a problem and Ember CLI itself has been just astonishing this year, especially recently as performance improvements have snuck in from Steph Penner and build times have dropped by a factor of 10. I think this has become the delivery engine for the Ember message really. This is what's selling people in. And Babel, Babel 6 released on stage at Ember Conf. I think Babel's changed everything about the way we work with JavaScript. And it's weird to think that before Babel, again, ESnext and Tracer were these tools which people were experimenting with but didn't see adoption. It seemed like maybe just complicating things and then Babel arrives and Babel's, it's like a no brainer. It's just got that right feel. It operates at the right level. It just does all the right things. OK, so hard problems that we solve in the Ember community. Simulating data sources with Ember CLI Mirage or the Mirage project. Hands up, who uses Mirage in their apps? That's pretty much everyone. Excuse the croaky voice. This has been a real game changer. And it seems to have taken off in every step of the stack. Some people run production sites where the data is coming from Mirage just because it's such a nice way to model it. It makes testing easier. It makes development easier. And the people behind it, Sam Selleckoff and the team, just seemed so amenable to new ideas and so friendly. Deployment, Ember CLI Deploy has revolutionized this with the perfect mental model. And again, an incredibly open and welcoming team, including Aaron Chambers, who I don't think's along tonight. But he's done incredible work on that. Animation, this is another one where before Liquid Fire, it just seemed like everyone was punting on it. And there were answers here and there. And it was like, user didn't set element, hook, or just kind of roll your own solution. And then Liquid Fire landed. And it wasn't that it immediately solved everything. But it definitely gave us the primitives, I think, out of the box. The idea that the way you could declaratively mark up your templates to encapsulate animation. And I mean, this is maybe a half tick as fastboot is still alpha. But I think the progress so far is really, really promising. And fastboot definitely addresses this problem at every single level. And I think it's a really impressive act of thinking through a problem, like really, really thoroughly, like all the way to the end for every possible angle. And when it's out, it's something that should solve it in a one-line install for almost all of us. HTML bars landed in early 2015 and then was quickly followed by Glimmer and now Glimmer 2. And I think it's interesting the progress that has been made since HTML bars arrived. Handle bars seem completely fine for so long. The script metamorph tags seem completely fine for so long. But once the back was broken of that problem, it seems like progress has just come in more and more rapid iterations. Ember data became stable and has now hit version two in lockstep with Ember itself. We'll talk about JSON API in the next slide. But Ember data, I think, has reached a point where it almost, dare I say, it feels quite lightweight now. Like it's not the heavy cumbersome burden that I think we might have felt it was at the start of 2015. Now, its role and the things it does are very precise, very surgical. And JSON API version one was released. Now, this isn't Ember specific, but this is something that the Ember community was heavily involved in. And we all stand to benefit from. There are some good podcast interviews with Yehuda to speak about this. I think the last episode of the change log, he covered the philosophy behind it in quite some detail. I'd recommend checking that out. OK, now on to resources from the community. Ember 101, formerly Ember CLI 101, went creative commons and free. This is a wonderful introduction to Ember. So if you're thinking of learning Ember, you can pick up this book for free. You can contribute to it. It's excellent. Rock and Roll with Ember JS by Bama Turdy. I've not read this personally, but I know people who have and say it's fantastic. One from Pragprog. Matthew White, no relation to me, I don't think. I've heard this one's excellent too. Actually, if you look on emberwatch.com, there's a stack more fully fledged actual paper books on Ember. Ember Observer landed. This was announced by Katie Gangler at Ember Conf in March. Again, it feels like we've had this for ages now as well. It's become so important to everyone's workflow. And it's become, I don't know about all of you, but this has become my go-to resource for learning about add-ons. And also, once you have an add-on in the wild, getting that all-important 10 out of 10 on Ember Observer is the coveted prize. Ember Screencast. I've been really impressed by this. When I met Geoffrey Biles for the first time at Ember Conf, he said, yeah, I think you're starting up a regular screencast thing. I've got the domain name. I think I'm going to do it. Just churned out content, video after video, week after week. And these are excellent. So this is now a well-established, trusted resource. Podcasts, Ember Land. This is the one where Robert Jackson tells you what's really going on every couple of weeks or so, while we're subscribing to. Ember Weekend is a great companion for that. That's a couple of guys from Hash Rocket, I believe. That's really good. And the Frontside podcast, which isn't always about Ember, but largely is and often has interviews with people from the Ember community. And this is a brand new one. It's the podcast companion to the modern web UI meetups in San Francisco. And this one kicked off with Yehuda being interviewed by. I can't remember at all, but it was excellent. And the global Ember meetup is a really interesting new entrance to all of this. This is a virtual online meetup. Miguel, you're speaking in the next one, is that right? Yes, this Saturday. This Saturday. And probably the same time with the extended version of the tagging with the new. OK, for the uncut talk, tune into a global Ember meetup. OK, so that's the tip of the iceberg of what happened in 2015.  So I want to talk a little bit about what we've got planned for 2016 or the kind of things we're thinking about. This is about Ember London primarily. There's only a few points. I'm nearly at the end of this gigantic ultra-Rambel. So the first thing is diversity. Now, we are really proud of the community that's grown up around Ember here in London. We think it's done great things. We think it's contributed back to Ember in a fantastic way. But we're never as diverse as we want to be in terms of where people are coming from, in terms of gender identity, everything, the kind of context people are coming from. And we want to try and improve that as we grow. We think this community is going to grow this year. And as it does, we want it to grow in a diverse way. I spoke to a diversity consultant on the phone last week, and one of the things she suggested was to, it wasn't just a matter of me standing up here and saying, we're going to do this. I need you all behind me to do it. When I say behind me, I mean behind me and Ken and Nick, you know, organizers. So I want to give you some of the reason why we think it's important. So that first one, a richer mix of experiences and perspectives, is something that we believe feeds into the quality of the code we all write, of the kind of user experiences that we all write, of the kind of conversations we have together, the kind of apps we build, the kind of problems we try to address. The more and richer variety of points of view we have in the community, the better we think we'll be. And this is something that needs to happen for our industry as a whole. And if we can play a part in that, then that's wonderful. And finally, we just think it's the right thing to do. So what's the plan? Well, we want to have more diverse speakers. How are we going to do that? Well, largely it involves fielding speakers from other parts of the industry, from other meetups, from universities from academia and this kind of thing. And so this is something we need your help with. So if you attend any other meetups or conferences and you see a speaker and you think the kind of things they're talking about would be of interest here, then we'd love to know about it. And similarly, whenever I'm at a meetup and I see someone give an interesting talk, I always invite them to speak here. Secondly, we are beginning the process. We've always linked to the confcodeofconduct.com, which is a good kind of generic base for codes of conduct. But now we're taking a step of having our own so that we can spell out our specific policies around this. And I'll show you a bit of that in a second. Marketing is something that we don't really do in any significant way. We tweet about these meetups. We email the full mailing list, a bit of Google Plus. Beyond that, Slack a bit, I guess. But it's kind of like it's more word of mouth than anything at the moment. So we'll be looking at how we can reach out to wider audiences. And apart from that, we didn't know exactly how to approach it, which is why we've spoken to a consultant, Astridon, who's fairly well known and runs AlterConf in the US. And what I hope to do is commission her to work with us throughout the year to really, really do wonderful things and tackle this and make ourselves better and more diverse. So the Code of Conduct, we're taking a new approach with all these bits of the community. This is GitHub, EmmaLondon, slash Code of Conduct. This is the first draft that's in there at the moment. And I welcome everyone to come and give your feedback, make poor requests, suggestions. Broadly, what it says is we won't tolerate any kind of harassment of our members. Everyone who comes here should feel welcome, should feel comfortable, should feel safe. And if anyone is harassed, we will deal with it. We will have a clear policy on what will happen, how you should report it, how things will be treated in confidence. So the idea is this will make everything absolutely clear so that if anyone does have any fears about coming along to this kind of event, this should hopefully address them as best we can. So to support all these efforts, and largely to pay the commission, the consultant, we'd like to hire, we are seeking sponsors for EmmaLondon Diversity for 2016. So previously, when people have approached us about sponsorship, we've said, well, you can sponsor food at the Project Nights. And beyond that, we don't really have that many costs. This is something that would be great to put money towards. So if any companies are interested in helping us with this, we would be delighted. Next thing for 2016, Project Nights. So who's been to one of our Project Nights? So the way these work is usually they are the week after the meet-up. There's not one in January just because there's not enough time to organize. So one week after the meet-up, and the idea is it's a similar kind of gathering of programmers to work on any Ember project you fancy working on. So that could be making a pull request to Ember itself. It could be your own toy project. It could be your work project. Could be just to come along and learn or teach or hang out with people. But it's a meet-up without the talks effectively. And the thing that's different this year is that we want to host them in more different places. So last year, they were all hosted at the office I work in. This year, the plan is to build a roster of different venues around town with different companies. And we will move between these different venues each month. So you can probably just about make out who's on the roster. So far, we've got Alpha Science, Trover, with Associates, who I work for, and Zatnito. Again, if your company would like to be on this roster and host one of these Project Nights, make a pull request or email us. And this is a new idea from Deepan from British Gas, the Ember London Exchange. And the idea with this is for companies using Ember to exchange ideas, give talks at each other's offices, set up video calls. He's got some cool things in mind for this. But I think primarily it will start there with the kind of brown bag lunch exchanges and things like that. And again, we are taking the tack of hosting these things as GitHub repos. So it's really easy to contribute to them. Make changes, give feedback, and everything. And Ember Conf is coming up in March. Who's going this year? OK, I can't make it. I'm very sad about that. But it's going to be amazing. The lineup looks absolutely incredible. Enjoy it. So that's all from me. I'm sorry if it was a bit long. So, Ju, if you want to bring your laptop up, I'll get on with the first talk.