All narcissists end up losing. They ghosted you, they discarded you and They found someone else They rode off into the sunset and you were left broken and destroyed After they moved on to someone else, they demoted you They moved you to a lower position usually as a punishment and they put someone else in your place They replaced you Which caused you to experience overwhelming shock and grief after you have to witness them enjoying this new situation with their new supply You saw them going on trips and they seem to be happy and not thinking about you Which affected you it brought you down Which is why you shouldn't look at what they're doing. It's why you need to stay off their social media But if you've already seen it, it should wake you up It should give you a reality check Because they tricked you they got you to open up to them. They got you hooked They made you believe in a future with them as though everything was going to be fine, but then They ghosted you they moved on and Then they didn't want to talk to you anymore They changed completely Which left you in a state of confusion It messed with your mind because you thought you had something but then you discovered that they didn't even care They weren't even concerned about you Because when a narcissist decides to rub off their mask and Show you who they really are It's not a pleasant thing to see because they don't show any compassion for you Instead their sole intention is to punish and harm you To make you feel like you never meant anything to them Despite all of the time you spent together and everything you invested in them You're left feeling like you never even knew them and that is what shocks and overwhelms you Especially when you see them with someone else and then laughing and smiling. They're enjoying themselves They're having a good time and that is where many of you find my videos Because you don't understand how they could do something like that, but now you have the knowledge and understanding You're beginning to learn about what happened to you Because before you found my videos it didn't make any sense, but what you need to understand Is that they have relationships with more than one person at a time so you were never exclusive Because you were never number one You were never the only person who mattered to them They always had other people on the side who they were entertaining But you were given the illusion of monogamy as though you were the only person they loved and cared about But they were just playing games with you They were cheating you and playing you for a fool because they're never dealing with just one person at a time You were one of many people Who were being given the same treatment at the same time? They were never exclusive to you They just knew what to say to make you feel like it was just you and them because you spent so much time with them You spent hours talking to each other, but then they ghosted you because they were busy working on their next victim And then they came back to you again Because they needed to ensure that you were still longing and yearning for them Because after all of the love and attention they initially gave to you, they know you're still going to be waiting for them You're not just going to forget about them and move on You're going to hold on to them because you can't accept that it's over You thought this was just the beginning But you weren't the only one that they were doing it to you thought you could have had something with them You thought there was so much potential for growth and development in the future But there wasn't Because you were never exclusive to a narcissist You are never the only one So you never had a relationship You never had something special You thought you had something Because of what they were displaying to you But you were never theirs Which is how they moved on to another situation And they probably already had that person lined up before they even left you And they were giving them the same treatment that they had given to you They were never exclusive to you They're not exclusive to anyone but themselves and what they want and once they've moved on to this new person It's all over unless you're still waiting for them because you're hooked Because you're hoping there's still a chance that they could be exclusive to you Because you think that no one else can make you feel the way that they do you think you've lost something But the only thing you've lost is the validation Their ability to control how you feel When true happiness is generated from within You should be able to make yourself feel that way on your own Because that was a vulnerability that the narcissist noticed in you And they were able to take advantage of all that and use it to control you All narcissists end up losing They didn't win because you never had anything anyway You never had them, but many of you feel like they've won Many of you feel like you've lost Because you don't want to let go of something you thought belonged to you You thought you lost something But how could you have anything if they were never exclusive to you if they were entertaining other people from the very beginning? They just knew how to get you hooked And because they were never really invested in you It was easy for them to shut you down But they didn't get away with what they did to you They don't just move on and continue to advance without you They all go down in the end They are miserable people They're never happy with anything and they always end up being unsuccessful in achieving their goals Because they're very neglectful They don't give the proper care and attention to things that should be considered valuable and important So their lack of care and interest for you Should reveal to you how things are going to turn out for them Because their own attitude behavior and characteristics Ends up coming back to haunt them. It comes back to get them in the end Because they end up suffering How a person treats you is a direct reflection of how they feel about themselves So if they don't have any concern for you It's because deep down they don't value or love themselves But eventually they over exaggerate their performance They end up doing things they said they would never do They become the very thing they hate and then their lives are exposed And they reach a point where they're unable to continue or achieve anymore They might experience short-term success But it won't last for long because nothing good can sustain itself in that type of environment Just like a seed needs sunshine and water to grow Nothing good can develop in an environment Where there is no love or happiness Which is why narcissists do not continue to succeed or do well They all go down eventually Thank you for watching If you found this video helpful please give it a thumbs up Share your thoughts in the comment section Hit the subscribe button to receive the notifications If you would like to support the channel You can donate at paypal.me-naq-survivor You can book a one-on-one with me on my website It's Naq-Survivor at the Godu UK Thank you for watching and I'll talk to you soon


Hey, we're back this part three to our movie reaction to what? Hum dill day jukes an arm. Yeah, you say better than I do. I don't think so. Ah, it's probably chook not chook chookie The 1999 sounds a little bit on solid films. I'm a con Ashwari all right If you saw some YouTube obviously you saw very cut-up version if you'd like to see the full-on cut version though You can head over patreon. Is this part three? Yes I'll put the link in the description below in the pin comment and at the end of this we will talk about the movie and review it Okay, okay. All right Let's just get back into this It's the guy from the drawing man He's Jesus Just a pika sing this in rem lila Maybe maybe she's just saying drum Dosts yes, the couple of ghosts in love with the same girl Who will win in the end?  He doesn't know him They were making out Are you gonna have to do more than that I'm gonna take it, please Italians have no idea what to do when people are being intimate Yep, don't look this up Italians aren't very passionate people Don't look that up. Oh Samir your paneer is in your pocket Wait, that's not Samir the other guy. Is that paneer in your pocket or you're just happy to see me I don't think you should store paneer in your pocket Is that a mere con in to whom this slaps There's not one italian in this entire movie Thanks random stranger italian ron swanson He's like he's a bit extreme sometimes I'm rethinking this You're like a hundred let's bring it down to like a 20 I think you did a lot of coke before I do this She says i'm having second thoughts Poor by it's twice now for having loved you Same Essentially been solid directed this all right cool. There's no post-credits scene or anything like that what Maybe someone comes back and is like I changed my mind Uh, that is twice now Well, they just need to do the follow-up movie about I thought this was the follow-up movie the Her eighth life and then he gets her Yeah, I guess What did you think I liked it I thought it was absolutely beautiful and I loved the songs This is a masterful soundtrack. Yes, and this one gorgeous. Especially that last No, I can't sing like it But I loved it And um, I really wanted her to stay with her husband. So I'm glad that's what happened Uh, well salmon was like on coke there at the end and it's like So she probably got put off by that. She's like I don't want to be with a drug addict Yeah, um It was I enjoyed it as well. It was uh, it was what I was not everything I was expecting There was some very unexpected fart scenes in there that I that was jarring Funny, but I in Sanjili Benzali film these eloquent beautiful things I didn't expect a fart joke. Oh my gosh. It was still gorgeous. Yeah, it was Funniest thing though was when he knocked over The thing when they were coming down the stairs and then he meowed and she was like there's no cats in this house Super great, um Yeah, I feel bad for by even though like there's There's a bunch of like dated things in this in in in this film of like certain things that are said or like like that extremely weird scene where he is just Groping in on top of her and she's just saying no no no And then she she rightfully slapped him because he's not listening and then he says she's not worried.  Yeah, I didn't say whore, but um, maybe he did he said shit like 15 times in this which I'd never heard before um But yeah, like then those other instances of stuff like that would happen that He's like, oh, but I think a lot of that stuff was put in there to show what real love is and what Not so real love would be yeah, I guess probably I was happy she ended up with a j. Defkin in the end for sure I think he uh outside of dragging a woman across the house um and Staying up till four in the morning Those I didn't even understand what that was Yeah, I don't know why you like see okay. I'm gonna give I have to be up early in the morning he stays up till four to Right at four to have sex with her right before she prays That was a little strange. Ah, but anyways, um Yeah, I enjoyed like the the score and this is magnificent Sanjay liban solies, um It's not like it wasn't like, um Padma Vada Bajorau in terms of that level or even dev das Which is a step above in terms of just the the beauty, but there was a lot of extreme beauty in this It was extremely gorgeous. Um, you'll like dev das a lot. Absolutely gorgeous. You're gonna like dev das But this yeah the score was amazing the cinematography and the direction By it was one of the more quirky things. I've ever seen him direct though There's like a lot of like Almost 90 moments 90s bollywood moments at times that I think you know, I I don't normally associate with him Just based off of the stuff I've seen maybe I've just seen his more later works Yeah, um In and then four jokes how early into his career. What's this 99? Uh, well Probably a fee I think early 90s he started Or at least was directing in early 90s. I think Uh sanjay liban solie where that's a profession by direction Let's see This was his second directorial debut Um First was another salmon kai's apparently they're good friends. And I think it's also a short. Is that a shorya? nani peticar No, it's not a shorya. Okay. Uh, so this is the second film. Um, so he was still kind of finding his voice um Back then. Yeah, um, and I you could still tell it was him like extremely. Yes, um But I mean, he's lucky that uh his main actress like plays into what his Man, this was Go ahead. It plays into what his strengths are beauty like Yeah, so, um This was the a shorya rise show man.  Um in terms of just I thought everybody, I mean summa kana had his moments and he brought a lot of the more of the Humor, I think probably the film then than everyone else did Um A jay was was did well and all those boring guys. I thought the outside of the white people did well Um She was the one that was like she was the powerhouse. Oh my gosh in terms of active formants dance the intense She always she was on and every single. Yeah, she always brings the intensity man the uh her eyes You just can't watch something she doesn't be like see anything other than these She is like More expressive zoidish and allies Like just more intense more Fiery sad whatever emotion you need from her at that moment. It just comes out Yeah, um Just gorgeous. Her dancing is impeccable Um She was she this is a shorya rye Star vehicle uh this one Uh, so yeah, so this will be the earliest works. I've seen of him because uh The one earlier thing was in 1996 And this was in 99 dev das was 2002 black Which you have seen. Yes, this is with ren beer, which we haven't seen um Gershnach we've seen rem lila bhaji rap pan mavat This one that's uh the alia one the most recent alia one I think I think he's teaming up with yeah ren virnale again That's just gonna be amazing in a another sanjay lila mensali film Uh, which is gonna be fantastic. So wait, so am I almost done with Sanjay films So we haven't seen the ren beer one And we haven't seen his first one And then that's it.  You're right. Yeah, it was theaters. Um, yeah regardless, um This was uh, like I said the ishwari a rai show Uh with an impeccable score Agreed a fun Fun at times Uh Movie that had a lot more quirkiness than I was expecting I feel like Um a lot more like jokes and weird stuff Going on with salmon guns. I still don't quite understand. I guess it was just his father.  I guess god let him control the thunder that day or so it was confusing And I didn't fully quite understand it. Um And there's so many other like poignant moments of like Like they they show these families as like the the the dad's nice and they didn't remedy the families at all in this actually like Did the dad ever sing again? Did they just disown her? I don't know. Are they just gonna kick her out they kicked it They told the other girl who's getting abused to go back to her husband. I think I mean because Her husband bridged that gap with his dad before they left and it seemed like the dads were friends so maybe Maybe there's gonna be some sort of conversation there, but yeah, no, they never that never came full circle Yeah, well, I guess there's a need to she ended up being with uh jade devgen like they wanted so Um I guess they would have to be in a dress if they he ended she ended up with salmon I will say though that I would have thought Her mom would have like I don't know fought a little bit harder for her daughter But never well they were all about to send that girl back to her abuser, so That shows me what kind of people they are Yikes And we've seen that in many movies obviously before like especially like I guess I don't know when this is exactly Supposed to be I don't think it's supposed to be taking place in 1999 Or maybe I'm wrong. Maybe it is modern day uh, I guess In italy which they definitely did not encounter one real italian The entire time they were there that was just so random I think two crazy white people robs them shoots them runs away Some escalated really quickly that's over there doing coke lines off of uh Uh Anyways, it was uh, it was an enjoyable movie. I enjoyed it. It was um, it was what I wanted from a big bollywood um Venture in the 90s with some beauty and and epic songs and an epic performance by um Ashwari Rai and some fart jokes, which is just to top it off Uh Totally got me off guard. I was not at all expecting that I might even be one of the first fart jokes. I've seen all of indian cinema I don't even know if I've seen another fart joke.  Yeah Uh, but yeah a lot of fun, uh, really enjoyed it. Uh, love a good romance film um Let us know what you thought about the movie as a watch along. I hope you enjoyed it as well If you're watching this on youtube, obviously you saw a very cut up version of our reaction If you'd like to see the full uncut reaction You can head over to pay a turn up with the language scriptable and pinned comment You can check out this uncut reaction and others from rick and I to stefan. I uh a whole bunch And future ones obviously you'll get to see those Usually days to sometimes a week or so before they come out on the channel Um, so you go check those out.  So maybe and you didn't like it, right? I liked a lot of the elements of it in terms of songs cinematography score I think I even said I enjoyed the main threes performances But everybody else was and even them sometimes was insanely over dramatic I don't know if I'll have the problem anymore. I'm pretty accustomed To it and at that time. I'm pretty sure it was like four months in To the channel. I wasn't Still I am now Full circle it's come full circle. Anyways, let us know what the next watch along with stef should be Down below and Bottle the next watch along


It's going on everybody got Kyle and Sean here a little stash and trash action We are ripping 2023 certified football four box break 56 picker team saw got Charlie P with the Raiders in the Ravens Cory G has the Bengals Packers Vikings Dan B with the Dolphins D. Kill it has the Cardinals David H has Eagles David M with the Chiefs Emanuel L has Texans Heath F with the Cowboys Jake R has Bronco Seahawks John S has the Steelers John W has Rams Patriots Commanders and Saints Josh B has the Giants and the Jets Joven H has the Chargers we got Keith B Browns Nathan A with the Panthers Nick C has Titans Patrick H Has the Bears Paul H has the Falcons Richard H with the Colts Robert L with the Bills Scott M 49ers Tony K has the Jags Tyler H with the Lions and then Vinny S has the Bucks Alrighty guys. Good luck on this one. Let's see what we got see what we got Just eight spots to go on Star Wars last break of the nights Do have a little bit of space for personals not a ton though guys if you do want to get an order in Recommend doing it soon. Otherwise, it might rip with force tomorrow night.  Let's see a big CJ numbered here. See something big. There's a 275. That's gonna be a Lou Nichols Packers rookie bronze base base base Certified stars is Jameer Gibbs And redemption is Devon a Chan or a chain for the Dolphins II Freshman fabric mirror signatures nice one there on the base side of for the Dolphins Devon a chain There's Sam's boy Roku on Chris Olave Chuck Woodson and a 99 we got Jalen waddle II Another one for the Dolphins.  You know that play was gonna work And orange the 249 got Sean Clifford new generation Packers On the Jersey QB out of Penn State Travis Kelsie certified stars There's a Tyree Wilson a 399 rookie Raiders their aides Told Donnie has a good one. I need to hear it Micah Parsons and blue this will be Josh Allen to 49 certified baller Sean's gonna watch him lose in a few weeks Certified ballers got that right see if that brings a shady out right there. Yeah, come on. See that brings a shady out There's the mortals Marshall Fock we got Jalen Hertz to 140 not nice one there eagles Eagles Thank you Keegan for the follow on twitch you get there's a mirror signatures Rodney Hampton Throwback there to 99 mirror signatures Gman Rodney Hampton throwback Jerseys or something man box number two. Good luck here guys Keep her moving And we got an eight box mixer next and then we're on to flawless flawless Wallace sold out for I didn't got in here guys. I appreciate that We got there. You keek Lee got to 149 Felix Uzama for the chiefs Derek Henry. There's a rookie at 325 Devon weather spoon See I've been getting pretty lucky with defensive draft picks recently Here's a Jamal Anderson in 99 Immortals and RPA is gonna be Jonathan mingo to 399 Panthers on the RPA Jonathan mingo Travis Kelsey Gold team Travis Kelsey base Certified stars sake one to 275 Mack Jones Thank You Paul grabbing a star Wars spot Appreciate you mr. Paul actually that brings us down to eight. I was wrong before there's a Justin Tugger to 249 So eight to go on that 399 Luke schoonmaker new generation Just eight to go on Star Wars Shake that get some random mini boxes going hopefully see some sick sketches some really nice autos Trevor Lawrence. There's a tank Dell nice one there Texans a 275 Really good rookie season for getting hurt It's tanked there him in a CJ gonna have some connections going for the next few years Roquan Gold team and a 75 certified ballers is will levis Will levis certified ballers. He kind of is a baller man. He has been He's been playing pretty good Certainly think he's better than Tannehill. So I think he'll have that job next year Oh for sure. I think he'll have that job next year. He's good certified stars McCaffrey and muck's Musgrave in 99 Luke Musgrave a Metcalf and that's gonna be Darnell Washington to 149 for the Steelers Rookie Otto there the big man out of Georgia Darnell Washington Ronnie box number three luck here guys That's right. Dolphins legend Tannehill He certainly was man He certainly was You one second guys. Let me tweet out this Star Wars real quick rights. Good luck here guys box three Case 56 or four box 56 Jason e.i. Any update on the girl on the sixth floor, bro? What's going on with that? What's up going on with the girl on the sixth floor? Jayce Sean's gonna steal her Goodness gracious. There's a Justin Herbert and Michael Mayer to five this time the second one First one was to 10 now. We got Michael mayor to five. There we go Raiders That is sick on the true green rookie Michael mayor is a Michael mayor case right here It's a ten and a five Good lord immortals there. We got a Miles Murphy to 399 rookie. Yeah, right? Just the 101 and we were getting close Got the bigger ones out of the way They dial and cook and Michael Wilson gonna be the bronze RPA to 349 Michael Wilson for the Codinals Joker Cordero Patterson Chubb Gotcha ball. It's area McLaren and a 299 Chad Rylan rookie for the Patriots.  Well, it goes Paul It's Joe yourself. Yeah, right? Absolutely, the Kellen Winslow and to 299 new generation Chase Brown. Let's go Paul. That's awesome. That's awesome Calling cards lamb jacks a 325 or a she rice And she spent having a good year About the only receiver on the cheese that can score Touchdown 80 Perry We just had I think so There's a rookie gonna be a Bryce young to 349 rookie bronze. I was definitely thinking the over in this one. Should have gone with it Should have gone with it 80 Perry. I like it. There's a certified stars and Bronze is gonna be a 275 Grady Jarrett Get a nice one. Uh, Jamie's doing a nice touch a couple weeks ago. They're really nice catch on it Mortal someone 49 Marshall Falk and that'll be Fred Taylor. Nice one there to 35. That's Vaughn's boy or the Jags Fred Taylor Used to love him watching him and mojo Nice one there on the mirror signatures Wow, I happen quick Alrighty guys box for this is for box number 66 Yeah, yeah, I got a I got a pretty nice mojo Finest finest auto I'm back in the day. I got a couple years back I really like Maurice Jones drew when I was growing up man. He was one of my favorites There's a 199 Steve young and that'll be to 99 Shaq Leonard the newest eagle Eagle Yeah, I know that happened quick man blue nickels to 399 Pack G willy CJ JJ and to 399 freshman fabric a no Connell Raiders keeping it going Ain't no Connell in the freshman fabric RPA. Nice one. They're these starting QB for the Raiders Derek Henry certified stars Jalen waddle II Alongside. Oh puka. Oh, yeah, let's drop that. He got it Hey, Justin Fields Frances foundations and a 199 Desmond Ritter on the pink Desmond Ritter Hey, there's Paul with the masterwork spot. You're right this time. Thank you Paul with the masterwork spot I appreciate you my man down to seven down to seven on our last break of the daytime guys or the day Daytime certainly not that anymore There's a Jonathan mingo 249 new generation RPA Panthers. Do you know how many yards puka has? I I think I might be playing against him kind of worry about that 275 jack Campbell rookie bronze. I know he's got touchdown But hopefully doesn't have like a hundred yards or something Having to pay too close attention Okay certified stars DK And DTR 249 in the orange new generation jersey brown browns color match on the orange Over 150 oh god Last thing I wanted to hear next to him the last thing I wanted to hear. Oh boy. Oh boy He has well over a hundred. Oh man Dang it puka and to 275 Matthew Stafford. There's a guy throwing to him puka. I'm on puka Not the not the puka and shatty Now you're good next to me all good And on the back is going to be javon curson 99 nice when they're titans javon curse 299 the freak the freak All there goes there's more puka There you go. Alrighty guys that will do it for a certified four box number 56. We'll get an eight box mixer going next Let's do a quick hit recap Got eight box mixer and then two flawless cases and then we are on to um Star Wars We had patch DTR patch, uh, jonathan mingo brice younger 349 Had chase brown to 75 will levis to 399 schoonmaker 49 was um josh allen and a 249 was shonk cliff Hey gachi david Appreciate you my man. Appreciate you grabbing a spot Jays don't delete that please nine for 164 in a touchdown. Are you kidding me? So he's got 32 Wow Hmm not what I wanted here 99. We got uh javon curse RPA aiden o'connell. That's a that's a puka game right there. He's had a few of those this year 35 fred taylor There is um michael still got it michael wilson rpa. We got darnel washington jonathan mingo rodney hampton and rpa is going to be a or sorry just a auto is going to be davon han I know I saw just saw that one josh. We'll keep on uh adding to the tally. Yeah, whatever That'll be all right Hopefully tyreek can catch it up or something like that to tyreek do what they do you win Alrighty guys appreciate y'all on that one number 56. We will get the mixer going next


Hi I'm John Davies from Intel Corporation. We had the privilege of being able to cosponsor four sessions today on education ITU telecom world. I was part of the first session which was kicked off and moderated by Tim Unwin and it was the ministerial session. Over 10 ministers joined that session and they shared both their implementation practices and their challenges and issues on education transformation. All of the ministers I think the consensus of them was that teacher training was absolutely critical and delivering content in many different languages and sometimes indigenously was critical as well. It was very interesting around the ministerial table because as people brought up these issues it was also clear others on the table had addressed some of those issues and solved them so because the ministers got together the best practice sharing I think was just tremendous. There was really the value that came out of the session. What was very interesting was to watch a number of countries from Africa that talked about content and then along came Zambia with this amazing interactive content portal and out of that the countries were getting together this evening to share the best practices. Now the session was summed up I think beautifully by Prahima Sunoo that said there's a cost of implementing education but the cost of not implementing it is much higher.


Ruby refactoring with confidence and science. So I am Jesse Toth, I'm Jesse++ on the internet, GitHub, Twitter, everywhere else. I am the head of web technology for GitHub, which means I work on the web team behind github.com, work a lot on our Ruby on Rails application, especially focusing on database and performance things. And this talk especially will talk about one of the projects I did around those sort of things. So, tackling a large Ruby refactoring. What do I mean by large Ruby refactoring here? Well first, let's break it down, large. Either it could be the size of the system that you're trying to change, or, and especially this is the case that I'm talking about how risky it is to make this particular change. But then, what about refactoring? Traditionally refactoring shouldn't be risky, right? It's supposed to be just changing the structure of the code without changing your behavior. Usually you can do it in small pieces, and if you have good test coverage, you should be able to catch any accidental behavior changes, so it shouldn't be risky. But really I'm talking about, in this case, sometimes really risky changes, or maybe going a little bit further than your typical refactoring, not just changing the structure of the code, but maybe changing some of the underpinnings of the code, but still without changing the behavior of the code, such as maybe switching out a service or a gem, you want it to behave the same way, but you want the code that you're using underneath to be different, or maybe you're optimizing a query, moving it from talking to your model to raw SQL because you need it to be really fast, something like that. So any code interacting with this code at its boundaries shouldn't notice a difference, but things are gonna change underneath, and hopefully be better for whatever your metric you're trying to improve, which is usually performance. So maybe a better word for this is reworking or replacing. In some cases this might work, but I didn't want to use that because I don't think it emphasizes enough the fact that your behavior really shouldn't change when you're doing things like this. Rewriting is really the best word for this, but if you've ever done a rewrite, you know that that word has lots of baggage. Most people have some PTSD around the fact that rewrites usually don't work. They're very risky. If they are of any size larger than just a little rewrite, they tend to drag on and on, and more often than not they fail because you failed to duplicate the behavior of the original system and then you're left with either something that's not matching the old system or you just have to throw it away and I'm gonna try again. So that's very risky. But anyway, I'm gonna talk about some of the tools and techniques that we use at GitHub to tackle these sort of problems, and I'm gonna do that talking through a case study which was a rewrite that we did of the permission system behind github.com. So what we had behind github.com to control all of the permissions was a huge legacy system. And the thing that we wanted to do with this rewrite was to create a more flexible system to grant and revoke access to any of the things that you could have access to on GitHub. So repositories, forks, issues, pull requests, teams, organizations, basically anything. As you can imagine from that description, this would be a pretty far-reaching change.  It's a huge service area and it's extremely risky. So if you're gonna make a change like this, you have to have absolute confidence in the correctness of the new system when it's time to switch it out. So first of all, the first question to ask is why would we wanna do such a thing, especially with such risk? So to give some background on that, it requires a little bit of a history lesson on GitHub itself. So in the beginning, there was collaboration. So one of the very first features of github.com was that you could add other users to your repository as collaborators and allow them to have access to your code and to make pull requests and make changes to it. And then that worked very well for quite a while, but eventually got to a point where we needed a little more than that. So then there were organizations. As GitHub grew, we needed to place a way to give repository access to groups of people and to kind of control those for different groups and different teams in different ways. The problem was that these two different ways of granting access were actually written in separate ways. They were developed at different times by different people. And as most legacy code goes, it wasn't obvious at the time that these were kind of the same concepts. So they were developed in very different ways and represented in different ways behind the scenes. So this is what permissions look like to begin with. This was collaboration on repositories. So it's a super simple join table. Basically, you say, here's a user, here's a repository, they have access. That's about it. And then this is team members. So the way that you give access to repositories and organizations is through teams. This one's a little weird. And every time I look at it, I kind of shudder now because it's a strange three-way join table between, well, you have the team, first of all, but then you have users and repositories. And so to be a member of the team, if you're a user, you have a row in here. But also if you want to give users on the team access to a repository, the repository has a row in the same table where the repository is like a member of the team. And this is pretty terrible, but it's what we had. Hey, it worked for a really, really long time. So with that in mind, let's talk about how we would use these things to access permissions. So there were tons of places in the code base that needed to get lists of things that people had access to. So repositories, pull requests, teams, those sort of thing. And they all kind of access permissions in slightly different ways based on which kind of data they wanted. And in true legacy code fashion, even different places like the API and the web that wanted to access the very same list, they did it in slightly different ways because they were written by different people at different times. So here's an example of, this is what you would have to look at all the repositories that an organization controls. And probably this is only going to look at the team members side. It's not gonna touch the collaborators or permissions table. Whereas on the other hand, if you want to get every pull request that a user has access to, you've gotta look at both the permissions and the team members table and see what repositories do they have direct access to, which ones do they have access to via organizations and teams, and kind of put that all together. And what we saw with this is a lot of bugs and edge cases, especially around transitional states. And because, like I said, each place that was calling this was different, the bugs were slightly different each time. But the scariest thing was, like I said, around transitional states. And unfortunately we have a lot of transitional states on GitHub, so you can transfer a repository from one user to another. You can also transfer it from a user to an org or from an org to a user. And that would change how the permissions should be represented. Or you can do something fun, which is you can transform your user account into an organization, which should also change how the permissions are granted from the permissions table to the team members table. Or when you remove somebody from either a team or from your repository, you want that also to take away their access to the repo and that's considered a transitional state. And we started seeing problems with these lots of different bugs all over the place. And this was pretty scary to us and to users because when you say you're giving access or you're removing access, you wanna be absolutely sure that that's doing the right thing.  And beyond that, we started seeing performance degradation with this. So over time, as GitHub grew, and depending on the API that was being called, different places started seeing different performance problems, especially in our API where we had to grab large lists of things, like give me all of the repositories that anyone can have access to. And what started happening is that places that grabbed access to repositories started looking like this or grabbed access to lists of anything. They had, each time a performance problem came up, somebody would go in and optimize it and make it a little bit better. And eventually they just looked like huge chunks of raw SQL like this. But of course, they were all accessing slightly different things and they were all optimized at different times by different people. So these queries were slightly different in each case. Which of course means that it's difficult to build anything on top of this. Because everything's just this huge query, you really can't, I mean, if you wanna build something on top of that, you're adding another line to that query and potentially breaking the performance. So when our CEO, Chris Wandsworth, suggested, hey, let's maybe try to find a way to make orgs better, that sounded pretty hard. Especially because his suggestion was, well, let's change permissions in this way, in this way, in this way. But with all the problems that we had been seeing with the existing permissions, it seemed pretty impossible to be able to make any changes like this without first fixing the existing system. So that's where we put on our hard hat and said, all right, let's rewrite this. We need to be, we want to be able to rewrite the system in parallel to the legacy system. So we wanted to not touch this brittle old system as much as possible. And to be able to run experiments and do performance testing kind of side by side and use that in a way to test it without touching the other system. So we came up with these pretty simple goals for what we wanted, a super simple flexible interface to grant and revoke a general permission, so not specific to our repository permission or team permission. We wanted to be super fast since we were already seeing performance problems with the old way. Anything new we built need to scale much further into the future. And we wanted to be easy to integrate and operate with the existing technologies we had. We looked at things like graph databases and things like that, but it didn't make sense for us to go and try out a new technology. So we wanted to build it right on top of my SQL, the way the old system was built. So first thing we did was kind of spike something out. It was important to us to be able to test how the new system would potentially perform with production load as quickly as possible. So we didn't go down the wrong road and go way too far in building this new thing and find out that it just wasn't gonna scale at all. So the first spike was called capabilities and it was something that John Barnett worked on a bit. And it was a rewrite, but there was also a bit of refactoring that we had to do in order to even test this spike. Like I had said earlier, there were multiple points where even if the API and the website were getting the same list of repositories, they were doing it in different ways. So in order to be able to test this new system out a little bit, we needed to at least refactor those points so they were going through one single point and that way we could grab at that point and test the new system and the old system. But this is where we ran into a problem. So I think we tried to tackle the API and the organizations to see all the repositories that an organization has access to. But there was a problem with just doing the refactoring because it was this huge chunk of SQL. We didn't have test coverage for all the crazy complicated cases that we had in production. And even if we were able to duplicate all of those cases in tests, that would slow our tests sweet down a lot. It would still be hard for us to have confidence that we had tested every single case. There was also something that someone found where maybe there are bugs in the current implementation that people are relying on those bugs. So whatever we do, we have to duplicate the bugs in the system and make sure that we really know what the changes we're making are. So at this point, a bunch of engineers at GitHub kind of put their heads together and came up with this little hack to try to do this refactoring of this one path. And basically it was a way to test the behavior of the legacy system and the behavior of a refactored path together. Basically it amounted to dark shipping, the refactored path. So running it behind the scenes in production and then using our instrumentation library to subscribe to an event, basically compare the results of the two paths. And if they didn't match, throw the results of that into Redis and we could go look at that later and see what was the difference and why was there a difference. And this actually worked really well. It allowed us to gain confidence in these initial refactorings and it also actually uncovered some bugs in the legacy implementation. So like I said, this worked so well that we wanted to keep using it. So we pulled it out into a science library that anybody at GitHub could use if they wanted to test a new code path and they wanted to be really confident as they were rolling it out. So to give you an idea of what this looks like, we're gonna look at the pullable by method on the repository class. So to do science, you take your old method and you make a little science block and you give it a name. We tend to namespace the names of our experiments just so we could group them together because we ended up doing a lot of experiments. So then the first thing you do is you say, take the original code that was in that method, so the old pullable by method and we tended to pull it out and put it into a new method that we just named with like underscore legacy at the end and then there's the use block. Basically, you say whatever is in the use block, you want the method to continue returning that. That is the old code. And then you say, but try this other thing as well, which whatever the new code path that you want to refactor, we tended to pull it into a new method. We call this ability, so we named it with abs at the end but we said, okay, but also at the same time, try this new method, compare the results and tell us what the result of this experiment is. Also for usefulness in this, we added a context. So if things didn't match, we wanted to know who was the user, who was the repository that we're trying to test between each other. So then in a case where things don't match, you want to publish the results basically. And so our published method just uses active support notifications to instrument these events, which you can then subscribe to. And it looks kind of like this. Subscribing to a science event, you can subscribe to it and publish the interesting parts to like stats D, graphite, something like that. So we keep track of how many total times an experiment has been run and then do some timings on each part of the experiment. So the use and the try block, how long did they take to run just so we can compare that as well as the actual result. And then you can also subscribe to the mismatch event. So this is when things don't match. This event will be thrown and you want to keep track of how many times things go wrong, how many times do we have a mismatch? And then we just simply push the result of the mismatch into Redis so that we could look at it later. One caveat about this technique is you can only use this on code paths that have no side effects. So if you're doing something that's writing, you can't use this because especially if they were to change the same thing, then it's not gonna work. So using that we were able to do a little bit of spike and figure out what we wanted to do with the real system for permissions. So using that science we were able to get the refactoring into the little points that we needed to attach the new system when we did build it. So taking those lessons, we put together a team and we started building a new system that we called abilities and we called ourselves the ability buddies. So this is what we came up with for our system. It's super simple I think and I really like it. So you have actors and subjects and you ask an actor whether they can perform an action on a subject. So can this user read this repository? Then a subject would grant a permission to an actor. So you say if I'm a repository, I want to grant this user a right permission. And then to remove a permission, you just revoke any permission granted to the actor. The only additional constraint we had on this was that we wanted a cascading functionality for things that were both subjects and actors. So if I have an ability that was granted to a team on a repository and then I granted a user an ability on a team, we want an implicit ability here between the user and a repository to cascade up basically. So the user has access to whatever the team has given access to. But that was about it, that was the interface and once we figured that out, it was time to start implementing it. So we wrote the code that did the core of abilities in just a few months. It was pretty quick. So once it was written though, we needed to generate all the data to actually have the permissions in the new system. So we wrote some migrators that would iterate through all of the permissions in the legacy system and try to generate new abilities entries for each one. So the generated data, if you're generating it, it's only a snapshot in time. So then the other thing that we had to do in order to be able to compare these was to anytime that the legacy system changed, we also needed to change the abilities data. So any new permissions needed to be written to abilities at the same time and any changes needed to be updated as well. But once we did that, we were able to start doing some science. So we were able to add science to all of the read points that touch permissions and there are a lot of those all throughout the code base. And because there were so many different places that touch permission, we kind of had to split this up and parallelize it. So we had one half of the team doing the refactorings to find all the points where we needed to add the science. And then the other half was going through and using the things that already had science, analyzing the results of the experiment and seeing what was happening, were things matching, if they weren't matching, why and how can we fix that? So let's dive into what the data looks like that science collects and how we can use that to determine correctness. So this is dashboard that we built to kind of summarize and visualize the data from graphite that we had instrumented before. The graphs on the left give you kind of a day in the life of each experiment. So what does it look like in the past 24 hours? Is it mostly green, meaning no mismatches? Is there's a little bit of yellow in there, meaning a few mismatches and the ones that are really red are just, everything's kind of going wrong with those. And the right side gives a little bit of statistics on how often something's running. You can see how many times per minute it's run. What percentage of the time are we running this experiment? For things that are not very performant, you don't wanna be running this 100% of the time. Maybe you only wanna run 1% or 10% just to start collecting some data until you get less mismatches. So if you click into any of those summaries, you can see a more detailed view of each experiment. And inside here are the graphite graphs of what's happening. So those totals that we saw earlier and how many wrong is in there, as well as on the bottom there's the performance data. So you can see how is the new code that I'm testing out performing versus the old code. So you can see in this case, we're running a lot of experiments like 20,000 per minute and there aren't very many mismatches, but they're not zero. So if you see this graph in the middle that has some mismatches, then you wanna go and look at those mismatches and see what is the difference and what's going wrong. So in order to do that, like I showed earlier, we just throw these mismatches into Redis. So you could go take a look at Redis and for the particular key that we've named it, see how many mismatches are there. So for this poolable by, we have 3,000 mismatches. And you can just grab the first result by popping it off and saying, okay, show me the top mismatch. And then there's a bunch of data in there, the context that we push on, but then you can see the most important part is, tells you what the candidate did and what the control did. So it gives you the timing information. It tells you whether an exception was raised and then it gives you the return value. So for this method, the poolable by method is just a simple true or false value. So the candidate, which is the experimental refactored code, it returned false. But if you look at the control here, it returned true. So this was the difference. And we can take a look at the user ID because we pushed that onto the context and the repository ID and tried to investigate why for this particular case did it go wrong. And the results that we see from that, especially at the beginning, it was just bugs and abilities. We didn't get it totally right the first time. We didn't account for all of the cases. We didn't realize that the old system did this particular thing. So we just needed to fix that. In these cases, we would generally fix the bug, completely blow away the abilities database and re-run the migrators to regenerate all of the code in there. We thought once we fixed those bugs that everything would be good and we'd be ready to ship abilities, but then we started running into other issues. And that was with the data in the database. So if we look here, can show you a sampling of the data quality issues that we saw as we were trying to do this ability of stuff. So what we found in investigating some of these mismatches is that there were issues in the legacy permissions tables that were actually being masked by conditions in the code. So there was bad data in the database. We were pulling it out, but then something in the code was filtering it out and saying, oh, forget that. The database says that this is poolable by this user, but under this condition, we'll just forget about that and we'll say, no, it's not. And so we decided that if we wanted to be generating the abilities data from the legacy data, we needed quality data, it needed to be correct. So this is the case where we needed to go back and actually fix the old data. And you can see how often this happened. We had, this is just a sampling of a few data quality issues. We had quite a few of these and we ended up adding a lot of process around doing migrations or what we call data transitions to fix data like this. We ended up running them quite a bit. We actually ended up writing some throttling code to do these sort of migrations so that when we were deleting millions of rows that we weren't hammering our database during this time. So then once we fixed the data quality issues or we thought we were gonna be done except there was one more thing that we found that we didn't expect coming into this. There were still some mismatches and it turned out that the reason things were mismatching is that the legacy system handled things inconsistently. There is literally no way to duplicate the functionality in the new system because the old system was order dependent or time dependent based on when things happened and when the code changed there were different things in the database. And a lot of these centered around problems with forks and problems with forks that had different visibility so public repos with private forks and things like that. And so we actually had to stop working on abilities, take a break here and fix the problem in the code and come up with a consistent system. This actually took a couple months because we had to fix the code, we had to fix the data. Because these were privacy issues and it was really sensitive stuff we actually had to spend time contacting users and emailing them, warning them about the changes that we were gonna make. That was something that we did not expect. I think if we hadn't had something like science to test out this behavior we wouldn't have known it had happened and then we would have rolled out the new system and people would have started sending requests like what's happening, what's going on, things have changed. So we were really lucky to have found that and definitely thanks to these tools. The next thing we started to find was performance problems. I had showed you the graph earlier at the bottom that showed the difference in performance between the old code path and the new code path. And for these little things we optimized things at the beginning and there wasn't a big difference of performance there. The places we actually started to see performance problems were because we had had this dark shift for a very long period of time we started seeing some pathological cases. People were doing interesting stuff with our API. They were like removing huge amounts of permissions and re-adding them and there was one customer who was basically doing this every night. We saw something like this. So every day at 5 p.m. a certain customer would drop all of their permissions and re-add them and do this over and over again. And we would see this spike in replication lag in our database, which is not good for us. But because we had this kind of dark shift we were able to see this and that gave us an opportunity to optimize abilities even more. So of course in order to fix this we wanted to use our favorite tool, science. And in this case we actually did some science within science. So we had, we already had using abilities itself, science, but then we wanted to test two different code paths within abilities. We wanted a way to do some inference of some things and to not explode things out into the database quite as much. So we tested that out. And we tried it out, it worked. And when we shipped it and we stopped having these problems. So that was a really great way to find that out. So let's take a step back and look at what at this point in the story what have we really done so far? Well we've done a rewrite and a refactor. We've done some science. We've done some data quality repair. We've done some fixes for performance. All right, what's our progress? None of these things are using abilities. We haven't shipped anything yet. But that's okay, once we got to this point we were able to pretty quickly start shipping. So once we were able to fix the last bit of data quality issues, we could start rolling this out. So we rolled it out for organizations first and then for teams. So check, check, we've got two things. We've got a little bit of progress here. I had to do a lot of work upfront to get that but once we got to this point we were able to get things out and using abilities much more quickly. At this point the only thing left is repositories but repositories are the largest and riskiest piece of this. It's the most used part. It's the most complicated and we've been slowly chipping away at the bugs and the data quality issues surrounding it and we're almost to the finish line. In fact, we get to a point where within a week we've only seen one kind of mismatch around a particular data quality issue. And so we said, all right, this is gonna be the last thing we have to fix and then we can ship it. So I code up the pull requests to fix this. The problem was that when users were being deleted they weren't being removed from the teams. So the old record was just sitting around. So I said, all right, let's change the codes. These are always removed from teams upon deletion and then we'll clean up the old data with some of our transitions. So I wrote up this transition. I meant to clean up all the deleted users from teams. Instead I wrote a bad query and I accidentally removed every single repository from every team on GitHub. Oops. Yeah, you can see the outage there. But we were right at the end. Unfortunately science couldn't save me from my own mistakes but we were just about ready to ship, things were pretty much green. And also we have database backups. It's not, not everything is lost but we couldn't instantly restore the backups. So what we decided to do was to turn abilities on at this point to restore access to people, to use the new system a little sooner than I had expected to use it but it was an emergency test run. And we turned it on, we were able to restore access probably an hour or two quicker than we would have before restoring the backups. Once we got the backups restored, I reverted that and turned it back off. Given my previous mistakes I wasn't quite confident enough to really push that into production yet. But this is where we were. Everything was green. It was about time to really ship this. But I wasn't totally confident. I wanted to make sure one more time that everything was okay, that nothing was gonna go wrong when we rolled this out. So I came up with the idea to basically run through every user on GitHub and have a transition that just called, called to the associated repository. So I said for each user give me a list of all the repositories they have access to. And this is a way to basically exercise every single permission in the database. And I figured if we do this, we should be able to tease out any of the last mismatches that we'll find. So I ran this and there were actually no mismatches at all. Well you can see there are three mismatches there, but we had a, there was a particular issue with timing on one of our jobs when you removed a member from a team. It was a common thing that we had seen throughout the time. But there were no mismatches so it was ready to go. It was time to give it to the world and say you can have abilities. So what we did actually instead of just removing all the science code is that we flipped the use block and the try block to begin with so that we could continue having the science around it just in case any mismatches came up. We could still look at the performance so you can see where those graphs kind of flip the candidate and control changing positions basically. And we left that on for a few days and then once we were totally confident that there were no more mismatches and nothing was going wrong, we removed the science completely. So at this point everything's shipped. We've got abilities backing everything. And this was almost a year ago so this has been in production for a year. We've had no problems with it. We've even had people starting to build new things on top of abilities because we made something that was general and flexible. So that was also successful. So we have the code, it's open sourced on GitHub. It's called Scientist. There's a Ruby gem, you can use it as well called Scientist. So I encourage you to go out and use this for your refactorings, rewritings or anything where you really wanna gain confidence. Thank you. Hi, I'm back. Do we have any questions for Jesse? And while you're thinking of a question, I just wanted to point out that a little known fact about her, she has two cats and they're both named after a Disney character. She's a huge fan of the Lion King apparently because one of them is named Nala and the other one Eva after the girl in Wally. Okay, we have a question up on the left. You mentioned that Scientist, one of the caveats was that you can't really test things that write to data, it's only read out. Right. How does science or does it handle model changes? If you're using, say, the same tables, the same model names, but you're slightly adjusting the columns in those models or the fields in those models, can science handle that? And then secondly, the problem that's coming to mind for me is we're attempting to do migration from Rails three to Rails four right now. And this would be incredibly useful for me to say, as I'm moving these things over, let me test it as I go along to see the cases that I've not yet covered or that I still need to work on is science applicable in those situations? So for the first question with the model changes, I think if you were just accessing the different columns, you could probably use it to test different columns. Otherwise, the way we did it was two completely separate models just because if either one are modifying things, you don't want the overlap there, you really can't test the separateness of that. So I'm not sure, I might need a little bit more detail to understand the case, but you may not be able to use it for that. For the second part of the question, doing Rails upgrades, I've never considered using it for that.   So this sort of seems like a kind of a decent way for me to say, okay, I'm now in the process of migrating, let me turn on science to, I know everything's going to go wrong with my Rails 4 implementation at the moment because it doesn't exist. Let me start seeing what these results should be and as I build up my Rails 4 implementation, let me ensure that they're matching what my Rails 3 implementation does, as opposed to just, so something at a much larger scale and not as granular, which kind of seems like the opposite of what science was intended to do, but maybe as fits as well. Yeah, I guess you'd have to hook it in somewhere way far down, which right now it's, it kind of goes inside of your Rails application. I don't, the way the code works, I'm not sure how you would basically have to load up two different Rails applications and have it outside of both of those and running both, which it definitely wasn't designed to do at the moment, you might, you could test out something like that. Thank you. Are there any other questions? Yes, great. So by the time you ship abilities, I assume that you have written a lot of experiments and when the old system goes away, I imagine that there will still be some, potentially some usage of, from all the experiments that has been written at some sort of acceptance test. Do you see something like this or did the already written experiments go away after shipping? So the experiments do go away after shipping. We try to, if there are cases where there aren't enough tests around it, we can use science to compare the results and add more tests in our test suite, but it's not something that you wanna continue running in production forever because you're always basically doing double the work every time you're calling something. So we do it for, as much as we need to, to gather the data and have confidence, but then the experiments tend to go away as soon as you're finished. That's okay, thank you so much.


Hello, I'm Jane Cummins and I'm the intern at the Arlington Department of Public Works. Today I'm here with Shannon Linlau, who is the Waste and Diversion Compliance Officer. We're here at Spy Pond to talk about Shannon and her job, as well as some mistakes that you might be making with your trash and recycling. So to get started, Shannon, what are some of your most important jobs as the Waste and Diversion Compliance Officer? So, the main aspect of my job is being literally on the curb in the mornings before the recycling truck comes through, checking people's curbside bins to make sure that they're recycling properly. And then if they aren't, I would give them a sticker and then that results in their recycling just not being picked up that week. Okay, and what are some of the most common items that you have to put a sticker on that ensures that the recycling doesn't get taken? Soft plastics like plastic bags or bubble wrap and plastic films.  Those types of cartons and black plastic. So like black plastic takeout containers. Gotta go in the trash. Oh, okay. But just to clarify, it's not like a plastic milk carton, it's just like the papery cartons. Right. Plastic milk, like jugs are good. They're good. Paper cartons bad. Okay. I was curious about other rules and regulations similar to the ones regarding milk cartons. So I asked if there will be any changes to the system in the coming months. And here's what she had to say. Yes, definitely. This November actually, and this is coming from the state level, we will no longer be able to put textiles. So like old clothes, sheets, cartons, whatever, or mattresses in the trash. What made you interested in pursuing this job as the waste and diversion compliance officer? Sure. Well, I was advertised as there being a fair amount of walking. I love to be outside and I love to walk. I didn't mind the early morning hours at all and I've always been passionate about sustainability and like especially reducing plastic pollution. Although Shannon's job is purely beneficial to our community, she still experiences conflicts, most notably when people call in if their trash recycling got missed that day. Shannon also mentioned that she experiences some internal conflicts, like when she has to stick or someone for recycling incorrectly. I feel really guilty when I have to recycle people, when I have to stick or people's recycling. I feel bad about it. So I'd like people to know that we're not trying to get people in trouble. It's more like recycling education right on the curb is how I think about it to make myself feel a little better. As important as recycling is to our community, there are also so many other components involved in sustainability. Shannon talks about the other two R's, reduce and reuse that work in association with recycling and how they work into her work philosophy. My philosophy towards my work is that recycling can only get us so far, right? Yes. And I wish that people would also think about the first two R's, reduce and reuse. Yes. And what are the metrics for measuring success as the waste and diversion compliance officer? It would be really cool to see less contamination in people's recycling bins after time goes on, after I'm doing this for a while maybe, and increased participation at some of our weekend and weekday recycling events. Thank you for joining. Shannon and I here at SpyPond. I know I have learned a lot about what to recycle and what not to recycle, and I hope you have too. Stay tuned for more videos from the DPW similar to this one. Next up, you can hear about changes in textile disposal. Thank you for watching. I'm Jane Commins.


Hello. Hello, everyone. I'm Gyotin. I work at SocialCops. Now, before I start to talk about building for next billion, let me tell you what we do at SocialCops. SocialCops is a data company that exists to derive decisions that can solve problems that matter. For example, Google tells us the quickest route home, but not the safest. I can find a restaurant around me in two minutes, but what about finding a grade school for my sister? Now, before you start to talk about predictive analytics and visualization, the problem here to solve for is there is no data. So we build Collect. Collect is a data collection tool that has been put into hands of our 150 non-profits that leverage their field volunteers to collect data from remote parts of the world. Collect has been used for disaster management during Nepal earthquake, maternal mortality in Uttar Pradesh, and then water and sanitation management in Indonesia and also for village development in Andhra Pradesh, Maharashtra, and Jharkhand. So after writing 46 K lines of code, collecting over 30 million data points, and 50 field visits later, here's what I have learned about building for the next billion. So this is our average user. Who is she? She is Pramila. She is a seventh class pass. She can read, write, and speak only in Marathi. She lives in Raigad in Ruler Maharashtra. Recently I visited Ruler Maharashtra and there is lack of internet access and electricity. This is her first interaction with a smartphone which cost rupees 2000 and 3000. So while building for Pramila, I learned my first lesson, colors for next billion. Whenever a user answers a question, so collect is basically a questionnaire app, so there are different questions on that. So if user answers some question on the collect, so question number turns green, giving a notification to user that you have answered this question. But due to some buggy code, for one of the question type, the question number was not turning green. That was a small bug for a developer. But during a field visit, a field volunteer came and said, Sir, this is not working. So that was a big worry for a field volunteer. Imagine your grandparents, when they use a mobile phone, they will feel that they will break it. But how happy they will feel when they can play music or talk to your video. We're dealing with the same audience type. And also field workers, like interact with phone on the field, when there is a large sunlight. So using contrasting colors actually helps greatly in ensuring that users understand app faster. Now as we talked about icons, let's talk about how to use icons. How many of you know what this icon means? Everyone does. This is simple. But for a field volunteer, this is something new. He doesn't know what this icon means. So using text with your icons, like if you can see there, if you use text with the icons, it actually helps field volunteers or the users who are coming using the phone for the first time, that what does this icon mean? They will actually know that if I press this button or this icon, what will happen? Now as I mentioned, Promila can speak only in Marathi, right? And read only in Marathi. So let's talk about language for the next billion. So as Promila can speak, read only in Marathi. So we included Marathi language in the Collect app. But Promila was using an iBolt app, which does not support Marathi as a default language in the language settings. Which is a problem with I think all the low-cost devices. So what we did was, we included the Change Language button inside the app itself and gave user an option to select a language from a list of supported languages which is present in the Collect app. So if you're building for next billion, language has to be an app feature and not to rely on the phone. So this is a list of languages we had in Collect and we gave a button so that user can select from this list of languages in the app itself instead of going into settings in the phone and changing the language. Because you're not sure which phone user is using, right? So we included it in the app itself, so it doesn't matter the phone he is using, he can directly select a language from the app itself. Now let's move on to our lesson number four, which is building for infrastructure constraints. Before that, let me show you an interesting picture. So this is an image of tablets being charged at night during one of our deployments in Maharashtra. So if you can see, like, there is one extension and then there is one charger connected, one extension is connected to another extension and these, like, during a deployment, like, there are about 350 to 400 tablets that we need to charge. And area in, like, as Collect is deployed in areas like Riger where there is lack of electricity also. And field worker is always on the field during the day time and he is only able to charge only at night. That is also when electricity is present. So we can't consume large amount of battery, right? So best option, like, good practices would be if we are building for next billion or for the people who belong to ruler areas, right? So best option will be not to run background services all the time. Like, background services consume a large amount of battery as we all know, right? And then not to track the location. Like, it's better to track the location only when we need it. Not to track location every time. That is another main cause for battery drain. Another thing we can do is we can use battery historian which is present in Android Studio which is a great tool to track battery consumption by the app. So as we talked about battery, let's move on to network which takes us to our most important lesson building for offline world. So as collect is being deployed in areas like Riger, right? So in that demographic area, the internet is very slow or not at all present. The average speed there is about 10 kbps or 20 kbps. Not like 5, 10 mbps where we test over app and generally we say, yeah, we did this thing, yeah, it's working. So yeah. So collect experience has to work offline. We can't say that like we are not able to download your data because you are on slow internet because the main audience we are dealing with is in ruler area. So we can't complain that you have a slow internet or something. It has to work offline. So what we did was to give offline support either we can follow anything like we can go to with SQLite which is a traditional way of doing giving offline support or we can use Realm. Realm is a mobile-based database and it's very simple and way faster than ORM-based SQLite available currently in the market. Another thing is creating a seamless experience. As collect is being used by people who are using a low-cost device which is worth rupees 2000 and 3000 and they have a low memory and certain issues. So it's very difficult to provide them a seamless experience. No matter how many null pointer and out-of-memory exceptions we add but it's very difficult to give them a seamless experience and nobody want to see this message on a screen like your app has crashed, right? So like in collect we have one, like generally in a questionnaire they have 150 questions and if you answer, suppose I answer 50 questions and then app crash or phone goes off. That will be frustrating, right? Like I will feel like let me uninstall this app, right? But we gave an autosave feature so whenever a user answers a response it will automatically get saved. No matter whether app goes off or app crashes the data will be saved always. Like he can come back to the app and see the data. Ultimately, like we are building for people who are using phone for the first time and also they don't know how to report bugs. They can come onto Google Play Services and say that like give a feedback that this is the bug or this is the problem I am facing. So what I have learned in the last one year or something the best way is to go on field, ask your users see how they are interacting with the app, how they are observed their behavior with your app, talk to them, take feedback. I feel this is the best way we can build for the next billion. This is an image of a field training during Vijaywara deployment. So there were people who were using paper for I think their whole lifetime and they are taking surveys on paper. But and this is the first time they were using a tablet. So at the start of the session we need to train them how to swipe, how to turn on and turn off GPS. And also at the end of the training we generally ask them a question. How you feel about whether you want to use a paper again or you want to use mobile. And this is the number of people who said we will use mobile. And this gives you a satisfaction that if you see people who are using phone for the first time and they are using mobile app for the like and the app they are using is the UR app.  This is all about I have to say about next billion. Thank you guys. Any questions you want? Sorry. We are using that. And plus the like file storage as a backup. So we have two methods basically because most of the users are like from rural area. They can log out. They can log out at any point of time. So we need to keep a backup. So tomorrow if they log out by mistake or something. So we have a backup of that like they have. Because if they log in again we will have a backup. No auto saving is basically like we saved draft. So whenever some user answers the question we will basically trigger a method of like saving it into the XAML set. So there is an object basically answer object which will automatically get saved. So basically it's like it's not still production. That is like they are releasing updates every week. When I started it was 0.60. 0.60. Now it's 0.86. I think there is a new version. So they are also removing but there are some thread issues also. Like you can't use an object from one to another thread and there are some certain other issues also. Like if you want to write you can't write an object at the same time. Like you need to make a synchronous module to write objects. Then if those are the issues then still why you are using SKR? Because realm is provides you an object oriented support also. Like it's a good structure then you can define keys also. Like if you can write a good quality code like you can synchronize all the calls then it's like you can go with realm also. Like it will take a huge basically you can say that use time at the first but once you are comfortable with it it's like very good database. Have you used data migration as well? Yeah. Migration on film? Yeah, migration is there in realm. So they are building that. So what kind of feature is there? Is it good? So migration is basically something they are developing it. There is no like they haven't documented it properly but if you like debug it or try to build it sit for two to three days you will be able to figure out like how to do it properly. Have you compared it to any of the ORM existing like Greendow or something? No, I have tried I think active Android. Okay. Active Android I think which is a SQLite way but I like realm more. Okay. Any more questions? We have about five minutes more. So as you are having that string option in the app itself like not from the settings, right? The language. Language is in the app itself so if you want to select some language like I want to see my app in Marathi so I can just go into the settings of the app and then change from there. So have you abstracted out to something because like if you are doing it from the Android perspective it manages it all. But the problem we faced is like in low-cost devices there is no language options. Some languages are not like I want to see Marathi, right? Yeah, that's fine. So my question is like how you abstract it out like have you inflated the layouts again or like? No, so there is something in Android which is localization. Yeah. So you can use localization and call that function and you can change the language. So that is basically per session base. Okay. So it won't change permanently for all the apps on the phone it will just change for that app. Thank you.


Hi everyone, I'm Shintaro Mizuno from Entity Software Innovation Center, one of the R&D division in Entity Group. And my name is Toshikazu Ichikawa, also working for Entity Software Innovation Center. And thanks for this opportunity to introduce ourselves and some activities around OpenStack in Entity Group. So Entity R&D has a mission to evaluate and design OpenStack to be used in various Entity Group companies, including Entity Communications, Entity East, West, and Entity Docomo, and other Entity Business companies as well. So there are various implementation and use cases in Entity Group. So Toshikazu, can you explain some of the major use cases in Entity Group that you can explain to the audience? Okay. Shintaro, we have mainly three deployment in the production today. First one is Entity Resonant, which is our web portal company providing web service called GU. Second one is Entity Docomo email system providing email service. We are using OpenStack as a backend of it. And the third thing is Entity Communications deployment for the public cloud services. Thanks. We also have our own internal R&D cloud for our researchers to use so that we ourselves are the user and also the developer as well. So Entity Group is a big user of OpenStack and thanks for the foundation to award us as a super user. Yesterday was a very fantastic moment for us. But at the same time, we're also a big contributor to the OpenStack community as well. So Toshikazu, can you give us some numbers that we have been contributing around four years since its launch in bear release? Okay.  They are portal sites that generate more than one billion page views per month. As a backend, we are using OpenStack, including Swift, Nova, Granz, Keystone Horizon, and Neutral. We deployed more than 400 hypervisors and more than 1,800 virtual servers are already running in the production. More than one year in a stable and more than 80 service is on top of the OpenStack. Okay, great. So behind those use cases, we have more engineers contributing to the community. So can you give us their efforts in the community activity? Okay. So we have a lot of developers, including Entity R&D and even the other operating company. We have been ever committed to the OpenStack community with 64 developers, engineers, have ever contributed. It's great.  We are putting more efforts to the community. We are pleased to use OpenStack in many different more use cases, and we have more requirements that we need to upstream to the community. So do you have any specific areas that we want to contribute to the community for probably the next few cycles? Okay. I'm going to pick up two large topics. One is return request ID log mapping. It's a kind of request proposal from an operator's viewpoint. Today it's hard to analyze OpenStack logging for the troubleshooting, especially when one request goes through multiple components. It's hard to track one customer's request. Then we are proposing logging the request ID in connection with a single customer's request. That is already approved by the OpenStack close project spec, and we are now working on the implement in Oslo library and major components. Also, we are working on the networking perspective. We are deeply contributing to the neutron project. We are proposing features such as availability zone support from Neutron and even distributed virtual router with a Linux bridge. We want to enforce much capability to the Neutron project. Thanks. NTT Group has been a big contributor to OpenStack community and as well as the big user of OpenStack and a big fan of OpenStack as well. We will keep doing our contributing and we would love to share our experiences. We have various sessions in this summit. If you missed it, please check the website for the video. We have around more than 10 sessions from NTT Group to share our experiences. So please check on the video and please also join the session if you are here in Tokyo. We are expecting your feedback and we thank you for this great opportunity to be on the screen of this video. We hope to be in touch with all the community members and be a part of this big wonderful community. Thank you. We are going to keep contributing. Thank you very much for this great opportunity. Bye.