Before before we move to the lightning talks We have a special announcement to make so I would like to Announce Naomi Cedar. She's a director of the PSF and she has a special announcement to make so please Welcome Naomi It's like the tech could you put the standard on please? Well get rid we don't have any Laptop here so okay stage zero. Okay. Thank you. I yeah, I have a brief moment of PSF business to take care of And to that end if I could be joined on the stage by Mark Andre Lemberg, please You may know that the Python software foundation every quarter awards to people The community service award for their service to the community You may also know that much much more rarely We award something that we call the distinguished service award for truly exceptional service to the Python community And if I were to tell you all of the things that Mark Andre Lemberg has done for the Python community Well, I've already run over one talk slot today. I would be well over the second one. He has been core developer He's been a founding member of the PSF PSF board member. He has been almost everything in Europe Python So he's done a lot and in fact the problem is we couldn't recognize him because he was always on the committees in the groups that We're recognizing other people But we have at last found our chance And we actually made this decision a number of months ago But we felt that the only way the only place to give this award would be here At Europe Python and with Alexander's help Maybe we have surprised mark and if we have that is a rare thing indeed So without further ado On behalf of the Python software foundation, it gives me great pleasure To award the distinguished service award in recognition for decades of service core developer Python software foundation founding member PSF board member Europe Python organizer board member chair with great appreciation mark Andre Lemberg. So Thank you all, thank you. It's been a pleasure. Thanks. You really surprised me indeed Okay, so we've had some practice at applause awesome Welcome to the lightning talks. So day two of the lightning talks. I was worried yesterday We only had nine submissions, which meant that we ran out of Talks in the time that we had so yeah, we've got 21 today. So you you really took the a challenge to heart. This is awesome Can we have? Pavlo, he has a talk about 18 conjuctor here and can we have Jill with make your buck talk up to the front of the stage somewhere. It's awesome There's already had a good second day at the conference Yeah, he's using good talks We've got amazing weather. It's not normally like this the average rainfall in Edinburgh is a hundred and ninety one days of rain per Year, so that's every other day and that doesn't stop in July normally Okay, all right, take it away on Windows yes Hello, hi, so let me start the timer. Here we go hi, so a Secretness programming was a quite a hot topic during this conference. So I want to share a couple of recipes with you So I'm gonna be showing some code. There will be a live demo. So if you want to follow that's the bitly links of bitly Async hyphen Jupiter Let's get rid of the help. Here we go. So two things that I want to share with you is if you upgrading your Jupiter notebook environment and you notice like tornado got Upgraded to five point zero. It may break your Async code in your notebook. Don't be surprised and second thing. I will show how to run I pie widgets asynchronously Instead of synchronously. So actually I'll start with live demo without further do Can you see the code I guess so yeah, okay, so I have two notebooks deliberately side-by-side on the Right-hand side. We have synchronous I pie widgets on the left-hand side. We have asynchronous I pie widgets by the way I pie widgets If you're using Jupiter I pie widgets are great if you don't know about them. Have a look at them use them So let me see what I mean between so let's go ahead and run both this one, all right so Widgets require handlers so in interactive stuff. So you change something on the on the screen You need a handler if your handler is for whatever reason for slow like I have it here Time dot sleep so we sleep for only point three of a second. So it's a slow function But let's see what happens if I try start dragging though that slide and you see the top do you notice the top so the top number represents the argument and It updates on the same thread. So it becomes. Oh, yeah, they probably yeah, it's become slow Let's see what happens on the asynchronous side the bottom number is The process number so it updates with a delay but the top number there is no delay So we're not blocking the main thread and the difference between the handler. Remember, you can always write asynchronous code by just So here's the bit of code That does that. Okay, and the way we wrap it. We attach This curtain using To an existing loop, okay, so that's first part done Yeah, done that now. Yes, so the bad news Tornado 5.0 may Stop your asynchronous async.io specifically async code from working because they started wrapping async.io loop So you cannot stop. So when you start it like the loop is already running. I Can actually demo it. So here's the link. So it was actually in the release notes So here's a snippet that shows you yeah, you get an exception at the bottom of the screen It says the event loop is already running so we can actually go ahead and We can actually go ahead and check that the status of the loop and it says Yep, the event loop is running indeed. So we can close it now. How do we solve it? So we can create a task instead instead of Starting a attempting to start a new loop. So also live demo So there was a delay behind edit point point seven of a second delay Here we go. So we The way to solve it is just create a task and add a callback to to your future Okay, and I've got another way so inspired by one of the talks during this conference trio, all right, so Yes Trio makes it even easier more intuitive to basically to do it asynchronously. Yeah And it works in a notebook. All right Also, you noticed actually I saw only one presentation that uses this plug-in for the notebooks where you turn a notebook into a nice presentation and So here's the link if you are wondering how this presentation is made And the link to the slides also there if you want to take a screenshot and thank you Awesome Take it away Jill. It's okay. I have a plan B so How to make your bot blabber? Just change the talk name. This is not about AI.  This was the code So here you go I Just threw a die and the result was so the thing is that Twitter only allows you to send the same Honestly, don't know but if you send the same message to Twitter, it will it will block You won't let you send the same one. So After a few messages, you wouldn't be able to send anything. So this bot was a bit useless now the thing is I found a new thing called tracery by by Kate Compton So galaxy no not galaxy tracery is a super simple tool and Language to generate text. So this is basically the thing Imagine a text, right? So you have an adjective a noun something else So you can do that with format strings and variables and lists The thing is that Kate found is that she was doing the same thing all over over again. So She made tracery and I'll show you the grammar Later, but this is not about JavaScript and that's made in JavaScript. So I found this other project by Allison Which is called pie tracery which translates by tracery into Python now. Sorry about sorry about this Let's see some code So this is the grammar and yeah, no, this is I'll show you something better if we Wow Just arch Linux things That's okay. Let's make it work right so So as you can see some things change it change here So as you can see we have an action and A noun so we can just we just need to change that and I'll show you the code the code will be more complicated than what any of you Would write But here's the here's the thing. So we have the phrase which is I action animal, right? And then we have a list of actions and a list of animals What it does is the the grammar will pick one randomly Now there's some magic here as you can see The animals are singular and if you add the grammar dot s you see that animal with the dot s over there That makes it plural Yeah, I was going to say yes I see monkeys dogs pigs and I only had monkey and dogs So to finish this one thanks human beings also this talk was made with that Can I do I have time to show the code? Yeah? Yeah? Okay? You talk I have another one The one that breaks. Oh What yeah, this also breaks what's he called? You talk so, yeah So basically these are the options I could have said Cheers lads or choose cheers chaps or cheers human beings Yeah, that's it. Thank you. Awesome And so to include if you'd like to take it away with Python package scaffolding Yeah, hello everyone. This is my first the Euro Python lighting talk So I wanted to talk about make a Python package in one minute because I find in that There's not such a case in Python community to make automatically Make a package and for that I wanted to you if you have a laptop left up already so you can do now peep install Yehua and then we continue so what I will show you is a live demo and in the end in one minute you have your own Package so the objective is that you do Make your publisher scripts your from your heart is to pie pie and after talk. It's a very easy See one minute So what I'm going to do is a quick the Python package in one minute and then she added on the github and the publisher to pie pie That's the objective And I will talk While this the script is running. I will talk why I think is quite useful in this regard Hopefully it's already I installed my utility. So So what you need to do is that project name let's say put your Python to 2018 so showcase Python a package Scaffolding This is most time-consuming in doing this one minute. So And then for GDPR, I'm not showing my emails so your email at your Yeah, and usually in normal case I used to put your email just here so that users can contact you and For this mobile, this is a organization. I put it as this is a real organization again This is real one and and put it on and let's see Like this and now I'm going to make a command line interface So I press two and then command line name. Let's say you look at on 18.  I got there. Sorry a bit. I should choose to Okay, sorry I Still got time quick. I can still at home. Okay unlicensed. Let's see Okay contact email for now to save time and then and Then this is the choice should come on the line interface and then now is EP 18 Okay, now it's doing the job so I can you can each other talk So why why this talk? So you got a cool Python suite in the hard disk? Why don't you share it? Why so you share it on pi by I can everybody can use so but then it says I need a set up the pie Independently so Kenneth reads had a set up by a set of pie project Which is a template you can copy and paste. However, you need to copy and paste by yourself So it got 2800 stars at the moment and in addition you probably want to supply set up the CFG so so as that you can build a universal views and Can you put on the github as well so that I can contribute? Then you need to read me. You need a license file and For you to contribute or for me to contribute. Maybe you put a Travis file so you have a test running and for that Psychologically it takes more time than just it takes right this is so many fires for each project I need to do that and Then last question is do you want to repeat yourself every time you got a script? You want to share to the pipe I and a github you do want to repeat that so now So hopefully that has done the job is here So the the script says everything's up today, please review the changes before commit so everything's ready So what's what's needed for you to do? So what we needed to do at least do something so you look I done with you look I don't we do main and say deaf main Let's say Hello World world, sorry So that will be end of supposing it's your main file. So let's say we wanted to install That's it and then okay. Thank you. Jen Fu. I'm afraid that's your time up So Marcus is gonna talk to us about bike on a you in one minute one minute, okay So at we had a yesterday a talk that verse from the Antarctica You don't need to fly as far as there and you can fly actually fly then next month in end of August August 24th to August 28 Pike on Australia is gonna happen There's four special districts on the Friday one is about security and privacy one is general connu One is about education one about internet of things and the general conference has about 50 talks for invited speakers and It's generally a pretty amazing conference And there are also sprints on Monday and Tuesday and yeah ticket sales open. There's still tickets available They're still last time I checked flights available to get down there Yeah, thank you Look ma no slides old style like me started my program professional program career as program in 1984 35 34 years ago Start learning in 80 previous century And at that time some people told me, okay, you are doing a developer cool last a couple of years and after you will move to something else sales Clients whatever Lots of people don't that Personally, I think that that people don't like program go into program as a step to go somewhere else. I love program But I Still in buisness After all these years as the Sun say because I have a problem I Got bored easily After a couple of days or at maximum a couple of weeks I get bored of what I'm doing. I have to change so IT is the ideal field with their fast moving change To have someone that get bored easily and here I am never bought what leads to a Small detail. I have used the word problem on purpose because I hate the word problem the word problem Triggers a problem in mind if you are kind of neuro linguistics You will see that some words trigger some states of mind. We don't have problems. We have attributes. We have characteristics and May look like a parenthesis, but it's not I Think the most powerful Emotion for humans are gratitude some people say is that most powerful emotion is love But perhaps love it's just a state of profane gratitude to our partner and I would like Manifest my gratitude to the speaker of yesterday to X singleton that talked about autism being a program with autism because bring us a huge Example as we can put our attributes our characteristics at our service instead of working against us and so for that of Course it needs a Major part needs that I know myself very well so self-understanding is a Keep part so I can know What I am and put it at my service Time mark you have two minutes left. Thank you So I would not say to you to go vegan Well, first of all because I hate someone to tell me what I have to do But I can tell you that I haven't eaten meat for more than 10 years eating fish for a couple of years and recently go going entirely vegan And it raises a lot of questions, but one thing I can tell you I Never felt bad better in my life. I never felt with more energy. I never felt In a better mood even for for programming. I never felt My brain my mind working less cozy and Yesterday I have the honor To meet someone that I have never met before as the person of the kind of person someone that is Vegan from birth because have the lucky enough to have Vegan fathers that rise of their child as fully vegan and That out to rise me to refer to himself. That is our colleague. Hanyeri. So, hello, please if someone Wants to know more about their experience if someone he's Preparing to rise a child or have friends preparing to raise a child Perhaps it's interesting to think about what kind of substances we are going to put inside the body of our newborn baby Thank you very much. Thank you. Wow. I'm ready. Awesome. Take it right here. Hi. I'm Kio Smallwoods I'm a recovering cis admin I And recovering pearl programmer I work for a company called Flexitricity I'm going to talk about seabor which is a binary serialization format It was inspired by message pack and by inspired. I mean, I don't like the way you're doing that So we're going to do something different It's an ITF proposed standard 7049 is the RFC It uses a tag length value encoding, which is very common in networking It only uses three bits for the first part of the tag and then the rest of the tag system is extensible It's designed to be very compact to write encoders and decoders for constrained devices It supports recursive data structures so if you have a list that references itself or whatever no problem and The tag system has an IANA registry So tag 258 has been registered and it is a Python set or other languages that support sets as well The there is a pure Python implementation and by Alex Gronholm who I hope that I pronounced that right He's going to be giving a talk tomorrow About testing in 1120 so check him out It works in Python 2.7 to 3.7 probably He's just added support preliminary support for that It's really clear readable code. So it was easy for me to get started It's got a great test suite. It captured all of my mistakes very quickly and it's the only Python library that supports canonicalization which is The same data going in will always create the same binary at the end of it So your maps are sorted if you choose to canonicalize You can use it when you need richer types then JSON or message pack and when you don't want to write a schema like for protocol buffers or captain proto or other things like that and You definitely want it to be more compact than XML. I think everything is more compact And when you need to put it on a constrained device again What I use it for At Flex Tristy, we have a lot of time series data for frequency measurements and power and It's really good for taking a CSV file and putting it into a much smaller more compact format Although Apache Parquet is probably better supported for this use and faster It's also really good for simple passing between processes Particularly when you want to serialize date times Because with JSON you have to write a JSON encoder class anyway Messaging for constrained nodes it it really does take up that the C encoder takes up a tiny amount of space You can also use it for dissecting network protocols as a few new protocols that the IETF are working on that are based on seabor The DNS grasp discovery protocol There's also phyto to authorization over Bluetooth that uses seabor as its encoding And Sierra wireless uses it for a couple of their device metrics and things And In the future Maybe we could get in micro Python. I want to try doing an implementation in NIM because that looks cool And yeah, and that's it. Thank you, Keo big round of applause, please Take it away. Okay ready So I want to start my talk with a question to the audience. Please raise your hands Who do you know about? How many knows? Oh Maybe 10% of the orders more than I expected so This is the second question and Actually, maybe you don't know that every time you fire high python. You are important the correct module and it's not only that several Framers web frames no web frameworks are important it is one of the most downloaded packages has been there for 13 13 years since the 2005 and The start actually started as a hack. I thought Well, this will leave one year may be this project because the next release of Python They probably will fix this problem that this module the decorator modules meant to solve the signature problem of the decorators that Essentially, I want to preserve the signature or a decorated function. I thought this would they will fix in Python Maybe not because After 13 years the problem still in Python 3.7 Now there is the signature object. So the the problem is Was he then under the carpet by still there? Anyway, this to show you that the project is still alive. I make new releases every year This year I've done already three releases last year for releases. Okay, the beginning 10 releases the first year then Here there was a gap doing the Python 2.7 Python 3 transition Now it is the working it works for Python 2.6 to 7 3 1 3 2 3 7 every version of Python. Okay It's the joy to maintain. I have very very little problem Most people don't know about it so they don't ask questions, which is fine And essentially I do something major only when there is a new Python release Like for instance, I don't know you have a sync functions in Python 3.5 Then I want to decorate a same function. That's a good reason to do something new and just to To make that my point. What's the problem that I'm trying to solve? This is the recommended way to write decorators in Python if you look at the Python documentation as you see is kind of ugly because you have These func tools wraps to call you have a nested function, but it's not particularly Difficult to understand but the problem that is not does not work What does not work is actually you see this wrapper function the signatories star arcs star stars keyword arts So actually the signature of the decorator function is this one Okay, and you're to the documentation to tell you the truth that signature is this one While the original function of the different signature was X and Y Okay, this is a problem and the decorator model is able to solve this problem Also makes the syntax a bit nicer because you don't need that to have a nested the closure inside and If you look with any documentation tool You will have the right senior to refuse I Python question mark. You will get the right senior the right dog strings everything As I was saying I had this user request to decorate Async functions and you can do it and you need this kind of a sink decorator This is a new feature. I don't use personally these are sink stuff because until recently I was Stuck with Python 2.7 and now we're using Python 3 so maybe I will use this but I'm here to say to you if you use this feature and you find bugs Let me know the user to ask for this is happy So I hope is I will work for you, but I'm not and there is a very new Feature that they implemented three months ago and now you can do this kind of decorators Okay, we've a parentheses without and if you there are no parentheses essentially you have a default message Okay, this is and this actually I use in production because I need a decorator like that Okay, you can implement this in a no six or seven line of course with the decorator module try to implement this without It's decree trigger that it seems and that's what everything I wanted to say Check it out Take it away Okay, so My name is Jorge Jardines and I'm a software engineer Working for trust you at the moment and I used to say that I'm a better dozen than a coder And When I arrived to Spain some years ago There was a big bubble of dancing salsa there You actually ask almost anyone in the clubs and everyone used to dance salsa apparently so This is the question that you get more often when you go to a salsa club Do you want to dance and this is actually what you expect to to feel in the dancing floor? But it happened that what you actually receive or feel is that that your movements does not match with with your dancer partner and And Yeah, it's really fun, but not really what you expected so I Found some similarities that with some trendy topics in our industry Related to this So this is what I understand by salsa or by dancing in general Is that you need to keep the communication with your dance partner? You need to make sure that you both are found not only one side You need to listen to the music You need to keep the tempo So I found really that the issue is not In the way you dance or the way I dance the issue is that we don't teach this from the Very basic moments when you are start learning We are so focused on the figures and we don't go to the basis which is listen the music and keep the tempo So if you do less figures and keep more the tempo you maybe have more fun so in our industry we found so much this question and the people are Always saying yes, I am I I What do you mean by I? Well, we do a stand-up meetings We are fast We even do continuous integration and we do retrospective time so I don't know if you see some parallel lines here So salsa is trending a y'all is trending everyone dance also everyone is a y'all I feel the same so I guess that we don't dance the same I I at the end So I understand by Alice that that ability risk management and learning and innovation culture so Still we don't dance the same I Alice we don't do this in my opinion We need to have real continuous integration continuous delivery aesthetic code co-quality Embrace embracing the change the boss culture and continuous improvement culture refactoring That way we will feel like the real picture of dancing as couples Thank you. That's all What a great search in the evening. Thank you. Well, hey The big round of applause a big round of applause for all our speakers this evening


Good one, Diesel. Good boy. Every good day starts with this face. Now look at Bo and Diesel. It almost looked like you were gonna eat it in that shot there. How you doing? Anyways, yes, here at home again today. I ran into town early this morning and oh, I dropped the jelly bean. No, not the jelly bean. Five second rule. Five second rule. My house. Jelly beans off the floor if I want to. No, sorry Diesel, you can't have jelly beans. I think it would make you sick, I don't know. I'm not risking it. Anyways, yeah, so I went in this time back and I had this camera with me over here. Brought this camera with, you know, I was gonna do some vlogging in town and whatnot. I was all excited. Pull it out, try to turn it on. It's got no battery in it. I left the battery in the charger. Is there some reason with that camera? With a, hmm, that was a delicious jelly bean if you were wondering. I just made you all want jelly beans, random. Yeah, with these Sony PowerShot cameras, you have to take the battery out and put it into this thing here to charge it. Just the way it goes, just the way the cookie crumbles on that one. And so I forgot the battery at home, so no vlogging in town today. So we just got home and we're gonna install our, what you call it, cell phone booster. Oh, I can't talk, I can still have a sleep. So this is the external antenna. How'd a burp just sneak up on me there? I didn't see you coming. Whew, all right. In my eyes of water, how in the world, a weird burp. So this is the external antenna. This is what's going on top of my house. It's gonna point at the cell tower nearest to here, which is probably about, I don't know, 25 kilometers away or 15 miles or so. Apparently people around here use them all the time. So that goes on top of the house. You gotta stream that coaxial cord or whatever they call it. I got a hole in the side of the wall in this room over here already. I'm just gonna have to reseal it again once I push that through. All right, and corner down there, that's where the internet came through from the outside. So I'm just gonna push this cord through there too. Hopefully it'll fit. If not, then we'll have some other problems to deal with. Anyways, I gotta get that cord from out there into here. And this is the inside piece. This is gonna give me cell signal inside my house. And it works for up to 7,500 square feet. So I'm hoping it'll go through the walls of the house into the backyard as well. Then I can have cell service out there when I'm hanging out on the lawn with diesel and whenever I have company and stuff that everyone can have cell service here, right? Otherwise, no one's gonna come visit me if I got no cell service. You're living in the Stone Age, trucker. I just don't wanna come see you. I'm trying. Got the internet hooked up anyways, that's good. So we got internet here and now we're gonna get that going and we're gonna climb up on the roof. However, I think it just started raining. Say it ain't so, come on. Come on. Is it raining? Nope, okay, just looked like it. Nevermind, Taylor, let's get her down. She doesn't quite fit. Dang it, she doesn't quite fit. Well yeah, I brought the truck as close as I could here cause you're gonna laugh at me. You know what I'm thinking? I need to get on the roof, right? The ladder, but I have a truck. So I'm gonna climb up on the fender here, climb up on the hood, and then jump up on the roof. I am a genius. Who needs a ladder? Who needs ladders? I'm a genius, just saying, just saying. Worked perfectly. Who needs a ladder? You see, now all I need is my hockey stick with my Canadian flag at the end and I can fulfill Jay's dream for me. Proclaiming from my rooftop, this land is my land. But I don't got my hockey stick up here. I should go get it though. Jay Cannell, you all know him. He's the one who made that comment. I was laughing a while ago. I just thought of that now cause I'm up on top of my roof working on this pole right here, right? I'm gonna attach this thing to that. Figure this thing out here. But yeah. Worked like a charm. This land is my land. Not gonna lie, that was actually quite the accomplishment for me. I am not friends with heights. So, the fact that I went up on my roof by myself hooked that thing up there on that slippery roof. I got her done without a scratch on me. To me, I was like, that's right. Conquered my fear. I wouldn't say I'm afraid of heights. I would say I'm nervous of heights. I still gotta go back up there and hook up the cable. Goodie, let's go conquer our fear again. Not a fear. Well, we've been working all day. Well, I've been working all day.  Slacking off. Look at him strapped. Oh, it's been a hard day, man. It's been a hard day. Oh, you died. Oh, man, it's been a hard day napping. You're an expert napper, man. If you were paid to nap, you would be a professional napper. Anyways, so I've been spending all day working on this cell phone booster. I got it here right now. It's not gonna stay there. Trying to find the best place in the house where it'll work the best. Everything's everywhere, even worse than usual. And pretty much it goes out here, out the door. We gotta find a way to get it into the house. I've been spending all day trying to aim it right. And I got all kinds of apps on my phones and on the internet, got online to figure out where to aim it. And I got it in pretty good right now. But it's just going up on the roof, coming back down and checking. Going up on the roof, coming back down and checking. Going up on the roof, coming. You probably know what comes next. We come back down and checking. So do we even have cell service here now, or what? Tell you what, it's driving me crazy. See, do we have cell service here? Nope. It says that it's not working. See if I can call my house. See if it'll dial. It says I got one bar for donkiness. I got two bars here. Closer I get to this thing, but it should give better signal than that. It won't dial out. Very frustrating. My American phone actually has a lot better service in here than my Canadian phone.  See if this one has signal. Yep, this one's got signal. Just got a text message. See if I can call my house. See if this one works to call the house. Sounds strange. How does one phone have service? But the other phone doesn't, right beside it. It's strange. Anyways, we're trying to figure this out all day. So it's been a very boring day. And I haven't been able to vlog very much. I've just been trying to get this to work. I don't know. I got 14 days to return it if it doesn't work. I might end up just returning it. Just sticking with the landline. I don't know why I need a cell phone booster anyways if it's not gonna work proper. But anyways, we'll check in with you in a bit. Haven't been doing much today. All right, we're slowly cleaning up the house a little bit here. I still got my little gaming area going on right here with my chair way too close to the TV. Sorry, mom. I can always remember as a kid, you're sitting too close.  I'm not cross-eyed yet. I don't think I am anyways. What are we gonna do? We just edited a video from yesterday. I really liked the way this one turned out. It would have been a week ago for you guys already. This is where I was at home when I took the bike off-roading a little bit and there was a train and we moved along. It was a very productive day yesterday. I feel way better about yesterday than I do today. Today I got nothing done. Nothing. I worked the entire day. Well, in the morning, I went to church. I got that done. And then I came back home and I spent the whole rest of the afternoon and even trying to get this cell phone booster to work. And that's right. I couldn't get it to work properly. Sometimes it would work good. Sometimes it wouldn't work at all. Sometimes, boom, in the back seat. In the back seat. I'm bringing it back. I had 14 days to get a full refund. I'm bringing it back in two. Not impressed. Hashtag not impressed. But, oh well. It is the area that I live in. It's not their fault. I don't blame them. But the whole thing, and even if I didn't get it to work, I was thinking to myself, that's a $560 cell phone booster, right? And I thought that it would be great. Yay, I'll have cell phone service. Really, the only difference is that I'd be able to text people. Because I have a landline here. I would have canceled that if I had my cell phones working, though. But all I have to do, really, is just forward my calls from my cell phone to my landline when I'm at home. And then I have a phone. And I don't have to pay $600 for a cell phone booster. The only con is I can't text. Oh well, I guess we're just gonna have to live like a normal person now. Dang. I do have internet here, so I mean I'll still be on the internet. I can text people through the internet so I can still get around that. Yeah, so we're gonna make some food. Make some steak. So these are the security lights that I bought from my backyard. First zenith. It's gonna replace this light back here. Because that light back there only works with the switch. And I don't like that. I want it to turn on whenever there's a bear in my backyard. So I know, hey, there's something in my backyard. Maybe I should check it out. But, and plus it saves energy, right? It only turns on at night then. What are you beeping about over here? He's cranky though. I'm sorry I didn't use you today. Sorry, he's a grumpy little guy. So, what was I talking about? Lights, yeah. Those lights only turn on when there's motion, right? So then it saves me a lot of hydro from having to leave them on all the time. I'd also like to put motion lights on the front. These are the lights that I got for the front. Two of them for on the garage. I already had these from before. I was gonna use them in my house, my old house, but I moved. So I'm gonna replace the ones on the garage with these. They are also motion lights. Or I can set them to just come on at night. Because those are the lights that are on there right now. So those lights again are just on a switch, on or off. That's the only options, on or off. And when I'm not home, there's no one to turn them off.  So they're always on. And just for safety, set at night, there's some light around my house, right? So, instead of, I was thinking about getting a boot. A boot, I just said a boot. That Canadian. I was thinking about getting lower energy bulbs, fluorescent bulbs. I figured, well, they still gotta be on all the time then. So I'm just gonna replace them with motion lights. The ones in the front, they'll turn on as soon as it gets dark, so they're on all night. But the one in the back, it'll just be on motion. So if anything moves anywhere in my yard, you'll be in a spot light pretty quick. So this is the movie I bought yesterday, remember? Did I cut that out? I don't even remember. The box is a Walmart. And it looks pretty interesting. I like these kind of videos that are sort of set in the future. It really shows the producers and directors' imagination of what the future could be like. Those kind of things, so. Except the only problem is, I don't know where I left the actual movie. I can't watch just a case. It gets boring after a while. I mean, she is pretty, but I actually wanna watch the movie. Where's the movie? Diesel, what'd you do with the movie? I wanna watch the movie. Is it in there already in the PlayStation? You have no idea how often I lose things in my house. It's always been this way. It doesn't matter where I live. I will lose everything 10 times every day.  That's everything, 10 times. I can never find anything. I'd lose my head if it wasn't attached to my body. Where are you? It was hiding amongst all of my mess. This is my production area, my YouTube zone. It's got cameras and wires and remote controls for the TV and external hard drives and battery chargers and cameras and accessories and jelly beans and a movie. Movie was hiding under the jelly beans. All right, so we found the movie. And we moved it. But, I bet you ain't gonna put it in there already. It was in the PlayStation the whole time.  All right, we're gonna watch the video now. We're also gonna be making some steak like I've been telling you about and looking forward to to starting to smell good. Steak and fries. You'll notice that I eat a lot of meat. I am a very carnivore pro meat kind of guy. No offense to any vegetarians, vegans, nothing against you guys. But I eat enough meat to make up for all of you, so it's all right.  That's what I always say. I don't mean any offense by that, but it's every time that I meet a vegetarian that I'm comfortable enough to joke around with. It's not that I know I'm just being sarcastic and joking. I always say that joke.  Yeah.  You totally understand why people are vegetarian. There's multiple, multiple, millions of different kinds of reasons and I have respect for that. If you were to come to my house, I wouldn't force you to eat meat. Absolutely not. But you may have to watch me eat meat. Sorry. Where did the day go? The day's just gone. It's after midnight. I'm going to bed. I gotta be up early. Gotta bring my truck in to get the estimates done. I'm a little weird about this. I like my curtains to be even. Fully closed. I need to get different curtains for the front of this house. You can see through these curtains. I feel very uncomfortable when people can see into my house at night. Like you see the windows back there. They have no shades on them or no curtains, nothing. No blinds. Makes me nervous. I don't like how people can see into my house at night. There's nobody back there. Like there's no one could possibly see into my house from my backyard unless if they were in my backyard illegally. But you never know. It's creepy. I can't walk through my house in my boxers and I don't like that. I like having the freedom to be free in my house. My house. Shouldn't have to wear clothes in my house if I don't want to. But that makes me very uncomfortable. Anyways guys, we did nothing today. We had a failure with the cell booster. It was disappointing. And we're gonna return that tomorrow after we bring the truck in to get the estimates done. Hopefully that'll go well. Tomorrow's vlog will be us going into Winnipeg to get that done pretty much. So it won't be just sitting around here all day again. So don't worry. Thanks for putting up with this vlog today, guys. Thanks for putting up with me every day. Thanks for putting up with me again tomorrow. 4 a.m. Central time on the same channel here. If you want to see some more Charter Josh videos, you can go down below to the description. There's more links to other videos of me on the road or whatever I was doing a way of doing a year ago. Go check out the description there. I'll see you tomorrow.


straight ahead of me in the bighorn sheep blocked by bighorns we can't get to them I'm because of all the sheep up there there are 21 bighorn sheep in front of us and we've counted so far some to the left there there's some on top of the waste pile there and there's a bunch right here one big full curl one right over here I hope you can see it in the video and then and then these are right on top of the waste pile and there were some on the right side of the waste pile too so we're blocked by bighorns good morning youtubers good morning Tom and Julie here I think Julie has a portal right in front of her there and that's where we're going to go in so let's go see what's in there how's it look no monsters coming out at you once again we have hand-hewn timbers in here don't we and the lagging and the gobbling on each side got a little buckled timber right there boy look at the size of the rocks on the sides here wow that's pretty wild isn't it I'm not going to hang around under this broken one very long all gobbling mixed in with the timbering the timber sets some huge rocks in here like this I know it's crazy they really went through a fractured area here didn't they with this drift it is kind of crazy looks like it goes dead straight these are some skinny little timbers aren't they I don't see much of a vein of anything it's so fractured how could it have been you know it's not like a standard fault line in contact zone maybe they're going back to the intersect one I can see ahead of you it goes pretty far doesn't it interesting it looks like all mechanically placed rocks no animal sign yeah except for rats of course that's looking down the down the drift as far as I can see still same thing straight as an arrow okay here we are at 300 foot mark that's looking back at the portal this area was traditionally mined for silver since the 1860s I just talked to one of the guys that's with the mining company that is currently holding the claim here a Canadian company which most of these areas are owned by Canadian mine companies there's miss Dorothy anyway you said they're still sampling for silver 160 years later let's say first started mining silver in this area changed here yeah now it kind of changes from this fractured stuff oh listen you can hear the the echo now it's solid here comes some miners graffiti so now they must have intersected some of the stuff that they're interested in what does that say Stanford oh the Stanford geological survey august of 48 and there's some other stuff I can't read we've seen the Stanford survey in some other places they really shouldn't be painting them up but whatever they did so here's the left we might as well continue straight for now it goes back a long ways yet now I don't really see a vein here either but I'm sure we will that whole first 300 feet was just to get back here to intersect some of this other stuff it smells a little sulfury with 500 feet and there was a mark on the wall there was tracking here at one time for the or cars here's another intersection looks like is a cross cut both ways no just just that way and some backfill and gobbling here and then that hits the face does it let's be sure yeah this hits the face right over here yep let's get back to the main line oh here's the head frame you know what that means there's the sign of the old hoist there's a ladder so we're here at this vertical shaft it's got an interesting set of bearing or bushing that would have held the shaft here their metal it almost feels like lead hey maybe it's a babbit so here's the ladder there is a ladder here it doesn't look like it's terribly deep i'm guessing 60-ish or so and it looks like there's possibly a working level down there so let's explore more of the mine and maybe we'll come back and go down the ladder okay there's the shaft let's go continue down the main line finish this one what'd you see a cross cut probably 40 feet to the face well what did they see there that they were falling oh this thing right here yeah looks like the strap from strap rail probably an inch and a half by a three sixteenths flat bar that's all that looks like that mica yeah it does yeah yeah i've been stabbed by a track before like that i'm not looking at it and all of a sudden there's one that's peeled up like that what do you see ahead of you there still this keeps going here's a little different colored stuff and here the echoes is low so this is a softer area here the two by fours are still here for the rail so we've found the lid to a pot some dynamite paper it looks like there's some more mineral mineralization looks like somebody picked at that yeah predominantly silver mines in this area and a little bit of gold here and there according to the guy from the mining company who is still assaying different areas they dig different pits and they do different drillings well here's a lot of strap here's the first curve in this thing we should be able to still see the portal back there yeah this is the first slight curve and we're at a thousand feet i think station 10 plus a hunt 10 plus zero that if you were a surveyor you know that's a thousand feet 10 100 foot stations cross cut to the left at an angle 50 feet or so long we can put on there in a second i think we're nearing the the face of this main drift looks like a slight amount of backfill near the face unless it turns the turn okay oh so this is just a car couldn't see a weird yellow circle there sure it hurts my back to scrunch like that how's it look okay what's getting hot back here we're not got cool again what if there's air moving through here yeah it got hot and then it got cold not cold but cool not as hot all right there's the face narrow's down um i'm gonna see if this fluoresces this is under a UV light i don't know if you can see any of that a few different colored things gotta watch it that's just bad for your eyes there's a bat we won't disturb him he's sleeping i can see him breathing though you notice him breathing yeah his little body was pulsing look at all this draft all right i'm gonna check back when we get back to the shaft we're a cross cut whatever we miss a cross cut here's some advice about mining mining is good if you got the money sucker looks like it's from may either may 20th of 25 though how do you read it somewhere 25 29 so if you got the money mining's a good gig i guess that means they don't make a lot or it takes a lot to make some is this the right that he skipped all right let's go down here looks like there was track so it probably goes a little ways at least out of those short drifts they don't bother putting track in those short cross cuts it's funny they took the ties out of the other main haulage and here they took the strap and the tuba force here's some timbering some natural lagging on the left here i guess and on the right uh mill timbers but natural lagging his tree tree parts that lumber smells good didn't it yeah had a nice tree snow what's that say it looks like July 29 i can't read on anything tom July first 29 july 1st 29 wasn't me i wasn't around until 56 this is following something because it's drifting around these little mines can surprise you how big they are because there's not much of a waste pile outside this thing and it's gotta be i think it was about a thousand feet plus whatever we got here here's 200 on this drift looks like the rail is still intact up here and then we have an intersection just a short cross cut one big timber right here in the middle neat little mine oh here's some real real uh track kind of narrow this rock wall this has real soft material this light stuff whatever it is one of you a geology guys will know there's kind of a seam there every time you come to one of those places you get a lot of it on the ground i see something back here a barrel a little glove there that's a cardboard cardboard box oh yeah there's there's another glove there yeah there's a pair actually a pair set those up on the rock maybe see them a little drier i'm gonna cool it off again a little bit maybe there's an airflow here something's going on cool rabbit there's a rabbit in your live one you have big one poor guy he's right here if you go right on corner this must be the end because they're the end right there oh yeah he's lost in here let's chase him out yeah we can get him out come on buddy you're all blocked this way oh he went yeah he's pretty lost in here yeah come on big guy too i just saw the tail end of him come on we'll get back a little bit here's oh that's a nice barrel and then a muck sheet and then the face so let's see if we get this rabbit to go by us here could get him in this barrel and haul him on we'd have to cover him or something could grab him by the back legs if you catch him he's pretty quick yeah i know he hasn't been in here in a long time he looks pretty old come on you okay big guy come on oh he tried to crawl in a hole come on out of there come on come on you go by me come on come on you don't want to go in there come on go the other way other way other way other way there he goes there he goes go right come on might still be able to see him here you ran right into my leg he might have been a little blinder by the light he should see the fall right yeah if you if you went down this far poor thing was scared up an hour how long was this you're not there yeah he must have gotten turned around he wouldn't be way back in there well lisi's not in the cross cut anymore maybe we'll see him out here i don't see him down that way well that was something kind of excitement i guess i don't know where exactly where he went lisi's not stuck back in the back of that cross cut okay tom is going down an old set of ladders old old set of ladders he's roped in so theoretically he's roped in and there is a large bat down there no more well he looked me but maybe he runs away yeah maybe he's not all right see you down there that's a rope bat rope hook all right yeah sure so here's the bottom of the ladder it's got a little funky spot there and a little switch over there probably usable though but you don't know so much you get down here there's one drift going to the left here and there's another one going behind me there's a little teeny hole a purple one real small one looks like a child's toy probably good for getting in small crevices this area of the mine doesn't get much visitor many visitors i don't think since you gotta climb those sketchy ladders so this coming dead's in dead ends of a shoot let's see where this goes it's very red the material is up in here excuse me it looks like it's gonna run out here yeah i can see the end of it up ahead here let me make sure yeah that's the end of it apparently they like the other way better they weren't finding what they wanted this was about 50 paces all right so tom's making his way back up the ladder they look pretty old what are you getting there you said your bat took off yeah he went out the mine or in that direction anyway just are you just hanging out yeah i'm resting okay there he is wow what a trip what a workout mr bat didn't get me no you just have all got me okay all right good job thank you here's one last look at these beautiful hand hewn timbers you can still see the x marks in there so that's going to conclude our exploration for today's video it was a fun little mine we hopefully got the rabbit going the right direction we never some go out but anyway we appreciate you guys watching the video we hope you enjoyed it and we'll see you next time thanks for watching


And 30 seconds early, but I'm Gabe. I'm giving the talk on Postgres 10 performance in U. So just to give us a quick kick off, who? Slides not working. Who here uses Postgres? Kind of show of hands. Oh, a whole lot of folks, okay. Wasn't sure if we had any MySQL users or anything else, so. So we're here for a lot of the same reasons. We recognize that Postgres has a lot of really great things that are going on for it in the community. Native JSON support, native hash support. There's a lot of great community standards for data integrity and insurance. Continued excellence in a lot of really large and really expensive subqueries. And a lot of numerous native extensions and extensions supported by the community with standards built in natively. So we're all here for the same reasons. But Postgres 10 got even better. As a performant database. So some of the things we're gonna talk about today are native table partitioning, hash index resiliency. A big one for a lot of folks which is full text index searching on JSON and JSON blobs that is native now. And a big one that I'll probably talk about which will be a little more in-depth parallel queries and how they affect large joins indexes and other portions of the code base. And query analyzer accuracy and changes to how the query planner does efficient work as well. So you'll hear me talk about some phrases that you'll probably sound familiar if you're an avid user of Postgres but I'm not gonna jump into in-depth how much they work.  These are also changes that happen in Postgres 10 but I'm not gonna cover them in this talk. But a little bit about me first. I joined Heroku a little over a year and a half ago. I've been primarily working on Sinatra and Ruby on Rails applications for the better part of the last four years. Primarily right now in my role, I'm focused on Postgres and high availability modes and large data growth. So growth in the terabytes of data. I've worked on a lot of different data service engines more than I can list. I'm not gonna go through all of them all there. That's too much time. But on the side, I'm trying to be a lot more positive. So I'm trying to do a lot of self-help and life coaching things. So I'm trying to focus on a lot of inspirational things. So on the side, I do some super secret project work on this social interest platform because I like Facebook, but there's a lot of negativity on there. So some of the things that I'm looking for are some basic things that any social interest platform would provide, which is profiles, but I'm also looking for curated content. So inspirational quotes, TV characters, movies, TV shows, foods, the list goes on. And I'm imagining a lot of usage. So we're gonna just jump right into it and the first thing that I'm gonna talk about is inspirational quotes. And this is one that I actually saw when I was traveling to New Zealand which I thought was fantastic, which is happiness is not a destination, it's a way of life. And I'm hoping to see millions of these populate throughout the code base and through the platform. But wow, that grew fast. So even though quotes aren't very big, they're just small string texts, most databases even at large scale have significant amount of RAM, tons of disk space, but some usage scenarios that most quotes are never updated or revisited after a week. And most people only want the most recent and up-to-date quotes, they don't wanna see stale stuff. So that's kind of tough. That one table grew really quickly, some quick math there. And that's way more memory than I have. So a little bit about how Postgres and some of its caching system are gonna work here, it's gonna attempt to load as much stuff in memory as possible. And then it's gonna try to use what it has in memory. And if there's overflow, it'll start doing disk writing with pointers to the files. And it'll open them up at one at a time as it does file writing. So it actually will manipulate the disks a lot more. So caching is done this way because if you access the data, obviously, you're probably gonna need it sooner than later. So it's gonna try to do that in a couple different ways. But the other problem with this is that temp disk space is not free. I know we'd all like to believe it is, but it's actually not. There are a lot of other overhead costs because usually temp disk space also happens with where the data is stored on the database on disk. And caching records in large quantities can have some significant impacts, especially as you're trying to concurrently access the same table multiple times with large data sets. So one of the real issues, and this is something that some people might have seen in the past is that we can easily stop connecting to the database entirely if we run too many of these simultaneous really heavy bloating or locking queries. So who likes fast queries? Everyone, I'm hoping everyone likes fast queries. And databases staying up and continuing to connect. Yes, yes, big thumbs up in the front row, thank you. So a great thing that they actually added as Postgres 10 is native table partitioning. There's been extensions in the community for a while that do this, but let's go into some of the big wins on native partitioning. So native partitioning actually can entirely avoid the vacuum and bloating process. So bulk loads and deletes and offloads can happen without causing overhead to your Postgres internals. So slow accesses to your database connections, things like that, they're gonna be a lot less costly because you're not stressing that one giant table as much. Our most common used rows are only gonna be the sections of the partitions we care about. And a lot of the seldom used data can stay where it needs to on slow disk storage or even off on cold storage. We can move it off entirely. And one of the great things about this is that queer performance improved dramatically. We got a lot more bang for our buck when we actually do the lookups. Previously, we'd have to do best effort random index scans and then random access reads to the disks. With large data like this, disk data storage is kind of fragmented in some scenarios. And so in average and worst case scenarios, if you actually do complexity, some of these cases actually end up being worse than sequential scans. So native partitioning is a great way to optimize for this. Another key note to have here is PG Partman is still in 10, it's still supported, but it doesn't have some of the parallel support that we're gonna talk about a little later in this talk. Some other small noteworthy items for native partitioning. It's not natively supported in ORMs. Obviously, you saw me writing raw SQL in the migration script. There have been gems to try to make this more consistent in things like active record, but none of them are really well supported right now. A lot of, another great thing about this though is tables can be created from your partitions. So if you need to do what I'd like to call YOMO tables, your month tables, this is, or even week tables. This is something that you could absolutely do. And this is a great way to even segment, even data, even further as it migrates through. And it's an easy way to understand your data over long periods of time. And you can even create partitions from your partitions. I would add a partition meme, but I feel like they're overkill. But so now we wanna do a pivot. So we've had a lot of use of the platform and people are starting to create products. And now we wanna do home delivery. So how do we get that done? Well, we obviously need customer addresses. So this seems pretty simple. And we want a mobile app to be able to access lots of different customer addresses when they're going and making deliveries, right? We wanna take customer service seriously. But we do have a lot of customers and we have a lot of addresses. So addresses aren't uniform across North America. Actually, they're very different even between the US and Canada and even between different states. Different states have different rules and regulations about where delivery drivers can stop and other really important features and gotchas like that so nobody gets ticketed on the street. So there might be some important instructions we also need for not just customer satisfaction but for driver notices as well. So we can kind of store this unstructured data already in Postgres, right? HStore already exists today. A lot of people I'm sure use it. It's not new. It's a key value store. But implementing it seems straightforward enough. Let's just add the hash, right? Yeah, man, those address lookups are real slow. Real slow. The other problem here is that when somebody's trying to make an update to this, our drivers aren't gonna get it in real time. It's not gonna happen. We could add an index to improve lookups though so let's try that and let's see what happens. So let's add a hash index on usage and hope for the best, right? It seems like it should work. Maybe things improved. But ooh. Show of hands, anyone seen this error message before? Show of hands who uses hash indexes in here? Not many people. Okay. Well that's good to know. So probably a lot of people know for good reason that hash indexes aren't really well supported in Postgres 9x for a lot of different reasons but the key one being replication is not supported. So it's not treated as a first class citizen. The other one is you get a lot of misleading failures like that that kinda looked like a data corruption issue but actually that's an index corruption and that's because the hash index can't keep up. So there are other things about this is that if we have extremely large data sets to pull from sometimes you have to completely access the hash index and rebuild it from scratch which is not a great experience. What's the solution if we can't rely on this hashing mechanism if we really wanna use age stores since it's mostly supported? Well Postgres 10 actually finally brought it into the light and made it a first class citizen. So it added first class citizen support such as replication. It made it crash safe through shutdowns and failovers. Replication is done faster lookups so the growth of each page for that index are actually consistent with regular indexing practices and the pruning happens much more consistently as well so indexes don't go as stale as often. So going back to this we can rely on this much more consistently and have better peace of mind and we do see actual improvement now checking through and updating the hash itself. The biggest thing that I'd like to point out here is the locking concurrency issue was a huge problem and we use age store at Heroku for a lot of different use cases. The reduced locking for concurrent access for records is a big win for us even though we don't use a lot of hash indexes age store in general for the few places that we do that this is a really big important thing for us. So going backwards we're talking about more curated content on my social media platform and I wanna talk about movies and TV shows and characters that we love that we can't get enough of for any big nerds out there. I'm a huge Flash fan. I met John Wesley ship for the first time earlier this year which was fantastic. So I wanna list all these movies and TV shows and I wanna get the characters for all of them. So each show can have potentially 50, hundreds, maybe even thousands of characters if anyone watches Game of Thrones. I'm sure everyone's excited. So each record can have a lot of varying data over time and some things that we'll wanna try to watch is genres, catchphrases, famous accessories and items or even just certain ways that they dress or look or interact with other characters. I'm sure some people are gonna get a little schwifty after all of RailsConf. So we need some way to keep that data unstructured but co-located with the general structure of how we wanna address media and characters in general. So let's use a JSON blob because this seems to make sense for this scenario. We can query third-party data sources and we can get them in consistent JSON formats so we can reparse them over and over again and we can get updates. The great thing about this is that we do have this flexibility but it opens us up to a lot of other gotchas, one of which is each row is dynamically sized because of this. So it's not consistent what size each record is and because we can have so many different data sources analyzing each character, each record, this could potentially get a little hairy in tracking how this information changes over time. So how can we keep this unstructured and not have to parse it all or create constant relationships through this? So we could create multiple tables. We could add a lot of one to many relationships. We could modify our Rails app entirely to manage this information. We could even do partitioning based on some subset of where the data is co-located per show. But as much as we could do all those things there is some other easier item to do and that is using the full text index searching that's been added in Postgres 10. So because we can store whole request bodies, we can store custom made JSON blobs. This allows us for a lot of more no SQL dynamic schema operations and this is what we're really caring about especially when we're trying to get this off the ground in initial stages. This adds a lot of flexibility. So one of the great things here is that there's a lot of examples of how to do this online now on the Postgres materials but this keeps our data structure unstructured and flexible. So the big thing about this is indexes can actually filter on a lot of different views and kind of perspectives that you wanna do for the type of index searches you wanna do on your JSON since they are treated as first class citizens as well. The other great thing about this is this gives us ways to target folks on the platform that we're looking at if we wanna target them for being big Marvel fans or maybe somebody can't get enough of Joey from Friends. Maybe he just wants to say how you doing all the time. But I'm a big fan of Thor so a lot of hammers for me.  That was the big one that got the platform recognized and people love Throwback Thursday. So let's create a throwback stream of quotes for folks who wanna know. On this day, my favorite posters posted these quotes. So we wanna be able to replay a stream of old quotes. Well, streams should include all of the posters that I liked and followed, not just my own posts. So that's a lot more quotes than you would normally see and even getting one month of posts. And that's a lot of queries, especially if you're following a few hundred other people. That's a lot of quotes, especially for power posters who post multiple times a day. So, yeah, here we go. So Postgres 10 did a lot of things to optimize how we do these types of queries and how fast they come back. So we're gonna, I'm gonna touch on three. There are more optimizations in the parallel areas on how to return queries that Postgres 10 did. I'm only gonna touch on these three because these three are usually the big ones that people talk about. So we're gonna talk about parallel bitmap heat scans, gather merges, and merge joints. So let's jump into it. So in nine six, parallel searching and scanning was introduced. It was a little limited though. So it used to only do sequential scans and sequential scans were only done on the primary tables. So this kind of created the problem where you couldn't rely on indexes. We had a little bit of a trade-off here where you either had to rely on your table caching to be really effective or your indexes to be really good, but you couldn't get an overlap here, which was kind of painful. The big thing about Postgres 10 is that changed when they added way more parallelization to every level of your table schemes, including indexes. So you can do parallel searching through your indexes first. You'll actually go through and construct the only the index files first so that you are much more methodical and strategic on how you do your queries and how the results are loaded. And so this requires a lot less data loaded up front, a lot less caching, and way less results that's done to parse through and a whole lot less disc reads. So the next one we're gonna talk about is gather. And gather is actually the mechanism in which Postgres introduced how to do parallel query construction and result retrieval. So it was introduced with the idea of workers, obviously, to do asynchronous processing. And the big thing here was that there were multiple layers on how results were returned. A lot of the time they were returned in arbitrary fashions, mostly in the order of how fast they returned from each worker. So unfortunately, if you're adding searches or sorts to this portion of the mechanism, this is gonna add a significant amount of overhead because you're requiring not just the search of the B-tree, but you're also requiring an extra order layer to happen after the initial result sets are retrieved. So a small example here is, I have a bunch of IDs in my B-tree and I just wanna retrieve the primes, right? So here's a small example of a B-tree. The issue here is that when we're going through and checking through primes, the leaf nodes in some scenarios have more content than either the roots or some other leaf nodes. So certain leaf nodes will return faster than others. The root node is our first returned response. So 17 was actually the first node. I'll go back here. So 17 was the first one that we actually saw on the left side. If you'll actually look on the three leaf nodes to the left of 17, 11's only got one. So our first records are gonna be 17, 19 and 11, depending on how fast they return, then two. So we actually get stuff out of order. So another pass is gonna be required to sort through the somewhat strange return results of the workers themselves. And so they were only trained to return as fast as possible with no context. Well this changed in Postgres 10. So Postgres 10 changed this by optimizing the worker collection results. So gather merges actually sped up the parallel worker result retrieval because they also kept context of where they encountered the data that they actually cared about and they kept them into account as the result sets were returned. So it removed a lot of overhead for sorting after the fact because at the end it ended up being mostly just a flat line. So this was really beneficial especially for really large data sets traversing huge tables. So kind of revisiting this. We can see that we have a whole other side of the B tree we didn't touch, right? I kind of glossed over it, but for good reason because really when we parsed it down we actually held context. It did actually move through the B tree and construct how we thought we were gonna see everything. The reason we kept context here is because we kept context of the locations that we visited each node at. And so this is much more natural ordering of operations from an intuitive mental standpoint. So it's easier to mental map how our data is retrieved as well too. So the great thing is is because as we merge the leaves and the roots we basically can turn this into a link list and it becomes a simplified array that's mostly natively ordered even before we add other order clauses. If you add the order clauses they're kept in context along with the parallel workers. So they know what to do and when they're ordering as they return results. And then the last one we're gonna cover here is merge joins. And this one while maybe performance wise doesn't always pack the most wallet is one of the interesting ones for me as well. So kind of applying this to checking all the other posters I care about for throwback Thursday. I need to join on the other customers quotes, right? So previously 9.6 only allowed this on nested and sub query loops or hash joins. This doesn't always happen on the primary query of the table. So a lot of our sub queries were optimized for this but not necessarily the originating one that we cared about. So this adds a lot of overhead and even though indexing and ordering is great it could actually make things worse and not better especially the indexing portion which is surprising sometimes but it can be a gotcha. So parallel merge join was added to consistently parallel do parallel operations for all of the merges at all levels. So all of the sub queries all the way up the stack from each sub table or even tables you've had to rejoin to get different data sets for temporary tables for example are now gonna be done in parallel as well. So this is gonna make things much more efficient. Outer sides of the joins especially when we do null comparisons or checks like that are gonna be the most effective because we're not gonna be as restrictive here and it's gonna be much quicker to run through certain sections of that with less data and combined with a lot of the parallel changes to the indexes gonna be way faster in getting all of the data that you care about. Again the parallel changes that also happen to gather merge will give you some natural ordering by default even before you specify order but will still maintain context for adding order as well. So the last thing I'm gonna touch on which I did mention this is a real world example actually from something that we track at Heroku which is part of tracking our issues on our internal repos and external repos as well so that we have a good list of items from when we get feature requests, issues, bugs, things like that and that is actually even though this table is pruned pretty consistently for the issues mapping the GitHub issue to a repo ID it's not huge but it's not tiny either. Here is the query analyzer for the table for a given repo ID. Execution time's not great. This index is basically not being used even though we expect it to be used in 96 it definitely didn't get used at all and so we end up doing sequential scanning through it and it gives us some pretty crappy runtimes. Here's the great thing. With all the changes with Postgres 10 the parallel index got figured out by the query analyzer and it's being used now.  Gonna let that sink in for a sec. That's a half a gig table. That's not even a big table. That's a tiny one. The changes that have occurred in the performance gains are huge.  That's another talk in and of its own. That is way too much time for what we have to get through today. So unfortunately that's, if you wanna hear me talk more about it or explain anything more that I've seen of my last year and a half, two years come talk to me after. But there are a few last minute gotchas with Postgres 10 with Rails that I wanna go through. When I touched on some of them but maybe some of you might not know. Rails 4 doesn't support Postgres 10. Unfortunately the connection protocols changed in how Postgres 10 works and so unfortunately Rails 4 only supports up to Postgres 9.6. If you wanna make the move to 10 you will have to be on Rails 5. Again, as I've stated before, partitioning is not native to the Ruby and Ruby on Rails ORMs. So our specific index types like hash indexes and JSON indexes for the full text query index searching we went over today and the analytics queries are all not available except for in raw SQL as well. So some things to note but that's one of the reasons I love working at Heroku is because we try to offer a lot of tools for stuff like this. So with the release of Postgres 10 and the support for that, we've added a lot of extra support in numerous extensions to support Postgres 10 as well. We've added managed Postgres credentials because the permission schemes are very confusing as we've found over the years. New updated hardware plans at the same price points improved CLI analytics because our analytics CLIs use the same query analyzer that they do there and the same PG internals that have had the optimizations done as well. And we're betaing a new feature right now for those to test out called Postgres connection pooling. So the Postgres extensions that we offer are numerous and most of them are natively offered. We offer a few external ones like PostGIS and things like that. If you want the complete list, go check them out. But the big one that I love here is the time travel one. That's a big favorite, especially with the new changes to partition. Postgres managed credentials was something we actually released late last year, but we did this for a lot of different reasons. We had a lot of customer asks for this. So Postgres permissions are really difficult to reason about. They're at a lot of different layers and they have a lot of nuance into creating granularity, understanding revoking permissions, things like that. We still allow customization when we issue managed credentials, but we've tried to take a lot of the guesswork out of initially creating a lot of these credentials and roles for tables. So we've tried to consider things like special tables or special characters. No one likes SQL injection. Bobby Tables can stay at home with his mom. Name lengths are another big gotcha. There's actually certain character limits that if you blow them, Postgres silently doesn't tell you. So that's another strange one. And we've also published some of the best practices as well. So if you wanna see some of the other things, you can check them out at the Postgres credentials dev center. But I encourage everyone to, even if you're not using managed credentials, consider things like separations of concern, principle of least privilege, and even better external control access. If you're using a third party vendor that needs data access, you wanna give them very restricted things. You don't wanna give them access to PII data, for example, to some of the things you insure for your customers. So the other big one that we did earlier this year was Heroku PGX plans. I was directly actually part of this project, which was fantastic. I was very happy and excited about this, which was we were able to utilize the latest hardware virtualization for better IO, better CPU and query optimization. And also we've added all of these plans under dedicated hardware so that you get consistent and stable performance. We've documented about it in the blog post, but the big wins here are that we've doubled CPU cores at tiers four and up. We're exposing some of the dedicated provisioned IOs for plans and we're increasing data disk size at every tier and prices either stayed the same or lowered. So please go check that out if you're interested in checking out some performance on the hardware side. The one that we do for a long time is one that I use on a daily basis, especially when people file support tickets for us is the Heroku CLI, especially for Postgres. We've used PGDiagnose and this is something that has been out there for a while, but I wanna reiterate because Postgres database tuning has been a big deal and other people have talked about it at the conference. We've had a Dev Center out there for a long time and we have the extension CLI to add on to the native Heroku PG CLI commands. So the big one is PGDiagnose, but if you're looking at sequential scans or caching and hit rates, some of the things that I might have talked about today, we actually offer pre-defined ways of querying that information through all your tables and schemas. So please go check that out as well. Now, the one that I know a lot of people have heard about and a lot of companies have started using is PGBouncer. So PGBouncer, for those of who are familiar or not, have added management for Postgres connection pooling. This is basically because there's not really this idea built into Postgres, so connection pooling is done outside of Postgres and this kind of adds insurances to making sure connections don't stay open longer than they're supposed to be, queries don't kill your database and we've added this as part of the connection pooling. That's how this is done. PGBouncer is in beta as Heroku PG connection pooling. So why use it? Well, one thing Heroku in general loves to see is stats. We love our stats. This is better visibility into seeing how our connections behave. This also gives a lot of guardrails to protect your database and keep it up, especially if you have multiple applications connecting to a single database. And this is really great for asynchronous workloads. For anyone using lots of active job or anything like that, this is fantastic. We've offered it as a beta. We're still working out making the best practices out there. We're still trying to get some feedback from folks who ask for more customer visibility or better use case scenarios. So we're still doing some public beta work on it right now. It is supported out there and it is well documented. So if you're interested in using what's on server right there for PG connection pooling, please go check it out. We still offer a client side build pack that can run on a dyno. It doesn't have as many of the built-in features that we'll be adding to Postgres connection pooling, but if you are interested in using it separate from the database itself on your application layers or something like that, it's still available out there. Whew. And that is it. What a marathon. Thank you so much. Please come check out the booth. I'm the last presenter for Heroku today. And as always, thank you again so much for letting me come and talk to you at RailsConf.


So one of the things that happens when we design and call out a method invoke a method is Please excuse my dear Aunt Sally. So PEMDAS and The first one is parentheses. So if we take a look at this bottom example here some XY What happens is I Evaluate what's inside the parentheses first in this case X X is going to be evaluated And you see that's where what we have here. So this becomes five this seven Becomes or this Y becomes seven those two things occur first and then I Assign it over here So what's going on when I design out a method when I use a method is something called passing by a reference and the idea Behind this is now that when we're working. We're we're looking at something in the sense Let's see. Give me two seconds. Go ahead and fast forward So when we're designing out this code say for example, I've built it out and I do something like index equals one System dot out that print line The value of X Plus X. I'm saying this is what it's going to be Now I'm going to make a second one. I'm going to make a method here called increment X now increment X if we kind of take a look Increment X is a void return data type. So it doesn't need to be stored anywhere So I come down here. I normally put it after my main method or before it really does not matter public static void increment And number again, like I said, I don't care what it was called outside of my method. I Don't care that is called X here inside my method.  You are number in my world And the same kind of thing I'm going to change this and I'm going to say that this Is now number the reason why is because there is no X inside here Again, it doesn't exist. Oh, well, it exists over here. No, no that exists in this world No X in this one. Notice how I highlight it. I don't see there's an X anywhere. I didn't name it X So I'm just going to call this guy number and This guy number and I'm going to do number plus plus So what happens when I write this code when I run this code? What are the values going to be? What are my outputs going to be and take a look write this code out see what happens? I'll pause for one second Okay, that was hopefully one second probably a little bit more but I pull this out and I bring out my Code so again files I'm going to compile out my test dot Java file Compiled out fine Java test The value of X is one Hopefully, yeah, that's exactly what I was looking at X was one So I printed that out then I go into increment. Well again What happens is anything when I call a method? It activates. I don't move forward in my main method I don't go any further in my main method until this is done So as you can see there's a lot to do the first thing I say is to print the value of number Again, remember I call it number not X So then all of a sudden I say well number plus plus All right, then print it again. Well number was passed by a value meaning only the binary numbers of One were passed on This is because an integer is a primitive data type a A primitive Data type which means it's very small especially in the computer scale of things so I can just Pass it around I can move it wherever I need to I don't have to be too careful with it because things like integer chars Boolean's doubles longs Floats even though they sound big, you know 64 bits 32 bits They're not really big in that grand scheme of how you know how much my processor can handle at one time So I can just throw them wherever So this just magically gets you know the 32-bit notation of one and Then it does whatever it wants to with it and then outside of this it just moves on and as you can see X never actually changed only number


our 244 channel is where you can find us across all our social media handles at Michelle Ashira is where you can reach out to me in this particular session we dive into an interview that looks at the entertainment service industry which is very vast and most of the case is a reflection of our own society and there is money in this particular business. Joining me is two gentlemen from Unlimited Focus Media I am joined by Cassenti and Stevenson I'm so sure those are not like they were you know the names of your ID but I'm standing for Musa and Cassenti so what's your names as by you know identification cat okay my name is James Kamita Nyu too my business name or brand name is Cassenti right yes from Cassenti production Cassenti production is a company by itself so I'm an audio producer artist and entrepreneur in the unlimited focus media right in the company we Cassenti production okay we'll get into that you know what right to you Stevenson right yes your name as by identification and what is your role in unlimited focus media yeah sure my name is Stevenson I am a film director content creator that is okay yeah so guys prior to us coming to you to have this conversation I had a brief interaction with these two gentlemen and they do a lot in the entertainment industry and we're just going to look at a couple of things they engage into this particular business and how they're making money in the entertainment service industry so starting off with you Cassenti you mentioned earlier on about your own production let's start off from there and how you got into just merging and to unlimited media unlimited focus media because Stevenson also had your own company prior okay let's start off with your company and what it deals with or and did you dissolve it or is it actually still up and running Cassenti production is still up and running we've decided to to join our companies together because we can help each other I can I can do audios for him you can do videos for my company it's very easy then there's a there's a chemistry between our companies that's the best thing yeah right one thing that cuts across you guys are also artists yes okay so back to Stevenson right I mentioned all that your musician and you actually still an artist I like to find out earlier on while growing up did you have someone within your life that opened a particular I within yourself and made you actually realize that you can get into the entertainment industry and make a business out of it I started as a musician and back then like seven years ago when I started music I did my first song of course I needed a video and so when I went to someone and I told him you know I want a video of this song so they charged me they did me a video but the video was not that good I was like I'm really household and I've given you my money and this is what you can give me so you are talking getting their place from TV stations and they're like no what is this compare these and this good quality and poor quality so I was like one day I would wish to be a film director okay you know back then I was I was milkoa Macanga was a Macanga so I didn't know that my hustle couldn't allow me to get good money to do good music and then I was like now me up not I'm not educated I'm not going to university to study these things how will I ever become a film director okay but I got a friend of mine I look a paparazzi work bigger picture I can do it so instead of being a paparazzi you cook a you picture it was like for the whole one week it was a wedding yeah what hindi you know what hindi's weddings it goes from Monday to Sunday so the deal was there but now you know me I don't know how to hold it in the camera but this guy was very confident you guy you'll go get we will do it to me and you we want money to won't employ people from outside so we went the first day it was a struggle the second day that day before the fifth day there were guys from Canada who were doing film the same same place so they started training us and even after the wedding they took me trained me that is how I became a film director so I normally direct my music videos and now I cannot do audio so Cassie raised me musically because if you listen to those songs that I used to do before yeah but there is a growth he has to do that group okay and that's why when my company stood I just decided to work with him because now he knows my weaknesses you know where I come from interesting like a centi Stevenson I say that he played the very major role in his life when he comes to just not change his talent as an artist so and also he mentioned something very important like when starting off as like a conductor to the fact that he was thinking I don't have good money to make a good production with the production how important is it when it comes to in this business is finances that really important and what will you tell someone who's watching us and they feel like I'm talented in music I don't really have that money so nowadays it's very easy it's not like a whiteback where you didn't have access to information like YouTube and stuff so the first thing I did it with him is develop his talents his singing capability I guided him through the things he can do because nobody was born knowing anything you just consume whatever whatever environment you are in and then you're inspired by the people around you so you might start when you're young you might start when you're old but nothing is impossible if you really really want it and the good thing is a very good listener he loves to learn he used to sing people used to laugh at him like we're not talent but he had that passion so he developed his sound his voice now he can I can probably say he can give you a pretty good performance how did you learn about audio production phase when it comes to your case my dad my dad taught us like I'm born in a family of music so I did I started developing music as at an early age so my dad taught us keyboard but production pretty much is self-taught through YouTube YouTube University is the best yeah so Stevenson but when you're having the story the wedding you don't know how to hold the camera so how did you get to the point where you can actually do video production for other artists and also other entertainers how did you learn and how long did you take for me I acted like I know yeah just that dream like you've been there so after this guy is from Canada talking some few things became a duration of one month they left to their country and so I joined YouTube as well trained myself that is where I started doing some free jobs of course I worked with Mr. T in some some services in churches and many other places and that is where I developed you know practice make perfect so I practiced and every time there is a software that is coming up I train myself from YouTube their tutorials there so those tutorials made me to become who I am today what made you resilient and persistent and not giving up because I never wanted to go to those I remember there are those places when you when you remember where you're coming from we just say yeah let me press on to this because I want to have something in the future to be more better than the way people so I could not even think of flipping so every I I I said it myself at the time like because then for you you probably heard that when it comes to huge role yes it comes to me learning about productions how long have you been to this business um about 12 years yeah 12 years yeah okay what are a couple of such challenges that you faced while starting up um it's not easy you have to really really really really like what you do you have to put your heart in it you have to develop that skill every day for people to start seeing what you have like when you see someone blowing up you don't know what they've been doing to cultivate to to get to where they are I saw a video of diamond the other day that I never saw so that means you have to develop this thing for a long a very long period of time even you I know you never just woke up and started this absolutely yes so you have to develop it's like you have to do it every day every day you have to put in the hours I actually we actually started sleeping early like this year after all those years so we never had time to sleep right yeah and it's quite an impact having you over and to have this conversation for being in the game for 12 years then have young people out here who are like they see the buzzing entertainers or learn due to social media they feel like you know I want to be rich as soon as tomorrow they get so they end up doing unnecessary things or getting so get themselves into a necessary advice to someone who is young and they want to be probably millionaires by 30 they can have their goals by 25 they should be you know living this lifestyle number one it's a process they should they should take their time to develop that skill then link up with people look for people who have the same thing you want if they ignore you go to another person you might find out your solution is around you never ignore any person treat everyone with respect because you never know who that person is going to be tomorrow and maybe the person you're ignoring today might have the key for your future tomorrow like where you want to go all right so let's look at unlimited focus media so looking back what was the inspiration starting this company and what are the advantages of operating as a company compared to Stevenson Acienda a particular project or an event oh sure unlimited focus is based in no limit that is no limitation that is nothing impossible because when I look at my background in all those processes that I've gone through in your story yeah I see an inspiration so the word itself unlimited it's an inspiration to me and it creates an opportunity for me when I wake up and I see I don't have a camera I say I'm not limited I can get it I can go and hire somewhere and do a good job and that job pays me I get a new camera a good one and that is what I do so I say I don't limit myself in anything even if today I get a one million project because they normally come I say I'm not limited I will do it I will not go and hire a retreat people I will do it unless I go and acquire some knowledge from them that I don't know they come sit under me they show me and then they leave and I do my job so I'm not limited and that's why that is the source of my inspiration from from from the word unlimited okay how long have you been operating it's the fourth year the fourth year and uh and uh most so many people who have uh made my company grow they are the influencers I've worked with different influencers I've raised influencers okay well they mentioned a couple of influences have worked at the moment I'm working with Henry De Sago and I've worked with Azir in a certain advance I've worked with some guys from Real House of Congo and many others okay so a couple of good names there when it comes to content creators sure well so uh Kasanti when it comes to offering value to your clients how do you ensure that you actually give value to to their money when they come to you when they ask for audio production video production considering that we have so many production companies um first we really really want to know what you need from us because when you come to us you believe in what you're going to give you now we want to know your vision so if you know your vision we'll be able to deliver something for you if you notice some weaknesses in what you need we will advise accordingly if it's vocal coaching if it's uh songwriting anything that will make you look better the end product is more important to us than than the process itself all right okay so before we go ahead let's check out a couple of projects that you guys have done and studying us off is one of the content creators Henry De Sago