Okay, so let's start with palettes. I'm getting rid of the Valentine palette scarlet. I really like this one. I got it, but I have hardly reached for it at all. I have so many palettes. It's a beautiful palette, lots of pinks and reds in there, but I have other palettes that have those colors in there, and that one just seemed to be very repetitive. So even though I did like it a lot, I'm going to get rid of it. I'm getting rid of the NYX Ignite. I think I've had this in another video before, but this is the NYX Ignite one. I swear I'm never buying another NYX palette as long as I live, and unless they come out and they say that they have a hugely different formula, this is the NYX Swear by It palette. I cannot get those to have hardly any pigment in them, and they just fall short for me. So I just, I'm not going to buy any more NYX eyeshadows. I'm getting rid of the Perception palette from Colourpop just because, again, the situation where I don't reach for it, and I've had this for at least a year, and I think I've only used it one time, and even though I keep thinking to myself, I'm going to go back and I'm going to reach for it, I really never do. So I'm getting rid of that one. I'm also getting rid of the Yes Please from Colourpop. This one is their best selling palette ever, but I find that for me, the colors are just too orange, too warm, too stark. And again, this is another one that I don't reach for. I think I've done a couple of looks out of it, and I just, I just don't reach for it. So the next one is a knockoff that I got off of. This is from Beauty Glazed. It's supposed to be a knockoff of the Hood and New Nudes palette, and I was going to do a comparison with it. When I looked at the ingredients, I wasn't really happy about it, and it really wasn't a dupe for it, 100%. So I did swatch that, but I didn't use it, and I'm not going to. I'm also going to be getting rid of the Too Faced Shadow Insurance. I don't like this one. It doesn't do anything for my eyes. That's absolutely spectacular, and I find that my eyeshadow does crease sometimes with it, so I'm not going to be using that one. This one is from Becca, and I can't remember what this one's called. This is the Soft Light Blurring Powder, and I just, I don't like this. It makes me way too cakey. I don't know why. I don't know if there's something in there that just doesn't work really well. Even my daughter tried it. She's 20, 22, excuse me, she's 22, and she didn't even like it. I'm getting rid of the Jouer Essential Full Coverage Foundation. I even tried this two more times right before I decided to put it in here because I'm like, I need to give this another try. I find that when I use this, it just makes the texture on my face stand out so bad, and it wears horrible on me. And I don't know whether it's just because I have really, really dry skin or what, but I just, I can't do that. It's really hard. I bought these, and I was going to review them for you guys, and I put them in products that I didn't like. These are the stamps for the winged eyeliner. One is a right and one is a left. You get two. Could not make this work, so I'm going to get rid of that one. This one is the Monture Big Mascara from Lancome. I like the regular formula, but this one is the waterproof formula. And I did like this the first time I used it. As a matter of fact, I used it at my daughter's wedding because I knew it was going to cry. But definitely after that, I just couldn't get it to work. I don't know if it dried out on me. I didn't use it soon enough or what. I went back in like three weeks later and just really couldn't get that to work anymore for me. This is a single of the NARS eyeshadow. And even though this is a pretty kind of a golden taupe color, it is pretty, but I just don't reach for it at all. So I'm going to be getting rid of that. I'm getting rid of two of the Jcat Glow baked highlighters. These are some of the driest highlighters I've ever used. You probably could mix something with them like a cream or something and make a cream highlighter, but I have so many that I'm not interested in doing that. For makeup revolution, I so wanted to like these because I love makeup revolutions palettes. But these were just not good. There is a waterproof and a regular mascara here. They need to make their wand or their stopper inside of the mascara so that the wand comes out with less product because it was so goopy that by the time I was trying to wipe it off, made a mess there, and then if I just went in, you would immediately just get a goopy clumpy mess. So I was really hopeful for that one, but it didn't work out. Speaking of revolution, I'm also not really crazy about their skincare line. They were kind enough to send me all of their skincare line, but you guys, I just couldn't get on board with it because after I researched the ingredients, they weren't that great. And there was other things in there that I just don't really want on my skin. Does that make sense? And the one that I did try, it really didn't do anything. The other thing that really is hard, when I look at a skincare, and it says something like what this one says about stabilized active collagen, the collagen molecule is too big to penetrate our skin. So when I see something like this, I just kind of just roll my eyes and go, I wish everybody knew that from Kota Lee. This one is their Beauty Elixir Soothing Glowing Complex, Soothing and Glowing Complexion. I could not get past the smell on this. It was so nasty that I don't even dare open this. It just really, it kind of made me a little bit sick. These are two foundations that I've had in my collection for a long time. They're in the old packaging. Even this is from one or two cosmetics, and this is called Cover Proof Foundation. This is a good foundation, you guys, but this has been in my collection for way too long. I have it in light, and then I have it in medium. And I do like this very, very much. And I used a lot of it, but when I got to, you know, opening my door for the summertime this year, I realized that these have been in the back of my door for a while, and they're just too old. I got to get rid of them. I have a couple of loose pigments. This one is the gold one from Too Faced. And then this one is from L'Oreal. It's an infallible magic pigments. This one turns from a powder to a cream as you're putting it on. Great concept. Couldn't quite get on board with what the, you know, the actual application was. It wasn't as nice as I thought it would be. When you swatch it, it's great, but when it gets on your eyes, it's not that good. And then when you use these gold pure pigments, it makes a really big mess. Plus, this one was just a little bit too stark for me. I probably will never use that color gold, so I'm getting rid of that. Some amazing cosmetics is this little tube of cream concealer that I got somewhere that I was going to try, but this creeps so bad on me, so I'm going to declutter that one. This one is from Estee Edit. Anybody remember when that was a brand? This was a good concept. In the top, you had a cream product, a little mirror in there, which you don't really need, because I don't know if I'd put my, I make up on with that little mirror. And then in the bottom, you have a loose powder, and they are coordinating, and it's a really great concept as far as how pretty they are. This has just been in my collection too long, so it's time to declutter. From ColourPop, this is one of their loose eyeshadow pigments. Even though this is a gorgeous pigment, it has a little bit too much glitter in it for me, and I found a lot of fallout from it. So I'm going to be decluttering that one. From La Rock, I have a little bronzer that is just very orange on me, so I'm decluttering that one. Oh, I missed a palette. You guys, I'm decluttering the Maybelline Soda Pop palette. I did like this. However, I just find again that I'm not reaching for it at all. Also, from Makeup Revolution, this is their High Impact Opulence palette, which is basically just a whole lot of shimmers of gold and champagne and copper colors. Did not look really good on my skin. That's the only reason I'm decluttering it. I do have another one that is more along the lines of the pink colors, and I do like it a lot, so I am keeping that one. Don't anybody say anything. This, I have tried so hard to love. This is the Too Faced Peach Perfect Comfort Matte Foundation. There are so many people that says that this is their Rider Dye Foundation. I don't know whether I'm weird or not, or whether it's just because I have super dry skin. This did not work. I couldn't make it work. I couldn't make it look pretty. I just, oh my gosh, I, yeah, I'm not happy with that one at all. I'm decluttering a Makeup Revolution Baked Blusher. I'm only decluttering that one because it just really isn't one that I reach for at all. I'm decluttering the NYX Liquid Suede. This is crystal metalized or something like that. It's the color I'm decluttering that, and I have a ton of nail polishes that have just gotten too old that I'm going to go ahead and declutter as well. From Colourpop, this is a Liquid Eyeshadow. I find that this wears off way too fast, so I've got flaking on my eyeshadow. This is so pretty. I really love the color of it, but I find that I've got flaking off of my eyes way too fast through the day, so I'm getting rid of that one. This is supposed to be a Lash Extension Mascara that I got off of Amazon. I did not do what it said it was going to do. Another Colourpop Eyeshadow. Some more of these nail polishes. I have a ton of nail polishes that I'm getting rid of. The L'Oreal Infallible Metallic Single. I don't mind this one. I think it's quite pretty. I actually got it because of Toddy's recommendation one time, but I just find that I don't reach for it that much. And boy, these shadows are looking pretty on my hand, but they don't perform like that. So, you know what, that's why I'm getting rid of them. From Wet and Wild, this is one of their Liquid Eyeshadows as well. This one flaked too. It was really not good at all. This is a JKat Beauty Liquid Eyeshadow. That one didn't work for me. This is a lipstick that I can't remember where I got it, but when I rolled it up, I'm like, I am never going to wear that chocolate cherry deep dark plum color. So that one is going. A Rimmel lipstick that is just super uber bright coral pink, and I probably will never wear that one. So I'm getting rid of that one. I'm getting rid of my Wet and Wild Photo Focus Concealer, because it's just too old, along with the same thing on a couple of my Makeup Revolution Concealing Defines. I do like both of those, but they're just too old. Lash Paradise Mascara. I don't like this mascara. I have not liked this mascara since I started using it. It's just not one of those that I pick up, so I'm getting rid of that one. I'm also decluttering a Catrice 18 hour eyeliner in black, and the only reason I'm decluttering this one is because I have a lot of other ones, and this one you can't get anymore. I guess it's been discontinued. So even though I do like it, I'm getting rid of it because I can't show it on camera anyway. I'm getting rid of my Wet and Wild Photo Focus Foundation because this is too old as well. I'm getting rid of a couple of the matte glosses from NYX that just don't work well on my lips anymore. I'm getting rid of a perfume. This is Euphoria Deep. I love the regular Euphoria, but this one was just a little bit too intense for me. I'm also getting rid of this Jelly Mutch color from Colourpop. I really like these. These are really good eyeshadows. However, this bright teal green, it just doesn't look good on me, and it's hard to work with up here. It's nice to work with on your eyelids or underneath your eyes, but this one just, it just fell short. So I'm getting rid of that one. This is a Rimmel Art Metallix. I'm getting rid of that one. Been there too long. A Sigma Eyeliner that has been there too long. This is from L'Oreal, a true match concealer. This one didn't conceal for me, so that one's going from Sephora Collection. This is one of their eyeliners, and it's just old. It needs to go. I'm getting rid of a Tarte Shape Tape and a Rimmel Stain Matte Concealers that just don't work for me. They're super-duper dry. I'm also getting rid of an Anastasia Beverly Hills Very Nude Color Lip Gloss. I love these lip glosses. I even put them in my recommendations, but this one is like so pale that it actually is lighter than my light skin. So that's saying a lot. Then from Chouet, I actually did get one of their Metallic Liquid Lipstick Sets in the holidays, and this color was just too dark. It's just really, really dark even when you put it on, since it is a liquid lipstick. It even gets darker, so that one's got to go as well. I'm getting rid of a bunch of brushes that just, I just never use them. So this one, these two right here are from Luxie. They're fluffy brushes, and then this is an angled brush, a brush, excuse me. And then this one is from Bare Minerals. You know, it's got a flat end on there, but I don't use these for my foundation. This is an amp brush. It's really old for brows, and then this one is from Color Pop, a fluffy brush that just wasn't cut well. And then this one right here is I think is a Morphe, and it's just, I would never use that for anything specific. I'm also getting rid of this It brush, even though I am a huge fan of It brushes, this one's got to go. It's domed, and it just puts down too much product. So unless I was a person that would use a ton of mineral makeup, this one really wouldn't be a good one, because it's really just a buffing brush. Okay, I'm going to put my glasses on for the last of these, so I can go through it fairly fast. From Color Pop, one of their satin lipsticks, I don't like that it's too dry. From Ulta Beauty, this is one of their cream eyeshadows. It's called Opal, beautiful color, but I find that the cream doesn't last, and it creases really bad. From Jaycat Beauty, this is one of their Prism Eyeshadows, Prismatic or Prism Metal Eyeshadows. Love the color on this one. It was a pretty purple, but it didn't stay on, and it flaked like crazy. From Butter London, this is the Glazing Lustrous Liner. This one really made a mess and got everywhere. This is the lash domination from Bare Minerals. Couldn't build any lashes with that one. This is from NYX. It's called their Slip Tease. I think they came out for this in the winter, and it was okay, but the color was really bad on me, and I don't like it. And then does anybody remember these little teeny lip glosses that we've gotten sets? I don't even remember how long ago at Ulta. I loved getting these because you got to try so many different colors, but at the same time, it just is too old, and it needs to go. That's it. That's all of them, you guys, I think. Yeah, so that is everything for today's video. I hope that you did enjoy it. Please give it a thumbs up if you did enjoy it, and don't forget to subscribe if you're new to my channel. We'd love to have you with us. Thank you guys so very much for spending part of your day with me. I hope that it is a wonderful day, and I will see you guys all in my very next video. Take care of yourselves. Bye-bye.


Welcome back to another episode of likeable science here on Pink Tech Hawaii, where each week we explore more things like about science. I'm your host, Ethan Allen, and today we're going to discuss another fascinating new innovation of filtering water. Filtering water is a really critical technology for millions, hundreds of millions of people around the globe who lack access to clean water. Today joining us via Skype is Dr. Terry Denkovich, the founder of Folio Water. Terry, it's great to have you on the show. Welcome. Hi. It's good to be here. Excellent. Thanks for joining us tonight. Tonight where you are, today where we are here. So Terry, just let's just jump right on into this and tell us a little bit about how you got involved with this business of water filtration. Well, the project started for me basically around 10 years ago. I was a PhD student at McGill University. I was in the Department of Chemistry and my project was to add an antimicrobial chemical to paper in the application of a water filter, which is pretty, pretty broad. I could have gone many directions, but I started to kind of go down this point of use for developing countries route pretty quickly when I saw that there was like such a huge need of people who didn't have access to clean drinking water. Yeah, exactly. There's a real, I don't want to say market out there, but there's a real need for this in the third world in developing nations where people sometimes have to walk great distances to get water and basically don't just have taps where you can turn on and get your water right away. I see behind you, you've actually got some of your apparatus here. And I think it's a real nice demonstration here. Well, maybe while you set it up, I'll just say a few words. It's one of the key issues in a lot of places to have a very simple technology. And you can see our audience can see Terry is taking this piece of essentially a coffee filter looking like thing, folding it up very simply into a classic coffee cone shape, putting it into what looks like actually a coffee filter type apparatus, although it's slightly different. But in that this actually attaches, it has threaded ends on both ends, but it attaches to bottles. So you have a bottle of dirty water and a bottle of nothing in it that is clean. If you want to put your clean water in. Yeah, actually, this is the well. So she puts, this is presumably the muddy water or dirty water. Okay, and then, now she puts the other bottle on top, the empty bottle on the bottom, and then. And this is the exciting part, where you flip it. And bit by bit that water now is going to filter through. And the substance in your filter that's doing the antimicrobial is, of course, nanoparticles of silver, am I not correct? Yeah, that's correct. And as I understand it, and please, you know much more about this than I. Silver is quite toxic to, essentially a wide array, virtually all microbes, but not particularly toxic to all people, at low levels. Just make sure this is stable. Okay, it looks like it's pretty good. Okay. All right, sorry, I don't want to have that filter in the background. Right. It'll be pretty stable. Can you please repeat your question? So I was just saying this is, it's impregnated with silver nanoparticles, and the mechanism by which the silver is actually toxic to the microbes isn't apparently well understood as I read. Yeah, from what I understand, it hasn't been figured out the specific mechanism. There has been many observations that have been made by scientists. And some of these include disruption of the membrane structure to the cells. So it looks like there's holes that form. Sorry, this is actually forming a little bit of a pressure seal tonight. So there's a better flow. Anyway, so once the cells have little, the contents start to leak out and the silver can bind irreversibly to DNA and proteins because of the strong linkages between sulfur groups on them and silver itself, that forms a very insoluble compound that's well known in, you know, silver chemistry. OK, OK. And so you sort of, in your graduate work, you came up with this idea of putting the silver nanoparticles into the paper. And then I suspect you had to spend some significant amount of time figuring out just how to get the silver in the right concentration in the right way into the paper. Yeah, so a lot of the project during my PhD was looking at not just one, but several different methods to add silver nanoparticles to paper. Like you can think of many different ways. But so kind of a job is to kind of figure out how many different ways, which ways work the best, which ways don't work so good. So I started with, like, for example, a compound that was very reactive that had some toxic products, byproducts that worked really well. But I wasn't sure if that was good for the long run. So I started to work with nontoxic materials to make the nanoparticles. And that's what we used today. But yeah, a lot of the PhD work was figuring out some of the basic synthesis methods and then group of concept in the lab, which means you test with lab grown bacteria to see if they, you know, you hit the right concentrations of silver. So it's not too much silver in like the drinking water afterwards. You want it's OK to have a little bit of silver in, right? But if you have too much, that can be not good for people. Right. So it's like a fine balance. You've got to work there. Yeah, when I was talking with your former, I guess, one of your former professors, Jim Smith, he said the same thing with his matty drops.  Yeah. Yeah. So you started doing these and then at some stage, you must have begun to realize that this was going to be a worthwhile a product, you know, an actual rather than just being graduate work. Right. Yeah. So so basically, I graduated about almost five years ago now for my PhD. And then you mentioned Jim Smith. He was my first postdoc advisor when I was a postdoctorate researcher at the University of Virginia. And that gave me an opportunity to work with while someone who obviously knew a lot about working in developing countries. And so at that point, Jim had been to, I believe, Guatemala and South Africa. I'm not sure what other ones, but definitely those two. And he had a lot of experience with taking innovative technologies to the field as you're well familiar with. And one of the things I did in that postdoc was going to South Africa and seeing really what life could be like in some of these rural or poorer areas and seeing how well this filter worked in that setting with like water sources that people did drink from without any sort of purification. And so that was, you know, obviously very eye-opening. And basically, I saw, oh, wow, this actually could be effective, but I need to kind of, you know, alter the design and look at what makes sense for what people use like in their household. And this was not the design that we had back then. This is a modified, updated version. The version back then was a little like lab, less smaller, like more what you used to characterize something in the lab setting. So it's indeed so.  And we were discussing this the other day about sort of scaling up production, right, because these paper filters will each one will filter so much water. But at some point, you basically it's it's done. It's got a relatively limited life, right? It is paper. And so you're trying to produce, ultimately, you produce in these booklets and here's a picture. I guess we have all in now of the one that your current production machines were rolling out rather rather impressive quantities of the fully a paper. So at each stage, I'm sure as you're sort of doing this differently, the way in which you get the paper impregnated with the silver must vary.  Is that been a challenge? Well, yeah, of course. But it's something that luckily the the technology to start with worked out pretty well that each scale up hasn't been as bad as I don't know. It hasn't been something we couldn't do, I guess is the best way to put it. So the scale up first started from me going from making tiny little squares that are yay big. It's something that you could cook in a, you know, a lab oven or prepare and lab equipment, which is usually small scale up to something that would that I did actually in a commercial kitchen in the church kitchen, which use big bacon trays size of paper. And now it's just on a paper machine. So this is like our bound pack of papers, just tree crafts, but it's just I don't have any roles with me, but it's just like a big roll of paper that's this color. And this stuff could be made much, much quicker now. We made me more paper this past year than, you know, any year before that, for example, like in the tons scale, not in the, I guess, pounds, grams, orders of magnitude bigger. But yeah, the scale up there, the process has changed a little as it does. But, you know, it's still was something that even the first time that we tried it on the paper machine, we still got something that we're like, hey, that looks like it's not so bad. And, you know, it's something we could work with. And now it's even better as we've done this a few times using industrial machine equipment. Excellent, excellent.  There's lots of different approaches now to this idea of filtering and purifying water. There's this wonderful, these foliar filters that you've got. There's Jim Smith's thing, the matty drops, which of course are not a filter in any sense.  As yours does, but yours filters it as well. You deal with sort of two liter quantities at a time in yours. I was talking with a young gentleman from South Carolina of Southern California last week who and his colleagues have developed a little sort of cylindrical sort of filter that can run 380 gallons a day or something through it. And so yeah, there's all these different things that I think it goes to show. There's not a sort of one size fits all solution here. There are different people are going to need different amounts of drinking water at different times for different needs, right? Yeah, and different cultures too. Sure. And it matters if you're on your own or if it's a whole family. Are you sort of in one stable spot or are you having to carry the water with you? All these things will come into play as to what's great. And what's really impressive, I think, is these technologies are very appropriate. That is, all of them, the ones I'm seeing now are these quite low tech sorts of things where they may be based on very high tech nanotechnology, but they're very simple to use for the end user. And that's really critical because things like reverse osmosis is just way too complicated for a lot of the settings where people are most in need of the water. I can say from my own experiences that I've worked with now a couple of industrial designers. And it's not, industrial designers don't always go into the sort of market. Just as scientists don't always go into this sort of, you know, like this is a kind of, well, for lack of a better term, a low profit making driver for the most part that people do it because they're passionate about helping others. But I've met many industrial designers that have been like, oh, I don't want to make just the latest flat screen TV or, you know, the newest chair or whatever and have been interested in solving like this sort of problem. And this design came out of, it wasn't exactly what we had from the industrial designers, but it was influenced by a lot of their work and it combined with engineering designers too. Like, so, you know, this is not just me who made this. I'm not going to take credit for that, but it's taken a team to come up with this idea. And this is actually our first version. We have a new and improved version that's going to be done in a couple of months, once. Excellent. We're going to have to jump out for a short break here. Dr. Terry Denkovich from Folio Water, CEO, founder of Folio Water is on via Skype with us today on Think Tech Hawaii. I'm your host, Ethan Allen, and we'll be right back. Aloha, I'm Kaui Lucas, host of Hawaii is my mainland every Friday here on Think Tech Hawaii. I also have a blog of the same name at kauilukas.com where you can see all of my past shows. Join me this Friday and every Friday at 3 p.m. Aloha. Okay, I'm here with Brent Opegaard of the Faculty of the School of Journalism and the Department of Communications at UH Minoa. We've had a number of shows. We have a movable feast going on and we talk about journals and we talk about language. We talk about communication in general and we talk about the effect of that on the country and on individual people. Brent, it's so good to be able to discuss this with you in our movable feast. Oh, it's my pleasure. This is a great opportunity. You'll have to come back again and again. Okay, deal? That's the deal. Brent Opegaard, I'm Jay Fidel. We care about everything. Thanks. And you're back here with me, Ethan Allen, your host on Lakeable Science here on a Friday afternoon on Tech Hawaii. Joining me via Skype today is Dr. Teri Dankovich and we're talking about her new folio water filters that are silver impregnated paper filters that kill off the microbes as well as sort of physically restraining sediment and all going on through. Indiable tools for the people who don't have access to clean processed water. Before I get back into conversation, though, actually thinking about the various different technologies that are being used now to filter water and the various different approaches that people take, reminding me of a story I heard years ago about sort of multiple approaches. These scientists were training chimpanzees and trying to study the intelligence of chimpanzees and they set up a problem for this one chimp. They put the chimp in a room and hanging from the center of the ceiling was a bunch of bananas the chimp wanted. And in the room was a step ladder set up and a long pole. And they wanted to see whether the chimp would pick up the pole to knock the bananas off or would move the ladder over to climb up and get the bananas. And the scientists walked on in with the chimp and the chimp looked at the pole, looked at the step ladder, reached up, took the scientist by the hand, walked the scientist out in the middle of the room and climbed up on the scientist and took the bananas down. And I just sort of thought, in a sense, this is the same, it's a sort of parallel situation here, right? There are many ways often to get to the same goal and some creative thinking will sometimes get you there really fast. Anyhow, sorry, a little off the side there. But so, Terry, tell me this, what do these filters filter out and what don't they filter out? All right, so these filters, as we've talked about before, contain silver nanoparticles, which are lethal for lots of different types of bacteria and also some types of viruses. I personally haven't worked with viruses, but I've seen in the literature reports of a few types of viruses that silver can kill or inactivate.  And so we've tailored it for the microbiology. It can also remove dirt particles, as mentioned before. We have not tailored it to remove heavy metals or pesticides. So there might be some physical absorption to the paper, being that there is some absorption that will happen, but it's not, it doesn't have any extra chemicals to really focus in on those heavy metals. So no, it doesn't remove lead if you were wondering. Right, you know, one simple paper filter can't do everything, but it does amazing, amazing things. And about how? So you've been producing these now for, you said, for several years, and how many do you think you've probably produced at this point, distributed? Doing the numbers because we're just starting to get a track of these things as a company because we've just started to distribute over the past few months. And basically we're at about 1500 of the books format and the books depended upon where we have distributed them have had between 10 and like 26 filters. So probably about 20,000 filters for your magnitude. That's got to feel good. I mean, you know, you've helped probably 20,000 different people then avoid gastrointestinal illnesses and serious. Well, it's not necessarily one to one. It's, you know, it's a pack of filters. So it's probably closer to the, you know, 1500 number for a number of people or maybe a little bit more in 2000. Yeah, that's still it makes a real real difference in people's lives. So I had colleague get gastrointestinal illnesses traveling in the islands here and no fun at all. And so what do you do produce any educational materials with these? Or do you just sort of figure that people are going to know how to use them? No, no, we some people are some people understand very intuitively upon seeing the demo. They're like, yeah, that's easy. But we don't want people to be confused once we're not there. So this is the inside of our packet. It's basically just a picture telling you how to do it. And then down here, you can see this is just how to fold into a cone and then kind of guidance on when to change the filter and when it's OK to use. So people will feel like we do also recommend, you know, based on 100 liters. But most people don't remember how many liters they filtered. So that's not necessarily the best guidance. And on the back right now, we just have a cartoon that one of our friends made that explains the science behind it. It does have words in English, but, you know, eventually we'll get that in foreign languages as well. But it's mostly a cartoon. But this is this is kind of this is like our first product that we're selling. This is the third prototype we've had as a lot of people have heard on the news, the three couple book version that I worked with a nonprofit called Water is Life a couple of years ago. And excuse me, that that was written in English and Swahili. And I can show that to the camera too. So you can kind of see that. And this one, unfortunately, the cover is actually very expensive to me. And the filter sizes aren't optimal. They're kind of small. So we move beyond that. It's more of a first version to the newer one. And we're still learning how to best print on the paper at mass production scales. So maybe in a few months, that's what I do. So right now we're just going to print on the package in. And maybe in the future, we'll we'll get the mass production of the print insorted as well. And so you have this, you've got your website up where you sell the packets and all. What are your plans? I mean, how do you want you want to scale this up and begin to really distribute these more broadly? How do you how do you see yourself going about that? So right now we've been working with a lot of smaller nonprofits and church groups. Like, for example, we've worked with a medical clinic that is run by a doctor here in Pittsburgh and the medical clinics in Honduras in a mountainous area. And we've also worked with rotary groups that often go to various countries and do water projects. And that's like basically to start to get the word out. But we really, our goal is to move up to local distributors in country that can distribute to either pharmacies, grocery stores, corner stores, and get it so that it's a sustainable supply chain because we're recommended to change the filter. Well, it depends on how many people are using it, but every week to every couple of weeks.  One of the pushes that we're trying to make right now is into like the Dominican Republic and Haiti. And there's problems in both countries. I mean, the aftermath of Hurricane Matthew is still a problem that hasn't received as much attention since October, really, but it is still like hundreds of thousands of people who have lost everything because of that hurricane and, you know, basic necessities are really unique first. So that's one of one of our first steps for this year. And so I like really want to just trust that one and we have someone that's really focusing on that area in our company as well. It's sometimes these areas get very hard hit. We've just had one of the islands that I work on got hit by a water spout, which is very unusual. And it just went over and really destroyed a, you know, ripped the path of destruction across the island. Fortunately, it wasn't very wide, but it, and even it was in its path was pretty well wiped out. And so these people lost rainwater catchment systems and roofs off their houses and all that kind of stuff. So how will you know when you have succeeded in this business? Well, I think in the short run, we'll know when we've sold a bunch of products that if people use them right, they're going to get clean water. If we start to look at it more as a business, we'll have to look at it as a certain amount of growth per year and, you know, the whole return on investment and so on and so forth, the business metrics. But, you know, our background is more from the heart of the social enterprise side of things more than the for profit, just pure business. So we're going to look at things, how we make impact on people's lives. And that's hopefully more than just clean water, but better health, less disease. And it's a little bit harder to measure, but, you know, we're hoping to work with partners in the university to like public health doctors and people like that to help understand some of those impacts better. Sure. One of the things we're doing to look at this issue is looking at school attendance rates and seeing if when they get clean water sources, does school absenteeism drop because basically if fewer kids are sick for fewer days, you should see you should see a nice increase in school attendance. And again, that's a win-win-win, right? The kids are healthier. They're spending more time in school. They're learning more. So it's everyone's advantage. And then their parents also don't have to stay at home with them too, so they can also earn more money with their jobs too. Exactly, exactly. So what advice would you give to aspiring scientists who want to make an impact in the world just very briefly? Well, I think I got to where I was because I didn't give up. There's so many points along the way that I could have chose to do something else that would have been maybe paid better. It would have been, I don't want to have to work as much. I feel like I work a lot of hours, a lot of the days. And but I like what I do. So that's, it doesn't, it's not a negative for me. Excellent. So I think you have to believe in it. Well, thank you so much, Cherie. I've really enjoyed talking to you, learned a lot as I always do on the show.


Good afternoon, everyone. Thanks for coming for this session. My name is Ramkeer Ramakrishna, and this is Joshua Cohen. We are at Twitter with Platform Engineering. And today, we're going to talk about how you might apply some neat machine learning algorithm, what's called Bayesian optimization, to the problem of tuning performance in an automated manner in the data center. Right, so let's see. This slide's a bit light over here, but this represents the service graph of Twitter. Twitter runs on microservices. And along the edge of the circle is the set of microservices that together make up the Twitter experience. The edges over there are the edges of the service graph, so that, for example, with the magnified portion over there showing the TFE router, edges going from the TFE router to the other microservices represent the calls that that microservice might make to its downstream services. And in turn, those might make further calls to other microservices and thus, together, satisfy the request that came in at the front end. It turns out that, as you can tell from the number of services that are listed over here, there's a very large number of services that together make up Twitter. And it's about several thousand services, microservices together make up Twitter. And for each of these microservices, there might be several instances, horizontally-scaled instances, that together make up each service. And of course, these services might come in different sizes, depending on how much load they have to, how many requests they need to service. And based upon that, if you look at all of the service instances running in the data center at any time, there might be upwards of several hundred thousand such service instances running. Think of a service instance as a UNIX process, for example. Now, many of our services are actually built out of, they're on the top, the Java virtual machine. So one way of thinking of the service instance is as a JVM. There are, of course, non-JVM service instances as well, running as well, but the JVM makes up a large percentage of the services that we run. These services run on heterogeneous hardware. There could be several generations of servers in our data center. And they're all taking, our service instances might be scheduled on different kinds of hardware. So at any time, a service instance has no idea which piece of hardware it might be scheduled on. And in addition to that, depending on what the service does, it might have different resource requirements. It might require either a large amount of memory or a small memory depending on how much load it serves. It might make use of several cores or only a few cores depending on how much parallelism, how much concurrency it supports. And so the data center actually is composed of many, many different service instances running different flavors of applications. Here's a typical performance stack of any particular instance. So if I look at the stack over here, at the bottom is the hardware of the host and the kernel and OS running on top of the hardware. And in this particular case, we have two Mesos containers running on that host. And each of these containers, in turns, running a JVM process and the microservice is executing on top of the JVM. So it's a layered system. And each layer of this service stack might expose a set of parameters which you might tune for optimal performance of the microservice. Here I list, for example, H1, H2 as being the hardware parameters that might be tunable. Of course, that tuning may not be done dynamically. Maybe it's something that you set when you release the machine into the data center. The kernel might expose certain tunables. The Mesos container might have some parameters set, things such as the number of cores that the container can use, the amount of memory that the container has, the amount of network bandwidth that the container is allowed to use. And then the JVM itself has a whole lot of parameters. And many of us are familiar with tuning various things such as the heap size and various other heap shaping parameters and so forth. In order to maximize the performance of microservice. And finally, the microservice itself might be tunable in terms of various service level parameters that you set in order to change the size or the user experience. So the performance of the microservice itself can be considered a function F of the parameters lower down in the stack, which I list here as HKJMNS. And turning first to the JVM, which is what we are initially targeting, the hotspot JVM has hundreds of parameters. In this case, I'm listing a few of them over here. And if I look at the count of a version of the JVM that's about probably a year old, we find that there are about 7 and 57 tunable parameters. Now, not all these parameters actually affect performance, but there's a large percentage of them that do. So you could say that there are several hundred parameters that can be tuned, and that might affect the performance of your application. These parameters, just even focusing on the JVM, these parameters, there's lots of these parameters. And the performance of your service might be sensitive to some but not to other parameters. Well, we a priori may not know the performance sensitivity of your service to the settings of each of these parameters. Some of the parameters themselves are hardware dependent. So for example, there might be something that tweaks the JIT compiler, and the JIT compiler might emit code that might be better suited to certain CPU instructions. It might be able to make use of CPU instructions for specific hardware better than with older hardware, for example. They might also be mutual interdependency amongst the parameters. So for example, I might say that, oh, I need a heap that is this big. And then once I have fixed that, it might be that the young generation has to be tuned after you figured out what the size of the overall heap might be. So there's lots of interdependencies amongst the parameters. Clearly, hand-tuning something of the scale of Twitter with hundreds of thousands of services, each with different requirements and different performance characteristics is a difficult exercise. It's something that's impossible for any small team of engineers to do. If what you can, how you can scale down the problem is you say, OK, maybe the top few parameters that affect performance are these. And then you focus on those specific set of parameters and try to tune those manually. But even if you were to do that, something a manual exercise would be time-consuming. It would be labor-intensive and very likely error-prone, especially as the number of parameters increases. You have to keep track of what experiments you've run and try and plot a highly multi-dimensional performance surface. As a result, what happens is when new microservices are written, it is often the case that you look at some microservice that resembles one that has already been written. And so you say, hey, this set of parameters worked well for this service and my service kind of looks like the other one. And so I might be able to set my JVM's parameters like they set theirs. And as a result of this, there's a lot of cargo-culting of configurations that goes on. And perhaps you have the time to tune this cargo-culted configuration before you launch your microservice. Perhaps you don't. You find that the performance is reasonable, so I'm going to just make use of it. And you don't have time to tune it, for example. But even if you spent a lot of time tuning something like this and said that, OK, I'll pick these five parameters, tune my microservice, for these parameters, do a whole lot of experiments, maybe I spend two or three weeks performance tuning this service, by the time it's next week and the service owners start rolling out there, the next version of the service, the performance characteristics have changed. And so what was optimal two weeks ago may no longer be optimal now. In addition to that, because of the layering of our stack and because of the fact that in a large data center, we don't know which hardware instance our service might be scheduled on and because of the fact that there might be upgrades going on at various levels of the stack. So for example, I might be changing the version of Linux in my data center from one version to the next. I don't know, a priori, which of these two platforms my service will get scheduled on. So there's quite a bit of, there's a lot of churn going on in the data center, there's a lot of heterogeneity and trying to get optimal performance across the entire spectrum and making that be optimal over time is a difficult problem that no amount of manual tuning will address. And so it turns out that if we follow the manual approach, then it goes without saying that many microservices will in fact be operating below optimality. And so the question is, how can we get a handle on this apparently intractable problem and make it automated? So our approach is to actually leverage some of the literature in industrial engineering and operations research, which have looked at problems of optimizing engineering devices. So for example, you have a device that exposes a set of knobs and by tuning the knobs, you want to find the optimal operational setting for this device. It's an old problem that has been addressed in various engineering fields. And the way to do it is to say, hey, the function F, which is the performance metric that I'm trying to optimize, is a function of x1 through xn, which are the knobs that I'm going to tune to turn. And what my objective is to find a configuration A, which is the settings of x1 through xn to the values a1 through an, which is the knob settings, such that my performance metric F is maximized. But there might be constraints on this optimization problem. So for example, supposing I want to maximize throughput of my service, I might still have some constraints on the latency that the service responses might have to satisfy. So there could possibly be a constraint predicate G. There could be a bunch of constraint predicates, and I can actually represent it as the conjunction of a set of individual predicates that must be satisfied for the optimal configuration to be acceptable. So that's how we would model this performance optimization problem. And then just to give you a little bit more of a flavor of the kinds of constraints that we might want to model, I might say that the parameters x1 and x2, which are parameters to my performance function, might should be in the relationship x1 is less than x2. So that kind of constraints the set of configurations that are acceptable. And that reduces the space within which we must optimize. In the context of the JVM, that might mean, for example, that the size of the new generation or the young generation is smaller than the size of the heap. Or that, for example, there's something called the max-tenuring threshold, which determines how many GCs an object has to survive before it gets promoted into the old generation. And the JVM itself might set some limits on what the value of that setting might be. There could be more complex constraints that relate some complex function of one or two variables with another function of other variables. And then the last one, the constraints and the behavior I've already talked about a little bit before. But further making the problem more difficult is that there are sets of parameters that affect the performance of my device. In other words, the function f, which we may have no control over. So for example, my service doesn't know which other service or which other container it might be co-hosted with on a specific host. And they might be intercontainer crosstalk that I don't have any explicit control over. There might be variation in the load across time, which I may not be able to control. And it is possible that the optimal settings for one load are different from the optimal settings for another load. And then I've already talked about not necessarily having control over the hardware where I'll be scheduled. And so when there is uncertainty in these hidden parameters, that appears as noise in your objective function. So the performance tuning exercise itself would consist of designing a suitable performance metric and then deciding on and refining the set of knobs to tune, and then using an iterative strategy to tune these knobs. And of course, not all knobs are visible to us. So pictorially, that might look like this. The performance engineer picks a set of knobs and picks settings for those knobs and then starts up the system that he's trying to optimize. He makes some performance measurements of the metric f that he's interested in optimizing. And he makes a couple of these measurements with different settings and tries to figure out the shape of the performance surface. But the question is, can we automate this by having a black box tuning assistant, something that will try different parameters, look at the shape of the function, and maybe learn what the function shape might be like. And for complex systems, the shape might be extremely complex. It might be a highly multimodal, non-linear system, as we know most of our systems are. We make use of a technique called Bayesian optimization from the machine learning literature. It's a statistical technique that can be used to learn potentially noisy objective functions. And we've already talked about how we might have noise in our system, iteratively and efficiently. And by efficiently, what I mean here is that through a small number of evaluations or experiments of the response surface, we can quickly find our way to an optimum configuration. And that's what we are saying here, that in a very few small number of iterations, we should be able to find an optimum setting for our parameters. This technique has existed for a while. It was first talked about in the 80s by Marcus in the former Soviet Union. And then there's been more work recently. And we are going to be leveraging work that was done at Harvard in Toronto a couple of years ago, which we have in-house today. And this technique works well with non-linear, multimodal, and highly dimensional functions. So it won't get caught on local maxima. Let me walk through a quick example to show how Bayesian optimization works in practice. Supposing we have taken three measurements of the system, and those are the three. So I'm here looking at a single variable system, performance shown on the y-axis over there, and the knob settings shown in the x-axis over here. And we've had three evaluations that give us that set of points. We might look at this and say that maybe it's worthwhile to look at something that's of setting the value at minus 4 because presumably that setting might result in higher performance. Now, it turns out, and we do not know this, it turns out that the actual function might have this shape, which would still produce the same set of settings. So I'll walk through an example where Bayesian optimization will try and learn the shape of the surface as we go through. So the way Bayesian optimization learns is that it models the function, the unknown function, as a stochastic process. It's what's called a Gaussian process. And this is a distribution, it's a probability distribution over functions. One way to kind of think about this is that at any point of the space, of the settings of the input parameter, there's a distribution of values that the function might take. So for example, if I pick the setting two, we have some uncertainty as to where the value of F2 would be. And so that uncertainty is represented by this cloud, which is a normal distribution with a given mean. And as we make more measurements, this probability distribution will get modified. So we start with a certain prior distribution and as we make observations, it gives us a posterior distribution and we thus iteratively define and go forward. So the way we figure out what the next point at which we should look to make a measurement is that think of the best point that we've got so far, shown over there at the top. And we slice the probability distribution and look at the mass of the probability distribution that's above it. And so if I take any specific point over here and think of the probability of improvement, it would be the integral of the probability distribution that lies above the curve. And that's true of each point. Now I could use the probability of improvement as something that would drive the choice of the next point. It turns out that people have looked at this and found that expected improvement might be a better measure. So for example, I might say that the value of the improvement times the probability integrated over the slice that the part that lies above it would give us a much better measure. So here we've plotted the expected improvement of this Gaussian process given the best value that we have so far. And then it turns out that it's mathematically easy to optimize this function, which is called a surrogate function, the acquisition function. And so it turns out that this point would maximize the expected improvement. And so we pick that point and proceed from there. We make a measurement over there and we find that it actually doesn't improve our performance much. But we keep doing this at every point, refining the probability distribution, the Gaussian process that we have. And over a couple of such iterations, every time picking the best point that the optimization of the expected improvement acquisition function gives us, we eventually find our way to the, to somewhere near the optimum. One thing that I want you to note over here is that there are points at which we actually pick up and do and evaluate the function at points where it actually performs much worse than the best point that we had so far. There are several alternate approaches. I won't go into the details of this because I'll probably run out of time. But a Bayesian optimization has been used at Twitter for a number of optimization problems. I list a few of them over here and we are applying it now to JVM Performance Tuning in the data center. Note that this kind of technique could be applied at pretty much every, any layer of the stack. We are now focusing mainly on the JVM and so that's all that I'm going to focus on now. But there's no reason why the same technique would not extend and be applicable to almost anything in the performance stack, including, for example, the Mesos, the container level which somehow got skipped in this slide. So I'll actually very quickly go through a proof of concept that we did where we picked a bunch of JVM parameters, some of which I've listed over here for a total of about 30 parameters. So taking these 30 parameters, we ran Bayesian optimization over it. And the setup was of a large production service to which we applied this. But we didn't apply it in production, we applied it in a staging environment. And the performance metric that we were optimizing for is listed over here. It's the number of requests per second divided by the GC cost. And so intuitively we want to increase the throughput of the system and we want to reduce the garbage collection cost. And intuitively what that means is the latency of your requests would be minimized. Wild throughput is increased. This shows how the, you know, on the X axis is the iteration number and on the Y axis is the relative performance improvement with respect to the original settings that the service was running with. And we can see that by about the, probably the 20th or 25th iteration, even though we are trying to tune a system with 30 parameters, each of which might take up to 100 different values for a huge space, like 100 to the power of 30, even though we were searching over such a huge space and this could possibly be a highly nonlinear multimodal system. Within 20 iterations we had found our way to something that doubled the performance of the service. And by the 70, whatever 78th iteration we had performance that was 2.2 of the original performance. So that's the optimum over there. And I won't go into the details, but basically this shows the metric as, you know, over time. The blue line represents the tuned system, the optimized system and the yellow line represents the unoptimized system. And looking at the two parts of the optimization function, one was the requests per second, that was the incoming ambient traffic that is the same for both the optimized system and the nonoptimized system, and the improvement was entirely because GC cost was reduced dramatically. I won't actually go into the optimized settings. I'll let you look over them in the slides that we have. But basically what one can take away from this is that if we can optimize, if this optimization carried over well to production, then we might be able to extract a data center footprint reduction and therefore a cost improvement in the data center. And so that's really what we are trying to get at. The key takeaways is that you have to be aware of hardware heterogeneity. You have to factor out hardware effects when you're doing these experiments. There might be load spikes and seasonality and you have to factor those out by running sufficiently long experiments that will average that out. You have to have baseline configurations so that you can normalize your evaluations. This is something that I kind of mentioned a little bit before. You have to run experiments long enough that long range effects appear. So if you don't run your experiments sufficiently long, then the long range effects may not be visible and what you thought was an optimum turns out not to be an optimum after you've run for about a day or more. In order to increase convergence speed, we want to reduce the amount of noise as well as to increase the quality of the optimum we want to make it robust with respect to noise. We can actually, the technique that we are using can in fact be parallelized so you could have several experiments running concurrently to speed up the convergence. And finally, suboptimal suggestions such as the one that I pointed out earlier, you need to be able to detect those and stop them early so that you don't affect the overall quality of service especially if you're running the experiment in production. And the last bullet basically says that staging is not production. If you optimize something in staging, as we found out the hardware, it doesn't translate automatically to production so it makes sense to run these evaluation experiments in production. So I'll actually let Joshua now take over the talk and talk about an implementation of this that he's been doing. Thanks, Romkey. So yeah, so based on what we learned from the original prototype that was built, we've started building a system that we call AutoTune. It's a Bayesian optimization automated tuning service. If you put that all together, it spells out boat. So we like to hit these memes at the peak of their popularity. We can also just call it AutoTune as a service. We take what we learned from our prototype and apply it to a general service that anyone can use. The key goal is that no coding is required. It should just be configuration. It should support any type of service and it should be running continuously or on demand. And yeah, it should just basically, as you would want from any service, be really easy for our engineers and SREs to run. So this is an example of what an AutoTune configuration file looks like. Essentially, you give it some parameters that you want to optimize for. I'm just showing one here for the sake of space, but it could be any number of parameters. You give it an objective query, which is essentially the function that we're trying to optimize. A job key to identify this in our Aurora Mesos cluster. The range of instances you want to experiment on and how long each experimental evaluation should run for. This is just a brief systems diagram, which isn't super interesting. So let's talk about how Aurora and Mesos actually help us to build this AutoTune service. I assume everyone's familiar with Mesos. For those who are not super familiar with Aurora, it's a Mesos scheduler. Initially developed on Twitter, later open sourced. Patchy project was designed for microservices. So what does Aurora bring to the table that lets us build this system? It's got homogeneous jobs, instance level scheduling constraints, programmatic access to the API, fault tolerance automated deploys. These are all things that make it easy to build our AutoTune service on top of Aurora and Mesos. Before we delve any deeper, just a brief primer on how jobs are configured and executed by Aurora. Jobs are a top level construct in Aurora that composed of n identical tasks. Tasks themselves are composed of processes. Individual task execution is managed by Aurora's executor, which is called Thermis. Aurora itself is responsible for tasks being resilient to any failures, either of the tasks themselves or host failures. And Aurora and Thermis together facilitate service discovery. So other services in the data center can find those and talk to them, which will be important when we want to send them production traffic. Aurora also provides a thrift RPC API that lets us have full control over anything we need to do when scheduling our experimental instances. So this is sort of the overview of the process that we need to take when we go to launch experiments Aurora. By the time we're talking to Aurora, we already have our suggestions from Twitter's Bayes-Op service. So effectively we need to find the primary task config from Aurora, inject our suggestions from the Bayes-Op service, make sure that we're consistently running it on the same hardware platform, and then finally pick an instance to run it on. Ideally Aurora jobs are homogenous, but during Canaries and Updates they might not be. So we need to pick a task config that identifies sort of the base level task config that we can make all of our experimental changes to. We use a simple heuristic for this, which is just picking the most common task config among all instances, but if we needed to we could do that differently. So I'm talking a little bit about task configs. This is what an abridged Aurora task config looks like. The key things to notice here on top of the obvious, like these are the resources to use, are it allows specifying task level constraints. It's got some data for Aurora's executor and it allows us to specify metadata. And so this is what Aurora Thermos's executor config looks like. We have a list of processes and there's constraints among those processes that define the order in which they should run. So this constraint here says that the stage process must run to completion before the run process completes. You might be asking, so if we look back here, we have, you can see in the command line, there's a little bit of auto tune specific stuff here. So you might be asking yourself, how can we guarantee that that's all in there? For the sake of time, we have a helper built into our Aurora install at Twitter that all JVM processes run through, which gives us a convenient point to hook into to add those necessary flags. If you'll recall from that command line, we are looking for this auto tune config file and effectively what we do in the auto tune service is inject a new process that generates this file for us, which effectively exports a bash variable that contains all of those command line flags that we want to pass to the experimental instance. But the process by itself is not enough because we need to make sure that that runs before all of our other processes. Otherwise, we won't know that they've picked up those settings, so we also inject some constraints that guarantees that our process runs first. As Romkey mentioned, one of the lessons we learned from our prototype is that hardware differences definitely impacted the optimal settings. So thankfully Aurora allows for us to make scheduling decisions based on mesos attributes. So we just configure our mesos attributes with our mesos agent with an attribute specifying the hardware platform, and then Aurora can schedule on that constraint. The last thing we need to do before we can launch an experiment is pick an instance to run it on. We want to make sure that we're not clobbering an existing experiment, so we inject a little bit of metadata so we can tell our auto tune experiments from normal instances of a production service. So we finally have modified Aurora task config. This is where Aurora really helps us out because of its built-in job update support. We just initiated an Aurora job update with this task config. If the suggestions are bad, Aurora automatically rolls that update back. Twitter doesn't go down as a result of this because we are experimenting on production instances of these services. Auto tune then detects the rollback and updates our base op service to let it know that those settings were no good. Once the experiment is running, Aurora's built-in support for service discovery means that this instance is taking real production traffic just like any other instance of the service. If the suggestions are bad and the service fails, Aurora will restart it, will detect if there's too many restarts and market is bad. If we're also monitoring metrics, while these are running, if the metrics are bad, we market is bad again. So finally, experiments will run for whatever the service owner duration was specified as. Once the evaluation is complete, we check the objective query, feed those results back into the base op service for the next round of experiments and repeat the thing over and over again until, as Romkey demonstrated, we eventually find the global optimum. So eventually, experiments will convert to an optimal setting and this part is currently manual in that SREs will take those settings and apply them to the service, but hopefully one day it will just be automatic and we'll just keep doing this over and over again, continuing to feed those settings back into production services. And so looking back at the diagram that we saw earlier, we replaced this black box tuning assistant with AutoTune. We replaced the performance engineer with the base op service. We have Twitter's existing monitoring infrastructure and it just keeps running and running. So this is great for the JVM, but what can AutoTune do for us outside of the JVM? One area that I'm super excited to explore is automatically sizing Aurora instances. So instead of engineers having to figure out that I need four CPUs and 12 gigs of RAM, AutoTune will just figure it out for you. So in conclusion, given the scale of the problem and the possible gains, it seems clear to us that some form of automation is desirable in this space. Like we do for QA, continuous performance optimization appears inevitable for efficient operation of microservices. Base opt, because it drastically reduces the cost of search for an optimum, appears well suited to the, as the technical basis for this kind of work. Yeah. And yeah, so at Twitter, we think that because of the current state of, we believe that the current state of the art and containerization combined with commoditization of machine learning technologies opens up new frontiers and operations, scalability and performance engineering. Things that were previously unautomatable are now automatable. Our work on AutoTune focuses on a small piece of the stack and uses a tiny subset of what's available in machine learning today. As you move your platform and your infrastructure onto services like Mesos, we encourage everyone to examine what opportunities for automation and optimization are available to you. And in the negative 15 seconds I've got less, I'm happy to answer any questions. Not at this time. I'm right. So yeah, so I mean the underlying machine learning, the Bayesian optimization system is available. Open source, it's called Spearmint. So if you look on GitHub, you can find that. But the orchestration piece that is AutoTune itself, maybe one day, but at this, right.


Hey, welcome to Sunday morning coffee. Oh gee style. Yeah, we're gonna do an audio today because why because I woke up way late and That was crazy. I'm really glad my body got to sleep so much. However, my brain had things to do I'm surprised it didn't wake up my body. I got a family commitment today a couple of client sessions and I Need to get moving. I really really want to connect with you though. I Know how important consistency is you find a rhythm and When that rhythm gets disrupted It's a little it's not well, it's disappointing, but it also kind of It can set a ripple into your week or your day in a way that's maybe not Not as positive as it could be it could be a disruption So I'm I really do value a consistency rhythm and showing up you guys know that I really value that so Although there's this time of massive change and shift for me in my life and each and every one of you has your own things I know You guys have a lot going on. They'll just stop for a second. Can we just stop from it and talk about you? Let's talk about you for a minute. Okay.    I think I'm going nuts all this stuff But the fact that you have the wherewithal to even acknowledge The weight that you carry says a lot a Lot about how you show up. I'm not impressed by how much you're doing. I am inspired by How much you claim awareness for how the things in your life are impacting you and Where you can when you can you make choices?   Oh My friend, I'm just gonna tell you that is called stress that is stress and Despite all the stress loads that you have and the pulls and the tugs at your energy Not even the human stuff, but the energetic stuff that that is like working its way under your skin Like you're handling it You are managing this life And the I know some days are not so graceful.   I mean how many times did you run up and down the stairs because you forgot what you went upstairs for? Okay, okay, that's not just like an old person thing or a hormonal thing That is a stress Thing yeah, it means you have a lot of energy That is moving through you and around you and yeah, my friend you are affected by that you are You are so just let's just stop for a minute. Let's have a nice drink coffee here. Just a minute And let's just feel into you The incredible awesomeness of you because just showing up Just showing up is the reward. You are the prize You're the grand prize, you know why Because when you show up over and over and over again, I know sometimes I know I know doesn't make sense every day Things don't make sense. They're not supposed to because guess what? You're not just ruled by your brain You are a beautiful brilliant intuitive body a gorgeous and shining spirit and An unbelievably unfolding expanded heart So it's you know, there's a reason why things don't make sense because they're not supposed to Stop wasting your energy and your time trying to figure it out It's not time for that. It's time to just feel into you right now and just say hey, I'm doing it I'm showing up and it might not be in action You might not be taking these bold steps or doing these bold things, but you know what? Just getting out of bed is a bold move these days. Have you felt the energetics of the world? Yes, and what about the cosmos and the universe and the astrology because you are made of stardust and because of that You sparkle in different ways that sometimes your body doesn't doesn't really make sense of So let's just take a minute of gratitude for you Okay, cheers to you for doing this thing this life thing. Okay, and and then there's a purpose for it It's not just a mundane Hamster wheel don't feel Like you aren't doing it right or you are not doing enough or you're not doing I'm telling you how to feel Don't you love it when people do that doesn't just really work for you when somebody says oh don't feel like that Oh, yes. Oh, I'm sorry. You're such a genius and a brilliant Consultant for me and wise one. Oh, you know more about me than I do. So yeah. Oh my god. You're so right. I Will just Not be that way.  You may have just been super productive or you might feel kind of sad because maybe maybe you Just got some sad news or or some of you might you might have some worry feeling.  You are the light in the room. Did you hear me? Did you hear that you are not sucking the air out of the room? You are not making people feel bad and making the room in a bad mood and remember Oh if you had that much influence, can you imagine how great the world would be because you could be just shining your awesomeness And everybody would just be magically responding in awesomeness back Doesn't happen that way doesn't You are not sucking the air out of the room. You are the light in the room You are the light in the center of the universe and I feel you right now And I'm just gonna tap on your little belly. Do you mind if I do that? Let's just tap our bellies I know I know The first thing that might come through is judgment or I might feel odd to actually touch your body If it does feel odd to touch your body like your stomach or your heart. Oh my gosh start just doing that because Your body misses you your body needs you your body deserves appreciation Your body is not the bad guy or the bad girl or the bad one that makes your life hard Even if you do struggle with illness Or dis ease Your body is not the bad guy. Your body is extraordinarily intuitive and trying to hold you as a sacred container That's what your body does. So touch it.  It's hard, but squishy bellies are beautiful Our bellies are so soft our bodies can be so soft and squishy and I think that's beautiful this beautiful It's beautiful you are beautiful in this now moment because you showed up because that's what you do You are present for life Giving yourself every opportunity to feel Feel your squishy belly How come how can we like rubber dogs bellies or our cats bellies or a little baby belly rolly skin and be like Oh, you're so squishy and so cute, but when it comes to touching our bellies, it's kind of like It's not flat. Well, of course. That's not flat my dear first. You're sitting down And you're all hunched over and two Uh bodies are not meant to be flat We're not Barbies I saw the movie. Yes, we should do a whole Sunday morning coffee about that and maybe we will We're not plastic We're people we're human and we change our bodies change And that's on point It's supposed to change and oh my god, let me just tell you I am so hormonally in menopause and The way the body all of a sudden literally you wake up one day and your body is so different I'm like, wait, what just happened here? I got all this skin like hanging out around my belly. I'm like, wait What And oh my goodness, there's so many there's so many different things I can't even begin to explain because that's not what this podcast is about and you're probably not that interested in it In this moment, however, some of you can relate to me the body changes And it reminds us Of all the seasoning we've had all the marinating we've done all this lifetime thus far this lifetime Forget about the past lifetimes. What about this lifetime girlfriend? You have shown up over and over For the first marriage for the second marriage for the third marriage for the for okay. No judgment. No judgment For the kid for the graduation for the childbirth for the neighbor for the best friend for the moving to one state Moving to another state for the starting the new career for the stopping the old career for the New education for the new class you're taking for the new Book you're reading for the new the all of it. There's all these things that have contributed to the beautiful seasoned Essence that you are you are so experienced.   Don't you just go? Oh, yes. Yes. Yes.   You are the reason I got out of bed this morning. I jumped out of bed and was like Oh, I gotta do sunday morning coffee And had I not put the eye patches under my eyes I might actually I might actually be doing this on a video You know And over time I'll figure it out. I'll find a rhythm because of uh You know on the townhouse here. We've got kind of unstable internet sometimes and it just I'll figure things out Just like you have just like you are Just like you are you I cannot I am not I cannot tell you more from my heart from my truth from my center core how much you impress me and inspire me Honestly This is my truth You are you are so so inspiring to me Thank you for showing up Thank you for being you in the unis that you are today Thank you for showing up Thank you for being here.  I actually don't even have it written on anything. I should I didn't come this far to only come this far Hey I appreciate you. I see you. I feel you In your unique shining light that you are you contribute so much to this world And I know because I feel it every week On above life channel. I feel it every week. You have no idea how much you are That kindling for my fire my passions my life purpose You do make a huge difference you make an impact Thank you Thank you And with that nurse squishy bellies It's like we're caribers Oh Above life channel sunday morning coffee listeners unite Light comes out our belly like caribers anybody know that reference? Youngsters you're gonna have to google it google the caribers caribers stare. Okay Now i'm getting goofy. I got to finish my coffee and get about my day take off these puffy eyepatch things Thank you to my friend who recommended them Eyes my eyes are so puffy here in the morning. So i'm like oh, you know puffy eyepatch It's the start of the week. It's the start of the day. I look forward to uh connecting with you again I hope I've inspired your spirit and filled you with hope today Alexa turn off I think it's Alexa that's setting alarm Oh Yeah, I think so. Oh my gosh. I have to get used to this Thank you so much for being here. I hope I've inspired your spirit Encouraged you to live your life. This is your life after all and you are living it Just show up and live it. That's how we do it. Thanks for being here