Hello, who's managing your career? Carrier can be a noun or a verb. If you think your manager is managing your career, think again. If you imagine just doing great work will boost your career, think again. If you feel the performance management system in your company or wherever you work is so aligned with how you're doing, and things will naturally automatically follow through and help with your career, think again. So what makes career management challenging? What are some ways to lean in and proactively plan your career? Let's take a look at it through these different topics, which I call the six C's. We'll look at context, which are on myths. We'll see some challenges about technical roles, primarily a topic that we want to focus our conversation around, which would be, then again, around growth and success and what it means. And the central part of today's theme and topic would be around form, which is around cultivating networks. And we will close with these interesting paradigm that I can surface to you, called ninja, spiders, and forests. And you might be wondering, what on earth do those things mean? And then, of course, we will have some time for clarifying some questions that might have come my way as part of helping prepare for this topic. Let's have a chat, shall we? First, let's talk about myths and paradigms about what career means and what career opportunities are. Now, the top portion of these slides are some of the common myths or popular myths that you might have heard of. The first one is about we have to strategize. We have to play the game in the locations where we work, play, and operate. But I'd love to tell you that it's not just about strategizing and playing the game anymore, but it's about understanding the culture and thriving in it. That is what is most important about building a career. The other myths that you would have heard and you would have seen, even if you go to your favorite search engine and type, it's always about ladders. Carriers are about laddering up, following the crowd. But it's not. It's about climbing. Yes, but climbing in all different ways and doing it in your way. That is what carriers are all about. The third one, which is a very even more common myth, is about career is being a path. You always hear the term career path. There are specific milestones, there are pit stops, there are speed limits, and so on. But it's not. It is about setting course and navigating to your favorite destinations. So carriers are about understanding the culture and thriving in it. It's about climbing and doing it your way. It's about setting course and navigating to your favorite destinations. Let's take a look at career growth. Some of the key paradigms for career growths are it's all dependent on your manager or by managing other. But it's not.  That is how career growth happens. The second one, which is another common thing that I've heard, among so many people that I talk to and mentor, is about career growth. It's about doing a lot of things all the time. It's not. It is also by doing what you love. And it's not cliche to say it's what you love and loving what you do. But it's true. It's important. That is a central part of how career growth happens. You've heard these other terms about career growth being navigating jungles, handling turbulence. Well, I'd love to tell you that it is back to fundamentals and foundations. It is about investing time, driving impact and bringing people along. You might have heard these things many times before, but I'd urge you to reflect on these things.  Okay? Let's take a peek at, you know, I told you that we're gonna talk more about technology and technology roles. Let's take a look at some of the challenges and all the different options by which career growth can happen within technology roles. Let's take a look at some of the roles. It's a very simplified view, but provides an interesting perspective on how some of our roles can be applied. On the X axis, you have technical skills. And obviously all of us have some form of technical skills. It could be a practical skill, all the way to an expert technical skill and you can define what these terms mean, but you get the point, right? There is a gradation of different levels of technical skills that each one of us have. And on the Y axis, let's talk about product skills. Product is foundational and fundamental regardless of the type of roles that we all play in the technology world. You can have an operational model of product or you can have a strategic thinking about what product management is. Okay? There is a Z axis which is around domain skills and we'll keep that as whatever the domains can be for now. And the four grids that emerge from a two by two like this or on a graph like this is on the X axis, you can be very strong in project process management and you can get into more and more engineering management based on growing on the X axis. On the Y axis, you can still be project to project process management which is very fundamental and foundational and you can start getting really good at product management. But the place that I would like to focus our conversation on is this amazing space called technical program management or product TPM as it's called in some companies. And that area is also a very interesting fascinating space because it blends process, product and engineering management into one space for technical program management. So why am I talking about all these things to you? I wanted to give you a flavor, one framework about how we think about technical roles and how we can be applied to the type of career growth that we want. So let's take a peek at a few options of career growth. So on the extreme left, you see that the same set of four boxes that we talked about before and you can try that particular combination, go all around those particular roles, you can rinse and repeat, but then change it by domain. You could be in health care today, you could be in climate change tomorrow, you could be in any different forms of domain that you want and you could try process, product, technical program management, engineering management that way. You can grow your career that way. Boredly rotate roles across domains, the first one. The second option for career growth can be how do you create, craft, and cultivate your brand? Who it is that you are? Who do you want to be? Now, when you take a look at the technical program management that area that I specialize in and I work a lot in is you could be a fixer of things, you could be a builder, you could be a connector, or you could be a scaler. I could be great at scaling large scale systems. I could help fix things that are broken or I could help create and build a new thing. So you find your brand and you start developing your persona and then you can have career growth that way. You can change each of these particular roles or you can go deep in any one particular role and become the best known scaler for technical program management. That's another way to grow your career. The last option which I also wanted to surface with you is bread versus depth, single versus multiple. What does this really mean? Every one of us has a style, every one of us has an intention of our repertoire. And so the goal is how can we learn to be good at one and then perform and master and then get better at it over time or then switch over. So I can be a master of one program or I can take care of multiple programs that I want to run. I can go deep into one domain, one specific set of systems and keep doing really great there or I can go bread twice. So technology roles and career options. So think about these as you're thinking about career growth as well. So we talked a little bit about the myths and paradigms. We talked about the different career options and so on. So now why don't we have a conversation about what growth means and what success should be defined as or how should we think about it in that particular way? Now what does growth mean? So let's take a parallel. I'm fascinated by trees and this whole notion about arboreal aspects. So what does a tree actually need for a tree to really grow? Like if you think of career growth in parallel to how a tree grows, a tree or some kind of vegetation requires these three core engines, core elements. Sun, sunlight, water and nutrients. That helps this arboreal nature of growth. So similarly, there's a very strong parallel to what happens at work. So you need three things. You need sponsors, somebody who knows the space and investing in it, somebody who knows you and investing in you. The spaces, the problem spaces that are very important and that's a true form of nourishment that's required for us as humans, for us as individuals to operate in and the necessary skills and attributes just like nutrients that are required for us to actually grow. So remember that sponsors, spaces and skills are very important and you need to find it through your mentoring circle, through your manager and through your own self-reflection on how you can develop and grow those. So what does success demand? It's a very complicated workplace today, as you know. We all have to deal with so many different aspects to it but I hope this will give you a summarized or a TLDR view of what success means in today's workplace. So there are these bunch of boxes. Let's start from the first one where I have a star that says know your potential. I think that's a very important thing. There is a time and place for all of us to reflect and know who we are and what our potential is. Then you find your purpose. I think it's the second and most important thing here. If you don't know what your purpose is, if you don't know how to seek it, then you can't develop your skills. And if you can't develop your skills, you can't induce passion, which is very important.  Once you do that, then the rest of the things will begin to follow more logically that you will land impact and you will not only grow yourself, but you will grow heroes around you. People who kind of rally with you and that helps you build and cultivate your network. Once you cultivate your network, the next few things are very fascinating. You will find people starting to become very happy. You will start having a much more cohesive sort of people around you. And then comes the time, a very important time of not being quiet here when it comes to career growth, but being able to narrate your story or narrate the story that you've been able to create. And then you start again tracking and watching the trends. That's what I call friend the trends aspect. So this is the cycle of what success demands in today's world. The more you do that, the more you consciously are aware of and take control of it, the better your performance will be. Know your potential, find your purpose, develop your skills, infuse your passion. So that's one phase. Once you do that, you can turbo boost your core. You land impact. You grow a group of people who are all going to be heroes. It forms your network. People are happy. Then you take time to tell the story about what actually happened.  So let's take a look at this, the network part, which is another fascinating piece that I really wanted to talk to you about. So what does networks mean? All of us know this notion. Some of us go to business school and say, I got to develop my network. LinkedIn is a crazy place for developing networks, right? Crazy in a sense in a positive way and also in sometimes in an organic way that you want to build your networks. But what is it actually, right? So let's take a peek at how we ship music and then we see how we ship products. So let's focus on the left side. Now for a successful production of a music album or a performance, it requires an ecosystem of players that have to get involved in it, right? You need to have a terrific orchestra. You need to have some engaged producers. You need to have some invested advisors and then you have some passionate followers and you have some really, really fond fans of yours. In order for all of these things to happen, then you ship a great label, okay? So this is a paramount importance. This is a paramount importance in order for this ecosystem to actually work and that is the cultivation of the network, a successful music producer or a successful album will have to do to ship music. So let's take a look at it in the technology world of shipping products because we are after all talking in product school and it's very important to know are there parallels? And of course there are. Just take a look at how fascinatingly similar these things are. Now, if you want to ship a great product, right? You obviously need to have a great product team and which is sort of an aspirational thing for a lot of us is how do you become a great product manager? How do you become very good at product sense? And you need to have invested sponsors. You need to have some engaged mentors, very similar to the advisors in the music side, right? And what about teammates? You have to have and we depend on a terrific teammates and committed partners, right? Partners who all of us have to deal with either internally inside the company or outside. So a similar ecosystem is absolutely important in order to cultivate and create and ship products, okay? So just as if we ship music, it's equally important how we think about shipping products as well. So let's conclude. I told you that we're going to talk about some fun things about warriors and spiders and Mike or Rizal for us. Are carriers determined by these three things? Remember we started our conversation of this presentation today that our managers, the ones that determine your carriers. The answer is yes. But managers are coaches. And remember what coaches are and who they are. They're extremely important people in our lives. They see in us what we can't see ourselves, right? And this is exactly a quote from Pep who, if you're a football fan, a Man City coach who talks about it that way. But what is more important than managers are to be on a great team, whatever the team means to you because as you know, a great team achieves more and more boosts camaraderie. And that's a very, very important part of our careers and determined today. So it's not just about managers who are about coaches but being on a great team. So you have to be on teams like Man City. It's not just by having Pep as your manager but by being on a part of a great team. The second one is our carriers determined we ask this question just by great work. Absolutely, yes. There is no substitute to hard work. As Edison would have talked about and some of you might have known the quote. But also we have to make sure that we all have that purpose, that passion that we skill it with. And then we bring it home with solid performance. So purpose, passion and performance is as important in addition to just hard work, okay? Performance review systems. Every one of us has it in every single company that we work for. Yes, as Drucker said, the famous management thinker, if you can't measure it, you can't manage it, right? So there has to be a way that we have to work within the performance management system for carriers. However, what is more important as I called out very early in the presentation was understanding the culture, knowing what actually matters and making sure that the key metrics in that culture that you work with, in the realm of the project and the impact that you actually do, how it actually matters, how it actually matters to move those metrics, okay? So managers plus team, hard work plus passion, purpose and performance. Yes, performance review system, but also understanding the culture, knowing what to ship and how it moves the metrics, okay? So if you have to still want some simplified view about thinking about carrier management, think about these next three things that I'm gonna give you, I'll leave behind. One, don't think about carrier ladders anymore. It's ancient, okay? It is about ninja warriors. It's about ziplines. It's about being in these gyms. It's about all forms of growth can happen. It's no longer just laddering up or down, okay? Next, do not think about isolated actions. So just as if you're working on individual work and independent affiliation with the manager is not the solution anymore, right? You have to build and work over time with a part of people, with the network of peers, mentors, well-wishers or even your family. And that is when the network will work for you, okay? So don't think about yourself as an individual entity trying an individual relationship with somebody, but it is about a microcosm of people that you would have to go along with. The third, try please not just being the tree. It's no longer important to become the best tree that you ever can or the stronger tree, the taller tree or the most powerful tree, but you have to become integral to what is called the Maikori Jal forest. A Maikori Jal forest is a fascinating place because it is how all the trees communicate with each other. Trees talk to each other, trees warn each other in times of distress and trees nourish each other and provide signals. And so Maikori Jal is a type of fungi that is known to communicate with each other. I'm just drawing analogy or parallels to it because that is what career growth happens. So don't think about just being a tree but think about helping your forest develop and your forest become as successful as yourself. And that is when true things happen. So if you like so far on what we talked about and you're wondering who on earth am I? So my name is Kaushik, Kaushik Sethuraman. And that's my LinkedIn profile. I am a technical program manager at Metta, formerly known as Facebook. And if you like what you do, I'd love for you to consider talking to me and talking to us as a company. And some of the roles that we have on our team are called the technical program management roles or technical product management roles. So feel free to give us a shout out and take a look at us.  If you have questions about these topics discussed today ping me on LinkedIn or other channels, I'd be happy to answer those. If you, again, to reiterate, if you like what we learned today and have an interest in technical roles at Metta, please do consider the Metta TPM roles and other open positions. Or not just at Metta, but these type of roles of interest to you, I'm sure these things would be a fascinating place for you to look at in the industry. And remember how we started this presentation. Who's managing your career? Well, by now you perhaps arrived at this insight. It's you. You manage your career. Remember, it is after all your tree, it's your forest, and it's your career. Our tiny, quirkus rubus, sapling that you saw, has now grown into a full, big, large, strong oak tree. And if you remember, our oak tree now nourishes the mycorrhizal forest that it now lives in. Thank you.


Hi everyone. Sorry for the late start. I couldn't get into the meeting before. My name is Sean McGiven. I'm the discussion back indeed at GitLab. Just share my screen with the update of what my team and I have been doing for the last five weeks and what we'll be doing in the next five weeks. So in 10.0, we shipped some stuff, which is awesome. 10.0 being released on the 22nd of September, so a week and a bit ago. We shipped group issue boards, which fit from the previous release, which is awesome. I think everybody at GitLab who does any kind of planning or managing issues across a release really needed this, and it's awesome. So thanks to Felipe and Simon for that. We also have Jira Development Panel Integration for Commits. So as well as I was working on that, we hope to add other types of things that we can integrate into the Jira Development Panel in the future. But that's an awesome start. And Mikhail had worked on this before he joined GitLab and he finished it up in 10.0. That's a first-time contributor badge. So when you submit your first merge request to a project, you get a little badge that lets people on the project know that this is your first merge request and to be even nicer than you would be otherwise. We also have some community contributions on that subject. So one was to filter issues or MRs by your emoji reaction. So if you've added a star emoji to a bunch of issues, for instance, that you want to bookmark or favorite, you can filter by those easily now. We added a setting to when you push to a merge request, at the moment we follow the diff through for each comment on the diff and try and match it to align in the new diff. If it's outdated, we show it as outdated, but we don't automatically resolve the discussion. We make you do that manually because it's possible it just moved, but it still needs to be addressed. This setting lets you go to the other behavior of just resolving it if it no longer matches the new diff. We added a move quick action. So quick actions are etype slash and then move in the comment box and you can move an issue, which is super cool. And we got some time tracking information in the CSV export for issues, which is cool for people who like CSV exports. In terms of the team in general, most of them were at conferences this last weekend. So Yaka and Douglas, among other people from Get Lab were at Yiruku in Hungary. And Felipe on Oswaldo among others from Get Lab were at the conf in Brazil. So I've spoken to Yaka since then and it sounds like she had a great time and I'm sure Felipe Oswaldo and Douglas and everyone else who was there did as well. We've got a couple of concerns. Lock issue, I said here miss 10.0 and will be in 10.1. I'm still waiting to hear about that. I'm not quite sure what the hold up with that is, but it should be in 10.1 that freezes on the seventh. So we definitely have enough time to finish that. Valerie and Dimitri have been sort of working on the Rails 5 upgrade. Maybe not necessarily as a main project for either of them, but definitely as a thing they've been working on, but it's just not enough people. And the longer we leave it, the more we have to do because the rest of the team is busy adding new stuff that they then have to account for in the Rails 5 upgrade. So it's kind of a vicious cycle there. We'll try and get more people dedicated to that in 10.3 so we don't leave it hanging over our heads for too long because the longer we leave the existing MR, the more likely it is that it's going to be really, really painful to pick up again like if we left it for, you know, three, four months from now until we tried to get a team on it to finish it. It would take a lot of time just to get up and running again. So those are the two main concerns I've got at the moment. Next I've got two things that are pretty cool that are coming. So first of all is comment on images, which I bet both of these I hope will make 10.1, but I'm not 100% certain yet. So this is just you're in a diff view and you can comment on text and you can add discussions to the top level, but you can't comment on images or other files. So this is adding the ability to comment on a particular coordinates position by coordinates in the image and have a discussion thread on each of those coordinates that are selected. So that's looking super cool. Like I said, don't know if it will make it definitely in 10.1, but I certainly hope it will. And then another thing is saved board configuration. So this is really useful as well. At the moment you can only save a milestone on a board and then any other filters are added once you view the board. What this will let you do is save configuration to the board itself. So you can say that this board is for the CI CD team and anybody who loads it automatically has the CI CD label applied to their filters along with whatever milestone you pick as well. So those are two super useful things. I didn't include a last slide for questions. So I'm just going to go to the chat now and see what's up there. Yep, these are both cool features. Saved board configuration, like Europe said, especially is huge because it's like root boards. It's one of those things that helps us make boards better as well because we can actually use boards more and the more we use them because of dog fooding, the better we can make them. John asks if saved board configuration is a C or E feature. I think it is EES, but I am not 100% certain and Victor should answer that for certain, for real in the chat probably or on Slack because I don't want to give the wrong answer. I'll give everybody 30 seconds for any more questions. 30 seconds is quite an uncomfortably long amount of time to spend just staring waiting for questions but I'm done now. So thanks everybody. You've got 22 minutes back before the team call.


All right, microphone is on, beautiful. Good morning, it's still morning, just a few more minutes. Good morning everybody, thank you for joining. The last session before lunch, so I gotta get everybody excited.   I've heard it's decent. My name is Inko Fuchs, I work at NetApp, I work in the cloud group, and my team's responsible for a number of topics, including OpenStack and object storage. So I thought, well, that would be a good idea, let's talk about object storage in OpenStack environments. And so throughout the session, I'm going to talk about things like Swift, I'm going to talk about things like proprietary solutions, I'm going to talk about some of the benefits of each. But ultimately what I want to achieve is that you come out of the session and you have some idea about questions you should ask. I've seen quite a few customers implement object storage and then some of them were disappointed. And often it was because they didn't ask the right questions in the beginning. And sometimes it's hard because object storage is a relatively new technology. And a lot of people are not as familiar with it as they are with NAS or SAM, with block or file storage. Some are totally new to storage because they have been worried about applications and they didn't care how the storage guys did the storage stuff. So maybe a quick question, who comes from more of an application background running or developing applications? Who's coming more from a storage background? What is the rest? Both systems? All right, great, perfect, thank you. It's an interesting mix. Another quick question, who has already implemented object storage? Whether it's Swift or something else? Not much? Okay, great, perfect, thank you. Really appreciate it, that helps me. So let me start the discussion with talking about some of the use cases. Why is object storage so interesting? And one is cloud applications. So with the emergence of applications that are either born in the cloud or have been transformed to run in the cloud, we see more and more applications that were written to actually work with something like object storage. So if you look at Amazon S3, where you look at Google Nearline or Glacier from Amazon, those are all object stores and they use object-based protocols to work with them. Restful HTTP, it seems a natural choice when you're building new applications. It's much easier than doing a file or block interfaces, much less complicated because the object store does all the data management and where data sits and keeping it available and what the location is and all that kind of stuff. And it works for a lot of the workflows and workflows very well. The second one is where we see object storage a lot is data archives. So for those of you that worry about not just the transactional data right now, right this moment that is supporting your application, but also about storing data for a long period of time. That's very relevant. If you're building applications that serve, let's say, the financial industry, you might have data that you need to store for 30 years, financial records, loan applications. When you look at the healthcare space, life of the patient plus X and that X varies by region and by law, those are very long retention periods for a data. And so you need to think about, are there technologies that I can leverage that make sure that data is actually intact and accessible 30 years from now, 60 years from now and object storage has a great answer for that. And then some of you, like the previous customer, some of you come from the media entertainment industry. So actually, there's a lot of data being created in this industry and it needs to be retained for a very long period of time and you're reusing it all the time.  So you have a topic that comes up and suddenly you need to bring data back out of an archive of some kind and we bring it back into your life online storage. Those are some examples for use cases. Now, if you look at OpenStack specifically, Glance is a great example for use case that benefits from object storage. You can build these large scalable environments to store your Glance images. When you're using Cinder, Cinder's block, right? So you got these big blocks of data. A lot of times you have a project where you need the Cinder volume when you're working with Cinder and then you're done.  More and more of what we're seeing when you're running into an enterprise IT requirements that you need to hold on to that for compliance reasons or because nobody wants to sign off on the deletion that happens often. So you can use a very scalable object store to just move your Cinder environment over there and leave it there and archive it, keep it for a year just in case you need to bring it back. The board of the cloud applications already talked about. Big data, more and more use cases about data analytics. So you have this internet of things that is creating all of the sensor data. I'm not gonna bore you with that, but you're creating a lot of data that you need to collect because you don't know what kind of analytics you can do maybe five years from now. Maybe data that is not or that you're not able to analyze today or that doesn't give you relevant results today, maybe five years from now it does. So you want this big bucket that you can throw data into and keep it. And then long-term archives, which I already mentioned. Now, one question that I got a lot is so what is object storage? You know, how is it different to file and block storage? One way to look at it is how do you connect to this storage infrastructure? And typically what you use is one of three interfaces. It's either S3, that's what the vast majority of the world is using. There is Swift, of course, very closely aligned with OpenStack. And then there's CDMI, the Cloud Data Management Interface. CDMI is an interesting one because CDMI was developed by SNEA, the Storage Networking Industry Association and it is an ISO standard. It is a very, very powerful interface but very few people use it. So pretty much everybody decided it's S3 and or Swift. So it's interesting when you evaluate solutions of this environment and maybe think about, well, maybe today I'm gonna do Swift or today I'm gonna do S3, maybe tomorrow I'm gonna do CDMI. Depending on your application that you use case, you might choose different interfaces. The second one is how do you address the data inside the storage infrastructure? And so most of you are probably familiar because most of you raise your hand with storage, are familiar with file and block. So block, you need to know the sector on the drive. Great, sounds complicated. The second one is file. You need to know what server or NAS storage system your data is stored on. You need to know an IP address. You need to know a share name. You need to know a directory structure all the way down to the sub directory and then you need to know the file name and the file extension. Sounds complicated. With an object system, you only get that object ID and that's it, that's all you have to know. Well, plus one IP address. So I liken this to comparison to valet parking. Who has a car? Who has driven a car ever in your life? I'm mostly just making sure that you're still, you know, physically capable of moving.  You have to memorize what level you're on, what row you're on and, you know, I'm assuming you know what your car looks like, right? But if there's, you know, three Chevy's parked next to each other, you actually have to remember license plate number.  If you define the correct service levels, it might be washed, maybe the tires are rotated, all that kind of stuff. But that, you know, it eliminates having to know where your data is.  I don't care where you get it from. I don't care where you parked it as long as it didn't get stolen or vandalized or anything, against service levels.  Here in Vancouver, how great is that, right? Wouldn't that be nice to have such a valet service? So that's something that object storage can do. It completely virtualizes and abstracts the physical location of your data from your application. And then you have metadata.  How do you get your data back? How do you get your car back? Well, hopefully you know the make, the model, the color, maybe even the license plate. That's all metadata, that describes your data, that describes your car in this case. So if you know what that metadata is, you can find your object back, even if you lose that ID. Not a great idea to lose your ID, but you can get your data back. So that was kind of just a very, very quick overview of what object storage is. Now, when it comes to a natural implementation of an object store, there are a number of criteria. And I'm just gonna touch this really, really quickly because hopefully by the end of the session, you can just go through all of those without my help. But reliability, durability, and availability, when you run an enterprise IT shop, those are very important criteria, right? You want to make sure that your data is available, that your applications are running, otherwise you might be losing your job. You want things like policy management, that determines automatically in the background where your data is being stored. You don't want to manually move data around. Multi-tenants, obviously in large environments. Elasticity, you can grow, you can shrink. You want to lower costs. Well, who doesn't like the cost model of the cloud, right? So if you're thinking about us, one cent per gigabyte per month if I do it in public cloud, can I beat that price with an on-premises infrastructure?  Yes, you can. With object storage, you can beat the cloud economics depending on the use case and all of those asterisks and footnotes, but you can. So don't walk away from it just because you think you can't beat the price of public cloud. Suffer defined. We get this question a lot. Can I do software defined? The reality is most people, most traditional enterprise IT shops don't go software defined. They go with an appliance, easy. You plug it in and runs, but you need to have the option to go software defined and use the infrastructure that you already have. And then finally, that global namespace that what I just described with the valet parking, it doesn't matter where my data is stored. I want to get my data back whether it's stored in New York and LA and Vancouver. I don't care. I need to get it back from there. So let's talk about the obvious choice, Swift. Now Swift, really interesting, and the first one I really love is the OpenStack community has decided to separate the Swift API from the Swift implementation stack. So you get a choice. You can use the Swift API without having to use the implementation stack of Swift. And that means that it gives you a choice to determine what you're going to use for the implementation. What are you going to use to store your data, to manage your data, to ensure data durability while still using the API that you want to use. You get the global namespace. It's very scalable. It offers some to your distribution options and there are limitations to it, but it's there. And they just rolled out erasure coding. Now to be very clear with the erasure coding is the erasure coding is great. There's a lot of stuff with the devil is in the detail. So if you're comfortable with picking your own library and doing your own quality assurance testing, the erasure coding beta is certainly very interesting. Now there are some things that you can do to improve on a Swift deployment. So this is when you go from a Swift deployment on just using individual hard drives to a Swift deployment that's using something a little bit more enterprise storage. The first one is usually when we talk about enterprise storage in Swift deployment, the first question is, that's more expensive. Well, that's not true and I'm going to show you why. The other one to keep in mind is if you're running, if you're building an open stack environment as an experiment and then you move it into enterprise IT, one of the pushback that we get is it takes up too much space. I don't have room in my data center. When was the last time if you're a service provider that you talk to your data center guy and he said, oh, I got tons of room in the data center. Just ship me another 200 service, no problem. You don't hear that very often. And so the ability to reduce rack space by 70% can dramatically lower your costs and your complexities. And the other thing is predictability. If you're running an environment that needs to run all the time, you want to minimize the disruption to your environment. You want to minimize performance impacts or availability impacts in the environment. I'm going to go through details for all of them. And then of course, HAA, you want to make sure that there's no failure. Nobody wants to lose data. So typical environment that's Swift. So if you just read like five minutes of Swift and how it works, this is one of the fundamental principles that we see in most Swift environments. And that is that you store three copies of data. Why do you do this? Swift is designed around running on commodity hardware. Commodity hardware fails. You have no protection. So what you have to do is you have to store multiple copies of your data so that if something breaks, you can repair it, right? So what you do is similar to HDFS, if you're familiar with that, Hadoop Distributed File System, you're storing three copies of your data. If something happens, you can rebuild it over the network. But that means that if you store a petabyte of data, you need to have three petabytes of capacity because you're storing three copies of your data. Now you can do things a little bit differently and I'm using our product as an example, this NetApp E-Series Storage, Block Storage that you put underneath Swift. And what we're doing there is that we're using an erasure coding algorithm that we call Dynamic Disk Pools to spread your data and your redundancy information over a large pool of disks that can be dynamically adjusted. That's why it's Dynamic Disk Pools. What it gives you is an advantage of even traditional rate in that you'd have very, very minimal impact if a disk drive fails because you're only hitting very few segments of a particular data set. So if you're using something like Dynamic Disk Pools, you only need to store 1.3 times the capacity of your data. Because now you don't need three copies anymore, you just get that little redundancy effort, that's 30% in this case, on top of that. And you can even do a lower if you wanna do some other things in different rate levels, et cetera. So benefit is your system runs more smoothly, runs more predictably, without creating all those redundancies that you need. And that helps with rec space and everything else. So if you look at the business impact that you have in this environment, if you use a traditional rate, rate six in this environment to protect your data, and you're looking at how long it takes, we stopped doing the math of three terabytes because once you get to four and six and very soon 10 terabyte drives, the numbers just kinda start looking a little bit ridiculous. But with Dynamic Disk Pools, you see that your impact, how long it takes to repair a drive is much, much shorter. That translates to a more optimal operating environment for your enterprise IT shop. So instead of having a business impact where everything slows down dramatically for a fairly long time, you make it shorter and you make it less impactful. And then ultimately if you look at how much data is actually affected by it, there's just because we're doing the slicing across a lot of disk drives, you have much less. Now, I talked a little bit about density. So availability is great, performance is great, predictability is great. So tell me again about density. So this is a typical Swift environment built on regular disk drives. And if you do this with, in this case, our box, this is a 4U storage system, 4U 60 drives. If you're interested in hardware, there's actually really neat layout. It's kinda like these multi-level ovens if you do Christmas cookie baking. Nobody does that, all right? So you have these trays that you pull out and all the hard drives are kind of laying in there as cookies. So if you need to replace something, you pull it out, put a new drive in while the system is running. But you only pull out a few drives because you have that tray. So you look at some of the competing options 4U 60 drive means you have to pull that entire thing out. You didn't make sure I'm not falling off the stage here. And then you have this butterfly mechanism and you have to go in top. Now if this is at the top of the rack, you need to tall ladder and you need to hope that there's no ceiling above you. So it's actually very, very cool hardware design. So you need the storage system, a couple of service and you're done. So that can dramatically improve your workspace. And then of course you can scale vertically because of our infrastructure and you can scale horizontally because of the intelligence and Swift. So if you wanna make Swift run better, run more efficiently and less space, that's a great option. Now what if Swift doesn't give you enough? So Swift software is great. There's a lot of movement and it moves very fast. Very dynamic engineering team, good stuff happening in there. But there are some things that are missing, a dynamic policy engine where you can adjust where data is placed over time. The data durability framework where you do regular health checks, make sure your data is still intact. So you're not going back in 30 years and recognize the data got corrupted 29 and a half years ago. The flexible tiering over time, what about support for tape? Now tape isn't sexy at OpenStack, I get that. But a lot of customers ask because they made an investment over the years and spend a lot of money on tape. So if you can tell them, well, if you really want, we can actually include tape. You don't really want to do that probably, but you could, it's investment protection. I'm gonna talk a little bit about Eurasia coding and S3 support and all that stuff in just a moment. Appliances, very, very easy deployment. What about auditing and reporting capabilities? So I want to introduce a product here that we built. And I think you all know or most of you know that NetApp has been doing a lot of work in the OpenStack community, but one of the products that we provide for the OpenStack community is called Storage Good Web Scale. It's an object storage product. Goes back more than 10 years in production environment. It originated in the medical field and that's relevant and interesting because that's where a lot of our data durability features are coming from. If you think about somebody that does surgery preparation, he's gonna take out a kidney. You want to make sure that he's taking out the correct kidney. So that's where a lot of our algorithms are coming from making sure that the data is not corrupted, it's not changed, it's not lost, it's always exactly identical to what was originally stored. And this is a solution that we're proposing in this environment. Let me walk you through how we answering some of these requirements that we're hearing from our customers in the OpenStack space. One is data placement. The value of data changes over time. So storing all the data in one single spot is maybe not the most efficient way to do it or not the most cost effective way to do it. So if you're thinking about just your on-premises deployment or hosted deployment, you can have flash, you can have disk, you can have geo-distributed erasure coding. I'll get back to that in a moment. You can even have tape. Plus you have the option as a tier to use public cloud. So if you have an S3 compatible target, either on-premises or hosted that you can write to that you want to leverage because maybe the economics of the cloud makes sense for something, this is just one tier of storage that we support. And you can manage it across multiple data center locations. And that can change over time. Maybe when you're ingesting data first for the first four weeks or so, maybe you say performance is the most important factor in determining whether this data should be stored. Both physical location as well as tier storage. But maybe after that month, cost is the most important factor. So you want to move you to something that's more cost effective. The way that we're doing this is with a dynamic policy engine. In a policy engine, you go in the policy and you say, well, here's how this particular set of data should be treated. And we can do this on a per object basis. Every single object can be treated differently from policy perspective. You don't have to do it on a bucket basis or container basis. You can do this on a per object basis, very, very flexible. Say this is my policy and this policy determines where data is being stored and for how long. You can say it's gonna be stored here. You know, let's say in our data center in New York, performance very important is going to be stored there for the first 30 days on very, very fast disk. After 30 days, I want my copy in my much, much cheaper data center in Oregon or in Nevada, wherever it might be. I'm gonna keep it there for 30 years and then I'm gonna throw it away. But during those 30 years, I'm going to do a health check every six months to make sure the data is still okay and hasn't been changed. One important thing here is that what if you change your mind? What if you acquire a company and there is a new data center? Or let's say cost factors have changed. You now want to go to public cloud with Amazon, have it hosted or something else. With this policy engine, you can actually have all that data being moved dynamically in the background automatically once you change your policy. You don't have to manually move those hundreds of millions or billions of data objects that you stored. It happens automatically. I talked about data integrity. That's very important, right? To make sure that at every single step, ingest, transport, replication, readout, the data is always assured. It's always correct. Nothing ever changes. If it breaks, we automatically fix it. We regenerate the object on the fly. The application will never know. Also, playing into this is data availability, making sure that data is always available. Regardless, even if you lose an entire data center, you want to make sure that all the other data centers still have data that you can recreate or just create another copy. Data availability, even if sites go down, it's very important. Especially if you're looking at a truly global deployment. Sometimes internet connection to Asia goes down. Can you still operate even if the copy of the data was originally stored in Taiwan? You want to be able to do that. One of the concerns that customers have, and I think one of the drivers for the adoption of OpenStack is security and security in the cloud and security across clouds. We offer encryption. So if you're not encrypting data on the client side, well, we are doing it on our side. If you combine it though, so if you're doing data encryption on the client side, and then you're doing encryption again with us, your data is very well encrypted. That's a great way to look at it. And then obviously, we support the standard security access mechanism from S3 and CDMI, for example. And then the option to actually take your data and tear it to the cloud, if that is what you choose to do. We have a lot of customers say, you know what, ultimately I want to do public cloud, but I need to go with an on-premise infrastructure today. There might be legal reasons, might be lost, might be compliance rules, whatever it is. Maybe sensitive customer data. They're saying I want to do on-premises today, but I want to have the option to leverage the public cloud. So with us, transparently, you can use the public cloud to store your data. And this might just be like a disaster recovery copy. Maybe you can afford to have one big data center, and that's where you run all your on-premises environment, but you choose to say, you know what, my disaster recovery copy, instead of building a data center or leasing one and having hundreds of servers to sit in there, maybe I'm just going to use Amazon for that, right? So that is one option. Now I promised I would talk a little bit about erasure coding. With erasure coding, erasure coding is something that's been around for a long time. If you look at rate, rate storage, that is a form of erasure coding. So we use rate and we use dynamic disk pools. Again, those are different ways of doing erasure coding. We also do something that's called geo-distributed erasure coding. But the way that you can imagine how this works is look at the way, you know, a rate system works.  So you can spread your data and erasure coding data, your redundancy data across a lot of disks. Geo-distributed erasure coding works exactly the same way, but across physical locations. So now you can geo-distribute erasure coding across, let's say, three data centers or five data centers.  But you're doing this without the cost of storing individual copies of your data in each site. So traditionally, what you do is you store full copies in each data center.  If you lose one, you can recalculate and repair that data. And again, it's transparent to your application. Now what that means is it's probably gonna be a little bit slower.  But if you think about the cost and the impact in your environment, this can make a lot of sense. Think about you're storing a lot of data and you're moving from full copies to geo-distributed erasure coding, which you can do automatically with our policy engine. Once you move to geo-distributed erasure coding, you have roughly 30% hit on your capacity as opposed to a 300% hit. That means that suddenly you might have a petabyte of data storage just available for the next application. That's what it really boils down to is you have a more efficient way of storing data, meaning that once your data is in that right mode, that means that you can free up capacity for the new data that's coming in. And all of that happens automatically in the background transparent to the application. So when you look at deployment options, I talked about software defined before and I said, well, there's kind of the I want to do software defined versus I want to have an appliance kind of discussion. It's a little bit philosophical. With our product, you can actually do both. So you can mix and match your software defined and your appliances. So I had a customer that I talked to two weeks ago and he was running a project and be close to the deal. And he said, you know, I just invested into two petabyte of your competitor storage. And I thought I was going to roll out my applications this way, but I was wrong and so now I made a mistake and I, well, I can't give it back, right? I bought the stuff. He said, that's no problem. We run software defined. We support third party storage. You can use that storage investment that you already made and just run our software on top. So you got that investment protection. No problems there. We have the four you 60 drive appliance. And I talked about the supply. There's the same one that I talked about earlier, you know, with those cookie racks that you can pull out and you got your cookies, your disks stacked in there. And so what we did is something actually very neat. We put all of our software right inside that appliance. So you don't need a whole stack of servers on top of that. So from a density perspective, now you're talking about a four you system that has 60 drives in it with up to six terabyte each right now and the entire intelligence to manage all of that data, the policy engine, the data durability features, the interfaces, all of that is inside that appliance. So again, it's just very, very efficient, very easy to deploy. Now the question is, well, that's all very interesting and go thank you for that. But can I use this for OpenStack today? Yes. So we support Glance today, we support S3 and CDMI.  So you have that choice today. And we are actually doing an implementation of the Swift API right now. Now we have made a decision not to go with the gateway, not to go with the bridge, not to go with the shim but rather implement the Swift API natively inside our core software stack. This gives you a much better integration, it's a much more efficient way of doing it, a much faster way of doing it, but it takes a little bit longer. So that's why, you know, I'm not here today telling you that it's available today, but it is coming very, very soon. And if you're thinking about implementing something, if you're thinking about a proof of concept or a pilot, let me know and we can provide the software to you. I want to give you a very quick, very quick customer example. And for those of you that come from the storage world, you're probably thinking, okay, well, why would I go to object storage or what about network attached storage? You know, well, in go your storage system for NAS, they score, they scale to 70 petabytes. Why would I even consider object storage, right? So this is a customer that I actually used, previously used about six petabytes of our NAS storage, network attached storage, storage file based. And what they recognized was that they were running into problems with their applications because their intellectual property is all about their software. And they're ingesting billions of files every year from all of their 19,000 retail locations, 8,000 kiosks across the globe. They're getting all of this data coming in.  They need to store it, they need to manage it. They need to do some cleanup work with it. They need to do some alterations. They need to store it in different ways. And they need to keep track of where all of these files are. And for every single file they need to store which server is it on, what's the file share, what's the directory structure, what's the name. And they need to keep this redundant in case something goes wrong. And then they just have to hope that nobody goes in there and manually moves the file. What they recognized and what they decided to do is, okay, these database are getting so unwieldy that I need to do something differently. I can't do interactive lookups anymore in my data. My customers are getting frustrated because they can't offer certain services to their customers. And so they decided to say, okay, we need to go to an object store. And they decided they needed to be on-premises. And so they implemented storage with WebScale. And one of the side effects that I thought was very interesting is that they so massively shortened the length of the rows in their database where they used to store all the path information and how many copies they had and where they were physically stored by replacing all this information with object IDs. So the rows have gotten so short now that now they can do interactive lookups. They can provide interactive services to their customers. So their choice to go to object storage was not the scalability, but it was the simplicity and the automated and transparent data management that allowed them to optimize their applications by going with an object. And I thought that was an interesting perspective that I wanted to share. Now, I've been going through this at 100 miles an hour and I apologize. A lot of stuff I wanted to get across and wanted to make sure that I get all this information across here at the session. Now, we have a booth here, S13, with a beautiful view of the ocean out there, of the bay. There's another session actually this afternoon at 410 where my colleague Alex McDonald and Mark Carlson from Toshiba and they have, I don't know if you have seen that, the Toshiba booth. They have an object-connected hard drive. It's kind of a cool idea. They're going to talk about Swift and extending it with S3 and CDMI here and they're going to go much more technical. What I did here is really more of an overview and introduction. They're going to go really, really deep into those protocols, what the differences are and why you would want to combine Swift and or S3 and CDMI and what the benefits would be. If you're more interested in erasure coding, there's a great webinar that the Storage Networking Industry Association did explaining erasure coding and why you want to do hierarchical erasure coding which combines local with geo-distributed. That webinar is available. It's free. It's available without login. So check that out. And then of course, I have some product links here as well. If you have any questions, please contact me, either email or Twitter. I respond, trust me. I really do. And with this, I want to thank you very, very much for attending the session. I hope this was useful and I hope you have a great show. Thank you.


In this network we're going to do some summary statistics, some of the most basic things that you can do in data science. We're going to import some data and we're just going to describe that data and that really is one of the first things you can do when you start analyzing data, once you've got the data in Python and you've cleaned it up, just try and understand the story, the information that's hidden inside of that data. We are notebook 05 descriptive statistics and as I just mentioned one of the first things that you'll do when you analyze data it's very difficult for human beings to stay at for instance a big spreadsheet file or massive amount of images or audio waveforms, no matter what our data is in data science, very difficult just to stay at it and get some you know get some get the information that's hidden from it. We have to tease that knowledge that information that story that the data wants to tell us we have to get it out and one of the first things that we do is just we summarize that data and that's what the descriptive statistics or summary statistics that's what it's all about. So let's look at the packages that we can use in this notebook we're going to import pandas so import pandas as pd the namespace abbreviation so we've imported that namespace with all its functionality into python here then we can use a new one scipy scientific python it just expands on what numpy can do even further it's got a module inside of it called stats and in stats you get a bunch of statistical functions i'm going to say from scipy import stats so i'm not using a namespace abbreviation so if we want to use the functions in in that stats module we'll have to say stats dot from google dot co-lab we're going to import the drive function because our spreadsheet is in google drive so specifically to google co-lab and our data being in our google drive we have to use that drive function and then as always i use this percent load underscore ext that magic command just to then print my my spreadsheet files my data frames out to the screen nice and neatly next up we're going to mount our drives remember that's drive dot mount and then we pass this argument which is a a piece of string so it goes inside of quotation marks forward slash g drive and i showed you in the notebook before that that folder structure where this file is saved and this right at the end this data sub folder there i've got to put all of that as a string after the percentage cd change directory i want to change to that right at the end that sub folder because that's where my spreadsheet files are so i'm going to run this cell remember that's going to open a new login for me i have to re-log into my google drive give it the permissions that it requires and then copy that security key that it's going to give me to enter in the little block that is going to appear here and there you go it's happened and i can all you can also see the percent ls magic command there that's just going to show me all the files that are inside of that and the one that we're going to use in this section is the client underscore data csv file so once again i'm using the read underscore csv file from the pandas package so pd my name space abbreviation pd dot read underscore csv and because i'm already in that folder i change directory to that folder i can just reference the csv file directly and we're going to assign that to a computer variable called df to hold our data frame object let's just have a look at the data frame object just to make sure it imported properly and there we see we have a first underscore name last underscore name very important to have those underscores when you design these so that you don't have these illegal characters so we can just use the dot notation when we refer to that column and we see age number of children email address all of this is fictitious data job title home loan more than one vehicle financial literature review savings investments etc so some economic data pertaining to specific customers or clients so let's examine this data frame object as we always do we're going to call some of the attributes we've seen before first one is the shape attribute and that's going to tell us how many rows of data we have so how many data on how many subjects or how many observations all those are synonyms for the word rows so there's a thousand rows along eight or should say eleven there eleven variables so if i want to see the names of those eleven variables remember that's the columns attribute so df dot columns and that shows me all the variables that we're dealing with a society data so every rows and observation or subject every column is a very specific variable with a very specific data type so let's look at those data types at least what pandas things they are so first name is an object which is a string which is a categorical variable so it is for last name ages a 64-bit integer a children 64-bit integer email as an object in other words categorical variable or string so it is with job title and then home loan it sees those as bool values boolean it's two or false values more than one vehicle it sees that as true or false values so let's just look up there if we look at those indeed home loan and more than one vehicle it was captured as true or false so pandas correctly identified those as boolean values and then you can see the rest there so the first thing that we're going to do is we're just going to do counting and we're going to refer to these as the frequencies how many times does one of the sample space elements or each of the sample space elements in a variable how many times does it occur and we've seen the unique method before so i'm going to say df dot invest remember that gives me back just that column in a pandas series format so df dot invest now that i have that column as a series object i can call the dot unique so the dot unique method on that and that's going to show me the sample space elements inside of that invest column so i see it's three one two five four and remember it's in the order in which it accounted that going from the top to the bottom now a much more useful one method i should say when it comes to frequencies is the value underscore counts value underscore counts so once again i'm going to say ds df dot invest so it gives me a pandas series object of that column the invest column and then i call this method value underscore counts so not only is that going to give me back the unique elements you see there two five four one three but it gives me the frequency of each of those so you can see how many times the value two appeared in that column five four one three and again what it's going to do here it is just going to go down the frequency in a descending order so the the sample space element in my invest column in my invest variable that occurred most commonly was two and that occurred 210 times so that's a frequency we can also do relative frequency so what we're going to do is just divide by the sum total remember there was a thousand observations i just devised by a thousand so that i can get a proportion or a fraction of each of these instead of the absolute value which is the frequency the actual count that's now going to give me a fraction of the whole which is which is what we call a relative frequency and all i have to do there is use one of the keyword arguments for the value underscore counts method and that's the normalize with a z so this is all american spelling so df.invest again that gives me the invest column in pandas series format and on that i call the value counts method and i pass the argument normalize and i set that to true remember true and false as far as the actual python words are concerned the keywords is uppercase t and uppercase f so if i do that i now see the fractions and the fractions must always the proportions must always sum to one 1.0 that's the the the the sum total of them and if i wanted to do that as a percentage i'm just uh if you really wanted to do that i don't really see why but we can just do some broadcasting where we multiply each value by a thousand and once we do that by a hundred and then we get percentage 21% 20.8% 20.1% etc so a little exercise for you there if you're interested just to stop the video and and do that for yourself more interesting though we're going to do grouped frequencies so how do we do grouping now grouping occurs by the sample space sample space elements of a categorical variable so you take a categorical variable and it'll have a sample space some unique values and we're just going to group by those and then we might count the frequency of some other categorical variable so what we have here in the first instance we're going to create this this what we call a contingency table and we're going to do home loan which is a categorical variable and more than one vehicle which is also categorical variable and we're going to pass those two categorical variables in series format because i'm using df dot and then the name of the column and df dot the name of the column that gives me back remember pandas series object and i'm going to pass those two elements to the cross tab function so pd for pandas pd pd dot cross tab and then pass those and look what it's going to do it gives me this nice little table so it says home loan here false in two and then the second one is going to be across the top there so that's more than one vehicle false in two so if we look at no home loan and not more than one vehicle there were 268 and a home loan with a not more than one vehicle 239 and then the two values uh two four four more than one vehicle and two and false for home loan and then right at the bottom the 246 more than one vehicle and with a home loan so that's a two by two contingency table two rows two columns and that's what you can see there with these four values then two rows and two columns now measures of central tendency we've talked about how to count things frequency relative frequency and as the course goes on we're going to see many more examples of those so remember this is just an introduction so let's do some point estimates or central tendency and what we do there is we take a variable and all its values and we calculate a single value that is representative of the whole and that's what a mean or an average is taking a bunch of numerical variable values and I express a single value that is representative of all of them and the first one is the arithmetic mean or the average and remember how you do that you just sum up all the values and divide by how many there are so let's see what the average or mean age was of all the customers so I can just say df. age so that df. age is going to just give me just that column as a panda series object and then I call the mean method on that not passing any arguments and that's going to give me the mean age in our data set and that's 44.739 years now what is the mean value of customer savings now remember I could just say df. savings but I'm just reminding you here if you do have illegal characters like spaces in your column names or your variable names you can always put them inside of quotation marks inside of square brackets so df and then savings and then dot mean and that's going to print out for 30,136.49 as far as the average is concerned now let's do something much more interesting I want to know the age the average age of only the customers that have a home loan and remember we do that with conditionals and I'm using a bit of a shortcut here and then I'm not using dot loc or dot i lock I'm just going for it directly df and then all the rows that I'm interested in let's go down the home loan column and look for all the values set to true so equals equals true so that's the only ones that are going to be selected from that give me the age column and of that age column give me the mean so that's how we would construct that that's going to give me the mean or the average age only of those customers that have a home loan so you can also do the mean average or the mean age of the customers without a home loan so that equals equals false as far as that conditional is concerned and then instead of putting age like that just showing you a little bit of a variation I can just say dot age because age contains no illegal characters something a bit more interesting what if I wanted to do those two steps all in one and here is a really a method that you're going to use a lot the group by method I'm saying df dot group by and then inside of parentheses because this is a method I'm going to pass one of my columns inside of quotation marks this time so home underscore loan group by whatever you find in the home loan and then the age and then the mean so let me show you what it looks like and it'll make a little bit more sense so in the home loan it found two sample space elements false and true and then it gives me the mean age of each of those so the customers without a home loan the average age was 45 and those with a home loan the average age was 44 so that group by becomes it's going to become something very useful to us and we're going to use it a lot remember there's also the geometric mean and that's calculated a little bit differently and that comes from the stats library so I can't call that as a method on a panda series object so I'm going to say stats dot g mean for geometric mean and then I'm going to pass my panda series df dot age and that's going to give me the geometric mean so that's just multiply all of those together and then take the nth root where n is the number of observations that we do have median now what the median does it puts all our numerical variables in ascending or descending order and then if it is an odd number of values it's going to take the one right in the middle for which half of the values are less than and half are more than and if it's an even number it's going to take the middle two and just take their average and that works very well when I data when our data is v skewed when data is v skewed we have outliers we're always going to pull that average towards the outliers and that value is now no longer a proper representative remember that's what we're trying to do with point estimates one value that's a proper representative of all the values and in that instance the mean might not be so so we have the median so let's have a look at this what is the median saving of customers older than 50 again we're just going to use some conditional there df and then df dot age greater than 50 so it's going to go row by row in that age column and then from that we want the savings column so I could also just have said dot savings dot median and anyway dot median will calculate the median for me so let's look at the median savings of the participants or the subjects or the customers that were older than 50 years of age now remember how we put those conditionals together as and and or so what we're looking at here is where the home loan is true and the more than one vehicle is true as well give me the median of the savings so referring you back to the previous notebook where we looked at these we just call catenate them together and and or so the next one we're going to say either the home loan is true or the more than one vehicle is true and then give me the savings column and the median of that savings column excellent and there's a little exercise and solution set for you now we come to the mode now the mode is where we don't have continuous numerical variables remember continuous numerical variable can have many decimal places we just truncate those maybe the apparatus that we use in the lab can only give us until a certain number of decimal places but in actual fact the decimal places go far beyond that and we don't do a mode for that mode means the value that occurs most commonly so yeah we're really going to go for either discrete data or categorical data anything that has a very fixed sample space elements and if there's one sample space element that occurs most commonly that's going to be the mode sometimes two of them are equally have the equal highest frequency and then we'll call that a bimodal variable and then you also get trimodal and even more multimodal anyway let's look at the number of children so i'm going to say df dot children dot value counts and remember it's going to do that in descending order so it sees three children that was the most common so three you can think well it is really numerical but you can't have a fraction of a child so it doesn't really help to to express the average number of children now you see that all along but but really that is a very difficult concept because what is 2.18 of a child very difficult there so i tend to use any kind of discrete data rather use the mode next up we're going to look at measures of dispersion now that we know that we can take a data set a variable and we can calculate the single value that's represented of the whole we also need to find out how spread out the data and those are measures of dispersion so the first method of dispersion that we're going to talk about is the standard deviation and the variance and they actually refer to the same thing and you actually calculate the variance first take the square root of the variance to get the standard deviation so what is the standard deviation well that's the average difference between each value and the mean for that variable so I just take all the differences between each value value number one and that's difference with the mean and I sum all of those up and divide by how many there are so that's the average the problem is some values are less than the mean and some are more than the mean so we're going to get negatives and positives and we sum all of those up you're going to get to zero so the standard deviation is zero and that doesn't really help us that's the way that the that the mean is calculated so what we do is we square each difference and remember when you square negative value it becomes a positive value so all our values will now be squared and if you add all those squares all those positive differences and divide by how many there are you're going to get to the variance and what you'll have to do is to take the square root of that to get to the standard deviation but that's exactly what the standard deviation is the average difference between each value and the mean for that specific variable it's very easy to do and I'm just showing you one example here we're going to just group by the home loan so all the sample space elements of this categorical variable we're going to group by that then take the h column and take the standard deviation and that's the std method and if you can see up here there's also a dot var method there and so it's either dot std or dot var for standard deviation and variance and you'll see the two standard deviations there the range is this the difference between the minimum and maximum value and remember how we get min and max there we go df.age so that gives me our our column there of only that that column the values for the age column or in series format and then I call the dot min method on that so the minimum age is 25 as we can see then the maximum age is 65 and the range is going to be the difference between the minimum and the maximum so I'm going to take the maximum minus the minimum and that gives me the range 40 so when it comes to age age of animals age of organisms age of human beings I like to express the range many times in the reports you'll see the standard deviation but that doesn't really tell me uh that much I want to know what the minimum age and maximum age was because I want to infer the results of your lab to my lab and and it's kind of neat to know the youngest and the oldest age that brings us to the quantiles now remember the median that just chopped up my my all my values new continuous numerical variables in half I put them in ascending or descending order chose a value for which half were less than and half are more than that value but I needn't only chop it up into into halves I can also chop it up into quarters and for that we'll get the first quartile the second quartile and the third quartile so the first quartile is going to be a value for which a quarter of my values are less than and three quarters are more than the second quartile is going to be the median in the middle the third quartile three quarters of the values are going to be less than and one quarter more than and just for completion's sake we actually also have the zeroth quartile in the fourth quartile that'll be the minimum and the maximum values and the way that we do that is with the quantile you can see there the quantile method so for age again that's df.age the quantile and then inside of a python list I'm going to pass all these values 0.25 0.5 0.75 so we actually put that in fractions and that's going to give me those values so as far as the age is concerned we see that at 0.25 so we know that a quarter of customers were younger than 35 half were younger than 45 and three quarters were less than 55 or you can obviously see it in the opposite direction a quarter was older than 55 but I can express any of those fractions so here we just go at the 95th now we call it percentiles so we're going to say df this time I'm going to group by the true or falses in the home loan take the age column and give me the 95th percentile that means 95 percent of the participants and we see 63 for both so both those within without a home loan 95 percent were younger than 63 or we could say 5 percent we're older than 63 and into quartile range is the last one I want to talk to you about and that becomes very important when we look for outliers in our data or when we want to create box and whisker plots and that's coming up in the next notebook very exciting notebook visualizing data and all I'm going to do is I'm going to subtract the value of the first quartile from the value of the third quartile so it's that difference between the third quartile and the first quartile that gives us the inter quartile range so I'm just going to say df.age.quantile 0.75 minus df.age.quantile 0.25 and that gives me the inter quartile range of 21 so there you go just starting to tease out that that story that the data is trying to tell us by summarizing it in some way either calculating a single value that's represented over the whole or then giving us some idea of how spread out the values are


Touch is an important part of social interactions, including human-robot interactions. That's why we designed capacitive touch sensors for the Blossom social robot platform. We formed nine flexible sensors by crocheting conductive thread into the robot's soft exterior. We can use these low-cost sensors to program and study touch-based human-robot interactions.


Everybody forced here rip in 2023 a leaf exotic football 20 box case number two random hit style We'll type in the hits as we go along here and then we will random off at the end to see who gets what? Good luck everybody one hit per spot in this 60 hits in the Metal autos per box. Yes, these are all top-load. Yeah. All right first up a Trenton Simpson Trenton Simpson nine of ten Glen Numbered seven of eight It's pretty cool and a zebra looking car Jaren Hall here. Jaren Hall is six of nine Yes, there's some pretty big names in these Did you hear this on like Thursday or something? It's Kyle on Thursday, right? Yeah. Yeah. All right. That is a Noah Sewell Numbered six of seven Noah Sewell six of seven next is Jalen Milrow four of four the Jalen Milrow and then there's a Jake Hainer or jaker Hainer as we would probably call him around here two of 13 Jake Hainer the diamond back right there That was almost a pick that was almost a catch that was almost the this that play hat at all Next is a Don Tavion wicks autograph number five out of 15 That's what Steve would call them. Well. Yeah, because of because of because of Jake Because of Jake, that's why five of 15 Don Tavion wicks Luke Musgrave Numbered four of eight on the Luke Musgrave. Yeah, it is kind of like Jalen Hertz I guess they're both J H and a Zay flowers Seven of 13 Zay flowers. I'm sorry to hear that illegal, but yeah You should it's interesting got that money for all the fantasy leagues that were in together Again full sellout promo tonight Flawless down to 19 random team style. It is 459 at 99 the mixer is under seven there's under 800 bucks. It's like right at 800 and That has 14 to go. Yeah, $800 799 99 Shawn picked out the thumbnail for it this week. No, I did not I promise you that Sean's one of those people online that is just like I've had it with the Taylor Swift stuff Ronald Washington two of eight on the auto before I haven't seen a Taylor Swift post in like three or four days although I was walking through Publix yesterday and One of her songs came on But that I mean that's Publix they they've always been coming on inside of Publix's And come on don't watch it in two of eights No, better. I think sin is just with the action Roshan Johnson four of seven on the auto you guys You've never seen the the the cat video where there's the cats like they did it kind of show you we were talking about it Don't I'm Johnson. You guys don't I'm talking about That's how I want to say Roshan Johnson in every time germane Burton One of one germane Burton on the like ice leopard print something going on. There we go one of one You would have bought a spot if I said no, yeah, I'm sure oh Sean Johnson I'll show you guys in between breaks. I'll show you guys. It's pretty good It's like old old internet video Miners turnover What happened dude? Oh Man, they're like right there too Oh Anybody make contact with it Like good that good camera angle, huh? Hang on what is going on hang on guys this place Did someone touch it though while they were out? That's that's the question Well, they rule it fumble right or like recovered recover. That's what I mean We're covered Xavier Hutchinson number one of three on the auto. I don't know what's going either way yikes Dallas They might dodge a bullet if that gets really really looked at I don't know It's really hard to see if it was touched by somebody. I do I I don't yeah Don't look at it. Trey Simpson Trey or Trey Benson Trey Benson on the auto here five of six. I don't have time to watch this whole thing I don't think anyone touched it, but that one dude's foot was so close The ball didn't move though. So yeah Trey Benson five of six check this one out dual autograph Dan Marino Joe Montana Marino Montana do a lot of one of only one how about that one? Crazy good luck for the Niners, but they forced the fumble to be fair like it's not like all luck, but That one was close We're not gonna read that it's already got Josh Wiley auto 206 Josh Wiley Too nice prospect. I thought I was the marino in Montana Too late we pull out to Sean can spell that one 713 too late we pull out to and a Bryce young redemption Very nice, I can't remember if they there was some they had him for I don't know Ha, what's up Kyle? Kyle remember all along Johnson you know I'm talking about Kyle knows Bryce young redemption. I believe I wasn't sure I think they had some of them They definitely has a bright song out of someone like live ones that they could send out to people I don't know though. I know they're they have decent stuff to offer in place if they don't have it So they're telling me they're calling any read during the day or tell me about that. It's funny All right, you got a Devin Brown auto Devin Brown three of five. Yeah, I don't think he's gonna spell right even by look Roche on yeah, see Kyle knows Devin Brown three of five got an auto Tyler Scott Tyler Scott one of three good case so far and Joe Thysman Joe Thysman two of two Kyle you see Paul's question Sam texted me at 12 30 at night Asking about like because he couldn't find a product and I just woke up thinking like why it was like I woke up at like 9 30. I'm like, do you think I was gonna respond? It's Tyree Wilson Number three of seven maybe didn't expect it, but maybe so maybe some Tyree Wilson three of seven on the auto watch what back? Got a Dwayne at McBride number seven of 13 and Clinton Johnston number two one of 15 Yeah, I'll make I'll make a note to take a look Paul. How would you put the pen? Come on Kyle Check stranger thing all right that video of Sam and Dylan try to find that box after main brakes They're not even want to watch Sam and Dylan do brakes Andre Eosee boss on the auto here three of five Andre Eosee boss Kendra Miller auto Three of seven of the Kendra Miller and Ladanian Tomlinson that's sick 203 LT to three LT to three Sunday fun day guys, let's get it filled on up Sunday fun day Phil grabbing a spot in both of the remaining breaks. That's what's up 13 on Sunday fun day now again full sellout promo tonight Yeah, and then the Remember guys Sunday fun day also has a season-long promo. There's 20 prizes One spot on the Sunday fun day mixer is one spot in the drawing at the end of the regular season There's Brady helmets break credit there is What there's a Bradshaw and Lynn Swan helmet as well or some other cool stuff though It's listed in the description that link right there Tanner McKee 404 Tanner McKee Walker Howard One of ten Walker Howard and Jason Taylor Jason Taylor two of six that guy was all right, I guess all right halfway through we got 30 more autos coming up You're just typing like number name, right? You don't have to type all that's There are a lot of Tank Dell on an auto here eight of 15 Tank Dell Got a Michael Wilson number to two of eight Michael Wilson two of eight and will Anderson junior now Three of five solid box as well all rookies that time will Anderson junior three of five Mr. Anderson one ref says down there others didn't signal anything What happens all right? I say uh Foskey foes key Foskey two of three Sean's dialed in right now.  That's a that's a fumble Whoa, what a play is that Rico Lewis?  That is an insane play. That's crazy by him Dalton UK. Nice big-time defensive plays right now in this game Christopher Vizina numbered five of six Elijah Higgins Elijah Higgins seven of seven and Bijan John Robinson two of seven cut that ball on his hip you guys see that today Cut the ball like on his hip like in stride and like ran it in it's pretty cool Bijan bedbath and Bijan Bijan mustard. Oh my god. This this game is gonna be turnovers back and forth. I guess Cameron Peoples Cameron Peoples 1 of 13 Okay 18 on flawless guys Sorry guys texting rich 8 today go on flawless false on that promo tonight mazy Smith number 12 of 15 Andy Reed five of 10 big red Big red auto Good luck y'all on that one some good stuff in this man. LT got the marino in Montana Bijan Jonathan mingo on the auto number seven of 13 Jalen cropper five of nine cropper. Hello jaker Mayfield Jerome bed is auto six of ten the bus Kyle Why is Venice as well some cool vets great young players good mix in this product. I kind of I like this a lot Those best parts are all top-loaded all designs as well. What's with me and getting all these? Stabbed by cardboard Anyway, Cam Smith Cam Smith three of eight Ithaca Roman Wilson now number three of seven Chris Paul and Michael Mayer Michael for mayor 13 of 13. Oh, yeah kiddle is balling him at Etienne are carrying me.  Oh boy Miles Murphy Miles Murphy five of seven Charlie Murphy got Chase Brown number two of seven and Jamir Gibbs eight of eight There we go eight of eight Jamir Gibbs. Yeah, I'm excited to check my fantasy score now things to etienne and kiddle I I Still probably lost my team is not good Well, all my guys are on by or like sake ones out Then there's like seven people on bio eat this week Josh Downs. He had a good game today Josh Downs on the auto three of 15 Sean is Fort Knox locked right now got Hunter loop key number two of two and Jackson Smith and jigba Number one of eight Chili's I don't want it. I don't want to chili's burger Ithaca gifted one membership one membership was gifted. I got a tie Simpson tie Simpson auto two of eight Dan M with a Sunday fund a spot on a 12 on Sunday fund a Chris Rodriguez jr. Two of three on the auto See rod two of three and Quinn Ewers Two of three. Good luck on that one as well Let's see why this break filled up so fast. I'm sure he was a baller Phoenix Jaden Reed numbered four of ten There we go K. Shawn Bouté one of three and Boat nicks Three of four Boat nicks Like that was a pretty good case random time. Good luck Randomized names and hits same amount of times put them side by side. We'll go five times on each. We'll read them off one two three four Five Scott C Down to John s and the hits five times. So it's gonna be a name and serial number on each of those one two three four and Five Kincaid down to Benson All right, Scott C. Kincaid Nicole Tyree Wilson Dan M. Noah Sewell Nathan B. Kendra Miller Florian with Xavier Hutchinson Alexander s Joe Thysman Kyle B brought Glenn Jessica G with Dwayne McBride Mike F Roman Wilson Jessica G Trenton Simpson Aaron s Bryce Young Redemption Adam R. Donald Washington Mike F Chris Rodriguez jr. Stephen s hunter loop key Valerie with the merino and Montana duel George M. Elijah Higgins Valerie tank Dell John s with Roshan Johnson Michael a Jason Taylor and the Von Oshane HN Richard H with Jaden Reed Adam R. Chase Brown Brian L is a flowers Benjamin L Michael Mayer Valerie Jonathan mingo Peter C. Ladanian Tomlinson George M. Josh Wiley long J Jerome Betis Scott H J. Hainer Matthew L. Isaiah Foskey Dave C. Will Anderson Nicole with Michael Wilson Valerie Tanner McKee Michael a Tyler Scott Dave C. Quinn Ewers Gavin Jalen Cropper Justin H Jerome Burton Henry C. Bonix George M. Ty Simpson Benjamin L. Walker Howard Pete M. Don Tavion Wicks Wayne P. Maisie Smith Kristen P. Miles Murphy Neil M. Luke Musgrave Valerie Josh Down Stephen H. Quentin Johnston George M. Andre you see us Clay F. Cam Smith Matt P. Jaren Hall James M. Christopher Vizina or Vizna Stephen H. with Devin Brown Dave C. Cameron Peoples Kyle B. Jameer Gibbs Adam R. Tully Tui Palotto Ben H. Andy Reed Valerie Vijon Robinson George M. Jalen with Milrow Stephen S. with Jackson Smith and Jigba Pete M. K. Sean butte and John S. Trey Benson Let's get those posted next break coming up in just a few minutes here