Hey, everyone. Welcome. This is Women in the Tech Workplace. I'm Alana Burke. I work for Chromatic. You can find me online on Twitter, druple.org, and GitHub. So these are three of my guinea pigs. And I named them after three really important women in computer science. Their names are Ada, Hedy, and Jean. And those women were here long before any of us. Ada Loveless, Hedy Lamar, and Jean Bartick were some of the early pioneers of computer science. Women have been a part of this industry since its birth. Women aren't new to the tech world. We've always been here. So why are we still struggling to fit in and succeed in the tech workplace? Once upon a time, computing was seen as women's work. Programming was seen as similar to running a telephone switchboard. In the first half of the 20th century, programmer had a negative connotation. Computers involved a lot of manual labor. When you see those old paper time cards that people had to punch, men didn't want to do that. They thought it was secretary work. So what happened? Well, the industry exploded. By the 1960s, there were more jobs than programmers to fill them. Two male psychologists, William Cannon and Dallas Perry, were hired to find recruits who would enjoy programming. So they profiled 1,378 programmers. And only 186 of them were women. They came to two conclusions. The first was the people who liked programming and enjoyed solving puzzles. The second, a problematic conclusion, was that satisfied programmers don't like people. That personality test became widespread and created the stereotype we still have today, which excludes women. This test made the reign of the nerds a self-fulfilling prophecy. The industry self-selected for anti-social, mathematically inclined males. And that created the perception that programmers should be anti-social, mathematically inclined males. So what happened to women in computer science? This graph from NPR shows that there was a peak around 1984 and then a really steep decline. Around that time, there was a huge demand for computer science classes. So universities like Berkeley and MIT started restricting admission to students, only those with a 4.0 major, but only those with a 4.0 could major in electrical engineering or computer science. These barriers to entry pushed even more women away. Computer science became not only nerdy, but elitist. The only way to become a programmer was to already be a programmer. By 2001 to 2002, only 28% of all undergraduate degrees in computer science went to women. By 2004 to 2005, the number had declined to only 22%. Data collected by the Computing Research Association showed even fewer women at research universities like MIT. Women accounted for only 12% of undergraduate degrees in computer science and engineering. In the United States and Canada, granted in 2006 to 2007, down from 19% in 2001 to 2002. Many computer science departments report that women now make up less than 10% of the newest undergraduates. I know that when I was in school from 2004 to 2008, I was often the only woman in my computer science classes. So why did this happen? We talk about the pipeline a lot. A lot of people blame the pipeline. They say there aren't enough women in tech. There aren't enough women graduating. There aren't enough women applying. But is that the root cause? Women need to see other women in tech careers to want to sign up. It's not enough to just get girls into computer science and coding classes. There have to be successful, visible role models in tech. I know that I was interested in computers all throughout high school. But I thought it was a thing that weird nerdy guys did. I never thought it was something that I could do or that I wanted to do or that I would be successful doing. Or I thought I would just be too creeped out by the nerdy, stereotypical guys in the computer science department. And then I had a really amazing role model. I had an amazing faculty member who I had in a basic class that I had to take who took every female student under her wing that she possibly could and said, look, I am a successful woman in computer science and you can be too. And she was really successful at that. And we loved her. And I'm still in touch with her today. And I think it's women like that that make a huge difference. We need more visible role models. Because right now, what women are seeing in the tech industry, scandal, wage gaps, hiring bias, bro culture, and the few highly visible successful women are just not cutting through that noise.  In 2015, Intel set a goal that 40% of all new hires would be women or underrepresented minorities. They got 43.1%. And Intel chief executive Brian Krasanich said, if the pipeline was such a big problem, I would have come back as a failure there. So if one huge tech company can do this in one year with one goal, anyone can. Don't believe that the pipeline is a problem. So getting women into tech, one problem. Keeping them is another. The leaky bucket is the name given to women trickling out of the industry. The tech world needs to attack this issue on both fronts, attraction and retention. Getting women into tech is pointless if there's no effort to retain them. But we can't make assumptions as to why women leave. So why are women leaving? One of the main reasons is pay. Women are actually more likely than men to leave because of pay. And the other reason is bias. Why don't women leave? They're actually not leaving because of motherhood. Companies assume that women of child-bearing age are leaving because they're having children. But that's really just not the case. Most women who leave their job stay in the workforce. So when it comes to retention, don't focus your retention strategies on motherhood. The difference between men and women's reasons for leaving are really highly overstated. So don't assume. Talk to women in your company if you're losing women. And develop data-driven retention strategies. I read a really fascinating study called the, it was called Gender Bias Against Women of Color and Science, and they talked about the five biases pushing women out of STEM. It was the Center for Work-Life Law. They studied decades of gender research as well as doing some of their own studies. And they identified five key biases encountered by women in STEM fields. So the first one, they called Prove It Again. So women often have to provide more evidence of competence than men in order to be seen as equally competent. The Harvard Business Review did a follow-up study on this. And two-thirds of women reported having to prove themselves over and over again. They had their successes discounted and their expertise questioned. The next bias they called the Tight Rope. Women often find themselves walking a tight rope between being seen as too feminine to be competent or too masculine to be likable. Women are also more likely to take on administrative tasks or things like kitchen duty and to be looked down on if they don't. I know this one is one I can really identify with throughout my career. I know a lot of other women in the tech industry too.  Do I prove I have to be a girl? Now am I being too girly? Am I trying too hard to fit in? I also identify with the other part of it feeling like I wanna do administrative tasks or clean up after things. Although I don't think I've ever been treated poorly for not doing them, but I know this is a really, really tough one, especially in our industry. The third one, the maternal wall. So when women have children, professional opportunities dry up. There are strong assumptions that when women, that women lose their work commitment and competence after they have children. But women who remain indisputably committed to their work are then penalized because people assume they're not good mothers. So it's a catch 22. People assume you lose your commitment to your work when you have children, but they look down on you if they assume you are prioritizing your work over your children. You literally can't win. A quote in this study from an Asian American immunologist said, I have to fight very hard to show that I am a good scientist as well as a good mother. A black microbiologist said, there is an assumption that your career is more of a hobby than a career. And I think these quotes are just heartbreaking. The next type of bias that they identified, they called tug of war. Sometimes gender bias against women fuels conflict among women. Women often exhibit the same bias against women in traditionally masculine careers that men do. And when women have experienced discrimination early in their careers, they may distance themselves from other women. And I have to say that this one really bums me out. Women have to deal with, and we just have enough to deal with, and we should be in a position to support one another. If you're a women developer or anyone, of course, who's met another woman and you've questioned her technical skills based on her gender, or been surprised that she was a developer or an engineer, I want you to sit with that and really think about what made you think that or assume that, and think about how you can change your thinking in the future because that's really important. You want others to think of you the same way that you think about them, right? I don't want to judge women for thinking about this because, like they said, a lot of this is based on how we've been treated, but we just have so much to deal with that we should try to make sure that we're lifting each other up as much as we possibly can. And the last kind of bias is not a surprise, it's sexual harassment. In a study of 60 scientists, all women of color, 34.5% reported sexual harassment. A 2014 study of scientists found that 64% of 666 respondents had experienced some sort of sexual harassment while doing field work. Sexual harassment is also a major problem in science fields at university. Kelly Ellis, Susan Fowler, Julianne Horvath, Katrina Lake, and Whitney Wolfe. What do all these women have in common? They are just a few of the women who have spoken up about sexual harassment at tech companies like Google, Uber, and GitHub. Sexual harassment is a huge and ongoing issue in the tech industry, and we are just starting to pull back the curtain. This quote from Katie Levinson in Lean Out really nails the culture of hushing up and keeping quiet. We don't call anybody who talks about sexism in tech a whistleblower. But there is a revolution happening and women aren't keeping quiet anymore. So what needs to change? Break the silence, keep talking about it, call it out. We also need clear codes of conduct. We need clear reporting and follow-up with no retaliation. Most women don't report harassment, some don't want to take the risk alone, some fear retaliation, some don't know who to report it to or they don't think anything will be done. Some don't want to end someone's career, they just want to stop the behavior. We also need a very serious personal commitment at all levels to stopping this. Everyone at every level in every organization needs to be committed to this. Also in the university level when it comes to illegal discrimination against parents and pregnant women which has also been reported in the science field. Universities need to take this really seriously. If federal funding is involved, Title IX also covers women against this discrimination and harassment. So some tools for change. A really interesting site called Know Your Title IX. It empowers students to stop sexual violence. It has a lot of really good information. Another really interesting thing is called bystander training. Sexual harassment training as we know it is mostly useless. At worst it enforces gender stereotypes. Bystander training, on the other hand, empowers people to do things like call out offensive jokes, ask a coworker if they're okay after being the butt of a crude joke or offering to accompany them to HR. It helps to create a dialogue and tell people that it's okay to speak up. We also need to promote more women. Research has continually shown that companies with more women in management have less sexual harassment. It's partly because harassment flourishes when men are in power and women aren't and men feel pressure to accept other men's sexualized behavior. So speaking of promoting women, let's talk a little bit about hiring and also imposter syndrome. Women are often less confident than men and their technical skills. Men apply for a job when they meet only 60% of the qualifications, but women will generally only apply when they meet 100% of them. Oops, just trying to make that gif happen. We do not all have the confidence of Olivia Pope, unfortunately. But is it just a perception and confidence issue? Imposter syndrome is very real, but women often do need to meet more of the qualifications to be hired than to their male counterparts. A 2011 McKinsey report found that men are often hired or promoted based on their potential and women are hired or promoted based on their experience and track record. So what do we do here? Do we just have to get better? Do we just have to be so much more amazing at everything that we do? Do we make sure we have a genderblind application and hiring process? Do we hire for skill? Do we hire for merit and ability? Let's talk about that one and talk about meritocracy. There are some words that come up when we think about meritocracy. These are all quotes. A lot of them from companies like Uber and Google. Super pumped. Meritocracy and toe stepping. I did not make that up. We always see words like rockstar, guru, ninja, unicorn. People refer to themselves as a self-made great man. They say, let's get people like us. We don't make mistakes. Always be hustlin'. He's a top performer. He's the chosen one. Well, what's the problem with getting people like us? This is a quote from the book Lean Out by someone who's sort of an avatar called fake Grimlock and they say, if you let someone build tomorrow, tomorrow will belong to someone else. They will build a better tomorrow for everyone like them. So meritocracy, at its basis, is selecting people on the basis of their ability.  It doesn't work. The term became popular after it was used in a 1958 satirical essay by Michael Young, which pictured the UK ruled by a class that put aptitude and intelligence above all else. It results in hiring people like you because you view their ability through your lens and ability is a function of education and privilege. So say you're an MIT-educated upper middle-class white man who majored in computer science. You're probably going to think very highly of the ability of another MIT-educated upper middle-class white man who majored in computer science, especially compared to, say, a working-class woman of color who went to night school and got her associates and then self-taught with online classes. If they can code the same, what's the difference? They have the same ability, right? But you, MIT-educated upper middle-class white man are thinking, well, the quality of his education is higher, but now you're judging his privilege and his background and not his ability. So you think, well, what's the worst thing if we only hire a billion programmers as long as they come from all walks of life?  And people should never be hired or judged on one characteristic. It doesn't matter what it is. And a lot of startups say, okay, but I need to hire people now, so I'm just gonna hire people I trust.  Yeah, okay.  Nobody, white dudes. Because guess what? Being a woman of color and tech is even harder. In a study of women in STEM fields, black women were 76.9%, more likely to report having to provide more evidence of competence than others to prove themselves to colleagues. Again, Olivia Pope. 48% of black women and 46.9% of Latinos report having been mistaken for administrative or custodial staff. This is from that same study of women in STEM. These are scientists being mistaken for janitors because of the color of their skin. This is messed up. Black women also reported being openly confronted by negative racial stereotypes. The postdoctoral advisor of a biologist, quote, turns to me and says, hey, do you have any family on drugs or in jail? This is a woman who is working on her PhD. Let's move on to something else. Work-life balance. Slack has it, right? They have signs in their office that says work hard and go home. That should be the norm. Why is that not the norm? That's what we should do at work. We should work hard and go home. Women are especially penalized for having to leave at normal hours because women are more likely to be primary caretakers of children or relatives and they're penalized for being seen as less committed to their jobs. But a healthy work environment recognizes and encourages a life outside of work did a whole talk on work-life balance last year. Along with work-life balance, we also need good benefits. Big tech companies now offer many benefits that are aimed particularly at women and mothers. Childcare, parental leave, egg freezing, IVF, surrogacy, paid parental leave for partners, concierge and convenient services for new parents, fancy. Shipping breast milk home when traveling for work. This is a huge step up, but equal pay at 40 hour weeks would be even better. Especially because many of these benefits kind of seem aimed at delaying motherhood for the sake of career or getting moms back in the office a little faster. So let's talk about that pay. By the way, I found out today that it's equal payday and I didn't know that until I'd already left the house. So this is very, very on point. Hired.com did a study called the State of Wage Inequality in the Workplace. So here's one. On average, women are offered 1% less money than men for the same role in the same company. So here's a graph showing that there's a high variation in the salaries offered to candidates for the same job, the same company. And the number of times women are offered less than men is almost two times greater than the number of times that men are offered less than women. Not cool. They also found that 63% of the time men are offered higher salaries than women for the same role at the same company. On average, these women, these companies offer women 4% less than men for the same role, with some offering women up to 45% less for the same role. Age also plays a role. Women ages 41 to 45 get paid 90 cents on the dollar compared to men. Women ages 20 to 25 are doing the best at 97 cents on the dollar. Naturally, experience plays a role in salary. If you think of 13 to 14 years experience, the wage gap really wouldn't be a thing anymore, right? But women with 13 to 14 years experience earn 92 cents on the dollar compared to men. I have so much to look forward to. And when broken down by race and gender, black and Hispanic women made the least at 90 cents on the dollar compared to white men. They also did a state of salaries report, where the average preferred salaries and then offered salaries by race. And this shows a significant difference in the salaries asked for and received by candidates according to race. With black candidates receiving the lowest, lower still were candidates who identified it multiracial with an average preferred salary of just 123K and an average offer of 128K. So, what can we do about all this? How about that? Equal pay or equal work. We can also hire women, find them, recruit them, do the work, stop sitting around blaming other things. Don't blame the pipeline, the leaky bucket. Don't passively wonder why there aren't women in your company. Talk to women. Ask women to look at your job descriptions. Avoid loaded and masculine terms. Don't ask for ninjas and gurus and rock stars. Don't have beer in your office. Stop with the ping pong and the happy hours. Women don't want to play ping pong and drink beer in the office, okay? It's not that we don't like ping pong and beer. It just makes us think that it's a bro culture and that's not what we want. I actually recently wrote a blog post where I asked all of the women in our company like what made you want to work at Chromatic? Like what drew you in and what are things that have made you not want to work at other companies? And one of the things that a lot of the women said was one, that we didn't say we wanted ninjas and gurus and rock stars and that we didn't apply to other companies because they did want ninjas and gurus and rock stars. That language matters and it gives you an impression of the company and its culture and what's going on there. And you also really need to assess and evaluate your culture and find out if it's friendly to women. And if you don't know, you have to ask women. If there aren't women in your company, you have to seek some out and find them and talk to them. Talk to women in other companies. Talk to consultants. Talk to them. We're not scary. We're here. And we will answer your questions. You have to do your research. You have to approach this as any other business need. Making your company diverse isn't just some pipe dream. It's vital. It's how your company is going to grow and have different voices and have different perspectives and be intelligent. It's so important. I just can't stress that enough. I feel like there are so many companies out there that just sit around going like, where are the women? And they don't do anything about it. So I want companies to just reach out and talk to people and talk to women and put it up there with all of the other important business needs that they have. That's my last slide. I finished a little bit early. If there's any questions or Q&A, I'd be happy to answer. Thank you. Hi, Dave. I have a question. You were the first female to join the Chromatic team at the time, all white men. What advice would you give to a woman looking to enter into a team that might be in the same position? I did give it a lot of thought. I researched the company pretty heavily. I googled you guys. I did. I wanted to look and see if there was anything that gave me pause about you guys as individuals. What did you guys... Your website looked great. Everything you said on your job listings already... There was nothing there that were red flags as a company. Then I wanted to see as individuals what you guys seemed like was... What kind of things did you say on Twitter? Did you seem like total dude bros? Were you into social justice? What kind of things? Everything that I saw about you as people I liked and when I talked to you... We addressed it. We brought it up in some of our first conversations and that made me comfortable. When I came on full-time we spent the entire dinner talking about me being the first woman and me being an ambassador for the company and that made me really comfortable and that you guys were passionate about diversity and that you were excited to bring a woman on. That made me really happy. Those were all really good signs and I knew that you guys had been... It showed me that you were trying and that you were happy to be increasing the diversity and this wasn't just like, oh, yeah, we're just a bunch of white dudes and now you're just a woman that you were happy and this was purposeful and that just all made things feel really good to me so I would encourage other women to do that. Have the conversation see that these are individual people that you want to work with and somebody has to be the first woman unless it's a woman led company in which case you have to be the first guy so do the same thing, I guess. Does that answer your... Okay. But now we have five women which is pretty cool. Anything else? Anybody? You know, when we're trained in a very serious way and so then when the workplace has been shut down so saying that we don't want we don't want to be right here with the camera we don't know how to let down we're afraid that we'll be perceived to let down our guard as not as competent as we want to be seriously human no matter who those things are still being considered. It's a sort of a vicious circle that we kind of call it because so I think your point about talking to women and what I heard you say to the folks who hired you that they're willing to have that conversation right up front and that you're able to say things like the whole thing probably doesn't work for me here's what I need to have fun that's really powerful and that's something that's not happening I was saying that if you have that conversation immediately a lot goes up and if you learn how to deal with it so I think that you will get away with some of the things that sort of fit these persons will work a bit well let's talk about work that out a bit and you should not but it's great. Now I totally agree I think there's also another layer to that of companies that tend to have their level of serious might be like a different you know the way that they see themselves and the way that their culture is is something that is often just not as friendly to women it's not even just about fun but they tend to also be the cultures that they don't take sexual harassment as seriously and when people are drinking in the office especially during the work day that's not going to be as safe as a place to be for anybody so I think a lot of women see that as a turn off not just because of the fun aspect but like are you taking the things you need to be taking seriously seriously or are you all just like yeah let's just have fun and bust out some code this is just so I think especially that sort of start up vibe where it's like nah we're not a business we're just like some dudes having fun and I think that that's that can be a real sketchy area for women oh yeah sorry the answer might be obvious I mean Google indeed all these things LinkedIn but where would you say is the best place to find women to hire I think that's a really good question there are a lot of there are a lot more job sites that are for women specifically these days I haven't been looking for a job recently so I'm not so I'm not sure I know that a good one that I recommend to people who are looking for remote jobs is called power to fly because they specifically try and place women in like flexible remote jobs but I think that that is a growing like a growing industry of like job sites for women but if not there need to be certainly more of them just step up I will say first that I'm upset that so many of your anecdotes deeply relate to my experiences in the tech workplace so I can definitely feel you there but I just wanted to ask so I work in a non-tech environment and I'm the only technical person there I'm also up against a strong age difference in my supervisor lineup about 20 years difference in the youngest person there so I just I want to ask about your questions for advocating for your technical needs resources professional development when you have non-technical people above you I think that can be a really tough one and depends a lot on your relationship with like your managers and your higher ups I had dealt with that one once and what had worked for me was I had a really non-technical direct manager but I had a better relationship with his manager sometimes just wind up having to go a little bit over his head even though that was uncomfortable but he understood my needs a lot better because he just came from a more similar background so sometimes just finding someone else in your organization who can relate to you a little bit better can be the best thing like even if it's not the person that you're supposed to and that can really depend on your organization but just finding the right person but I think also just finding a way to back up your needs and saying like and also not asking saying like I need this because data and I need to do this because data and instead of just saying could I, may I and just putting it out there especially because you're younger and asking permission might make you feel like you're a kid just saying like can I please do this and instead just stating your needs and can help you feel a little more you know just putting it out there and saying that it's what you need I have a suggestion in your review process can you ask specifically to have a line of someone who's doing the same job as you in another area give feedback because we even at Penn State you know they don't do anything but they do it in the review process they do allow feedback from a colleague on the same level maybe that for the review process of it all yes exactly like my title as multimedia specialist too so I could say somebody from another department like my friend Nikki back here since we're doing the same type of work she could review me on the work that I'm doing the technical work so then my boss can do that thanks microphone is for tall people so yeah I've been in tech since about 2000 and the majority of my career has been with companies run by men working mostly with men which is not unusual I currently work for a company called communicate health and it's actually a women owned women run company we're not just a web development company though we do communications for the public health sector so we have researchers writers developers designers so we have a kind of small development team within our larger team but even on our development team in a company run by women we have seven developers and only two of us are women which is interesting so I know we're working are we're actually dedicated to trying to get a more diverse group of people in our in our employees now and that's something that the company is actually like striving for so what advice would you give a company trying to find more women technical women like where do you find them how do you go about finding them like actual real world ways of recruiting them and seeking them out I think one way would be to get involved and look at like you know if you're like local groups like you know who's teaching you know say if you're looking for a javascript developer who's teaching the local girl development javascript class you know who's teaching the local you know like ladies learning code class or things like that like reach out and you know see who's doing that like who are the local subject matter experts in the thing that you want to do like look on meet up look on twitter and like actually seek them out and like recruit those women directly and find them like that's that's what I would do like that's what I you know I was the I'm the track share for the back end development team and like I actually went out and like searched for women in you know the space that I wanted to and send the messages and was like hey I would love for you to submit a talk and that was a much broader thing but I think especially if you're in a location yet it's even easier like find organizations and find the women in them and or just go to meet ups and talk to women and put your name out there and things like that hello I mean for me I often think about like impact and intent of our privileged positions like I'm a company you know whether we have a beard I'm a dude and I always tend to think about just you know what position can I be in to really help people and you're kind of talking about privilege before metaphor is to that we use are very powerful it could be a sports metaphor to contribute to like bro culture the pronouns we use a lot of latent things that we do that are implicit that cast value in meeting we really need to kind of look at that so something I tend to think about less of a question I'm curious what you think yeah I totally agree and also like coming from a place of privilege like you have the best vantage point to call that stuff out because the other people in the room are going to be less comfortable like you know someone's saying like hey bros or hey dudes or whatever you are the strongest person in the room to call that out and say hey maybe don't say that or if someone says an inappropriate joke or anything like that like you have the privilege the most comfortable privilege to call that out because you're not going to get flack or if you do it's not going to affect you in the same way that it's going to affect someone else so the people with the privilege are the people who have to do the calling out yeah come up this might have been answered earlier so I apologize but I don't think there's much difference between the needs of a man and a woman but I'm just curious if there are any specific needs for females is it more because of historical values historical reasons that causes these values or is it more intrinsic for example like safety in a workplace historically has been a little bit more difficult for female than for men I don't think that the needs are different I think that we've made the reality different so we need to change that reality so that we're facing the same thing like you know our safety we both have the same needs for safety but our reality is different because we're treated differently you know like because you know in general women are preyed on by men in a way that men aren't preyed on by women so men don't have safety needs that we that women have so it's not it's not that our needs are different you know like or in terms of pay like we don't have different needs for pay but we aren't valued in the same way that men are so we need to change our reality to match the reality of men does that make sense does that answer your question okay I think there was someone else over here who was about to stand up and then oh come on maybe this is like a bigger and harder question but I couldn't help but notice that there's more women in this room than men and this talk seemed like it would be a really important one for all the men in our community so how do we get more men in a room like this do you have any ideas for that I think that's a tough one and I'm happy to see that there are men here I know honestly and I was happy that this was in the business track I had some discussions with the tractors and we decided to put it there and one of the reasons was to get more men in it I had originally we thought about putting it and maybe being human and we thought no let's you know this is a business related discussion this is about the workplace and maybe we'll get some more guys in here and I'm happy to see that there are guys in here but no you're right and I'm not sure how to get more men to just keep trying go for it come up to the mic I was just going to speak to that because we have a few diversity initiative groups at Four Kitchens right now and there's like four different groups who are kind of tackling different issues like marketing like how do we fix our marketing to like be more inclusive how do we fix our interviewing to be more inclusive etc and when we opened up the groups to volunteers it was immediately like all of the women who volunteered and so I just called people out and I was like we need more men like there's also diversity issues on that end if we don't have like you guys supporting us and being a part of the conversation so call people out and just like if you have your team say there's a session I'm going to you should come with me because I know that you know it's important to me for me to have that support so that's my advice hey I'm being represented by the guys on my team here so ladies bring your male team members to the diversity sessions just saying is there anyone else we've still got a few more minutes yeah come up part of the talk but I just wanted to say that I think part of the thing that separates us puts us in a different place from men is that things that are expected of men aren't expected of women we're raised in a different way where we're not challenged you know things aren't expected so we're forced to do these things but also part of that is us sort of supporting this male stereotype where we're expecting men to know things we're going to them for help we're asking them to do like certain manly things and we're treating them as if we expect them to have their manly role so I think part of it is not just thinking about how men can treat us but how we can treat men and not sort of like supporting the stereotype that they have to be doing their men things because it sort of reinforces the you know the need for them to feel like they have to act in that way where they have to act like they know everything and they have to act like they have to take care you know do things for women and you know not expect things of women so I just want to mention that part of it is you know thinking about the way we treat men because it's also difficult for them to have these expectations that they've grown up with that we haven't necessarily grown up with and yeah just like kind of keeping that in mind also yeah I know I've definitely thought about that one before and there's been times where I've like asked a male co-worker to grab my suitcase asking him because he's a guy or is it because he lifts weights competitively hmm so sometimes you can overthink that one too but no that is definitely a good thing to keep in mind is there anything else one more time that just made me think of the whole issue of toxic masculinity and is something that I feel like needs to be combated in order to make things more equal for women and that includes equal parental leave parental leave for dads too so that it's seen as not a women's issue but a human issue which is a big topic for me right now yes I like your future geek shirt yes this is not a question but just on terms of hiring and job posts there are tools out there that you can use you just put your job post in and it'll read through you'd be surprised what other words get picked up so I think it's a good idea to write as more masculine coded versus feminine coded so I always make sure to run my job posts through this just to make sure that they're more feminine coded because it's shown that like men are still going to apply to more feminine coded job posts than women are so women are going to mostly apply to feminine coded job posts so just things that come up are like things that sound super competitive will generally deter women so I can never remember the URL for it but I usually find it so put your job post in and that's a good place to start if you don't have any women in your organization to review your job posts it's an easy place to start what is it T-E-X-T-I-O dot com okay I think it even flags things like outgoing because you know some women don't want to be like hey team how's it going every morning really who does anybody else got a couple minutes run up I missed some of your presentation too so if you covered this already apologize you talked earlier about like a boozy culture being something that might want to be avoided from a discomfort level too if you're looking at making it a more inclusive culture kind of from the top or at least from all of the different groups levels I don't think it's arguable that there tends to be different cultures depending on the disciplines in the technology company right sales marketing development that kind of thing too so besides not having a boozy culture what else do you think might be a consistent thing that we could either proactively do or proactively avoid that would still be applicable whether a female was in sales or marketing or development or anything like that where it's maybe that would be a great fit for a development manager to be concerned with but not necessarily something that would I mean you want a sales person to be outgoing right so you wouldn't necessarily want to avoid that in a job description for a sales person so what are the other things that maybe we can do from a consistency standpoint that would apply to all these different groups and cultures within a larger organization do you mean in terms of job listings or in terms of like other like extracurricular type yeah extracurricular type or just how communications and are done throughout the day stuff like that I didn't necessarily mean job description but when we talk about even job descriptions right I mean there are some very specific roles within companies and again you don't want a sales person to not be outgoing for instance right so and something might make any developer frustrated whereas it would be nothing to any sales person so but there's got to be consistencies across those groups in general I would think well I mean obviously I think it's fine to use a term like outgoing for a sales person I was thinking more like you know tech culture really off-putting term to a lot of people one thing that I find that I see in more and more tech cultures is like a lot of like hey we go hiking and we do yoga and we have all this like in-house like working out and that can be like you know that can start to border on like the ableist kind of thing and you know not everyone can or wants to work out and again that can also be kind of veering into like bro culture of like hey we all want to like get pumped in the morning and you know not everyone's into that so you know obviously offering workout classes is one thing but when you start to make them feel compulsory or feel like something that everybody in the company has to do like that needs to stop and I'm seeing a lot more of that not just in the tech culture but like all over the place if anyone reads ask a manager it's a really fantastic site but there was just a question the other day about I think or maybe I was reading an old one about like mandatory yoga and I was like why would you do that like that's the worst thing I've ever heard of because the person was saying that they like couldn't do it so things like that like actually I mean anything that's mandatory that any mandatory fun should never be a thing like no one should ever have a mandatory fun but I mean thinking about anything that can make people uncomfortable like you know the only thing that should be mandatory is you know doing your work and communicating with your co-workers and your clients there really shouldn't be anything else that's ever mandatory and you know things like you know you should never have a drinking culture it's you know obviously you know like we got to do a book on we have a few drinks but some people don't drink or like I only have a couple drinks because I get migraines and no one ever pressures anybody and it's fine and you have to make sure that that's a totally okay level like it's okay to not do things and it's okay to just be yourself and that needs to be like the level that it is and things like that so I hope that answer your question I don't run a company so I can't answer like how all departments should be run I would say it's important to men and women but I know what's really important to me is work-life balance so anything that like verbally promotes that you know oh it's okay you know she has this great thing to go to like just you know saying that we you know we should be able to to turn off so okay well it's almost one so I think we can wrap it up thank you you had an amazing question


Hey guys, today are really quick. I want to talk to you about the Vitamix Mother's Day sale that begins April 17th and goes for the month through May 17th. Everything's on sale. All new models, the non-reconditioned ones are on sale, so take your pick. Whenever one you want, you're gonna get the Mother's Day, holiday, Christmas, they're the best prices all year. Since they're all on sale, you could take hours and days and you probably have that these days to look and research. We have a video, which Vitamix Buy in 2020, a guide to read through on our website, or you can email or call us. We'll help you. We'll give you a personal recommendation. If you just want to know what to get right now, most people will fall into two categories. The best one you can get, which is the A3500. It's the least loud. It has the most features. It's the one we have, one we use every day. It's actually the most popular machine that people get through us. The best deal and the actually the best dollar-for-dollar, the best value is the Certified Recondition V1200. A great model. I'll put a link to the video that we talk about it. So either the A3500 or the Certified Recondition V1200 is probably going to be what you want. Once you pick, use one of our links from the description here in the video or on our website. That will get you the Mother's Day sale price, free shipping, and it'll tell Vitamix that we sent you. So we'll get credit for the referral. You already know this, right? Then be sure to notify us of your order. You can fill out this little form that says register VIP, notify us. That's how we can track your order, make sure you're well taken care of, and then send you our ebook, which has 50 recipes. We'll also get you a few onboarding tips and tricks for getting started with your Vitamix. We want to make sure that you get the most out of this thing. All orders, they come with 30-day returns, and then also the full warranty. A lot of warranties out there are kind of BS. This one's more like an insurance policy. Anything happens, they'll send you a shipping label, send it in for service, they'll send it right back to you, it'll be up and running in a few days. Guys, you might be thinking, which accessories do I need? We find people are most successful when they take minimalist approach when getting started with the Vitamix. Get the machine, use what comes with it, and then down the row you can decide, yep, I want to get another container or some attachments. They go on sale periodically through the year, take the minimalist approach. And guys, subscribe to our channel if you haven't already. Hit that bell to get notified. If you appreciate videos like this, hit that thumbs up. Thank you so much for watching. Happy Mother's Day to the mothers out there, and we'll see you in the next one. Bye guys. This is, I normally come to work to my office with green juice. We're living in quarantine life right now, and we don't happen to have most of the ingredients for green juice. We have some greens, but we're missing apples and oranges and bananas. So I made a protein shake, and this is full of ingredients that we kind of just always have during the freezer or during the cabinet. And it's pink, and I like that. From Mother's Day it makes you think of when baseball players wear pink gloves and their bats are pink, and it just kind of makes me happy to see and think of springtime and baseball season. So happy Mother's Day, happy springtime, cheers, and thanks again for watching. Bye guys.


Hello and welcome to the session. In this session we discussed the following question which says use the vector method to find the values for lambda and mu so that the points A with coordinates 1, 1, minus 3, B with coordinates 7, 6, minus 5 and C with coordinates lambda, mu, minus 1 are collinear. Before moving on to the solution, let's recall the vector equation of a line that passes through two points with position vectors vector A and vector B is given by vector R is equal to vector A plus lambda into vector B minus vector A for some scalar lambda. This is the key idea for this question. Now we proceed with the solution. We are given points A, B and C. A has coordinates 1, 1, minus 3, point B has coordinates 7, 6, minus 5 and point C has coordinates lambda, mu, minus 1. We take let vector A, vector B, vector C be the position vectors of points A, B and C respectively. Then we have vector A is equal to i cap plus j cap minus 3 k cap vector B is equal to 7 i cap plus 6 j cap minus 5 k cap and vector C is equal to lambda i cap plus mu j cap minus k cap. So now the vector equation of the line A, B is given as vector R equal to vector A plus T into vector B minus vector A for some scalar T. That is we have used this key idea where we have vector equation of a line that passes through two points with given position vectors vector A and vector B is given as vector R is equal to vector A plus lambda into vector B minus vector A where lambda is some scalar. So in this way we have found the vector equation of the line A, B. Now substituting for vector A and vector B we get vector R is equal to i cap plus j cap minus 3 k cap plus T into 7 i cap plus 6 j cap minus 5 k cap minus i cap plus j cap minus 3 k cap. And so we get vector R equal to i cap plus j cap minus 3 k cap plus T into 6 i cap plus 5 j cap minus 2 k cap. Further vector R is equal to 6 T plus 1 i cap plus 5 T plus 1 j cap plus minus 2 T minus 3 k cap. So this is the vector equation of the line A, B. Now if we have that the line A, B passes through the point C then we have vector C is equal to 6 T plus 1 i cap plus 5 T plus 1 j cap plus minus 2 T minus 3 k cap where we have T is some scalar. Now substituting for vector C which is lambda i cap is mu j cap minus k cap. So we get lambda i cap plus mu j cap minus k cap is equal to 6 T plus 1 i cap plus 5 T plus 1 j cap plus minus 2 T minus 3 k cap. Equating the components on both the sides we get lambda is equal to 6 T plus 1 mu is equal to 5 T plus 1 and minus 1 is equal to minus 2 T minus 3. Now that we have minus 1 is equal to minus 2 T minus 3 this means that 2 T plus 3 is equal to 1 or you say that 2 T is equal to minus 2. So from here we get T is equal to minus 1. Now putting T equal to minus 1 in this lambda and mu we get lambda is equal to 6 into minus 1 plus 1 which is equal to minus 5 mu is equal to 5 into minus 1 plus 1 which is equal to minus 4. So that we get value for lambda is minus 5 and value for mu is minus 4. This is our final answer. This completes the session. Hope you have understood the solution of this question.


Yeah, we on boss talk one-on-one. When you, when you, when he was down there and he gets ready to come home, you know, I know now this is where the celebration begins, but I know it's hard as y'all go through it. Oh yeah. Bumpy did a splendid job doing the Free to Pimp C campaign, man. You didn't know it was that big until after it was over. I knew it. I liked it. I seen it. I loved it. I seen it and I respected it. It was going hard too. He was going hard. He was ripping everybody. Everybody was trying to get him on verses and everything. Everybody was getting it. But every time he ripped them up. Yeah, Free Pimp C. Yeah, Free Pimp C were killing them, man. The T-shirts and everything. It was just on point the way he done that. And I always, I always think about that time because that was a time in itself where, and then I felt so bad when Pimp passed because I knew it was like a double M in a burn because he already had went through so much, man. I thought about that as well. You did, did you? Oh my God, yeah. And you can kind of hear it in that next, what is it, not the Trilogy, but that two-trill album. Yeah. He wasn't passionate on that. No, no. He even alluded to it on that show. Like it was tough on everybody, man. But like I said, for him, the way he was campaigning for it and then to not see it come to fruition. Cause he had already, I know in his mind he could see this big picture happening. You don't know. I had just come back. I was going through a divorce at that time, right? And I called him and I said, say I'm going through divorce, bye, bye, bye.  You know, I knew. Come on down here with us, bye, bye, bye. I said, well, man, you know, I got a, I always used to take my vacations from the car dealership. First week of October because of the fare. Yeah. I always did. Yeah. And he said, come on down. And so he said, you can stay at the, my high rise on West Timer. I'll leave the keys for you at the front desk. So I went down there. And when I got there, he was still there. Come on upstairs. He was still there. So we went to Port Arthur, I stayed in Port Arthur like two, three days, you know, that was an adventure. Wow. You know, yeah, that was an adventure. I got to stay, sleep on his couch for two, three days and Queenie, not Queenie, I'm true. No. Hey, Queenie. Yeah, man, that's bonus. Yeah. Channara, Channara drove me back to Houston. And so that's when I stayed at the high rise then I hung out with Queenie after that. But it was, that's when he gave me the shirt. Yeah. A couple of other treatments and things like that. And then fast forward to Thanksgiving. So that was October and I can, bro, the stuff. Bro, look at that. The stuff that he was telling me and playing for me and the phone calls he was taking in front of me. This is what he was planning before Drake. He was planning a double CD. Okay. Half rap, half singing. See, Channara stayed away from that singing early, 92, 93, 94, 94, he didn't like that. I heard him say something about his high pitched voice. He didn't really like it. For some reason, he just didn't like singing. And I remember when he sung, a lot of people didn't know that that was him singing on having things. Yeah, I did. And then when I, before the video came out and I said, bro, that's... That thing was hard too. Oh, yeah, yeah, yeah. So he, I don't know why he shied away from that so much, but by that time he had embraced it. He had started embracing. So he was going to do a song that was going to be produced by Timberland. This is what he's telling me. Okay. Produced by Timberland, him and Justin Timberlake. Hey. Was that going to go to him as well? Man, come on, man. That thing will still be rolling red now.  He said, come on back down here with me. We about to hit this road. I got a radio show.  I don't know. We was doing that for eight and a half years. And I had already, because of the divorce and everything that I was going through, I was planning on retiring because I, you know, I had two daughters at that time. So I got to play that. Yeah, yeah. And it was on Saturday night from 10 to midnight. I'm like, who going to watch my kid? That's it. You know, so I'm a single father at that time. I'm planning, planning this all out. So he said, I got a radio show. And I never met this gentleman, but he says, it's going to be you. You're going to control the boys and P.M.C. and Lil' Duvall.  I said, okay. So he comes down for Thanksgiving. We do the show at Blue. Okay. You know, we chop it up. We stay together.  And then he said, now some, some, some soft. So it showed up for him. He wanted to stay at the hotel. So, so he stayed at the hotel and eight days later.  It was just. Yeah. That was, that was crazy. And he and I were, see Chad was, I did the first interview with him out of jail on the radio. I think I heard that. I listened to it. I listened to it. Man, we got some classic interviews. The most classic one. Was the one where he's standing outside by the fence. I'll show you some pictures from that night. Yeah. But we did some interviews, man. They on YouTube? YouTube, yeah. On YouTube. Some people recorded and they uploaded it on YouTube. Oh yeah. Yeah. And he would just call me anytime he wanted to vent. And he'd be like, blah, blah, blah, blah, blah, blah, blah. And he'd get on, I'd let him get on the earth and just do his thing. He, he did that twice before he did that laughter interview. Yeah. I heard that one too. That is classic. He did one where he really went off on skip cheating. It's on. I think I heard the one tonight. I listened to that one before or earlier today. Yeah. We on Boss Talk 101.


Greetings! This is O513. If you're watching this briefing, you have been asked to learn about SCP-001. Let's begin. Item number SCP-001. Object Class. Unnecessary. Special Containment Procedures. SCP-001 does not need to be contained. In the event of SCP-001 occurring, all personnel, including D-Class, are to be honorably discharged and may do what they wish with their remaining time. All sapient and non-aggressive SCPs are to be released. Any further SCPs which can be decommissioned are to be. All remaining Foundation sites are to be run by the Artificial Intelligence Systems. Specifically, bread instances of SCP-514 are to be released globally. Description. SCP-001 refers to an event occurring shortly before the cessation of all life on Earth. Whilst SCP-001 has not yet taken place, it was discovered through various pieces of information gathered from extra-universal foundations and other similar groups. See attached document 001-A for a list of such communications. It is important to note SCP-001 is not the cause of the apocalyptic scenario, merely a pre-occurrence response to it. SCP-001 is, according to records, distinctively recognized by certain key features. During an SCP-001 event, flowers are observed to spontaneously appear and bloom over approximately 90% of the viable land service of the planet. These flowers are universally referred to as vibrant, bright, beautiful, and or words to this effect. The weather will clear globally, with an ambient temperature recognized as comfortable by the majority of the populace. Air pollution will additionally clear. During an SCP-001 event, the global populace will become aware of the fate of the Earth and of its inevitability. They will also experience heavily decreased levels of violence. SCP-001 will occur exactly 24 hours before the death of all life on the planet. This concludes today's briefing. Remember to like, comment, subscribe, and hit the bell using your encrypted YouTube account, and support the broadcast at patreon.com.


Hello everyone, welcome to the talk today for accelerated and standardized deep learning inference with cave-serving. The presenter today is Dan Song from Bloomberg and David Groove from NVIDIA. We have two agenda today. The first part we will talk about accelerated deep learning inference with cave-serving. The second part we will talk about cave-serving V2 inference protocol. Deploying ML models at scale is one of the most challenges for companies learning to create value through AI. As models become more complex and more deep, this task becomes even harder. As a data scientist or ML engineer, I want to serve standardized deep learning models like TensorFlow or PyTorch with minimal efforts and at scale in a unified way. I may also want to bring in custom pre- and post-processing before and after the inference. Running inference on deep learning models can be slower on CPU. I also want to accelerate the inference by deploying models onto GPUs. GPUs are powerful computer source, but deploying a single model per GPU can underutilize GPUs. I want an easy way to serve multiple models behind a single unified endpoint, which can scale to hundreds to thousands of models. In order to save the computer source, I will also want to auto-scale based on the workload and allow me to scale down to zero when there is no traffic sent to the service. In order to ensure a safe production route, I want to deploy models with zero downtime and can use multiple deployment strategies like shadow, canary, and blue-green robots. In order to solve all these problems, Cavesurfing is an open source project founded by Google, Seldom, Bloomberg, and Microsoft and IBM on the CooFlo. CooFlo was the perfect meeting ground for these companies. Cavesurfing is following all the open source principles to build open and cloud-native serving solutions. Cavesurfing provides a Kubernetes customer source for serving ML models across deep learning frameworks with a simple, intuitive, and consistent experience. It encapsulates the complexity of auto-scaling, networking, house-checking, and server configuration to bring cutting-edge serving features like GPU auto-scaling and canary rows to your ML departments. It enables a simple, pluggable, and complete story for production ML serving, including preprocessing, prediction, and explanation. Cavesurfing codifies the best practice of Kubernetes data guidance. With Knative, Cavesurfing is able to enable serverless inference and automatically scale up and down according to the inference workload. Cavesurfing extracts the common model serving features like request response logging, multimodal polling, batching into a cycle agent so that all integrated model server can benefit from all these features. Cavesurfing departments are immutable. Every new department results in a new version, and the traffic is migrated over to a new version only after the new parts are ready and passing the readiness check to ensure safe production loadouts. It also employs various other route service such as canary and progressive loadouts. The main Cavesurfing components are Istio Ingress Gateway, Knative Autoscaler, and Inference Service. Istio Ingress Gateway routes the external request to the inference service. Each Inference Service pod contains two or three containers depending on the features involved on inference service YAML. The main containers in the key inference service pod are Qproxy, Cavesurfing agent, and model server. After the Ingress, the request first hits the Qproxy, which involves the concurrency limit and timeout. It then goes through an optional sci-fi agent if the request rebounds logging and batching are enabled. The request finally hits the model server for inference. So model server for each framework implements REST or GRPC handlers, and it can load multiple models. So inference service pods can be auto-scaled based on CPU or inference workload with KPA. Cavesurfing 0.5 release promotes the inference service API version from V1 Alpha 2 to V1 Beta 1. It supports a standard inference for TensorFlow, PyTorch, Escalon, SGBoost, and other frameworks. It also provides SDK for users to plug in custom component to while still benefit all the common serving features from Cavesurfing. The V1 Beta 1 API further simplifies and enables a data science-friendly interface and also maintains the flexibility Kubernetes pod template spec provides. In about a few YAML lines, you could describe all the infrastructure you need to get up your models up and running. On the other hand, user can still specify advanced fields we need to. Cavesurfing not only provides a unified interface for control plan, it also tries to standardize the data plan protocol across ML frameworks, which we'll cover later. Since 0.5, we also added a multi-model serving capability to improve the resource utilizations. For TensorFlow models, Cavesurfing uses the TF serving as the underlying model server, which is a flexible high-performance serving solution which supports stable model format and graph depth. TF serving uses the TensorFlow REST and GRPC prediction protocol, which is similar to Cavesurfing V1 protocol. For PyTorch models, Cavesurfing integrates TouchServe, which provides an easy way to serve both eager model and TouchSquib model, which can serve the model without a Python environment. Cavesurfing is currently working with TouchServe to conform to the Cavesurfing V2 prediction protocol. In V1 Beta 1, API, we mainly support three components, predictor, transformer, and explainer. Predictor is a required component, and the fields on predictor is mapped to Kubernetes deployment hot template fields, such as replica, service count, node affinity. On the predictor, user can specify the model framework, which naturally maps to container fields like command, arguments, environment variables. Same applies to the transformer and explainer component. NVIDIA Triton InfoServer provides a cloud-inference solution optimized on NVIDIA GPUs. The server supports multiple deep learning frameworks, such as TensorFlow, PyTorch, Onyx, with both REST and GRPC endpoints. It supports model report with versioning and allow multiple models to run semi-tuners on the same GPU with batching support. On-inference service spec, the storage UI is pointed to the model report, which can contain multiple models. As you can see from spec, we also set the OMP num threads in environment variable to improve the inference performance and reduce the resource contention. As by default, PyTorch spawns number of open MP threads, same as the number of cores available on the node, which could overload the CPU as a default CPU limit for inference service part is one. You can also choose to schedule the Triton InfoServer part on GPUs by specifying it on specify GPU under resource section. You can also choose to add node affinity or tolerance to schedule to a particular node, such as T4GPU. As you may know, Bloomberg's customer service is an essential part of Bloomberg terminal. Ours represented the work very hard to provide the best customer service to our clients by answering questions with high quality and fast speed. The reps are pushed the content to help answer questions in the smart resource window. This use case is powered by our fine-tuned BERT model and deployed onto Caves serving on production. For the fine-tuned stage, the data we are using are categorized and annotated FAQs, which gives us half million question pairs. The problem is formulated as question similarity with inputs of two questions and output as a similarity score. The BERT model is saved using export save the model API which contains complete TensorFlow program, including where weights and a computation. It does not require the original model building code to run, which makes it useful for sharing and deploying. As you know, BERT model at the inference time requires significant computation time and doing this on CPU can be slow, which can take a second, often time can take seconds. This helps them challenge to meet the latency requirement for this real-time use case. Deploying the BERT model to meet all the production requirements is a challenging task. You need to take all latency throughput scaling, house check, save flow out, monitoring, or into consideration. And also how can you scale to serve hundreds of BERT models with your limited GP resources? First, to accelerate the BERT inference, we deploy the fine-tune model on KF-serving with TensorFlow serving component, which expect the tensor inputs. Since the user input gear is the question pair in text, we also deploy the KF-serving transformer for the sentence segmentation and tokenization. KF-serving then automatically was up to call to TF-serving. In this way, you can scale transformer and a particular differently. Why also you can deploy it as a single unit? By deploying the model on GPUs, we achieve 20x speedup. We also experimented deploying the BERT model on a mid-year try-to-inference server, which achieved a similar performance gains. We have tested the performance on a mid-year V100 GPUs with 24-layer FP16 BERT squad model. By deploying the BERT model on GPU, we can get to 15 millisecond, which usually takes three seconds to run inference on CPU. On a single try-to-inference server part loaded with BERT model, the latency increased linearly with the concurrency without batching the request. We can scale up the try-to-inference server instance automatically with KF-serving. As you can see, from the previous table, you get the best performance when container concurrency is low. So here we try to set the container concurrency to one and let K native auto-scaler scale up the inference service part automatically based on the in-flight concurrent request within a time window. By setting concurrent concurrency to one, the inference service part automatically scales up when container reaches to the max concurrency. In this experiment, the perf client generates the inference service request to your model and measures the support and latency over a time window and repeats the measurement until it gets stable value. For example, when the in-flight request is five, it scale up to five parts to handle the concurrency. From the result table, you can see that the average latency for higher concurrency can maintain as low as when concurrency is low. There are some latency spikes which are caused by the part code startup time which includes model downloading and part spawning time. When container concurrency is specified, K native activity is also injected on the request parts to do smart load balancing to make sure container is not overloaded with request more than the concurrency limit. As you can see, the support does not scale linearly perfectly because of the inference service code startup time. One way to alleviate the issue is to cache the model on PVC after downloading from the remote storage so that the models can be directly mounted onto all the parts. GPU autoscaling based on GPU metrics can be hard. K native implements a request volume-based autoscaler which allows you to scale to and from zero both on CPU and GPU. The number of parts needs to scale up the load is calculated by the in-flight request and the concurrency target. If the container concurrency can handle five requests concurrently and your in-flight requests are 50, then K native automatically scale to 10 parts. Batch inference is a process of aggregating inference requests and sending this aggregated request through the DR framework for inference or at once. Cave serving was designed to natively support batching of incoming inference requests. The functionality enables you to use your compute resource optimally because most DR frameworks are optimized for batch requests. You can configure max batch size and max latency on inference service batch section. The CAFSA cycle agent then knows the maximum batch size that the model can handle and the maximum time the agent should wait to fulfill each batch request. On the inference service spec, user can also enable scale down to zero to save compute resource after batch is done. Over the time, you need to both pre and post processing before and after the inference. Cave serving provides the SDK to user to easily implement a transformer which can communicate to model server with a standardized data plane protocol which expects tensoring and a tensor out. Transformer and predictor can also be deployed and rolled out as a single unit. In future, Cave serving also plans to support a few outbacks transformer like text tokenization and image transformation. Cave serving allows chain transformer with any inference server. Here, we can easily swap to use different model server as long as they speak the same data plane protocol. So pre-process handler transforms the raw data into the tensor according to the V2 data plane protocol. And you can see the post-process handler transforms the raw prediction into a response for downstream consumption. The transformer does not enforce a particular data schema. It could be a list of text or images. Deploy a single model on GPU can underutilize the precious GPU resources. Cave serving touch serve Triton or allow co-locating multiple models onto the same GPU in the container. The two requests arrive simultaneously, one for each model. They can be scheduled onto the same GPU and work on both inference computation parallel. Cave serving as a training model, customer source, two-year-old scheduling models onto the inference service at scale, which abstract the way the model placement logic way from the user. The models that are placed on the same inference service customer source can be assessed from the same service URL. With the Cave serving multi-model serving design, we started to decouple inference service and model artifacts. Training models can be placed onto an inference service before it reaches the configured memory limit. Cave serving training model scheduler can spawn new shots for the inference service to host models at scale. It also send health check purpose to each model endpoint to reflect the model deployment status on training models customer source. On training model customer source, user assign the model to a given inference service. User also specify the model story UI and a model framework. So memory's resource estimation is used for scheduling models onto the inference service based on the capacity left off given inference service shot and the schedule automatically spawn new shots to host the new models if all current shots are at capacity. The training model scheduler decides which service shot to place the model and writes the model configuration into the shot config map. The cave serving sidecar agent then reconciles the config map to pull new models and remove deleted models in the model repo that is monitoring onto the pod. The agent then sends a signal to the model server to load and load the model. After model is successfully loaded. On model server, it can then be assessed from a unified service endpoint with the model name specified on the UI or pass. While we develop the multi-model serving, we hit fuel ascapabilitation as well. For a single, it's still ingress gateway. We handle limits about running 2,000 services. What do we want to actually scale to 100K models? Can we support a 100K training model customer resource? We are actually bound by the customer resource limit on SCDD. Even we can deploy 100K models on 2,000 service, we may still hit the limit of the number of squished service we can create on the gateway for routing the models. So the phase two multi-model service to find out these limits. We are excited that KF-Serving 0.5 is released with the V1 beta one APR and Triton with V2 protocol. And here you can check the V1 beta one IFC. IFC doc, multi-model serving is in alpha steps and the next step is to make it a production rating. KF-Serving is an open community. So we love your contributions and you're welcome to join our work by weekly working group meeting. And here's the GitHub and example links. Thank you. I will hand over to David Goodwin for the second part. Hi, my name is David Goodwin. I'm going to be talking about the KF-Serving inference protocol version two. You might have also heard this called the V2 data plane protocol. So before we talk about the specifics of the protocol, let's look at kind of the domain where this protocol is important and is meant to be used. So the diagram here shows kind of a representative way of deploying, I say a containerized inference server. So in the gray box on the right, you see there's this inference server that is capable of performing deep learning or machine learning inferences on behalf of whatever clients there are. And it is inside of some kind of deployment environment. It has access to the model repository, which is holding all the different models that it might need to serve. And there are also maybe some load balancers and some auto scalers, et cetera. Those aren't required. This is just showing kind of a representative example. And then on the left, you see there's some clients. So these are clients that want to directly or indirectly take advantage of some deep learning or some machine learning models. And so they can communicate, for example, maybe there's an ASR, an automatic speech recognition kind of service that they want to use, which will in turn need to do some inferencing. And there could be multiple of these workloads. And again, there can be load balancers and the deployment. The protocol is concerned about how the clients communicate with the workloads and also how the workloads then communicate with the inference services themselves. And the protocol is meant to be a single protocol that can be used in any of these areas. So given that background, for the KF serving inference protocol, why do we need this standard? Well, like most reasons why you need a standard, you want the, say the inference clients, which in the previous diagram were the things on the left that needed to use some inferencing. You want them to be able to talk to multiple different servers or services to increase their portability to make them more interchangeable. And on the server side, you want to allow as many different types of clients as possible, maybe ones that weren't originally meant written even to use your server to be able to use your server or service. And that increases the utility of your service.  So some of the high level of current requirements or support both ease of use and high performance, which we'll talk about some, they need to be extensible. So the KF serving protocol talk about is, has a core protocol, which is kind of the required part, but there's a extension mechanism in it that allows either for in the future, there to be a standard extensions added to it. So optional parts of the core or you could have a server, we'll see the server specific ones, which are individual inference servers can decide to implement their own extensions. And there needs to be both a GRPC and an HTTP REST and a JSON based implementation for this, which we'll look at also in a minute. So I mentioned there's the core protocol and the extension. The core protocol, which we'll go through in some detail is required for all conforming servers. So if you want to, if you want to write an inference server and you want to say that your inference server supports the KF serving inference protocol, you need to implement all these. And they include in points to understand if the server is live and ready to get kind of metadata about the server. And then for all the models that are available on the server, all the deep learning and machine learning models that are available, whether the model is ready and metadata about the model.  And then again, extensions are optional and currently the standard doesn't have any, there's no standard extensions. So there's no optional parts of the core. But we'll see there are, some extensions have been implemented by specific inference servers. And in particular, we're going to look at some, the Triton inference server has implemented some extensions and we're going to talk about those briefly. Okay, so the liveness and the readiness endpoints. Together you could think of these as the health endpoints. And on the server side, there's two different endpoints that you can see whether the server is live and or ready to receive requests. And so for instance, in something like Kubernetes, you can directly use these for liveness probe and readiness probe. And on the model side, there's just a ready and that can be used to indicate if the model is ready to receive requests. And so this is some example here. Again, there's an HTTP restyle version of this protocol and it just returns for these returns that uses the HTTP status code to indicate whether something is live or ready. And here we're asking if the server is live and you get an okay back. So that means the server is live. GRPC side has, you know, it's very similar. Of course it's using the RPC style, but there's a server live and it would turn a bull a true frost response that would indicate whether the server was live. All right, so let's move on metadata. There's another core part of the protocol. And the server metadata is, you know, things like name versions, what extensions are supported. The model metadata is more interesting. You can kind of ask, so what, you know, for a given model that's on the server, you can find out what versions of the model are available. The protocol allows, you know, the server to have multiple versions of the same model. And then, you know, most importantly, maybe what input tensors are you required to send into the model and what outputs can you get back? And for the inputs, you can see here, I'm showing for this example and going forward here, I'm just gonna kind of show the kind of the JSON HTTP examples. The GRPC, protobuf-based GRPC examples are very similar. It's just a kind of different syntax.  So here you can see from the URL we're using, the slash of E2 is slash models, slash ResNet gives us the metadata for the ResNet 50 model. And we see that there's just one version, version one is available. It's a TensorFlow graph def model, which is just the type. So the platform is a type of the machine learning or deep learning framework or representation for this model. That's kind of informative. One of the advantages of the protocol, of course, is that it's the same protocol. You use the inputs and output tensors and you communicate with the servers the same way, no matter what the actual underlying machine learning or deep learning model is. But for informational purposes, it's there. Inputs, in this case, there's only one input and one output on this model. And you get the name of the input, the shape and the data type, which is very important, because this allows you then as a client, you can then, if you want to discover, well, what kind of data shape of the data is expected and what data type, you can get that from the metadata. And similarly for the outputs, it'll tell you what type of output tensors are going to be returned to you and their shape and their data type. So, and now for the, again, the primary endpoint or API that's in the protocol, of course, is for the inferencing. And again, this is just showing the HTTP, the REST style. But here's the endpoint. Again, we saw before V2 models ResNet 50. So for the ResNet 50 model, we want to do an inference. There's that infer there. And in this case, this is the ResNet 50 model. We're going to send in, this is just a picture I picked. And so ResNet 50 is, if you're not familiar, as an image classification. It was a very famous image classification. It was a very famous image classification, deep learning model. And it takes in an image. In this case, you can tell just by the shape. This is a 224 by 224. So a very small image with three channels. And it'll do a classification. ResNet 50 will do a classification and tell you what that image is. And in this case, ResNet 50, it's been trained with the ImageNet dataset, which there's like a thousand different objects it recognizes. And one of them you can see from here. So we, on the left here, you see the post, post this, we send in the input and tell the shape that we're sending in. And the data here, I didn't list all the data because we're going to talk about that. There's a lot of data. So there would be just an array of the, JSON array of the data. And there'll be, you can tell from the shape, there'll be 224 times 224 times three elements in this data array, which is a lot. And then the output coming back, this is what the protocol requires, the format that's required by the protocol that the server needs to respond with. You know, repeating back the model name and version and then giving you the outputs back. And the outputs in this case, you can see the name of it there and the shape. It's just a single element. And in this case, it's a single element that tells you the classification of this item. And it's a coffee mug. So it did quite well actually on that classification. And so here, by looking at this example, we can see, you know, the inference protocol like the other ones I just talked about earlier, it's very simple. Very simple. It's just the necessary kind of basic information that needs to be sent to the inferencing server and back. And all based around having input tensors and output tensors with their data types and shapes. So that's the core protocol we just talked about. Again, the required core parts of the protocol that every conforming server must implement. And in doing that, in implementing that, you'll have a server that can talk to any client that also implements protocol for any type of deep learning, a machine learning model that that supports and send in the tensors, get back the results in kind of a standard seamless way, which satisfies the primary requirements of the protocol or any kind of standard protocol for that matter. So Triton Inference Server here. Triton Inference Server is an open source inference server. I have a reference link later on if you're interested in taking a look at it. But it implements both the HTTP REST and the GRPC KF Serving Inference Protocols. It's a multi-framework, multi-model. It's both GPU and CPUs. So multi-framework, I mean, if you have a TensorFlow or a PyTorch or an Onyx model, again, that all those types of frameworks are supported for inside the inference server. It implements, again, the core. So it's a conforming server. It also has some extensions. And then so briefly I'm gonna talk about extensions here. So the core is completely sufficient that we just talked about is completely sufficient for making an inference server. However, especially in the area of performance, it can be lacking, especially for the HTTP protocol or primary HTTP protocol. So the extensions that Triton implements, there's some per model statistics. So you'll notice in the core protocol, there was no way, say, to find out from the server how many inferences possibly had been performed for a particular model or how long, on average, were those taking that kind of statistics like that. Triton adds an extension that provides that information. Also an extension for model repository management. So not only can you query which models are available on the server, but you can load and unload those models or load new models and unload models. Also as an extension in Triton, there's support for a stateful inferencing, which there's a lot of language models and other models that are implemented, say with an RNN type implementation where there's state in the model. And so you have to, the order of the inferences received is very important. There's support for that for performance. Now the last two items are primary performance bullets. Tensors, as we saw in the example above, you actually send the tensor data is part of the JSON message. And that can be very expensive, sending that over the network. And so if you're communicating to the Triton Inference Server on the same system from another process on the same system, you can instead communicate by system shared memory or by GPU shared memory. And that then you don't have to send the raw data over the, actually over the network, even local network. Or you don't have to encode it, say into JSON or into a GRPC ProtoBuff. You don't have to do that in coding decoding. You can just access it directly in memory. And similarly, the last one, which is just an extension that only applies to the HTTP REST part of the protocol is a way to communicate tensors using binary data still going over the network, not using shared memory, but a much more efficient way. And we'll talk about that in the next slide. So, and I call this one out. So for all the extensions I just showed here, again, these are extensions. They are optional. And so the servers don't have to implement these. In fact, they're not part of the standard anyway. They're just Triton specifics. Servers could implement these. They're Triton's open source. These are the specs for all these is published. And again, I have a link later. But again, they're optional parts. However, in particular, I want to call it the HTTP JSON, the HTTP REST, the JSON implementation for high performance. If we look at, if we go back on the, we have this binary data extension in Triton, which resolves a very important problem. So if you look at the code on the left or the output on the left where we post, this is the same what we did before where we posted, we sent in a single small image, a 224 by 224 with three channel image, which is quite small. There's still 150,000 FP numbers in this data array, which of course I didn't list them all. And so that's a lot. And that's just for a single small image. And the problem is that this is very readable, right? It's very usable. I mean, it's easy to generate this JSON in a lot of different clients, a lot of different languages, a lot of different toolkits can already do all this stuff. However, to do that, you first have to, so if you have this image or this 224 by 224 by three set representation of this image as floating point numbers, you first have to basically encode them into strings. So print them basically. So that's time consuming. And then you send them over the wire to the server, which is on the other end. And then it has to read this JSON, it's basically just a string, right? So it has to actually parse these strings into floating point numbers, 150,000 of them just to do your inferencing. And so that becomes a huge bottleneck. Basically makes for any, unless your model has very small input and output tensors or if you just don't really care, this is a research model or something on the side, you don't perform, it doesn't matter to you, then this is not really acceptable. It's not really useful for our production kind of deployment. But still the advantages, again, going to the ease of use, this JSON is very easy to generate and there's a lot of advantages to doing it this way. But so let's combine those. And so this binary data extension does just that. So on the right, we can see using the binary data extension, basically the header part, the JSON part is more or less the same, except you have this extra parameter in here, which you basically say that, hey, I'm not giving you the data here in JSON, it's just, but I'm just telling you how big it's going to be, which is a little bit redundant, but that's done intentionally just to be redundant. And then you just, after the JSON message, is just the raw binary data. And there is an extra header required in your post, so the server can figure out where the JSON metadata header is, separating it from the actual raw data. But in doing this, now there's no more, you're still sending 600 and 2,000 bytes of data, you can't, that's kind of unavoidable, that's actually your image, you have to send it over. But there's no encoding or decoding overheads. The data can just be pulled right off the wire and kind of sent right into the model. So, and so for instance, this is just one data point running on a local network, so actually it's minimizing the, it's kind of removing the network part of this. If we send 128 of these requests in time that, just to kind of even it out and make sure there's no blips, doing the binary data really has like a 17x speed up, a huge amount. So you kind of remove that bottleneck from the problem. So that again, that's an important example to show how the protocol, through its extensions in this case, but in general, the protocol does allow the usability and because the flexibility of the protocol, you can't have extensions like this that also allow you, along with say the shared memory to get great performance as well. And so as I mentioned, there's a couple of references here in closing. One is to the protocol itself. You can find it in the KF serving GitHub and comments are welcome on that. Take a look at it. And then for Triton, this is a link that you can see from this GitHub, you can find actually the full Triton repo, but this link directly takes you to the extensions and the inferencing protocols that Triton implements. Thank you.


Hi everyone, in this video, we're going to talk about how to see what time it is in Python programming language. To do that, first what we need to do is to import time module by typing import time, import time. After importing the time module, we need to call the string format time function to specify how we want the time to be displayed. If we want to display time in format of hours, minutes and seconds, in 24-hour format, we will type next code. time dot string format time str strftimE and we will type %h colon %m colon %s h is for hours m is for minutes and s is for seconds. We will close our string and press enter. Now if we want to display time in format of hours, minutes and seconds in 12-hour format we'll type next code time dot same function string format time but with first argument i next is same. I will copy them minutes and seconds. Now I will press enter. If we want to add am or pm we will include %p in our function argument. For example, let's put it here and press enter. That's all in this video. Thanks for watching. Please like share and subscribe. Happy coding!