Hi y'all Liz here with smart business moves and Tom is not here as my co-host today but you know who is here An amazing guest. We have Bruce Vance. Hey Bruce. Hi there Ah, so glad that you could join us here today Bruce and spend an hour Helping our our little group of people that follow us figure out kind of where we're going. It's kind of a scary time, I mean For a lot of people it feels not as scary because okay, we're at least past the big pandemic piece But there's more stuff coming down the pipe. There are more changes coming, right? There's going to be a lot of changes. I think coming and it's it's not going to be easy You know we in cleaning we went along pretty much the same thing for about 2800 years We have actually found residues of soap like compounds that are 2800 years old Are you kidding me? Are you exaggerating? No, no, that was just in clean facts. Wow And then in the 50s we had the first great revolution in cleaning We got the detergents See see this is why we brought you on here Bruce because you're gonna have all the facts and all the details All right, so for everybody first before we jump right into it all I'd like to at least have Bruce introduce himself I know that most of you probably know who Bruce is why he's the expert why We turn to him when we want to find out about cleaning nobody knows more about actual cleaning in the Bruce bands Oh, look at this. Tom's coming on here We'll let him come in a second, but while he's figuring out how to get in here if you can Bruce Why don't you go ahead and tell us about yourself introduce yourself tell tell our audience who you are and Why we look to you so strongly as the leader Okay, uh Let me see if I can get straight in here. Hey, Tom.  I have been in the cleaning industry Well, we've been in town and country cleaning from For 32 years and I've been knocking around the cleaning industry a little bit before that and carpet cleaning Working for some carpet cleaning companies Probably close to 40 years in this industry I was I was involved in getting the first certification international certification set up I was the the technical advisory chairman after David Kaiser Who deserves a lot of credit for that too and I don't want to take away from it And We figure that in that class we have now had representatives for from about 500 companies In 10 countries and at least five continents All right, and that's the hct program that you're talking about the hct program House cleaning technician program So for those of you that haven't heard about it yet, you can find out more about it Through issa arcsie All right, go ahead go go on Continents wait a minute. So i'm assuming and ardeca is one of the two that didn't make the cut. What's the other one? um Leave it to tom to ask the hard questions. I'm trying to think maybe it was in africa Okay As far as I know, we didn't have anyone from africa yet Yeah, we will fix that we had we had one gal who actually when we're doing a live class in raleigh Who actually had come from mumbai? Oh That's that's a track right there. She said she looked on looked on the internet and it said there was no other class like it in the world So probably true. Yeah, and uh But anyway, so that's kind of my background and I'm still I finally am off of the tack chair although they put me back on as vice chair When you say tack chair, you know explain what that is I mean, this just isn't like a couple of people getting together in their basement over You know a 12 pack of beer and coming up with a bunch of no the ii crc is the largest certifying body in the world for the cleaning and restoration industry They are american national standards Institute approved for their standards They have very strict standards on how you do these these classes and these courses These are not something a couple people as tom said sat in their basement and did it We had representatives have representatives from chemical manufacturers from Equipment manufacturers we get input from the um Uh flooring manufacturers we have a lot and of course the cleaning industry itself So it's no one person's idea and it's what we have to do to Verify each question is intense And tom was involved in the first one run through on this So he has some concept of what i'm talking about in his consensus built among industry experts. Liz you were participated in In normalizing the answers to the question is a lot of this gets kind of nerdy, but it took years to Get the final product. Yeah, I remember Liz coming off of one of the meetings there and going these people are intense Yep, I remember that too. Was it only one meeting that I said that no, I don't think so but Yeah, I mean it's a very serious thing, but As we're moving forward. I think we the science is becoming more important We as I was saying we tell us why why do you think that first? Well When we went through the pandemic the pandemic changed a lot. Yeah um A lot of the trends that we're starting got sidetracked Uh, a lot of the Thought on how to do things Uh, turned out to be very wrong where where we started Um, we you know, we had people running around with uh electrostatic sprayers spraying disinfectant on everything in the house And then we found out that the phone might transmission is very minimal uh phone might being from a a contaminated surface And what I I always like to remind people that disinfectants are regulated by the epa As pesticides So would you feel comfortable if I said I'm going to make you healthy by spraying pesticide all of your house For no good reason for no benefit for no benefit and in a home Actually, even if it was for a good reason I'm still not in I still don't sound good, right? No, you know, it depends what you're you're doing though I mean, there's a lot of medical treatments are you know, like chemo and stuff like that It's kind of the same thing but you know the cure it's it's worth worth the trade off When they were when they were thinking that it was transmitted through surfaces through fomites Then the thinking was it's worth the trade off, but if there's no efficacy then you're just poisoning yourself for no good reason Probably in home there was Even if it had been phone might transmission true It's you're using a a bazooka to do a job you could do with a Much lighter gun as it would but even multi family or you know common area if you were sitting for instance, if we had a we do several sororities and We didn't need it there, but if we had had say a norovirus outbreak there We would definitely have used that in the Common rooms and and especially in the dining room and such Because first of all, it's not really only effective on a hard non-porous surface Um, I know they show it's being sprayed on all kinds of things, but uh Couches And depending on what you're using But just looking at Disinfection and because this was a big deal first of all The fogging they talked about epa said uh fogging really doesn't work Because you can't control what the dosage on the surface is and how long it stays wet You've got you've got to In order to actually disinfect the surface it has to be clean to start with So you'll notice that uh CDC kept saying clean first Then apply your disinfectant and then depending on what you're trying to kill or inactivate in the case of a virus How long do you have to keep it wet? Because and one of the issues we've been we were coming out of the Pandemic with is a concern over antimicrobial resistance Because if you look at the hierarchy of of how you Inactivate or kill the easiest thing to inactivate is the coronavirus So on many of these disinfectants It was saying one minute one minute for coronavirus two and a half minutes for Say hiv virus Five minutes for gram negative. I mean gram positive and 10 minutes for gram negative So if you were spraying this and wiping it up after a minute You were exposing the gram positive gram negative and they were kind of going oh, this is interesting But you weren't keeping it on long enough You're getting them drunk, but they so we're up the next day stronger than they were before That's a good way of putting it. Yes And so we're seeing that there's and this is still somewhat controversial in the industry but we're seeing that There's question about how effective our quaternary ammonias Which are basic if you go out on a grocery store, that's what 90 of your disinfectants are That's the staple. That's the go-to as they're beginning But if you I don't know what you guys noticed at issa That nobody in the last two years has shown a new quaternary ammonia all the new disinfectants have been either hydrochloric acid or it have been I mean hypochlorous hypochlorous, yeah Yeah, what do you say? Or they have how you said hydrochloric is like, no No hypochlorous not to be confused two different. No, no, please don't confuse those Or they've been hydrogen peroxide Right and those all work by basically burning what they're trying to get their their oxidation reaction, which is it's a little harder to get away from to become immune to burning Okay, before we move on further and go deeper on down this I'm going to ask you to just take us back because Bruce you're right when Everything was coming out one of the main things that we heard was you have to clean the surface first and then disinfect Right mean What's clean mean? Yeah, because you know, you hear you hear clean and people are like well, I am cleaning and I'm using a disinfectant That's how I'm cleaning it Yes That is a is the 64 000 question and one that there is I know siri is beginning to start a series on trying to figure out How to measure clean Interesting because you know, how do we know what something is clean? Cleaning basically is removing soil if you're just going to go to a very basic Definition as opposed to disinfecting which is actually killing whatever is left Now we have done studies and I know tom has done studies and we have shown That just using a high quality microfiber and plain water or deionized water We can achieve readings on ATP meter that are superior to using a disinfecting scrub cloth in fact, we've got one on our website where we had A desk that was 1500 On the ATP meter we cleaned it with a lysol disinfecting scrub cloth And got it got one side of that down to 350 Cleaning the other side with a high quality microfiber or hospital grade microfiber And deionized water and got it to 50 Wow, what are those numbers 350 50? What are those numbers? Okay? Let's back up a little bit. What is ATP? You know, what the world are we talking about? Yeah, okay This gets this gets ahead but this is one of the things that worse as we are talking about where where are things going How do we how do we measure? How do we start figuring out what is clean? Yeah, good The ATP meter ATP is adenosine triphosphate And it is the energy molecule in all living things So an ATP meter basically measures the organic load on a surface Does not tell us whether it's living or dead But it gives us an idea Of what's there and how effective we have been in in removing it. Okay On the meter that we use So hold on. I have a question real quick so if if Is it possible to have organic matter on a surface that is Dead or inactive and so it doesn't show up on a measurement or it would show up on a measure It would okay. So if you have a lot of bacteria there, it's going to have a high reading If you just cut up a tomato there, you're going to have a high reading. Okay In general on the scale on that particular meter Anything 30 and under would be good for a commercial kitchen Okay, 70 you're under that's the meter.  Okay for the rest of the house 200 let's clean it 500.  Yeah I'm sure that that's at least a big chunk of mine and I speak into my phone So I'm constantly spitting onto it So I'm sure that's hey I'm sure that's at least a good chunk of it there But we you know we're talking about what is clean. How do we know something's clean? Okay Well, what we have always done is we look at it. We it looks clean. Okay. It's uh No feels clean Ah smells clean Now one of the problems there's some problems with that. We can't see uh allergens. We can't see micro uh bacteria micro organisms And many of the smells that we consider clean Actually work against us You know, you have people who go out and go. Oh, I love it. I can smell the chlorine bleach. I know that it's clean right I know what clean is my grandmother taught me how to clean and my bathroom is supposed to smell like a pine tree My kitchen is supposed to smell like a lemon right, absolutely now you notice though that when a a tanker of chlorine gas Falls over and breaks open that they run around say everybody come out smell how clean it is Yeah, they they they they get them out of there because it's it's dangerous Um, the lemon scent is uh, laminine Laminine they have to you see they suddenly start doing a lot of testing of air quality after And during the pandemic stuff they hadn't done before And they found out that the lamb uh, laminine Which is your lemon scent combined with the uh Ozone in the air to form formaldehyde Oh, oh, I did not know that what they discovered. I was uh on a webinar which uh Some people were doing this research. We're talking about it. Is it the terpenes? Which are your smell molecules in your essential oils? Are extremely active chemically active? And so they get up in the air and then they start reacting with whatever's there And so you get some very interesting things that are developing from that So they may not be quite as safe as we thought they were Okay, that's so that's Taking us a little bit closer to why things are changing why we should be looking for Things are gonna be different Things there are going to be a lot of things if you went right before the As I say, we are in the we had the one big Uh Revolution of cleaning in the 50s when they came in with the disinfectants. I'm not disinfectants, but detergents We've started another one when with the microfibers with the green cleaning And trying to figure out what is green. What's not green? And you know, how we can get things that are better and better to From a green point of view and as we went before the pandemic we had a Very interesting interplay between the people who were doing engineered water of one form or another and the chemical companies Chemical companies weren't real happy about the people who were doing the engineered water and that would be things like deionized water ionized water and Electrolyzed water Electrolyzed water is Gaining you had gained a fair Foothold because you could use it. That's your electrolyzed water Actually, this was this was an award We were an early adopter and they thought so much of us. They actually sent us. I so overplayed everyone Let's see that was ionized. That was it. Yeah. Yeah, it was uh active ion Electrolyzed water and my positive and negative whatever it was all awesome It sucked that if you left water in it it actually grew biofilm And you were one of actually shooting more Organic matter under the surface than what was there you were Wow things aren't always perfect when they first start We'll talk about that a little bit when we get into uh, some of the surfaces we're cleaning too. Okay But uh, then the pandemic came along and disinfect disinfect disinfect and it was like a goldmine for the chemical companies We're beginning to see some traction again from the Uh, especially the electrolyzed water electrolyzed water allows you to basically get one stream that is uh hydrochlorous hydrochlorous hypochlorous acid and the other one which is sodium hydroxide which can be used for cleaning Which is really just kind of like my soap is like It's lie and and what you're doing is lie Yeah, it's lie. No lie. Yeah. No lie. It's lie. All right With the electrolyzed water are and with the other things you see where they say This is soap free and we haven't used soap in this industry in 50 years So don't tell me about that but detergent free What they're doing is they're using alkalinity to Mix with the soils and especially the grease whatever kind it is That's on the surface to create soap Basically suponification And then that's what's doing the cleaning is the uh, they're basically creating soap on the surface Bruce we had a guest on I don't think it's probably over a year ago a gentleman by the name of marty paris. He uh Is founder and CEO of a company called um, anilair and and they make split stream Machines that produce the hypochlorous acid and the sodium hydroxide It's really cool. He explains how you just take a big tub and throw some salt and water in it Suck it into a machine that puts some electricity to making heating chemicals. It's it's almost magic And it works And it will do a lot of your basic cleaning if you're in uh, hold on I just need to make sure that everybody hears this Bruce van says it works So I don't want to hear any more like conversations or arguments about how it doesn't work. It's not real But there are some caveats coming was there I want people to at least hear this part because you know how people just shut down They're like they haven't in their head, but if Bruce says it's works. All right, let's start from there Okay, let's let's look at a difference. Most of these things are geared to janitorial Yes If you're doing janitorial cleaning and you're in there every day You don't have to have the really big guns heavy-duty cleaners if you're in our residential cleaning and they You know you cleaned on monday and tuesday morning. She dropped some egg on the floor and kept it there for um Until you came back two weeks later You're gonna you may need a little bit more elbow grease to get it out of there Yeah, now when you're doing electrolyzed water you can you can vary the strength of your chemical by how much salt you put into the mix Okay If you get too much you're going to burn something because if you have too much oh h in there you've got a problem So there's a little bit of a learning curve on this um, the zero res which is the carpet cleaning franchise that uses this This process had a lot of trouble with browning carpet until they figured out exactly what the right mix was Okay, that makes sense We're kind of dialed in Based on the whatever the saturation level is in the water It gets to the point where you can only put in you can put all the salt you want and only so much of it The water will only hold so much of it. They calibrate the machines that way That's kind of the way an aisle there did it But then the product that it was spitting out you would dilute it and okay cut it by You know two to one five to one ten to one depending on what it was that you wanted to do Excuse me And what they're doing here, this is getting a little technical They have they are using a non buffered solution That means that um There's nothing there to keep the pH at a certain level as you dilute Most of our cleaning agents on this other hand do are buffered So the idea is that as the as you dilute it down your pH doesn't change that much Okay, think about the difference if you have if you pour water cold water into a glass And just let it sit out there that temperature is going to go up But if you fill the guy the glass with ice and then pour the cold water in it's going to stay Uh cold a lot longer Yeah, that's sort of buffered. It's the glass would be the buffer Yeah, the ice would be a buffer in that case. Okay um Keep trying to find an an easy way of Describing that buffering agents. Yeah so That was pretty good, Bruce. I think I think that the Yes, the um split stream works It can be effective It is um But it's not going to do everything we need to do We're going to have the times we need a stronger degreaser But for instance, we take uh deionized water, which is just pure water With a high quality microfiber hospital grade microfiber And this is all what tom taught us and uh It um, we can do like 70 of our cleaning with that It will not clean it will not do your a hardwater film It won't do your heavy build-ups of grease But it will do all that intermediate stuff The advantage of the various empowered waters Is that You make them in-house If you're talking green They are there's no, uh Petroleum there's no carbon footprint in trucking it across the country It's right there You're making it So you're you've got a lot of advantages there. Will it will Some of these especially The deionized water we use uh be approved green. No Because it's the people who do the approval green seal and such Charge a lot of money to get the approval And if you're making it in your, uh sink Or on you might going to pay $7,000 to have the thing approved green But it really is quite uh So this stuff is coming up and that's going to be an issue. Uh, I'll be curious to see how it plays out over the next few years Uh, I think that It right now like the anil air is I think about $2,500 uh I think it's I mean they've got two machines out now. I mean we have one and uh That's time. They only had one and I think it retails for around 15,000. It's not a particularly cheap piece of equipment So it yeah, go ahead and they have a lower end product I'll say a little in a smaller product that I think retails for around eight maybe Okay, so I saw and they were just introducing that at issa last year Yeah, I I have looked at it. I uh, it was outside of my price range. So So I don't think we're going to see That machine, but we are going to see some things get smaller and uh more price conscious There are some small little things that you can get that will supposedly do the work for you for Like I I want to say as little as a few hundred dollars more more people are coming out with the ozonated handheld tight Stuff that you can charge and they're relatively inexpensive. You might want to speak to that bruce rather than a gadget like this You're using uh ozonated water, right now The last time I looked at it. I haven't been able to find it largely because I've been tied up with other things that uh at issa but uh The last time I saw it they said that we had 20 minutes It was a sanitizing agent and 40 minutes. It was a cleaner Which meant you which means you'd have to be running around recharging it all the time Oh, yeah, now I understand they have improved that but I have not seen it. So I can't Again, it's a fairly well proven System But for for a maid service it didn't seem to be uh at that point There's a there's a company called Tresano that has a technology and they call it stabilized Ozonated water and they've got some of the trick that supposedly keeps the Free uh, I guess free oxygen level up there for for a longer period of time like days maybe Yeah, I had heard that that was out there as I say I have not seen it So I can't speak to it So But all of that's kind of kind of cool. But um But what what how does this play into our future? What are you seeing? You know, well, I care I see a number of things one is we're going to have to be More aware of what we're cleaning Okay, um Talk to us about that. What do you mean? Although, you know, there's there's the number of surfaces that are out there Are just amazing um You we've got stone which you know, the marketers like to talk about the fastest way to cash for the cleaner That's the fastest way from cash Um And when I do the class I like to talk about I'm going to show you how to how you can make a $10,000 mistake in less than two minutes And uh Okay, bruce. I have a story for you that you will like that goes right along with your story. I think um one year So my birthday is march 31st Tom's is the very next day And so Alonzo Adams is the one after that. That's right. And Alonzo's is the very next day after that holidays Is two days after that who says aches is on the third on the fourth on the fourth All right So a lot of people having birthdays right around that time period so, um one day one of my employees had called me and said hey lis I know that it was just your birthday and I hate to have to tell you this but We dumped over a bucket of vinegar on our clients Our clients marble entrance Um Hey lis can you come look at it? It looks kind of scary well I was not happy. I was nervous until I realized that Yesterday was my birthday and today is april fool's day. It's not only tom steward's birthday And it was April fool's day I was never so happy to have someone play a prank on me That was the best prank ever But they were trained so they understood the chemistry and the surfaces and of course Of course But I thought bruce would like that. It was just a prank in my case They take the training you provide and use it against you. Yes, they did Uh-oh you're muted bruce Sorry, I had a cough Years ago we had we didn't we never had these kinds of finishes to deal with you know, nobody had marble floors But we had one client that did and we cleaned it with something we thought was going to be safe And we burned the floor $20,000 later We learned that's when I started going hark. I need to know what i'm doing And that was back before inflation. That was back when $20,000 was a lot of money Yeah, I mean a big clean with $75, uh, you know for a 4,000 square foot house.  You need to do a better job when you're done. You need to you know buff it Yeah Yeah, and it's it's there things there's key landing agents in the uh, uh the dish soap that uh Attack the counter Is it only in dish soap only in dawn bruce? No, no most of your Keeling agents make uh detergent more effective by softening the water effectively It's not quite that simple, but that's that's close enough And so most detergent have key landing agents in them because it makes them more efficient But you know if you have one that is stone specific it either has very specialized key landing agents or it doesn't have any All right, and they these things don't end up on sds sheet. So it's very hard to know um, I mean we happen to know that uh uh Shackley's basic age does not have key landing agents because Somebody in our leads group who has a great big shackley business also happens to be a biochemist called their chief chemist and asked But that's the only way you know, you know so I just say use a uh uh Use a stone specific cleaner and a microfiber or or water um We're also seeing you know, uh six seven years ago. Everybody was pushing natural surfaces Which are notoriously hard to clean um Now we're seeing a big move towards cleanable surfaces The lbt well you have the vinyls for instance And we have a luxury vinyl tile and luxury vinyl tile originally came out and in five years went from zero to 20 of the entire flooring market It just exploded the laminates that we're now getting from Major players like pergo like mohawk like canoleum and shaw Are pretty well waterproofed Although I was a little disturbed that uh Shaw said On their water some of their waterproof flooring says do not use water to clean it Okay What would they like you to use? Well, they have a product. Of course they would love you to use but uh I wonder if it contains water Of course it does That was my sarcasm in play but I I uh I had been on a uh Little focus group of four of us on how do you clean this stuff after? uh post construction cleaning And we never said anything about you don't use water So I emailed jim mannes who is the uh there after care manager said jim. What's your thinking on this? You know, we would normally use a flat mop with water or some kind of cleaners. Just that's perfect It's buckets and puddles. We don't want So again, we have to keep up on what's really happening out there for instance vinyl Steam mop should be fine, right? I would think No, not on the seams What they found was that the super uh energized vapor can get down in between the seams on most of it Where it condenses But it can't come back out now Ah And so they're having mold problems. Yeah And they fence so this is the thing we've got to kind of keep up on you know, what's happening They put this stuff out there. They don't know what's going to happen with it until they get it out there When they first came out with lvt luxury vinyl tile the original luxury vinyl tile they Suddenly realized it's a thermoplastic And it will if it gets hot it Increases where am I there and so if it was in front of a Sliding glass door with a southern or western exposure in the south It deformed Which is why they came out with what they call spc which is stone poly core Which is probably put limestone in it which gives it dimensional stability But I was talking to one of the aftercare people I said you got limestone in here. I said, yeah I said, you know one of the new green cleaners is the hydrogen peroxide cleaners. I said, yeah Yeah, I said, you know, they're all acid the side don't you and he went we never thought of that So as we go out we have to know what we're doing and we cannot necessarily rely on What the uh Especially not what the uh Dealers and stuff. Oh the deal. Yeah, you follow the follow the anything that's written you want to follow because that'll Be take care of your warranty at a relieve you of any kind of liability yes, um, I was at a Class of the american hardwood association put on and the fellow walked out the instructor and he started with this The number one enemy of the wood floor is the maid And what they're telling us is that 50% or more of all warranty claims are maintenance And they are no longer accepting responsibility for maintenance And to give you an idea of the volume we're talking about I was talking to one Uh flooring inspector who as of october this was right before the pandemic She had handled 85 million dollars worth of claims and one of her compatriots had handled 125 million Is this primarily like residential residential and commercial Okay, I hope the majority of that was But you're you can see the kinds of dollars we're talking about right And so they're not they're basically putting us on notice that Hey, if it's the maid you need to go talk to the maid Don't talk to us And so these are insurance This is like basically warranty claims from the manufacturer and if that fails you go back to your gl policy But if you're the professional and you did something you weren't supposed to do You may want to speak to that bruce. They're not going to help you either Yes, thank you A lot of people feel That they if you have a liability policy it covers everything that goes wrong in the house And I've had people correct me on um Some of the facebook things saying oh no no no no everything that anything goes wrong in the house is covered under your liability I've been involved in enough cases where somebody has come to me for help After the fact when I can't really help them anymore And they're going my insurance company won't pay off. Why? Because it covers an accidents. It doesn't cover stupidity So If you are walking across the room I don't tell somebody this when they're you know, but I do in a class You can say things to a group that you can't say they're an individual especially after they've made a mistake But if you're walking across the room and you trip and the bottle of bleach goes flying And you have to replace a rug and a couch that's covered under your liability insurance But if one of your technicians or you saw on the internet that you can take bleach and get a spot out of the carpet Which you can not I don't recommend it.  You're supposed to know what you're doing And that is a big shock with you. I've I one case they had Uh, they wanted to get a wood floor really clean. They glugged the uh, the cleaner a few extra times And went after the floor to clean it up real well and turned it into a white sticky mess Because they started to dissolve the polyurethane $1,800 clean I saw another one in a this was in a school Uh in three months the new cleaners who went in there Uh used a mop on the wood floors did $18,000 with the damage Wow And that was these are these are damages that have come out of the company's pocket Manufacture no insurance company's going to pay for it.  Yeah, the one we have is called willful damage And it's not all that expensive, but you've got to find a true commercial insurance agent who understands this stuff Because most of them have no clue what you're talking about So we're going to have to be aware of what we're doing and and keep up on what the changes are So we know again what we're doing Well, how do you keep up on these changes Bruce?  I mean you're actively Take a lot of the knowledge and research and things that you learn like break, you know I don't call it breaking news, but the latest developments and you share that information Oh, absolutely rc si who shares it with its members And cleaning business today is still in working Yeah, last I saw yeah cleaning business today as a resource that Well, what will publish bruce has written for cleaning business today, you know over the last You know, it's having its 10th year anniversary april of uh this year if you can believe that And rc is having its 20th year anniversary Wow, that seems so fast to me a digress but uh plugging into cleaning business today plugging into a rc si or two things you can do to At least you know be exposed to a lot of the concepts that are um Yeah, what's coming up? What's what are the problems? What did we think was going to work and it doesn't? um Bruce can I ask a Couple of questions. I want to get in before for the hour completely gets away from us. We were talking about microfiber wipers and You know, we were an early adopter. We've been using microfiber cleaning towels for 20 years and this was back when It was really you know, everybody was cleaning with cotton tarry cloth and other fabrics and You know over the last several years I guess you start seeing more research where people are concerned about the microfiber that's coming off and The wash that's getting into the environment and they're getting like marine life that's got all this like microfiber building up in its system and um, I see some companies that are actually getting away from the multi-use microfiber wipers to like single use disposable wipers Is that a friend that you think we're gonna gonna gonna see getting any momentum? I think we will some in the Janitorial I'm not sure we will as much in I I've made I've met two Rather established and successful make that three well two that are doing it in the third one that is Is considering it that have made that move and there's a couple of things behind that From a cost standpoint Arguably it's a little more expensive on the surface But a lot of these companies since covet More and more companies that go to solo models where their technicians will just get up in the morning Pick up their phone Log into their favorite, you know, software and I forgot one in mind But we don't need to go there. Anyway, find out what homes are cleaning they go out They clean a couple of homes and they go home and The one problem that you're always dealing with is rotating the the towels that they're cleaning with and where they're going with disposable the cost of multi-use microfibers higher in a You know, solo remote workforce model when you throw their next layer in there Disposable Really starts becoming a wash no pun intended now that I think one of the questions would be can it be as effective? Oh From a consumer standpoint The feedback i'm getting is their customers Like the idea more because they know it's it's it's it's clean that that it hasn't been used to somebody else's house And they don't have to worry about it being properly laundered in terms of efficacy and how well it removes soil Yeah, who knows? Yeah, I mean see if you look at the perfect cleans they have testing showing a 99.99 percent removal of Pathogens in general and up to up to 99.9999 on some Yeah, and we we've done studies with with ATP. You mentioned some of that earlier and the higher in microfiber Definitely does a better job of removing soil right and that is That gets up to the level of of disinfection In terms of removing it and The other question is a lot of the stuff that we're talking about here obviously We think it's important, you know because We when the industry, yeah But we are in the minority and you know the general industry I know that you're you're you're kind of kind of been spearheading this for for years bruce and You know, you've had 500 companies take ACT certification That's awesome, but You know, there's a lot more. Yeah. I mean that's still It's still a small slice of the larger market and Consumers don't seem to really yet grab this Do you think that some point there's I mean with with with the thing that that you're doing in the body of Science this building that there will be some type of regulation coming in play where You know the government will Either stayed or maybe even at a federal level start getting a little more active and Making sure that professional cleaning companies are doing it in a responsible way I don't know. I do know That a number of states Now require a pesticide license to administer disinfectants Okay So that's one step There It's going to be difficult because everybody has their favorite way of cleaning Grandma did it this way I mean the reason we have we we're cleaning Excuse me hardwood floors with vinegar and water It's because grandma Had used soap to clean her floors Especially her linoleum and vinegar Rinsed the residue that was off. So she said vinegar gets it squeaky clean And when they used when they came in with Paul your thing used the oil soaps Uh, and if they had to go back and do any kind of repair to the finish It wouldn't stick because the oil soaps. They just said, I don't know use vinegar and water It's like uh, Liz's grandmother cutting the ends off the ham Exactly exactly Uh, it made sense 70 years ago, it doesn't make sense now And we're going to have we're still going to have clients who want to Uh use all natural products, which they think are are uh healthy We had a very interesting experience in one class. Uh somebody wanted Was using in fact a number of people in the class were using thieves oil Which is something that a lot of the independence love. It's natural. It's safe. It's wonderful And we pulled and they said could we look at an sds sheet for it? So we pulled the sds sheet and yes, it was pretty not not a big deal for people And that said do not allow to get in waterways. Do not allow to get into Sewers do not allow it extremely toxic to aquatic life so it's like Oh, I thought this was just safe and natural and wonderful so Which which I guess if you really break that down, it's almost impossible to use that in a residential siding Well, it's It's it's uh, yeah, you have to be more careful of it than you think And we see this all the time And I don't know what it's going to take for sure to get around that I do know that we're going to have to be careful With air quality You know good vacuums vacuum that's becoming much more important and people are beginning to think more about air quality We're going to have to be careful about fumes coming off now right now people think about People are geared to think about the uh, volatile organic compounds by which they're thinking of these horrible petroleum things But when you take the top off a bottle of vinegar you smell it instantly, right? You're smelling volatile organic compounds in that case in fact, uh, green seal probably would not approve vinegar because it has too high a Voc content and too low a ph So as we are You know moving forward I think there's going to be increasing interest in Trying to get this stuff right and trying to get so we're not damaging products, especially for the cleaning industry Because we are we are having more responsibility and uh You know, I know that uh, a lot of the individual cleaners Have a buy because when they destroy something that you don't have enough money to their soup roof but Which has Happened by the way in both of the cases I talked earlier about the wood floors uh But I think that we need to be in it as an industry we need to be thinking about Uh, what's really going on here? What are what we're doing? Really being that helpful are we getting it clean? And again, you know, how do we tell it's clean and we now have some tools to do that As you as an industry matures It tends to go from being able to determine success from a subjective method looks clean or as uh when I first got involved in Uh carpet cleaning and restoration came part along with that you felt, you know, the wall feels dry today you you uh If you did that you end up in court Because you have mold growing so We are now moving in this industry from subjective to objective Where we can actually determine is this system working? And I think that is the what we're going to have to hope is The direction that we go more and more because that's really when we become professionals And not just a drudge or not just the lady who does for um Mrs. Jones And as we talk to our clients We want to become more of a consultant for them I remember walking through a house and it was a new house and I'm Um, we had done the post construction cleaning on it and I walked through with the couple I said now this is such and such a product I uh surface you need to do this to clean that you need to do this to clean that you need She said do you do ongoing cleaning? I said yes Said good We start basically, you know, okay, you know I I think that It is changing and a lot of times change starts slowly before it happens quickly and There's a lot of things that are happening within the industry that are all positive You know, I see a lot of cleaning companies that are growing from uh from a revenue and from up from bottle line profit standpoint Companies are scaling up levels That you know, even five years ago you would have a hard time imagining and certainly On the 20th year anniversary of arcsy when you would get together and you know It was a big deal if you're a million dollar company and you know, you know, it's You know, I saw some numbers and and just about half of the companies that are made central are over a million dollars a year and An impressive number of them or two or more and it's you know, if a lot of things are happening There's better training more companies are bringing in professional management techniques the technologies enabling a lot of this There's even private equity that's coming into the residential cleaning space and you couldn't even imagine that five years ago All of that is just raising the stakes the more you have the more you have to lose and the more important it is that you incorporate professional tests and the The the the the you know issues that bruce is bringing forward here All of that's going to force it to happen more quickly moving forward You're going to see a lot more happening in the next 10 years in this regard than what we saw in the last time Absolutely, absolutely, and I think we're going to have I think we're going to have better cleaning. I think we're going to have better Uh understanding of what it is we're trying to do I mean we saw that just in the pandemic the advice we got in the beginning and that that wonderful article in the atlantic called That derrick thompson run wrote What was it? It was a hygiene theater That what we were doing really was just the Theatrical not accomplishing anything. We were basically it was just just for show We are right at the top of the hour But before we go bruce, is there anything that you would like to to share with the audience? Do you have any hct classes coming up anything that that we need to know about that would Be useful in this regard. Well hct class. We have coming up. Let me get make sure I've got the dates We have a zoom class Um in february on the 9th and 10th If I want to sign up for that, how do I do it? uh go to arcsy arcs i and uh go to events And it will have hct and you can sign up for it there All right, so for everybody. That's arcsy.org. I think and Go to events look for I think they actually have it listed as hct Yeah Versus The menu in the in the here i'll share my screen real quick. That's good and They'll also be one in april but there if you go there, they'll have a list of There we go If can you see that?  Let me let me back up. We'll take it from the top Okay, you go to arcsy.org. It will bring you here. I go to education and events I go to here click on live and virtual education and training and Here you go the ii crc Hct certification program I'm going to click on that And I'm going to drop this in chat Okay, great And you got something else going on bruce I will have one in april in uh same from you can go to the same place and that is 13 or 14 Ah, you've got it right there good. Is that also a Zoom meeting that's a virtual. Yes The next uh, we will be doing an in-person class in vegas at the show That's all the way in october, right Yeah, so I don't even think they're taking registrations for that yet. No No, and if somebody has uh you know wants a in-person class for uh A private we can arrange that too. And how would I do that? um You can call Our office here at uh<PHONE_NUMBER> And that will uh, that'll get get to me and I'll<PHONE_NUMBER> 967 7592. Yes Make it okay. There we go. No, did you put his no so that's bruce's number.  Yeah, that's our office number Oh, well then go ahead and distribute it to all your crazy friends. Oh, especially if they're dirty Yeah, it'd be either home clean Yeah, yeah, or uh tappel hill or pittsboro. Yes, okay, so Uh any we we're we're we're at the top of the hour bruce any any closing thoughts I think we've got an exciting time to come. I see there's been a lot of change in the last five years uh, and I think some of these uh Trends are going to continue And I'm not sure where we're going to be in five years, but I would we would have had a hard time When we started this thing in fact even when we started the hct What's that now 12 years ago? Well, we started probably 15 years ago. Wow Um Invisioning what we have now and what we would have to be teaching And you know, I think in five years. We'll all be making more money I certainly hope so. It's an opportunity here for us Yes, I think I think the people who are professional will do well Yeah, all right. Thank you sir. Great. Thank you so much bruce Talk soon. Bye everybody. We'll be back next wednesday five o'clock Easter. Bye. Bye. Bye y'all


Hi, I'm Zor. Welcome to Unizor Education. We will talk about something which is very interestingly called the fundamental theorem of algebra. It sounds really very impressive, if I can say so. Well, the name is historical, and some time ago algebra was primarily concerned with solving polynomial equations. And the higher degree of the polynomial equation, the better. How to solve the polynomial equation of third degree, fourth degree, etc. Well, there is actually some history behind it. I think it was Cardano who came up with the formula of solutions for the polynomial of third degree. I'm not sure about the fourth, but starting from fifth degree, I think it was French mathematician Galois who basically proved that it's not possible to express solutions of the polynomial equation of the fifth and higher degrees in radicals. Just as some kind of a formula, basically, which combines the coefficients of this polynomial. Anyway, the fundamental theorem of algebra is very interesting, and I think it is really quite amazing property of the polynomial equations. First of all, it considers the polynomials only in the field of complex numbers. So, we're talking about polynomial of nth degree, which means the old, let me use the argument z as traditionally used for complex variables. So, the polynomial of the nth degree of the argument z, its general form is this, where a zero should not equal to zero because we are claiming that this polynomial is of nth degree, which means the z to the power of n must have a non-zero coefficient. So, this is a general form of the polynomial of nth degree, and it is assumed that z argument is a complex number as well as coefficients from a zero to, oops, that's my mistake, this is a, n, z, n. So, all coefficients from a zero to a n are also complex numbers. So, the fundamental theorem of algebra says that this polynomial, if used in the equation, must have at least one complex solution. So, the result is some kind of a complex number z, which if substituted, it will result in zero. It's not an easy theorem, and this is one of those theorems which I'm just telling without proving that. Obviously, there is a ton of material about how to prove this particular theorem. It's just beyond the scope of this particular course. However, well, first of all, it's very easy to remember it. Like, any polynomial has at least one complex, mind you, solution. Now, why am I saying complex? Well, for obvious reason, this has no solution among real numbers. But there is a solution among complex number, x equals i, and x equals minus i. So, again, any polynomial has a root solution of this particular equation is called the root. It has at least one root. Now, there are a couple of corollaries, well, something like simple consequences from this particular theorem, which I would like actually to present to you with the proof. And here it is. So, let's assume that we have this. This is a polynomial of the nth degree. Now, I know it has at least one complex root. Now, the very simple sequence of this statement is as follows. Let's say c is a root of this particular polynomial, which means if I will substitute it here, I will have a0 c to the nth plus a1 z to the n minus 1, et cetera, plus a... Yes, I'm substituting c, which is a root a minus 1 c plus a n equals to 0. So, this is given. I know that this c exists because of the fundamental theorem of algebra. And the statement of my theorem is the following. That this particular polynomial can be represented as z minus c times some other polynomial, but the degree should be n minus 1. So, this is b0 z to the n minus 1 plus b1 z to the n minus 2 plus, et cetera, plus bn minus 2 z plus bn minus 1. So, this is a general polynomial of n minus 1st degree. The statement is that if z equals s is a root of this polynomial, then it can be represented as a product of z minus c and some other polynomial of degree n minus 1. And I'm going to prove it. Now, just to make my life easier, I might actually use a sigma symbol. Now, sigma means a summation. Well, just in case I'm sure many of you have seen it, I'll just repeat what I mean. So, this particular polynomial I can always represent as sigma of ai z to the n minus i where i from 0 to n. It means I'm summing these elements, varying the index i from 0 to n. If it's 0, it's a0 z to the n, which is this one. If i is equal to 1, it's a1 z to the n minus 1, which is this one, et cetera. But up to the very last one, when i is equal to n, I will have an and z to the power of 0, which is 1. So, I can use this notation and now I can rewrite it as this. So, this is corresponding with bi z. In this case, it's n minus 1 minus i, right? Because it's an n minus 1's degree polynomial, and i from 0 to n minus 1. So, this is basically another formulation of the same theorem, which I might actually use it just as some kind of a shorthand. So, I have to prove this, knowing that c is a root of this polynomial. Now, the fact that this is a root, I will write again in this format, c to the n minus i is equal to 0, where i is from 0 to n. So, this is given, and this is what I have to prove. Well, let's do it in a very, very simple manner. We will just multiply this by this. I'll open the parenthesis. If I will multiply z to all these numbers, I will have, now let me put it explicitly without the sigma sign. It will be b 0 z n minus 1 plus 1 from z, it will be z n plus. Then I will multiply z to the next one, so it will be d 1 z n minus 1 plus, et cetera, plus. The very last one would be when i is equal to n minus 1. So, it will be b n minus 1 z to the power n minus 1 minus n minus 1 and 1 from here to the first power. It will be just this. So, this is the result of multiplication of z times all of these numbers. Now, I have another one. I have minus c here. If I will multiply minus c by all of these guys, I will have the following. I will have minus c b 0 z to the n minus 1. Now, it's not an accident why I put this member under this member because they have exactly the same power of z. So, the first member would be b 0 times z to the n minus 1 and times minus c would be this. And I put it under the z to the n minus 1. Now, to get the power 1 of z, I have to use i equals to n minus 2. So, it will be minus. These are all minuses. Again, c b n minus 2 z and then for i equals to n minus 1, I will have minus c times b n minus 1. So, this is the result of opening the parenthesis and this is expression which I will have. And my question is, is or is not possible for this to be equal to this? Let's just think about how can we make it. Actually, it's very simple. Can I choose b 0 equals to a 0? Yes. In which case my z to the nth power would be exactly equal to my original, where is my original? My original polynomial is this. So, for z to the power of n, this polynomial has a 0 coefficient. This has b 0. So, if I will assign b 0 equal to a 0, now this polynomial and this will have exactly the same coefficient as z to the power of n. Now, how about the power z to the n minus 1? Well, I have b minus c, b 1 minus c b 0. It should be equal to a first, right? Because the a first, a 1 is the coefficient at z to the n minus 1. Can I do this? Well, actually it's very easy. b 0 I have already established. It's this, which means b 1 should be equal to a 1 plus c a 0, right? So, I can establish b 1 using this equation. Now, how about the next one? Well, I didn't put it here, but do you understand that this would be this, right? From which b 2 is equal to a 2 plus c b 1. c multiplied by b 1, which is a 1 times c plus a 0 times c square, right? If I multiply c by this, I will have a 1 times c and a 0 times c square. Now, b 3 minus c b 2 equals a 3. From which b 3 is equal to a 3 plus a 2 c plus a 3 c square a 1 c square plus a 0 c cube. So, this process actually can be repeated for all these members. Now, the very last one, I will write it down and I will just write it down by analogy with this, but obviously it can be proven by induction. It's a very simple thing. The very last one here would be of b n minus 1 minus c b n minus 2. It should be equal to a a minus 1, n minus 1. From which b n minus 1 should be equal to a n minus 1 plus a n minus 2 c plus a n minus 3 c square plus etc. plus a 0 c cube a n minus 1. Again, it can be proven by induction from all these absolutely equivalent equations. And the only thing which is left here is this. And it should be equal to the coefficient which is free of z or z to the 0 degree which is a nth, right? So, in addition to all this, I have this. Now, if I didn't have this, I have already found all my coefficients from b1 to bn minus 1. Now, obviously, I can't really just use these without checking that this particular equation is also true. Is it true or not? Well, let's just find out what it is. If this will be a true equality between these two things, then these solutions are actually everything which we need. These are exactly the coefficients of this polynomial of n minus 1 degree which is needed to be multiplied by z minus c to get the original polynomial. So, all I need right now is to check if this expression for bn minus 1 substituted to this would give me an identity. So, let me wipe out this. You don't need it anymore. It's already done its deed. Now, well, let's find out. I get this one and I should multiply it by minus c, right? Or, by the way, instead of this, I can do it this way. a n plus c b n minus 1, it should be equal to 0, right? So, that's the equivalent equality which I should check. All right. So, let's check what bn minus 1 is and we find out what it is, multiplied by c. Now, what happens? a minus a n minus 1 times c then plus a n minus 2 times c square plus, etc. Plus, the very last one would be, well, the one before last would be a1 c to the n minus 1 and the very last one would be a0 c to the n's, right? Each member is multiplied by c, so this exponent is increased by 1. This is what I have from this. Now, a n plus this one and it should be equal to 0. Is it equal to 0? Uh-huh. Absolutely. Why? Here is why. Because c is the root of the original polynomial. So, original polynomial, which is this one, has a root c which means if you substitute c instead of z, you will get 0 and this is exactly what we have to check. So, this last equation really is always satisfied if I choose b's like I did before. And that actually finishes, this is the end of the proof that any polynomial of the n's degree can be represented as a product of z minus c where c is a root and some other polynomial of degree which is one less than the original. So, bn of z, if any general polynomial of the n's degree in the field of complex numbers is always represented, can be represented as a polynomial of the smaller by one degree and z minus c where z is the root. Okay. So, this is the first corollary, the first very simple, well it's a little lengthy but it's really simple consequence of the fundamental theorem of algebra. Because the fundamental theorem says that c always exists for any polynomial including this one of course. And that's why it is possible to do it this way. Now, let's do a different theorem. It's another very simple consequence of corollary. Since I know that this is true, then exactly the same theorem can be applied to this polynomial. It also has some root according to the fundamental theorem of algebra, right? So, if I will call this root c1 which means my first root. Now, this can be represented as this polynomial of the n minus 2 degree and c2 would be a root of this polynomial, et cetera. Obviously, it's true for this as well. And finally, it would be z minus c1 times z minus c2 times et cetera times z minus. Now, for any polynomial real polynomial including the polynomial of the first degree I will have some kind of root. So, the polynomial of the first degree would also have some root. Now, what is the polynomial of the first degree? Let's go to this. Well, obviously this is something like this. This is the first degree, right? And it has a root. I mean, obviously it has a solution to the equation az plus b is equal to zero. And we can represent it as this particular solution times some kind of a constant, right? Actually, this constant is supposed to be equal to a where cm is the solution. So, there is a constant at the very end. This is the result of the very last representation of the polynomial. The first degree has z minus its root and some kind of constant, which is no longer a polynomial. So, no longer our fundamental theorem is applicable anymore. So, that's basically the final form. Any polynomial of the nth degree can be represented at. Well, incidentally, a is not just a. For obvious reason, this is a zero, the coefficient at z to the nth degree. Because you can only get z to the nth degree from the multiplication of these if everywhere you take z. z, z, z, z, and you will have one actually coefficient at z to the nth degree, right? There are only n multipliers. So, to get z to the nth degree, if you open all the parentheses, you have to multiply z by z by z n times. And the coefficient will be one, so you need a zero to convert it into a zero z to the nth degree. Basically, that's it. This is the end of this second corollary to the fundamental theorem of algebra. Any polynomial can be represented in this particular form. And by the way, incidentally, it means that it has exactly n roots. Because every time I have a root, I can always represent it in this way. So, I can always find n roots not necessarily different, because sometimes c1 and c2 or whatever, they can actually be exactly the same value by accident. But anyway, there are n of them, and it's exactly n of them. Because every time I have a new root, I'm reducing the power from n to n minus 1, from n minus 1 to n minus 2, et cetera, down to zero. So, there is no other solutions. It's always n, and you can always represent your original polynomial in the field of complex numbers again and repeat it again and again as the product of these z minus roots. And as I said before, some of them might actually be exactly the same values. Like, for instance, if you have z squared is equal to 1, well, I'm sorry, if you have z minus 1 squared equals to zero, this. You have root 1, and then you have another root 1. So, there are two roots 1, c1 is equal to 1, and c2 is equal to 1, right? So, the roots can be multiple, how should I say it? Multiple roots can have exactly the same value. Let's put it this way. It can be double roots, triple roots, et cetera. But in any case, the theorem is always formulated as a polynomial of the nth degree in the field of complex numbers has exactly n roots and can be represented in this particular form where a0 is the coefficient at z to the nth degree and c1, c2, et cetera, cn are roots. Okay. Basically, this statement, it seems to be a little bit stronger than the original fundamental theorem of algebra because fundamental theorem says that any polynomial has at least one root. This is a slightly stronger statement. It has exactly n roots. However, these two statements are basically, I mean, this is kind of consequential and the derivation is so simple that in many cases I've actually heard about the fundamental theorem of algebra stating that any polynomial has exactly n roots. Well, it doesn't really matter. It's basically, it has some maybe historical value, but no more than that. That's it for the fundamental theorem of algebra. Try to go through this material again, maybe just from the notes and try to follow the proof, whatever I have presented here. There is one in the notes. And again, don't forget to register and go through the exams. Thank you very much. That's it for today.


remember that even in your company they are ex-wix people that you're being recruited for what come so ns light mystic that you don't do in your company then probably is going to result in something that you're not going to uh to to actually uh accept when you not even plan for it because most of the employers some companies are trying to close down some companies are trying to lay off some workers either laying off what you call the old workers or even the new workers


Hey, good morning. Good afternoon. Good evening. Wherever you're joining us from welcome to cube Khan 2022 My name is Raj.  Thanks for attending this session I'm nilanjan a software engineer at harness and a core contributor to the CNCF litmus chaos project Where I'm responsible for developing chaos experiments for various types of failure scenarios and the requisite tools Required for injecting chaos into various environments.  So today we're going to talk about chaos in any applied to the fintech domain Before we jumping into chaos in English. Let's spend some time on fintech domain. What it is Here goes a problem statement Non-banking services companies would like to offer banking services and products Example could be a cab company such as uber Which provides digital platform to its drivers and passengers would like to offer art alone Why would they do that? I mean for obvious reasons. They want to Increase the revenue streams and also they want to serve their loyal customers Make them happy by providing some competitive, you know interest rates so If so, why they have to look at some fintech providers, why can't they do themself again for some obvious reasons, you know Ubers primary business is not banking a be They don't have banking because of that. They don't have banking ecosystem and They don't have banking license and They had to go through even they do they would go to a lot of Compliance and regulations of banking sector, which is too much for them. This is where FIS would help the Fintech companies are fintech consumers where it exposes banking services in the form of APIs and these fintech consumers Consume those APIs and build some innovative products Provide great customer experience to their end users is a win-win for everyone So, how does a typical fintech technical architecture look like it? It would be something on these lines Here I want to kind of convey a couple of things one the complexity of the architecture and introduce chaos engineering So the complexity of this architecture is not intentional It is a wall over the period of time just like any other organization FIS architecture is also evolved For the last three decades or so FIS is building products in the financial services. It build It's its products on all sorts of technologies like a mainframe windows based web applications It evolved right now. We are in the Cloud-native computing era where the architectures built on micro services base architecture at API gateways Kafka and everything in in between But these modern architecture need to integrate with legacy architecture because you still have certain capabilities provided by legacy architecture While FIS is going through the digital transformation phase so in this context You have to understand or this is a classic distributed systems architecture Out there and distributed systems architecture Brings its own challenges the challenges around resiliency Where things could go wrong in Anywhere in the system. What do I mean by things could go wrong? Like, you know, there could be a network partition that could happen between the systems There could be a service disruption. There could be a Fault that could be happening This is where chaos in reading this plane could help identify those issues and Resolve them thus providing superior customer experience So what is chaos in reading? I'm at the discipline or a practice experimenting on a system in order to find how resilient it is meaning whether The application can withstand to the turbulent or faulty conditions You know, what I'm what what do I mean by that? Let's work with an example. The example could be There could be a network latency or Generally latency between two microservices in your in your stack and What if the if the if there's a latency on the downstream system, how the upstream system is impacted, right? You want to understand how resilient your application is and overall stack is Just like any other engineering practice chaos engineering practice as you know, three pillars depicted here You have people you have process and you have ecosystem so next few minutes, I'm gonna double click on ecosystem and a little bit of one process and explain to you What the journey that fys went through and the thought process that we went through and how we put together this Ecosystem because it's not just a chaos tool. That's good enough to implement chaos in reading practice But you need to envision and think about Ecosystem So here is the the ecosystem that we have put together Why we have done this One we want to find the toil Within the within the chaos engineering practice and automate it and make it reputable process and Second we need to scale it So why we need to scale it we need to scale it because Organization like fys has Multiple hundred products and we need to find build an ecosystem and tools and the processes so that we and we build this consistently and in a repeatable way so that We can apply this across multiple Multiple products that's how we scale it and second we want to Automated so that we can implement chaos engineering for every release that's going into production Because chaos in itself brings in some additional work So we want to identify the toil around that automated so that it is easy consistent and reputable process For for the application or application teams So as you can see there are four or five pieces in this ecosystem one on the top your cacd pipeline you have application under tests here you have a load generators you have a apm tools and You have chaos tool itself plus the captain so I'm gonna talk about this each one of them here So when you're executing a Precursor to the chaos test is a generating load on an application under test and put it through some load Why we want to do that because most of the issues that happen in production Happen under load. That's why we want to generate some load and the application that we are injecting Chaos into that's where the load generators would help Okay, that's great You have the load generator generating the load on the application and you have chaos tool injecting chaos through its agent into the application, but We want to monitor the health of the application when we are injecting the chaos. What do we want to monitor? You have Several metrics up and down the stack, right? You have a matrix at the application level your matrix at the process level and your matrix at the host level such as You know response time throughput error rate At the application level your process CPU process memory Utilization if it's a Java JVM based application or J boss you have a thread pool connection pools You have a JVM level matrix such as garbage collection and all and you have host level host CPU memory and network IO and this coyote type of matrix that you want to monitor, right? in order to monitor those metrics you need tools like They're not raised plunk and Prometheus. So that that's why they are critical part of this ecosystem now These tools are out there. They are collecting the matrix And you are executing the chaos Experiment and you want to measure the chaos experiment, right? Meaning when you introduce a network latency, you expect say 2% errors But you are seeing a 10% errors today. You do that evaluation manually without any automation, right? But with the with the tools such as captain Which helps with automating these evaluations where you define your SLIs in this case an example that I just mentioned error rate and SLO which is not more than like, you know, 10% Right? You define that you qualify that in a YAML file and you put it in the in the Bit bucket pretty much like some sort of a github's model here and You integrate captain with the APM tools down below Now when you execute a chaos test You tell captain. Hey captain. I executed this chaos test for this period go evaluate for this five minutes or this 10 minutes and Then it goes pulls that SLIs SLOs pulls the matrix from the Dynatrace evaluate and give you gives you hey, you know for this test you you said you wanted a 10 10% error rate is acceptable because you introduce chaos, but I Saw like a 30% error rate. That means your chaos test failed. Then you further dig into it to understand why that Why why there are so many errors whether you introducing a latency in one API caused errors in other API That's for the evaluation Triaging that you have to do So but what captain can help you with is can help you with avail automating that evaluation process There's a bond toil that you have That is eliminated Now this is all good and if you want to automate this further That's where says CICD Would help you what CICD can help you is help you defining a workflow around chaos engineering ecosystem and Help you trigger it's a load generator Through the through Jenkins are CICD pipeline and then trigger chaos Experiments and finally trigger Evaluations that will talk to captain which will give you a boolean well value like pass or fail Using which you can make a decision whether you move this software to the next Next environment or production or wherever you want to or not So this is a kind of an ecosystem that we have built to a Automate the toil be Scale this across the board by consistently and repeated Consistently implementing this in a repeated manner. So now the floor. I'm handing the floor to Nilanjan who's gonna spend some time on chaos in eating or sorry, I will say Litmus chaos tool itself Nilanjan floor is yours Litmus is a tool set to do cloud native chaos engineering It helps both developers and sre's to automate the chaos experiments at different stages within the devops pipeline like Development during CICD and in production which leads to increased resiliency of the system It adopts a Kubernetes native approach to define the chaos intent in a declarative manner via custom resources It uses the operator pattern and relies on custom resource definitions to define the experiments On top of that it must provides chaos inter dashboard where one can orchestrate and visualize the results and metrics of the conducted chaos Experiments as well as litmus API for programmatically injecting and managing the chaos From a top level litmus chaos uses CRs to define the chaos intent and manages the chaos orchestration via operators There are different CRs such as chaos experiment chaos engine and chaos result while the chaos operator That reconciled the chaos engine CR in the course of a chaos experiment execution Given that a service account with sufficient our back permissions for the experiment has been already defined One needs to define the chaos experiment CR manifest for specifying low-level experiment attributes such as experiment tunables container images etc and The chaos engine that binds the experiment with an application instance as well as defines how to perform the chaos experiment in terms of Mounting volumes for the experiment parts or overriding experiment tunables Retaining or deleting experiment parts post the chaos and so on Upon the creation of both these resources chaos operator reconciles the chaos engine CR to create the chaos runner Which consumes the chaos experiment CR data for experiment creation and then creates the Kubernetes jobs which creates the requisite experiment parts for running the experiment logic in the end The job also updates the chaos result CR which Summarizes the result of experiment runs as well as updates the chaos engine to bring the chaos execution to an end You can schedule chaos experiments to run them later export metrics of the experiments via Prometheus exporter and get chaos Execution events from chaos engine and chaos results during the various phases of execution The unique value proposition offered by litmus includes first of all being cloud native chaos Experiments that allow you to validate your entire application stack beat cloud native Kubernetes resources or cloud infrastructure or even Legacy infrastructure with the broad range of experiments that it that is offered by litmus second being least privilege Principal chaos injection which allows for chaos engineering in security sensitive environments using granular RBAC over individual experiments and use of just-in-time Execution of privileged containers that limit the abuse and misuse of target environments Thirdly it provides declarative pre checks and hypothesis validation Which is leveraged using litmus probes which can be used for validating Experiment steady-state conditions with less to no programmatic intervention in a simple and declarative manner For fourth, we have conditional auto-stopping of chaos injection Which is one of the features of the litmus probes where the failure of the probe condition check Leads to the safe abort of the chaos experiment which prevents any harm to the target resources Fifth it provides custom chaos recovery actions and it can be defined as a part of the chaos scenario in a Declinative manner to introduce custom chaos recovery steps that can execute conditionally Sixth it provides declarative custom tasks, which aids at the early running the tasks that simulate real-life Conditions for the chaos execution. So for an example Running a load generator to simulate a network traffic is what we can do here finally the last one quantification of system resiliency by use of weighted experiments and probe success criteria to quantify the result of your system as a metric score Now let us take a look at how some of the litmus chaos experiments can be leveraged in real-life situations to evaluate system resiliency Thank you, Nilanjan Thanks for that that nice overview on litmus tool So let's dive a little bit deep into the experiments that litmus chaos provides Here I want to kind of focus on how do you measure the experiment, right? Just like Any test you want litmus test or litmus experiment to be measured Litmus offers wide variety of experiments here I took three experiments as a sample and I want to introduce how we measured When we execute this experiment for example When we execute pod HTTP latency, we measured A third pool connection pool utilization error rate and throughput why so again back to that microservice a and microservice b example If we introduce latency between a and b we want to understand how is the impacted the resources done on a is impacted Pod memory hog if you introduce a memory saturation type of event and a microservices are Kubernetes architecture the pod is killed with whom kill Event we want to understand whether that the APM systems are alerted because of that event How is this a service response time of API's of that exposed on that part and as well as does The pod memory hog on a particular part or a whom kill event of a particular part caused any wider impact Right, that's how it helps you with the broader stability resiliency issue or understanding the stability resiliency of your application HTTP status code so if you want to inject 500 errors on a microservice running on a pod in say account creation workflow, right? What you want to understand is when you introduce 500 errors how it impacts your business metrics such as account creation rate How resilient your overall application stack is how it is impacting your your business metrics So earlier talked about kept and doing a chaos evolution. So how what is the example of a chaos evolution? Here the way we envision Chaos evolution as a kind of three phase manner your pre phase pre chaos phase during chaos phase and post chaos phase pre chaos phase you can think of a Like a Study state during the study state you may define The metrics that you want to measure and you you define the thresholds example, you want to measure throughput error rate response time CPU memory connection poll utilization thread poll utilization say for example for Response time you want three seconds throughput hundred transactions per second Error rate less than 2% and now executed the test study state five minutes as depicted here, right? You give the captain saying hey captain. Here is my SLL SLI definition and Here's my five-minute window. I executed the test go evaluate it The captain comes back saying that it's all looking good green study state is good Now you introduce chaos example a network latency Now typically when you introduce it latency the throughput will go down the error it will go up But your hypothesis is there error rate could be impacted only one API where you introduce the latency and it could You know cause maybe 5% error rate You feed that your SLIs that you want to evaluate and their objectives are thresholds Again in this YAML file and feed it to captain and kept and tell captain. Hey captain. I executed this chaos This is my chaos phases between 10th minute and 15th minute and here is my SLI SLL definition for this go evaluate Captain would evaluate and comes back to you whether it is past or fail if it's within the bounds It's a pass if it's outside of bounds fail That's that leads to further investigation and that will help you find why there are more errors, right? Are any other issues why the connection pool utilization? You know went up during the chaos, but it's saturated. You're expecting it to go up to maybe 50% But it may went up to 100% right or CPU you're expecting to go up like 5% But it went up to 50% So those are the things that captain can automatically can evaluate and give you kind of as some sort of a heat map and pass fail result and post chaos What you need to do is Which is you define your metrics very similar to your study state The post phase is more of a recovery phase right where you remove your experiment at the 15th minute and 15 to 20 minute you won't understand the application Metrics are back to normal, right? So then your SLI SLL are very similar to what you have in study state and then you give the captain saying hey captain go Go evaluate my post phase. These are my cellos SLI SLI's and Evaluate it and captain evaluates and comes back if it's all looking within the bounds pass Otherwise it mark it fail. So that's the kind of a power the automation that captain brings you in evaluating your chaos experiment So finally, I conclude this talk here. Finally, I will leave the session with a thought that Usually the chaos in leading is looked at as a more of a SRE problem But it can help you validate your architecture and also helps you with the You know measuring business metrics does help the business teams ultimately providing greater customer experience and product quality and also With the architecture validation, obviously it helps with the stability mean tentative detector mean time to resolve type of issues That helps with the SRE with that I open the floor for Q&A. Thank you very much


Next question is from Grant Lee. How would you recommend starting a career as a trainer right now with the COVID situation? Maybe, maybe the single hardest thing to do right now. I got it. Maybe. It's so weird. It's got to be up there. It's so weird. If COVID wasn't happening right now, my recommendation would have been go to a big box gym, get certified, learn the ropes, learn how to sell training, learn how to sell yourself, build up clientele, prove to yourself that you can become successful, be the top trainer there. Then if you want to do better, then you can go into a private studio and do that kind of stuff. But gyms are closed, and the ones that are open are limited. A lot of towns and cities are saying things like you can be open, but only 25% capacity. So that is out the window right now. If I were to start as a trainer now, that wouldn't even be an option. What I would probably do is I would build some kind of a online authority, social media following, and then I would try to train people one-on-one outdoors in person. And I would meet up, you know, hey, meet me at the park. I'm going to take you through a workout, or I'm training three people at a time. We're going to go hiking and do some resistance training outside. I think the demand for health and fitness is just this high, if not higher than it was before. You know, here's the thing about personal training. People who tend to hire personal trainers tend to be in higher income brackets. People in the higher income brackets right now are more protected from the economic downturn than people in lower income brackets. So personal trainers, if you position yourself properly, there's probably a lot of people out there that still want to hire a trainer. You just have to do it outside of a gym. Yeah, it's just more difficult to just stumble across them. And so you really have to do a lot of work at, you know, how do I market myself and promote myself to attract, you know, that type of a client?  That's something that, I mean, I was able to structure my whole business around that. So that's a potential option now, which I think a lot of smart trainers have pivoted and made their business mobile so they could go travel and then account for the costs of travel and all those things to bring into it. But if you're a brand new trainer, you know, getting time under your belt, it's really difficult right now. And I've seen some tech companies, and I know one of them was like Fusionetics, I think I just saw was looking for to hire online coaches just solely, right? So they have the platform and they're trying to like pull in trainers to just help kind of coach them through these, you know, portals that they have provided for them. So I think there's like, there may be ways to at least get introduced to people online and then kind of spawn off and create what you're talking about with a social media presence to get, you know, more people, but it's going to be difficult. Like you got to get creative. Well, this is a good conversation because Andrew and I were just having this conversation maybe three, four days ago. Andrew's behind the cameras. But is it aspiring trainer? And, you know, and that was one of the things that he was vocalizing to me recently is just, you know, out of my love everything I do here. But I also really, really want to, you know, move into training clients. I mean, that's what his degrees in. He's been around physical therapy. He's been listening to, you know, all the information that we've been presenting for a long time. And so we talked a lot about this. And one of the things that I was explaining to him that is going to be one of the hardest things to do right now. And it's one of the most important things for you having a consistent business is lead generation. You know, the, the old school method or what we would recommend most times to people is what Sal said, which is go go to a big box gym.  That's 1500 to 3000 opportunities that you have as a trainer to potentially get them as a client. That doesn't exist right now. So how are you getting just people in front of you or listening to you or paying attention to you? Now, depending on what medium speaks best to you or what you use the best. So whether that be writing like blogging or you're good on camera. So YouTube or maybe you're great at writing Instagram posts or maybe you have quick short facts or stuff that would go good on Twitter. Whatever the medium that I think that you do the best on, you want to double and triple down on that on adding value. So you can just start to gather your audience or gather your potential leads. So that would be my first piece of advice and then flip the script on like this being such a struggle right now. Like it makes it difficult for you to get out and meet people, but this is where I would be doing my studying and learning. It's like, okay, I don't got a lot of time to be out with people and seeing people. I guess this is a great time for me to further my education. So I'm going to be trying to learn as much as I can during this time when we're not out socializing and being around people. And then I'm going to share that information that I'm learning on the whatever platforms I do best on. And what I'm not trying to do is sell people yet. I'm not trying to convert you into a client. I'm not trying to sell you on personal training. I'm just trying to provide information for free to add value to the people that potentially are paying attention to me and begin to build your network. Don't think about the dollar amount. Don't think about training the clients right now. Just think about generating leads and your litmus test on if you're doing a good job of that is are you adding people? Do you write a blog and does it get shared? Do you write a post on Instagram and do people comment on it or do people DM you about it? And if you're not getting people to comment on something that you're providing for free, then you need to up your game. You need to get better at it. Well, that means you further your education or redraft whatever it is that you're putting out there. My main focus right now would be educate myself and then use one of these social media platforms to add value to others lives to start to build a network of people.