Okay, thanks Adam in the studio. We're here on the floor in Cloud City. You're in the middle of all the action. The keynotes are going on in the background. It's a packed house, I'm John Furrier. Dave Vellante's on assignment, digging in, getting those stories. He'll have the analysis. He'll be back on theCUBE, but I want to welcome Chloe Richardson, who has been holding down the main stage here in Cloud City with amazing content that she's been hosting. Chloe, great to see you. Thanks for coming on theCUBE and kicking it off day two with me. No, not at all. Thank you for having me. It's really exciting. What you guys have got over here, very fun. We're inside theCUBE. This is where all the action is. And also the Cloud City is really changing the game. If you look at what's going on here in Cloud City, it's pretty spectacular. No, I mean, the atmosphere is absolutely palpable, isn't it? You can just feel it as people walk in and see what the future looks like for the telecoms industry. It's really exciting. And you've been a great job on the main stage. We really love on your content. Let's get into some of the content here. Actually, the keynotes are going on. We're going to have DR maybe fly by the set later. We're going to check that out. But let's check out this video tape of, this is telcoDR. You got to check out this reel and we'll be right back. We'll talk about it. TelcoDR burst onto the global telecom scene this year, making headlines for taking over the huge Ericsson space at MWC 21 and for building Cloud City in just 100 days. But why did the company go to such trouble? And what is their unique offering to the telecoms industry? And what drives their dynamic CEO, Danielle Royston, or DR, as everyone calls her? Cloud City Live, caught up with DR, away from the hustle and bustle of the city to find out. Hi, I'm Danielle Royston, coming to you from beautiful Barcelona. I'm here for MWC 21. About 100 days ago, I decided to take over the iconic Ericsson booth to turn it into Cloud City. Cloud City has over 30 vendors and 70 demos to introduce telco to what I think is the future for our industry. We're gonna have three awesome experiences. We're gonna talk about the new subscriber experience. We're gonna talk about what's in store for the new network and the future of work. And I'm really excited to create a community and invite awesome telco executives to see this new future. It's been a really tough 18 months and we didn't know what MWC 21 was gonna be like in terms of attendance. And so from the get go, we plan this amazing experience that we call Cloud City Live. At Cloud City Live, we have two main components. We have the speaker series where we have over 50 speakers from Amazon, Google, Microsoft, as well as CSPs and awesome vendors talking about the public cloud and telco. The second part of Cloud City Live is the queue. Think of this as like an ESPN desk of awesome tech interviews focused on telco and the public cloud. Hosted by John Furrier and Dave Valente, Dave and John are gonna talk to a variety of guests focused on telco and the public cloud. It's a great way for our virtual participants to feel like they're at the show experiencing what's going on here. So excited to have them as part of the Cloud City booth. There's a ton of innovation going on in telco. And 20 years ago, Elon Musk said on his mission to Mars, I like Elon Musk. I'm on a quest to take telco to the public cloud. Every year at MWC, there's always a flurry of announcements. And this year is no different. At this year's MWC, to Togi, a startup that I invested $100 million in, will be launching. To Togi is introducing two products to the market this week at MWC. The first is a planetary scale charger.  The second product that to Togi is introducing is a planetary scale BSS system built on top of the TM form Open APIs. Both of these products will be available for viewing in the virtual booth, as well as on the show floor. The public cloud is an unstoppable megatrend that's coming to telco. I'm super excited to bring to you the vendors, the products, the demonstrations, and the speakers, both to people here in Barcelona and virtually around the world. Sonating insight into the origins of telco DR, why public cloud is going to truly disrupt the telecoms industry, and why DR herself is so passionate about it. If you'd like to find out more, come and see us at Cloud City. Okay, thanks for rolling that reel. Chloe, I mean, look at that reel.  I've seen a lot of power players in the industry. She's got guts and determination, and she's got a vision, and she's not just, you know, making noise about telco and cloud. This is actually a lot of real good vision there.  No, it really isn't. And for me, it's almost like the next moonshot. It's the moonshot of the telco world. She's innovative, she's exciting, and if we've learned anything over the last 18 months, it's that we need that in this industry to grow and for the future of the industry. So, so exciting. I think she's a real inspiration. And I love the fact that she so takes a tiger by the tail because the telco industry is being disrupted. She's just driving the bus here. And I remember I did a story on Teresa Carlson who was with Amazon Web Services. She was running the public sector and she was doing the same exact thing in that public sector role in DC and around the world. She opened up Regents in Bahrain, which as a woman, that was an amazing accomplishment. And she wasn't just a woman. She was just a power player and she was exceptional leader. ICDR doing the same thing. And people aren't going to like that. I tell you right now. People are going to be like, whoa, what's going on here? No, of course, it's always the way with pioneers. So isn't it? And then in time, people thinking, what is going on? We don't like change. Why are we being shaken up? But actually afterwards in retrospect, they think, oh, okay. I see why that happened and we needed it. So really exciting stuff. Making things happen. That's what we're doing here in theCUBE. Obviously the main stage doing a great job. Let's go check out this highlight reel. If you're watching and you miss some of the action, this is, I'll see the physical event back since 2019 in February. But there's also a hybrid event, a lot of virtual action going on. So you got theCUBE virtual. You got a lot of content on virtual sites. In person here, we're going to go show you a highlight reel from what we did yesterday, what was happening around the show. Enjoy this quick highlight reel from yesterday. Okay, we're back here in theCUBE. We're on the main floor. I'm here with Chloe Richardson, who is emceeing, hosting and driving the content on the Cloud City main stage. Chloe, it's been great here. I mean, so far day one. I was watching your presentations and it's fireside chats. You've been hosting awesome content.  Yeah, no for sure. We had Scott Brighton on yesterday who was our opening keynote on the live stage. And his session was all about the future of work, which is so relevant and so pertinent to now. And he talked about the way it's changing. And in 10 years, it's going to be a trillion dollar industry to be in the cloud at work. So really interesting. I mean, yeah, the atmosphere here is great. Everyone's excited. It's new content every day. And that's the thing. It's not stale content. It's stuff that people want to hear. People are here for the new hot trends, the new hot topics. Really exciting. Yeah, the next big thing. And also it's a physical event. So it's since 2019, this Mobile World Congress has been a massive event and it hasn't happened since February 2019. That's a lot of time that's elapsed in the industry because of COVID. And people are glad to be here, but a lot of stuff's changed. Yeah, it's a different world, right?  Everything has changed, digital transformation, migration, obviously cloud, which is what we're talking about over here at Cloud City Live.  It has changed in the last two years. We know that. But what about the future of Mobile World Congress? How do you see it changing in the next few years? Oh man, that's a great question. I mean, my observation, I've been coming to this show for a very long time, over a decade and a half. And it's been a nerdy show about networks and telecom, which is basically radios and wireless and then mobile. It's very global, a lot of networks, but now it's evolving. And a lot of people are saying, and we were talking on theCUBE yesterday, Dave Vellante was commenting, that this show is turning into a consumer-like show. So CES is the big consumer electronics show in the US, in Las Vegas every year. This show has got a vibe because all the technology from the cloud players and from the chips are getting smaller, faster, cheaper, more capability, lower power. So if you look at the chips, the hardware, it's less about the speeds and fees. It's more about the consumer experience. You got cars. I was talking to a guy yesterday. He said, vehicle e-commerce is coming.  And you could be on your app driving down the freeway and go, hey, I want some food.  So that can be happening now in real time. You can do all kinds of other things. So a lot of new things are happening. Yeah, that's exciting. Do you see that as another disruption for the industry then? Is the fact that it's moving to be more consumer-focused, is there anything we should be worried about in that space? Well, I think the incumbents are going to lose their position. So I think in any new shift, new brands come in out of nowhere. And it's the people that you don't think about. It's the company that you don't see. And we've got DR on the main stage right here. Look at this. We're sort of walking out of the confidence of a pro. Yeah. She just walked out there and she's not afraid. No, well, as she said in her video, she is ready to wake them up and you can see soon she walked out. That is what she tends to do today. I love her mojo. She's got a lot of energy. And back to the show. I mean, she's just an example of what I was saying. Like in every market shift, a new brand emerges.  They were going to run out of cash. When Steve Jobs brought back Apple, he consolidated and rebooted the company. The iPad was a seminal iPod with a seminal moment. Then the iPhone, it's just a restless history. That kind of disruption is coming. You're going to see that here. Oh, it's exciting though, isn't it? To be future ready rather than future proof. But actually, I wanted to ask you something as well because we are seeing all these cloud players getting hot under the collar about Telco. Why are they so excited? What's the buzz about? Why, as you're in AWS and Google Cloud, why do they want to have a slice of the pie? Well, I think they're hot and heavy on the fact that Telco is a ripe opportunity. And it used to be this boring, slow-moving glacier. It's almost like global warming now, the icebergs are melting and it's going to just change. And because of the edge, 5G is not a consumer wireless thing. It's not like a better phone. It's a commercial opportunity because it's high bandwidth. We've all been to concerts or football games or sporting events where stadiums packed, everyone gets bars on their Wi-Fi but can't get out, can't upload their pictures to Instagram. Why? Because it's choking on the network. That's where 5G solves the problem. It brings a lot of bandwidth and that's going to bring the edge to life and that's money. So when you got money and greed and power changing hands, it's on the table and the wheel's spinning and it could be double zero or it could be lucky seven. You don't know. And that's certainly enough to get all the big players hot and bothered about getting involved. And I suppose it circles back to the fact that DR is really leading the charge and they're probably thinking, okay, what's going on here? This is different. We want something new. You did notice an open run is something that we've been talking about over the last day or so. We've heard quite a few of our speakers over here at Cloud City Live mention open run. What is it all about, John? Because why all the buzz if 5G is such a hot topic? Why are we getting excited about open run? That's a great thing. 5G certainly is Google Drive the main trend for sure. Open run is essentially an answer to the fact that 5G is popular and they need more infrastructure. So open source, the Linux foundation has been the driver for most open source software. So they're trying to bring open software and open architectures to create more entrepreneurial activity around hardware and around infrastructure because we need more infrastructure. We need more antennas. We need more transceivers. We need more devices that could be open. So in order to do that, you got to open up the technology and you want to minimize the licensing and minimize a lot of these proprietary aspects. What have we looked at? So on Wednesday we've got a great keynote from Philippe Longbois who is CEO and founder of P1 Security and he's coming to talk to us about cyber security within the cloud and within Telco. So you just mentioned there, open run is all about having open source, about having that space where we can share more efficiently and more easily. What does that mean for security though? Is it a risk? I think it's going to increase the value of security and minimize the threats because open source, even though it's open, the more people that are working on it, the more secure it could be. So yes, it could be more open in a sense that could be excluded by hackers but it could be open to also protect. And I think we've seen open source and cloud in particular be more secure because everyone said cloud's not secure, open source isn't secure and as it turns out when the collective hive minds of developers work on things, it gets secure. And it's interesting isn't it? Because we have seen that there has been an uptick in cyber security threats but actually I was speaking to some leaders in across various industries and particularly in tech and they were saying actually there's not been an uptick in attempted threats, there's been an uptick because with this open source environment we are able to track them and measure them and defend more efficiently. So actually they're being batted away but the number is probably the same as it always is. We just didn't know about them before we had this open source environment. There's more money in threats and there's more surface area. So as the tide rises, so do the threats. So on a net basis it's more because it's more volume but it's pretty much the same. And look at this money involved, they're organized. There's a business model on attacking and getting the cash out of your bank or ransomware at an all time high. So this is like a big problem and it's beyond the government that's at individual freedom. So security is huge and I think open source and cloud are going to be, I think the answer to that. Yeah for sure. And it's again about collaboration isn't it? Which we talk about all the time but without collaboration the industries aren't going to be able to work together to promote this environment. So yeah, it should be good to talk from Philippe on Wednesday. I was just saying security don't download that PDF if you don't know where it came from. The phishing is always good. Well we got some great stuff coming up. We're going to have a great day. We got a video here on Mobile World Live. We're going to show this next segment and we're going to toss it to a video. And this is really about to give the experience, Chloe, for people who aren't here, right? To get a feel for what's going on in Barcelona and all the action. And if you look at the video, enjoy it. I'm Daniel Royston, CEO and founder of TelcoDR. But you can call me DR. Ready for some more straight talk about Telco? It's go time. Let's do it.  It sure is a great time to be a tech company. I mean, if you're Amazon, Microsoft, Google, Grab, Twilio, DoorDash or Uber, life's pretty great. Just look at these stock prices over the past five years with their shareholder value going up and to the right. Totally amazing. But where's Telco? Dare I add our stocks to this awesome chart? Let's compare these fabulous tech stocks to AT&T, Vodafone, Telefonica, Tim, American Mobile and Zane Group. Huh, not so great, right? Yep, I'm talking directly to you, senior Telco execs.  Why is it that Wall Street doesn't see you as tech? Why aren't CSPs seen as driving all the tech change? Why is it always Apple, Amazon and Google who get the big buzz? But more importantly, why isn't it you? Before I came to this industry, I always thought of carriers as tech companies. I gave more of my money to AT&T than to Apple because I really cared about the quality of the network. But I also wondered why on earth the carriers allowed all the other tech companies to take center stage. After spending the last few years in Telco, I now understand why. It's because you are network people. You are not customer people.  It's crazy expensive to build a network and it's expensive to buy spectrum. It takes operational chops to run a killer network and it takes great skill to convince Wall Street to finance all of it. You Telco execs are amazing at all those things. But because you focus on the network, it means you don't focus on the customer. And so far, you haven't had to.  You don't have to be the best, just don't be last. Everyone else's MPS is in the 30s too. Their mobile operating are just as terrible as yours.  Let's talk about the cost of that. The cost is not measured on market share against other MNOs. The cost is measured in lost RPU that the tech guys are getting. Everyone knows about the loss of texting to WeChat, WhatsApp, and the other OTT apps. But it is not just texting. The total adjustable market or TAM of the mobile app disruptors is huge. Instead of remaining network focused, you should be leveraging your network into a premier position.  That is not what I'm talking about. I'm talking about love, customer love. There is one thing the highly valued tech companies all have in common. They all crush it on customer love. They look at every interaction with the customer and say, how do we make the customer love this? Like, Netflix has easy monthly cancellation. Amazon does no questions asked returns. Uber gives users a real time view into driver rating and availability. Compare those ideas to the standard telco customer interaction. The highly valued tech companies don't have the network oligopoly to fall back on like you do. To survive, they must make customers love them. So they focus on it in a big way and it pays off. Their MPS is close to 70 and they have app ratings of 4.5 or higher. A far cry from your 30s MPS and app ratings of 3.5. If you wanna have those huge tech multiples for yourself, you have to start thinking about these guys as your new competition, not the other telcos in your market. The crazy thing is, if you give up using your network as a crutch and put all of your focus on the customer, then network becomes an asset worth more than all the super apps. Let's step back and talk about the value of super apps and becoming customer centric. Retooling around the customer is a huge change. So let's make sure it's worth it. We aren't talking about 25% improvement. I'm gonna show you that if you become customer centric, you can double your ARPU, double your valuation multiples and drive big shareholder value just like the tech companies on that chart. Now, let's talk about the customer focused super apps. There are hundreds of companies and a variety of categories vying for your subscribers disposable income, movies, food delivery, financial services. Who are they and why does Wall Street give them such high valuations and like them so much? Well, first, look at what they are telling Wall Street about their TAM. They broadcast ridiculously huge TAMs that are greater than the telco TAMs. You know who should have a ridiculously huge TAM? You, hello. What I'm saying is that if you've got what's yours, you double in size. And if you take the TAMs they throw around, you'd be five times as big. When I think about the opportunity to double ARPU without having to double the CapEx to build out the network, I say to myself, hell yeah, we should totally go do it and do whatever it takes to go get it. For example, let's talk about Grab. Grab is a Southeast Asian super app company with an expected $40 billion valuation. Grab's customer focus started in rideshare, but then leveraged its customer love into wallet deliveries, hospitality and investing. ARPU is now larger than the telcos ARPU in countries where they compete. And they have a higher valuation than those telcos too. Imagine if you could combine a great user experience with valuable services that helped grow your ARPU. That would be huge. So how do you build a super app? I bet right about now you're wishing you had a super app. Everyone wants a super app. A lot of money has been unsuccessfully spent by telcos trying to build their own. I bet you're saying to yourself, DR, your pie in the sky sounds great, but it has no chance of success. Well, I'm betting things are about to change. There is a public cloud startup called Tatogi that is gonna help carriers build world-class super apps. To have a successful super app, there is one key metric you need to know. It is the KPI that determines if your super app will be a success or a flop. It's not about the daily act of users. It's not the average order value. It's not even gross merchandise value. It's all about the frequency of use per day by the user. That's the metric that matters. How many of you use that metric in your telco apps? Do you have a team driving up user app interactions every day? Most telco apps are used for top-up or to check a bill. This is a huge missed opportunity. Super app companies excel at building great experiences and driving a huge amount of interactions. They have to. Their business depends on it. They have to be customer-focused. They have to keep bringing the user back to the app every day multiple times a day. And you know what? They do a great job. Customers love their super apps. They have great user experiences, like Apple credit cards, no information-required application process. They have high net promoter scores because of customer-friendly policies, like how DoorDash retroactively credits fees when you move to a better plan. And they have great App Store ratings because they do simple things, like remember your last order, or allow you to use the app rather than force you to call customer service. Customers of successful super apps love it when new services are added. And because of the customer love, every time something is added to the app, customers adopt it immediately. New services drive frequent daily user interactions. So our problem in telco is we have an app that is only open once per month, not multiple times per day. And without frequent opens, there is no super app. Hmm, what do we have in telco that we could use to help with this problem? I wonder, while you don't currently have a mobile app that subscribers use multiple times a day, you have something that's 10 times better. You have a network. Subscribers already interact with your network 10 times more frequently than any user with any of the super apps. But telcos don't leverage those interactions into the insanely valuable engagements they could be. Worse, even if you wanted to, your crappy over-customized on-premise solutions make it impossible. Thankfully, there's this new tech that's come around, you may have heard of it, the public cloud. When you bring the enabling technology of the public cloud, you can turn your network interactions into valuable super app interactions. And there's a special new startup that's gonna help you do it, Tatogi. Tatogi will leverage all those network interactions and turn them into valuable customer interactions. Let me repeat that. Tatogi will leverage all those network interactions and turn them into valuable customer interactions. Tatogi allows a carrier to leverage its network and all the network interactions into customer engagement. This is something the super apps don't have, but will wish they did. But this magic technology is not enough. Telcos also need to move from being network focused to being customer focused. Tatogi enables telcos to chase exciting revenue growth without that annoying massive capex investment. Tatogi is gonna help you transform your sucky mobile apps with the crappy customer ratings into something your subscribers want to open multiple times a day and become a platform for growth. I'm so excited about Tatogi, I'm investing $100 million into it. You heard me right, $100 million. Is this what it feels like to be SoftBank? I'm investing in Tatogi because it's gonna enable telcos to leverage the network interactions into super app usage, which will lead to an improved subscriber experience and will give you a massive jump in your arpu. And once you do that, all those telco evaluations will go from down here brrrr to up here. And so, I've been talking to some folks, you know, checking in, feeling them out, getting their thoughts. And I've been asking them, what do you think about telcos building super apps? And the response has been, click eh. Everyone says no way, telcos can't do it. Zero chance, total goose egg. One suggested I build a bonfire with $100 million because then at least I wouldn't waste years of my life. Well, I think those people are dead wrong. I do believe that telcos can build super apps and make them super successful. The public cloud is changing all parts of telco and Tatogi and super apps are fundamentally changing the customer relationships. In one month at MWC, people will see what Tatogi has to offer and they will understand why I'm making this bold call. Because Tatogi takes the value of the network and the power of the public cloud to help telcos move from being network-centric to being customer-centric, boom. If you wanna make this transformation and reap all the financial benefits, you will have to compete for customers with a whole new set of players. You will no longer compete with the network-focused guys like the other telcos. Instead, you will be competing against the customer-focused companies. These players don't have a network to fall back on like your old competitors. They know they have to make customers love them. Their customer loyalty is so off the charts their customers are called fans. So if you want that big money, you will have to compete on their turf and make the customers want to choose you. You need Apple level loyalty. That bar is Uber high. We will have to give up the security blanket of the network and change. Instead of MPS in the 30s, it needs to be in the 70s. Instead of mobile operating in the threes, they need to get five stars. I'm betting big that Tatogi will make that possible. I'm gonna help you every step of the way starting with my keynote next month at MWC. Join me and I'll share the secrets to converting your super valuable network interactions to make your super app a massive success. We're going to have an amazing time and I can't wait to see you there. Okay, we're back here in the cube here at Mobile World Congress in Cloud City. I'm John Furrier. Chloe Richardson, filling in for Dave Vellante who's out on assignment. He's out getting all the data out there and getting stories. Chloe, what a great keynote by Daniel Royston. We just heard her and Huawei. Major action, Major, pump me up, punch in the face. Wake the heck up, Cloud people. Cloud is here, she didn't pull any punches. No, and I mean, the thing is, John, there's trillions of dollars on the table and everyone seems to be fighting for it. And you heard her up there. If you're not on the public cloud, you're not going to get access to that money. It's a free for all. And I think the Cloud people are like, they might think they're going to walk right in and the telco industry's going to just give it up. There's not going to be, it's going to be a fight. Who will win? Who will win, but also who will build the next big thing? Someone needs to die in the media conversations. It's always a fight, something's dead, something's dead, but it keeps living. All that kidding aside, this is really about partnering. I think what's happened is telcos already acknowledged that they need to change. And the 5G edge conversation, the chip acceleration, look at Apple, they got their own processors and Nvidia, Amazon makes their own chips, Intel's pumping stuff out. You got Qualcomm, you got all these new things. So the chips are getting faster and the software's more open source. And I'm telling you, the cloud is just going to drive that bus right down Cloud Street and it's going to be in Cloud City, everywhere. And it's going to be peeping on the horn as it drives down. John, I'm not a stalker, but I have read some of the things that you've written. And one of the things you mentioned that was really interesting was the difference between building and operating. Break it down for me, what does that mean? That means basically in mature markets and growing markets, things behave differently and certainly economics and the people and the makeup and the mindset. So the telco's been kind of this mature market. It's been changing and growing, but not like radically. Cost optimization, make profit, install a lot of cable.  Cloud is a growth market and it's about building, not just operating. And you got operators, carriers are operating networks. So you're going to see the convergence of operators and builders coming together. Builders being software developers, new technology and executives that think about building. And you want people on your team that are going to be, I won't say wartime, you know, lieutenants or generals, but people who can handle the pace of change. Because the change and the nature is different. And some people want slow and steady, keep the boat from rocking, but in a growth market, it's turbulent and the ride might not be quiet, first class ticket to paradise, but it's bumpy, but it's thrilling. No, of course, is it similar to the old sales adage of Hunter versus farmer, the parallels? Yeah, I mean, there's a mindset. If you have a team of people that aren't knocking down new opportunities and building the next big thing, fixing your house, get your house in order, you know, refactor, reset, reboot, replatform with the cloud and then refactor your business. If you don't have the people thinking like that, you're probably either going to be taken over or go out of business. And that's what the telco with all these assets, they're going to get bought, roll into a SPAC, special purpose acquisition company which is super hot in the United States, a lot of roll ups going on with private equity. So a lot of these telcos, if they don't refactor or replatform, then refactor, they're going to be toast and they're going to get rolled up and eaten up by somebody else. Yeah, sure, it's interesting there, isn't it? Because when we think of telco and tech, we often think of, obviously we've got the triad people process technology and we think the process and technology really take the forefront here. But like you said there, people are also so important because you don't have this right balance. You're not going to be able to drive that change. We had, obviously it's got right on the stage yesterday and after his session, somebody came up to me and just said, I'm interested to hear what that means for education. So how can we establish this new generation of tech and telco leaders from the grassroots with educational associations, establishments? How can we encourage that? I wonder, is this something that you talk about often? I mean, education is huge and this highlights the change that telco is now part of. Telco used to be a boring industry that ran the networks or moving packets around and mobile was there, but once the iPhone came out in 2007, the life has changed, society's changed, education's changed, how people interact has changed. So you start to see people now aware of the value and if you look at during COVID, the internet didn't crash, the telcos actually saved our assets and everyone survived because the network didn't break. Yeah, we had some bad Zoom meetings here and there and some teleconferences that didn't go well, but for the most part, we survived and they really saved everybody, my opinion. So they should get kudos for that, but now they're dependent upon healthcare, education. People care about that stuff. So now you're going to start to see an elevated focus on what telecom's doing. That's why the edge is got trillions of dollars up for grabs, but education, there's negative unemployment in cybersecurity and in cloud.  You can make 100,000 pounds and be unemployed six months and then be employed. So negative unemployment means there's more jobs and people to fill them qualified. Yeah, it's interesting you mentioned that because I was talking to a cybersecurity leader who was saying a similar thing. There are now three million vacancies in cybersecurity and there's such a skills shortage, there is nobody around to fill it. So it's an interesting problem to have, isn't it? Because it's reversed to what we've been used to for the last few decades and obviously telecom's is in the same space. What can we do about it? Do you think to actually bring people in? I think it's going to take leadership and I'm a big proponent of kids not going to university if they don't have to. Why spend the dough if you don't, money if you don't have to, you can get online. I mean, the data's there, but to me, it's the relationships, the mentorship, you start to see women in tech and underrepresented minorities in the tech field where mentorship is more important than curriculum. Community is more important than just going through a linear courseware. Nobody wants to sit online and go through linear courseware. Now if they have to get a certificate or degree and a accreditation, no problem, but communities are out there, so that's a big change over, I'm a big fan of that and I think people should get some specialized skills, you can get that online. So why even go to school? So people are figuring that out. For sure, and also even transferring, I mean, so many skills are transferable nowadays, aren't they? So we could easily be talking to people from other industries and bringing them into Telco and saying, look, bring what you know from your retail background or your healthcare background and help us at Telco to again drive forward, just like DR is saying, it's all about the next big thing.  Now the jobs that are available weren't around five years ago. So there's no like pedigree or track record, there's no like, everyone's equal. So you could, the democratization of the internet now is from a job standpoint is people are leveling up faster. So it's not about the Ivy league or the big degree or silver spoon in your mouth, you've got the entitlement. So you start to see people emergent and make things happen, entrepreneurship and American immigrant entrepreneurship. People are billionaires that have no high school diplomas. It's interesting you mentioned that John because we can't have more than five years experience in this space, we know that. But in telco, there is a problem and maybe it's, again, it's a flipped problem where telco recruiters or talent acquisition leaders are now asking for kind of 10, 20 years experience when they're sending out job descriptions. So does that mean that we are at fault for not being able to fill all these vacancies? Well, I mean, I think that's just, I mean, I think there's a transition to the new skill set happening one. But two, I think you got to be like a chip engineer. You can't learn that online. But if you want to run a cloud infrastructure, you can. But I think embedded systems is an area that I was talking to an engineer. There's a huge shortage of engineers who code on the microprocessors, on the chips. So embedded systems is a big career. So there's definitely paths you can specialize. Space is another area you've seen a lot of activity on. I see Jeff Bezos and Elon Musk is going to be here on a virtual keynote, trying to go to Mars. And Daniel Rosen always says, who's going to happen first? Mars, Colony, or Telco adopting public cloud. Some people think Mars will happen first, but. What do you think, John? I think Telco's going to get cloud. I mean, first of all, public cloud is now hybrid cloud. And the edge, this whole internet edge, 5G, is so symbolic and so important because it's an architectural beachhead. And that's where the trillion dollar baby is. So the inside baseball and the inside money and all the investors are focusing on the edge because whoever can command the edge wins all the dollars. So everyone kind of knows it's a public secret. It's fun to watch everyone jockey for the positions. Yeah, no, no, it really is. But it's also quite funny, isn't it? Because the edge is almost where we were decades ago but we're putting the control back in the hands of consumers. So it's an interesting flip. And I wonder if with the edge we can really enhance this acceleration of product development, this efficiency, this frictionless system in which we live. And also, I've heard you say hybrid a few times, John. Is hybrid going to be the future of the world no matter what industry you're in? Hybrid is everything now. So we're the hybrid cube. We've got hybrid cloud. Exactly. You've got hybrid Telco. Because now you've got the confluence of online and offline coming together. That is critical dynamic. And you're seeing it. Like virtual reality, for instance, now you're seeing things. I know you guys are doing some great work at your company around creating experiences that are virtual. Exactly. You got companies like Roblox went public recently. Metaverse is a good time to be in that business because experiential human relations are coming. So I think that's going to be powered by 5G, gamers. So all good stuff. Chloe, great to be with you here on theCUBE. And we're looking forward to seeing your main stage. Great. We're going to send it back to the studio, Adam and the team. We're waiting for DR to arrive here in Cloud City. And this is theCUBE from Cloud City. Back to you, Adam in the studio.


Our second lesson. In this lesson, I will try with you to build the first neural network, a small one. We remember from just the recap from what we did in the first lesson for the logistic regression. We had our input X, these were the data sets, and in the data set you remember we had features. We used, for example, for the classification of the picture, we used the pixels as a feature, but for other objects you can use as a feature. This is your interpretation, it comes from the domain of application. Features can be different, they are related to the application. We have our weights and the bias B, and we had our function C equals W transparent X plus B, these are of course all vectors. And then we used the sigmoid function just for the activation to make a decision between zero and one. This is what we did, and the output was a zero and a one, and based on that we get the prediction. And then we were looking for the loss function of our prediction and the label, this means this is the difference. But based on the logistic, we have the logistic equation of course, and we are summing up about all examples. Then we have the cost function, and for the cost function, based on the cost function, then we are calculating the differences dW and dV. These are our partial derivatives of the cost function dJ, dW, and dJ, dV. And last but not least, after all these calculations we are getting the new weights, and this is W minus alpha dW, and dV is B minus alpha dV. And based with these new weights and the new bias, we are doing this the same. And then if you are doing it a lot of time, and based on a really good training set, you have a good trained logistic regression work. In our case now we will see we will also do a network. That is just what we discussed now. And the small difference now is that we have, here you see we have our each and only one node. If we are working with neural network, then at least we have three layers. We have the input layer, we have the hidden layer, and we have the output layer. And the figure, I mean the connection figure which we are using for the logistic regression, now we are applying to every of these nodes. And you see every input is connected to every node. And every connection here has its own weight. In that case we have only three weights plus weight coefficients, and plus the bias is means four perimeters. In that case we are getting three by four already 12 perimeters for the first layer. And here we are getting plus four, because the bias is not included. The bias you should always think this is a special included. Here we are getting plus four, this means we should train, we should train already 16. What is very simple example, 16 perimeters. Again this is not a problem to train 16 perimeters, but it's just only to give you an understanding how the number of perimeters is increasing. So for the training we have now to calculate not only one set, here we have only one set to calculate and using this set to apply to this input to apply the sigmoid function. Now we have at least three sets, three times to apply this. Of course we can do this using not any more vector, but using some matrix. How this we will discuss later. And get an output. An output here, again here in that case we are writing a sigmoid function, but we will discuss is this maybe necessary, is it maybe necessary to replace the sigmoid function with another activation function. Why I will discuss this later. And then we are applying this output again here at that point. We have a second set of weights plus the bias. We are getting a third output. This output we are applying for this output again we are applying the sigmoid function and we are getting our prediction. This is our prediction. And then we are checking the prediction against the label. And again if this is fitting we are happy and if not then we have to calculate all the differences so we are going back. So we are calculating our DA, we are calculating our reset and last but least we are calculating our DW and DB. And then we are applying the same formulas and we change the parameters. And running this again. This is then running in a loop. How to fix it. First of all about the representation of a neural network. As I said already the neural network, the neural network we have this input layer. We have at least here one bit layer and we have an output layer. If we have binary classification then we will have here only one neuron. If we have let's say a classification which is used for example for the unmanned vehicle driving. Then of course the class is much bigger. Of course we should say this is a traffic sign. This is a NASA car. This is a bicyclist. This is some people who is walking. This is a tree. Don't drive to the tree.  This is the car beside you and this is the car behind you. So all these objects should be detected and of course also their locations. This means in that case we have already not only a binary classification. We have a multiple classification. And the algorithm is a little bit changing but not really. The idea behind is always the same. Only the implementation is much more difficult. But if we have a binary classification that the activation function at the output is always a sigmoid function. Of course we have one or zero. But here at these neurons or units we will not use the sigmoid function. We will use another function. If we have a multiple classification then we will not use sigmoid function. We will use a softmax function like this. You can learn later. But it's just only for your understanding. Of course it is not necessary that we are using only one neuron on the output. But it is simple to start with simplification. So as you remember for the calculation this was the input multiplication with the weights plus bias. And this was the argument for the application of the activation function. And the activation function is deciding is this neuron firing or not. This is the reason why it's called activation function. If it's zero it's not firing. If it's one then it's firing. I will not discuss now how similar this is to the work of our brain. But at least some similarity is there to the work of the neurons. And this is the reason why this was also called the activation function. So in the case of a stellar network we have this bigger now. We should apply this to every of these neurons in the hidden layer. This means we are calculating this for this weights. Then we are getting one output. And you see now what we need is not only a notification or let's say an indication on which neuron we are working. We also should say on which layer we are working. Because this is layer zero. This is layer one and this will be layer two. We are always starting the indication with zero. And you understand if this will be a deeper network then it's just we are getting additional layers. It's quite easy to add layers. The same is then working for the other neurons. We have one in the middle here again. You see this is for the second, for the third and so on and so on. So if I put this together, you can see what is written here. These are vectors. But why to multiply this as a vector? I can put these vectors as rows. I can put it as columns in some matrix and then transpose this matrix. So we have two matrices. We have a matrix X. We have a matrix X where I have here my data sets. We said we have m data sets and I have my matrix W. And in the matrix W I have here my weights. This is the matrix W for layer one. And here I have the first weights, the second weights of this layer. And here I have four. Let's say how much I will need an X. No, sorry. This is one, but this is an X. And to multiply this, I'm just only transposing. And then I can multiply it. And then in one step, at least, I calculate my sets for the first layer. So I'm adding, you can add here some vector B also from the first layer. So this means this can be this operation. This operation can be written at one line. Of course, from a mathematical point of view, you cannot really, of course, the output here is a matrix. And here I have a vector. Because matrix times matrix gives me a matrix. But adding a vector, this is not really possible. From a mathematical point of view, I could enlarge the dimension multiplying this identity matrix. But in Python, this is quite easy. Of course, this is a broadcasting of the operation. Python understands that it should add this number to every output. So this is quite easy to write. It's the same like in Matlab. In Matlab you have also this broadcasting of the operation. So when we have this given input, when we have this given input, we calculate it. Then we apply to this input our activation function. And this is now the activation function for just the whole, for the first layer. Then we have this output and the output is the input on the second layer. Of course, you should make sure that the dimensions of your multiplications are always in the right shape. So this is from a practical implementation. From a practical implementation, one of the, let's say, the tricky things, where you should pay attention to that. Otherwise, your algorithms either will not work or you will have problems with the algorithms. And last but not least, we are getting the output. But in that case, this will be a number. And this also will be a number. Of course, we have only one prediction, only one prediction. But if you have multiple classification, then, of course, you will get the vector of output. This is one of the important points for neural networks. Yes, it's more that the first neural networks were introduced somewhere in the 60s. And they started to work with the neural network. Maybe somebody of you remembers the name Prasek Tron. This was one of the first neural networks which were used. They were used as activation function, a unit step function. Every engineer knows what is a unit step function. And I hope that also the people from computer science know this. You see, this is this function. It makes sometimes also called heavy side function. With this function, they tried to do some applications and then came Marvin Minsky. And in one paper, he showed that the applications of this Prasek Tron are very, very limited by theoretical reasons. Not because the people are stupid, but because the theory behind and the neural networks were forgotten for 20 years. And only a group in Vancouver from George Hinton, he was working still on that and he tried to improve it. He did a lot of effort and they introduced also new activation functions and then they reached a really good success. What I told you in 2020, it was even that group, people from that group who had to break through. And based on that, it was working. So what is the problem? The problem is the following. If we have this function, this kind of function as activation function, then this function has a big problem is starting somewhere here and somewhere here because we have a saturation of the function. And this means even if the arguments are very, very different, the output is more or less the same. So and you cannot have, not anymore you can have a distinction. And this means, we will see this later, I will show this later, then you are replacing the input not with a non in your function, you are replacing this with identity function. And if you replace this with identity function, your algorithm is not learning. And if it's not learning, why to spend time? It's a waste of time. What we are doing is now that we are looking for other activation functions. As I said, at the output, if we have a binary, if we have a binary network at the output, of course we will use the sigmoid function. Of course it's good for a distinction between 1 and 0. But here we need functions which are more sensitive. So and one of the possibilities is of course the tension hyperbolic cross function. Because the tension hyperbolic cross function, first of all, it's not so steep. And the second you have at least an output between minus 1 and plus 1. And the last, let's say the group of hidden, what they really introduced was the use of this rectified linear unit function, which is 0 at that area and getting, let's say, it's a linear function at that area. But overall it's a non-linear function, but very easy to implement. Because it's just the maximum between 0 and the input. If the input is negative, then 0 is greater than a negative number. And so the output is 0. And if the input is positive, a positive number, or 0, the maximum of 0, 0 is 0. And the maximum of 0 and the positive number is the positive number. And by the way, what is the derivative of this function? What is the derivative of this function? This is just the constant function. It's that function. Because the slope here is 1 and the slope here is 0. So we are getting for this real function and we will need, we will need the derivative also of the activation function in our calculations. We are getting for the real function as the derivative just the unit function. So this is really easy. The problem, the problem of this function is the negative part. Because the negative part cancels out everything. So at that area you are learning, here you are learning nothing. So you should be, you should ensure during your calculations that the arguments for the activation functions are not getting negative. Otherwise you have too much units which are not learning. But a way out could be a so-called leaky real function, which you see here, the slope of this function in the negative part, the slope is less, but nevertheless you have a small slope. So also you are learning something in the negative part. How do you determine the slope on the left side of the rally? By experience. I will show you, for example it could be 0.001 or minus, sorry. Sorry, plus 0.001. It's a hyperparameter, you are defining it? No, it's not really a hyperparameter. You just implement this as the used activation function. Why don't we just use a linear function here? This one? Yeah. Because you are not learning nothing. You are replacing the input with the output. At that place, here, at that place we have some activation. And if the input is like the output, if we would use this function. Yes, the input is like the output. You are not learning enough. But in case of positive numbers, the input is... Yeah, but the negatives are cancelled out. This is a linear function, this is a non-linear function. So what we are doing is the following. We are replacing, we are replacing at that point where we had the sigmoid function at that layer with another activation function. Of course there are a lot of activation functions, as I show you at the table. It is not necessary now to read it. I only want to show you that this is not only two or three activation functions. There are a lot of activation functions you can look to tables. And it is not necessary, you find it in Wikipedia. The source for me was also Wikipedia because this... You never... Never else you get such a nice, such a nice overview. Because you have also here the properties of these activation functions. For example, monatonic, the derivative is monatonic. Each kind of continuity you have, what is the range for that and so on and so on. But nevertheless, of course I can use, as you said, the activation function, that this is not really good for learning. We have the binary step function for the perceptron or unit function. We have the logistic regression which we are normally using which we are normally using for classification. Tangent hyperbolicus and also arc tangent hyperbolicus, arc tangent. Both are used, both. But in the case of the arc tangent, you should have to be sure that your arguments are only between minus pi over two and plus pi over two. I see you are good students, not like mine students. They would ask you why. They forgot already what is arc tangent. And here you see also other functions. It is maybe also possible that that part is not linear, but here it has some special shape. But the good use for that function is, and also for the linear function, that the derivative is very easy to calculate and very easy to implement. And if we are thinking not only about the theory, but we are thinking also how can we make an algorithm which is working quite fast, then of course we should have also this in consideration. So, going back. But it would, that one looks linear even for... Which one? For negative numbers. This one? No. Yeah. Yeah, that one. Yeah, but I'm thinking about the range from minus infinity to plus infinity. From minus infinity to plus infinity to slant infinity. It is just, the parts are linear, yes. This is the idea. This is a stepwise function. It's a stepwise function. The parts are linear, so the derivatives are quite easy, but the function overall is slanted. And I have not this problem which I sometimes have with the real functions. But then the derivative is like this, and like it switches. Yeah, of course the derivative is something like that. This is the derivative of a league 3. That's true. But this just only... The factor is not very big. What's the reason that we decided to have something linear instead of 0? Because there it was not learning anything. But this or this one? Yes. Which one? What means... Why did we switch from that one to another one? Because it was not learning anything. Because sometimes if you have negative inputs, then it's not really... If you have too much... Yeah, in that case you should make sure that you have not too much negative inputs. In that case you should not think about that. And in the case of this function the problem is the saturation. The problem is the saturation. So this means if I'm taking here an argument, and I'm taking as an argument 1 million, the output is the same. If I'm taking in that case an input at 1 and 1 million, the output is very different. But it avoids this function, avoids the problem of an identity function. Because the identity function I said at least for classification is not learning. And you also have to have in mind in the logistic regression, in the logistic regression we are using a logarithm. And the argument of a logarithm should be positive. Yeah, this we discussed already. This means we are replacing the sigmoid function now with a different kind of activation functions. The activation function on the first layer on the set or on the different layers should not be the same. But the activation function for all units in one layer should be the same. So if you are going to the first layer, you should define normally the real function is used. But if you decide that the tension hyperbolic fits more or you like it more, then you can use it. But please use it for all units in the first layer. Then you are going to the second layer. And in the second layer you can use another activation function for a deeper network. Again, think about which kind of activation function makes sense. And please have also in mind changing the activation function means to implement elements. But if we replace, and this was your question, if we replace this with the linear function, then we have here C1. We are getting this, then the activation is C1. And this is going in here side. What we are learning, we learn nothing. Because in that case, if we are using this activation function, what we are doing is we are just multiplying matrices. If we have a look. This is the input. We have as an output, we are getting the multiplication of these two matrices with that. And we have two different parts, but it's looking in the same way. Do you have the input? Can I have one more question? You mentioned that it's not important to have, we can have binary like in that layer. But it's important that we have a linear function. In the first layer, the idea is to replace the sigmoid with one of these functions. This is not true. We should not replace it with a linear function, but this sigmoid function for a binary classification and for a multiple classification with a softmax function, which is something like the probability. If you have, let's say 10 classes, you are calculating something like the probability for every class in the output. And then you say, okay, I am taking the class and you say, okay, this is probability. How this is implemented? This is not really a probability, but very close. So this is about the activation function. Next discussion is about derivatives. So as you see, we will need also the slope. We will need the slope of our activation functions. We will need the calculation for the calculation of the new weights. So how to calculate this is quite easy mathematical exercise. If our sigmo offset is 1 divided by 1 plus e power by minus z, then sigma prime is then minus e power by minus z divided by 1 plus e power by minus z squared. Everybody agrees? Or should I show it? Then we can say this is the same. This is the same like 1 minus e power by minus z and plus 1, minus 1. And this is 1 minus e power by minus z squared. And we split this. We are getting here 1 divided by 1 plus e power by minus z. And minus 1 divided by 1 e power by minus z squared. And this is sigma offset minus sigma offset squared. sigma offset 1 minus sigma offset. And this fits always. So we always know what is the derivative and we can plug in.  So it is quite easy to use in an algorithm. So this we are using for the derivative of the sigmoid function. And in the same way, if you have tangent hyperbolicus offset, this is our new g offset, the new activation function. Then g prime is the derivative. And this you can calculate. This is 1 minus tangent hyperbolicus squared z. Look in any formula.  Sine hyperbolicus by 2. And this is the cosine hyperbolicus. Also by 2. 2 is called out. And then we have this much. So this is also very easy to use. Tension hyperbolicus. This is the tangent hyperbolicus function. Sine hyperbolicus divided by cosine hyperbolicus. And if you try a little bit, you will get to the same point. I will not use my time. You will find it in every book. So this means, if I call this a, then also this is 1 minus a squared. Very easy to calculate. So you see as the activation functions, we are not only looking for nonlinear functions and we are not only looking for nonlinear functions which fulfill the requirement that they give a sensitive output. We are also looking for activation functions which are easy to calculate where the derivatives are easy to calculate. Because we need this, we will need this in the, we will need this in the calculation of the derivatives of our cost function. So, and the derivatives of the real and the linear real or we discussed already, sorry there is something missing. So this is the unit step function in that case. And this is the unit step, but this side is a little bit higher. But normally we are taking here a value which is possibly but very, very small. So this means the slope which we have here should be a very, very, let's say, non-steep slope. How we are coming to the gradient. As a parameter, we have now the input of our, the number of units in the first, this is the dimension of my matrix for the weight in the first layer. This is the dimension for the bias vector. This is the dimension for the matrix on the second layer. And this is the dimension for the bias on the second layer. And if you have a look, this is always, this is the same. These are the same numbers and these are the same numbers also. So you have to have this, and you are checking the shapes of your matrices. You can do this in such a way. Because for this matrix, we will use the transpose. For this matrix, we also use the transpose. So then we are calculating our cost function based on all these matrices. And then we should take, get the gradient descent. And the gradient descent is calculated in the following. That I, I'm now checking what are the difference, what is the derivative of my cost function by the components of the first layer and by the first bias. And this I will use to improve these, these components, these parameters. In the same way, in the same way, I'm checking this also for the components on the, on the second layer. Yes, we are always calculating now the derivatives. So the implementation then would be the following. That from a formal point of view, I have my first output. Then I'm using the activation function on the first layer on this, this output is now the input here. I'm getting an output here. This output is used as an input to the second layer. We are using, we are using this, this weight and we are using this bias. And last, at least, we have here an output. And please have in mind in that case we are using the sigmoin function. If you implement this now for the backward propagation, we are first checking the difference at the output. So this is our predict, our predicted values. These are the labels. And the difference between predicted values and the, and this predicted value and the real labels gives us the differences in the second layer. We are using the differences in the second layer and multiply them with this formal, with this value. And we are getting the differences in the weights in the second layer. Because this was the input, as you see, this is the input to the first layer. And if you are taking the derivatives, then just A is left. So, and last, at least, the bias on the second layer is just the sum, is just the sum over my d-sets, which I have divided by the, the axis is just the programming question in which dimension I am looking for the sum, going for the columns or going for the rows. And then, in the same way, I am now, not in the, sorry, not in the same way, now in a special application I am calculating my d-sets on the first layer of these d-sets. And this is the multiplication of my weights from the second layer with these kind of differences and multiplied with the derivative, with the derivative of my activation function. And this is multiplied by element. This means this is not a matrix multiplication, this is just a multiplication. So, and based on that, if I have this value, then I can calculate the differences for the first, for the first parameters and also the differences for the bias. And if I have this, this, this, this, then I can apply this and change, change the parameters w, my w is in the first layer, my bias in the first layer, my parameters, my matrix w in the second layer and also the bias in the second layer. So, and this computation figure, now we are doing 1,000 times, 10,000 times, depends on you how long you will learn. The question, how often you will repeat this, this is again a practical question. Check the learning curve which I gave you. This means I have your differences or do you have a saturation of the learning curve if you have a saturation of the learning curve just right up. It makes no sense. This is in case of supervised learning. Yes, this is totally different. This is totally different. Let's say in the case of unsupervised learning, what we are doing is, first of all, we want in the case of, we can apply different kinds of unsupervised learning, but let me speak about the class today. In the case of clustering, it means we have some data and we want to find out which data are similar. But the question, how much clusters I will set, this is your decision. This is not given from the data. Of course you can have a first look on the scatter plot and say, okay, it looks so like there are three clouds or four clouds or five clouds. Then you can decide how you will calculate which data are going to which cloud. For example, you can try to work with the centers of cavitation or the mass centers. For every cloud you try to calculate the mass center, then you are calculating the distances when the distance between two mass centers is different than you choose the minimum. Then you separate this. Based on that, you can try to give labels and then make a supervised learning. But it is unsure if you really have the right number of clouds. This is just a practical question based on the problems, based on the practices and of course also on your experience. Because you can say, okay, it looks like three clouds, but I know there is an error in measurement and this error in measurement is responsible for the certain cloud. So for example, you can switch from unsupervised, just do unsupervised learning. First you are doing unsupervised learning and based on this you can try to make a supervised learning when it makes sense. This is, okay, if you like this to do it every day in the afternoon, after dinner you have nothing else to do, then of course you can do it for fun. But normally because the implementation of this takes a lot of time and the cleaning of the data, I didn't speak about it, but cleaning of the data takes most of the time, then I saw you will have a summer school in data science and I think the lectures will spend a lot of time on the data cleaning and you will see how much ever you will need for that. Then you can try to apply this different kind of order. Another, let's say application is reducing of the dimension of your objects or of your data. For example, if we are looking to the colors of a picture, then of course we cannot have only RGB from 0 to 255 and 3 channels, we can have also other values. In that case it makes sense to reduce and based on this, on the reduced picture nevertheless see if you can work with this. This is quite often done with a principle component analysis where you are working with the singular values of the matrix which you get from the pixels. Again, who knows a little bit are numerics and unfortunately I'm coming from mass, so I know how much computation effort you need to make a principle component analysis because this is something like the calculation of singular values of matrices and these are one of the most computation consuming operations and calculations which you can do in mass at all. So it's always a question makes a sense or not, but nevertheless if you have the reducing then you make your operations and then you go up again to the original dimension. But again, do you need it or not? It depends from the real application. Okay, but in the case of supervised learning with deep networks we will need to calculate the derivatives and to improve this with a learning rate. So coming to the back propagation which is the most challenging part. You see, the computer is so afraid about back propagation that he might show it. I'm sorry for that. So computing the gradients in the case of the logistic regression, maybe you remember this. This was, we go for the backward propagation, we go this way. And last but not least, we want to know what is the derivative of this value by w, by the w's, and by b. This is what we want to learn. Of course we could do this in such a way that we calculate this in one step. But then you don't have a computation scheme which you can apply in a program like a sub-program. So for this reason we are using another application. First of all we say, okay, we calculate the derivative of our cost function of our cost function by a. And this is quite easy.  So this we know. And then we calculate the cost function by set. And the cost function by set using a theory from mass of the chain rule of taking derivatives is dL by dA, dA by dZ. So dL by dA we already calculated. And this value, this value by set, this is just the derivative which we saw and which we discussed of our activation function by set. Of course A is nothing else like our activation function. And we discussed, we saw that the activation function for the activation function for let's say the sigmoid function has a special form, the activation function or let's say the derivative of the activation function for the attention hyperbolic function has a special form and so on. So this is also predefined from the activation function. And then it's quite easy to multiply this but have in mind we should multiply this element wise. We should not multiply this as vectors and we should not multiply this as matrices. So and based on that, based on that now we can calculate dA. Of course this is this value times the derivative. So we have dZ and on the output here we have our dZ on the other ways and then we can multiply and calculate dW and dV. Of course dV is the next sigmoid like dZ and dW is like dZ times X. So having in mind this computational scheme which we are getting from the logistic regression we can try to apply this again to the neural network. Because this was the computational chain this is the computational chain for our neural network. So we are starting again here and we are calculating dA on the second layer. And based on that this is we can calculate also dZ. This is the difference. And based on that we will calculate for the second layer our dW and dV and make then later our improvements. And then we are going back here we calculate our dA for this function and you can replace this sigmoid now with another activation function and calculate dZ based on that multiplied element wise with this product and now we have more or less everything. So we know how to calculate our dAs and we calculate now dW in the second layer our dV in the second layer our dW where I have it our dW in the first layer and also our dV in the first layer. And these are the values which I need to learn because now I can apply this first I calculate the vectorial application and I apply this for the learning algorithm this is the implementation already written how we can write this in Python and the update then is that I say I am learning this I always take I always say dWL is WL minus alpha dWL and I learn or dVL sorry, sorry, sorry, sorry VL this is the bias of layer L minus alpha dV on layer L So we are coming now to the question of initialization If you remember we initialized our indelogative regression sorry how do we get from here that we are actually correcting our weights because we measured how much different we were from inside now we are looking let's say so the back propagation gives us an insight how the output error is propagated to the different weights and biases and we should improve by this value we should improve this output sorry, these weights this is like an error propagation which you have if you are measuring something and then you see how this error measuring is propagated in your calculations I just want to see where this one I will discuss now I only say this is the way how I can improve it I say the new weights are the old weights minus learning weights like the difference and the new bias is the old bias always a layer L the only layer which we will not improve nothing is the input layer because the input you cannot improve the input is as it is now we are coming to the question about initialization in the case of the logistic regression I told you that I can initialize with zeros the question is can we also initialize with zeros let's see what happens and if I have my matrix with zeros and the bias with zeros if the matrix is zeros you remember this I multiply with my data plus zero then I will get here zero and it doesn't matter which kind of activation function you are using you are getting zeros you are learning this so this is the reason why it makes sense to use another way of initialization and the easiest thing for the initialization is if we are using random numbers so for these weights and also for these weights we initialize with random numbers and sometimes also it depends if you want the random number if you want the random number very small then multiply it again with some factor for example 0.01 so the random number in any case if this is a normal distributed random number is between zero and one nevertheless it's quite big so you can make the weight much smaller maybe hopefully you will learn better the bias, the initial bias could be zero it's not necessary you can also choose a random number but quite often we are using zero this is not the only approach for the initialization if you go to the literature one initialization is called the Hei initialization is already a Chinese influence for this Chinese also and another initialization is the Spanish name maybe I remember this one but it means you can have a look to the literature and find out different initialization methods because a good initialization shortens you the computation time or maybe not the computation time but the computation effort and if you know time is money then you are also spending money but let's say no algorithm which tells you for this application you should use this initialization for this application you should use this initialization this is a question of experience so a good start in any case is this kind of initialization but isn't it possible to create some kind of algorithm which can exist? no there are so many different problems where you apply machine learning and on one data this initialization with this you are learning faster on the other data with this initialization okay you will have your topic for the master thesis sorry maybe for the PNG let's say so I don't know such a kind but this should be discussed if this is really possible because in that case you also should be able to classify the applications for example now I am working on an application and then algorithm which is trying to guess what kind of features and cell pipelines our business can use based on their company profile so it's also like the same configuring something and somehow collect data maybe you can do this in that case you are checking the business cases and you say if I apply this to this business case then it makes sense to work with that kind of applications or with that kind of algorithms like a recommender system this is possible if you work on a recommender system and you have a breakthrough then you will get the same money as the team of one time from Netflix this was one million dollars not Netflix this was Netflix announced the competition the competition is over but they announced a competition to build a recommender system for the applications so that they have improvement of the benefit of 10% and of course one million dollars is not so less money and a lot of groups were working on that but the winner was not implemented this was the funny thing why? it was good the results were good but the effort for the implementation was too high and they decided to work with the second winner I think I have to reconnect but if there is any data I mean do people say it's not so that you have a table but of course one moment I will just but you should check the literature it is the only thing that I can tell you it is not so I didn't see at least in the literature you get the table and say these are the use cases these are the activation functions which you should use in that case and these are the initializations but one thing that I can tell you if you are working with a sigmoid function over all the network you will not get really good results with sigmoid functions so in the written layers it is recommended to work with so what we want to do we have a data set like that and we want a classifier or is classifying the red and the blue dots and you see the distribution is not really good distribution we will do this with two different kinds of algorithms we want to apply logistic regression and then a simple network we will see if we have improvement on that the matrix which we are using is just the so the shape of our x is 2 by 400 so this means we have 400 dots and of course we have also 400 links so first of all we will use simple logistic regression in that case I am a little bit lazy and I am using a library skill on with the skill on a library where we have logistic regression we will close the regression ok maybe to explain for people who don't know what is cross validation if you have your data then you can normally let's say the first thing what I told you was to split the data in a training and a test set but the problem in that case is the following if you make your training and you want to test it then you can use the test set only once then of course you can use the same data but again to reshuffle in a random way your data or you are working with other methods to pick the data out the improvement is always if you want to make an improvement you always have to reorganize your data a better way in that case is to split your algorithm in a training set a cross-validation set and a test set so for example this could be 60% 20% and this up to you a little bit to move here so what you are doing, you are training your algorithm and you are checking it with the cross-validation you are not satisfied, you change the hyper parameters of your algorithm check it again on the cross-validation and you do it as often as you want and last but not least you are satisfied only once on a test set what is the advantage the advantage is that you are avoiding the bias problem because every time if we are using if we are using our test set a second time when we improved already our algorithm, it is biased because we used already the results of the test set to improve your algorithm I know that in practice this is done but at least from a statistical point of view not really came and for this thank you for your remark in that case we are using the cross-validation I have a question what is hyperparameters for simple logistic regression why we do cross-validation in this case in that case I think 60, 20, 20 but this is automatically you just plug it in in the library this is like working on TensorFlow if you plug in a comment on TensorFlow you don't know what are the parameters behind and it's the same with the library here you should have a look directly to the library but I think it is this splitting what I showed you, 60, 20, 20 but you will also find 80, 10, 10 this is also possible so the important thing is to show that with the logistic regression we have not a really good result for the for the classification and you see the accuracy is about 47 percent and 47 percent even for machine learning is very, very bad so now you can try to use a neural network and here you have the input we have only one hidden layer we will use four units in the hidden layer the input we have only blue and red and as the activation function we will use here the tangent hyperbolic function and then the output we will use a sigmoin function so and the prediction the prediction is done based on this on this the prediction is if our output is greater than 0.5 then we will put it one and now we should decide it's one red or blue and otherwise so the cost function cost function for this is the same as you know already the next thing what we should implement is the layer size and the layer size is taken from our matrix X also for Y you remember this was the first the number of rows is the number of data and then also and Y is the number of labels which we have an hour later and we rejoin this these numbers how actually do we decide common layers unit? in that case we are speaking about this swallow network only one about let's say but even networks do speak on Wednesday of course there are some special things to have in mind what about nodes? how many nodes too? in that case we choose only four there is no rule which how many nodes I would say use a simple modeling rule, kiss, keep it stupid in that case I am using four nodes but what you will see later we will change the number of nodes and see if we are getting better results the point is the following not always if you are increasing let's say in other words if you are expecting that the increasement of the number of nodes increases your accuracy this is not true there will be some saturation and then maybe even the accuracy is going down so you check it because adding nodes is not such a problem and you can check it by let's say four nodes five nodes, ten nodes, fifteen nodes like cross validation you can but in that case it's really an other organ and we have not the problem so again you see the size of the input layer is five, the size of the hidden layer is four and the size of the output layer is two and this was also what was expected so about the model parameters so we want to have in the model parameters we want to initialize we want to initialize here the parameters for our weights and also for our bias in the first and in the second layer and you see I multiply this with a small factor you can also use even smaller factor you can also skip this factor but then the initialization the numbers are quite high because the random numbers between zero and one have mostly only one one precision of let's say the MSB is one and we want to have MSB zero and then again we are also taking the shapes and we are building something like a library so in the library we have different words and these words have values and first of all we are using the initialized values and later we will renew the values after every calculation so just only though that we see these are random numbers they have at least the second only the second position of the zero is different from zero of course we have also negative ones and the bias the initial bias is zero so now we are doing the forward propagation and as you can see the forward propagation is just using the mass which we discussed on the slides this is nothing special the only maybe important thing is that we are using the dot function and the dot operation attention from the NumPy library but this is just because we wanted to to work with matrices and this is just a test case we are forwarding we are checking the forward propagation test case and then see what is the output and for the output we are just reading the means if the means are more or less in the same region we are sure that the algorithm is working properly so the next is the cost computing I should implement the cost function but again the cost function is following the cost function is following the way how we define the cost function just only applying just only applying NumPy operations but this is just only because this is a matrix and if you would apply normal operations you would get the narrow as an output so we are coming to the challenging part the challenging part is the backward propagation this is in every implementation the backward propagation is the part where you spend most of the time so what we are doing you remember for the backward propagation we needed our parameters we needed cash and then we should calculate our increasements for C2, W2, B2, C1, W1 and B1 and to calculate this we are just following the formulas which we used you remember for the input arrow was this difference this we multiplied if we are multiplied as matrices this part with the output from the layer and so on and so on it's just following the formulas and here we are putting this in the library grats so we have now a library if you put in the input of your function here we have the parameters and we have cash and we are taking always the new values and the new values they are updated they are updated in the cash let's make a test more or less fine now we can try to work it's not that we are not finished so now it comes to the question of the step size this is just a demonstration if you take the step this is a small step and you see with the gradient descent you are reaching the minimum in that case the step is too big and this is just only jumping you will never reach the minimum and that's the point but even if you would have a longer time it would not reach because you can show it because the search this can be shown by mass the search direction is always perpendicular to the gradient this is always perpendicular and if this is too big because this is going up here again you are jumping just in this direction this is just a function for updating the parameters I will not show this in detail because I want to come to an end this is my example in Coursera I can give you this example later in the files which I gave you and now we have the model we are taking the parameters we initialize the parameters and then we run this in a loop we have the forward propagation we have the cost we have the backward propagation we have the updated parameters and it is just up to us to define how often we will go through this loop to make the training so now we are coming to the predictions as you see it looks a little bit better of course we have still arrows but of course in that region we have red points and in that region we have blue points most of the points here in that region and in that region we are red so this kind of classification is even better now but I guess it will get better if we repeat it many times the iteration was 10,000 but we can't improve anymore of course you can improve but it is a question if you have time of course I can change this 15,000 time just to see how much you see sorry this was an achievement of course it is already saturated the vision of my clients is what I want not really of course you see this is a problem what not really can be improved at least if I use such kind of boundaries if the boundaries would be nonlinear if I would use nonlinear boundaries then of course it could be improved and what would be the problem if nonlinear functions were done? no no this is nothing I am speaking about the discrimination the discrimination functions because you see the discrimination is done by a linear function of course it is possible to use nonlinear discrimination functions in that case maybe we can have it but this has nothing to do with the activation function the accuracy is about 90% it depends if the red points would be cancer and the blue point non-cancer I would say not it depends from the application but the sensitivity in that case will be not very high in my case 80% of my clients yes it depends from the application but in any case we have a big improvement against the accuracy which we decided in the beginning with the logistic progression and if you remember I defined machine learning, solving a concrete task defining a performance measure and run this with some concrete algorithm so let me run this for you it takes a little bit more time and this is a question of who asked me this? can you still see the fold that each wave is actually now this is of course if we can see we have to stop the algorithm to see what is in the library what I am doing is the following I am writing the new weights in my library and update the library and forget the old values and then I update again forget the old values this means I have to stop at the step I don't know 100 and then take the next step and somebody asked me what means adding units in the layer and this is for one hidden unit, two hidden units, three hidden units four hidden units, five hidden units and now we see if I even have two hidden units the accuracy is not improving but the calculation time and the calculation effort is of course increasing so this means not always increasing the number of units in the layer brings you better results but of course I would have better results if I would put the second layer I had to make another assumption that the discrimination curves are not linear of course here I have linear discrimination curves if I use nonlinear discrimination curves it will be better but with linear discrimination curves I am not sure that there will be and we have linear in one layer discrimination curves are undoubted may I ask a question for the hidden layers the nodes of the hidden layer should be equal or not different you will see this especially if you go for convolutional layers they are special playing they are playing on the dimensions of the hidden layers of course not otherwise it depends on your data now it depends not only on the data it depends really from the architecture what you want to do if you give me a little bit more time on Wednesday so for people who are really interested I can show a little bit different architectures and then you will see that the layers are very very for example if you take a Alexa architecture and compare this with a Google net architecture normally called Inception architecture and you will see the Alexa architecture has five layers the Inception architecture has around 21 and more it's much deeper and of course the dimension of the layers is then taken based on the function what you are expecting the layer should do because if you play on the dimensions on the layers because you are using in the layers different kind of filters this is the idea of a convolutional network the reason why it's called convolution of a convolution operation and this means there is really an idea behind which kind of dimensions I am using in that case we are just testing adding say ok fine 4 is fine but there is no special calculation in advance to say ok 4 units this is the best one it's just you see this is for 50 layers this is for 50 layers sorry this is for 50 layers this is for 20 layers and here already the functions are a little bit non linear nevertheless it is not taking this one here if this function would go this way it would be much more precise maybe I didn't test and also here if this will at least take away or classify these two points and that point as a red one there are other data sets and you can play with these data sets so I give you this file and then you can take the data sets are in the let's say in the zip file and then you can check it with these kind of data or you load your own data you can also have your own set up lot based on that because the program is managing such a way that the dimension of the input data doesn't play any role you can use any dimension only the dimension of the label set and of the input let's say the x values should be the same you should have as much numbers can we use neural networks for regression problems not only for classification can we use a neural network for regression problems where y dependent is numeric not classification problem you mean for prediction so in that case the activation function is a simple linear regression yes it will be linear regression but of course this is possible to use neural networks for prediction and it is done I showed this here only in that case because for the prediction you have no backward propagation and I wanted to show also the backward propagation because a linear regression for linear regression the gradient descent is much easier and the derivatives are much easier and the backward propagation is much easier so all this let's say the things which make the paper or the salt of this problem stay out but coming back to your question this is possible and it is used so this means I am done for today and who is interested I heard on Wednesday at 10 o'clock so my plan for Wednesday at 10 o'clock is is the usage of complex numbers let's say so I cannot say yes or no I don't know but for what you would like to use complex numbers okay I never saw it this is the only answer which I can give you but I think it's quite easy to go to Archive X and to put these two search theorems deep let's say neural networks complex numbers and you will get if they are papers or not so my plan for Wednesday I wanted to tell you on Wednesday I wanted to show you what is if we are adding additional layers but I will do it it's mostly for let's say full connected neural networks and a little bit but just only a little bit show what changes if we are going to convolutional networks but nevertheless I will show you examples so running examples where you can later the the materials are on the server I gave you the input to the server so you can download the input if you have an anaconda environment running which has installed all the necessary things this is you know first of all I found out I was doing these files with Python 3.5 and 3.6 and already not everything is working until now I couldn't find out why so this is always a version question I cannot assure you that it will run on Wednesday let's say so these files should work but the other files where I am using I try to install TensorFlow and I am going to get it always multiple error and now I should go and sit down and find out why I have this multiple error but it's a version problem and also in some cases they are running so you are working already with TensorFlow 2.0 and I am using here TensorFlow it was 1.2, 1.3 these are elder files and I have nobody who is doing for me to update them the maintenance of files is always a big problem but I think you know it is but it's a big problem thank you