We're Hack 5, I'm Darren Kitchin here with Shannon Morse and a very special guest, it is none other than Dark Tangent himself. Hey Jeff, how are you? Hey, I'm excited to be here. I've watched your show for eons and we've never actually met. I can't believe it's been 30 Defqons and that hasn't happened. No, no, but I feel like I know you through your YouTube, but I definitely know Shannon. Yeah, we actually did a panel the other day, it was the Defqon 101 panel. Thank you so much for the kind introduction and it was great to finally meet you in person. Yeah, it feels like I think with COVID, I didn't realize when I got to Defqon, like a weight lifted off my shoulders. Yeah. Then I didn't realize she was there and soon as I got around like my people. Yes. It was like, wow, I didn't realize that that personal contact is just we are such social creatures. I had the same feeling. I'm definitely an extrovert. I know I'm the weirdo being here. I'm the extroverted one. But it was so interesting coming back to Defqon 30 after being away for such a long time. I actually saw some of my friends in person and I started bawling my eyes out and I didn't realize how much. You didn't know that you had that? I didn't realize I missed them so much until I came back and I feel like that's one of the important roles that Defqon plays in our lives. As hackers and as information security experts, we're kind of a more than a community in a sense. Yeah. Many of which came to my wedding, like we're a family. It's a weirdo family. It is a weirdo family. I think maybe that's a difference though between sort of the hacking culture and the infosex culture a little bit, where infosex sometimes can become kind of transactional career advancement. And with Defqon or with hacking conferences, it's a little less about career advancement and more about learning the thing, do the thing. I agree. Yeah. I think that's maybe a different mentality. Getting into the scene early on, that was one of the biggest draws for me and I would love to hear your personal story on this. But the thing that I found as a kid on IRC in the 90s was that finally there was a place where it felt like I belonged. And there's been this sense of community that even, you know, now 20, 30 years into it, you really still have that kind of family feel of like, this is where your brothers and sisters are.  Well, so in the lore of Defqon, there are all the hacking conferences before me. There's about three or four. There's like Summercon and Bohocon and Pumpcon was just starting basically the same year and they're all invite only. And I was young and new and I didn't have the invites. So when I did Defqon, I wanted to make it open, not closed. But that's the fork in the road. If you become open, you can't really turn people away. All people who are interested can attend. There's no gatekeeping there. So you don't know who's going to show up, right? So you've just got to try to set the tone and hope you're... And people in those early days, there's so full of promise of what the internet could be, the utopian world it could build, that everybody wanted to figure it out and they all showed up.  And I think early days finding a home like just resonated because before then we were these weirdo internet people. But then since we planted our flag early, there was a structure as new people came along to kind of find their tribe. And if you look at Defqon, you can be a hardware person. You can be a social engineer. There's a million little communities in a community. Yeah, absolutely. So speaking of gatekeeping and accessibility, something that I noticed very early in the scene was that there was still that mindset of kind of like the bravado, the I'm-leet and I'm not going to share with you. RTFM. And I've got to be honest. As a kid, I really wanted to go to some of the very early Defqons and try to convince my mom. You know, with the other phone freaks I hung out with in pound 2600 and stuff. And obviously couldn't go as like a 13 year old. Right. But now you do see 13 year olds on the floor. Yes. And you walk the conference and you see how accessible it is and how there are so many games and contests and villages that are like, hey, try this lock pick out or try, you know, all these different things out. Has that been part of the fabric and was that part of the foundation of starting Defqon, going to your statement about how they were all previously invite only? Right. I think a couple of things happened. It became more accessible, less secretive when we started doing it publicly. And then later on when the search Alcavista and then Google and then Amazon, all of a sudden you could search on security. And so the amount of time you would spend as a hacker trying to figure out the secret was reduced because you could find it faster and you had more time to actually do the thing. When I was young, we were young. Like half my time was trying to figure out who knew the answer and convince them to tell me the answer. When that shrinks to like 10%, it gives you that much more time to come a little jealous of the current generations where they can do the thing. Absolutely. And they've got frameworks, you know, and mentors. Mentors are so important in this industry and being able to get into, you know, hacker education, hacker research, one of the things I've noticed through the weekend, interviewing with tons of different villages, which by the way, thank you for that opportunity, is how welcoming each of the villages is. Every single one that I talked to mentioned that even if you come in there and you've never done blue teaming, you've never done love picking, whatever it might be, you can go in there and find somebody that you can talk to. Yeah, they're so excited to find other people like them. Yeah. So I think the other difference is we really started about 10 years ago or 9 years ago, we really started to double down on is DefCon hacker. Because in the early days, everything was a hacking conference. But then InfoSec grew, careers, we got jobs, all these fantastic opportunities. We could do this as a job, not a hobby. But naturally then, all the cons started to look like InfoSec cons. Yes. Because who's the sponsor, the product manufacturer, right? And so it just naturally had that gravity. But because we existed before there were sponsors or whatever, we had to figure out how to make a con that worked with none of that. And then later on, as the industry grew up, we didn't need to take the sponsor money because we'd already figured it out. So we fortunately had charted this path where we could stay more hacker where less people could influence us. Right. Well, now here we are 30 years later and it's like, what's the difference between hacker and InfoSec? We think about it every year, but I really think it's sort of joy of discovery, like spontaneous sort of exploration. It's things that are not, like you don't necessarily get an InfoSec. It's not become better to some, it's a response tool. It's more like, oh my God, that's how satellites work? Yeah. Right, the company's not going to pay you to go learn how the satellite works. But you've got this innate curiosity that we're trying to feed. And so a friend of mine years ago said, well, I send my team to Black Hat to learn how to be better at their tool use, but I really send them to DEF CON to learn how to think. Yeah. And I think that summarized like mindset versus... Yeah. Hacking is a way of thinking for sure. What I'm really curious about is how, as our scene has matured into an industry, as hacking has become, you know, evolved into InfoSec, how do we preserve the origins of InfoSec? I know that archive.org does good work. I know that text files. In fact, I even have text files from the BBS scene that I wrote, preserved sadly, poorly. What can we do as a community to ensure that as it continues to evolve, that the essence of the seeds that planted originally to start this stick around? You know, it's a really good question because we're terrible at preserving our own history. And I noticed this on the hacking conference world, so I started a project called infocon.org, and went and found every hacking conference I could find and preserve the videos, and I keep updating it. I know there's been a few books blowing up the phone, trying to, or exploding the phone, I think it is. Yeah, it's a good freak one. Yeah, I'm freaking, but there's not that many good books on the history. And I've heard that there's two or three that people are trying to write before they get too old and forget all the stories. So I really hope some of the old timers write that. I would love to contribute to anything anybody's writing out there if you want to talk to somebody about our early days. But if we don't do that right, it'll be lost. Yeah. It's just so fun watching Hacker Jeopardy when they ask, like, this happened 30 years ago, and people, they either get it or they're completely lost.  Terminator 2, 30 years ago. Going into DEF CON 30, it's, for me, even coming back here after three years, it's a bit overwhelming. Do you feel like DEF CON in general has changed a whole lot, even given the past three years? Oh, definitely. Yeah, and I think it was, you know how in your own personal development, there's different stages, and you look back and you go, oh, that's when my perspective changed. Around DEF CON 3, people weren't coming up and telling me things that happened at the con that I didn't know about.  Describe that feeling to me, because I might know a little, I'm starting to know a little bit about that. It's gotten bigger than me. What does that feel like to you?  I'm not going to, like I put in all this energy and I don't get to learn. And then you're like, but it's kind of like, it's bigger than me now. Yeah. It's larger than me. And people are, I mean, all these experiences, and I don't have to be there to partake with them. When you noticed that, did you feel like you were giving up your baby? It was. Yeah. And it took me a couple years to really, because at first I was like, tell me everything that I missed. And then after a while, you're like, you know, I'm okay. There's too much going on. Yeah. And then when you hear those stories secondhand, you're like, that's awesome. You know, that people really did that in the shower. You know, okay then. Wait, what? I haven't heard that story. And hopefully nothing involving Quickset cement. No, no. We did have some cement in the toilet one year. Yeah. In fact, I really appreciate coming down the escalators this year. You see the big old placard for the Alexis Park. Yeah. Oh, yes. We had to go. So good. We originally were going to do a giant rug and we figured somebody would roll it up and walk off with it. They probably would, yeah. So we did a decal on the floor. I've seen people steal sofas from some of the sweet parties. Yeah, that happens. I really wanted to do this year closing ceremony like a confetti cannon. But then you have to pay to clean it up. Oh, yeah. So I was trying to figure out how to make each piece of confetti collectible so the people would take it with them. Wait, so can we spoil it? Is that actually happening? No, but I wish we could. Maybe next year. So if I ever do a confetti cannon, I'm going to make each piece collectible so the attendees will clean it up. Yeah, it really has gotten so much larger. I haven't come to at least the last 10 years. And what are the kind of thought processes that go in your mind when you're thinking about as the attendance rises and keeping the cohesive nature of it? Because I know we've gone from small little hotels to slightly larger hotels to multiple hotels. And now we have a whole convention center. That's crazy. So I think what's going on is DEF CON is always a reflection of the community. Because we can't force the community, right? We can just sort of steer the super tanker or kind of try to direct the orchestra. But really, they're doing what they want to do. So we try to set the tone. We try to set the example. We did this with our Code of Conduct. Yes. We did this with our transparency. Thank you for that. First conference to do a transparency report. And so the thinking there is we first create our Code of Conduct to create standards. And then we report, are we following them? To try to force us to follow our rules and then hopefully other people like that they imitate us and then hopefully other people are helped. So every once in a while we do things to try to elevate the community or provide some direction. I don't say it's, what do they call it, something signaling, a virtue signaling. No, I don't either. Absolutely not. In fact, I'm glad you mentioned Code of Conduct policies because I feel like a lot of other hacker conventions look to DEF CON to figure out what they should do in the future. And when you created that policy many others created their own COCs. It was okay for them to follow. It was okay, yeah. It was more comfortable for them. It became our standard opportunity to ask for that when we would sponsor conferences. It was like making a new protocol. Yeah, no, when we wrote that we had an eye for other people hopefully copying. But so when you're being kind of told whichever way the community is going people will lament, oh it's not like it used to be. It's changed. It's like, yes, look around you. Look at the diversity. Look at how the workplace has changed. It's not just network engineers anymore. Technology is in every industry.   You wanted people to understand what you're doing and now they all do. And so now they're all here. Yeah. Straight up, I used to work in restaurants. I graduated in hospitality administration. So I could run a convention, but I didn't know a thing about hacking before I came to DEF CON and started working with Hack 5. Unlike the dark tangent who obviously, you know, aspired to create the nation's largest hacker company. Yes, of course. From day one. That was hilarious. Like, do you want it, other than the birthday story, do you want to like give us kind of the little nugget of how this came to be for you?  Not a great hacker. I was an okay hacker. And then... You didn't get caught, like Darren did? No. I never got caught. But I ran this Bolton board and I was the U.S. hub for like 12 different hacking networks on Fido Network. It wasn't on Fido, but Fido Compatible Network. And when one of those networks was going away, PlatinumNet, we were going to throw a boy in a wave party. But all the... Here's a Canadian network, but all the users are mostly in America. So he's like, let's throw a party, but we got to do it in America because that's where everybody is. And I said, okay, great. Well, where could we do it? Maybe boss Vegas. And then all of a sudden, his dad took a new job, he disappeared, and I've never heard from him since. But I had already had like one foot in, I'm throwing up. So I was like, well, if we're doing this, I'm going to invite everybody, not just that network. Yeah. I'm going to invite everybody on IRC, pound hack, pound freak. That's what I was, pound freak. Yeah. Downnet, Fnet? Fnet. Yeah. All right, downnet. On Usenet, I was sending faxes to everybody. I mean, I was like spamming everybody that I could for free. And it just turned into this party, right? And you quickly realize, like I didn't really sign up to become a conference organizer, but you have to learn how to do it. And so I went looking for books on, well, how do you do this? There are no books on this.  And so it's like, kind of a little bit like hacking, like the way you order things, optimizes the cost, the way you do ship things, it's the same kind of puzzle mentality. Yeah. It's very logical. When we first started getting into e-commerce, I look at like all of the regulations and all of the hoops that we have to jump through for EAR and everything else. And I'm like, ooh, a complex system. What can I do? I get to understand this and find the optimal way to approach this problem. Yeah. And that's like hacking isn't just like Unix systems. Right. It's a mindset. And I love hacking logistics. And when you see, when you're paying with a credit card and they're like, oh, you got a sign and I'm like, oh yeah, they're going for the extra 0.1% discount for the extra in-person profit. You know, like you can see the process. It's true, it's true. And when they don't, you're like, wow, they must be making so much money. They're not trying to max out their, you know, piece of whatever, gonna change for you. Yeah. It's really fascinating. That hacker mentality is kind of what we try to teach. And as the community has grown and there's been more people, you can't just get into a room and meet everyone. So it's been a big challenge over the years is, how do you help people find their tribe? Yeah. And when the villages, the original villages were Wi-Fi village and hardware hacking, and now we have 30, yeah. I mean, the conference is, the villages make the conference. You've got the tracks, but because you put the, the talks online, you can actually go and enjoy that not having the fear of missing out on the talks. It's wonderful.  Yeah. And they're making this. How does that feel when it's like delegated like that? Yeah. So what we do is, instead of, you know, villages and contests and events, and now we're just thinking among the creators. Yeah. These are the people creating the content that people want to see. It might be a two-day contest. It might be a one-day village. It might be something, but they're the creative energy that some, and we have about a thousand of them it takes to put on a death march. And about a thousand creators to make this, and about 500 goons to help act as the oil. And so it's about 1,500 people to make this kind of experience.  And so we spend a lot of time trying to figure out how do you take 20 something, 30,000 people. And then you're like, okay, hardware. Hacking Village. Voting machine. And next thing you know, you're sitting down with 10 people with the same interests.  I met some, oh, okay, locks. High security locks. You know, now you're at a table with those people. And we just spend a lot of time giving off ramps. So even though there's a huge mass, how can you find your people? Yeah. And we think about that a lot. It might not be visible to anybody, but that's our secret social engineering strategy. Yeah. So Jeff, as the guy behind the convention, what is one tip that you will give to somebody who's coming from a completely different background and wants to go to DEF CON just to see what it's like? Yeah. The number one thing is you go up to somebody and say, I don't understand what this is. Can you explain it to me? And people are so excited that you care about the thing that they care about. And they just want to share their passion with you.  It's true. Now that we're on DEF CON 30, I feel like it's so exciting. It's so good to be back. And congratulations on 30 years going. 30 years going strong too. Yeah. Where do you want to see DEF CON going from here? I feel like there's just onwards and upwards. Also, by the way, Caesar's Forum should have asked you before building this place about Adigan as second floor so that we have twice as many ballrooms. No. We signed the when we were about to sign signed the agreement movie from Paris Bellies. Yeah. They showed us the plans that originally had two floors. Really? That originally designed up two floors. I freaking knew it. These should have done it. It was. And then, remember, they've been bought so many times that they're in the process of being bought by another group. Yeah. Yeah. And they tried to shave all their costs and they decided oh, one floor. Well now, no, I don't want to go there. Please don't do that. No, no, but there are other. CES is there. It's terrible. No. So the thing about that is there are places that are bigger, but we need air walls. Yes. We need to divide and those giant technical centers have no air walls. Yeah. So you can literally go broke trying to put those spacer walls in so we are stuck in only a certain time. So true. So does that mean that as Defconn grows, it hits a capacity or what do you think? What do you think? Right now, thinking at some point they have to be fragmented. Not, like to say it's not already with all your DC insert area code here or NPA here. Yeah. So we have the Flamingo, we have the length and harrows and so we try to localize things like all the workshops are one location. So I could see if we really had to keep growing, we had to add another property. Yeah. You know, I would hate to have people walk across the street to Cesar's, but if you did go to Cesar's Palace, that's another giant place. I think Vegas can provide. I just don't want people to walk any more than they have to. And I really love this street of food behind here. Yeah, I do too. You can go out here and have an In-N-Out Burger and it feels like home. It's great. What's your step count for the weekend? Yeah. Well, funny you mentioned that I turn off all the wifi and Bluetooth on my phone. Wait, why would you do that, Jeff? Don't turn off the wifi. I think it's something like some Bluetooth ducky and wifi, ducky, rabbit, what about the zoo? We have a petting zoo. Yeah. When can we bring all the animals for the petting zoo, Jeff? So instead, I won't know, I'll turn on my phone and everything. I'll sink it and I'll be like, oh, that's what I do. Well, if it's anything like me, it's probably about 10 miles. Yeah, per day. Yeah, it's pretty crazy. So when do you start planning for DEF CON 30 wonderful? About a month. Yeah. About a month, I believe. And then around the first of the year we start announcing all our calls and then it really like for us in the back end about a month from now for the rest of the community, probably six months from now. So I really got to know where do you go in September to recharge? Well, after this I'm going to Singapore. Ooh. Yeah. So that'll be nice. I hope it's on vacation and not work. No, no. I've been living there. I got trapped there for COVID. Oh. And so now the family's there. Oh, nice. Cool. Yeah, you're going home. Yeah. Which is weird though, which is it's on every day. The sun sets and rises about the same time every day. Like nothing changes. And it's also like a cyberpunk novella at night. It's crazy. Yeah. All the light and everything. And so then I come back here and I'm like, I'm at North Lexa, Seattle, where I'm from and it's like, I can wear a jacket. Oh, I have got clothing options.  It's just one other thing I want to mention about 30 this year I thought I kind of had it under control emotionally. And you know, we had Defcon 29 last year and I was pretty good felt really good to see everybody last year. But I was on the escalator coming down and I saw everybody and I saw how much energy and how much positivity and all of a sudden it felt like this invisible weight was lifted on my shoulders is a weight I didn't know I had. Yeah. And then I got down and I was around people and I was like, oh wow. Like, how did I survive before? You know, there's like the colors have been turned down and I didn't realize it. Now that I've been around everybody, it's like really energizing. I don't know maybe it's just because I've been so far away in another country, but whatever it was, it just awesome. Yeah. Oh. Well, we're just honored that this is, that our community is still here and that it's so strong and it's such a great feeling. I think you're stronger than ever. Yeah. Which is amazing. Every year we did Defcon and we were like, okay, that's it. That's as many people in the world that are willing to get in an airplane and come to Las Vegas. There's more conferences than ever. Yes. There's more hacking things than ever. At some point we max out and even with a thousand B-sides and other events and InfoSec still keeps growing. Yeah. Which just tells you there's this insatiable first demand and I'm sure like. It's not just the demand for the talks and the villages. If you look around it's the people. Yeah. That's what makes it for me. I'm sure you've seen it probably with your YouTube channel like just more and more viewers as people are just. It's crazy. They want to know. Yeah. When I first started on this channel I never would have thought we would have. How long do you know? You're in the million, right? We're almost to a million. We're over 800,000 now. It's weird too because it's the longest running show on YouTube technically. It's crazy. But also still not just shy of a million subscribers. We're niche. That's okay though. We're niche. It's authentic. We're a million. I think DefCon, we got our 100,000 subscriber a little more. I think we're around maybe almost 200,000. Right? I mean you're... All right. Everybody go subscribe to DefCon. Go subscribe. Yes. Also I really got to know what again is that website as it's like so important to preserve our culture. Yeah. Infocon.org. And the community this is so awesome. A guy sees Infocon.org and says, that's amazing. And he creates InfoconDB.org. He takes all my data and he created an inverse database of all the speakers and their bios. So you can do a reverse lookup. That's so smart. You can do a reverse lookup and you can say Dan Kaminsky, Black Hat 27 and it links to the video on Infocon. And now, I mean it's just An IMDB for hackers. Awesome. And it's like So cool. He did that for years and we'd never met each other and we ran into each other and he's like, oh I'm the InfoconDB guy. Awesome. Why are you doing that? It's like, I wanted a project with all the scripting language and he figured he could learn how to do it by doing this new project. That's a great way to do it. Because we can is the best way to hack. Yeah. Right. So there's more content than ever. There's more community than ever. All you got to do is have the courage to come out. Yeah. And where can people find out more if they want to join if they want to go to Defconn in the future? So Defconn.org obviously and we have a pretty good community on Reddit. A lot of supportive people. We got our Discord for the people who can't make it here. We really try to support the community year-round with the Discord. I'm not a big fan of the Discord privacy policy. Yeah. Same. So maybe in the future we'll think about maybe doing a Masterdown or something. Yeah. But we didn't have time for 30 this year. But I'm always curious in the community what they would want to see. Yeah. Well, I miss IRC and Usenet. What was it? Undernet? Or one of them? Yeah. Yeah. All right. Got down that dropped. Undernet's gone. Yeah. I'm blanking on the one you're talking about. Yeah. But they had Someone's yelling at the video right now. They had a weird takeover split. It just seemed like things aren't we're not happy in IRC land for a while. Which is weird because I heard once there was drama on IRC. No. There was a conference that over 30 years has also never had that. No drama. No hotel damage. Nothing. No hiding. Yeah. Nothing at all. Yeah. This year. That never happens here. I'm sure your insurance premiums are wonderful too. There's somebody who every year counterfeits the badges. Yep.   I'm like how did you figure that out. So there's like a whole another aspect of physical security and tamper evidence that you didn't even know existed until the guy's telling you about melting the record. And it's just so cool that you can spontaneously learn that stuff. Yeah. I can't imagine you know the next 30 years I can't even like what what can you even imagine for DEF CON 60. I mean I know there's going to be some weird network satellite orbital data center cloud hacking. I mean there's just you know under seas probe something or other. We're going to be like hacking our holograms. You know soon as we're on this AR VR it's all going to be about how to make prompts pop up for stuff that's not supposed to pop up. 100% you know it's going to be in AR and have an experience right next to somebody that's not having that experience. Yeah. It's going to be I'm really looking for the AR world. I don't really care for VR. Yeah. Well we'll be here next year with our Google Glass on hopefully making that happen. I'll bring my glass yeah. No I won't I promise. Yeah. Right. Thank you so much. Yes thank you so much for joining us today. I really appreciate for making this convention. Thanks for having me an excuse to meet Jen. Aw. Thanks for making such a place where all hackers belong. We hope to see you guys next year if not online. Yes. We love you. Take care. Trust your technolest. I'll talk my last.


Hi everyone, Joe for Jaspy's CaseBreaks.com here with 2021 Mosaic Baseball Choice Edition. One box, Ram team break number three. This actually comes from a fresh case, by the way. So we've got three different dice rolls happening here. First, we're going to give away six choice spots. So we only sold 24 spots out, right? Going to randomize those 24 names, top six, get extra spots. Second dice roll will be the break itself. Third dice roll will be this chrome filler. I guess the top spot is going to get two boxes shipped sealed of Platinum Chrome Light Edition. Everyone else will get these teams. So we'll see who wins those spots or the boxes in the third and final dice roll. But first, let's give away those extra spots. Let's gather everybody's names, new dice, new lists, and let's roll it and randomize it two out of five is seven times. Top six after seven, one, two, three, four, five, six, and good luck seventh and final time. Adam, Adam, Lonnie, Adam, Lonnie, Vince. Congrats, after seven times, a little buy one, get one scenario for you. Extra spots going your way, that's nice. So hopefully those extra spots will come in handy in the giveaway part at the end. All right, now let's gather everybody's names again. Now this is for the break itself. The break itself is pretty nice. Mosaic Baseball Choice Edition. Let's roll it, randomize it three out of two, five times for names and teams. One, two, three, four, and fifth and final time. Obi-Wan, what's going on? How are you? Obi-Wan, fifty-five in the house. Three out of two, five times. Things are well. One, thank you for asking. Obi-Wan Kenobi. Now that is a name I have not heard in a long time, long time. All right, there are no trade windows and filler breaks here. Unless you trade really quick, it is a choice box, so. Here's the second half of the list right here. You know, Drew, there is monster potential in a box of Mosaic Choice. So that's not a, that's not a crazy ask from a fresh case to. All right, there are your two autos right here. We got Chris Paddock to sixty-four, Chris Sale to sixty-four. Chris Paddock for the Padres, that's for Adam. Chris Sale going to the Red Sox, Lonnie. We got Jonah Heim and Ryan Castellani. Jonah Heim for the Rangers, that'll be for Lonnie. Ryan is for the Rockies, that's for Adam. Dodger's Zing the Rockies, eight-four in the top of the ninth. Freddy Freeman's probably what is his last at bat of the evening. There's Noah Cindergar, this is Mets edition, that'll be for Lonnie. And behind, Whit Merrifield, your autographs. We've got Jorge Mateo, rookie autographs for the Friars. That's gonna be for Adam and the Padres. Freddy Freeman working the walk. And behind, Cool Whit is, this is a nice one, Keebert Ruiz. The big Dodger's prospect. He's a national now, but this one's still Dodger's edition. That's with Lonnie, that's with the extra spot that you want, Lonnie. All right, nice, not bad. And Whit Merrifield, of course, will go to the Royals. Aaron with that one. Okay, now, let's see who's gonna win those boxes or those teams. Let's gather everyone's aims again, one through thirty, for our third and final dice roll. And then we'll match you up. Name on top gets the two boxes. Next handful of names will get teams. Second name on top will get the Dodgers. I think those teams are organized, sorted by most expensive, down to least expensive. So it's gonna be top sixteen. A little over half the break, we'll get in. A little over half, after three and a two, five times, good luck. One, two, three, four, and fifth and final time. After five, from seventeen on down, sad times. But I appreciate everyone giving this a shot. Right here, thank you. Top sixteen, you're happy. Nester's happy. Lonnie is happy. Nester, Lonnie, Aaron, all happy. And the rest, Adam, Adam, that extra spot coming in handy. Lonnie, Lonnie, Adam, Adam, Adam, Aaron. Lonnie and Han. So congrats to the top sixteen right here. Han, two boxes going their way. So we're about to break open the Hobby Edition, but they also released a LITE Light Edition. So you'll be getting two of those boxes sealed. And you can sell it if you want to. You can rip it open if you want to. Hopefully there'll be something cool there. We'll put a little rooftop next to your name. Well, not next to Han's name, because he's just getting those boxes. So with Lonnie, we'll put that there so you know that's the spot that you want from the filler. Aaron with the Marlins, Adam with the Mets and the Padres and the Cardinals and the Blue Jays. Lonnie with the Nats and the Cubs. Adam with the Giants and the Guardians. Aaron with the Tigers. Lonnie with the Rays. Nester with the Rangers. Lonnie with the D-Backs. And Nester with the Rockies. So there you go, gang. I'm Joe for JaspysCaseBrakes.com. I'll see you in the next video for the break itself. Bye-bye.


Hi everyone and welcome to JSA TV live here today on the floor of ITW at National Harbor and joining me is Ben Edmond, CEO and founder of ConnectBase. Thanks for joining us. Thank you for having me. It's a pleasure to be here.  I mean, it's we're only on day one, but I already just before this interview some news hit my inbox and I'm excited to very timely here, I think, to share that with our viewers and to talk about, well, why don't you tell us? Tell us what the news might have been planned from a tiny perspective. Maybe. I love it. I'm excited to share that we acquired Last Mile Exchange and it is a fantastic combination between who ConnectBase is and what we just acquired to create the first real global platform out there and excited to share more. Yeah, and so it's been a big year of like busy last few months for sure, but sort of what does this mean for your current customer base and the LMX network? What does all this mean, bringing that together? Sure. It means a couple of things really succinctly. One is reach. When you look at our customer base, which is predominantly North America and their customer base, which is predominantly Europe, bringing those two things together brings global reach for both sides and the ability to have an American operator reach out into all the great things that LMX brings to the table and vice versa as a Last Mile Exchange customer that was automating a tremendous amount already across Europe, now being able to reach into North American networks, which is very complex and very fragmented. It really brings a lot to the table on both sides from a reach perspective. I think secondly is just the simplicity of doing a very hard job. Our customers have a hard job of participating in the market, building networks, winning customers. It takes a lot of resources and now with one platform they can plug in and reach almost 400. Yeah, amazing. I mean, if this isn't big enough news on its own, it's not the only thing you've been doing, right?  What are some other highlights for over the last year? Well, there's a number of things. ConnectBase has been a growth engine for ourselves and for the market. We raised capital in the fourth quarter, $22 million, oversubscribed round, bringing in new investors, leading the round. Digital bridge ventures led the round and all of our existing investors also participated. So the new capital, making an acquisition, and then, oh yeah, by the way, we've built a lot more capabilities and brought on a whole bunch of new customers as well. So it's just this constant investment in what our ecosystem is all about. So, I mean, it's a lot and you're doing a lot. You're obviously busy and succeeding in a lot of areas. And I mean, as we sit here on day one of this show and I know that you, no doubt, have a pretty full agenda for the next few days. What are the top things that you want to be, the conversations you want to be having with your network here? I think it's really centers around three things. One, from existing customers, how to speak to enable them to do business when we're not in the room. The reality is more of the world's going digital. They need us to fulfill how each side needs to monetize their networks and buy networks. So making sure our existing customers, wherever they are in the world, understand how to have a conversation with a new supplier, an existing supplier that's using ConnectBase, how to ask the right questions and onboard them. Two, at 400 or so networks that we're serving today, there's still a lot that we haven't addressed yet. We want to onboard those new networks into the ConnectBase ecosystem and make sure we're telling our story why we are a great return on investment, why we help monetize their network, why we enable them to securely do the things that they need to do. And then the third is really just an advocacy for the ecosystem to digitize and elevate how we do business. We don't get looked at as an industry as the best customer experience and ConnectBase is on a mission to change that. So what's next then? Once you leave here, you're focused on the year ahead. What are the priorities? Yeah, ultimately we have to prioritize, but the short answer is more. We are in our early innings of a massive growth cycle. We want to be a platform that is known throughout the industry across every continent because network is ubiquitous. It is how the world plays. It is how it learns and how it works. It's how people are doing what they do to entertain themselves. It doesn't work without connectivity these days. It is our job to help reduce the friction and how connectivity works and elevate the industry. So true. Again, being here is such a great start for this with all of the right people, a lot of the right people anyway here to have these conversations. So any further notes on this as we wrap up and as you think about the week and then the year ahead? I think if you look for some of the things that we're rolling out to the market, we're introducing things like near-net automation on the right-of-way, where customers can do these things themselves, things that we had to do for people over the last seven years. We're putting those tools in their hands, elevating our global footprint to now over 2.4 billion locations, all exciting initiatives that will help people sell more or buy better. Right. Amazing. Ben, thank you so much for joining us. It's always a pleasure to catch up. It's my pleasure. I appreciate the chance to be here. If anyone's interested in learning more, don't hesitate to go to our website at ConnectBase.com or follow ConnectBase on LinkedIn. Yeah, ConnectBase.com or on LinkedIn. Absolutely. Fantastic. And thank you again, and thank you viewers for tuning in to JSA TV, coming to you live today from the floor of ITW at National Harbor. Until next time, see you soon.


In condensed matter physics and material science, an amorphous from the Greek 8th, without morph, shape, form or non-crystalline solid is a solid that lacks the long-range order that is characteristic of a crystal. In some bolder books, the term has been used synonymously with glass. Nowadays, glassy-solid or amorphous-solid is considered to be the overarching concept, and glass-bomor-special case. Glass is an amorphous-solid that exhibits a glass transition.One polymers are often amorphous. Other types of amorphous solids include gels, thin films, and nanostructured materials such as glass. Amorphous metals have low toughness, but high-strength amorphous materials have an internal structure made of interconnected structural blocks. These blocks can be similar to the basic structural units found in the corresponding crystalline phase of the same compound. Whether a material is liquid or solid depends primarily on the connectivity between its elementary building blocks so that solids are characterized by a high degree of connectivity whereas structural blocks in fluids have lower connectivity.3. In pharmaceutical industry, the amorphous drugs were shown to have higher bioavailability than their crystalline counterparts due to the high solubility of amorphous phase. Moreover, certain compounds can undergo precipitation in their amorphous form in vivo, and they can decrease each other's bioavailability if administered together. For 5


Hello, everyone. Jim White here from IoTeX Systems. Welcome to my talk on lessons learned in creating an Edge Management with open source software. So let's begin here today. Again, the agenda for our topic, if you haven't had a chance to review it online, is we're going to talk a little about what Edge Management is. We'll talk a little bit about the requirements. I want to provide you an argument, if you will, a story for why we think cloud and enterprise solutions are needed in certain spaces, but not necessarily in this Edge Management solution. And we'll talk about some of the differences there and why I think Edge Management has some unique requirements and why we've looked to open source technology to find a solution. And one of the reasons why we did that is certain benefits that we think this solution has to various edge environments. I'll give you a demo of some of the work that we've done today. And then I want to speak to some of the challenges and the lessons learned, allowing you potentially to follow down some of the same path in finding out how to get an Edge Management solution around what might be your Edge solutions, your Edge applications out there. So really quickly, let me just provide an introduction of myself. And of course, I'm here because of my company and so I want to also introduce them. So again, my name is Jim White. You can find my email address there on the screen. Please feel free to email me if you have any questions about today's topic or in general about some of the work that I'm involved in with regard to IoT. Specifically, I came from Dell Technologies where I was the director and team lead of a project that has since become EdgeX Foundry, part of the Linux Foundation. I was the chief architect and lead developer of a project that became EdgeX Foundry. It was called Project Fuse. So yep, apologies in advance. If you do go out and take a look at EdgeX Foundry, you'll see that a lot of the original code is mine. Thankfully, there's a lot of better people out there working on the system today and you'll find it in great form with over six million downloads, container downloads to date. Today, I still serve in the chair position of the technical steering committee and kind of the unofficial ad hoc lead architect. My company, again, where I am the CTO today is IoTeX Systems. We're a software company. We build Edge Middleware is the way we like to put it. We're based out of the UK, although I'm in the United States. We build vendor-neutral software platforms to which we've taken EdgeX and commercialized it. The logo for our product is down there at the bottom of the screen called EdgeExpert. We also produce a kind of a real-time, critical-time version of that called EdgeXRT. So we're in industries like industrial IoT, verticals such as utilities, energy, oil and gas, mining, they're typical cases where you see IoT solutions out there today. And we work with a lot of the big partners. As a small company, as a startup, we usually don't go out and work directly with companies, but we're working with our partners such as Dell and HP Accenture and such. And again, we leverage and believe in open source technology. So you're gonna see the solution I have today to talk to you about is all about open source technology, but again, things like our EdgeXpert and XRT products are also based in open source technology as well. So that's a little bit about me and a little bit about my company. Let's talk now about Edge Management. So you hear the term Edge and Management and maybe the first thing you thought was, well, what the heck is that? That would not be an uncommon reaction. In fact, knowing that, you might have seen in my talk, little blurb or abstract that I tried to do a little bit of definition about it, but the first thing I wanna do today is set kind of the schedule straight in terms of what exactly is Edge Management. And we'll talk a little bit about the solution that we at IoTeX have undertaken to try and provide for that need. So the problem you have, if you've built an application that works the Edge like we have at IoTeX or that you find again an open source like EdgeX Foundry, the problem you have with that is how do you get that application out to the Edge? So you've created this great app like we've run into with our clients and they say, that's great, we love it. Now, how do you get it out to the thousands or tens of thousands or millions of boxes? How do you plan to deploy that application? Even if you're able to deploy it, how do you plan to monitor and manage that application at the Edge? How do you even know how many nodes you have or where those nodes are or how to find out if those nodes are up and running? And by node, I mean the host computer that you're running on. All of these essentially talk to one of the larger problems with regard to Edge and Edge solutions is that everything has to be done at scale. In the enterprise, we talk about tens or hundreds of types of systems that we're running on. In the Edge, we're talking about thousands, maybe millions. We have one pilot project where the pilot itself is 80,000 nodes. So how do you deal with that number of compute hosts and how do you deal with getting your application and monitoring your application once it's out there? That is all about what we call Edge management. So there are two parts essentially that we see to Edge management. There is what we have as node management, prepping the box, if you will. Typically, this means you're running, at least in many Edge-type environments, you're running on a somewhat resource constrained box, typically what's referred to as a gateway. Dell and HP pictured here, but there are plenty of providers of gateways or Edge solutions or Edge hardware, if you will, out there. So the first problem you have when you're trying to get these systems running is figuring out what boxes you have and how do you get those boxes prepped for application? How do you get things like OS and infrastructure on those boxes? And then the other problem you have, what I call bits to boxes, once you've got the box prepped out there at the Edge, how do you get your applications out to the Edge? For us, that means Edge Expert, XRT and even the open source Edge X Foundry. But what if you're running something like Greengrass? We talked to some of the Amazon folks and one of the first questions they had for us when we talked to them about things like our applications was, hey, how do you guys get your applications out to the box? So again, we're talking about node management, getting the box ready, application management, getting the bits to that box once the box is ready. That is what forms Edge Management or at least how we form the term Edge Management. So Edge Management includes node management, which again is provisioning those boxes or those platforms out there, getting the OS installed, dealing with upgrades, connecting the nodes to appropriate networks. Maybe you've got to get things like Wi-Fi configured. How do you install and update drivers and firmware? And then how do you update the infrastructure that's going to run things like your application? For example, Docker, or if you're using Kubernetes or something of that nature, how do you get that set up? How do you get your virtualization installed if you're working with virtualization at the platform? And then once you have it all installed, what's the health and status of that platform? How do you track that? How do you monitor things like CPU and memory and disk space? Finally, what about security? If somebody is trying to access that node, that box inappropriately, how do you know that? And how do you establish the preventative measures to prevent that going forward? So there's a lot just in node management, but as I mentioned, that's only half the story. The other half is application management. So once you've got the box ready, how do you deploy and orchestrate your application? And the configuration that goes along with it. How do you deal with it in either a container or non-containerized form? Today a lot of solutions only deal with containerized types of applications. What if you're dealing with a resource constrained device? How do you deal with non-containerized? And then you must be able to update the apps and the configurations. You may have to start, stop, and restart those applications on occasion, especially if there are certain changes that necessitate that. And like the box itself, you have to learn how to monitor the application, what devices are connected and being able to send out alerts when something's gone wrong. You also want to monitor the data that's coming through those. You're doing this so you can get all that sensor data from that operational technology area of your systems and you want to be able to know when that data's coming through and what to do with that data. And you have to deal with things like provisioning the various secrets or provisioning the actual devices or sensors that are connected to that box. And that typically means working with the applications to help onboard things like that. So there's a lot to worry about when we think about edge management, both node and application management concerns. So my company is based in the UK and so a little bit of a slapstick humor here. When I talk about a lot of my colleagues there are a lot of people we work with who come from that part of the world. I typically get this answer, it's no worries mate. You know, we'll just use enterprise and cloud technology to help manage our nodes and manage those applications. Doesn't that enterprise or cloud technology doesn't that work just as well? At the edge, can't we use Kubernetes or OpenStack, OpenShift, Mesos, fill in your favorite cloud or enterprise technology of choice here. Well, to some extent, you may find that you can use elements of the enterprise or cloud tools that are out there. A joke that we have amongst us at IoTeC when we're talking to customers is typically we get this answer that, well, Kubernetes will solve everything. You know, what's the problem? It doesn't matter, Kubernetes will solve it. And there's a web post out there I've provided a link in my slides to that. Let's use Kubernetes, well, that gives you another eight problems to have to solve. Kubernetes is big, it's powerful, but it's big. Sometimes cumbersome, sometimes complex and we're trying to solve things in an edge environment where resources and some of the availability to tackle things with an enterprise or cloud system like Kubernetes gets to be pretty darn difficult, if not impossible. So when trying to think about how to solve your edge management needs, I'm not saying don't use enterprise or cloud technology if it works, in particular, if your edge is actually pretty thick and resource rich, it might work. But use of enterprise or cloud technology is also problematic for several reasons. You need to know all the nodes and applications that you're deploying out there and these kinds of solutions. When you're dealing with hundreds, as you typically do in a cloud environment or maybe even 10s, that's not too bad. Imagine knowing all of the nodes in a Kubernetes environment where you're talking about millions. Just trying to keep track of all the IP addresses and all machines are pretty darn difficult. You can't afford to do that in a manual sense. You operate in a heterogeneous environment, it's not just one OS, it's several flavors of OSs, real-time operating systems and regular operating systems and how do you deploy and manage across that environment? Again, the applications in this environment are not just containerized, they're non-containerized. Some of these resource constrained environments don't allow for dealing with containers at the edge, it's just too small to deal with. You also have solutions that need to run on-prem versus everything being on-cloud and not always do some of these enterprise or cloud systems deal well with that. And you have unique edge needs, which I'll address here in the next slide upcoming. The marketplace isn't providing the solution in these cases. We found in many cases, people are trying to use an enterprise down approach, they're trying to take approach where they're using a Kubernetes or an open stack or some other solution and drive that down the edge. We've seen some failures in this realm. VMware had a product called Pulse, they just recently ended life. I would submit to you that that's one of the reasons it struggled. Cloud providers don't always have the answer either. Again, we mentioned Amazon with Greengrass, they're asking some of the same questions. How do I get my application out there? And some solutions, even though they were on a path that I think may have had some results and may still have some results, they went proprietary or I should say largely proprietary. So that creates a bit of a problem in terms of being able to take your solution into a marketplace where at least some of the pieces don't have to be always manufactured by a single vendor. You want to kind of bring your solution together with the best of breed approaches and that often means using open source technology. By the way, I'm getting a couple of online chat questions. For example, what about something like K3S or a stripped down version of Kubernetes? And absolutely, we will find that in some cases those solutions too will address or better address some of the needs at the edge. But again, there are questions about things like non-containerized environments or environments where even something as big or small as a K3S or a Kube edge could have some problems. We're talking about things like PLCs and control units out there. Sometimes those are not going to always address our needs out there. So again, there are some unique needs that the edge has. When I see an edge system, I'm looking at some of these kinds of constraints, resource constraints in terms of memory CPU. Again, when you think about big Kubernetes, K8s, we're talking about a need for about 700 megabytes of memory, minimally to get something up and running. Yeah, again, Kubernetes at the lighter end, K3S or Kube edge do have some smaller bases from which to work, but then you still have to worry about where does the other pieces fit and how do I get it connected to the server? Again, scale, we're talking about thousands or millions of nodes, even trying to address the address space for all those nodes gets to be complex in some of these solutions. The network connectivity is going to be intermittent. Again, some of these solutions don't work too well without a pretty solid connectivity. And that's norm when you think about edge solutions. Things like IoT systems on the back of a shipping container or on a rail line, those are by nature going to be in and out of connectivity zones. Thing connectivity creates all sorts of havoc with regard to security and dealing with things like containers, being able to poke holes in firewalls and getting access to various protocols. And the connectivity needed for those protocols can be a real frustration and one, not typically something that enterprise or cloud developers have to worry about. And security, local constraints in terms of being able to make sure data only goes so far in your fog environment and access restrictions. Again, not always the type of thing that cloud or enterprise people have to deal with. We have a case too where in many cases the systems out there are legacy, they're brownfield, they have an old OS on them. They don't always support the same types of updates that the modern OS that you've had to deal with in the enterprise and cloud environment deals with today. The lifecycle of an OT system out there in the field is measured in decades, not in months or years like it is in typical IT environments. You don't have the ability just to update overnight. And lastly, there is heterogeneity, not only in terms of OS, but the platforms you're going to reach. Again, sometimes because of that legacy equipment, some equipment that has managed to stick around for tens of years is combined with equipment that just came out last year. And so you have to be able to deal in those kinds of environments. So those and many more are some of the unique needs you'll find that you have to be able to address with these edge management solutions. So what we encourage is people to start to look at, well, what can we do from an open source perspective? Yes, we can use some cloud and enterprise tools. As you'll see in the demo I'll provide you and talk through, there are pieces that we are using from the cloud and the enterprise. And we bring those together with some edge agentry to offer a solution. Benefits is when you're starting to use open source are typical benefits, faster to market. There is an ecosystem that supports a lot of these open source capabilities. And so we can leverage a lot of that technology and the knowledge about them and start to deal uniquely with the unique problems of the edge. Trying to find a proprietary solution. Yep, you might be able to find a solution, but then as you start to get in all the nooks and crannies of edge management, you may find you're gonna have to pay that vendor a handsome amount of money to try to take you into all those corner cases. So it is cost effective. It is also something that's typically operator comfort. When we look at some of the UIs, for example, that we're using in the solution we're proposing, Portainer being one, then you'll find that a lot of the IT operators that are out there are already pretty familiar and use that technology. Can we reuse those same user interfaces to give them that same comfort and feel when they're operating over our edge environments? It allows us to leverage best to breed solutions. Hey, the solution that I'm gonna show you in demo today is one that we know still has a lot of work to do. And there are gonna be pieces which we'll have to replace over time. But by bringing together the various elements of open source and bringing those together in an integrated and well thought out manner, we hopefully provide the right abstractions as well to allow for those pieces to come and go as necessary as new elements start to take advantage of new technology out there. We can replace the elements that are lagging behind. Again, difficult to do in some sort of proprietary setting. So we're allowed to swap in and out various elements when and where necessary because of the use case, because of constraints, because some of the choices we made early on were expedient but long-term may not fit the needs. So what is our solution looking like? Well, right now we're in the midst of getting ready for kind of general release of what we're doing. We're expecting to have this released here in the next few weeks. In fact, I was hoping as I put this talk together back in the winter timeframe that we'd actually be out by this time this conference came about. But we're still just a few weeks ago and we're not gonna be able to meet all of our needs right with the get go. If you're familiar with the EdgeX Foundry project and you're familiar with our community, we have a saying in the project, crawl, walk, run. Crawl things first and then work your way up to running. And this certainly will be a crawl episode. We're gonna be able to support basic node definition and management, being able to add and remove nodes and register those nodes, being able to get the status of those nodes and then be able to do simple basic application management to add, remove and deploy. And to start with add, remove and deploy containerized applications as we'll see. Moving to non-containerized over time. And we wanna be able to get some metrics off of the box and off of the applications, being able to know for example, how much CPU or memory a application is using. And graphically display that, give at least users some comfort and feel that display they're seeing gives them an indication of whether things are trending up or down. The components we're using in our solution to start off with AWX, Protainer, Prometheus, Grafana, tools again that you see pretty much in enterprise environments, yes, but also being used and made available at the edge. In particular, trying to provide the agentry that fits down in some of these tighter spots. IoTeX, my company is pulling it all together with a command light interface and providing the glue between these open source elements. You'll notice too that a lot of the user interfaces as I have a slide to it to address here coming up are gonna be again those user interfaces that operators are very familiar with. Over time, we hope to change that. So we have kind of an amalgamated user interface that provides users kind of a one-stop way to see everything. So in version one, the first step in our new tool for edge management will be to have the operators define the nodes they wanna manage. This is tricky. We'll talk about some of the lessons learned. When you think about defining the nodes to manage in a large system and say a cloud environment, you might be talking about again, 10s, maybe hundreds if it's super large and defining that set of nodes in that kind of environment is not overly complex. It can be a bit complex, but with a file you can usually define the nodes pretty easily. When you're talking millions, that gets to be a little bit more of a challenge. You can't just sit down with Excel spreadsheet and start to define the nodes. We'll see that we've got some solutions to that but more solutions are on the way. Once we've defined the nodes and we use our command light tool and AWX with SSH to do that, then the next step is to deploy or get the infrastructure down to those nodes. Now, we're not deploying the applications in step two. We're simply deploying the infrastructure necessary to run our applications. To start with, again, we're dealing with containerized applications and so we've chosen to work with Swarm and using Docker, but as we move forward, obviously Swarm is a pretty big beast and not probably gonna meet all of our needs at the edge as we go forward. We know that, but that's our first attempt and we're looking at orchestration deployment options for not only container but non-containerized solution. So it's getting that infrastructure down in place to help allow applications to run. You'll notice too that we are constrained at least initially. Our first version will be on Linux. As we go forward, we wanna be able to support this on multiple OSes and there's a question in the chat about what version of Linux are we standardizing on. We're starting with things like Ubuntu, but again, we have applications that are real-time critical and so we're gonna have to operate on our tosses and some pretty small kernels. So as we branch out, again, we wanna be able to deploy infrastructure that works on whatever you've got out there and work in a multi OS very heterogeneous environment. But again, we're gonna crawl in our first stages here. Once we've got the infrastructure in place, then we work with, at least today, we're working with Pertainer to deploy the application.  They call their edge agent and working with the Pertainer UI and their edge agent, we can then trigger the infrastructure to go and deploy our applications. Again, using containerized applications to start with that will have to be grown out to deal with non-containerized as we go forward. And then finally, once we've got the application deployed, the next step in this whole process is to allow the monitoring to occur. We're using Prometheus with a Gravana dashboard to be able to suck up that data and understand what's happening at the edge with our nodes so that we can trigger things like alerts when we see something out of range and go take action. This will be something else that we'll talk about in the lessons learned when I complete the demo here for you. This is probably gonna suffice for initial needs, but there are a lot of other areas in terms of monitoring that we'll talk about. And this is one of the things that we hope to gain some attraction on and have people help us think about how we start to instrument various applications at the edge to be able to provide more monitoring capability, more on that coming up. So when we look at the user interfaces, as you could see from the little brief intro to the solution that we're presenting at this point, a lot of the interfaces we're using come from the tools that are already out there. Pertainer, AWX's UI, Harbor Gravana UI, all user interfaces that operators are going to be pretty familiar with, at least if they're coming from an enterprise or cloud environment. We provide a command line interface to bring it all together. And we do think the command line interface is gonna be necessary for scale. When you're dealing again with millions, you're gonna need to want to write scripts to handle things when we're starting to talk about doing any kind of add move update. We will also be rebranding a lot of user interfaces to help obviously give our product a little bit of a distinction, but going forward, we wanna also start to bring these user interfaces together, so that the operator really does have a single pane of glass, but still maintains a lot of the connectivity tissue back to the original user interfaces that they're familiar with. Okay, so with that, I've spoken enough for right now. Let me have Jim, our technical expert in the background, queue up our demo. This demo is gonna be presented with the help of one of my cohorts in crime at IoTeX, Tom Fleming is our product manager in this area. So I'm gonna have Tom take it away in this prerecorded demo. Okay, so this demonstration is going to aim to follow the steps that you've already seen in the slides and to look at how that's done in practice. I'll start by describing the setup a little bit. Firstly, we need a number of nodes in order to manage them. So we've got three virtual nodes. These are created using Docker machine. You can see them down here in the bottom left terminal. They have IPs ending in 167, 168, and 169. We'll refer back to that a little bit so it's worth remembering those. But these are gonna be our nodes for the context of this demo. In terms of server components, well, the machine that we're on is the server. As we can see here, we're running a number of containers. These containers are our open source components of our setup. We're running AWX, Grafana, Prometheus, Portana, a number of other ancillary services, as well as an empty TT broker that's used as part of the Edge Expert aspect of this demo. We'll refer back to that a little later and you'll see what happens when we use that. So, step one is all about defining the nodes we want to manage. So we can do that using our CLI. Our CLI has a number of commands. One of them allows us to manage nodes. And when there, we can add, remove, and view the nodes that we've got. This case, obviously, we want to add. Now we've got a number of options here. We can either specify each node individually by providing its name, its IP address, and the description. Or we can specify file that contains one or many registrations for each node. We've already prepared a file. I'll show you what that looks like. So this is the file. It's the same information, name, address, description. It's just one or more specified in the file best config. So this has been step one. This has been defining the nodes we want to manage as well as providing an overview of our demo setup looks like. Okay, so now we can move on to step two. Step two is about taking the setup that we've got, their definition of those nodes, and deploying that infrastructure. So we've got VH1, VH2, and VH3, and we can use the CLI that the command we have before. If we pass a file, the same file that you've just seen, we're passing that configuration and we're gonna hit go.  We need to wait a little bit for this to finish working, and then we should see that these nodes are set up. We can view the status of the nodes whilst this is in progress by again using a CLI command. So we've got a view option, and if we do view minus a, we're gonna see all nodes. Now these nodes say unassociated. Associated or unassociated is really portendous beat for either it's not ready or it's ready. So at the moment they're not ready. When they go associated, they'll be ready, ready to be monitored, and ready to have applications deployed on. Now, we would like this status system to be worn nuanced, and we're actively working towards having a more descriptive set of statuses that tell you whereabouts your nodes are in terms of provisioning process. So we need to view again. Ah, and see now we've viewed the nodes, we can see that they're all associated. This means that the nodes are ready to go. If we head to Portena and this browser on the right, and we refresh this list of end points, we can now see that we've got V edge one and V edge two, V edge three. Both registered and associated. We can do a similar thing in AWS. Firstly, we could go to jobs and see that the job that set up those nodes has completed. It's green, which means it's successful. And if we were to look into inventories, our IoTech inventory and hosts, we can see the three edge nodes that we've added to the system. Okay, at this point in the demo, we're removing onto step three. This is all about deploying applications to the system. This is where we're going to deploy an instance of IoTech's Edge Expert to those three nodes. Now, in order to do this, we define firstly what's called a template. Template basically points to a, it describes a configuration that we want to deploy. Okay, at this point, we're on step three of the demo. Step three is about deploying applications to those edge nodes that we've already registered. These nodes are registered and associated, we're ready to go. V edge one, V edge two, V edge three. What we're going to do is deploy an instance of Edge Expert to those nodes. Now, Edge Expert has already been, the configuration we're going to use for Edge Expert has already been predefined and hosted on a public Git repo as a Docker swarm composed file. So what we need to do is provide a description of that to the system to add it as a template. Naturally, we have a number of CLI commands to do that. We can see that template is an option and we're able to add, remove and view templates. If we view the existing templates, we can see that a large number of default templates is this within Portainer. For example, Redis or Jenkins, you can use these templates to deploy those instances on edge nodes. What we want to do is to be able to see Edge Expert appear as one of those templates. Now, what we need to do is to find a configuration which we've already done and you can see it here. This is the configuration for the Edge Expert template and it's fairly straightforward. Type is a unique notion in this case. Type refers to either a single container, a swarm stack or a composed file. These are the types supported by Portainer for the time being we just support swarm. Swarm is type two. Everything else is either a title or some general metadata, the URL of a logo, as well as the URL of the repository that the composed file is hosted in and the composed file itself. So this is a swarm composed file for a specific deployment of Edge Expert that's going to run in this demo. So what we need to do is use the CLI commands to add that to a template to our system. So again, we've got templates and we know we can add and we can see we need to provide the path to this file. So if we just do this now, provide minus F, we provide it with correct path. That's the final name. We run this command, we see that the template has been added successfully and now if we run the view command again, we can see that in addition to the default templates, there is now an Edge Expert template that's ready to be deployed on our Edge nodes. Now that we've got the template set up, we can look to using the CLI to deploy that template on each of the nodes that we already have. Again, we can look at the commands available and we can see that managing applications is one of those options. We can both add, remove and view the applications that we have available. In this case, we're going to add and we need to provide it with a number of pieces of information. One of them is the template that we want to add. In this case, it's Edge Expert. We need to provide it with type. This is a little bit of a hold over seeing as we just support swarm at the moment, but it's type two. And we can just pass minus A to deploy against all of the nodes available for now. So if we run this command, now we've got to wait a little bit of time for those nodes to deploy. And then once they're deployed, we start seeing data coming through the Mosquito subscriber over here. The Edge one is deployed successfully. Fairly shortly, we should see some data appear in the terminal below, as well as above we should see output showing that the Edge two and the Edge three have had Edge experts successfully deployed. And the amount of data we see should increase as more of these instances are deployed. We started to see data come through. The Edge two has now been successfully deployed so its data should start to come through. The delay here is all about Edge expert getting set up on these Docker machines which individually are not particularly powerful. So it's a little bit of time taken here, but now we're starting to see data from these instances. And we'll wait for the Edge three to be successfully deployed. Now that the Edge three has been successfully deployed, we're seeing data appear in the MCTT subscriber. We can also go into Portainer to inspect what has happened. If we select the Edge one, we can navigate to stacks and see that the Edge expert stack has been deployed. We can also navigate to containers and view more detailed information about what's been deployed. For example, we could look at core data, view its logs and see what kind of, so we see that data is being produced by the system. We can also use our command line tool to look at both where a particular application has been deployed and what applications have been deployed on a particular node. We look back at our app commands, we can see we've got a view option. If we run view, we can do this one of two ways. We can say, well, what about template Edge expert? Tell me about who's got Edge expert deployed. If we run this command, we can see that the Edge one, VH two and VH three all have Edge expert deployed. Similarly, we can run a similar commands app view, but this time if we provide node name, the Edge one, that's going to tell us that we've only got Edge expert installed, but this would give us a list of containerized apps if there were any others that are deployed on the Edge one. So in addition to using Portena, we're able to use our CLI to view this kind of information. Finally, we're on to step four. Step four is all about metrics gathering. So we're back where we were at the end of step three. We've got our app deployed, it's running. We've seen that in Portena. We can go in, look at logs, manage whatever we like. We can use the CLI to inspect where the application is deployed. Now what we want to do is look at metrics and information for the nodes and be able to monitor their running. We use Grafana and Prometheus to achieve this. We've got a Grafana dashboard set up here. This dashboard is pre-configured. If we enter this, we can start to see some information. I'll just change the parameters to make this a bit more responsive for the purpose of the demo. So here we've got a dropdown box that allows us to select which node we want to be the information from. This is by IP at the moment, 167, 168, or 169. Those were the IPs of our Docker machines. And we can use the dropdown box to navigate between them. We can see that we've got 11 containers running. These are two containers for our agent tree. The rest are the EdgeExport application. We can look at CPU usage, memory usage, and disk usage. If we extend the history to the last 15 minutes, we can actually see when the application was deployed on this particular node. We can see a spike in CPU usage, as well as a significant increase in memory usage, at least from a completely empty system. Similarly, we can look at disk usage and network throughput. This allows us to monitor what the system's doing at any time, and we can select any other nodes. It's important to note that the data being shown here is just a small subset of the data that we are currently gathering from each node. The Grafana dashboard can be configured to display any combination of the metrics that Prometheus is gathering. So that concludes the demo. We've followed step one, which is to define the nodes to be managed using our configuration file. We've then done step two, which is to deploy that infrastructure and to add those nodes to our system. We've then defined a template and deployed instances of that template as applications to each node. Those applications have been deployed and we've seen that data is now flowing out of Edge Expert and into MQTT, and we can view that information by subscribing to that broker. Finally, as part of step four, we've shown how we can gather metrics and select individual nodes to view information about what's happening on that node, as well as to view information and historical trends in order to understand what's been happening on the node at any one time. Thanks, Jim. Well, thanks everybody. Let me take you now to a couple of quick summaries about some of what we've done and some lessons learned. And before I do that, let me also give you an indication. Yes, we know that the solution that we have out there is still one where we've got to do some work. Our roadmap includes dealing with non-containerized apps, as I mentioned, deploying other applications. So today we're dealing with Edge Expert and XRT, but we want to be able to deploy anybody's application, whether it's containerized or not. Probably means we'll have to deal with certain packaging constraints, like it might have to be, for example, a WM package, but we're working on that. We're going to put in place an API proxy that allows for more stateful control, in particular refining as one of the lessons learned. Configuration around an application is pretty critical and how do you deal with that in combination with your application? We want to be able to deal with scalable node definition. If you saw in the demo, we're still dealing with a file, essentially, to define the nodes or you do it by adding one by one. That's great for tens, maybe hundreds. That's not going to work for millions and we know that. We want to enhance things like the telemetry and the metrics. We want to improve the application template. We want to deal with generic actions, like maybe you want to send a request down to the system to form some sort of command line issuance. We want to be able to deal with certificates and secrets and then also provide for options. A couple of you in the chat mentioned things like, hey, could we use Kubernetes in Places form or could we use some other orchestration tool? And the answer is not yet, but that's where we're heading. And again, as I mentioned, we want to consolidate the UI and also deal with more node management capabilities. So that's where we're heading. So what are some of the things we've already learned? Scale, scale is important. Edge and cloud scale does not equal edge scale. Again, tens does not equal millions. And when you're starting to deal with millions, just something as simple as getting all the IP addresses for which you have to deal with is not easy. We want to talk about volume of traffic, but in terms of the number of nodes, the amount of traffic coming from each node in an edge solution is typically going to be fairly small, at least compared to some of the enterprise solutions that are out there, but the number of nodes is what is overwhelming. So you start to add that up and you have an incredible scale issue. And how do you provision and configure those thousands of nodes, where every node typically has to have small amounts of configuration that are specific to that node, even knowing what it is, its name, for example, it gets to be very difficult beyond a certain level. And how do you even test for that? We're using things like Amazon services to be able to test at scale, but that starts to be pretty cumbersome after a while. So testing solutions like this gets to be a challenge. Still need agents at the edge. How do we get them there? How do we coordinate multiple agents? Do we need to start to consolidate that? Those are questions that we're wrestling with. And applications are one thing. It's a collection of services in many cases. So how do you start to orchestrate those and make sure all the services come up with their dependencies and such? How do you onboard sensors? Again, given some of the security constraints and some of the unusual protocols that are out there, as I mentioned, one of the big unique differences of edge computing. And there's a tremendous amount of tech that we're finding in some of these enterprise and collage solutions. Hey, some of these solutions gang work really well in environments where you've got plenty of resources. And in those cases, some of the warts can be glossed over by throwing a little bit more or beefier hardware at it. Doesn't work well when you come down to the edge. And we're finding that even in things like the user interfaces, some of the solutions out there have some tech debt which we're gonna have to figure out how to either overcome or replace. Application metrics is one where we're finding most applications aren't really outfitted to report themselves well what's going on and how things are behaving at the edge. And that's something we need more standardization efforts around. And lastly, many of the UIs, while they are comfortable to the operators, well, they are also many UIs. And we eventually wanna work toward a single pane of glass or at least reduce the number of user interfaces so that you're only dealing with one, maybe two in most circumstances and only briefly going to other user interfaces where there's really a problem you need to dig into. So many of the challenges still left to be addressed but also at least as far as we're concerned, many things we've learned about what's working and what's not working in these edge environments. So what's the call to action? Well, we're still working and we're still trying to find solutions here and we're still working with organizations to help collaborate on these efforts. We do have, for example, Intel signed up as a collaborator with us and they're bringing some technology to the bear which is already out there in open source that we think will help with node management. We encourage others to join us in this effort. Work to help standardize some of the APIs especially around things like metrics, collection, provisioning of sensors and devices. In EdgeX Foundry, we're certainly doing that and trying to facilitate things like these types of management solutions. We're hopeful other applications in the edge space will as well. So join us through things like LF Edge which is the umbrella project around EdgeX Foundry and others out there to help address this and please contact me if there's some additional information I can share with you with regard to what we've done and where we're heading. And with that, ladies and gentlemen, I'd like to thank you for your attention and let me check here quickly to see if we've got some questions that have popped up online. I think I've addressed the questions about some of the K3S and stripped down Kubernetes already and where those are gonna apply. We certainly wanna work towards that with regard to our open source efforts and trying to find a way to deal with other orchestration mechanisms in place of swarm. We know that's not gonna work on time. We've talked a bit about the versions of Linux that we're working with today, but we wanna make this a platform independent solution. Do we have any support for K8s today? Not directly, but that's also like K3S or anything at the orchestration level. We really wanna make it agnostic with regard to your orchestration tool of choice. And so we're working to that venue where you can essentially pick and choose which type of orchestration is at the Edge. See, would be nice if you could also find a time to lab more on how you ensure observability in a product, instrumentation. Yeah, that is a key question. We're finding that a lot of our nodes, a lot of the hardware out there, provides some ability to start to understand metrics around member usage and CPU. Your orchestration tool can help do that. For example, Docker can help do that. But when it comes to the application, you're really beholden to the application in reporting through some sort of API or through some sort of interface, what's happening in the application? Some applications do things better than others. In EdgeX Foundry, we're learning through this episode, we need to do more, we need to provide more metrics. Starting with a release that came out last year, November called our Fuji release in EdgeX. We at least offered some facilities there, some metrics around things like the applications use of memory and CPU. And now we're starting to look at how do you facilitate getting more specific metrics, like maybe the number of messages coming through or failures on the part of the application out in terms of some form that can be used and digested by a management solution. That's where we need some of the standardization and the APIs to help support that. We're hoping to drive that through things like EdgeX Foundry and LF Edge, but we could certainly use some help there. So great question. Do you plan to unify the various open source frameworks into one IoTeX management environment? We are a software provider, so anytime we could build a framework that is enjoyed by all, we'll take it, but we also know that you're gonna wanna have choice, and that's why we really use open source technology. And so while we're trying to unify how the pieces work together, at the same time, we wanna facilitate your choice in certain areas so that you can use what works well, for example, in orchestration, or maybe what works well in terms of displaying the data through whatever tool like Grafana you may choose to use. So our hope is to unify the framework, but maybe not be absolutely religious about which pieces are used and allow you some choice there. Let's see, other questions. Is there a way to collaborate as an individual? Absolutely, as I mentioned, please get in touch through things like the EdgeX Foundry Project, through other open source efforts out there. We'd love to have that support. I don't know of an open source effort today that wouldn't like to see individuals as well as companies participate. So grab my email address again from the information that's been provided through this conference and happy to collaborate with you. The demo seemed all well and good, but it was a little bit blurry to read. Appreciate that, yep, in the context of a conference like this online can be a little bit difficult. We will have the demo actually available through IOTech website soon as soon as the product is actually announced here in a few weeks. So again, contact me and happy to try and share that information with you as soon as it is made available. Let's see, I think I have addressed all questions. I'm kind of looking through the list one more time. I don't see any there. So just kind of pause in here for another second. And again, gang, I will bring up, let me go back to the previous slide. Again, my email address is on here, so jim at iotexist.com. Happy to correspond, not only about the effort we're doing around edge management, but also if you have questions about EdgeX Foundry, LF Edge, or some of the other efforts that are going on in IOT, happy to communicate and collaborate with you there. And as mentioned, EdgeX Foundry and LF Edge are the two projects that we're very much in touch with today through IOTech and where some of this work is ongoing, but you'll also find lots of others out there. But look, do a Google search for EdgeX Foundry or LF Edge if you want to help kind of collaborate at that level. With that, I'm turning to my wonderful staff here at OSS and Vanessa in particular, seeing if there's any other chat questions or issues which I need to address. Vanessa, are we good? Looks like, I'm not seeing others. Yep, oh, she has left the building, I'm being told. So Matt's telling me we are all good, so thanks. Thanks to the Linux Foundation and all the folks here as part of the open source consortium putting on this OSS ELC endeavor. It's been a great, great amount of effort to get this going and they've done a fantastic job. With that, you've got one minute left for the end of this conference or I should say conference session. I'm gonna give you that minute back, you might need a coffee refresh before your next one. Thanks everybody, been enjoyable, wish you all best and good luck in your IOT endeavors.


We can start with decline, because I think I don't want to decline maims. You start on the back, because he thinks he knows what decline means. So it's like, like, if you don't do something, it's like, decline like...  And actually we can write thank you yourself. You've got some opportunity. Yeah. That'd be the definition now. But it's still this year. But we have to find tech's evidence of this to get to him. So... Hi. I need these. Honey, these will publicize through still mysteries decline. How many? Like, they'll decline in mysteries. Like, decline in... Decline in... Like, decline in... Decline in the colonies.