There's a layer of liquid all around the outside here, in between the inner globe and the other globe, so that it's a nice bearing, you see. But it's moving. Now, I received this in the mail, on the same day that Martin Gardner got one in the mail, with no letter, nothing, explaining what it was all about around here, it was just free, the FedEx man came to the door, and a big sorrow phone box had that and gave it to me. I unpacked it and I set it up. I thought, who sent this to me? So I looked at the bottom of the globe, and it said MOVO, M-O-V-O, so I looked them up on the internet, I found out where the company was, and I called. I called Martin on the phone first of all, and he said, yes, I got one of those. He said, they're wonderful, and it turns constantly. I said, yeah? I said, who sent it to you? He said, I don't know, and I said, I don't know either, I'm gonna find out. So I called the company, and they put me in touch with the president of the company, and he laughed at the phone. He said, I didn't know whether Martin Gardner would call me first or you would. I said, well, he's calling you too because I'm calling on his behalf. I said, what is this thing? Well, he explained it to me. Now, I already partly figured out, it's powered by lights. Is there a ring of photoelectric cells all around the top here, okay? And I figured that much out because when I turned up the lights, it stopped. So that figured, all right? Now, what happens is there's a computer chip connected to the power source, which is the photo cells, and around the base, the equator, pardon me here, there are a number of longitudinal, longitudinal, longitudinal, I can say it, I know they can, coils, arranged like bicycle spokes. Now, what the computer chip does is at random, it chooses any one of those and puts all the photoelectric electricity into it, which turns it into a compass, is he which tries to turn, now that's so, this is North, it tries to turn North and South. But as soon as it goes 30 degrees, the computer automatically switches to the next clockwise one, and so it's a frustrated compass, it's always trying to turn and it gets stopped each time as he has to start to turn, that makes it into a motor. It's interesting, is it not? Sure. Beautiful thing, this is the, this is, there are four versions of it, everybody. This is the NASA photographs, it's made from NASA photographs of the Earth in various times and seasons and whatnot, and so that's the one I chose, I would have chosen, pardon me, had a choice of it, but the inventor told me that he was very pleased that we were pleased with it, and he said he chose it because it was the most scientific one. Well, this might be something that Norm Endman has never seen, and that would be interesting to know because there isn't much that Norm Endman has ever seen. Okay, this is just a copper pipe, all right?  Now, how does it do that? Yeah. Stand up if you would be so kind, thank you.  Hold it a little bit higher so you can see all the way through there, okay? It floats down. Now, it'll come up the end in a second, and here it comes, and it's done, there it is. But you see it levitating down the tube. The reason is this is actually a very strong magnet, a neodymium magnet that is copper plated, or nickel plated, pardon me, and this is a conductor. Now, when you pass a magnet by a conductor, that induces a magnetic field, right? It induces a magnetic field of the opposite polarity, and so that is being supported on a magnetic field as it goes down here, and it only touches the side occasionally, and that resist is there. Isn't that an interesting throne run, though? Now, I'll show you one little double the effect here. Well, I'd get a smaller ball here. Now, you know this is a magnet now, but this is a much smaller one, but what's this effect now? Now, a German scientist a few years back in Buffert-Hollover, as a matter of fact, showed me this. This is quite an interesting experiment. This is a magnet that, you see, I've marked a black mark there on one of the poles. Watch what happens now. You see it going chum, chum, chum, chum. It's not going smoothly. You think the reason is? Come on. Now, I couldn't figure this out. Since this is south, and this is north. As it turns over, it's north and south poles are turning over, you see, but it's resisting it. It's resisting turning because it's resisting the magnetic field of the earth. If I do it at a greater elevation like this, it goes down relatively smoothly, but still a little bit jerky, and when I do it like this, you see the distinct motion like that? Now, this is the one that re-puzzled me. Watch it now. Very smoothly. You see it doesn't jerk at all because it's going east and west. It's not opposing the north and south poles, you see. It's just going east and west and it doesn't care. But it does spin along with this north pole on one side and the south pole on the other side, attracting the magnetic poles of the earth. But I showed this to kids, and it really gets them thinking. See, it's very smooth when it goes this way. And the marvelous thing is, when you take this, you do this, but then it goes away and it ends up in this hand again. That's astonishing. That leaves it has me surprised. I don't know how that works. I have no idea whatsoever. Try to separate the magnetic discs up. I'm going to try very hard to do that. Just getting them apart is so difficult. No, my hands are not very strong now. I just can't do it. But they're held together so tightly, I can't even get one of them off. Ah, there, I got one of them off. Okay? But they're held on, and I've got to bring them together slowly because they're such strong magnets, they will break, they will actually shatter because they're very brittle. And I'm putting it, oh, gotta be careful not to touch my hands. Hot, did it again. He's easy. He's really easy. The technology norm is going in at such a rate. It's really hard to keep up to it. Yeah, you can't keep up to it. Oh, wait a minute, I'll show you. Oh, maybe I don't have it on this phone. Let me see if I do. This is the iPhone. I don't remember whether I put that program on it or not. Let me find out. No, I'm not seeing it. No. No, I think maybe I don't have it here. No, I don't have it on this very present phone. But there's a program in here. You just press a button on it and you hum or whistle any tune into it. And a little thing turns around and suddenly it comes up with the name of the tune or the symphony or whatever. You know, you can go ta-ta-ta-ta, ta-ta-ta-ta, Beethoven's Fifth, and it'll show Beethoven's Fifth. First movement, it gives you, and another thing too. On books like this, it's a regular book like this, any old book that you pick up, there'll be a barcode on it to some place or other. Let me see one on here. Probably in here someplace. No, I don't see one on that. Let me show you. But maybe if there's one cover on it, I'm sure we'll have it. Let's find out. You have a bar, yeah, and a barcode on here. Now, and there's one on here too. That barcode, if you hold the iPhone, if you've got the right program in there, up to that, you press a button on it, and it'll show the name of the author, the publisher, the date of publication, number of pages, all that information shows up immediately. And on a product, if you hold any product up to it, you know, something that you bought, I don't know, there's no barcode on this ruler. Well, there is. Here's a code right there. If you put it, it'll say the ruler, the name of the manufacturer, he leaps company and say, it'll give you all the information about it. But on your telephone, now, I remember a time, you may remember the time too, in the long distant past, when you bought a telephone in order to make telephone calls. That's been forgotten now, because people say, I can also make phone calls with this. Oh, really? Oh, I'm sorry. Sorry, sorry, didn't mean to offend you. But it's astonishing what you can get into a phone cell. Come on. This is getting ridiculous. You can play games with it, you can play everything but pole, whether I think. You're complicated for me. Yeah, me too. Me too, it's gone beyond me now. We should just relax and enjoy what we can get. I still use Q-tips and a few things. On Q-tips, I can understand. That's not too difficult for me to grasp, you see. But some of the computer angles, I just can't get to that at all. But Q-tips, oh yeah, I've mastered Q-tips. But anyway, all the books that you see lined up on the floor here, right up to this point here, are all books that I have written in different languages. I've got 11 different languages. That's great. Congratulations on that. This, for example, this is in Chinese. And this is what book, I don't even know what book it is because I can't read the Chinese. I have to look for the illustrations. Psyche can rest here. Is it? So it is. You're absolutely right. Good for you. I agree with you. Those are both these things. And I'm looking for a special shelf to put these on. We haven't got room for it now. I've got room for my books. I can't write any more books as we can't store them. But, you better keep writing when you need them. Oh yes, I've got one coming out called A Magician in the Laboratory. And I'm working on it right now and it's gonna be quite a big book. I've got 27 chapters, I think. I'll plan that. And I'm working on that. For more of James Randy and the Educational Foundation, make sure you visit randy.org.


Hey guys Say bear 1,000 here. You want to explore this cave with me? Who's up for it? I'll be back with you in just a minute Stay tuned Okay guys, we're here. We are at Danes caves You don't know Danes cave in yes Danes caves. There's a lot more caves Then what you're going to see Right now since we take you up on this so stay tuned and we're going to head to the cave and We'll be back with you in just a second Okay guys, we're here at the cave. Here's the cave. Check it out guys It's cool. This is like a little bridge here This is also known as vandal cave because as you can see The things people do here, but this is like a natural bridge That we're standing on right now because if you look over here It's another section We're going to be going right around here, and I mean I did bring the night vision camera And we're going to be going down in here so I'm going to catch you off and We'll be right back as soon as we get down in the cave Why not?  It's very slippery here. Mommy wait here, okay Fun parts going to be coming out Very slippery Did you hear that? Okay, we'll be back with you in a second. That's where we came down in it right there Through that hole so I gotta go get monkey room. I'll be right back with you Yeah, you see by the bottle how small that hole is I know that's I can I can't do that.  There's no way you gotta be flexible for that one Oh, I'm going down in it. All right. You have a good time. Let's take your camera with me. I'll feed you dinner I'll just hand it to you Oh, wow Wow, isn't this cool? Need some light So I can show the graffiti Okay, we're in another one And here Of course, they have all this writing people's names Kind of cool, isn't it? Wow Look at that, babe Cool, huh Hard to believe that we live what 20 minutes away from that 22 minutes. Yeah From here. Yeah, that's something, huh? Yeah, can you show that hole down there? There we go He's my backup light so I can show you all. Yeah, I'm using this light for her on her camera. Yeah, she's on her phone right now Yeah, that's cool Wow, so I'll be back with you in a minute.  Wow Can you see? Look at this camera turn on here. I am not attempting this one. It's too small for me I think my hips will hit the walls But a couple of kids while they're in their 20s, I would say or even a little older They went in there curling a guy They thought it was cool I'm going I'm recording All right, you can see oh it is now though.  No, I'm good. Just try and get some shots for you. Any since you can't come back in That's all right.  You can put a couple people standing up. But yeah, it's pretty small Help me too Filming you Now you're using my same that you made fun of a week or so ago Yeah You can't be real big getting in there. I was crowning you're filming me filming you Now you're using my saying that you made fun of a week or so ago Yeah Well, look look at you. You're touching the walls Yeah, you can't be real big getting in there. I was crowning Any bigger and you wouldn't fit in Now here she'll be powered baby. Yeah, that's why I gave you the shirt. So you'd be all right There you go, babe. So where you going, babe?  Look at wow Me and bruner stay here Yeah, we watch you If you think I can get in there, I'll do it Yeah There he is going you're filming my demise, right? Yeah deeper into the hole He was watching his daddy. See he watching turn the mic on And you're filming my demise, right?  Yep What'd you find Just a cavern. Yeah, just a room a big room Pretty good size. Yeah, I filmed it for you. Okay He's getting his exercise today Another one now There you go And I'll be back with you. Okay Right now against the ceiling and my belly's on the floor right up in here That's where the kids got stuck Back in 1999. They tried to make this turn to the left here They got stuck There's very little oxygen in here And it's going a little further Dead end's right Back there, but it goes down Get down in there rough down about About 15 feet And then goes way back up in To the left right up in there The one I got all day Explore this one to get way down in that bottom there I'm gonna find some bats for you guys I heard something When we was way up In there Something was like crawling around It's not may have been a bat, but I couldn't see anything. This is the ceiling Right on Monkey's waiting on us Let me get my truck back out of here We're only back in about 23 yards But what This is cool. I can get The top of the cavern we are in right now. See my big eye or light You can see that Oh, yeah, definitely got come back the headlamp and This cave in depth I could probably Spend several hours just in this cave alone Judge off here guys I'll be back with you Just a second. I'm going Here we go. I just came down Here and did some see that you can see your breath in here But I'm sweating. I just came in here I did some night footage or night vision footage But that's what you're seeing That's my breath But this is what I'm looking at right here Now this is not the night vision of course But and I'm not smoking it just it's this chili down here. You can see your breath But I can say I'm sweating We were just up in here with the night camera or the night vision way up in there I don't know if you can see that But we was way up in there That splits left and right. You'll see it on night vision. Hopefully But on the whole day because I want to explore this Let me get dark soon We're at 530 right now I want to get back in here I think I can wiggle my way around Back through that hole I think what's going on I'm excited to be down here I think the hole is big enough for here But this is cool Like I said, any mist or fog you're seeing My brother's mouse. Look. I just seen a mouse I don't know if I got him on camera or not These things Look at these Echo But I also want to get Back in there From I can see Like to where it either turns or drops down It's good 30 feet back in there Because I don't know where that mouse went. He was right up in here Okay monkeys Wait patiently on us She bear 1,000 guys Always doing adventure I knew she could get down here It's known how much of a problem she have getting back out I'll see if she wants to come down And here we are in daylight Hi monkey Seen a mouse down there Yeah And some kind of cricket looking thing. I don't think it was a I don't think it was one of them cave crabs, but it was really neat Yeah, I zoomed in on him, but I couldn't see any of my glasses, so I was only looking at through here, but it's pretty cool Yeah, that's cool So let me climb up out of here Huh? You have a hard time No, back here about 15 feet you got a duck, but after that you come into a cavern Big dome cavern And after that you got ducks some more and you come into another cavern And it splits off So I think we're going We're going it right here Here, let's do it right here in front of us How's that sound? Now we're going to do our closing right here guys Right here in front of the peace sign on the big oak tree. You okay there? All right, we're going to sign off for now. Thanks for joining us on another adventure here at Shaber 1000 and Monkey 1000 Monkey 1000. All right, we're going to go for get start I think it's Sasquatch over there Is there a bear? Well, there are bear out here I know We're in a forest I know I don't want to be dinner Time to go All right. We'll see you guys soon. Bye. Bye now


Happy new week everyone. This is the weather forecast for Longmont covering Monday through Thursday, March 23 to 26 I'm chief meteorologist John is worth for Longmont public media The new moon occurs Tuesday March 24th, so it'll be in the vicinity of the Sun for that whole day Behind the blue sky so rising and setting about the same as the Sun nothing to see here so let's get to the weather for This week we are definitely enjoying spring like temperatures and it is spring so enjoy the spring I'm looking out further in the miles to see if there's any chance that we have another shot of winter and Climatologically, we certainly can get snow in April and May and we've even had snow as late as early June Little bits, so it's probably not over but in the short term and make medium range and I don't see it We have high pressure starting out this week with a storm system down here in California It's kind of a mirror of the last system that gave us all the snow and rain But it's vastly weaker and it's moving pretty quickly So won't be as much of a thing at all So as Monday morning When I'm recording this Comes by we have this little trough sitting down here in Nevada Central California this little piece of energy here will break off and zip pie of the state today By late night Monday in the Tuesday morning You can see that the system is here a low kind of consolidated to the east of the mountains and is sliding quickly Off into the plains and there's mountain Moisture and snow going up there not a lot in the way of rain or precipitation down in the plains, but there'll be a little You can see in the 10-day Ensemble this Monday night Tuesday morning chance of rain is a pretty good agreement between all the different runs of the GFS But there's no snow temperatures here all above freezing this blue line across here We'll get to this next system at the end of the week shortly So by Wednesday, we have that trough that was out there the southern part of it remain behind I kind of connects with another ripple in the flow aloft in the northern jet Making one kind of bigger trough and that's going to start heading to the east and because it is a Larger feature it's going to go a little slower. It's got this ridge that was keeping us warm for out the early part of the week Slowly pushing off to the east. So this is going to hang around a little longer as it passes For the five day though the snow Going through the end of the week is not very impressive. It's up in the mountains mainly. There's some places in the six to 10 inch range and in favorable locations, but most of the mountains are only getting a few inches and There may be a dusting down in Boulder and the western suburbs of Longmont and Hygiene and like that for rainfall and not very Exciting either we've got some local areas of up to an inch or an inch and a half or so up here in the Northern mountains down here. We're generally around a tenth to under a quarter of an inch of water So putting the whole week together we've got fifties and sixties for the beginning of the week with a little chance of the afternoon overnight precipitation Monday into Tuesday Really nice weather after that Thursday and Friday some rain chances It won't be all the time and it won't be very much But you'll be cloudy you'll smell that spring moisture in the air and it'll be kind of ice We cool to the fifties and just barely below freezing at night into the weekend We start to crawl back up towards the sixties with sunny skies again For more local news take a look at the Longmont Observer at longmontobserver.org and their more frequent text updates and graphics for the weather that I post as conditions warrant For Longmont Public Media, this is Chief Meteorologist John Ensworth. Keep looking up


Are we live? All right, sweet. Well, sorry for the delay. There was a little scheduling mix-up here This is spark hara late, but ready to go and I'm Michael McCune. I'm a developer with red hat, and I'm joined today by Chad Roberts who's also a developer with red hat and Nikita Konovalov who's a developer with Mirantis and I'm gonna start with a little disclaimer What you're gonna see some of the code and the demonstrations that we're gonna do are not upstream yet Just a fair warning. These are in github repos. You can download them and experiment at your own leisure So what is spark hara? We started with an ambition to create a Heartbeat monitor for open stack based on the logs that are created from service controllers This was also a way to dog food open stack so that we could use Sahara to deploy a data processing cluster and then consume the logs through zikar and Display them for someone to watch It's also a silly name. So spark hara I don't know how we put it together, but it's a portmanteau of spark and Sahara and it just sounded funny to us and Also, we wanted to show a Window into the possibilities of how you could analyze log data Using spark and then visualize it to see what's happening with a stack in real time So you could see errors bubble up or you could see perhaps performance issues as they happen in real time And why do we do this? We wanted to create a real-world use case a user story that could show how you could consume these services To provide something for operators We also wanted to use kind of straightforward methodologies There's a lot of technologies out there that can be used to you know sort logs and accumulate them We wanted to show how you could do it in just a straightforward manner And we also wanted to kind of inspire all of you to take a look at this and say hey What can I do? Where can I take this? You know the possibilities are endless So what do we use to build it? Of course we use the core services as you know We need to then we use Sahara and zikar and trove and on top of those we use spark and zeppelin and the Mongo database and we're gonna get into how did we use these components and and what are they? So at this point, I'm gonna hand it over to Nikita and he's gonna talk a little bit about Sahara and spark Okay, thank you Michael so Sahara is an open stack service and Its main mission is to provide two components to the user first of all It provides a provisioning API which allows user to spawn Hadoop clusters in minutes And then when the user has its own cluster with Hadoop services running Sahara provides the data processing API as well So I'll go quickly about what makes Sahara and what is under the hood of these Components and then we'll go through the demo. So Sahara is a truly open-stack service which exposes all its functionality through the RESTful API This RESTful API is reachable by a Python client Which is an SDK to other applications if they want to use Sahara clusters. We also have CLI so the Administrators can operate with this and do all the operations through the common line From the user perspective the dashboard in horizon Displays all the templates all the jobs and all the clusters which are operable in Sahara with the tenant tenant filtering name filtering And all the ux stuff you might want to have from a UI Of course Sahara is complex complex service which has a lot of components under the hood so all of these components are checked in the open-stack gate jobs as well as the Provisioned clusters are checked by third-party CIs for non-fake integration testing that allow to spin up Hadoop and check that it's really working in our in your open-stack cloud So let's go a bit deeper into the Sahara. It has a provisioning engine So this is a component which decides how the open how the Hadoop cluster is being provisioned on the OU cloud The main goal of this engine is to cover all underlying operations and Make them being provisioned automatically the engine operates in steps and all these steps are locked in our new Recent feature which is even locked so the user can track how the provisioning is going When the provisioning is finished the cluster is operable, but it's not this final state so the cluster is scalable and user can edge or remove nodes depending on his workload and Change the name change the size and the services run in a cluster Sahara also takes care about cleaning the resources when the cluster is not used anymore. So the deleting cluster Will remove all the VMs volumes floating MPs and whatever has been created for this and you leave your cloud free and Clean so the provisioning engine has been tested to prove to spawn 200 nodes VM cluster in a matter of tens of minutes and We're actually right now in a process of migration to heat provisioning and This will allow more flexibility and more reliability when you provision a larger Hadoop clusters So what is inside what will be provisions Sahara provisions the Hadoop in plugins manner, so there are just different distributions and Sahara has plugins for each of them to provide a Hadoop That suits your use case the most so there is a vanilla plugin, which is an upstream Hadoop build build by Apache and It is a reference implementation or what a plugin should look like We also have a clodor distribution and Hortonworks data platform being provisioned as a complex Hadoop distributions Which have more than just Hadoop under the hood and they are Usually recommended for production use cases There are a few solutions other than just pure Hadoop builds, which is a spark platform And we're going to talk more about it today But also we have a storm support appeared recently, which is a best Scenario for streaming processing and my power with their my paraphrase implementation is also available in Sahara The versions for all of these components are of course different and we're trying to keep them up-to-date with the recent releases So the plug-in and the Hadoop versions are almost always up-to-date with the current upstream of Hadoop Then so the elastic data processing is the component that Is responsible for running the workloads on an existing clusters? Basically, there are two types of workloads Sahara can run today is that first one is Hadoop MapReduce And that's a big data classics with its big and big queries High-vehicle like queries, but also we allow to run any generic jar file or shell script so if your job is compiled and You have tested in a standalone Hadoop cluster. You can just submit it through EDP and it will run also there are second Second type of jobs, which are the streaming and we have spark and storm plugins for those and of course park and storm Submission process is a bit different, but it is available through all Sahara CLI OpenStack dashboard and other interfaces So you can experiment with that Now let's get a bit closer to spark and today's talk. So Apache spark is a fairly new data processing engine and it is One of its main advances is that it runs in a different modes It's not bound to any specific clustering engine. It has its own standalone mode Which is provisioned by vanilla spark plugin, but it also can run on yarn or mesos clustering which are supported by yarn is a yarn and yarn engine is supported by the clodera plug-in and it is being supported in HDP, but that support is in development right now Spark is a bit different from Hadoop. It does not provide a storage layer. So it operates on the storage which is provided and it is Not bound to a map reduce concept, which is the matter in Hadoop so this is a just quick comparison on what are the main differences and These things you should think first when you decide which engine you would like to go Hadoop or spark So in a case of functionality provided again So Hadoop is mainly built around map reduce concept and it is doing it very well so spark is More very provides you more variations It has over 80 built-in functions for not only map and reduce but also different filtering multi-passing and other stuff Storage is very important for big data processing and Hadoop tries to store all its data on your persistent disk storage, which is As good as you can configure, but this is the Hadoop relies on the storage for data processing while spark tries to keep as many data as it can in the RAM and run The data processing in the RAM and only store when it's really necessary As for the non-map reduce and streaming processing Hadoop has this possibility, but it's very limited to the that jar file, which is provided by the Hadoop distributions and Spark is way more advanced in stream processing because of the micro batching transaction functions and Almost any possible data source that you can provide it As for the scaling capabilities both of these frameworks declared to be infinitely scalable which is almost True, but very hard to test because they are relying on the network storage and other infrastructure So usually the framework itself is not a limit to its scalability. It's other stuff so The question that is being asked very often when people are start comparing their spark clusters to Hadoop So is spark always better? Is it more has it has more functions? It is faster in RAM processing. The answer is The use case for each data processing job is different and you should see the benefits in both frameworks but spark is better when you have Reasonable amount of data and you have to do multiple passes while processing it so when it fits in RAM your choices this park and It spark is always almost always better for streaming support because it has a richer data source support as Again, I've said spark is supported in Sahara as a standalone implementation is vanilla spark plug-in and CDH will provide your spark running its containers on the yarn system HDP will be providing this functional to version Right now the standalone mode runs one of the zero or three point one spark releases and Cloudera also has one point three release, which is Available from Cloudera five point four distribution The configuration is available through Sahara interface as well as the Hadoop configuration So the core site and other Environmental options can be passed through the Sahara API They can be stored in templates so you can use You can have the configuration that is reusable and All of them are still provisioned automatically. So Sahara will handle everything like provisioning the keys spawning up services Formating the storage and everything for spark cluster as well as the authentication for Keystone and Swift data sources, which is required to run the data processing on OpenStack The spark pull workloads are available by ADP and Sahara uses spark submit script under the hood But that doesn't mean that your cluster is limited to this spark submit tool because this Cluster is ready for any external tooling that is ready to operate through Spark's APIs so any spark python java compile script or Shell script will be executed Either through a spark API or EDP API or directly by user. It will run in the same mode of fashion so in case of the data sources that is supported by Sahara right now there are HDFS and SWIFTS manually is coming soon and To talk more about how we can operate the data on the different data sources. I'd like to handle it back to Michael and tell you the Exactly use case. We're trying to achieve with spark and Sahara today Thanks Nikita All right, so we'll just go over Zeppelin is a web-based UI that allows you to interact with a spark cluster and It's kind of like an ipython notebook. You can you can do visualizations. You can run code samples directly from your web browser It also allows you to collaborate So if I create a notebook that has some you know, particularly nice algorithm I could share that with Chad and then he could take it and run some visualizations Pass it back to me and that in that way we can share in on some of the live processing that we're doing We also used a trove to set up databases one of the One of the things that a trove can do is it's a database as a service you can deploy relational or a no SQL databases You know can do my SQL Maria Postgres Mongo Cassandra Redis. It really simplifies deploying the database for you You can use it to create instances or clusters of you know for your databases And you can also use it to resize database volumes. There are some other operations that are available with trove You just have to kind of explore and see how you can copy databases back them up shard them those kind of things And then we also use the car Zekar is OpenStack's messaging service. We used it as a queue Between multiple endpoints. It can work in multiple tenants It has a restful API like the rest of the OpenStack services and it supports event broadcasting task distribution and point-to-point messaging so you can you know configure your endpoints so that something can push messages into the queue and then Zekar can message the subscribers to let them know that a message has arrived for them or you can pull messages off the queue So that those are all the technologies we used to kind of put this together So the next part is you know kind of what did we do? so The OpenStack service controllers create a lot of log data and those logs have really useful information in them But it can be difficult to kind of pick through them all so In the spark our workflow the OpenStack service controllers create the log data and then we have a process that takes the log data and feeds it into a Zekar queue and As it goes through the queue we have another process sitting on the nodes with our spark Processes and they pull messages off the queue and feed it to spark Now once spark has got it then we can do the processing and you can hook in Zeppelin You could normalize the data and send it to a database. We used Mongo You can also send it out to just about anything you want in this case. We used an HTTP endpoint Spark is pretty flexible in what you can do you're not limited to just a map reduce kind of interaction You can you know do whatever you want to you on the data that comes in and So to prepare for this we needed to create some custom images and this is where the the disclaimers and provizos come in We were using spark 1.5 Which is one of the latest versions out there and we created some custom images that we could deploy through Sahara to talk on these We also needed to configure DevStack to add in the Zekar plugin and the Trove plugin So they were available to us and then we needed to customize the Sahara cluster operations a little bit So that they could deal with spark 1.5 because right now Sahara is only using up to spark 1.3 So just had to tweak it a little bit so that we could get these these new images in there So the next part was moving the log data The first thing that we did was configure the controllers so that the the logs were Formatted in a way that was easy to consume on the endpoints The next step is reading the log files because they're just existing as files on a on a server somewhere And we need to move them to Zekar So we wrote a small Python application that just tailed those log files and pushed the data onto Zekar So here's an example of How the controller might be configured in this case it was the Sahara controller We just basically told it to Output to a log file in opt logs and then the next four lines are where things get interesting that's where we started to format how we want the messages to come across and Really, we just went with a simple approach using a double colon to separate between various fields of the log So the the date time stamp what the process ID was, you know What what kind of level came back from the log? What was the log message? Etc. Now reading the logs This is where the the Python application that was sitting on the same node as the controller It could be sitting anywhere as long as it can access the logs We just ran this command let it tail the logs as they happened and Instead of making it a multi-threaded approach We just went with a single process per log Started up and have them feed the logs into Zekar in those processes We would then hand off to Zekar So we just wanted to take us you know a simple approach to this just tail the file for every line That's created just send that line straight to Zekar There are probably better ways you could do this you could use more complicated technologies, but we just wanted to keep it simple Also tagging the service type This is something that the approach we took was to to based on What socket was being used we could identify what service was was sending their logs in there Other ways we could have done this we could have tired We could have tagged the service at the point where the rock logs were read and sent them up as something more complicated But we just wanted to keep it as simple log lines coming across and then log line Kind of chunking was a problem we came into where sometimes when you get an error a service controller if it's configured in debug mode will send a stack trace and Those stack traces don't come out cleanly formatted. So we needed to make sure that that those would get a Compiled into a packet that could be sent across to Zekar as a whole so that it didn't appear out of order when it came onto spark So receiving and processing this is this is happening on the spark nodes and the spark nodes need to be able to Listen to a Zekar queue and then take from the Zekar queue and push them into the socket streaming Endpoints for spark And then you have you have choices about what you want to do with the data at this point because this is another area where things Could get split up or sliced in a way that may not be advantageous We went again we went with a simple approach. You're just getting these log packets across take the whole packet and send it to to spark So receiving the data again We had another small python application that we would run once per service that we wanted to listen for and In this case you can see we just set it up for the port We wanted to send it to so that spark knew what service and what queue was based on you know The service type that we wanted to send it across We we could have put all the messages into the same queue and tagged the data before we put it into Zekar But again, this was the first approach we took it may not have been the best And then this application would move the information from the Zekar queue onto the same node where the spark Application was living So feeding spark There are there are different ways we we can get information out of Zekar We chose just a simple approach of looping and Reading the queue to see if new messages were available And if not we just sleep and then you know wait for new message to be available We could have used the v2 Zekar API to set up a subscription model where Zekar would have sent us log messages as they occurred Now sending the messages to spark is where we get into How spark processes incoming data there is not a Zekar socket streaming Method for spark at this moment, so we had to use a tech socket stream which basically reads new lines separated lines that come across the socket and Forms them into rdd's which are the the basic primitive of sparks processing So it would based on a time slice spark would read some number of lines that came in on a socket And then it would ship those off into an rdd to be processed This is this is where things could could possibly slow down because of the way that we're separating the messages How we're how we're feeding them to spark there's a lot of timing issues involved here And it it gets pretty deep in the weeds, but yeah, you'll see So we've got some data. It's in spark now. What do we want to do with it? This is part of the spark application We wrote and as the log lines come in one of the things we want to do is normalize the data So you saw before we use the double colons to separate the fields We want to break them apart now, and we're going to turn them into a JSON object that we can put into mango So first we normalized it then we could store the data you could put it into mango You could you could build a more complicated schema if you felt like it and put it into a more structured database Then you could also signal an external application This is kind of the next step after we stored it We took that we wanted to say how would anyone know that this data was stored in the database if they weren't just you know Looking at the database so we wanted to signal an outside process in this case an HTTP endpoint that new data had arrived We don't want to send all the log lines to this endpoint We just wanted to let it know that new logs were available and some number of them had arrived and that here's the IDs They can be grabbed from the database There are other things we could have done that with this too We could have used other services to initiate actions based on information that was coming in Spark could have been set up to to signal an external server that you know something had just happened and you needed to take action Maybe send an email to an operator to let them know that a bunch of error messages had just arrived And maybe you should look at your stack The options are really endless here spark is very open and what it allows you to do So normalizing you know what do we want to store? How should we store it? We went with a very simple approach again just packaging up the log files and pushing them into Mongo and Just it the way we put them into Mongo was we stamped the time on them We put the account of the log lines that had come in within that RDD and then mark was there an error in that? Packet and then the log line so we just shipped each one of those packets to Mongo which made it easy to consume on the on the other end So storing signaling you know wherever you want to take this Getting the data out of spark is the next point and and like I said We just kind of packaged this up pushed it into Mongo We signaled a listener on an HTTP endpoint to let it know new new information had showed up We could have also pushed this information back to a zikar queue Which probably would have been a better approach because it could have been consumed by more services then Again, this was the first stab we took next time. Maybe we'll put it on the queue So now we're going to get into how did we visualize the data so at this point spark has taken the data It's normalized it we've put some of it into the Mongo database and What else are we going to do with it? We've sent some to an HTTP endpoint We've also got Zeppelin set up and the data is available to us So at this point I'd like to turn it over to Chad and he's going to talk a little bit about Zeppelin Okay, so one of the tools that we use to visualize and explore the data from Mike's app was Zeppelin So recall from an earlier slide Zeppelin is our web-based UI kind of ipython notebook like to let you kind of Fail quickly which I did a lot of and iterate based on that I See there's two kind of basic use cases that that I used and Kind of went with on this the rapid interactive development for the data exploration Kind of see what you got play with it a little bit see what you can maybe make out of it Zeppelin works really well for that And the other part it also comes with them kind of very cheap and free Visualizations, it'll give you some charts And you can do quick simple reporting with that When you get a chart in Zeppelin you can actually just take a link you can link that paragraph They're called there and you can send that to somebody else and they get essentially like a read-only Version so they can see the chart and they can if you update it it'll update their chart live for them Even if they're looking at it So it's kind of a quick nice to the The Zeppelin can give you So for this example Spark hara we took the data from Mike's app that was loaded into Mongo hosted by Trove Oh Then the Sahara tie-in Spark was of course running on Sahara And then we will show you a quick simple chart that was created That is of course shareable the first thing in Zeppelin here, and this isn't necessarily spark hara related This is just a quick You know hello world sort of thing In Zeppelin this is showing the spark pie example if you're familiar with spark I'm sure you've probably seen it on the spark documentation It just calculates pie using the what Monte Carlo method. I think it is the top paragraph there is Using scala And the bottom paragraph is doing the same thing but using pie spark The bottom paragraph you can see there's that it starts with percent pie spark That just lets it know that you're invoking the Python interpreter Scala on the top It's the default so you don't really need To put the percent anything if you want it to you can put percent spark there So then we get into the more spark hara bit And so one of the use cases was to quickly look at our data and maybe play with it a little bit So what we do here the first few lines you can see where we're grabbing the data from our Mongo database We're just kind of sucking it in and we're gonna create a Data frame and if you're not familiar with spark a data frame, but just think of it as kind of a Database table that you can it has columns and you can query on it But it's not really a database. So don't tell anybody that I told you it's a database So here we just quickly we slipped it in and we Displayed the columns that were created in our data frame so we can see that our data has account some errors some log messages Timestamp and a service that's created along with it there And then we also ran a quick query the last line Before the output starts there We're gonna see that I and we're doing this on two services. It looks like Sahara we had a lot of log messages from Hopefully it was a good day not a bad day for Sahara a neutron. There was if the few messages also processed there This one I hopefully yeah, you can read it. We're at the big screen here a lot of this again Is some boilerplate at the top getting the data from Mongo again loading it into an RDD The bottom part of this is where I'm actually doing something a little more interesting with the data grouping it into like time buckets the The thought here is I'm going to be making a chart out of this So I'm going to kind of creating some time series sort of data that you can easily chart There's a little bit of extra code here that doesn't really We're not making all the charts that I originally have done in a different version of this talk We're just going to be showing one of the charts. So some of this code is a little extraneous for for our purposes today But it's fairly basic Pi spark stuff nothing too magical there really So we've loaded all of our data into this data frame Here is how we make a chart. It's just we have four lines of code To generate all this magic percent SQL this lets us know we're going to be doing, you know a query here Pretty straightforward Select on the hourly errors that was the the table the data frame that I created And you notice that there's a drop down there Sahara's the currently selected service There's a little bit of code there, but You can see where service equals and then there's some braces you can actually create drop downs You can create text boxes that the user can dynamically Enter and change the query and the graph will update as soon as you change that So it you know that's quite a bit of functionality to get in just a few lines there And it takes you know this whole thing probably just took a few minutes to create really and you could probably do it Quicker because you're probably a better spark coder than I am So what else can Zeppelin do I couldn't quite didn't want to do a whole demo on Zeppelin It has interpreters for spark py spark SQL hive flink Cassandra lots of other things. It's open source You can you know if there's a framework you wanted to work with you're you know at it. I'm sure they'd love to have it Dynamic dependency loading we actually did that the the drivers to extract the Mongo data You can load those drivers kind of runtime. You don't have to even have them on your system Zeppelin will go out and grab them for you It can handle streaming jobs if you're interested in Zeppelin you can go and run their tutorials There's a really cool Twitter based example It'll grab a bunch of Twitter feeds and you can see what other people are tweeting Not all safe for work tweets. Don't make the mistake. I did You can load other charting libraries if you're not happy with the the basic charting they give you you can use angular and Grab your own charting library and then use that there's examples for that. I think maybe on the the user list If you wanted to low run your notebooks via set schedule, you could also do that. There's options in the UI For that but like I mentioned earlier for reporting you can provide links to the individual paragraphs And share them like that Yeah, the last thing of course a Little caveat here Zeppelin's not built into the spark plug-in for Sahara by default There's a few options you have to get it though. None of them are that extensive It's fairly simple to just install and start manually So if you started a regular Sahara cluster with spark, you could just log into that cluster and install it yourself I have a couple of Repos that you could run One of the Sahara image elements lets you build the image with Zeppelin already on it And the Sahara one is a slightly tweaked version of Sahara or the spark plug-in that will start the Zeppelin process for you Not necessary if you wanted to just do it yourself though And also there is a community contributed plug-in for Ambari that will also Start up Zeppelin for you So with that we go back to Mike for some more visualization All right. Thanks, Chad. Okay, so we've seen what Zeppelin can do. What did we do? We rolled our own We built a small flask server that as I mentioned before was getting hit by spark telling it that there was updated log information in the MongoDB We used D3.js to do the visualizations and we also had some non-visual receivers So learning about information that was coming in from the spark server But not necessarily visualizing it if you wanted to just hear that okay some errors had arrived in your system Maybe it's time to send an email to an operator to let them know, you know Check out what's going on and then beyond that you could also script actions that come in off of this So you don't just need to visualize it You could actually create your own scripting out of this So maybe if you knew that a certain type of log message was going to come through and you wanted something to happen Based off that you could set your spark app up and have it send to a to a scripted action that might do more things It might tell Nova to shut down or it might you know might lock the system down. Who knows This way this is just a small example of one of the applications we wrote in this one What's happening is all the logs the logs from two services are being are being collected here The top graph shows you a total count of all the logs coming in and the bottom count shows you a graph based on two services the the orange colored line is Sahara's logs so most of the time Sahara is pretty quiet. It doesn't do much It runs a periodic task and you see a little blip there the top one is neutron Which is kind of noisy sometimes so neutrons kicking out lots of information for us And then you see around the the 50 second mark is where we started to do an operation with Sahara We started to tell it to register an image and what you can see with the red line on the top is an error has arrived And in the bottom pane here, you can see that we've clicked on that and now the error has shown up and you've seen it So this is one example. I'm gonna Hopefully the demo gods will be nice to me and I'm gonna show you another example so What I'm gonna do here is I'm gonna this is all gonna be run on my local machine So I'm not using DevStack. I'm not using an external cloud service I'm just gonna run it here for the purposes of this demonstration What I'm gonna do is I'm gonna start a keystone server on my machine and I'm gonna visualize Login attempts that are happening in real time. So what's gonna happen is I'm gonna start some services here and The first thing I'll do is I'll start up The flask application that will do the visualization and I'm gonna start up This application that will feed log information from Keystone to that And I'm gonna start the spark application Now at this point, I'm gonna have to drop out a presentation mode here All right. All I have at this point is just a graph. Nothing's happening. You can see that If I if I go back to here Keystone's not creating any logs right now And hence we don't see anything happening out here. No logs are being generated But what I'm gonna do is I'm gonna start a process that will that will create login attempts And what it's gonna do is it's gonna look like someone's just logging in they're valid any normally It's successful and every once in a while someone will miss their password So once every 10 to 20 seconds, there'll be a failed login attempt All right So at this point we're starting to see some activity come through and what you're and what you're seeing on the the blue Line is valid logins and there we go. Someone has someone's forgotten their password at this point or something Someone's failed a login attempt So I'm gonna come here and and I'm gonna attempt to click on that And if I scroll down I can see the logs logs have populated here and I'll scroll through and I say okay Warning there's been a failed login attempt. Okay, you know, I look at these logs We scroll back up and it it doesn't look like it's that bad. There's there's been very little activity here So probably someone forgot their password Now they're good. They log back in it was no big deal But what I'm gonna do now is I'm gonna simulate Someone trying to brute force a password. So what you're gonna what I'm gonna do is I'm gonna have it create 10 20 30 40 bad login attempts per second over several seconds and we'll see if we can visualize it Okay, so now we're seeing some activity. We're seeing a bunch of invalid login attempts and what's happening is First of all keystone's gotten really noisy because there's a bunch of activity happening But we're also seeing that okay What what the heck is all this and why have all these why have all these bad login attempts started to happen? So if I if I click on one of these And and I can work this like this. Okay, so You can see that all right. We're looking at logs and they're being sorted and We can see that okay. There's something happening, you know in real time these things are these things are being interleaved We were seeing bad login attempts and if I needed to I could come down and click on one of these points And this will be very challenging Based on this, but there we go and now you're seeing just all the all the errors Collated together based on time. So if we needed to look and see, you know, all right What just happened, you know, we could go back through these logs and look at it that way and So this is one example of how you could use this to to work on the logs that are coming through spark in this case All I did was separate all the login attempts and just look for look for the keywords of warning and You know, you've made a request that requires authentication So it was a very simple approach to just picking out those log lines and informing that something has happened And we you know, we can hit another one if we want to just for fun All right, so where does that leave us? If I can get back into presentation mode, all right So having said all that and shown you the demo what what did we learn from this? Pi spark works really well It was pretty enjoyable to write the application in pi spark because it was Python It was very easy to interact with all the pieces of OpenStack We could use OpenStack clients as long as we had them loaded on the spark nodes pi spark could access them The OpenStack Python clients I you know Arigato goes I must to all the OpenStackers who work on those things because they all worked beautifully It was very easy to use them and then manipulating the log data was also easy using the Oslo logging engine It was very easy to separate the the log lines It was easy to break them up and feed them back through to our database and normalize them in spark That all worked great things that could have worked better image creation is still a really big issue and Whether we're creating images for Sahara or we're creating images for Trove We still have to package these images and then deploy them to the cluster and we ran into a lot of difficulty In some cases getting the images to build properly I Don't want to name names, so I'm not going to so what next This was all pretty custom built and we injected a lot of the the routing and the pathways by hand We could increase the integration between these components And when I say creating OpenStack applications What I mean is that if we had written this at a pipe as a Python application that interacted separately with Sahara and Trove and all the different pieces we could have had it pull IDs for the different components and put them all together Automatically rather than rather than us having to write the individual pieces and plug all the endpoints in this could have all been done from an Application that sat at a higher layer than even Sahara and would have interacted with the cloud in the same way And it could have even injected values into our spark application, so With all that being said Does it scale We didn't get to the point where we could test the card a large enough level that we could say alright Here's 10,000 logs a second. Can it handle it? I'm pretty sure a spark can handle it Especially if we scaled out the spark cluster, but it's difficult to tell if that would have happened purely with the solution that we were using So if you're thinking about building something like this and you wanted to grow it out to scale You might look at some of these other technologies Kafka is a cue mechanism that you could use in places a car Fluent D and log stash are ways to aggregate logs and You know transform them and do work on them in flux DB is a way to store log information similar in a similar manner to log stash Kibana would have been an option to Visualize this had we been using elastic search and then Grafana would have been another way to visualize this if we didn't want to Read our own custom, you know D3 JS application so I Think we've come to the end of the road Any questions? All right, well Thank you all very much Go visit the Red Hat booth go visit the Maranthus booth


What's going on everyone? Sam here with late sports guards do ripping two boxes of twenty two twenty three Penny NBA hoops for John H. Good luck, John. See what we got for you, buddy There's a Benedict Matherian the Pacers Walker Kessler rookie Keegan Murray Steph Curry. There's a Desmond Bain wrote to the finals in 2022 for the Grizzlies Jalen Williams Joc Dyson Daniels Check home green rookie Al Horford purple. Oh, it was a Jabari Smith You're in the pool and nice Bryce McGowan's the 299 always good to pull a numbered rookie out of this That'll work. Congrats, John. Peyton Watson. Why prime time James Harden. Caleb Houston It's gonna be your auto nice rookie auto their hot signatures to 25. There you go. It's numbered all It's not first numbered auto. I've seen out of hoops actually congrats on that John. We'll take that and And Shreya hypnotized home green rookie Williams. There's Christian Braun look at dodge. That's pretty cool. Actually, I'm not I haven't seen any articles about it now That sounds pretty awesome though. That is To 2022 road to the finals DeMar DeRosin my girlfriend used to live like Probably less than 30 minutes away from Cape Canaveral. It's just pretty cool, too So we would see we saw we were in Tampa. We saw one of the Rockets launch recently. That was kind of cool I enjoyed that Donovan Mitchell on the jamtastic March on Bochamp. There's RJ Barrett nice one there to 75 Gravity parallel for the next kind of a color match there, too, which is cool. That's on that John Palos. Yeah, it's prize. You see get like one or two palos a box. So hopefully it's coming up soon here Jalen Williams. He's actually kind of a stud too for the thunder Trey young there's a palo. There he is So Chan and purple wise camp Patrick Baldwin Purple James Wiseman third year now. She's 30 or wise man. That's weird to say. I'm a 30 year Lamello. God She didn't I be there you go. I know you're looking for Ivy's there John. Congrats on that buddy. Joe LMB He can marry Rick Shouldn't church Jalen Durant There's Paul George and Rollins and a red back of James Wiseman. Nice one there, buddy Second box here. Good luck, John.  There we go. All right, John see what we got in box number two See a big auto man.  I'll see an Ivy auto for you buddy. That'd be awesome. I've been like my third March on Bochamp. We got next. Oh, there is base rookie and Pat Connitan green color match for the bucks to 99 Nice one there Christian Wood Purple take L'Arabia. He can marry base rookie for the Kings Paul George wrote to the finals of Is that Duncan Robinson on art signature? I don't think so actually smack. I don't believe I have yet. I Don't think so. No, there's Luke Canard red back For the Clippers Duel vertical I have knots. No, I've not pulled one of those purple old Karusha. That sounds pretty cool, though I do love a good dual auto. Those are fire Rontan Kyrie and like my third not playing nice Keegan Murray It's a 199 congrats on that. There we go. No, I've not smacked. That's kind of nasty though Zion Kobe on card duel. That's insane It's why the price of like some of those like Like late 2010s boxes have gone up so much because there's Kobe stuff in there Christian colloquial Is that Alvarado? Yeah, Jose Alvarado to 2022. That's pretty cool It's crappy defender legend You're good. You're good, man. No, no worries. No worries. You see I got you, buddy There's my alkyd Brannum nice one there for the spurs on the NBA rookie, Hank nice one there and Debuck I did I like breaking Star Wars chrome galaxy. It's fun, man. I enjoy it. Oh, it's main Jane There are some sneaky monsters in there. Oh Never leave the last box classic motto classic motto, of course chat home grid Absolutely nice purple curry. There you go That's on that there John Bogey. There's a Palo rookie again and Redback Malcolm Brogdon for the Celtics nice one there And Deiren Fox That'll do it for your break there John. Congrats on all the hits buddy. We'll get those right out to you, man


Welcome to ThinkTech Global on ThinkTech. I'm Jay Fidel. Today we're going to cover the post-election status of the TPP, the Trans-Pacific Free Trade Partnership. Our special guest is Russell Hanma, APEC master plan author and attache for the United States Trade Representative Office. The campaign this year, the presidential campaign, raised the question of whether we as a country should continue pursuing the Trans-Pacific Partnership. That's a free trade agreement within the Asia Pacific region. Donald Trump, President-elect, has said that he contemplated issuing a notice of withdrawal from the TPP membership. Japan has recently adopted the TPP in the National Parliament and some countries have said that if the US does withdraw from TPP, they will pursue other agreements instead, such as the trade agreement that China is pursuing under the ReCEP. That's the Regional Comprehensive Economic Partnership, which actually competes with the US, since the US is not a member of ReCEP and China is not a member of TPP. So dropping out of TPP could have enormous consequences on trade between the United States and our Asian trade partners. Today we're going to explore these events and possibilities with our guests Russell Hanma and examine the consequences, the ramifications of what Donald Trump has in mind. With due regard to his appointment just today of Rex Tillerson as Secretary of State, the stakes are high. We'll also take a look at Trump's replacement, potential replacement of Michael Forma as US Trade Representative and who he might appoint next, since that will tell us a lot about which direction Trump will pursue in office in January. Now let's see what Russell has to say. Welcome to the show, Russell Hanma. Thank you, Jay, for inviting me again. I remember when I attended the show back in March 22nd of this year regarding the Hawai'i's place in the Asia-Pacific region, I did speak about my APEC master plan back then and the consequences with the TPP, the Trans-Pacific Partnership Trade Agreement might have an impact to it. I remember I mentioned back then that because of this presidential election between Hillary Clinton and Donald Trump, my sense of direction of the future prospect of our trade agreements that's going to affect us in Hawai'i tremendously as well, the Asia-Pacific trade partners that we are engaging right now. So I think what the ramification what Mr. President-elect Donald Trump said back in when he was elected in November's election that he said he's willing to withdraw from the membership of the 12 members of the TPP. And that has really had a big impact in Council's questions with our trade partners and our trade negotiator because as you know, Jay, we've been negotiating this trade TPP for the past seven, eight years already. And matter of fact, in Hawaii, we have hosted it two times already, one in Big Island and one in Maui, and got the TPP members to come here for the negotiation meetings. We're going to lead the lead proponents of this agreement, haven't we? Yes, Jay. I think we show that the Hawai'i is capable of hosting these kind of trade delegates in the trade agreement because we want to make Hawai'i the place or hopefully if we can ratify the TPP and make Hawai'i the headquarters for the Asia-Pacific region and have a Secretary General in Hawaii to oversee the TPP trade agreement. So it has an effect on Hawai'i directly? Yes, I think there's roughly the study that we did. I think there's about roughly 400 companies in Hawaii who's going to be, who are exporters, who can export their services as well with their products. That's the perfect thing for Hawai'i. Exactly, we have our you know, many hooded water people, we have our coffee, chocolate, some of these, some of the goods and services that we provide here. We can even give a lot of these engineering consultant tourism as well. Hotel developers can go to help these countries to how to do a successful development and with international partners. Let me unpack some of this with you. So what is a, what is the Trans-Pacific Partnership? I mean, how does a free trade agreement work? How does this one work? I think you got to really look into the export and import and what the TPP does is they're more concerned about taking off the trade tariffs, like the U.S. Customs. Like there's a lot of non-tariff and tariff items and there's roughly about 18,000 items, like you know, from watches, clothes, clothing, automobile parts, medical supplies. So you look at what the U.S. Customs has in terms of liquidation when they justify the commodities and you get, they have a tremendous amount of database with 18,000 items in there and they, they, doesn't they, which country is coming from what commodities and what products are being imported from that countries and they have a, they have to set that database. And how, how serious is a tariff? How, what percentage or amount does the importer have to pay on the goods that are imported? Okay, you know, right now in terms of like a bilateral agreement, like country to country, I can set a good example, like Japan and United States. For example, automobiles, I know that we want to protect our, our three General Motors Chrysler as well for with our three Detroit, you know, big three over there. So what they're trying to put is tariffs, like an automobile parts, like pickup trucks or like 25% tariffs. For example, like Toyota, like we have a Tacoma trucks, they're charging them 25% import taxes. They want to make the fair market value so they don't want them to compete with the American pickup truck. So this is a, this would be Trump's way of putting American manufacturers at an advantage against Asian importers. Yes, I think he made a statement like, uh, colonizing China, like, uh, because I, you know, they're worried about the dumping law, currency modification and all that. So he was saying that they want to put a tariff of 45%. So none of those, anything that we bring in from China, made in China, uh, products, so they'll have 45% more compared to their invoice value. Yeah. And that's going to have like, uh, so what it does is it makes the American products more competable with the China's products. For example, you, you're selling a product for $2 on this, uh, item from China. For example, it could be slippers or, uh, any kind of plastic materials that brought in. And, and you put a 45% tariffs on that one. So that would be like, instead of purchasing it for $2, you're going to pay for like $2.90 cents. Yeah. And, uh, and that way the American similar product can, you know, when you get to that from wholesale, from X factory price to wholesale to retail, there's a markup value. So what it does is it sets the retail price equivalent. So that's part of the protectionist movement. So let me get the sequence. Under the TPP, what all of these tariffs would be for, for given, they would be deleted, or would they simply be reduced? I think most of it goes by case by case. I think, uh, uh, but within the TPP with the 12 countries, they kind of agree like most of like 95% of the items that's going to be brought in is going to be free trade, but they're, but they're going to have a phasing out period. Like for example, agriculture is like fruits and beef and poultry and stuff. They don't want to do it right away. They're going to have a phasing out period. I think they're going to, in terms of other, uh, commodity like automobile, automobile parts, it's not going to happen right away. It's going to have a phasing out period. What period of time is it? I would say within about, uh, five years, they want to see if they can. So it doesn't do violence to anything. Exactly. Exactly. Now I guess he's interested in this because he wants to, uh, make America great again in terms of manufacturing and all that. So he wants to try to discourage, uh, importing goods from Asia, uh, and he wants to, um, you know, therefore, um, make American goods more competitive in the American market. But, um, I guess the question I put to you is, will that happen? Do you have any thought about whether that will in fact happen if we drop out of TPP? I think that it's not going to happen overnight. I think what, uh, Donald, president like Donald Trump is trying to do is like, he wants to persuade, instead of having his multilateral kind of trading with a lot of members, like 12 members, or China has this RCEP with, uh, 16 members with 10 members, the ASEAN plus six members. And we even have our trade agreement with the European Union on the T-TEP, which stands for Transatlantic Investment, uh, Partnership. Uh, so that's going to have effect on us as well. But I think in terms of importing in, if you look at the history of a trade, you know, ever since the 18th or 19th century, we had trade with Europe, trade with Asia as well. And if you look at all the investors that we have in United States, we've got Germany, France, Japan, South Korea, all the European countries, they've been investing in the United States for all these years. So, you know, they already start up factories here. They have their subsidiary companies. They're doing, you know, they're part of our economy already. And this goes hand in glove with their investment. Exactly. And if they want these countries to come and invest again, they can set up the factories here, uh, retool their, uh, equipment and create, uh, more additional jobs. And I think you're, you're, you're going to close that window opportunity if we don't, uh, pursue the TPP. One other point before we take our break here is that if the United States, if, if there is no TPP, we're not a member of the TPP, all kinds of things will flow out of that. But one of them that comes to mind, we can talk about others too, um, is that, that doesn't mean that, um, we will have the advantage in selling overseas, because if I add a 25% tariff on cars or a 45% or whatever you want, you know, on slippers, that means that China and other countries, um, will respond accordingly. And they will add a 25% or a 45% tariff on stuff we'd like to sell in China. And so it's tit for tat, it's reciprocal, isn't it? Isn't that what happens? I think, yeah, I think there would be ways of retaliation, maybe if we do give it like a 45% tariff to China. I remember like in the 1980s when we had a trade war, kind of similar with Japan. I remember when President Ronald Reagan was the president back then and, uh, Yoshiharu Nakasone was a prime minister. And we had so much trade, uh, issues with the U.S. and Japan back then. Even it affected Hawaii without his foreign investment from Japan was buying up golf course, hotel, resort developments. That was back in the late 80s and we learned quite a bit from that and they became our, uh, you know, best partners and, uh, and their allies and, you know, in those kind of aspects. So those were good investments, I think. And I hope that maybe in the future we can have a good relation with China and hopefully we can iron out the differences. And part of a good relationship is to go easy on tariffs. Part of a good relationship is to have free trade agreements, yeah. Yeah, I think so. I think China was showing an example that they want to be a member of a TPP. And all the other countries in the APEC, I know that even India, they're not even a member of APEC. And they're, they're showing us they want to be a member of APEC and TPP. Let's take a short break, Russell Hanwa. Um, and he is the APEC master plan author and deals with Trans-Pacific Partnership. We'll be right back for more about that subject. Thank you, Hawaii. Asia in reveal. I am Johnson Choi, the host. I'm looking forward to see you next month, December 15th, Thursday, 11 o'clock. Right here again. Hi, this is Jane Sugimura. I'm the co-host for Kondo Insider. And we're on Think Tech Hawaii every Thursday at 3 o'clock. And we're here to talk about condominium living and issues that affect condominium residents and owners. And I hope you'll join us every week on Thursday. Aloha. Aloha. I am Reg Baker and I am the host of Business in Hawaii with Reg Baker. We broadcast live every Thursday from 2 to 2 30 in the Think Tech studios in downtown Alululu. We highlight successful stories about businesses and individuals and learn their secrets to success. I hope you can join us on our next show on Thursday at 2 o'clock. Until then, Aloha. We're back. We're live. I'm here with Russell Hanma. He's the APAC master plan author and he knows a lot about the Trans-Pacific Partnership in that capacity. And we want to find out more about how this does work or would work. So right now the U.S. does have some free trade agreements out there, don't we? Don't we have some? Oh, yeah. Actually, you know, we're kind of generous because we have this trade agreements with this developing nation, the Third World countries. And you know, we're giving them a free trade agreement and a status in that way. Yeah, sure. And we help them out. It's a way of giving them aid, I think. Exactly. To encourage their development. Yeah. Even we have one with Africa. Nobody mentions, talks about, they might be the next region that's going to grow after India. And it's called American Growth Opportunity Act, which is AGOA. So I know there's a lot of trade agreements that we've been initiating, but I think, you know, President like Donald Trump needs to understand what these trade agreements really stand for and what we went through to ratify it or, you know, all the past presidents went through as well. It's not easy because people do oppose on the basis that you're competing or making it more difficult for them to manufacture in your country. Exactly. And I think the world is changing so fast and even if Third World developing nations are coming up to the same par now and, you know, with the internet, with the communication and technology enhancement that everybody's got the same amount of information. So they're really, they all want to be, you know, they want to have quality of life in their country and do well and stuff. Sure. But why, why don't we just go to the bottom line and make a, make the world a free trade agreement? Why limited to Asia and Europe and Africa? Why don't we just say, no more tariff. We love you all. We want everybody happy together. So we're going to have a free trade agreement that covers every single country on the planet. A, why not this happen? And B, is this ultimately where we're going? I think that's like a utopia kind of thing. It doesn't happen overnight. We've got people like President like Donald Trump who's, who wants to protect the United States industry as a nationalism kind of concept. So, you know, we have a leading agency that sits on top is called WTO, World Trade Organization. And they look at the, they're in Brussels in Europe, the headquarters and they look at the overall of the trade policies.  And a good example, we had one in the past about the solar panels from China. And there was a lot of concern because they're selling it below the fair market value in terms of anti-dumping law. So kind of a duty act. So they had a grounds to make that argument under WTO. And how do we do on that? I think solar homes or, you know, some of the other companies in the United States that looked into it. And they, you know, found out that it was justified that there were dumping law regulations. That mechanism with the WTO, were we able to stop China from dumping? Oh, yeah, definitely. They want to be a member of the WTO. So they want to make sure that they're in a law-abiding country. So the world is looking at them now. So ever since they were part of the, they became a member of International Monetary Fund. And so, you know, they're part of the banking system. Trying to be respectable. Right. So they can actually do the currency manipulation. So then I want to, so we have this continuum where ultimately, hopefully someday, I know it's not going to be in our lifetime, but someday in the world we'll reach the utopia of free trade everywhere. But we're not there yet. Yeah, I don't think so.  There's all kinds of special interests, political and diplomatic considerations and all that. But right now, so we have one with Europe. We have one with Africa or about to be, I guess. We'd like to do this one with Trans-Pacific Partnership, except for Donald Trump. And we have a competing one. Just talking about the four corners of our conversation here. But China's trying to set up for its interests. What's the advantage of doing with one neighborhood and not another? Why 12 members of TPP? Why not 15? Why not 18? Why not 36? How do you make that decision? Is it a diplomatic issue? Is it a strategic issue? Is it a military issue? Why do we limit it to 12? Yeah, that's a good question, Jay, because, you know, when I proposed my, when I drafted my APEC master plan, I drafted in a fashion where I wanted to include all 21 countries or APEC. Yeah, it should be part of the Bogar doctrines on 1994 Bogar. It states that by year 2020, all APEC countries throughout the Asia Pacific region should have a free trade area in that region. And that's why we're pursuing this TPP to meet the 2020. And as well, China is pushing for RCEP, which is trying to meet the requirement of the Bogar doctrine. So right now, what is the Bogar doctrine? Bogar doctrine is when it happened in 1994 in Bogar, Indonesia, there was an APEC conference. And that's when the world leaders of the APEC got together and say, all right, we're going to have this trade agreement. And by year 2020, we're going to have all members to be part of the free trade agreement on developed countries and undeveloped countries. Open-minded. Yeah, exactly. So we still got, I guess, three more years to pursue this. So hopefully, if we're going to meet the requirement. And just recently, this past about a week ago, there was an APEC conference in Peru. And when the trade ministers and the leaders got together and they said that we don't want to have protectionist movement, we want to have a free trade. We want to continue pursuing the free trade area for Asia-Pacific region by year 2020. And hopefully, the members, and the dialogue, it came out that they wanted all members of APEC to be part of the TPP. And so that's interesting. Yeah, yeah. I mean, hopefully it'll get there. Let's take a moment to talk about APEC and why APEC wants to have a master plan, what it is for APEC to have a master plan, how influential, how important is that, and your role as a trade, working for the trade representative, U.S. trade representative in developing this plan. Can we put that together so we understand? Yeah, I think overall, because what APEC does is it's a pretty much an independent agency. Alternate leaders. Asia-Pacific economic cooperation. And the headquarters is located in Singapore right now. And they just changed the leadership. I believe the Secretary General is a gentleman from New Zealand, one of the trade ministers. We consider him as ambassador. He's a senior official there. So he's supposed to oversee that and hopefully see if we can come up with some kind of regional trade agreements. But APEC is so diversified. It's 21 countries. 21 countries. And you're actually contracted with APEC? No, I'm not contracted. I did the contract with the U.S. trade representative.  So I'm just, I was a trade delegate on this, when they had the negotiation in 19th round in Burnel, Darsamo. And I did a presentation on my APEC master plan as a trade delegate from the USTR's office. So that was one of my involvement. It was a long time, Russell. Yes, I've been, so I'm pretty much an advocate. And I like to make it work, but I want to make sure that United States and Hawaii benefits from this trade agreement. And we're such in the middle of the Pacific. We're an island here. And as you know, we are the gateway of Asia already. And we're the hub of Asia Pacific. We have to hold on to that. Exactly, exactly. And I think our future for our, you know, our next generation of kekis coming up well, as well, if we can maintain our economic growth, we need to get involved with the international trade and some of the issues. We have to be on the route. We have to be connected between the U.S. and Asia. And, you know, not only in that, but I think it helps from tourism as well. Even some of these investments, developments, real estate as well. You know, if you go from a military aspect with your U.S. Pacific command here during the RIMPAC exercise, we got all the APEC countries through the minister of defense to precipitate in Hawaii to bring peace within the region. So what do you say to the manufacturer in the Midwest who's, you know, Tita Tottering because he's in competition with China or, well, some member of that 12. And they're, you know, selling cheaper in the U.S. What do you say to him, you know, on the question of how, what he can do to survive? And what do you say to him to sell TPP to him? How can you make him happy? What would you say? Yeah, I would say, you know, TPP goes both ways, not only for exporting, but it goes for importing too. If that gentleman on the other side of the fence wants to do business with the United States, not only importing American goods there, we want them to invest in the U.S. soil as well. And hopefully maybe they can set up factories or they can renovate new buildings or start up new, you know, shopping malls or development or real estate development. You know, in terms of investment, and we've seen that happen with Japan, South Korea as well and Taiwan since the 1980s and 90s that we had that trade war with Japan in the early late 80s and 90s and Japan said, hey, man, we're going to start investing in the U.S. soil. And they start building factories here with the automobile industry, electronics firms are all over, you know, and so there are often factories in the United States. It strikes me that we could do TPP, but we would have to step up to new technology because our labor expenses are likely to be higher against some countries in the 12 or in Asia. But if we got really smart about the technology and rebuilt our factories and became the best in the world in terms of efficiency of factory production, then we'd have no problem being part of TPP. And we do swell that way, wouldn't we? Yeah, exactly. But you've got to look at two ways, Jay, if you're going to have efficient manufacturing production in the factory automation, because instead of using manual labor, you've got to use robotics or machinery to do the continuous production, flexible machine system kind of operation. So that might take less jobs in terms of white collar, I mean, the blue collar job industry with the union workers. So that's another concern too, you know, if you want to bring up a high tech, but with Donald Trump's elected, he's got to realize the working force with the labor. You know, I don't mind seeing some of these industrial products like we have our, you know, we have a stronghold for industrial like washing machine, dryers, some of these kitchen kind of appliances. I think we're leaders if we can manufacture that in the US, and even in our air condition units, anything that's heavy industry products. So to conclude, Donald Trump has made some real threats on TPP and it sounds like, you know, he's going to try to stop US membership in TPP. So there's camera one over there. Could you please talk to Donald Trump and see if you can change his mind. You have one minute to do this, okay? Okay, this message is to President-Elect Donald Trump. Congratulations for your presidency election. I know there's a lot of controversy right now going through with your election, your selection of your cabinet members, but and my concern is like what the ambassadors from the other countries are trying to persuade you as well. I know there's ambassadors, Maripuni from Singapore as well, ambassador from Australia, Joe Hockey has brought it up to your staff with your transition team that TPP just reconsider ratified in the Trans-Pacific Partnership. I think in the long term, if you realize consequence, we can make it work because TPP is going to be changing anyway. You know, if you set the terms and condition of the TPP, the members are going to eventually see they're going to meet periodically and as well, there's 12 members right now. We want to include other members in the APEC. So it's not static. Yeah, right. So there are no terms of the members. This is a dynamic arrangement that TPP is not static. I think it's an important point. I hope you listen to Russell. And hopefully, hopefully we can make the United States or Hawaii the headquarters for TPP. And that's going to bring us a tremendous amount of competitive edge to us in the U.S. creating jobs and made in USA products. Thank you, Russell. That's Russell Hanra. He's the APEC master plan author, part of the Office of the U.S. Trade Representative. We've been talking about TPP. We've been talking about Donald Trump's inclinations on it, and maybe he should reconsider what he has said so far. Thank you to our staff here, Zuri Bender, our senior production manager. Thank you to Robert McLean, our floor manager. This is Think Tech broadcasting every hour on the hour with things that you need to know about Hawaii and the world. Thanks for watching, and thank you, Russell. Thank you, Jay.


Today we are going to demonstrate secure firmware upgrade on STM32 microcontrollers and today's modern IoT world, most of the devices including smart thermostat, vacuum cleaners, refrigerators, IoT variables, health monitors, most of them are connected to internet. Anything connected to internet are valuable for the security breaches and we are going to provide the security on the STM32 microcontrollers including the features of like peak proprietary code read production and read production and write production and also memory production unit and firewall and today this microcontroller here will show you how you have to do secure firmware upgrade and when you press like this and it will ask you like I mean whether you wanted to get upgrade or like I mean whether you wanted to continue the firmware which is inside whether if you wanted to upgrade the image and you could do like encrypt the firmware and download and decrypt it and store in the flash we are providing the security on STM32 by validating the firmware after you download and also like I mean each reboot you are making sure that the firmware is valid and making sure that it's secure.


First of all I'd like to thank IMPACT and I'd like to thank the jury for this one more time. But standing here today one more time again I feel extremely humbled again and extremely inspired to see so many women achievers on this list along with me and I'm happy to see so many more people joining the list which really means that there are a whole lot of women out there who got a huge amount of potential who are out there waiting for this to happen to them. We need to really embrace the women workforce. They have a lot to bring to the table and they have a lot of different things to bring to the table because we think differently, we act differently, we behave differently, we are multitaskers, we are good people, managers and so I think the advertising and the media world really needs to embrace the women workforce and make of course it all eventually depends on merit and potential as well. Having said that I think we should embrace the women workforce in fact we've launched women at Viacom 18, it's an initiative called WAVE in which we are actually taking a lot of initiatives to make the women force that much more dominant.