Hey everybody, welcome back to Magic orthodoxy. My name is David and this is a magic review. Hey, today we're going to look at the real man's wallet from Steve Drawn and Murphy's Magic. What is it? It's a card wallet utility. Basically, it's a real leather wallet, stick it in your back pocket, go through a card trick and be a selected card, any card could be a signed card.  Alright, so a couple of questions we always ask about all of our magic reviews. The first of which is what is in the box? What is in the box? You're going to get this real leather wallet. Give you a little look inside there. You can see it as a billfold. It has little places for cards and then you have this chamber right here where you can have the card revealed from. You're also going to get this nine page photocopied instruction sheet. They're black and white pictures. It's all done kind of like on Xerox paper, not color. And like I said, nine pages, but it should give you all the information, the details that you need to perform and execute this trick. Alright, so is it what I thought? As with most magic wallets and loads, you kind of wonder, okay, what style of wallet, what style of load is it going to be? And so I think if you're familiar with card to wallet utilities, you kind of know some of the different ways to get a card into a wallet. This wallet certainly doesn't use anything new that you haven't seen before. So it wasn't that I was surprised like, oh, wow, there's this creative new way to get a card into a wallet. But I think it did exceed my expectations and that's nice. Alright, so how are the angles? For the angles, you will have to be able to palm off a card. Alright, so you just have to be comfortable with that. And I think you're going to find that with a lot of card to wallet effects. However, Steve Drawn and the people that wrote the instructions, they recommend a gambler's cop. So if you can do a gambler's cop or a palm, and typically you do this kind of like on the off beat, right, you're not doing that right in front of them like, hey, watch me do this. I think you'll have no trouble using the real man's wallet. Can it be inspected? For the most part, I think this would be inspected. I don't know that they would find the secret to it on just a casual examination.  I said for the most part, as its name implies, it looks like a real man's wallet. So what would there be to inspect? Is it well made? Absolutely. This is one of the best leather wallets I've seen on the market. It's the exact same size as my personal wallet that I carry. I got my wallet from Fossil, and if I hold it side by side with this wallet, length and width, it's exactly the same, pretty much even thickness. It's exactly the same. I think you load this up with your unique cards and items, and it's going to be passable every day, every man's wallet. How much practice does it require? Like I said, you've got to be able to do a gambler's cop or do a palm on the offbeat, but I think it's not going to be that impossible for people that are already familiar with the card to wallet routine. How much setup and reset is there? They recommend a type of load that gets you into a get ready, but it's not anything that's going to take time to do. It's very quick, very simple, very painless. I would recommend using your driver's license actually. If you're going to stick anything into this section right here, they recommend your business cards, but I would stick your driver's license right there. That's where people would put their driver's license in the window, and I think if you go to reveal their card and it's behind your driver's license in a snapped compartment, I think that's an even greater kicker and certainly possible with this utility. All right, so what are the positives? The positives are it's small. So many wallets out there are huge and big and encompassing and just like I would never carry that. It's a good size. It looks like the wallet that you carry with you everywhere you go. And like I said, there's a lot of really cool things that you could do with this. Definitely looks like a real man's wallet. Definitely well made, great utility. I think this is the one you're looking for. Negatives. All right, so like we say around here, there's negatives to every magic trick. These are just the negatives that I see.  You know, in this day and age and you're paying $30 for the wallet, I really would have liked to download or a video that would have kind of just explained it. Just want to put that out there to Murphy's. Maybe they want to just get a magician and you know, take them off the side, film something for like three or four minutes, and just kind of walk us through the routine in addition to the nine page letter. I just think people are more willing to watch a video or to have that option. I think there is something that does get lost in the written word and especially people that just aren't familiar or as familiar as learning from text. I think that would be a great plus. All right, so is it worth your money? It's 30 bucks. Hey, it's $30. That's almost what you would pay for a leather wallet anyway. So why not buy a magic leather wallet? Next time you need to buy a wallet, buy a wallet that doesn't matter trick. You know, always have a trick ready to go. If someone hands you a deck of cards, it'll work with any deck of cards.  I think $30 is well in the pocket, well in the wheelhouse, definitely worth your money. Pick this one up. This is the one you're looking for.  This is the one. This is the one you want. All right, so who would like this? I think anybody who's been looking for the perfect card to wallet, this is it. This is the one you want. Okay. All right, so this is the Real Man's Wallet from Steve Drawn. And if you'd like to purchase your own, you can find it one of your favorite Murphy's Magic suppliers. Thanks. Bye.


Live from Santa Clara, California, extracting the signal from the noise. It's theCUBE, covering Nextwork 2015, brought to you by Juniper Networks. Now your hosts, John Furrier and Stu Miniman. Okay, welcome back everyone. We are here live in Silicon Valley for a special presentation of theCUBE here at Juniper Network's Customer Summit. This is theCUBE, our flagship program. When we go out to the events and extract the signal from the noise, I'm John Furrier, the founder of SiliconANGLE. Stu Miniman, lead analyst on infrastructure and cloud at wikibond.com. Our next guest is Ron Windwards, with server-central director of network engineering. Welcome to theCUBE. Thank you guys. Great to be here. Before we came on, we were like, I can talk networking, let's talk networking, because Juniper is in the middle of a major transformation. Their horizontal scalable platform, really powering a lot of innovations.  But all the action right now is happening in that middleware kind of cloud data center market where I got an infrastructure that I need to architect for software-driven, server-centric architecture. What's your take on the landscape with Juniper and just the overall data center architectures today? So in networking and service provider, the old antiquated way was you run a cable somewhere if I have to reach somewhere that is across a data center, maybe across a couple data centers or a metro. I've got to get cross-connects. I've got to rely on people. I have to deploy. I've got to maybe deploy expensive equipment or maybe I have to reposition this into my cabinet. And that's, it takes a long time. It's expensive. It's not always viable. So now we're looking to the network to maybe be able to provision in infrastructure across the backbone, maybe not itself, but the ability to do that and reach anywhere in the network with the control of the operator, being able to build the network, click provision services across a metro instead of having to redeploy equipment everywhere. Talk about the dynamic over the past five years. Obviously the cloud movement you've seen with DevOps, software development now, since the iPhone came out, we just talked about earlier about mobility and internet things, really exploding with data, new types of applications, more service area for security. Talk about what's changed in the past five years because five years ago it was relatively stable and five years earlier, 10 years ago today from today, provision, rack and stack, things are clean, things are good, we've got our apps running, got off branch offices, now it's like, whoa. Yeah, so for a long time networking was very stale. It's very exciting again. And I think that that comes from the customer base. I think that it comes from the innovation in the service provider space and the providers of the hardware, the manufacturers. So the applications you talked about, apps have exploded and those people are server people who are used to, I'm able to see real-time analytics, I'm able to see real-time data and make insightful decisions based on what I can see out of the application. That hasn't really existed until recently in the network and we're starting to see that now. So the customers are driving the need for, I want to see metrics that I can use to make insightful decisions or maybe identify a problem more quickly and react more quickly or maybe use automation to react before I can even have a human look at something to mitigate a problem, let's say, or an attack, let's say. So the previous five years coming up to now have been kind of a feeling out where this stuff is gonna go and I think as an industry, maybe in the past year to 18 months, we really have started to grab hold of where we're going to go and now we're starting to see, now we're really starting to figure out what's going to be next. So, Ron, I'm wondering if you can sketch out for us a little bit about what your network looks like, how many people you have and what's kind of day-to-day? What are the activities that kind of keep you busy and where's the innovation happening? So, Server Central is a data center and co-location company. That means a lot of things to a lot of people but primarily it means we're a place to put your servers, your critical infrastructure, your critical data and a place to have it reliably out there on the internet on redundant power, redundant internet service and that's the start of it, right?  Server Central is a company that can really tailor to your needs for I can provide you with managed switch, I can provide you with managed firewall, maybe I'm providing you with transit and really complex BGP service or maybe I'm just a place for you with great power and great hands and you operate your own network. So, Server Central really does all of that. We are a partner of Juniper, we have been, we're a var of Juniper, we've been Juniper users for at least 10 years. We use Juniper MX as the core of our routing. Every customer who is a Colo customer connects with redundant connectivity to two diverse Juniper MXes. If you're a customer who has managed switch or managed firewall that's done on Juniper EX or QFX, firewall is done on SRX, so we really, if there's a need in the network and if customers are asking for it, we're going to look to Juniper to see if there's a solution that they can provide and we're a very strong partner and love the power of Juniper and Junos. Yeah, it's a really interesting space. So, you've got both the, you know, kind of bring your stuff and we can hold it or we can deliver you with service. Absolutely. In either way, it sounds like you're handling the network for them. I mean, one of the questions we have is, you know, some of these big network changes that are going on, getting the skill set to, you know, program it, automate it, you know, curious to hear what you're doing with SDN, but, you know, if I'm the average mid-range customer, it's like, that's probably a lot more heavy lifting than I'm willing to do. Yeah. And therefore, you know, I go to somebody that can handle it better. Exactly right. What are you seeing and what's the pitch to your customers? So, the pitch is, we are a provider that can provide you anything, you know, bespoke custom network solutions.  That's where we come in. The same people who build our network can build your network and, you know, it can be that.  My team can do that, you know, our company can do that. So, it really is, it is a very custom, you know, whatever you need, we can make something custom to fit the bill. Okay, and I'm curious, do you see customers starting on one side of the business and shifting to the other or shifting back? You know, what's that dynamic look like? Yeah, so, a lot of what we see, because we do offer managed switch and managed firewalls. So, customers who have stayed with us and grown with us for a long time, we maybe have deployed their initial infrastructure, but they learn more about networking as they go along, because these are very bright, talented people who are building great companies, right? And they're learning more about the network and learning, okay, well, maybe I can handle some of this myself. So, we're seeing people move to being able to, maybe they don't want to control the network, but maybe they want the analytics and the information that can help them decide the performance of their network and how to grow it, you know, going forward. We also do see metal, you know, traditional collocation, going to hybrid cloud, that's, you know, the past year and a half, two years, we've seen that. And so, people want that, I don't want to deploy metal servers anymore. I have my need for metal servers, I have my need for virtualization, and I need a way for you guys, server-central, to marry the two networks together. And, you know, that's been something that we're using, you know, specifically with some of the contrail and SDN stuff, too. Yeah, so, one of the big trends we see is customers are looking to access, you know, more in different types of data. So, some, what they own, some, they're tying into other cloud services, sometimes they're doing things in the public cloud. How does that tie into what they do? Yeah, yeah, so, again, it's back to that traditional infrastructure and collocation type model. But as you said, maybe I want to spin up 100 VMs somewhere in another cloud provider like an Amazon. We have capacity to those providers where we can provide them with a VPLS connection to our handoff to them and maybe get them dedicated service, maybe they just use our own existing connectivity to them, like maybe it's not a dedicated cross-connect, maybe you're just using our Amazon peering, for example, so, you know, to have the ability to get customers to wherever they need on our Juniper network, you know, it's Juniper that empowers, that enables that for us. So, about managed services, because this is a big deal now. You can manage cloud, I got managed infrastructure. The OPEX, CAPEX thing is happening, we know that. The economics are there, there's incentives. Cloud has economic deployment incentives. You guys are in that managed business, right? You help customers there. What's the big trend in that market? Is it just capacity by the clients to go there? Is it budget, timing? What's the need? What's holding the world back from having turnkey managed services? Almost like we have electricity today. Almost thinking like, soon it's going to be that way. Yeah. You're in the front lines. What are you seeing? So, we see people who maybe want to start out maybe with metal, or we see the other side of that. We see customers who want to start out with virtualization just to maybe put some of their applications or try some things here and there. And what we actually see is a combination of both. Eventually, people, there is a need for metal and there is a need for virtualization. So, customers will see grow into both sides of that and really create a fabric of both sides for customers. It's not that I'm used to exclusive metal and all this stuff's happening. But ultimately, they're kind of, I won't say groping, but they're engineering, they're iterating to ultimately a hybrid cloud market. Exactly. Which is not a product. You can't buy a hybrid cloud. You kind of engineer it. Yeah, sure. Yeah, so there is a challenge there when I have my metal infrastructure and I've got my cloud infrastructure and how do I connect the two together and make them on the same subnet and all that stuff that we rely on networking to stitch together and to make that ubiquitous access for everybody in the network. What about an open stack? What's your take on that? I just want to throw this. It's just popped in my head so it's thrown out there. Yeah, I think it's one of the technologies that is powerful in the market. We are not using it, but plenty of people are. Are customers asking for it or not really? You're not your wheelhouse? Not for us. We have a very specific type of deployment. It's not so much an open stack. There are open virtualization platforms that we'll use. There's some custom things. There's some VMware things, so there's, you know, it's... You guys have a lot of services. I want to just kind of take a step back.  What's the hottest stuff that you guys have? What's the big sellers? What are people buying? So, A, people come to Server Central because of our ability to provide resilient networking, high performance networking, knowledgeable staff. And then the things that they're buying are the big virtualization platforms. They're buying managed switch, managed router. The classic stuff. The classic stuff, but I think a lot of it is also for our ability as a service provider and our care for customers. So it's classic stuff, but everybody's network is not classic. Everybody's network is unique to themselves. I would agree. What are you guys known for? A better question. What are you guys known for? We're known for reliable networks. We're known for knowing what we know and doing a good job of what we do. And we're known for uptime, uptime and performance. It's interesting. You become what you're known for. Yeah, sure. And so that's your company's DNA. Yeah, absolutely. How about the cloud? How do you see the cloud in the future for you guys? I mean, taking that to the next level. What are some of the challenges you guys see and what opportunities, what gets you excited? Yeah, so it's that room in the data center, right? Room as in where am I going to put things in the data center? I have a cabinet of hypervisors and then I have another cabinet of hypervisors and then maybe I have to grow into another computer room. The cloud is that footprint, right? I've got to stitch all of that together. So the things that we're looking at with contrail, contrails in our labs, QFX is something that we're using to build this, stitch this network together. So where are we going with the cloud? It's going to be how do I provide this, how can I touch anything in these huge data centers and make it one footprint, one thing that is an overlay that is invisible to the customer? My final question is share with the folks out there that are watching that have a data center. All this pressure, they need a partner. What advice would you give them to take that first step? I'm almost like, do they jump into the deep end and like lash around for a while? Do they go into the low end of the pool and kind of put their floaty on, go out to the networks? I mean, it's all kinds of different approaches and how they attack the future. Yeah, I say, what would you say? Interview the people that you're talking to, do some research, look at companies like Server Central, look at companies who will allow you to talk to their engineers, the people that are going to build your network, let them know. Put their toe in the low end, they were jumping the deep end, what would you recommend? Kind of different depends, right? Yeah, I say it depends. I say, go on data center tours, see the facilities that you're going to go, be partnering with, and I don't think that it's not a viable thing for everybody to just jump in headfirst with all of their data center needs into a place. So, when you do do that, get to know the people behind the company, get to know the people who will support you. My final, final question, I just want to just jump to my head was, Jennifer Blatnick said earlier that the new kinds of innovators are revolutionaries. You see those guys all the time because you're, again, you're in the front lines. What's the mindset of that revolutionary in the customer base? The ones who really want to make the change happen, the ones that are pushing, hitting that brick wall hard, breaking it down. They hold you accountable. They're the people who look to you as a partner and not your service provider. They look to you as your partner and if they hold you accountable if they need you and they look at you as part of their staff. If somebody signs up for a managed service, they expect you to act as an employee just like any of their other employees. So, those people who are the innovators, the really fun, interesting customers, they're going to hold you accountable too and we expect that and that's part of what we do. They expect what they want. They want their hard charging. They want you to be hard charging but reliable. Yeah, absolutely. Ron Wynward, thanks so much for coming on here, sharing the insight on the networking stuff from the trenches, from out in the front lines of the networks, this is theCUBE, sharing the data here at the Juniper Customer Summit. We'll be right back with more after this short break.


Hello DevCon and hello IOT Village. My name is Alex J. Boran. I work as a security research director for P-Defender out of Romania. First of all, I would like to give a quick thanks to the team that made this presentation possible. They are Radu Alexander Bazaraba, Alex Lazar and Yvonne Melnychuk out of our office in Cluj. Now we're gonna dive right into it. This talk is going to be about some of the findings that we've found, I guess, in the past five years while doing vulnerability research in IOT. Now a few things about myself and about us. I think it's the third time that I've spoken at DevCon and IOT Village. I'm happy to say that I managed to speak at the last edition of DerbyCon, simply because it's such an awesome conference. We've been doing vulnerability research in IOT since 2016 and we found vulnerabilities with a very high severity, so let's say an eight plus in the CVSS score in more than 20 brands. We published 12 papers amounting to only a small fraction of our total findings and that is because we had very poor or no reply from vendors. So one of the biggest challenges that we faced in these five years was getting the vulnerabilities that we found fixed and since a lot of the things that we found had a very high severity, disclosing them without having them fixed may have resulted or could have resulted in mass exploitation. So we imagined that we found vulnerabilities that affected let's say 10 million devices and the vendor didn't reply. So one of the biggest challenges that we have is getting a hold of the vendors and getting them to reply and fix their stuff. Roughly four years ago, we started to focus a lot on IOT Cloud platforms and these are some of the topics of the IOT Cloud platforms are gonna be a hot topic in today's presentation. So far we've published about 12 papers. So the list is here on this slide, you're gonna get the slides after this, but I want to emphasize on the huge amount of papers that we didn't publish and that includes some well-known brands and mind you, the list that's on this slide is not the whole list that's in our internal repository back home. So I want to take this opportunity to emphasize once again how bad the response is for most of the vendors that build IOT. Most of them don't have security contacts and those that do have security contacts don't reply to those emails. We've even tried reaching out to them on Twitter, on LinkedIn, getting a hold of them by friends of friends. Do you know anybody that works at company X and that kind of thing. So again, I'm taking this opportunity to raise an alarm about vendors and the fact that they don't have security contacts and the implications that this has in the grand scheme of things in security worldwide I guess. So many unpatched vulnerabilities and that can still be exploited to this day. Some of the exploits that I'm going to outline into this presentation can still be used to this day in a large number of brands and vendors. So that being said, the first chapter and probably one of the coolest is reusable attack vectors and common vulnerabilities. Essentially these are vulnerabilities that affect more than one vendor. So stuff that can be used en masse and on many, many devices. One of the most common ones is hijacking the wifi password during the setup process. So essentially it's a technique that enables an attacker to steal a user's wifi password during the setup process of an IoT. So here's how it works. You get an IoT, you start configuring it. The first step usually is to connect to a small hotspot that the IoT makes. Let's call it awesome secure IoT wifi. And the mobile app tells you, okay, the IoT created this hotspot, connect to it. The next step is to then select your home wifi, enter the password of your home wifi and then there's a bunch of other 20, 30 steps about configuring additional stuff like setting a common name, maybe setting up alerts, notifications, different other types of settings. After that's finished, the mobile application starts to communicate with the device and usually crashes and you have to restart the whole process all over again. You restart it, you redo it and eventually you are successful in finishing the setting of the device. So this is pretty straightforward. I'm sure that all of you have done this at least once with some of your devices, right? So you set it up and then you start using it. Now what's happening in this last stage of the setup process is that this is the moment when everything that you've configured in the mobile app gets sent to the device. Usually this is packaged in either an XML or a JSON file that's being sent over to the device whilst your phone is still connected to that hotspot that's created by the IoT. Part of that config is the home wifi settings. So among the settings that are included in that file is your wifi SSID and your wifi password and other stuff like the friendly name calendar, email notifications. In some cases we've seen devices that ask the user for their Gmail username and password so they can send email notifications. It's bad, I know, but it's happening. So we've seen it in a number of devices. It's because they don't have an SMTP server so they want to use your Gmail account to send emails and yeah, I would advise against setting that up if you're ever faced with it because that means that you're gonna pass your Gmail credentials to the device and if the device gets hacked, they're gonna get your Gmail credentials as well. So a key thing to remember here is that that access point that the device creates in 99% of the cases is unencrypted without a password protection. It's an open wifi that anybody can connect to. And mind you, this does not mean that you as an attacker are gonna connect to the access point as we're gonna see later on, but the fact that it's open means that anybody can monitor the communications back and forth from that access point. So in a base case scenario, that config file that JSON or XML is encrypted. So even though you can see the communication, you're not gonna be able to see the contents of that config file in order to steal personal information. Mind you that this doesn't happen a lot. So this, as I'm saying, is a base case scenario because for most vendors, they're just simply sending it in plain text. But in some particular vendors, mostly the good ones with some reputation, they do encrypt it, there's a strong key and you're not able to extract the information because you need to have that key to decrypt the file. But that's not necessarily the end of the world from an attacker's standpoint as we're going to see later on. So what you're going to do is set up a wireless card, set it up on the same channel as the awesome secure IoT wifi. You don't need to join the network. You're gonna start seeing the traffic from that network just by having the antenna on the same channel, is it? Start Wireshark and profit. As a particular case, I promise that I will touch on the encryption part. We published two advisories. One was in August Connect, the August smart lock. I'm sure that you're familiar with it. And the other one in Ring Doorbell, both are fixed now. Mind you, these vendors, thankfully, had a very prompt response. They fixed the stuff quickly. They pushed the updates and everything is okay now. The vulnerability in August Connect was particularly cool. Essentially, they had that payload that's being sent from the mobile app to the device. They had it encrypted. However, the encryption key was hard-coded in the mobile app. And then, unfortunately, it was the same encryption key across all mobile apps. So if you collected the encryption key from one app, then you could use it to decrypt the payloads from all the other August Connects that are in the market. That's fixed now. Each setup process has their own unique encryption key, so that's okay now. Yeah, many people said that this is not necessarily the coolest attack vector because the setup process only happens once. So most companies rely, the security of the setup process relies on the fact that it's only happening once when you set it up. And then you don't need to do it again. So yeah, you need as an attacker to catch that window of vulnerability when the user sets up the device. However, as we're going to see, that can be mitigated. Yes, you guessed it. You just send the authentication packets to the device, you knock it off the network, and then you force the user into reconfiguring it. And if you're not familiar with what the authentication packets are, it's a standard in the Wi-Fi protocol that enables somebody, anybody, with an antenna close by, they don't need to be part of the network, they don't need to join the Wi-Fi network, none of that stuff. But anybody can knock out a device from a Wi-Fi network. You see all the MAC addresses connected to an access point, and then you just simply send the authentication packets to a MAC address, and that device with that MAC address is gonna be disconnected from the Wi-Fi network. And essentially what you're going to do is just keep doing that until the user believes that the device is broken, resets it to factory settings, and restarts the configuration process. So for that, we've prepared a small demo in the context of actually August lock. So here we see wires are being started. The first thing that we're doing is just seeing what's on the network. And here we see that our target is the beauty salon, Afrodite beauty salon. And we see the access points MAC address. And then we start to see what devices talk to that access point. And at some point we're going to see our target. And right now what we want to do is just knock out the target from the network. So we're gonna send the authentication packets to the target and kill it. So in parallel we're going to start another air dump to monitor what's happening in the Wi-Fi spectrum because at this point the user is gonna get pissed off and reconfigure the device. So we should see the access point of the IoT popping up. And there it is. August won't connect. So at this point the user has started the setup process. And now what we're doing is just we're raising an antenna on the channel. In this case it's channel five. And monitoring the traffic that's being done by that access point. So we're just watching what's happening between the IoT in the setup mode and the rest of the world. And now we're going to see the packets being exchanged from the mobile app to the IoT. We're just monitoring on HTTP because most of the stuff is happening over HTTP protocol, the smart devices, small web server that receives the configuration file. And in a few seconds, and there we have it. We just follow the TCP stream. And mind you, this is on hard mode so to speak because this package is gonna be encrypted. But as I said, we already had the decryption key because it was hard coded in a mobile app. And we're just gonna decrypt that configuration package. Packet. Actually package is not wrong. And there we have it. We have the Wi-Fi password that was being sent and the SSID. So you see that the configuration that was being sent to the IoT was this is the beauty salon for the IoT. And the password is, well the translation would be beauty without wrinkles of the Wi-Fi password. Obviously this is something that we set up for fun and games. Anyway, so this is how you hijack the password of a smart device by exploiting the setup process. Moving on, my personal favorite. Abusing cloud implementations to get RC. So what's happening with cloud implementations in IoT is this. This is the way a smart device communicates with the mobile app. You have the mobile app with you when you're roaming outside of your house, maybe in another country, whatever. And then you have the cloud that's really in the communication between the smart device and the mobile app. And there are a number of applications, protocols, systems that do that from MQTT to custom made web services that enable that. But in this particular case, what we want to exploit is some parsers that parse input from the mobile app. So what's happening is that the mobile app tells the device, open the door, close the door, change the setting, turn the lights on, turn the lights off, schedule vacuuming the house or stuff like that. Which you're doing remotely from the mobile app. And all of these can be treated as inputs by the parsers on the IoT. So the mobile app talks to the cloud, the smart device talks to the cloud and asks the cloud, do I have new configurations? Is my owner telling me to do anything? And then the cloud says, yeah, sure, here's the new config, here's the new commands that you're getting to the device. So now in this scenario, we have the mobile app outside of the house. We have the device behind NAT, so behind firewalls. It can be behind three routers, it doesn't matter because the device is gonna get input from the cloud. And you have that config file which is relayed by the cloud to the smart device. The thing that we want to exploit here are the local parsers that process that input from the mobile app. Because when you change a setting, all that thing is processed by code on the device. And since it's processed by lines of code, if that line of code is not properly written, it can lead to command injections, it can lead to buffer overflows. And a little known fact is that in IoT, buffer overflows lead to predictable remote code execution. Because ASLR is not enabled in the IoT space. The operating systems in IoT support ASLR. However, the binaries are compiled without positioning dependent executable support. So in 99%, we haven't met an IoT that made use of ASLR yet. And we've studied like a few dozens of them. So if you find a buffer overflow in the code, then that can lead to a predictable remote command execution. Case in point, the Erie Max Smart Power Outlet is still not patched. I mean, it is patched, but nobody applied the patch because the update process in the Erie Max Smart Power Outlet is very not, it's not user friendly at all. So up until today, if you know somebody that owns an Erie Max Smart Power Outlet, you're gonna be able to achieve a very predictable, very easy remote command execution on their devices from anywhere in the world. I would advise against it obviously because it wouldn't be ethical and I would advise you to tell them about it. The full details about this or on labs.bedefender.com, just search for the Erie Max paper and you're gonna have the full details on how to exploit this ability there. It's a command injection in one of the functions on the device. Guardzilla also had both buffer overflows and command injection in some of their devices. Unfortunately, the company and their cloud died during the pandemic. And as I was saying, the attack vector is that you relay your payload, whatever you want to use to exploit the processors, the code that processes the data on the device, you relay it through the cloud and wait.  Since just by opening Telnet, for example, on the device it's not gonna do you any good if you're outside of the network, what you're gonna want to tell the device to do is just pop up a connect back shell to a handler that you're gonna keep on a remote server. So a quick demo on this part as well. We're gonna use actually the Eddy Max Smart Power Outlet. This is a presentation that I've done in Beijing a while back. Here you can see we have a MetaSplit handler that, hang on. So we have a MetaSplit handler that it's for the MIPS architecture because the Power Outlet is the MIPS architecture. It's a remote server. It's one of my VPS servers. And our goal is to tell the Power Outlet to download the first staging script, execute that script. And that script is going to be to download our MetaSplit payload and connect back to us. So this is a more detailed presentation that I've done. I would advise that you get the paper on it. It's pretty cool. The reason for that, for example, is a trivia. The reason for that weird domain, uy.md is because this string would be the password that we're going to set. There's command injection. They're using MD5 on the password. So that's our command injection. But this cannot be larger than 32 characters. So this is why we had to do a staging exploitation. So the script is going to tell the device to connect, get this A script. And this A script is gonna be telling the device to do more things, like get the MetaSplit payload, execute it, and more things. Now we're monitoring the FTP server to see if the device is gonna get the script. You can see it got the A script, executed it. The A script told it to get the other executable and then we got a shell. Just for fun and games, we're gonna run a few commands. Where am I in root? We're gonna get the wifi password from the config. This is the IP address. You can see it's a local IP address behind that. So yeah, this is the time I got a root shell on a device behind two firewalls. This was actually in our office in Romania, in Cluj. And I got to connect the shell on my VPS in Canada on a stage in China. So pretty cool. So moving on, another common vulnerability is MQTT. So MQTT is something that I personally wasn't familiar with until we started researching IoTs. MQTT is this very quick notifications and configuration servicing that can relay notifications and quick configs between the mobile app and the IoT. And MQTT has sort of a tree structure in the sense that it's usually not password protected. There's very rarely authentication on it. Or if there is, you're gonna be in the same, let's say pool of users as the other users of that MQTT server. So let's see that it does require authentication, but you are a registered user of that service. You want to see the status and the notifications of your own device. Now, what happens is that MQTT suffers from something like a directory traversal, if you will, in which if you're not configuring it properly, you can register to see the status and the formation about your own devices, but you can also go a few tiers up a few levels higher and register to the root of the MQTT tree and start receiving notifications about all the devices registered in that MQTT server. And that's gonna expose a lot of sensitive information like for starters, I'm up, I'm down, but then you get stuff like the IDs that are being used for devices by the manufacturers. And that's a very valuable information because very often the one thing that you need to interact with the device from another user is just that device ID. It's made in a non-predictable fashion, usually 16-bit format, but with MQTT, you can get that information through the MQTT stream. And we're actually going to see, we're going to see an implementation or broken implementation of MQTT later on. But play with it, research it again. We have a few papers on MQTT as well and some manufacturers look it up on our website. Another very common one, and this, as you probably many of you know, it's not just in the IOT case, it's bad implementations of S3. So S3, the storage service from Amazon. The usually the way it works is that I'm the device, let's say I'm a security camera, I'm creating a recording and I'm uploading it to slash vendor, slash my device ID, slash date, slash timestamp dot MP4. And I'm relaying that information to my owner's mobile app and telling them, I have this recording at this path and the owner can access the recording because they know the path to that recording. And in a normal implementation of S3, if you want to list the other contents of the folder, you shouldn't be able to let alone, you shouldn't be able to go up one level and list the contents of the higher directory. And in no way, shape or form, should you be able to list recursively all the contents of the bucket of that vendor. It's mostly something like directory index in web servers. Well, bad implementations of S3, that actually led to a lot of websites getting hacked as well is when an attacker is simply able to recursively list all the contents of the vendor's bucket. If you want to Google, for example, database leaked S3 bucket, you're gonna get so many hits about so many websites that had their users' databases exposed to the internet through S3 buckets. So the new release that we're, I don't think it's published yet at the moment when I recording this, but it should be published by the moment Defconn is live. If not, I'm sorry, I apologize. But it's gonna be published soon, so just monitor our blog to see it. It's Victor Cam. So spoiler alert, this release is going to include information leak, unauthorized access to the video feeds and pre-authentication, remote code execution. So a few words, it's an average, decently priced baby monitor. It has cloud support like most IoT's have today. You can access it from anywhere. And it runs on three platforms, depending on the model, IP6360, Nui and Tuya. Now, mind you, most of these IoT platforms are great inventions, they're great. However, the security reviews of these are lacking. So there's not many people doing security reviews for these cloud platforms. There's not many people doing security reviews on the binaries and the firmware running on these devices and the way they communicate with the cloud. We've been doing it. There's a few others that have, but I encourage that more of you start exploring this part. These are low hanging fruits if you wanna publish a research paper, trust me. You're gonna find a trove of vulnerabilities, including in big brands. If you just research this part, the way the mobile app talks to the cloud, you're gonna find indirect object references, you're gonna find directory traversal, you're gonna find the buffer overflows, man injections, you name it. All you gotta do is just look at the way the mobile app talks to the cloud and the way that cloud relays the information to the device. So the first vulnerability is an unauthenticated MQTT information link. So the camera uses the MQTT protocol to announce the status and to receive a URL location where it's going to send the video stream. So essentially when the camera is on, it needs to know where to publish the video stream and the MQTT protocol is the one that relays that information to the camera. Now, what we've discovered is that the MQTT server is at this IP address, so eup2p01.govicture.com on port 1883. And it has no authentication, by the way. So all you gotta do is use an MQTT client and subscribe to slash device slash the init topic and you're going to see all the IDs of all the devices that come online. So just connect to that IP address, that domain, that host, subscribe to the slash device slash init and you're going to see all the IDs of all the devices that come online. And as I was saying, there's no authentication necessary. And this is how the almighty exploit looks like. You get the tool is called Mosquito that's underlined sub. You use the host as a parameter, the port as another parameter and the topic that you're going to subscribe to slash the device slash init. And instantly you're going to see notifications from the devices. And this is bad because as I was saying that device ID is a critical, critically private information in most IOT implementations. If you get a device ID, in most cases you're not gonna need anything else to be able to interact with it. Even though obviously it doesn't belong to you, it belongs to somebody else. So the second thing that we wanted to do is get the video feed. Obviously without having to authenticate. So get a video feed of our camera as from an attacker standpoint. So we noticed that the way it works is that through the MQTT server the camera is told where to upload its video feed. And then the MQTT server throws the mobile app. Hey, the video feed is at this IP address. So we figured why don't we try to see what functions are exposed by the MQTT server besides, you know, I'm on, I'm off and these kind of things. And we discovered another function called command where we could use that to configure new parameters and new things on the camera. And among those we could tell the camera to upload the video feed at a different location. So as I was saying, the MQTT server has no authentication. All you need is a device ID which you can get from that previous step. You just register to slash device slash in it. And you're going to see all the device IDs registered in that MQTT stream. And then you just pick a device ID, send a command to it. So you tell MQTT to tell slash device slash ID slash CMD and tell it to upload the video feed rather than its default location. Just upload it to your own, to your own server. And on your own server you just have to register a small listener and you're going to start receiving the video feed. So as a demo, so in the first step we started to, we built our own web server, our own RSTP server. In this step, we're telling the MQTT server to tell our device ID so to our camera that the new URL for the RSTP server is our rogue, our attacker server. And again, no authentication necessary. All you need to do is your victim's device ID. So the command was sent and now we're waiting and we start receiving some traffic. Now we're going to pipe this so we'll also be able to see it and we're just going to open it with VLC. And there we have it. So this is the video feed uploaded to our own rogue attacker web server by the victim device that was tricked into using another location for the video feed. Moving on, no good deathcom presentation is good enough without a remote command execution. And much like with all the other stuff that we've hacked, this remote command execution works from anywhere in the world on all the devices registered by this vendor. And when we last look, I think there were about 10 million downloads of their mobile application on Google Play. So there's quite a lot of potential victims here. Now, the function that processes the URL command that's received through the MQTT is vulnerable to a stack-based buffer overflow. And the almighty exploit looks like this. So we're piping everything into one small file that we're going to send. This is the entire exploit. So not very complicated. And the command that we wanted to execute is just send us a connect back shell to our attacker server. So we bought it to execute netcat, our IP address on port 4444 with a shell. We're going to relay this with MQTT to the device. So again, this all is happening from different parts of the world. So we're turning MQTT server to tell the device to execute that command. And quite quickly, we received a connect back shell from the device. And there we have it. We have a shell for remote command execution. We have a full access, full foothold into the owner of that device's house. We can then obviously pivot. We can do anything that we want in that house. You know, we can start sniffing connections. We can start looking at other devices inside a house. Unfortunately, many, in most cases actually, the home network is not treated as hostile. And the home network is treated as secure. And as such, many of the devices on people's home networks are not properly secured. And many people keep default or no passwords on their storage servers, on their media servers, on weak passwords on the router, and so on and so forth. I mean, I don't need to tell you how many things you can do if you have a shell on a Linux machine inside somebody's network. So that being said, as end nodes, the trickiest thing about cloud-related analytics, cloud-related attacks against IOTs is from a blue team perspective. It's ridiculously easy from a red team perspective. It's insanely easy. Most of the devices that we've analyzed, we've managed to get a predictable remote command execution relayed through the cloud. So again, that means that you don't need to be on the same network with the device. You don't need to be in the same zip code with the device. You can do it from another continent and get a connect back shell from somebody's device in their basement behind four firewalls. And the tricky thing is that for the blue team, for the defense team, they're insanely difficult to detect and mitigate because the traffic is encrypted. The traffic between the device and the cloud is encrypted.  You're not gonna be able to do that with IOT in order to decrypt the traffic and do the backup inspection to pick up exploits.  The IP addresses used in communication are legitimate. Your anomaly detection engines are not gonna pick that up because it's a communication between the device and the cloud. You won't know about anything that was exposed or leaked by the device because, again, it's gonna look like legitimate traffic and most companies are blind to these types of attacks. And the only viable defense that I could figure out is to be able to pick up the moment when the device sends a connect back shell if the user gets a foothold on the device or maybe when it's doing traffic with their cloud at an unusual hour. But, again, this is an open case because, again, it's very tricky. Other end nodes. If you work for companies, and I'm sure most of you do, I strongly advise that you include IOTs in your security posture and that include your IP phones, PBXs, smart power outlets, smart light bulbs, smart, the climate control things. Most of the things in organizations are connected now. So just see whatever has an IP address and doesn't have a common operating system and talk to the vendor and make sure that you have a look at their security processes and definitely subject them to a pen test or ask the provider when they did their last pen test and also with what company. I remember I talked to the director of product management at one of the companies that was making a product that was ridiculously broken from a security standpoint. We hacked it in five minutes. They had, their security mechanism was something called a client ID. And if you know the client ID, then you can interact and get all the details about the client, including their devices and passwords. And that client ID was a number. And it was like a sequence of numbers. So our client ID was 308411 and then the next one was 308412 and so on and so forth. So totally ridiculous. And I asked the guy, okay, I'm not gonna blame you for doing something this ridiculous, but why didn't the pen testing company tell you about this because it's very obvious. And the guy said that they paid a lot of money for that pen test. So be mindful of who you get to do your pen tests and be very careful about that.  You're gonna want to see the pen test reports of your provider of the manufacturer of that smart device. And you're gonna want to validate the company that performed the pen test to make sure that they're on the level and they're doing a proper job. Whenever you hear somebody say military grade encryption, run away, whenever I talk to vendors, I ask them, so what steps did you take to make sure that the software running on your device is safe? And I'm actually feeding them with additional information. I'm telling them, okay, so how do you mitigate command injections? How do you mitigate buffer overflows? How do you mitigate improper segregation of user privileges and stuff like that? And like in nine out of 10 cases, the vendors tell me, we have military grade encryption. And I'm like, okay, whatever, sure. I mean, encryption is one thing, it's cool, but how do you handle application security? Application logic that's poorly designed, we have military grade encryption.  Yeah, another important thing to know about IoT is, is that many, like 80%, I would say, of the vulnerabilities that are discovered in the IoT space are not tracked by CVs. So if you use tools that correlate with CV details or CV.mitter.org or whatever, you're gonna be blind to about 80% of the vulnerabilities in the IoT space. So only like 20, 30% at best of the vulnerabilities in the IoT space are tracked with CVs. Make sure that you run security audits on all of smart devices. I've mentioned this, I'm gonna mention it again, and this includes your industrial devices, if you have them, even if you have like those solar panels or whatever power generators in these day and age, all of them have IP addresses, you're gonna want to make sure that they have been, they've gone through a security test before you start relying on them. Pretty much, this is it, I guess, so purchase any devices for connectivity only from trusted providers with reasonable SLA. This is pretty much what I've said earlier. You're gonna want to evaluate your vendors. And actually one last thing that's extremely important. Again, taking the opportunity to push for better standards and accountability from vendors. So for example, my suggestion would be that a very simple law could be put in place that forces any company that handles customer information or that has any kind of responsibility to have a security contact. It's as simple as that. You don't have a security contact and you don't reply to security researchers, you shouldn't be able to operate. We're doing this for free. I mean, we're not even doing it for the bug bounty. So a number of us are actually doing vulnerability research for free and vendors just don't even have a link that we can use to send them our vulnerability reports. And because of that, a huge number of users, tens of millions, hundreds of millions are exposed even today, even though we know about the vulnerability and we want them fixed. A huge number of our papers, like 60% of our papers are still on hold and we cannot publish because publishing them has a potential to create chaos in the internet and the vendors that make those devices just didn't reply to our emails. So with that in mind, I want to thank you all for watching my presentation. I hope you enjoyed it.  So this is my contacts, I'm on Twitter, my handle is Jamesu and you can check out all our papers at labs.pedefender.com. So thank you all for watching. I hope that I'm gonna see some of you on site at Defcon. Take care.


Good afternoon. Welcome back from lunch. For the next hour, we want to talk about ethics and integrity and data use and management. And may say, well, this seems a little far out there, but I think it's very important to look at a very brief history of ethics in research and in data management, both from research and clinical perspective. Look at the pertinent ethical principles that we try to apply in any use of data. And discuss the idea of data integrity and ethics within our data management practices. And then look at the applications for data management personnel in the ethical use of data. So we'll start with some definitions. Look at those ethical principles. Briefly discuss some guidelines and regulations that you should be aware of. And discuss the idea or concept of data integrity and what it entails. And then look at applications for our work with data. From the Webster's new Riverside Dictionary, integrity is defined as strict adherence to a standard value or conduct and also refers to personal honesty and independence. And the one that we often think of, completeness, unity, or number four, soundness, certainly applies to data integrity. Ethic or ethics. An ethic is a principle of right or good conduct or a system of moral values. When we use it in plural or ethics, the branch of ethics is a branch of philosophy dealing with the rules of right conduct. And those apply to any branch of work. Every branch of work has certain codes or certain principles of ethics that should be applied. When we talk about data and human subjects in particular, we think of three main ethical principles. The first one being beneficence, which basically means do no harm, but in addition to not doing harm, it also entails trying to get the most benefit for our patients or for our research participants. So we try to minimize harm and maximize benefits whenever possible. So that one's easy to remember. Beneficence, think benefits, okay? And not having negative benefits. Respective persons, which is also known as autonomy, comes from the idea that each person has the right to choose for themselves. And so this involves the concept of voluntary informed consent, not just informed consent, but voluntary informed consent, so that that person makes a choice about how their data are used or what types of research or activities they're involved in. And this one also looks at vulnerable subjects. People who cannot necessarily make decisions easily for themselves must be protected in some way under this principle. And so this is where we have protections for children, protections for prisoners, protections for the mentally ill. We include pregnant women and fetuses in this group, students, because they sometimes don't feel empowered to say yes or no. And I think I'm forgetting one group. Did I mention pregnant women? Yeah, okay. So I've gotten the majority there. Generally, you'll see this very much with pediatric populations who cannot consent for themselves and are reliant upon a guardian, often to consent for them. And so they must be protected to be sure that their rights are properly represented. So this is really the principle of justice. You could think of basically as fairness, treating people in an equal and fair manner. And equity, which is not exactly the same as equality, but equity, again, means fairness. And that means fairness in treating all patients the same or all research participants the same. Fairness in making sure that one group is not carrying all the risk while another group is getting all the benefits that applies to research. And fairness in dealing with the individual as far as what their rights are, what the benefits are that are coming to them. And then also fairness between institutions so that one institution is not getting a huge benefit while the other institution is doing all the work. So it applies on multiple levels when we talk about clinical practice or research. Clinical data versus research data. Are the ethics any different? Any opinions? Are the ethics different or are they the same? I hear multiple people saying the same. Yes, ethics are ethics. The principles are the same. How they are applied may be a little bit different. The regulations or guidelines that govern them may be a little bit different. But the ethical principles that we just discussed are really the same whether you're talking about clinical data or research data. And I can't stress that enough. Don't think that ethics apply to research, but in clinical practice we don't have to think about or do those things because in essence ethical principles are ethical principles and we should use them in all parts of daily life. And this has applications for us in privacy and confidentiality of records. Informed consent versus implied consent when using patient or participant data. The idea of data integrity or data quality which may not be dependent on the patient, more dependent on us but still very, very important in both clinical data and research data. And then data security and storage which are also really part of data integrity but we also often talk about them a little bit separately. So for research we have a large number of ethical guidelines that we use but I would like to bring to the group that we should also be thinking of these guidelines and practices when we're using clinical data as well. One of the best known is the Declaration of Helsinki and I did include that in your reading packet. I don't expect that everybody will have read it but it's something that at some point you should read and understand it's not hard to read and there are quite a number of things in that declaration that do apply to data and use of data. So any data manager that's going to work with clinical data or research data needs to be aware of these principles and this is an ethical standard that is used by international committee of medical journal editors so anybody working on a publication needs to have read this at some point to understand what the ethical principles are that govern medical research and are commonly accepted. The Council for International Organizations of Medical Sciences which we often call CHIOMS or SIOMS has also developed guidelines in collaboration with WHO and many international review organizations or IRBs often use the CHIOMS guidelines as well as the Declaration of Helsinki. I don't believe that's in your instructional packet but be aware that there are different guidelines although most of them are quite similar in content overall. Within the U.S. we often refer to the Belmont report which also gives guidelines for the review and use of research data and then Kenya itself has national guidelines put out by the NCST, the National Council for Science and Technology and within Kenya we need to be aware of what those guidelines are. I don't think I supplied that for the reading packet but I will get a copy and supply to ETA because I think it's important for all of you here within Kenya to definitely have a copy of those guidelines and to understand what they are. They do apply specifically to research but again if you look at them closely they have a lot of broad applications to clinical practice. Within the United States we have a number of regulations so they're not just guidelines, these are actual laws that govern how research is done and under some of them how data can be used and you may say what do U.S. regulations have to do with us in Kenya or what do U.S. regulations have to do with any international research or international clinical data management. Well the fact is at least in this setting we get a lot of money from the United States government to conduct research. And while it doesn't fall under these Code of Federal Regulations most of our clinical data here is covered by U.S. government funds through PEPFAR and so we need to be aware and understand when and how U.S. regulations do apply to our work even though we work in Kenya. For people working in other parts of the world if they're getting U.S. government money some of these regulations may apply to them. These include what we call 45 CFR 46. This is a specific federal regulation that governs the ethical conduct of human subjects research and we'll discuss that a little bit more. For any trials involving drugs the FDA has specific guidelines and rules so if it's research involving drugs or sometimes research involving devices then the FDA regulations also may apply. And then for projects which are funded by NIH NIH may have its own guidelines on the use of data and how data are used and why do we have to pay attention to them because whoever's paying for it has the right to make rules about it and so the funder can often have a say in the principles guiding our work. And finally there's something called HIPAA which we will discuss again. This is a U.S. law designed more for clinical records but also for research that guards the privacy of records and something that we also need to be aware of depending on our setups but here we need to know about it. So the U.S. Federal Code of Regulations 45 Part 46 covers human subjects protections it also covers the requirements for the operation and review by IRBs institutional review boards that are supposed to review all research and the protection of vulnerable subjects which we discussed about under the principle of autonomy. A human subject under this law is defined as a living individual about whom an investigator, professional or student who is conducting research obtains data through an intervention or interaction with that subject or identifiable private information. Now while this applies to research you can see it's also very applicable to our clinical setting because by that definition every patient or research participant that we come in contact with is a human subject. Privacy and confidentiality rules as I mentioned apply to both clinical and research data. This is a major concern for patients when it comes to electronic records and data. If you look through the readings I provided you when it comes to electronic data patients are very concerned about whether or not their data will be private and in fact within one of those articles patients sometimes do not go to places that have electronic medical records because they fear about the privacy and confidentiality of their data and so knowing how we protect these things and being able to educate participants and patients about this is very important as we know electronic records are growing and provide a lot of benefits to improving care. Privacy and confidentiality is a process that has to be safeguarded by every member of the team from the person taking down the information from the clinician or the healthcare worker all the way up through the data manager and data management group all the way to the medical records people who are storing and archiving data. At every step privacy and confidentiality must be maintained. So in the United States we have a very specific law called the health insurance portability and accountability act of 1996 which we know as HIPAA and it has issued regulations entitled standards for privacy of individually identifiable health information. For most covered entities who are covered by this law compliance with the regulations has been required in the United States since 2003. Now some of you here will know that are we covered under HIPAA here in Kenya? Do we need to pay attention to HIPAA here in Kenya? I hear a couple of no's. Does everybody agree with that? HIPAA does not apply in Kenya if all the data is collected and analyzed in Kenya. But within our setting we have a rather unusual situation and within many academic setting internationally they will come up with the same situation because sometimes data managers in the United States working for another university manipulate or analyze these data now that means that we are accountable to this regulation because they work for a covered entity that must apply all the rules of this law and so some of you will recall we had a huge discussion about creating data sets for our research on whether or not we had to pay attention to this law and the answer according to our IRAC and IRB was yes you do because Indiana University is a covered entity and if any data manager at IU is going to look at personally identifiable information we have to pay attention to HIPAA. So unfortunately the answer is yes we have to pay attention to it except in some cases where all data is only collected here and only analyzed here. In which case HIPAA does not apply. So this privacy rule in the CFR 45 establishes this category of health information which we call protected health information and a covered entity may only use or disclose to others in certain situations these PHI. It usually requires that an individual should provide consent or an authorization before any of these types of information can be disclosed or used. And that need for authorization though can be waived by an IRB when they say that it's very minimal risk for loss of confidentiality or privacy. The types of PHI and this is a subset there are some states which will list up to 16 of these but the most important subset includes things that could possibly lead to the identification of an individual if they were included in a data set. So name that's pretty obvious that if you release somebody's name into a data set that could identify that individual that could be a loss of privacy or confidentiality. All elements of any date except for the year are considered PHI. A social security number and I would for Kenya's purposes that would be the national ID number. Driver's license number. Any subdivision smaller than a state. That's a pretty tough one because most of our patients come from villages. But that village or location, sublocation is considered PHI because it's smaller than 5,000 people or smaller than a state. URLs or IP addresses for any individual. Vehicle numbers or license numbers can be traced back to an individual and phone numbers. These are some of the more commonly used PHI. Please take a look at that list because you might see it again. Just a hint. Although I hear all of your quizzes are open books so you know exactly where to go for that. So PHI only become personally PHI when it is received, created or received by a covered entity. So when it's sitting in our database it's not PHI. When you put it in a data set and send it to a data manager in the US to help create a data set who is working for a university or a health plan in the United States now it's PHI. Because a covered entity is anyone working in a US health plan, US healthcare clearinghouse or US health providers that transmit electronic information. And so in some instances for us in Kenya this now will apply if we are transmitting any data to one of these groups in the United States. So while I certainly don't expect people to understand everything about HIPAA you should understand where to learn about it and when it does apply in the international setting because it is an important regulation. A researcher themselves is not a covered entity unless they are a provider in a covered entity.  So my research comes under this law even though I am working in Kenya. And because we collaborate with many different international schools we are often going to be in this situation when we are using data in research. Research is governed by HIPAA if it is obtained from a covered entity that does not apply to us and as I mentioned the IRB may on occasion waive the HIPAA restrictions on use of PHI. Now we have gotten a waiver here at Moore University for the use of specific items within our data sets. That includes birth dates that includes patient locations that includes GPS data because we were able to argue successfully to the IRB in the U.S. that it would be virtually impossible for any data manager in the U.S. to identify a Kenyan patient based on those small pieces of information. And so they were able to give us a waiver which makes our job here much easier but not all IRBs are going to make those waivers and so sometimes you may be in the situation where you are going to have to really manipulate data to make sure you are not using any of that personal health information. So how does this all translate into practice? We've talked about some guidelines we've talked about some regulations why does that all matter? We've talked about some ethical principles Well electronic health information whether it's used clinically or whether it's used for research has a lot of important applications that are applicable to ethics and also has some misuses which are very applicable to ethics. So we know that electronic health information can improve quality and safety of medical care. That is beneficence, maximizing benefits. That is beneficence. We know that electronic medical records can be key to sometimes showing accurate outcomes in research on health or health practices that again is beneficence but that benefit is now extended to all of society and not just to the individual. If we analyze data from a lot of people and use that in public health practices or evidence-based medicine that again is beneficence. We are maximizing the benefits of that information for use to improve care for many. Confidentiality though and privacy can be lost if we do not properly safeguard the security of our data and so that would be a breach of autonomy personal choice about the use of information and that could be a breach of beneficence if it causes harm to an individual. Information sometimes is or can be used without a patient's knowledge or consent. That certainly applies to a breach of autonomy or a breach of the idea of respect for persons. We'll talk about that a little bit later on because there's often questions about who the data belongs to. Does it belong to the patient, the institution, to the research project? That is a question. So when can it or can it not be used without the patient's knowledge or consent or is that always required? But this definitely applies to the principle of respect of persons. And finally, depending on how we choose to design our databases or collect our data we could end up excluding some very important populations and in so doing we may change the analysis, we may change the way that outcomes are reported and if we do that improperly now we have breached justice. We have not paid attention to the correct design in a way that treats all populations fairly. Data integrity. Now I'm sure everybody working in data has heard a lot in all of your studies about data integrity and if you go online and start looking for definitions of data integrity you will find a lot of different definitions and these are just four of about ten that I found. The assurance that data is accurate, correct and valid. Accuracy and consistency of stored data indicated by an absence of any alteration in data between two updates of a data record and that it's imposed within a database at the design stage through standardized rules and procedures. Also involves error checking and validation. Another definition states that it's the exact duplication of sent data at the receiving end. Doesn't say much about the collection end. And then a final one, assurance that data are unchanged from creation to reception. I actually like this one, this last one because it covers all points from the person marking things down on a data collection form all the way through to the person storing that file or backing up that file or archiving that file. This one includes every step of the process but you'll see the common thread throughout all of these is talking about accuracy and completeness of data. And the idea that this is a process which starts with the first point of collecting data to the very final point of data storage. So think of data integrity as a process and it's a process that depends on collection which is accurate representation of the data. If a patient tells me something in the clinic but I think well that's not right and I mark down something else I have now compromised data integrity. I have not recorded accurately what the patient told me. Data transfer. Any time that data is transferred from one point to another it should be done in a way that ensures that that record is complete and is transferred in its entirety and is received in its entirety without change. Also storage and security preventing loss of data is just as important in data integrity as making sure the data are accurate at the time that they are received losing data also compromises the data record and so storage and security are very important. This involves sharing of data how much do we share who can see it and then the analysis of data the proper representation of those data when we start discussing what they mean. These are all important in data integrity. This fabrication and falsification is a very serious breach of data integrity and this is an important challenge to data integrity in research. There have been very highly publicized research where data have been fabricated or falsified in some way in order to make a certain research project or a certain analysis look better and that is a serious breach of ethical integrity. However, that is only one of the points as I showed you earlier in data integrity but one that we hear much about. Human error also contributes to loss of data integrity. If I mark down that somebody's liver function test is a 40 and when the person entering the data marks down a 4.0 into the database data integrity has been compromised is that a breach of ethics? I see one person shaking their head how many people think that's a breach? How many people think that's just human error and should be expected? Good. It's human error. While this is a breach of data integrity we would not consider that unethical behavior because human error is an accepted part of human practice. We try to minimize those errors as much as possible to protect the integrity of our data but we also recognize that unintentional human error will that is not a breach of ethics whereas purposefully fabricating data or falsifying data is a terrible breach of ethics and I hope nobody disagrees with that statement. Sometimes there are things that can compromise data integrity that are not unethical while there are other things that compromise data integrity that are terribly unethical. Try to learn to balance those ideas in your mind that there is unethical behavior and is human error which is an acceptable part of human practice although we try to minimize it. There has been quite a bit of concern about research misconduct because as I mentioned there are some highly publicized cases where people have fabricated or falsified data to make the results look better whether that's from clinical data or analysis from research data and so the US Department of Health and Human Services actually sponsored a conference in 1990 on data management specifically to summarize the ways in which conduct of research depends upon the responsible use of data and responsible data management and out of that conference they came out with a number of very important principles and guidelines and one of the responsible research begins with experimental design and protocol approval. Does that have anything to do with data? How many people here have been involved in writing a research protocol? No one's holding up their hand but I know three people in here at least who have been. So, yes, design of a protocol has a lot to do with data because every protocol should have a data management plan a data sampling plan a data analysis plan much of research design is about data and so understanding that means that the ethics of data integrity begin with the design of the study not just the conduct of study but the whole design of a study. Studies should be designed that involve record keeping in a way that will ensure the accuracy and integrity of the data and an attempt to avoid or minimize bias whenever possible. The design guides the criteria for including or excluding data from the statistical analysis and it should entail responsibility for collection, use and sharing who is going to see that data who is going to analyze the data and how is that data going to be used. It cannot stress this one enough and you will probably see that a couple more times. Everybody in research and I should expand that say everyone who deals with data from a patient or research participant so let's expand that to both clinical and research. Everyone in a project or a clinical care system has a responsibility to ensure the integrity of data that comes from the person collecting the data to the person picking up the paper forms and carrying them to the data entry operator to the data entry person and the data manager to the person filing those forms to the person archiving those forms and storing those forms everyone has a role and a responsibility to make sure those data are maintained and those data are kept confidential. The ultimate responsibility within research does sit with the principal investigator but the fact is that anybody who helps plan the study helps collect data, helps analyze or interpret the data or publishes is part of the publishing of the data or maintains records is responsible for helping to maintain data integrity it is a team effort it is not something the data manager alone has to grapple with but something that we often rely on the data manager to educate the rest of the team about and so you need to be as a data manager an expert in data integrity and know how everyone's behavior affects it and how you can help educate them to protect data integrity whenever possible data collection can be very repetitious very time consuming and quite boring and there's often a temptation to underestimate how important it is we've seen this on our own forms if people consistently miss fields or they don't understand what a field means and are often marking down the wrong types of information or incorrect information this severely compromises data quality and data integrity so it is a very important part of the process so those who are responsible for collecting data and in our case that is the clinicians all of the care workers within the other programs within our clinical setting everybody must be adequately trained and motivated to collect data in the most conscientious way possible and the most obsessive compulsive way possible and we are planning to do here quite a number of trainings throughout the care system to try and improve data quality to educate our clinicians and our workers within the different clinical settings about the importance of data quality data collection plays a role in that we should always employ methods that limit or eliminate the effect of bias whenever possible and we should always keep records of what was done by whom and when so this is very important in the design of a database for those of you who work with the AMRS we want to know who entered the data who viewed or changed data from a data management perspective because keeping records of what was done by whom and when is very important when we start looking at data quality and ensuring data integrity and finally analysis and selection of data oh good we'll have a bit of time for questions finally analysis and selection of data data does not stop data management and ethics of data management do not stop with the creation of a data set but it also goes as far as reporting how data is used in analysis is also governed by ethical principles we do not change analyses to give the outcome we want we report all the data we analyze all the data and we report what we find in an honest fashion however because we often can't report everything that's done we often have to choose which ones we think are more important and that's where judgments come in but we need to try and make decisions which are not just favorable to the outcomes we were hoping for but decisions that make sense with regard to representing the data accurately so we have to critically evaluate the reasons that we included or excluded data from a data set we have to talk about those we have to tell people what we did or did not include and why okay and we have to have a lot of regard for how that might bias the outcomes when we're doing analysis and we have to clearly document at each step of the process what we did why were these things done so that if somebody wants to come and do that analysis again they could replicate it and if we used an analysis technique which was a little bit controversial we chose a test that maybe some other statisticians wouldn't use we should explain why we used it and justify that and open ourselves up to comment by others on whether we had the correct analysis to do retention of data what should we retain it might be impractical to store large amounts of data particularly if they're on paper okay and so what do we choose to retain at minimum enough data should always be retained to be able to reconstruct anything that was done for an analysis now in a patient record from a clinical standpoint everything and we should retain everything for as long as is required by law by the country we're working in okay this will differ from country to country and from institution to institution okay I honestly don't know what the law is in Kenya as or if there is one as to how long more university or more teaching hospital has to maintain patient records for research data it may be determined by the funder may be determined by the country you're in may be determined by the country that funded the study so for many of our studies here we must retain data for five to seven years depending on the state okay the state law that applies to that project in the United States if they were funding that project okay but for clinical records that is going to depend on the institution and the country's law sharing of data who gets to see data this is important for responsible research and also for responsible clinical practice de-identified data should be shared and this is a conclusion again from that same conference we discussed earlier data should be shared so that others can verify your conclusions or analysis we don't throw analysis out there and then suddenly decide now we're not going to let anybody see what we did okay that is not considered to be ethical it should be verifiable by others but we do not share personal information on participants or patients we only share de-identified information, aggregate data so that personal information should never be shared because of confidentiality and privacy concerns data security involves two processes and I hope this is second nature to most of you by now but this is a good review it means both limiting access to data backing up data data backups we don't just do because we worry about it we do it because it's part of data integrity and data security loss of data compromises data integrity so we do not want to lose it and because we know that computer systems fail data can easily be lost we do regular backups to prevent that so we can rebuild data sets in places when necessary as far as limiting access that can be locking up all paper records password protecting all computerized information limiting access to any records to people who have the defined privileges to see that data defining privileges of access for different levels of users with an electronic system and then preventing outside access so that people can't hack into private information ownership of data I'm not going to give you an answer on this one but I'm going to make you aware that this is an important ethical concern within data whether you are talking about clinical data or research data but often comes up with research who owns it does the patient own it or does the hospital own the data you will hear you say patient I agree with you I think the patient owns the data I should not do anything without that patient's consent unless it's part of a hospital policy that these data are going to be used in that way and they've accepted care within that facility as is the case here but I personally as a physician feel that that data belongs to the patient not to me but to them however you will find administrators at MTRH or any hospital in the world who tell you that data belongs to the institution that's part of their record that's part of their mandate and so there's disagreement sometimes on who owns patient level data when it comes to clinical care research does the research own the data does the funder own the data and when you publish who owns the copyright does the public we sign away our copyright to that published information there are a lot of ethical concerns when it comes to ownership of data and there are no right or wrong answers these are things that are hotly debated all across the globe but something you should be aware of you know this is an ethical question when it comes to use of data who owns it and who has the final say on how it should be used and finally ethics in publication we've covered many of these but because many of you will be involved in creating data sets or analyzing data in journals you should understand ethics in publication have a lot of guidelines that are very pertinent to data and data management research should always try to answer a specific question we don't just go out and collect masses of information and then start doing willy-nilly analysis to hope we find some question in there we can answer research should always have a specific question in mind and we should try to answer that question through proper collection and use of the data that we get statistical issues such as sample size and what statistical tests we use are very important part of design to ensure that the data is very likely to answer the question that we want I may design a study with great research question but if I choose the wrong strategy to collect and analyze data with no hope of answering that question that research is now unethical because I am not creating a plan that will maintain any kind of data integrity okay IRB approval is always required when using human subjects human tissues or medical records and so there must be oversight by an ethics review board for the use of human subjects data in whatever form okay for research to be considered ethical we don't just go out and start doing collecting data and reporting data from human subjects without some type of ethical oversight data must be appropriately analyzed I shouldn't say should be it must be okay now there may be questions about what the appropriate analysis is biostatisticians don't always agree on the same test but inappropriate analysis are not necessarily ethical misconduct again this comes with the idea of human error or human judgment if somebody is doing a master's project and they do an analysis and they do an analysis with every intention of doing a good one and they analyze to the best of their ability they may report things that were not necessarily as well reported as they could have been okay they might even be wrong because perhaps they didn't know to use the right statistical tests is that ethical misconduct no that's human error again okay would have been better if they had the advice of a biostatistician or a statistician who could help them analyze the data but that is not ethical misconduct that's human error sometimes biostatisticians at the very top of their game disagree on which test is the best and so the analysis there may be disagreements about the ethical misconduct there is sometimes disagreement on how to analyze the data in the best possible way so that is not ethical misconduct however as I said before fabrication or falsification of data is always considered ethical misconduct finally sources and methods of attaining and processing data must be disclosed exclusions should be explained in full if I decide to remove something from the data set I need to say so and methods used to analyze the data should be explained in detail it's okay it's considered fine to go ahead and do other analyses from the data once your initial question you've made an attempt to answer that but again we don't just go collecting data and then start saying okay now what in here is there to analyze okay and finally bias which always exists should be discussed in detail in all publications or in all data analysis