Hey, good afternoon everyone. My name is Gavin Adams. I'm a Specialist Solutions Architect with Amazon Web Services and I'll be presenting deep learnings at the edge. I hope everybody is staying safe out there and welcome to the Embedded Linux Conference in a virtual environment. So a couple things here is just going to go over the agenda of what we're going to be talking about today. So there's a lot of interest around edge computing or edge capabilities, which is taking, you know, what we normally see is either data center-centric or cloud-centric and taking it out to the edge. So we're going to go through some of the different use cases for why you might want to use the edge. We'll then go through some of the implementation patterns. What we have seen with our customers and just, you know, through user experience, how these things actually can get deployed, some things that are good and some things that are bad so that we can sort of highlight those. Next we'll go through, you know, what happens when things go wrong. These are devices that are out in the field. So unlike, you know, in a data center or in a controlled environment, there's a lot of things that can go wrong, but there's a lot of ways that we can also address that through edge computing capabilities.  A couple things is that we're going to be taking questions at the end, as mentioned, and we'll do this through the chat mechanism. If we don't get all of the questions answered, we do have the AWS booth that you can go to in the Gold Hall and ask other solutions architects. But having said that, we've got a lot to go through, so let's go ahead and get started. So before we actually jump into our edge cases, this one will go a little bit over architecture terminology. As a solutions architect in the IoT space at AWS, I use the word cloud a lot because that's what we work with, you know, cloud cloud cloud. But when I talk about cloud as sort of a holistic view is, you know, consider this to be, you know, either your cloud platform you're using, or if you're not using cloud services, you know, the data center, your on-prem or on-premises environment, wherever you've got sort of, you know, central control, and I've got services that are available 24-7. When I talk edge, I'm basically talking about discrete devices that are at the edge. This can be one or many devices, but it'll be basically a combination of both, you know, hardware and software and then multiple ones, you know, out there. The reason we like to sort of, you know, break these up is that when you talk about cloud, that's where essentially you've got a lot of your elastic capability, where you can have services on demand, and that when you want to basically scale out or scale back in, you can do so in a very fast fashion. When we talk about sort of the edge component, and I'll talk about sort of the hardware and software, essentially it's a combination of hardware along with software that you bring, an operating system, system packages, and then your own components that make up an edge device. This can be either a physical device, or in some instances it could actually be a virtual type of environment. So it could be right on top of, you know, VMware or Hyper-V. So let's talk about the edge a little bit. So I mentioned it, there's a lot of interest into it, and, you know, if we take a look back, you know, what edge is, is, you know, you can see for the term is sort of an IoT, and the internet of things essentially comes from, I've got some type of device that's doing something, you know, in the real world at the edge. The way we like to kind of, you know, say that if you can address one or more of what we call the laws of the edge, is that it might be a good candidate for how you want to do edge computing, or you deep learning at the edge.   You want those to happen in microseconds or even single-digit millisecond latency. And if you had to take that data from a local device and send it out to the cloud, you're not going to be to achieve that. So that's a good example of essentially, you know, of how we can say that, you know, there's operations that should take place locally. That's a good candidate. The second one is around economics. As devices are now starting to go out into the environment, they're leaving the hardwired ethernet and Wi-Fi environments to sell your, or even to like Laurel WAN or other types of low-power WAN environments, if we now have network connectivity that can basically cover the globe, even from satellite, you know, communication, but it comes at a cost. It's either that you're going to have a significantly reduced amount of bandwidth. You're going from, you know, megabytes per second or megabits per second to potentially bytes per second, or that the cost of transfer data is going to be significantly more expensive. Transmitting one gigabyte of data on your broadband provider's network is going to be a lot less cheap than a one gigabyte across, you know, an LTE or a 5G-type connection. And then finally, it's kind of the law of the land, is that there are certain situations where you may want to have processing and there's, you know, values for it, but from a regulatory or sovereignty laws that there's limitations on what you could actually send out. A good example is that if you've got edge computing inside of a hospital environment, it might be very good to do some of those closed-loop operations inside of an operating room or a patient's room, but for, you know, personal identifiable information or healthcare reasons, that information should never leave the environment. So a couple of examples we'll go through here. Common one from an edge computer is sort of like, you know, a wind turbine farm. We've got a lot of very extensive devices that are close proximity to each other, but the farm might not be in close proximity to the rest of the world, meaning that they can communicate between each other but requires a cellular connection outbound. We want to do some operations on this. We want to do predictive or preventative maintenance on each of our turbines, which requires, you know, high velocity checking of vibrations, for instance, and that we want to essentially emit, you know, small amount of data to the cloud to say, hey, come out to farm one turbine number 22 for maintenance perspective. Comparing that to, like, the automobile of today, the modern automotive environment.  Keeping in here is that, you know, you've got a variety of different components, and you can actually include, you know, third-party ones that need to communicate with each other, and they differ. When a car is in a garage at home, you have got a good Wi-Fi connection, but when it's out in the street, you're now going back down to, like, to a cellular or LTE type of connection. And then the final kind of, you know, third example here is like site inspection. Very common where you might have a drone that you drive out to a remote site. This could be a refinery or an oil well or even, you know, a large farm environment. You program the drone to essentially do its autonomous operation.  When it's actually in that sort of autonomous mode, that's where Edge, you know, Compute of the Edge can do some differentiation.  So if we kind of categorize these three things together, the turbine, the automobile, and a drone site inspection against the three laws, is we can see that there's certain values that, or certain things that come into each of those that match up. So the wind turbine is that from the physics perspective, I need to be able to do thousands of monitoring, you know, measurements per second on vibrations. For the automobile, from the economic perspective, I want to capture a lot of video to help train my autonomous driving component, but I don't want to transmit that when it's on the road cellularly. I want to transmit that when I'm back into a Wi-Fi environment. And for the drone, is that there might be a corporate security policy that, you know, any data collected by a drone itself can not be transmitted across the public internet, has to go through a private connection. So when you think of your use case, think of how you can kind of tie, you know, a potential use case into three laws. If you hit one of these, okay, if you hit two or three, that's normally a very good candidate for an edge component. So speaking of that, you know, what does an edge implementation actually look like? So I'm going to be talking about this slide a lot. And essentially what we're saying is that, you know, starting from the bottom up, we've got hardware components, we've got an operating system, system packages, and then our user packages, which are sort of the orchestrated components that come back and forth from the cloud, that then add your business logical capabilities to whatever the device is. Starting with, you know, the hardware is we may need to interact with other things such as GPUs, for machine learning, inference at the edge. You might want to be a user connecting to fix your sensors. You might have different types of architectures that you're running here. But this is your common hardware that you might want to expose all the way up into your edge application. From an operating system perspective, we normally see this in the three different environments. A lot of times, customers will start with sort of a general purpose operating system. You know, they use a vune to a red hat internally. Or they're doing development on a Raspberry Pi. So the Raspberry Pi OS becomes sort of a, you know, the use case. These are kind of general purpose.  They're not really in sync with each other. The other one is sort of the embedded space from Linux is that you can use things like the Yachto or open embedded to build, you know, very specific and repeatable or immutable type of operating systems that can run on your particular devices. We also see customers are using more lean or thin operating systems such as, you know, Alpine or Busybox. Or even cases where you're going to sort of abstract out the entire operating system environment and use third parties such as Belina using the Belina OS as an underlying component for then deploying your edge capabilities. So moving up from the operating system into the system packages, there'll be a variety of these things. I won't go through, you know, much detail, but just to call out is that, you know, if you're running Python applications or Java apps, you're going to have to have a Java runtime or a Python interpreter for that. If it's compiled applications, you're going to have to have shared objects potentially or certain versions of G-Lib C.  For the machine learning and the deep convoluted neural networks, these might be full blown ones such as, you know, MxNet, Caffe, you know, TensorFlow, etc.  Call out also from a management perspective, ensure that you have, you know, something that's built into there that allows you to do remote monitoring and management as necessary. Your edge capabilities and orchestration provides some of that telemetry, but by having the ability to create a tunnel that can be outbound from the device to the cloud and then the cloud back to the device, is it allows you to do that type of remote monitoring or troubleshooting is necessary along with doing OTA updates. And then from the edge framework, this is going to depend on what you're doing.  It might be EdgeX or Kuvex or Kuvej or, you know, our AWS IoT Greengrass, which is a fully, you know, managed environment for doing new edge computing capabilities. So operating systems and system packages. And then on top of that, we have essentially the user packages. This is where, essentially your developers or you are adding your specific value-add business logic to the device to do what you want it to do. You know, machine learning at the edge, detecting certain events and taking actions. And we normally see this happening either through, you know, native applications that you develop and your SCP or copying over to a device. This may be that through it's an orchestration of multiple containers that, you know, you're doing through, you know, Mezos or Kubernetes. Or go back to the Edge frameworks where you now have, you know, a rich environment where you can do things like as containers or server-less environments such as functions along with other things such as, you know, APIs, queuing or stream management, all capabilities at the Edge.  We'll see that in a demo a little bit later on. So you might want to be able to build the capability of having, you know, local APIs, you know, either restful ones or just quick request response or endpoints that other types of devices can then interact with if they're also working with the Edge device. And then for the machine learning model is the ability to be able to have machine learning models downloaded and operate in the device. A long ability to have some type of new closed loop or round-trip operation for doing retraining and detection of drift so that you can ensure your models are running properly. Let's take a look at an example of all these kind of in operation. So we have on the left hand side, you know, physical camera, we have our Edge device, we have the cloud. So the first thing we do is we have a local process, a collector, that's taking in a certain type of format, you know, h.265, and it's using something like OpenCV to convert this into a frame that our inference model can actually consume. It's expecting it to come in a BGR format and resize to 300 by 300 pixels. Our inference could be a Python-based application, you know, maybe using ResNet 50 as our model and using MXNet underneath the covers. And then it's basically, it's incorporated into the training model that it's loading into. So we take in our video, which is, you know, a standard h.265 format. We're converting it into the format that we want. And then once the inference is done, we're only sending to the cloud the inference capability. So the list of the objects that are detected and the bounding boxes around it. So in this particular case, you know, everything on the device is very rich in the amount of data that's being transferred back and forth. We're having megabits per second from the camera into our device, but we're only sending bytes or kilobytes of information based on inference out to the cloud and only the things of interest. So let's take our basic example here. We're going to take it to the next level. So here's what we're kind of saying, all right, for the local streaming, we can use some of this G streamer pipeline that allows us to abstract out any type of cameras as to an input. We then bring those into our, you know, transformations to BGR in the resizing. And then in the green shaded section are all of our edge components that we're orchestrating. We have our machine learning inference model that's taken in the values and giving back, you know, giving back, you know, values back. This can either make it available to like a local web server and we'll see it in the demo or you might, we might be doing data enrichment which we then want to send to the cloud for a common web platform there. It also might be that we want to take a look at video of interest that might be useful for retraining our model. And then on the cloud side where we have the capability and resources to essentially manage not just one of these devices, the edge, but potentially thousands, is we may have a web server that essentially is an aggregate of all thousands of our devices that we can drill down into each one as necessary or see what the total aggregate is. For the machine learning trained model training component is we might be getting back video snippets from different views or perspectives and then doing the training of a model across, you know, a large amount of instances of, you know, GPU accelerated, you know, machines to recreate a new model which we can then deploy as a retrained model out to the edge to all of our devices as a single deployment over their updates to then have them take advantage of those new capabilities. Apologies there my story is getting dry. Just also call out and we'll go this a little bit further is that there's different types of layers that we can map from the edge device back into cloud capabilities which will help with the orchestration and automation of everything from the operating system all the way up through our machine learning models and compute functions. So with that I'm going to sort of transition into a demo here. So before I jump over and show you this in a YouTube video and sort of narrate what's happening, I want to kind of lay the land from the hardware perspective.  What you see here is essentially a conveyor belt at the bottom and at the center of that is we have all of our compute capabilities. On the right hand side we place pieces of pasta different shapes onto the conveyor belt and then our central edge device drives the conveyor belt forward at a particular speed and as the pasta comes underneath the camera it does object detection determine what type of pasta it is and what type of confidence level is there. The HVMI for the local user interface is used for a local interface that like somebody on a plant floor would be able to see and then we put the Wi-Fi for cloud connectivity which is sending that other data out to the cloud for a central type of presentation. So with that I'm going to stop presenting here and go over to a YouTube video and I'll do a little bit of narration on that. All right now I'm going to let this settle. So what we're essentially seeing here before I start this is one of our partner managers that was going through this. So as you can see here and I'll start the video up is that we see the pasta on the conveyor belt and at the center you can see the camera that's facing down with a light which is taking a look on that. What you're showing right there is a system on module or system on core of an IMX-8 processor which is running beside the green grass and the inference components. And you can see that the the pasta comes through and I'm going to skip forward a little bit here so we can take a look at the screen on the left. So this screen here is that local interface coming off the HDMI. So as you can see on the right hand side of the screen is that the conveyor belt is moving at you know certain frames per second and we're actually seeing the pasta there and then the left hand side is we're getting the inference results locally. So as the machine learning model says ah this is a piece of fusilli pasta or elbow pasta it can then basically you know increase the account and show what our confidence levels are for this here. So this is all the capabilities that's happening locally on the edge and this can be at you whatever speed that we want you know we can take a full advantage of you know HDMI full motion video these types of things. Let me scroll forward a little bit here and above this is another display which essentially is showing the cloud side so all of that data that's being collected locally is that we're only sending back a small JSON object or small amount of information about the current state of the situation or of the device and you note there on the left hand side as we'll see the pasta on current frame bump up when it sees a piece of pasta and then there's the other portions of you know what the the certain the inference things are. So this is a full you know web application workload that in real time is seen values coming back and then that is basically doing an update on the display here. I'll pause this and go back over to our presentation. All right so that was basically the hardware that you saw here. If we were to take a look at the architecture this one is very specific and opinionated for AWS but it kind of demonstrates the same capabilities. Left hand side is we have our device right hand side is the cloud. On the device side we have AWS IoT Greengrass which is our edge compute capability which is orchestrating all of our functions and our machine learning model. The little circles in there which have got a lambda symbol are our lambda functions and this is sort of serverless compute that we're running you know on a local device and this allows it to interface with the actual camera and the machine learning model to do the inference. The the lines between the device to the cloud a breakout into two. The top one goes into SageMaker and that is our machine learning set of services that allow you to train models and do machine learning operations and this is where we can take back that retraining data but then all those other boxes below IoT Core, Cognito, API Gateway, CloudFront etc are all the components that make up that that second display that we saw which is the you know the the overall view and the interesting thing about this is all those boxes we have got databases we've got compute we've got you know static websites reactive websites these are all able to be deployed through a single click through the CloudFormation template. So just you know just the demo here and I always like using YouTube because the demo guides are normally very good about this is that this kind of demonstrates how you can have local capabilities on a device and that if you want to interact with it locally you can do so but that you can also do those three laws still interact with things you know from the cloud side where you're on premises environment. So we've spoken about you know some of the use cases for for edge and also what some of the patterns are but because the devices are out in the field I can guarantee that there will be problems with them as soon as you don't have the capability of touching a device to either do something to it or even reset it that's when things go wrong. So the first one that is probably I would say 80 to 90 percent of issues that we see with edge computing is internet connectivity or intermittent connectivity. It's very common to hear that a backhoe has taken out you know a certain amount of internet links for the the northeast United States or for the southwest United States. It's very common to see you degraded you know capabilities if there's ZDoS attacks or you know bad routing tables and these types of things and what that basically means is that from a cloud perspective you may have a thousand devices out there and any one time you may only have a certain portion of your fleet that's actually active or available. So how do we have to deal with those devices or how should we deal with those devices that go from an online to offline state? Our CTO, Werner Vogels, has got a saying that everything fails all the time. So if you take that that particular statement you can design for failure and you should design for failure. Assume that whatever you put into the field everything potential is going to go wrong. If you design for that then you've addressed those types of failure points. So from a connectivity perspective is you know consider things that you know if the device is in an online state what happens it goes offline and what happens it goes offline for a couple seconds a couple minutes or it's offline for a few hours due to a some type of internet outage or networking outage. Understand the the interactions between the local components to the cloud back and forth so that you know if you are normally sending data to the cloud and you can't because it's offline how do you queue that data locally? Conversely think about also if you're in an offline state and you go online you imagine from the cloud that you've got your deployments you deployed version one to all of your devices. Then over the course of a couple days you've done versions two three and four but when your devices have been offline for that period of time so when it comes back online at version one and reaches out to the cloud do you then want to go ahead and deploy versions two three and four to get it up to date or can you go from version one all the way over to version four to complete that process. So those are some of the things to consider and from a timing or temporal perspective think about this in matter of minutes to potentially even years like kind of push yours out there but we do have customers for instance that have got shipping containers with edge devices on them. When they're actually in a port and in a shipping container yard they've got good internet access so they can say you know what's the the current condition of a cooling container. When those containers are actually put onto a ship and a ship is out at sea they normally have no zero connectivity but they're still running local edge computing operations. This might be to adjust thermostat values tracking conditions of pumps and compressors. So when they actually leave one port and enter another port that could be a two to three week period. How do you then sort of flush that data that's been collected?  So think about things around your intermittent connectivity from your device to the cloud. The other key when we see is that security constraints. A well-designed edge device will always reach out to the cloud. It will never open a listening port and looking for an incoming connection. It will always be the one to establish an outbound connection to a web to a always available service in the cloud. This could be a full variety of ports depending on your protocols and the actual end points might be different. A lot of times is that you'll have a fully qualified domain name such as you know myservicename.it.example.com. In this case here is that if you go to your firewall people and say can you please open the port port port three to myservicename.it.example.com. A lot of times I say we don't do that. We don't do fully qualified domain names. We only do IP addresses. So what's the IP address associated with that name? Sometimes it might be a single IP address. In other times especially for cloud services that might be thousands, hundreds of thousands or even potentially millions of IP addresses that that endpoint can change over a course of time. And most times the security people don't say no way we're not going to allow that to happen. So one way to address that is to actually reduce the access surface area. So instead of the attack surface area for like doing security testing is if you can reduce the amount of end points that a device has to get to and through using like this the VPN tunnel you can help address that from a security perspective. The tunnel can be something that's site to site like established from a firewall back into the cloud or data center environment or it can be a client side VPN application that takes everything on your local device sends it through a VPN tunnel out to a set of addresses maybe one or two IP addresses for resiliency and then only one port. But by doing that this allows you and to to convey to security people that here's a set of addresses and a specific set of ports which are easily accessible. There's a couple of design considerations that come into play here too. On the device itself design for your functionality. A common thing that I see from customers is that you may have your machine learning folk running Python 3.6 for all of their machine learning frameworks. NVIDIA has a good example of this with their jetpack environment. Jetpack today everything is designed out of the box to run under 3.6. That's the frameworks like the TensorFlow, the CUDA environment, etc. But you also might have some of your other developers that are writing other functionality that we're using Python 3.8. So in this case consider what applications or languages need to be available on your actual environment. If you're doing this from the Linux perspective directly you may want to say all right let's go ahead and install Python 3.6 and 3.8 and give specific paths to the applications that need this. And that's when you're basically doing sort of packaging for a common type of environment. When it comes to container-based approach you may say all right for our machine learning-based containers let's build in or put in the layers for Python 3.6 to support those people that are developing from that perspective. And let's use Python 3.8 for the developers that are writing other containers or microservices for other applications of our edge device. Other thing is we're going to be doing updates of these devices. These initially you know these will originate from the cloud. There will be a deployment or a manifest of all the changes that need to be made to the local device. And these can be things everything from operating system updates all the way up through our orchestrating component. Just to share that you have a ability to not break the device it's very it's very hard to unring the bell once you've deployed something that you're like oh darn this this is going to to break this that that does require a truck or somebody you're visiting the device. There's a lot of mechanisms out there to essentially help achieve this. A good example is Mender.io for instance has got the ability to basically have multiple partitions so that when you deploy a new version it goes in a different partition when it's rebooted either you see or grub itself can simply say did I start up correctly yes or no and if I didn't roll back to the previous operation and also try to consider if you can to use atomic operations or transactional changes. Instead of you know deploying a command to do like an app get updates on a variety of devices even if they're all the same version and the same patch level from a previous app get update there is a potential that they can slightly get out of drift but if you can build all of your updates into a single operation that if the transact can take place completely it's committed otherwise you do that rollback component it's always good to sort of consider that also the device itself is going to those varying types of environments the car the automotive example is good that you start off in a wi-fi connection your garage so you might be doing some updates to the operating system on the car itself when you're on the road and you're doing autonomous driving or other things you may want to actually collect data that is then used for your training purposes you know further so make sure your device can understand when it goes from one state wi-fi to cellular or back again from cellular to wi-fi you want to comes back into the garage and can do the upload of all that particular data also devices are going to be operating with other things in the automobile this might be with other interacting with other ECUs in other environments you may be talking to manufacturing this you know devices that put all of their data onto a windows files here just ensure that you've got the capabilities of your device to be able to interact with other either systems or sensors you know inside of the environment that they're deployed and the other one also and I should have and when I redo the slide I'm going to put this to top security is paramount every device out there should have a unique principle that associates that device to something in the cloud so that you can identify it so when it does make a connection that it can go through a authentication and authorization process to identify what the device is and what capabilities it has from the cloud side these are normally you know supported through either X509 certificates or other mechanisms so if you can ensure that you've got some type of hardware-based security integration this could be through secure elements TPMs or other ways that if you do deploy or when you do do deploy your certificates and credentials onto a device that you can actually ensure that they are you're protected as much as possible and that they can't be exploited but if a device does get exploited or compromised and you've got detection capabilities in the cloud for that have a mechanism to either you know reset the device to a factory state or the ability to rotate credentials to give it you know a new set of of principles that it's to interoperate with all right so that that pretty much goes over you know what we see from sort of the the design patterns and how to work it you know when things go wrong but one of the other key things is please leverage IT or your OT colleagues if possible this is a common thing that we see is that if you're in a shop that is using let's say a lot of Red Hat Linux under the covers and as an embedded developer an edge developer you're using a Raspberry Pi and you're using the you know you know Raspberry Pi OS which is a DB type of environment great for development but if you're looking to actually you know manage and deploy device to the edge see if you can leverage what is currently being used internally from your own IT organization you know commonly we'll see things where you know that you know you're you're set on a specific set of Linux and you've got those skills or capabilities but it also might be that you also have got you've built the capabilities for doing you know updates consider you know are you reaching out to public repositories or are you going to be reaching into to private satellites or PPAs for doing your updates also you know if you're a JavaScript you've probably got you know a set of Java JREs that you're going to use and deploy if you can align with what you have from an IT perspective that really does help from helping manage those components now once you have those sort of identified is we also say that you know we separate sort of edge capabilities from the underlying you know OS orchestration so you know extending things like using salt ansible stuff puppet etc for managing the underlying operating system devices is is a good thing then on top of that you can do things such as you know potentially using containers that you you want to use your rancher at the edge or kube edge or some type of your mesos environment for managing your container operating environment and then if it's a serverless type of capabilities you're deploying is you know you'll use things such as you open with help take those you know servers or functions you deploy in the cloud and deploy those out to the edge so if we move up to the next level which is sort of the assets these are sort of the the value add or the business logic that you're you're adding you might be deploying you know might be developing functions and micro service based architecture or it might be a combination of doctor images and doctor composed files that deploy all of your functionality at the edge it's also the the output from machine learning so the machine learning model that you want to deploy to your your your devices along with operating system updates firmware etc normally from this you'll have an end points where your devices can reach back into to say hey do I have a new deployment it gets the manifested deployment information back and then inside of that manifest will say oh I'm getting my functions from this particular URL or from these objects my machine learning ones from this one etc also manage you know fleet management if you have the capability of sort of your managing and seeing the health and state of your devices overall if it's got a solution for that you know please use that same for your status reporting and dashboards and then as mentioned discreet device security identity every device out there should have a unique certificate or principal and there should be nothing shared from that perspective from the machine learning this normally depends on the organization but I normally see the customers will have an IT organization and sort of a data science group that may be part of IT or separately but if you can use some of the same tooling and approaches that they're using for doing data science and machine learning using those of the edge is also great if you're using your tensor flow internally great maybe use you focus on using tensor flow light or LT at the edge the same thing is that you're if you're using mxnet or cafe etc is you're using those same types of outputs or models will help you understand the you'll help you and the the machine learning people know what's you know how things are happening in the cloud and then as you're actually deploying same thing it's like you know having a process for that I create a model I deploy it to the edge I use the inference on this to actually collect data I push some training data back to the cloud and at that point I can do the retrain into the model and using you know a lot of the scale out capabilities then taking that trained model and putting it back onto our devices so essentially having that sort of you know closed looped environment for doing model deployment model or data collection model retraining model deployment etc and then finally is that and this is where it gets a little bit sort of hand wavy it depends on what your development cycle looks like in the cloud is a lot of times if you're using like you know a continuous integration continuous deployment process you're doing a lot of deployments on a daily basis but if you can sort of incorporate that when developers are you know writing code and making commits that that kicks off a pipeline that goes through essentially the entire process of taking that code creating the assets going through a testing on either you know virtual environment or on local testing devices and then getting back essentially for the faster failure of a build is that essentially can demonstrate you know a pipeline that you normally use for like you know cloud-based for web-based applications that can be applied to edge compute components too we normally see this only when you're talking from the the the assets on up so the the blue box into the green boxes but this also you know when you get into like full regression testing to be all right we're making some significant changes with the underlying operating system and we want to do a full test of of OS system packages user packages etc and of course you have different types of mechanisms for having deployments you can be one box in this you can do in blue greens fleet deployments but have the capability for for how you're doing both over the air updates to send out things but also a rollback if necessary as we kind of mentioned on the what things go wrong slide all right that's you know 40 minutes then we've covered a lot here but what I like to sort of go is this you know sort of you know wrap everything up together and sort of summarize this so we'll go ahead and start with you know should you use edge or is edge a good use case and I'd say that the sort of litmus test is if there's a capability out there that addresses one of the laws of either you know speed or physics cost or you know from a jurisdictional or sovereignty perspective laws of the land you might have a good use case for that normally what we see is that if there's any type of machine learning that's being done at the edge where somebody says I want to put a camera out there and detect when somebody walks in the door or they're not wearing their personal protection equipment or when you know things like a manufacturing line look properly do I have the right posture or not those are always you know sort of your good indicators that you've got a good edge case once you do have that you know either define and create your own or adopt existing patterns to sort of build out that full stack component so you know think about the hardware think about the operating system the potential additional operating system you know repo packages or distribution packages you want to include then along with how you're going to orchestrate and deploy all of this business or edge logic that you want to put onto the device and then think about how these things are going to be deployed you know it's very common to see these in a your test or lab environment where you connect them to a hardwired ethernet or to a very consistent Wi-Fi these work fine you then put them into a remote terminal unit on an oil well in you know Northeastern New Mexico and things don't so it works well because when the sun's up it degrades the signal quality to the point that the device can barely get a connection and it you know it goes in and out so think about those sort of intermittent capabilities and then some of the things that we mentioned in the gotchas component for addressing the device and you know I really can't stress it enough but you know you know stand on the shoulders of giants take advantage of all of the hard work that you're you know you know IT or your operational technology the OT departments have done for for how they're deploying things and how they manage things too by doing that it helps significantly reduce the technical debt that you might be creating for a sort of a bespoke or a custom edge use case but at the same time is that it allows you to also have access to other resources for for troubleshooting or for potentially you know new opportunities and also keeping up to date for when they make changes that you can actually take advantage of those also and this is sort of the the things you know from my from AWS is you know AWS IoT green grass is the edge compute capability that we have that essentially addresses a lot of these these things we spoke about so essentially addresses the orchestration the ability to do container management at the edge along with you know lambda functions are sort of compute everything that can be deployed onto a variety of different hard work components and it also handles the machine learning where you can train you know a machine learning model in the cloud with SageMaker click a button and then have that automatically deployed through a set of you know prescripted connectors that will do that without you having to write this actual code and with that I'm going to open this up to Q and A I normally if we're together I would just you know point to you or point to you and it has a question but one of my colleagues Lisa Ralph is going to send questions over to me and I'll simply reread them and give you answers back here but first with a quick drink here all right so the first question is what will be the future of edge computing will there be any more evolution and in computing platform and I think the answer to that would be well first of all I don't really know what the future does hold but if we take a look you know historically what has happened is that things that are currently out there in sort of you know an innovative type of space that you're looking from one particular angle or another but there's sort of you know a one-off type of solution or one one-off type of situation normally can either evolve into the next type of technology or they they can essentially fade away I'd probably say you know you keep a weather eye on what things are happening in the the compute or the IT space or even the embed of Linux space in general and see where things are going is you know you know 2020 hindsight but but normally you know what we see from things such as you know from from markets or from the the analysts is that the the explosion of IoT devices into the tens of billions that are out there is giving us a lot of capabilities at the edge from a sort of discrete component perspective and now we're seeing sort of that same kind of you know waiver adoption happening from your edge compute or or edge capability which isn't just compute this machine learning other types of things so a good question so the other question is there a way to retrain an object detection model automatically from data gathered from the edge of a device using SageMaker and this is specific to AWS but the answer is yes so we actually have a green grass connector which is called the the machine it's a machine learning inference connector that allows for retraining so it allows it basically deploys the the capability of the edge to collect data and based on certain inference values if it sees probability dropping or or getting you know not hitting like a high confidence threshold to be able to capture some of that data it could be it could be you know video data or other types of you know input data that can then can be pushed up to the cloud and then automated from that perspective does require some work and coordination for how you you look to create your notebook and your training model or your training approach inside of SageMaker but that is definitely doable so the next question I love this one if an app needs Python 3.6 sounds fine you know you know 3.8 is simply just rolling out so you know should 3.8 be fine unless you're saying the Python isn't good with backwards compatibility and that's predominantly exactly what it is we we see customers out there that do this you've installed 3.6 it's like all right we have things that are running 3.6 and 3.8 so let's just do a sim link so we'll sim link our 3.6 to a 3.8 environment or 3.7 whatever is hard coded to use those particular paths and 99 9% of the time things work where things normally break is when it comes into some of the machine learning components in the case like IoT Greengrass we only support Python 3.7 so to get back to 3.6 you can do that we can sim link our 3.7 to 3.6 and at that point things normally work however certain things can break when you're doing different operations with machine learning models or like with the SageMaker Neo DLR perspective so that's why I I always like to say is you spend as horrible as can be sometimes spend a lot of time working with your Python environments specifically to see what is or isn't compatible and that's also sort of a good plug or use case for that that unit testing and that full regression testing of all the capabilities of your your IoT or your your edge capabilities there. Let's see so we have another question in regards to security and identity of devices in an IoT network so the question is are there any ongoing experiments to integrate hyper ledger fabric capabilities Indie Eartha Aries what is the future of blockchain blockchain technology in IoT I am not an expert in blockchain overall it is a question that we do hear a lot from customers normally right now sort of in the innovation phase or steps what I would say or what I have seen is that customers will take advantage of certain blockchain technologies for source data and this can come into things such as like an oil and gas industry where data is measured at certain you know remote terminal units things for like you know natural gas or oil expect then there are certain values that will determine the quality of the quantity of that picture product and that's actually that's actually tracked from forth all the way through distribution so from essentially an upstream all the way to downstream and we have been customers that are taking advantage of blockchain to essentially you know ledger those types of measurements there and then along the entire process of when they're using the technology unfortunately I am not that familiar with some of the other hyper led hyper ledger fabric capabilities out there but it's something that I'm keeping an eye on too to see how that can be sort of incorporated into an IoT environment what I would say is I never I never see that associated with the security or identity of a device it's more about potentially the data that it's actually creating and then sort of the like I said the chain of trust or the chain of custody for that data as it goes through a next in actual process so another question is are there any good resources to learned edge computing and Lisa kind of answered this but go to the virtual IoT booth in the golden silver hall is normally we'd have the staffed with people like myself solutions architect where you can come and have a one on one conversation virtually you can you can kind of do that today and simply bring up your particular use cases and have sort of a you know a one on one or you know even a one on many type of conversation in regards to that and there's also a great collection on the inside of the booth itself on a getting started guide so I worked with some of our marketing people to essentially say if you want to get started with Greengrass or Sage maker or our device SDKs or just IoT in general there's some good sort of links there directing to our developer guides that will start to explain what the services do and how they operate so another question as you know how should I compare the models for inference at the edge is there any tools for that I'm going to guess that the question is more around the quality of a particular model you know is model A doing the best or is model B doing the best when you're given a certain amount of data and what probably is that that comes back to the more data that you can collect from the edge to use as training data for your data sets the better in ML in general 10 or 100 samples is probably going to over train or over fit a model quickly 1000, 10,000, 100,000 sets of data is better and so that's why I'd say is that if you are looking to deploy training to the edge a lot of times the inference itself will happen cloud side when you're going to the training gyrations but then actually when you do deploy the model you could take sort of the the blue grain approach if you've got your entire fleet that's running you know model version one and you're getting back certain values and from the you know from the cloud side you're classifying or annotating the data of either correct or incorrect and you can use something like you say you make a ground truth for that you may want to say all right for 10% of my fleet I'm going to you know deploy model version two to the to the blue you know the blue fleet if you will and see what the differences are that comes that with that if it's better you can then say all right this this go ahead and roll everybody out to version two if it isn't you at least got some good data back that you can then use to make the terminate you know for the for the next cycle and with that they're going to be cutting the software in a few minutes I'm going to take one more question if there are any follow-ups please go to the database booth and also if there are things I'll be on the Slack channel for this this track later this afternoon probably about a couple hours or so can answer questions there and unfortunately that does going to be the last quest actually no here's the last question so coming from a developing country how can one start edge computing easily and teach community to to adopt it in the right services I'd probably say the best approach for that is find something find examples that are out there in the world that have worked and and kind of you know use those you'll play around with those and understand those are but take those lessons learned and see what that actual community that you're you're looking to potentially you know educate or elevate may have problems with for instance I do your mission work in Haiti and a situation there do you think it would be like you know water or power you know types of things when the key things there was they have that solar panels that they actually charge their cell phones during the day and you know that could be a potential opportunity that if you've got cell you know cell sites you know spread across the country they're doing charging and one cell is degraded or it's not pointed in the right direction that could be a good example of of getting some of that that edge data you know the operation of you know how many phones are being charged the conditions et cetera and be able to to feed that back to go out and make adjustments or whatever so with that I'm going to thank everybody for attending I do appreciate this I really wish this was in person so that you know I give you a heartfelt thanks to everybody but it's been my pleasure to to go ahead and present your deep learnings at the edge of you know what I've gone through you know battle train for the last couple of years and please enjoy the rest of the the embedded embedded links conference and with that I think we'll go ahead and close this out


Now it's time to introduce Bob, who is going to talk about communication breakdown coroutines. So please welcome him. Thank you. So first, Mike, can you hear me? Yes. Yeah, okay. I'll try to entertain you during this lunch break. I'm going to talk about communication on coroutines. And I'm a mobile lead developer at Quick with headquarters in Finland, and I'm based in Stockholm, Sweden. So coroutines, we had a lot of talk about them. So I'm not going to go into them. I'm going to talk about the communication. And basically one of the main problems I see is think of them as lightweight threads. So we're going to see why we might not take that too literally. And we're also going to see what do we mean by lightweight? Should we treat them as threads? Because they might be. And also, let's see if this works. Whoa, it's fast. The function main run blocking, it's going to be in every slide, but it's going to be invisible. So you have to stare at it so it's locked into your eyes for two more seconds. And now it's gone. But it's always going to be there. So we're always going to have the main scope to run all the coroutines on. And by lightweight, we've seen this example before. Here we create 100,000 coroutines with the launch coroutine builder. And it takes around 150 milliseconds to run on my machine. And now we switch to threads. And it's up to five seconds to run this. So yeah, they're lightweight. Let's go back to coroutines. And now we're using a dispatcher, the default dispatcher, which comes on my machine on eight threads. It's usually the number of cores on your machine times two. So it's very effective. And this takes around 800 milliseconds to run. And if we switch to IO, that doesn't really have a limit. I cranked it up to about 84 threads on this machine. And the more threads you're going to use, the more time it's going to take. And how about thread safety? Dispatchers, you can think of them sort of like thread pools, but not really. There are rules deciding on where you can run your coroutines. And thread safety, just a quick show of hands. Is this thread safe? Yes, show of hands. No, it's not. Because we run on possibly eight threads and we have a shared mutual stake. And the same goes for IO. It's just another thread pool-ish. How about now? Remember that function main, run blocking, we're running on the main thread. We launch 100,000 coroutines. But this is actually thread safe because we all run on main. Because coroutines inherit the dispatchers from their parents. But you might have a co-worker who does this, and now it's not thread safe anymore. So we have to be careful. And just to make sure, when I started learning Java a couple of years ago and threading a bunch of years ago, I actually thought this would help to add a volatile keyword or annotations to it. It doesn't. The volatile only makes sure that you don't use the cash value. You read it every time. But it doesn't prevent anyone else from reading the same value and writing the same value. So volatile doesn't help us. So can we treat them as threads and define? Someone mentioned the unconfined dispatcher as well. And I think it should be whatever. Instead, renamed to whatever. I just follow along. Because what it does, here we have our function main. So we're running on the main thread. And when you get into a coroutine with the unconfined, it says, okay, you're on the main thread. I'm just going to tag along. I don't care. So the first statement, print a1, is going to be on the main thread. And then we do a delay, which has a different dispatcher. And when it comes back, the unconfined coroutine is going to say, okay, you're on a new thread, new context. I'll tag along you. It doesn't preserve your context. So the a2 is going to be running on a different thread, or possibly often. And unconfined is for a corner case. I haven't seen them being used in production code yet. But you can have a similar result with this example. We launch on the dispatcher IO. We print a1 on thread worker1, say. Then we call expansion functions, which declares a width context and a different dispatcher. And here you can possibly end up in a different thread. You are going to end up in a different context, which can switch thread, and you can stay in the same one. And when you come back to the coroutine from the suspension and print a2, you can just follow along the other thread. So you might get this, and it's perfectly safe, but you can also get this. The thing to be aware of is that after the switch context, you're going to continue on that thread instead of coming back. So threads and coroutines aren't exactly one-on-one match. And just to make it even clear why it's kind of confusing to think of them literally as threads, is if you have a thread local, which contains your value on a thread. So if you switch thread, you're going to have a new value that you can share on that thread. So given the same example, we can have, or we set the local to IO first, then we read it. We call switch context. We read it again. We set it to default, and we come back and read it one-third time. So we can have IO, IO default, meaning we're all running on the same thread. The thread local stays the same during the entire operations. But we can also switch threads. And now thread local are thread safe. They're going to stick to their thread. And this is proof of that, because we switch threads. And when we're coming back to print A2, we're on the new thread with a new thread local value. But it reads wrong in your head when you have a coroutine with different thread local values. So I would suggest just for your own sanity to not combine those, because you are not... It's not readable code to have this difference. So we should treat them as coroutines. And another example from Dan Leb is if you use synchronized. This, actually, when you have a thread like this, or we create two threads and call a synchronized function, we're going to have starting, ending, starting, ending. The annotation is going to help us synchronize the code. So only one thread is going to be allowed in it at any given time. And if we change to coroutines, we launch two coroutines and we call the same function, but now it's a suspending function and we do a delay, this is actually going to print out starting, starting, ending, ending. And to understand that, we have to understand how suspending mechanism works in coroutines. What it actually does on a high level is when you call from a coroutine, when you enter the critical section, it's going to acquire the lock. So it's going to do the same thing. It's going to lock the function. It's going to print starting. And then it hits the suspending function and going to put the state into a continuation and suspend it. And then it releases the lock. So it actually divides this function into two. And when the suspension is done, it's going to acquire the lock again and print the ending and then release the lock. So that's why we can have this order, because during the suspension, the other thread comes in and takes over. So never use suspending with the synchronized annotation. You can do it if you don't call any suspension functions inside of it, but it's not safe. Someone will put the suspension function in there eventually. So let's do the communication like we should in coroutines. The third, we mentioned it and talked a little bit about it. It's often used with the async builder, which launches a coroutine and the last value of the coroutine, or if you return explicitly, it's going to be the third value. It's kind of like a future. And when you need the value, you call await on it, and then it's going to wait until the async block is finished and return the value for you. And the async is executed directly. Just to prove that we have this code. This entire block actually takes two seconds. The second async block is going to finish before the first one, but when we await the results, it suspends, so the second await won't be called until the first two seconds are done, but that suspend is going to release directly, so it's going to be like a regular call. We could also do this more manually with a completable deferred that you actually control yourself. So you don't have to use the async builder. You can use, like, an actor or just a regular launch on a coroutine. And what you have to do then is you have to call the complete on it to say this deferred is completed. It's done. You don't have to await anymore. This example is just to show that even though the third or completable deferred is a safe way of communicating, we still can't share a state because this can altering the object that we send to complete after we send to complete will actually still alter the object. So it's not safe to assume that whatever we complete is going to stick through forever, so we should use a val instead we don't want to do like this. But one thing that's good with a completable deferred is that if you call complete multiple times, it's only the first one that's actually going to complete it. So in this code, we're always going to have Bob sent, never going to have Charlie. You can still call it and you can call complete how many times you want. The first one is going to return through that it completed the deferred. All the others are going to return false because they don't do anything. And just to be explicit that it's communication between coroutines is perfectly safe. Second one we're going to talk about is channels and they provide a way to transfer stream of values, deferred are for one value, channels are for multiple values. So let's get familiar with it. Here we launch on the dispatcher default and we send two values on a channel and the send function is a suspending function. So it's going to suspend until someone calls receive on the same channel. And here we send two times and we receive two times. Just to be clear, this code will never terminate because we call send and then it suspends it will never get to the receive line of code. So we have to do it on different coroutines to be able to complete the code or we can alter the channel by adding a buffer. So now we have a buffer of one. So you can send to the channel is going to buffer the value and release the suspension. So there are a couple of different types of channels. We get the buffered one. You decide how big the buffer is going to be before send suspense. And then we have unlimited when send never suspends. You can just keep on hitting it and it's going to be like essentially a blocking queue. Conflated, we talked about a little bit in earlier talks. That's actually going to store the reason value. So send will never suspend on this one either. If it has a value, it's going to replace it. So it's always only going to keep the latest one. And if someone receives that value, it's going to be empty. So receive can still suspend until there's a value there. And rendezvous is the default way. One send and one receive function, they have to meet to transfer the value. Yeah, and also there are terminal operators or functions to channels like to list here. That's actually going to wait for the entire channel to complete and then we're going to make it a list. And this won't terminate either because it doesn't know if the channel is done sending or not. We have to close the channels to be able to have terminal functions on them. So let's see where they're excel. There's a pattern called fan in. You have many producers and only one consumer. So here we launch two. We launched two coroutines and we're going to send them to a race suspended function. That's just going to randomly release them in zero to five seconds. And then we're going to have one coroutine that's listening or receiving these functions or these channels. And yeah, we can either get Charlie Bob or Bob Charlie. It all depends on the random function. The other way around is called a fan out where we have one producer and many consumers. Like if you want to do concurrent work on a huge list, you can put them out through one channel and have multiple coroutines work on it at the same time. Here we just loop 30 times and send 30 items and then we close the channel. And then we can actually iterate over the channel with a for loop. So we create three coroutines that all have a for loop and we're going to get a result, something like this. And what I've noticed is it isn't always ordered this list. So I don't know why. If you put a delay in the suspending function, it's always ordered. So I have to look into that. We can also have, there are builders for these kind of things like a produce builder that actually creates the channel for us. You can call send instead of just send instead of channel dot send. And it closes the channel once the closure is complete. And you can also call consume each on the channel instead of having a for loop or multiple receipts. So it's going to keep on consuming, receiving until the channel closes. Next up is the mutex, mutual exclusion printed for Kotlin. So if you remember this code, it's not thread safe, but we can make it thread safe with a mutex. It's kind of a re-entrant lock. So you can lock a mutex, you can unlock a mutex and this function with lock actually first locks it, then it builds a try block and inside the try block is your code. And on the final block is going to unlock it. So it's a safe way of using these locks. And now this code is thread safe. It's fine-grained and it's custom, but it's safe. The same thing is with the synchronized example. We can actually make that work as well by using a mutex instead of synchronized. So this actually works and it prints starting ending, starting ending every time. Still have some time to go over flow, which is the Kotlin way of reactive streams. And it's kind of similar to channels, but not really. You have an emit function instead of send and you have collect or other terminal, but the most common one is collect instead of consume each. But they're basically doing the same thing. The big difference is that a channel is hot, meaning you have a coroutine behind it or multiple coroutines feeding data that is active all the time. A flow is actually cold until you call the terminal function or a terminal function on it. So it's not going to do anything until you collect and you can collect it multiple times and hopefully get the same result. You get the same execution. The result is up to you. So here we get value 1, 2, to 10. And we also have operators, like coming from the stream world, like filter, map, and also extension functions like here we have a range as flow. You can have a list as flow. But the more important thing about communications with flows and threads is that the collect or the terminal operator always determines on what context we're going to run on which dispatcher. So now we still have that function main run blocking. So we're running on main, both flow on main and collect on main. But usually we don't want to do that. We want to have a background job. So we can add the flow on which decides all the preceding operators that don't have their own context is going to use this one instead. So in this case we're going to flow on a worker one, but we still collect on the main thread or main dispatcher also. And just to show that it's only preceding, we add the map on print line as well. So map on is on main, filter is also on main. If we want to change that, we have to move the flow on. So we can move it down to under the map and now everything about it is on the dispatcher default and in this case on worker one. We can also just jump around if we want to. We can have multiple flow-ons. And one final thing, the width context inside a flow is, don't use it. You can get away with it if you're lucky. If your collect or terminal function is on the same dispatcher, you might get away with it. Otherwise you will have a runtime error. So use the flow on it to preserve context. Thank you. Questions? Are you going to have available the slice? What? Are you going to have available the slice? Yeah, they're available now at FOSTA. Thank you very much.


All right, let's uh, I want to start this out with some interactive, you know icebreakers, so Can I have anyone raise their hand if they've worked with GDPR before? Okay, a lot of you keep your hands up Keep it up. If you've worked with CCPA How about CPRA Know what it is or CPA or Or you see PA All right, no hands anymore. All right, so these are the things we're going to talk about today And I'm really excited to be here with with you all and talk about marketing needs privacy I'm going to save some time at the end for questions Make sure we have time to do an interactive conversation about anything that we talk about But I'm just going to go ahead and dive right in My name is John Doyle president founder of digital polygon And the first iteration of this deck was built in collaboration with Rick Buck He's the chief privacy officer at wire wheel their privacy company product company And our information is here and we'll be on a slide deck as well He couldn't be here today, but still want to give him a call out for this So what are we going to talk about today? There's really three key buckets I want to I want to talk to you guys about the first is understanding the current privacy landscape in the United States It's changing really fast and it's going to go into effect starting early next year With enforcement being applied and it's really important that we it's why I've been marketing teams Start to understand what's coming so that we can prepare maybe have these conversations otherwise what happened when GDPR is going to happen and legal is going to come to us two weeks before enforcement happens and say Why isn't this done get this done right? So, you know, I want to start preparing people for what's coming and how we can Think about this from a website perspective and interact with With these other teams that we have the second is industry movement. So talking about Big industry and how they're responding to privacy this is more just to reinforce that Privacy is here to stay. It's not going to change and speak become a part of our daily lives Which I personally think is a good thing as an individual It's a really hard thing as a marketer and as a business owner as our life is going to change Over the next couple years if it hasn't already And then lastly, I'll talk about how privacy in your website need to interact so that we can frame the conversation we have with the different members of our teams our legal teams our privacy teams And our downstream vendors even depending on what we're doing And then I'll leave you with some useful tools and triple modules and things that Could help you in this process and we'll open up for Q&A at the end So jumping right in this is the IEPP's state legislation tracker This is a great resource available on the IPP website that kind of gives you a high-level view of the United States What bills are in progress which ones failed which ones were approved? And at a high level 33 states have introduced 62 bills as of about two weeks ago This was updated on the 7th April 7th for IEPP's big conference. They hadn't DC about two weeks ago Five have passed through four states, so California has two laws Colorado Virginia and Utah have all passed in the last six to twelve months Utah being the most recent just passed earlier this year and There's 29 bills that have died and 28 bills that are still up for some state of discussion In addition to this, you know GDPR is a big one that everyone knows about but China passed PIPL Australia passed the Federal Privacy Act Brazil passed LGPD and more and more countries are starting to Jump into this I Don't know to be honest I'm Assuming both have agreed on some of these but I haven't followed the federal side of things It looks like more liberal states when I look at the the chart here, but Something to definitely dig a bit deeper on yep Another view that they have about the IEPP is the US state privacy legislation tracker This gives you a list of every bill that's gone to committee And it really gives you an idea a high level view of what Each bill includes and you it's not meant for you to read I'll go into a little bit more detail, but this breaks it into two sets of rights You have consumer rights Which is what consumers have the right to do when they come to your organization to your business and then you've got your business obligations So this is What are you obligated to do as a business to protect your users and we'll get into this a little bit I'm not going to dive deep in the laws for this for this presentation But you can see here in the list Some of the different rights that people have and we'll focus more on the website of things And and how our website has to support these Rather than going into too many details here, but if we break this down to the laws that have passed We can see CCPA CPR a CDPA CPA and UCPA or the five With the states and you can see that there is some consistency here, which is great for us as implementers of These privacy laws, but there's also a lot of differences that we need to take into account If you've looked at any big sites recently, you'll notice that there's always a California resident policy at the bottom there's a do not sell button now and These all came out of CCPA CPRA is going to extend California's Enforcement and laws and Virginia and Colorado kind of built off of CPRA. So those are somewhat consistent But they're still going to have one offs everywhere Go ahead Sure, I can I can do my best CCPA is California consumer protection act privacy act CPRA is Calus California Privacy Rights Act I believe CDPA is consumer data protection act CPA is is consumer protection act or privacy act and then UCPA is Utah consumer Privacy Act And if I got any of those wrong, they are on this slide in more detail So I can send you a link to the tracker afterwards And the the biggest difference here that we're gonna Notice is that Utah which just passed is way more business friendly Than the other three laws and it has less less user rights that you have to comply with and that's going to make our jobs even more Confusing as we go into a web and marketing implementation that now I have to deal with different Not just by country different laws, but by state and how do I track this? How do I know where you're coming from? How do I know you're a California resident if you're on an IP address in? Maryland and This these are some of the challenges that we have as web and marketing teams to to look forward to here Coming shortly Privacy changes are moving fast. Like I said last year we started with CCPA now We've got another four laws that have passed and there's two more that are in cross-committee, which is What you're looking like they'll get passed here in the next six months things could change of course and What I'm looking forward to at some point is a federal privacy Implementation because that should help standardize this But I don't personally see that happening this year maybe next year But something to keep an eye on and you know the IPP resources are a great place to to look for that so when we look at September 2021 to April 2022 we take the same graph and you can see that the color coding is extremely different This is how fast bills are introduced or killed and move forward And I'm not going to spend a lot of time on these slides. I just kind of want you to see How fast this landscape is moving just in the last six months? You can see how different the sizes of these charts are Just in the last six months. We've had more than double the bills go into a cross-committee or above with new states throwing laws up basically every other week and a lot have also died in Committee or somewhere through the process And you can look at what these laws included in this table in more detail if you're interested in getting getting more details on it Right, so one really important part of this is when do these go into effect? So these laws were passed in the last two years and now we need to look at okay When does enforcement begin because that's really what our timeline is as web teams and marketing teams to implement these changes The answer is January 1st, 2023 California Privacy Rates Act and Virginia's Consumer Protection Act go into effect on January 1st, 2023 Some of the guidelines for enforcement aren't even out yet Last I heard we're expecting those sometime in September So that's likely when you know your legal teams are going to start pinging you and if you haven't done any prep work Then your Q4 is going to be Probably a little hectic if you have to comply with these laws Colorado Privacy Act is going to be July 1st, 2023 and then Utah is going to be the end of 2023 and December 31st So this is kind of what the timelines are looking like we'll see when new bills get passed how they add them in and how much runway you have but This is this is kind of what we're working towards here with with most of our clients And on the privacy law side, you know privacy is here to stay There's more international laws. There's more us laws. This isn't gonna, you know, we're not gonna wake up tomorrow This isn't going away. So we need to start preparing for it as Web and marketing teams and we need to start building foundations for being able to support this And the new laws closely resemble GDPR, but everything in the US From a marketing side is opt out Whereas everything in GDPR is opt-in. So there are differences here and how you have to act with these different laws And that means that, you know, your marketing team Or your sales team is definitely not going to want you to throw a GDPR cookie banner on your site Stop tracking everything if they don't have to so now we have to get way more granular with how we implement these solutions how we target users in different locales and How we work with our downstream vendors as well So with that, I want to shift a little bit over to The marketing side and the tools that we use and I really love this quote by Tim O'Reilly because it's who has the data Has the power, right? We have the data that we use to personalize people's experiences to give them a better user experience to inform decision-making and To really improve our products or features. We use this data for tons of different things It's our responsibility Now to be responsible for that data a lot of times as marketing teams We're using tools that we have no idea what else they do with that data We're not reading through the 60 page contracts to say How else are they using the data that I collect when I put this marketing pixel on my site or this heat map on my site and these are the things that these laws are starting to push and protect and You know, it really is up to us and up to these vendors to start Taking responsibility for this data. We have to protect it and and to really respect our users rights to their data So with that, let's look at how the industry has started protecting their data in in the marketplace And I like to show this because it'll it'll give you a good idea of how much privacy is impacting business And if all of the Big guys are doing it. It's definitely going to trickle down to the rest of us, right? And it's going to impact what we do. So the first thing I like to talk about is the rise of privacy browsers You know, they're still don't have amazing market shares. They're not going to overtake Google tomorrow But they do exist and they are being used and a lot of the reason that you can't see how much share They're taking is because they actually block giving you access to what browser they're using right privacy first is what brave and duck duck go do and This is already going to start impacting your ability to market As more and more people start using these tools One thing that I pulled from IEP last week, this isn't even out yet by Google Probably the best release I've seen around privacy for Google is putting out this privacy guide and Google Chrome has a lot of privacy features In the tool, but this is the first time they've provided a nice user experience to guide people through how to use that So this is going to start impacting our ability as marketers third-party cookies and Other things like that because it's going to allow users to more easily go and manage their settings in chrome Third-party cookies are going away. Obviously this has been in the news all over the place for chrome They pushed it back to 2023 But it's still in the roadmap. It's going to disappear something we need to be aware of and start looking at different tech stacks to deal with The end of Google Analytics 3, this is again pushed to July of 2023 If you're not on GA4 yet, it's probably time to start looking so you have data The collection will stop in July of 2023 They won't delete your data for six months is what they're saying right now But you can run GA3 and GA4 at the same time right now So there's really no harm in you starting to get GA4 set up now while you start to learn to use it If you have played with it, you'll know it's a bit different than GA3. It's not just Plug and play so you want to prep for this well ahead of time And Apple is one I love to talk about tim cook actually keynoted iapp two weeks ago And some of their privacy features here if you've seen the headlines are costing facebook what 12 billion dollars is what they just said So these are going to continue to impact our ability to do business So mailing privacy protection. So if you sign up for Uh an app on your phone if you have an apple phone It'll now actually put in a fake email address that proxies back to your real email address. So Google I mean the apple is the only one with your actual personal email And when you sign up it it uses fake emails so that if anyone else has a data breach You're not impacted in that way And uh, sorry that was the hide my email the mail privacy protection if you use Apple mail it'll actually remove all the tracking pixels from your emails So I don't know how many people here are marketers or salespeople that have tracking on their emails where you can see when people open it and see click through rates and open rates, but Apple's mail system actually blocks these so that people can't track what you're doing in it and you have to use their app for this to work, but um, that's really going to impact our ability to calculate metrics to you know figure out if our campaigns are working or not and You know, these are the things that are coming and safari, of course has been blocking third party cookies by default since 2020 chrome is going to catch up next year But these are definitely important things for us to think about and us to Start adapting to And hopefully it gives you an idea that again this privacy thing is not going to go away Just going to continue to impact us and if we don't change We're going to be left behind Right, so we talked about kind of the current privacy landscape in The united states we talked about some industry movement now Let's talk about how this impacts us and how we should start thinking about this as website owners and marketers on our websites I like to break this down into three simple buckets Uh, really with all of these laws from a website perspective We need to provide information to users. This is telling people what we do with their data You probably already have privacy policies you have in there What cookies do we use? What information do we collect? How do we process that information? Some of these laws go beyond that. What are my rights as a california resident? Um, that's what providing information is all about The second bucket is facilitating requests So as part of these laws I as an individual if I was in california Have the right to request access to my data that you have To delete that data To rectify my data There's some portability laws in there So you as website owners, especially now that coveted is here Your website is really the place where people interact with your brand No one's picking up a phone and calling you most sites like organizations don't even have phone numbers to call for this So you need a way on your website to facilitate these requests Can be as simple as an email in your privacy statements can be as complex as a fully automated forum That sends tickets down to your downstream systems Um, but uh, your website really needs to support that and then managing consent is the last one. So this is Do not sell is cookie compliance um, do not sell my information and, uh There's a automated decision-making consent in there too. Um, so Uh, these are the things that you need to provide your users with If we take a look at it as more of a logical diagram here We're not going to talk about email and social too much today Although that is likely part of your stack But your website sits on top of all of these tools that collect data that you pass data through And that you interact with on a normal basis And a lot of these laws are going to require you to take those preferences that users set and make sure that all of your downstream systems respect them There's also this idea of being able to keep a log of when a user opted in or opted out so that you have this audit log so that when Some lawyer comes and tries to sue you for this information because this user opted out Uh, I say well, we have it here that they did or we have it here that they didn't and and what went wrong and what went right So something to think about, um as we're Managing these consents and collecting this consent is how do we prove it? So I won't go into more detail on all of these, uh for here but I do want to actually look at some examples of Websites that do a good job of providing this information and Really big industry has already started complying with the laws that are out there and are good places to look for examples If if you need some inspiration This privacy and security center is actually home depot I think they did a really good job of breaking out their information They've got tables for displaying what cookies they use grouped by category And what they do with each type of this cookie and how they process that data who they share it with And this is about saying what you do if you're open and honest about what you do with the data You should be fine One thing to note is you are going to Change what you do with your data as new tools come on as new products come on as new processes open So this isn't a Set it once and forget it. This is I need to build a process around this with my team So we're interacting on a regular basis To keep this up to date and keep it relevant And that's again, probably going to fall on the marketing teams because they own the websites Not something that we we typically do right now Facilitating requests. This is an example of a a dsar form in the industry's Data subject access requests is a lot of what this realm is called So delete access rectification portability. There's some opt-out category access requirements under ccpa here and There are tools and vendors out there to support you if you want to automate these dsars But I've seen plenty of organizations just provide email addresses and run it through a manual process behind the scenes Large organizations. I believe they passed a law last year a requirement that um They have to actually publicly post how many dsars they got How fast they were turned around in and how many they approved or rejected and make that publicly available to everyone So you can actually see uh, how fast they're turning these things around and it's taking for them to execute these requests And we'll see more automation come in into the market as as things pick up here Um and lastly managing consent. Um, again, these are cookie notices. These are do not sell opt outs and One thing that I thought was pretty cool is this global privacy control We'll talk about it more here in a minute, but uh, This global privacy control is a open standard that they're trying to push in the united states to say um I've chosen to opt out of do not sell and way for the rest of Uh companies to respect that right now. It's very granular. It's all browser based and There's no way for you to universally opt out unless someone respects something like this So if I go to one website and opt out there I go to another website I have to opt out there if I get open a new browser or my phone. It's all very manual and segregated um, so finding a way to centralize this and Allow users to choose is what this is going after How many companies actually pick it up because it hurts their ability to market? We'll we'll see But it's out there and it's definitely worth taking a look at And with that there are tools to help you Drupal modules Open source enables everyone. So I couldn't come to Drupal con without pointing out some Drupal modules that were here Cookies consent management and gdpr are two good ones to get you started You will see disclaimers on all of these modules that say installing my module does not make you compliant This is something that a lot of teams don't fully understand Just by installing a tool and putting a banner up doesn't mean I'm compliant with these laws And this goes for a lot of like third-party consent management tools too If I've got a tool that enables consent But if I don't hook that up to my tag manager or use their tag blocking scripts or something like that I'm not actually compliant clicking that button does nothing So I've seen a number of implementations that have kind of fallen off the rails because of that So it's important to understand a little bit deeper about what this tool is doing for you and what you need to do yourself And there's a number of vendors that can help you a lot of these vendors have Drupal modules to integrate their third-party products to help you Of course wire will is is a great one They should have their Drupal module out I think last next week, hopefully So we're creating that for them and we should get it out user centrics and consent manager.natter to cookie consent tools and all the big cookie consent tools have GDPR and then they have additional versions for ccpa And they'll have the other laws being added as they go into enforcement And they'll they're taking care of a lot of the geolocation and the different Nuances of each law so that can help you really get Some some speed out of their tools if if you try to choose to go with a third party to support you But you can do it yourself if you have a team willing to do that Um, again the gpc, I won't talk about this too much more But there are also chrome extensions for duck.go for example I don't know how many of you use that but it'll block marketing scripts. It'll enable gpc Um, and the first time I heard about it was this tweet from the old attorney general from california um, and like I said It's not at a mainstream level of adoption yet But I think anything that sets a standard that we can comply with so there's not 5,000 different ways to do it is a good thing in my book so Highly recommend taking a look at it and then I couldn't leave without talking about the daa The you your ad choices if you go to privacy policies around the us you'll see Links to your ad choices and almost all of them There's not a great way for you to tell downstream vendors to change the way you adapt information and This is kind of how it's done right now. So there's about 126 supporting organizations in the daa And if you click on the link you go there It'll give you a list of all those it's like facebook google etc You can check out which ones you don't want to retarget you and automate decision making for you And save it and that sets some cookies on your browser that should limit the way that they use your data You as a marketing team can't really control this So it's it's an interesting thing that's going to continue to happen over the next couple of years. I think with vendors Supporting this or not facebook released ldu limited data use. I don't know if you're familiar with that To allow you to to flip a switch on how this user is being retargeted to for ads with facebook But it's all over the map right now and For the your ad choices I go on my phone. I go on a different browser. I clear my cookies. It all resets So it's it's just something that us as consumers if we're going to do this need to understand And like I said, it's a pretty complicated market here Um Google consent mode is Primarily targeted at the us if you've done gdpr. You're probably familiar with the tcf 2.0 framework Google's products are already compliant with that framework But if you're not implementing gdpr and you need to implement a consent mode Google's consent mode is a good one to set up with tag manager And it'll it'll pass this data through to The google products to change how The consent status things for google ads happen. It'll change some of the way google analytics work with what data it actually collects To make sure that you're respecting this These policies that are in place for this So highly recommend taking a look at it if you're not doing gdpr and you need to comply with these laws It could be a good way to get you started Um right with that so key takeaways I just want to reinforce that privacy is not a button At the bottom of your screen It's it's that's doesn't make you compliant Your organization Has to be responsible for understanding these laws and what's coming Are you're going to be hit with enforcement? I'm sure there's already law firms lining up for the january 1st enforcement Just like they did with gdpr So it's important to get ahead of this secondly I took this from rick If you do what you say and you say what you do you should be fine Um, obviously work with your privacy teams and your legal teams to make sure that You've got these things figured out But uh, it pretty much is as clear as that if you're transparent with the data you collect You know how your sub processors are using it You know what your organization is using it for And you can say that and then you can protect that data and choose what you do with it Uh, that it really is as simple as that Um, and then you know ending it with giving Users the power to choose Um, that's really what it comes down to I as a consumer should be able to choose how you use my data Um, just like I do if I walk into your store Kind of but yeah, so Um, all right, so I want to leave you with with one thought Uh, together we can build a better internet. I truly believe that privacy is is a new pillar to building foundational websites Right next to accessibility right next to security It's it's our job to protect users data and their rights. And I think if we do it together The internet will become a better place I'll leave you with slides with some resources to those IAPP diagrams I had Also wire wheels privacy law comparison table They have a interactive table that has all the different privacy laws and a lot of information Comparing them. So if you're interested in digging a little bit deeper, uh, it's a really good resource for that And with that, thank you all for coming and I'll open up to questions and we can keep talking Sure So the question was why wouldn't I just take the most stringent standard and implement with that The answer is probably because your sales and marketing teams would get really really mad at you. Um You know, I if we're being good stewards and doing privacy first It's a way to do it. But when you're talking about business and you're talking about revenue because it has direct impacts Uh, it becomes a much harder conversation within your organization That is what i'm seeing around the market. Yes, it's a different policy for every place and we're doing As little as we have to to be compliant with the laws um Hopefully that changes with the federal regulation and we can be more consistent But um, we're just not there yet. So it's really up to your organization. How you want to interact with this How do you want to get ahead of it? And you know, if you think about, uh, first party data versus third party data Maybe you have some more power if if we go away from the third party data tools That you can build trust through users and and gain that consent back yep So that example, uh for the desar forms was actually an example of wire wheels um It was their desar product and uh, yeah, if you take a look at their website, they've got demos and interactive content for that I could probably send you a link of a few sites afterwards Sure. All right. So the question was, uh, there needs to be a connection between legal and marketing teams and Is google or anyone else going to put information out to help facilitate that? Uh, I doubt google will. Um, but I think a lot of these, uh, privacy product companies and privacy management consulting companies Uh, we'll help you do that and with the business obligations. There's also privacy impact assessments and Uh, things that organizations are forced to do. Um, which they also help with and that's that's about mapping your data and Doing assessment of how you're storing personal data and how much risk is associated with that Uh, that you're required to do under some of these new laws Sure. Yeah. So the question was, uh, How do we comply with these laws with users who Like how do we tell where they live if i'm a virginia resident or a california resident? But i'm in a different state or a different ip at work. How do I, uh, make that decision? and, uh The answer is i don't have a good one. Um, you know in your on your websites, you'll see like Statements at the bottom of the website for if you're a california resident you get these benefits you can execute these So for these are processes, uh A lot of the automation frameworks for validating this user is who they say they are Which you need to do if you're going to give someone access to all the information on them There's a lot of like, uh, k y n validation processes. You have to upload like your id Or things like that to validate. You are uh, actually who you say you are Um, so that's one piece of it for actual do not sell consent Um, I haven't really seen good enforcement Uh parameters on that a lot of people are geotargeting, uh, california, but again, it doesn't capture everything so, um You know, I I don't have a good answer on that. It's it's really up to your legal team on how Uh, they want to control that and how much worse they're willing to accept and do we just allow anyone that wants to click do not sell To be enforced or not? Um, or you know, is it only if you're in california not using a bpn, uh, you know It's it's a legal decision in my mind Based on risk Does that answer your question? Okay anyone else No, all right. Well, thanks so much everyone. I really appreciate it