If we can. Okay. If there's no one there. Yeah. Yeah. I think it's going to be a popular one. That would that be crazy. People can join online. We have 50 people online already. So we're, we're, we have quite a crowd here. So this one might go slightly over four o'clock. Two or three o'clock. Fantastic idea. So we're just going to, we're going to do it in here. One moment for people online. Okay, sorry for the, sorry for the, the mix up there, guys. We, we have a very large crowd for this presentation. So I'm going to go ahead and start presenting and the other group is getting set up in the other room to watch it live from there. And we will just get started. So it seems like this is a popular crowd, even though most of the people are no longer in the room here with me. But we're going to talk a bit about how to build a public dashboard and still sleep well at night, which someone told me today was a clickbait title. And I think that's probably true. But it's also important because there are, it's, it's tempting to build public portals. There are a lot of public portals out there, public access to data. But there are a lot of considerations that need to be taken into account when doing that. So we'll talk about a few of those. There are many more, but there are a few of the, the really critical ones and security is obviously of, of big importance and performance is another one. And making sure that you're doing this in a way that doesn't compromise or expose access to your operational DHS to instance or your operational system, whatever that happens to be. And so, just to specify what we're talking about here, we're talking about public dashboards the first of these, but I'll talk about a couple of these others as well because they are often kind of lumped into the same bucket. So public dashboards, the way that we're talking about them today are read only. They're accessible without any authentication on the public internet. And there might be some cases where you have a public dashboard that is only visible to a limited set of people on an intranet or maybe behind a password or something like that. The general idea of a public dashboard is that it's, it's for the public, it's for everyone right and COVID dashboards were very popular in 2020, 2021, where every country was publishing all of their data for the number of COVID cases every day. And everyone in the in the public was was refreshing every five minutes right to see what the latest numbers were. So that's what we're talking about here today. And this is talking, targeting the general public so potentially millions and millions of people that's important to keep in mind. And, and they're not, it's not targeting experts so it's not, it's not an operational system it's not a statistical system it's not something where you want, or you need to expose really sophisticated tools or want people to be trained how to use. And this is as opposed to a couple other use cases or things that are often bundled into the same concept. And when we when we talk about public dashboards or public portals. Sometimes people assume that we're talking about some other things. And those are just going to cover them quickly here because we're not going to talk about them today. Those are data exiltration which there's a couple ways to think about this one of this is push analysis, which is supported in DHS to a little bit of an antiquated support for that in DHS to but there are other ways to get data out of the system. And in particular to get data into the hands of to basically where people are anyway so a common request here that we won't touch on today because there's a lot of nuance and challenges with it is you have a minister who wants to be able to see their DHS to data. It's not data that you want to open to the public but the minister doesn't want to remember their password, for example, that comes up quite a lot. So that's another example of trying to get in a get data into the hands of people that should have access to that data without basically removing some barriers from from them seeing that we're not going to talk about that today because that's not the purpose of public dashboards.  That's another thing we're not going to talk about here today but is often confused here. So that is a whole nother can of worms a whole nother thing that we could talk about another time. So that's the main thing we're going to talk about. Everybody take a picture of this and scan that please and if you know of any public dashboards. Use this QR code to enter a simple form and at the end of this will just kind of list them out. These are public dashboards right so they should be open to the public. So what we're trying to do is just get a list from the hundreds some odd people who are in this session. If you know of public dashboards particularly those that are have DHS to as one of their data sources and go ahead and fill out that form and and to start us off actually we're going to give a quick an example of a public dashboard that was developed by the his Vietnam team for a while, and I'm going to turn, yeah turn it over to my colleagues from his Vietnam to do that presentation quickly. Okay. Hello. Hi. Good afternoon everyone. So my name is Sam. I work for his Vietnam but based in law, supporting Minister of Health to do the implementation of DHS to allow. So, for me, I will talk a little bit about the, how we make the loud dashboard. So I will just go to the overview the goals the aims and the use case then my colleague. Yeah, we will talk a bit about the technical part. Okay, so. Okay, so I'm like Austin mentioned most of the goals of the public dashboard is for us is also more or less the same. So we want this dashboard or not we want but the minister or the high level people, they need to access to data. So they don't, we already gave them the username password but sometimes they cannot remember they have problem accessing and most of the time they don't use laptops like us every day so they rely mostly on their phones and their tablets. So we come up with this idea so to help them to get access to data. So first of all, this dashboard that I'm presenting here is mainly about for decision makers for the high level people to be able to access from their phone. So the requirement is quite simple so that they don't need to log in so they can just go to their website, get the data immediately. And the second one is also, since they don't have to remember also where to get the data so we want to have one site. And then go through all the data that they need that can change the, the, the web just to see other programs. So the public dashboard will get data from multiple right now we get data from multiple instances but in the near future next phase we might need to get data from different sources. And as I mentioned, so it should be a mobile first so they can just view from their phone or tablets no need to use the laptop and then almost real time, right now we realize on the analytics so at least they get one day behind data. And most importantly, the graph the chart in the dashboard to tell stories to them immediately, so they don't have to ask, ask or ask each program like what does it mean, at least they can see and they can tell what happened with, with the data or with the program. So that's the goals. So in for the approach. So once we know what we need to, to have or what we need to build for, for the ministry. So these are the approach we do so, because in law we have different frameworks we have different indicators targets, we based on this one and most of the indicators on the dashboard are derived from here. So in, in last one thing that important is the national assembly they have 10 indicators for embers to monitor and also to report to back to national health assembly national assembly. So this year 1011 indicators we try to capture that in the public dashboard so that at least they get some update every month or every quarter. Apart from that also you will see SDG and priority programs, although we cannot get all the data here but at least we get some, some idea or keep them some ideas of how each indicator that are doing. Once we have the list of indicators or the areas that we want to show, we also work with each program, we want to work with our partners like the ratio, UNICEF, PSI, and also program in the ministry like MCH, EPI to define what can be shown and what it will be useful for the high level or decision maker to see. And the last one is also about making this as a living dashboard so we, we just want this dashboard to, to the high level people last last month, but we will continue to improve. Now we get the dashboard to many of the program and the users then we will get feedback and we will add more and try to improve. And now last one is just about the next step. So with this dashboard now they can easily get access to data. We will also use this tool as the one of the way to introduce or to encourage the data use culture in the country now that they don't have to use the user name and password to login they can get data at the hands and then it will promote the data use more. And we also plan maybe this to use the dashboard for the MND work so that they can also get the high level view of the program here. And I also mentioned now the data are from DH2 instances in the future maybe we want data from more sources for different purposes and at least they can have some information like a primate or maybe the for flood monitoring etc. And lastly that Austin mentioned but for us right now we are we mentioned is the public dashboard but not nearly the public. So it's mainly for the high level people to access without username and password, but at some point in the future maybe we would like to have this dashboard to be accessible by the public and at least the public can get some ideas of the country or at least like I said maybe from fun people that we want to know something about the areas of the communities. That's the overview. So we like just to show a couple of screenshots of the dashboard. So as you can see over there on the corner we will have different list of the dashboard that they can select and go through. But if you want to see the public and you can also access this website and hopefully you can also give us some feedback to improve. But as you can see now the graph the chart might be different from what you see in this to be as I said the main goal is also to tell the stories at least for the top row you can see the high level people when they see at least they know compared to last year the same period how we are doing for example the SBA compared to this time last year you see that is decreased etc. Yeah, and the same for family health. See just that's the one activities that we did in Laos and not go through in the detail but just to show you different type of chart the graph that we can do. And also the same on about health facilities that at least they get the idea of now how many facilities are there, how many health workers in the country. And by I said the almost real time when new data are entered then this graph should change and at least they know the progress or they can keep track of the progress. Yep, and one last thing is also about the mobile. So this is the main goal that they can see the same graph on the on their phone and then be able to quickly scroll and see the high level of the data. Yep, and maybe I can talk a little bit on the technical part. Hello. Okay, everyone's I'm here from here's Vietnam. I'm a technical implementer. Now we come to the sleepy pack. I will just maybe talk is very overall overall so you too can easy to understand. Okay, this is how does our public dashboard dashboard work here currently in Laos. In the public box we have currently we have two parts. First is a front end part is a page that Sam already show you in a previous slide, many chart many grab table. That is a front end part. And the main technology we use for front end part is react ZS and we include in size inside that in many useful library for showing the chart map by table. Nice. Yeah. And then about the data, the front end will get the data from the back end, you can see you can call it a middleware and we using the express GS. And this is a back end web based application form of the notches and by the help of the express GS we can easy to create the environment for the back end and also create many our custom API for the front end can use it for requesting the data. And then from the back end, we will request the data into the to to get the data. So the front end will, you can see here will not directly connect to the day to to request the data. Then back end we use the library I see us from that library, we have many authentication type to connect to the day to live basic authentication. We can to patch authentication by token to connect to day to, and then day to will return the data back to the back end and inside the back end we will transfer transform the data, combine the data into the object and then send back to the front end to use. Why we need to do that because we support we can request, we can connect request and getting data to multiple day to so maybe after you request to get the data, you want to combine that data into one chart. So that's will be in the middleware to do it for you, and then you push into the front end to see. And in the future, we like said, we will try to get data from different data source like from the Excel from my square and as well lights, right, that's if I in the future case. And for the security here right now we just using follow we just using the basic authentication username and smart world, but it's put into the back end and put in the same day to instant. So, as a container, the box script container. But maybe in the future we will plan to use live because we do use the token, or maybe we will use the event hook when every time the day to running after running analytics we can put the data again for update for the last ball we have the newest data every day. Yeah. Yeah, that's all from my path of the technical. Great. Thank you. And Sam as well. Thank you very much. Is it was at the end of the presentation. Yeah, great. Yeah. Thank you for sharing. And I think this is a great, a great example of kind of the progression that we see oftentimes with moving to public access to data right you want to get it in the hands of decision makers. But you also want to make it available to, to the broader public. And, and so we'll talk next a little bit about the different considerations that are important when going through that transition, especially when you go to the public as a whole. And, but before we do that, if we have a few minutes, maybe for questions for the history Vietnam team about the loud public dashboard. And then we'll dive into the, the kind of guidance. Just a couple, couple minutes if anybody has quick questions in here or online. We have one in the room here.  I would just like to know about the decision making process, who decided what elements they need on there. You talked about the National Assembly having very specific requirements. But we all know that there could be different requirements in terms of like general health in terms of maybe a specific disease and things like that. And what was the process of deciding what goes up there. And what doesn't go up there. Thank you. Thank you. So in terms of what that what indicators we will put up there. So, first of all, we have different indicators that at least at the national and even the provincial level that they need to monitor frequently. So those are the things that we put up there that every program they agree that they need to, to see that and also the high level people that they agree that they want to see those. So, apart from, from those things that the health program will decide and will tell us, like, what else did they want to see the thing that the boss want to see and also think that the lower level need to see. So, mostly we work with the program also with the visual that support each program to decide what additional information or initiatives that they would like to be added in the dashboard. There is one question, technical question online. And is there any caching at the Node.js layer here, or does the requests go directly to the DHS to servers. Is there any caching there. Yeah, we have the caching in the middle where after the one time you request the data is we restarting there also, so you don't need to request again when you reload the website. Currently, it will expire after 24 hours. Yeah, maybe in future we will set it and combine it with even hope to persist based on the analytics table time. And the analytics that this next question is about analytics table time. So, for you have real time data but it's every it's every day right so it's 24 hours. So you're running analytics once per day and then using that data. Is that correct. Yeah. We have a few more questions seems like this is an interesting topic I do want to get to the guidance. And so there's just this one one last question which is, and yeah maybe how do how do you do data quality checks before going public. I think actually I'll touch on that in a little bit later in the in the session here today so how to how to kind of validate and ensure that you have the data that you want before you that you want to make public before it goes public. So I think I'll get into that in the more general guidance. I think there was a question from the other room but I don't know if that's. Yeah, covered. Okay. Yeah, okay, we'll we'll go we'll stay with this for now and then we'll go through the guidance because that's a there's probably probably going to be some questions on that too. Thank you everyone for your patience as we go through this. Okay, so, and there are a few categories of considerations that we've compiled from different, different groups that have built public dashboards on top of the dashboards. And this is just sort of scratching the surface of what are some common things that everyone should be thinking about as you're making data public and DHS, giving the public access to data that maybe is collected or stored in DHS to So one important consideration, probably the biggest one that we see sometimes being overlooked is security right so this is should should be top of mind as you're making data public is how do you make that secure how do you make sure that you're giving the public access without giving them access to the DHS to system without giving them access to even be able to find that DHS to system in many cases. And in particular, hiding credentials so sometimes the easiest way to build a public dashboard is to put your username and password in some JavaScript and put it on the web somewhere, and then you have a pretty graph that you can see, but then whoever looks at that and knows how to right click and save you source can see your username and password and access your DHS to system. And just unilaterally that is bad, please do not do that. We do see that sometimes and it's even historically has been a way to embed visualizations in systems, but there are much better ways to do it. And that should be avoided when you're especially when you're giving access to the public. This has been something that with the kind of explosion of public dashboards for health in through the COVID pandemic. And lots of people know that there are operational systems behind them. There are lots of people that have access to these these dashboards and try to maybe can explore a little bit and and find ways to access the system. So, these are some very high level general security guidelines but there are definitely more to keep in mind. So, as an overall takeaway, think about security and think about security a lot when you're giving the public access to data to go through these individually. So, as a rule, if you're giving the public access to data, you should not give them access to the operational system that is being used by health workers or by the ministry to actually do their jobs, right. So, they can be completely separate systems they should be connected, but you should never give access to the operational system to the to the general public, and even accidentally so even if somebody is very technically savvy and can right click and save you source or do all sorts of different things they shouldn't you should protect as much as possible that operational system. And one way to do that is to hide the credentials as we said don't put the username and password in the browser. And there are lots of other ways that you might accidentally expose credentials so just be very aware of where the credentials for your operational system are stored. In many cases it's it's useful even to hide where the operational system is right so I'm saying operational system.  And if somebody goes there and then they can see the requests that are going to DHS to my country dot gov which is where your operational system is, you're opening up a surface area for someone to explore that and to try to hack passwords or send phishing links to other people to users of the system. Ideally you should hide that completely so in the example from Laos there there is a middleware there right so the browser doesn't talk to DHS to the browser talks to a another server that then does all of the talking to DHS to on the server side. And that helps to protect the the operational system because unless somebody knows knows the architecture behind the scenes, they can't just figure out where the DHS to server is and start to attack it in some way. If possible, it might be useful even to hide the fact that it's a DHS to system at all. So, this isn't strictly necessary, but it can help to kind of have defense in depth to not even expose that the back end of the system is DHS to there might be other systems it might be multiple systems that are coming together. But if you are using the DHS to visualization engine and you are using DHS to API is, even if that's going to another server and then getting passed on to the the back end DHS to, then it opens up the possibility of someone being able to figure out and use that to their advantage in some way. So if it's possible, it might also be useful to explore obfuscating those API signatures or the API calls that the front end is making. So this is getting a little bit technical I'm not going to get too technical in here but we can dive into what kind of how you would actually do this another time. Another important consideration is performance. So this is again, sort of similar to to what we said with security, in that if you have a million or 100 million people that are all of a sudden accessing the public dashboard that you created, you shouldn't. That shouldn't be the same as having 100 million new users in your DHS to system, right, which will probably take down your DHS to system, I would say definitely if you have 100 million users. And if you have 100 million people, this is a little bit of an exaggeration let's say even a million people if you have a million people that are accessing your public dashboard. That shouldn't change at all the ideally the performance or the stability of the DHS to system that you're actually using in your country for your operational systems. So that's another important goal or consideration here. And there were some questions about caching and the team in Laos answered as well saying that yes, caching is done in the middleware. It's very important because if you are saying every time somebody hits refresh in their browser, and you have a million or 10 million people that are hitting refresh in their browser to figure out what today's COVID case numbers are. You shouldn't have every single one of those refresh clicks go and hit DHS to that can be problematic. So instead, giving data. Close to real time is almost always better than than getting than refreshing every single time somebody clicks refresh so even if you cash for in the in the case of the that loud public dashboard it was 24 hours. Maybe you need maybe you cash for an hour, but it's public data so it shouldn't be changing based on who's looking at it so you can cash very aggressively here to avoid any issues. It also is possible to pre compute or pre generate a lot of the visualizations or the data that is being displayed in the browser to the public. So if that's a possibility for you there's some more technical considerations into how to actually do this. That's something to consider as well there are other other benefits we'll talk about for doing that in a minute. The next example or consideration is timeliness. So as I mentioned, probably up to the minute timeliness like real time data isn't actually useful or necessary for the public if you have an actual public dashboard. You don't need, people don't need to know what happened in the last hour in most cases right. One day is probably fine one month might even be fine in a lot of cases because people aren't even necessarily looking at daily daily data. A little bit different in the case of COVID but in most cases you're looking at how how is the country doing this month or this year compared to last year or last month. And so you can use that to your advantage to be able to avoid to address some of the performance impacts of trying to give real time data is that you don't often need it in most cases so you can do aggressive caching again. You can pre generate or pre compute things before people actually request them. The next consideration is privacy and access control. And this is not the same as security. So it's, it's related and sometimes we say security and privacy in the same sentence, but they're not the same thing. So in the technical security considerations we're talking about how people might hack your system. So privacy is a little bit different in that you don't want to expose the public, not necessarily people that are hacking, but you don't want to give individual personally identifiable information on your public portal, for example. And then maybe you, you don't want to give access to some sensitive aggregate data that you did that is maybe used internally in the ministry but isn't ready to be published yet. Right, so you want to think about what data is actually being exposed to the public and this is, there were some questions online about this as well. So the important consideration for public data access is what data is actually being shared and who gets to decide what data is shared and if you're if you're just all of a sudden clicking a clicking a button and exposing all of the data in DHS to to the public. The DHS to system that might be really bad, or it might be fine. It really depends. So you need to know in the context of your country, the politics in that country, and where what who gets to decide what's public and how what is the process to kind of approve data for publication. Data is in general a good thing but sometimes you need to make sure you're following the right process to make sure that you're not publishing even data that might be correct or up to date, but is not necessarily cleaned up enough yet and could be misinterpreted by the public so that's also something very important to consider is that you're not just putting the data out there but also you're you need to make sure that people understand what that data means and don't misinterpret it in some way. So generally I would say public data should be as explicitly selected and approved as possible. There you can automate that and you can reduce the kind of burden of that process, but generally you want to make sure that it's well known what data is going to be made public and how you're going to do that. And then there's a few considerations here on this next one next two slides that are a little bit less and less critical maybe, but are quite important when we're talking about access to the public. And, and that is usability. So we're talking about a very different user base when you're building a public portal than when you're building a DHS to system, for example, right. So you don't want to have to have a huge training campaign to be able to train people to use a sophisticated tool. You want to make it as usable and as accessible as possible to the people who are actually going to visualize that data or see that data and use it. And those are typically, they can be people that are less technically literate, for example, they're, they can be people who have all sorts of different types of devices from very low spec phones to every browser imaginable and coming from all different countries and different languages and all these different things. So usability and there's a lot of considerations here on this slide and the next one that I won't go into too. Too much detail, but they're important also to keep in mind when you're building a public portal and not just on top of DHS to but just building something that's accessible to the public. One way that this ties in with some of the earlier slides is that the performance in the browser is also very important to consider so you don't want it to take a very long time for someone to load that data.  And if you're able to do that by pre computing or pre generating some images or doing server side generation or aggressive caching, you can also have some of the security and performance benefits that we talked about some other considerations for usability, we won't go into too much detail here. Responsive design is also something we saw in the in the example from the law public dashboard that it's accessible on mobile devices, that's probably the majority of users these days will be accessing it from a mobile device rather than from a desktop so think about that when you're designing your your public portal. There's lots of other considerations here that are worth exploring as well. This one was brought up by one of the his groups so we collaborated with a bunch of developers from different his groups who have built public portals. And, and this was a good addition from that group and as well which is that user support and troubleshooting is very important because if all of a sudden you have a million people that are trying to call someone to figure out how to use this this public website. So to make sure that you have processes for that. Do they have an email address who's going to be checking that email address do you have a call center, all these different things so depending on the use case. It can be quite important and I think that was something that was brought up. I think that was a bit during the, like the explosion again of kind of public portals for coven data. Okay, now I wanted to share and quickly the results of this and whole for this quiz. I guess whatever it was called. See what it was. Oh no, I need to figure out how to get the results. They're going to be on my computer. So while I go through that. Yeah, feel free to go ahead and add some more few. If you haven't scanned this yet. Well, um, while I'm doing that does anyone have any questions on those on those considerations I went went through them pretty quickly but maybe we can on next step I'll talk a little bit about a few kind of low hanging fruit of easy ways you can achieve some of those with some some easy technical choices. I don't have any questions or additions to that list. Well, so I have Austin questions. The first one is, is this process applicable to almost all versions or are there any considerations. And second one is easier documentation available for the implementation part development part of this process to make the dashboard public. Thank you. Good, good questions so the first first question was around version, like DHS to versions you mean. Yes, yeah, so these are general considerations that it doesn't even necessarily have to be a DHS to system behind this, this public portal this is generally advice and guidance around how to build a public access to an operational health system, for example. So it doesn't matter what version of DHS to your running. There might be some nuances or differences in the way you actually implement some of those considerations based on the version of DHS to that you're using. And, and the second question was a good one is, is there documentation and guidance, and it's hard to have like a step by step guide for how to do this because it's a bit different in every every country every context. And, but that's what we're working on so I'll get to this in in a minute, but basically the next steps are, we have some of these kind of guide guidance for how to do this in a fairly easy way. We're going to be publishing that and also looking at doing building a reference implementation that someone could could base their implementation on if they wanted to that follows or takes in takes these considerations into account, and could adapt that to their local and that will also involve some guidance on how to do this technically as well as more expansion on the considerations that I went over today. Good question. Then can you hear us. We can yes go ahead. So we have some questions in the over spill auditorium. Thank you. Yes, I actually it's a question about the use of DHS to not maybe for the broad public but for partners working with the Minister of Health. Yeah, okay. Because you know, in many countries it's always conversation to go through the Minister of Health and the Department of Health information to ask for them to make the request. So I'm, I'm wondering if we can use the HIS to with an access that it's only consultation for not both the big but some partners working with the Minister of Health and if you recommend to do that or if you recommend not to do that. Yeah, absolutely. And I think that's to some extent what was done in Laos. I think. So it's, it's kind of a, a slightly narrower scope of what we're talking about here. So one way to do that would be just be to make a public portal and make it available to the public to everyone. But maybe you don't want to do that maybe there's some data that's a little bit more sensitive or you don't want to be misinterpreted or those types of things. You don't want to do this but put it behind a an intranet or a password on the on the public portal so still anyone, any of the partners can access that can can drill down can maybe see some details. They're not accessing the operational system. Doesn't matter how many people access that it's still going to going to have functionality in the operational system continue. So you could use a lot of these techniques to do that as well. And, but it's important to think about like putting it on the public internet means that it's public. So if you're just wanting it to be exposed to a certain subset so a certain set of partners or something like that. When it's when it's on the public internet it's public and you have to consider all of these things and you might be able to get away with avoiding a few of them potentially if you put it behind a username and password or give give people access in some other way. Hopefully that answers your question. One more question Austin. Sure. Yeah. Thank you. This is doing. I just wanted to ask about the reuse of the two analytics. Hmm. Yeah. When developing web portals. Most times would want to save time to come out with this nice public portals and already we know there are some analytics that have been done analysis have been done generated in within the highest two. And from within some version, I think 35 backward we used to have things to do with plugins and the like that could be used to help or aid and fast track the process of public portals. Is this something that's yet being maintained and if so, how simplified is it to allow public portals to reuse analysis objects that are within the days to. It's a good question. And so this is something that I think in in our technical guidance we'd like to make possible. And because there is a lot of power in the visualization engine of DHS to the analytics API is as well as the tools that have been built that you can design something in data visualizer and then it shows up on the dashboard and maybe you just want to make that particular visualization available to the public. The way the previous way to be able to embed that in an external website was not very secure so we have had removed that for that reason. But it is something that would we'd like to to make possible and it does kind of defeat the purpose of avoiding exposing that DHS to is there because you're using DHS to in some ways, but maybe that's not a that is important of a consideration of your use case and it would be useful to make it possible anyway, it, it is possible, even today, but we need to build some guidance around how to actually embed that visualization in a public website, and how to securely use a gateway to protect the operational system with caching and with hiding the the URL of the operational system and hiding the credentials of that operational system. So there are ways to do it, but it's and there are organizations and Hisp Uganda comes to mind and a few others that have have explored doing this in different ways. And, but we want to make sure that we have clear guidance on how to do that in while maintaining security and performance of your operational DHS to instance, which maybe thank you for that question that was a good question. And that maybe gets me to this first recommendation so I'm going to go through these six recommendations that are kind of high level. So if you do not adopt all of them. These are just things that you might want to consider to have some quick wins in terms of doing some of these. Having some of this protection for the data that you're making public and for your operational system. And so one of those that's kind of an easy win clear win is, don't, don't give anybody public access to your DHS to operational system. It's dedicated that only has public data in it. So even if somebody got a username and password for that system it wouldn't help them at all because all they can see is the public data anyway, right. So, if you have a DHS to system that are just two instance that is just public data, you can use the new aggregate data exchange service and application in the operational system to manually review and publish the data that you want to be connected to that other system. And then you have a, basically you can give, you can be a little bit, you're a little bit more protected by giving the public somehow access to that public DHS to instance. Even if you're not going to give them actually username and password you're still, you still have a kind of a layer of protection there. The second which I would recommend even if you have all of the public data in in a dedicated system is to use a proxy or reverse proxy is actually the correct term, which protects the DHS to credentials and limits the impact on the production DHS to system by implementing caching by performing rate limiting different things like this. Engine X is a one way to do this there are other ways, but the, they're very good systems for basically giving the public access to back end services in a performant and secure way. And so that is a general recommendation that you should you should probably always do that when you're having giving the public access to something. And if possible this this ties with in with number one again, you should do pushing data instead of pulling data right so you don't necessarily need to do this if you have caching setup correctly, but it's more predictable. If your operational system pushes the public data on a regular basis or maybe somebody manually approves it to a place where then it can be accessed by whoever maybe it's even published to Google Drive or something. You can push your data when you know when you know that you're ready to put this make this data public on a regular basis maybe once a day you can push it somewhere and then give other people access to it in some way using one of these other systems. So you can do that with the aggregate data exchange app you can also do it in other ways where you export to excel and upload that to an FTP server, all sorts of ways that you could could do that. So pushing it instead of pulling pulling meaning using the DHS to API to when somebody hits refresh in their in their browser on the public dashboard, pull that data from DHS to, it's much more predictable to instead push it from the system to somewhere where the public has access and three more recommendations I know we're a little bit over time so feel free to leave a few if you have to I apologize we had a little bit of a delayed start. But if, if possible, you might want to hide DHS to specific signatures so you would maybe hash the API calls that are being sent and then you use that hash instead of the actual API signature, so that people can't kind of manipulate it by saying All right, I see that this API call is being sent for this particular visualization or this particular set of data. What if I change the ID to this something else that I know exists or what if I try to like expand that out and find some data that I'm not supposed to have access to. So if you don't let them do that by like basically saying that the API is not is not the full DHS to API it's only certain certain calls that have certain ways to access it, that could be helpful. So that's something to think about. The fifth one here is use server side generation if possible this helps a lot with performance, not only of DHS to but also of the public website. You can pre generate the images of the visualizations that you want or the JSON for the data of this public data, and you do that on a daily basis, you do that on the server. Then the browser doesn't need to make a bunch of requests to the API over and over and over again, and you don't need to request it from DHS to over and over and over again. You can generate that on the server side, which makes everything go a lot faster and is more predictable. And then the last one is just don't don't reinvent the wheel. So don't be, don't build everything from scratch because there's a lot of tools out there that are very good for a lot of these things. So try to figure out how to piece together existing tools, rather than building them all from scratch and writing a bunch of bunch of code in node JS or Java because that's also a very easy way to accidentally introduce security or performance issues. If you're not very careful about how you do that. So again, what's next is we're going to be publishing some of this guidance the considerations that I shared, and consider continuing to collaborate with our partners his groups, all of you. So everybody that and how if you have ideas for how we can improve this guidance or what you'd like to see from this type of guidance. You can send me an email Austin at DHS to.org I'll share that on here as well. And we'd love to involve you in that. We'd love to share that guidance and also develop it into a more of a technical guide for how to do this. If you're starting from scratch, you can probably adapt that to your local. That you need to do it in your local context. And yeah, I think that's about it. I think, thank everyone for sharing the examples. And so we have another like 15 or 16 examples of public portals here. It would be great to work with all of you on, yeah, kind of curating a little bit of a list of those that that we might be able to. Yeah, to share or to, or to use as we develop some of this guidance. Thank you all for sharing that and thank you for your time. Apologies for going a little bit over, but hopefully it was a useful discussion feel free to come up and talk to me or send me an email if you have any questions after. Thank you. And thank you again to Salmonia for the Vietnam presentation.


Okay, this is what to expect. It's going to be an open forum. We have a good friend of mine, a civil attorney, and we have a guy who's a prosecutor, and we're going to, we tried to have a criminal defense attorney up here, Jennifer, but she kind of disappeared.   I guess she was afraid of you guys. She's right. So if you'd like to make a statement or... Sure. Hello. About that. That was good, huh? All right. Yes, I've complained about the air conditioning again. I'm happy to see so many of the people that I work with here in the audience. I could have a 100% success rate in spotting the fed. So anybody who wants to talk with me after, if you need a t-shirt, let me know. Well, maybe we can work something out because you can run, but you cannot hide. Very good job. Nice to see you again. I'm going to go hunt for Jennifer. I'm a federal prosecutor from San Diego, California. My name is Mitch Denver. And Peter asked me to do this, and I agreed because I thought it made sense to do a little bit of reality checking. Talk about what we really prosecute versus what people think we prosecute and kind of give some people a comfort level. And the reason I'm willing to do that is, you know, we talk a lot about what's legal and what's illegal. And a lot of the things that people do are, if you look at the books, are illegal, but they don't get prosecuted for it. And what that does is kind of embolden people to do things beyond the norms and say, hey, they didn't get hit for something that is illegal, like, you know, just jumping into a system and jumping out without access. Wow, that means that, you know, I do something illegal and I got away with it, I can do more because that's not the kind of stuff we're looking at. So what I want to talk to you about is what we really prosecute versus what we really don't and maybe that way clear up some of the myths and some of the legends and go from there. That's what I'll talk about, Brian. First of all, I'm not a fed. I've been just harassed by, I don't know, it's like, do people really think that feds are going to send some fad-balled guy out here to... Does this mean a little bit of family? Yeah, I must. I don't know. I tend to give the feds a little more credit. Next year they'll pull a double switch and have... flood the place with fad-balled guys. I remember. Yeah, yeah. All right, you know, it wouldn't cost much. Basically, I'm a civil practitioner. I am not interested in throwing anyone in jail. I just want your money. I'm here to... Well, yeah, I'm just here to scare the bejesus out of everyone and talk about the civil ramifications of unauthorized computer access and just where I see civil litigation coming from in the next few years and how it will affect newbie hackers, Uber hackers, or anyone else. I see considerable liability for coming for insurance companies and for insurance companies representing all the new start-up internet security consultants and firewall experts. And that's what I'm going to go into after the discussion of the criminal matters. Okay, sorry. So basically, after we touch on the criminal aspects, I'm going to hit you where your pocketbook is. I'm going to talk about where you may be maybe civilly liable, whether you're a hacker or in the internet security community. All right, let's talk about criminal stuff first. Here's the kind of stuff we do. Stealing. Stealing is wrong. Remember, thou shalt not steal. Remember that one, one of those commandment things? It's been wrong since we've had civilization. It's been wrong. It always will be wrong. It's wrong today. It'll be wrong tomorrow. Stealing is wrong. We should have been taught that as children. Stealing, taking something that doesn't belong to you is wrong. And while it could be a lot of fun, I mean, who didn't shop with when they were a kid? I mean, while it could be a lot of fun and exciting and challenging to take something that doesn't belong to you, it really sucks if you're on the other end of it, particularly if it's something personal to you. So stealing, if you're stealing, if you're out there stealing stuff, then you've got a problem with us. I don't care about people rooting around systems for the most part, but when you take something that doesn't belong to you, that's the kind of stuff that pisses people off, and what gets us going and gets us, we have a very slow, cumbersome process in order to go after people. But when one steals, then we actually crank it up. You heard the other day talk about a $5,000 limit. Well, that's not true.  It's actually not true. In the part of the computer crime statute, it's 18 U.S.C. 1038-4 if you happen to care a look. If you break into a computer with intent to defraud, then there is no $5,000 limit. It's a felony right out the bat. It doesn't matter. So stealing credit cards and using them, doing anything where you intend to get money from somebody by misrepresenting or by just by sleight of hand is the kind of stuff that gets us annoyed and gets the victim annoyed more appropriately, and we go ahead and look at people who steal. What else do we take seriously? We take malicious behavior seriously. It is wrong. It has always been wrong, or probably will always be wrong, to destroy or damage property that doesn't belong to you. It's just not cool if you're thinking about walking down a street and throwing a rock through somebody's window. It's the kind of stuff that, gee, that was wrong. And yeah, it's wrong in the physical world and it's also wrong in the cyber world. What we tend to look at because we're not terribly smart is that we look for a physical world counterpart to the kind of behavior that we see in cyber. So that if something is wrong and you know it to be wrong in the physical world, it's probably going to be wrong in the cyber world and it's the kind of stuff that you look at. So destroying property is just not a good thing to do. It's just not cool. So as a consequence, when we see that kind of behavior, malicious behavior, that is the kind of thing that once again gets us interested. Now that does have the $5,000 threshold just to make things perfectly clear if you do damage, delete, destroy information and it costs the victim more than $5,000 to repair the damage than it meets the federal jurisdictional threshold. And it's kind of silly when you think about it actually. What we're doing is encouraging bad net citizenship.  You're more than $5,000 right off the bat and that particular person can be looking at federal prosecution. If on the other hand you're a good ISP, a good net citizen and you have pretty serious security and you have system managers who understand security who catch the break-in fairly early and it's somebody, you know, actually consider that the two crackers have the same absolute intent. They just want to do damage. And let's say for example they do the absolute same damage. They go in, they get some access, they destroy personal web pages, they steal personal information, they just cause a ruckus. But this particular second ISP is serious about security, catch the break-in fairly early, have good backups, are able to restore the deleted files quickly, they patch up right away, this usually patch up right away. And as a consequence it's taken maybe a couple of sysadmins half a day to put back together. What's that cost to company? $1,500 in time? No federal jurisdiction, zero. So the good net citizen gets no action from us and the bad net citizen, somebody who doesn't give a wit about security, their case gets looked at by us, which is kind of weird and kind of disturbs any right-thinking person, I think. Maybe not. Oh, we have a question from the audience. In a major-stational standpoint, as a practical matter the prosecution is done where it's easier to prosecute. So in a case where let's just say somewhere from New York hacks into a company or hacks into a naval facility in San Diego, not that that's ever happened, the most likely site of prosecution would be the place of the victim computer system, because it's easier on the victim for the prosecution to occur there. The only time that doesn't really work is with juveniles. The juvenile system doesn't work that well on the federal side. Part of the reason for that is that the computer crimes are not listed as the lineated juvenile, delinquency offenses. So as a consequence, you have to get permission from the local DA to prosecute it federally. Oftentimes that's forthcoming, sometimes not. The problem with the federal system doesn't have any juvenile detention facilities. We're lucky in California, lucky in California, that because we have a very powerful juvenile justice system that a federal case, a federal juvenile case in California, the federal judges have access to the California Youth Authority sentence so that if appropriate, someone can be sent to prison. Also, we have a pretty good working relationships with our DAs around the state, so we get the juveniles taken care of that way. Are you a lawyer by some chance? Yeah, whatever you said. Sir? My question is about the legality of reverse engineering. A lot of people say on the internet, they're going to read through different materials that as long as it supports demonstration purposes or that you have this 24-hour window that is protected for anyone to go out there, that it was legal to present this information for informational purposes, could the use of it for the intended result is often considered legal? Is this a real distinction or is this... A lot of this... Did anybody hear that question? We talked about reverse engineering and the legality of posting... A lot of that will run into the civil arena. As a practical matter, I'll just answer this very quickly. On the criminal side, if we get into a situation where we have a battle of the experts where the prosecution has to call somebody to say, this is something that was created by reverse engineering and the defense expert can say, well, it was readily apparent, it really didn't have to be reverse engineered. We don't do those cases criminally. You're not really looking at a criminal case for the most part in the reverse engineered product, but it does have, on the civil arena, what is it easy to do? So I'll shift to Brian for that. Well, the intellectual property aspects of it, I'm not an intellectual property attorney. I know that the guys who do that work make far more money than I do. But essentially, there's a question of intent, what was your intent in posting? Right. And, you know, once again, you can come out and say, well, my intent was educational purposes, but if you have a history, if there's evidence of intent otherwise, basically, in the civil arena, you go before a jury and your burden of persuasion is preponderance of the evidence. It's not beyond a reasonable doubt. Preponderance of the evidence is more likely than not. 51% to 49%.   I don't know, you know. Sir, I can't hear you. For federal jurisdiction, for the most part, you have to have attacked a protected computer. A protecting computer is one that's used in interstate commerce or interstate communications. So what we look to see in a given case, let's just say you send a threat over the internet, an email threat to your next door neighbor. If it happens to pass through, let's just say AOL in Virginia, then there's going to be federal jurisdiction for it. If it never leaves the state of California or the state of Nevada, if that's where you are, then there could not be federal jurisdiction for that particular offense. If you put a modem pool within a given state, it would make it more difficult for us to establish federal jurisdiction. We'd have to prove that the communications involved actually went outside state lines. If we have a difficult time proving that, we would defer to the locals. So it would likely result in more local prosecution versus federal. So if a local connection to a local ISP within the state was raided by the Public Utilities Commission as being an out-of-state call that still does not call the FBI or anybody else that deals with federal issues into prosecuting that case? It doesn't matter what they call it, what we're concerned with is the reality. Okay, thanks. Wait for the mic. I'm not repeating questions very well. Who determines the damages for the $5,000 limit because the New York Times quoted some ridiculous number for being defaced. Can you just make up this number? No, actually the law requires that whatever losses that someone incurs as a result of a malicious act have to be reasonable. So it's not even what they actually incur. If you make the mistake of paying $50,000 for somebody that reasonably would charge $10,000, then if there's an argument about that, the argument is resolved in favor of what is reasonable. And we actually have to prove up the losses. So the answer is no, it's not just whatever somebody says it is. We actually have to document it, and we can only use it to the extent that it's reasonable. Which is why there's not a lot of cases on, well, there are some cases on website defacement. Most people just look at that as graffiti and it's entertaining without really bothering with it. I mean, it's true, but if you take the site down and make the information unaccessible for an extended period of time, then it becomes malicious. I mean, there's a line between something that's entertaining and kind of a fun sort of prank to something that's malicious is a continuum. It's one of those things that you know when you see it and you all know it when you do it. But the answer to your question directly is the losses have to be documented and they have to be reasonable. And on the civil side, we run into very difficult problems in proving damages in these kind of cases as well. It comes down to, well, was there trade secrets stolen? What's the value of those trade secrets? Yeah, we copied someone's customer list. We didn't take it. We didn't deprive them of it.  There's tremendous problems in proving up damages in these kind of cases, often requiring expert testimony from economists and you name it, internet security specialist. It's a difficult area. Trade secret theft is really difficult. Primarily if something is taken while in development, the issue is what's the loss. Is it the cost of development? The product may never reach the market so that there really isn't an actual loss. It becomes very difficult for us to figure those cases and there's not a lot of federal cases on it. Sir? Yes, sir. If you guys were IT managers, what steps would you take to educate your corporate attorneys on people you catch hacking internally, you know, like prosecution? That actually enters both of our areas. From my perspective, that there is an interest in corporations understanding what criminals are doing within their system, if in fact there are criminals within their system. I think it's important for them to know that and there's an interest, general societal interest in rooting out evil, if in fact evil is what is occurring. But there's a far greater interest for corporations to take a serious look at their security and it's called money. Exactly. And Brian will talk about that. My job, if I'm defending a corporate entity which may be used as a springboard for an attack or a security consultant or a security company who is charged and has taken on the job of preventing such attacks is not rooting out evil. It is protecting my client from liability. And the question you asked is nebulous and really huge. I mean, if you find your employees actually participating in cracking activities, you have to take a serious look at it. You have to talk about bringing in the authorities. You have to contact your insurer. Let me refine that again. Let's say you catch somebody doing something pretty malicious internally, maybe to yourself or someone outside. How do you guys, if you're an IT manager, how would you educate? Because normally your attorneys aren't very smart on anything really except for prosecuting DUI people or whatever. But how do you educate them? What steps would you guys take to educate them on perhaps something maybe hire a consultant? Was it some of the options that you guys would suggest? Is it just crazy question? Well, no. I mean, it's a very real world question and it depends on a number of factors. If I'm an in-house counsel representing a corporation and I become aware that my company's computers are being used for a tax or actually employees are being tacked from within, we bring the authorities in right away. We bring the authorities in and then we also, we do some severe risk analysis and find out to the best of our ability internal investigation on what's happened, what's the, if it's a one-time thing, if it's been going on for months, do some quick damage assessment, contact our insurers, get the ball rolling to protect the company's assets, whether it be springboard liability and whatnot. But definitely I think you have to bring in the authorities the minute you're aware of a second attack. One of the more interesting parts of that, to the extent a company has its assets kept in digital format, I mean they want to be able to protect them and if somebody steals their trade secret they want to be able to sue civilly or get the federal authorities interested in prosecuting. Unfortunately the law defines a trade secret as something that not only is secret but that the company has taken reasonable measures to protect. If the company does not engage in any security practice, how is it taken reasonable measures to protect its secret? So that's one of the easy doors through which corporate managers can start understanding that there's a price to be paid for only being worried if the printer is working. That's where I see in the next five years as an exploding area of litigation. The reasonableness of the steps taken by the companies to protect the data of computer network people to secure their systems, to prevent them from being used as a springboard, that sort of thing. One of the areas I was charged with here was to basically scare the hell out of the young hackers and the bottom line is that let's assume that there's been an attack and let's assume we've been able to prove damages. Let's say we have 5 million dollars of actual damages that we can prove in court. I'm not going to get that from some 20 year old college kid unless he hits the lottery or comes up with a great patent next year. But what I can do is I can go after the companies that he jumped through.  I will sit him down in deposition and I will ask him every technique that he used to pull this off and if I can somehow establish that he used an exploit that was known that they've been talking about here at DEF CON for the last 5 years that any reasonable security expert should have known about and should have taken steps to prevent. I will be attempting to dip into their pockets to make my client whole. And I see that. There's not a lot of cases now. You hear the scuttle button. There's law review articles coming out projecting this as the burgeoning area of civil litigation the next few years and I think that's where it's going to go and I think people in-house attorneys at the various security consultants or spring boards should be aware of that now. Do you want to see a little Jennifer? Hi, I'm Jennifer Granick. I'm a criminal defense attorney. Sorry I'm late. I was in the bar and I thought this was going to be in the room that the CDC thing was on and since it was on the TV in there I thought I don't have to worry until they're done with all of that so I apologize for being so late. But if you guys have any criminal questions you want the defense perspective on I've already turned them to the good side of the force. It is your destiny. That's right. I have a question about as far as intent goes and things like gray hat hacking and things where maliciousness is not always as apparent. Take for example someone who's showing a company oh you know I've broken your system this is your security holes this is what I got this is how you fix it but they weren't hired by the company and it wasn't so cut and dry how do you deal with the not so obvious? The question of intent in a criminal case is always the most difficult and you can own the only way and the jury's instructed that the only way that you can make a decision about intent is by looking at the person's actions what they said, what they did both before and after the action that we're looking at and we have to prove criminal intent it's important. The situation that you posit is one and there are gray hat hackers who go ahead and scan a system find a vulnerability the question comes up once they go ahead and exploit the vulnerability have they done so with pure motive or is their motive less than pure? If the message they send to the company is I found a vulnerability in your system and I exploited it and if you pay me $15,000 I'll repair it for you otherwise I'm going to put it on I'm going to post it then we have a situation of what we call extortion taking something that doesn't belong to you and it would be extortion I want to respond to that because I don't think that's an accurate reflection of what the law is 1030 does not the prosecutor is going to give you the gloss on it but the truth is that 1030 doesn't require maliciousness 1030 requires knowledge so regardless of the purity of your motives if you have knowledge that you're gaining unauthorized access you can violate the unauthorized access statute which is 18 USC 1030 and I have had cases where a security company offered to provide services to an internet business and the system operator of the internet business said they don't know anything their own security is not good and the security company said yes it is and the system operator broke into their system and stole one of the internet company's own proprietary documents that was stored on the security company's server showed it to his manager and said look they're not secure I stole our own proprietary information off their server like that and then the manager brought it to the security company and then the security company brought it to the FBI and that's a violation of 1030 because he gained unauthorized access his motives were pure to protect his company from the mistake of hiring a security company that was full of shit but nonetheless nonetheless he still had to meet me so maliciousness is not always an element of the statute your motives can be pure and you can still get to it if Jennifer hadn't been in the bar and was here when we were talking about this earlier she would have heard what I talked about at the beginning about there are things that may be technically illegal and we can debate forever the areas of technical illegality versus real illegality but the good prosecutors only do the cases where maliciousness is clear at least to us and with her that's true that's absolutely true there are bad prosecutors as there are bad computer system managers the law should not criminalize things that are not prosecutable the law should clearly say what's going to be illegal and not illegal and we shouldn't depend upon the good graces and discretion of some prosecutor to protect our rights your argument is not with me your argument is with the congress of the united states the representatives of the people here today if you want to change the law be my guest what I'm trying to do what we only try and do is take the worst of the worst off the street if I prosecuted every single act that's been made illegal by the congress I wouldn't be here enjoying your company today I'd be too busy so we're forced as a result of resource allocation to only skim the cream and those of us who do our job responsibly do that there are others and we both know them but the reality is if you're looking at what you can do and what you can't do if that's what we're here to talk about if you're knowingly engaging and malicious activity and you're doing damage or you're stealing something you're going to be talking to us and one aspect that becomes involved with trade secrets especially in large corporations is it actually secret or has it been made available to the public I've had a case in the civil arena suing an oil company who will remain nameless where I was in court jumping up and down trying to get access to some documents and we had an army of lawyers on the other side basically saying that it would be the end of the world as we know it if I was able to get these documents and essentially you know I have the documents in my hand and I got the documents from the oil company's website the night before so I was able to tell the judge you know the end of the world is here your honor but I still need copies of these documents given to me under oath and the upshot was it resulted in $10,000 discovery sanctioned for the oil company but what happens a lot of times is these companies the larger the company the left hand may not know what the right hand is doing they can be jumping up and down screaming trade secret yet their public relations department has posted the information in a press release six months ago so you know that's an issue that comes up from time to time to the civil navigator as someone that's working independently doing security auditing for someone what steps would you recommend that person take to protect themselves in the event that the company gets broken into afterwards whether you know as far as to protect your assets yeah you can knock off the ETS too I think what you need to do one is look at your form of business you may want to go into an LLC a corporation you want to get insurance you want to get E&O coverage and the amount of coverage you need I think you need to look at do a good analysis of your exposure what is the likely exposure or what is the possible exposure that is going to fall your way and go out and spend some money and get insurance and talk to an attorney talk to an attorney look at your documents some of these things you can contract away with your clients because you're in a risky business you may be doing everything right and still there may be some evil genius in the back of the room who may crack your firewall even though you did nothing wrong he just may have tripped on you might not have been negligent but if there is damage they're going to look to you so protect yourself by maybe changing your form of business look into E&O coverage and discuss with your clients a way to maybe contract out and get an indemnification clause from them you may not get it I wouldn't give it to you but they might depending on if your talents are so critical to their success and they want you they may be willing to negotiate with you and hold you harmless from many events don't count on it but it's worth the shot we're going to pass the mic around so people can hear what's going on I know you want to keep it rock out you had mentioned code section I believe 1300 or 1301 about criminalizing unauthorized access 1030 1030s, thank you what does this do in a situation where you've got a company like Yahoo that's checking into deep links at various other websites when those websites specifically don't want that deep linking done does that criminalize that behavior also? generally I'll just make it clear generally business to business behavior unless somebody's stealing someone else's property is not something that we look at deep linking I guess you're talking about a basic or exceeding unauthorized access if it's even that we recognize one of the things that's a reality even though Jennifer doesn't think it's a good thing if you simply exceed authorized access or access without authority a protected computer and obtain information and do nothing more than that that's a misdemeanor there is almost no federal prosecutor in the country that will willingly accept a misdemeanor case we just don't do them because we're too damn busy doing other stuff so if it gives you a comfort level to know that you can access a protected computer obtain information and you probably won't get prosecuted fine, you know, go home with that knowledge and sleep secure but on the other hand don't let that embolden you as my message to go beyond and enter the felony zone which is steal stuff or damage stuff because then we'll be talking to you I just want to add that there is a case that sort of touches on what I think you're talking about which is the bitters edge case in which a company called bitters edge like a bitter in an auction was doing mining of other auction sites and compiling them also people could see what the best price was on an auction and eBay then went back and sued them saying that they had gained unauthorized access to eBay's system by collecting this information and using it in their own business so this is something that we're seeing worked out not so much in the criminal context for the reasons that my colleague has so excellently stated but in the civil context don't try and make up now so in that case there could be civil damages as opposed to criminal prosecution alright exactly may I pass the mic back to you I just have a question of where on the prosecutorial and defense side is counter strike hacking or hacking back as we saw recently with the World Trade Organization denial of service attacks which were reversed and sent right back to the group of hackers which was malicious cause damage but at the same time was simply just a deflection of what they were doing the problem with counter attack is that to the extent that you are sniffing a system that doesn't belong to you in order to keep going up the chain then you're probably violating the federal wiretap statutes there are no protections so there's no exception under the federal wiretap statutes for trespassers and the federal wiretap statutes doesn't only apply to government activity but it also applies to private activity so if you can hack back to somebody without sniffing a system that you don't have rights to then you're probably in violation of federal law would you be prosecuted? the odds are, I would say no because we wouldn't want to be in a position of trying to explain to a jury why you're a bad person for going back after somebody who did some damage to you it's the kind of thing we would hope can be worked out in a civil arena particularly if you violated some federal statutes on the way so hacking back is not a good idea if you're going to be looking to the government to protect you down the road and maybe a bad idea if you're otherwise in hot water with the government because of the wiretap laws it's our favorite statute people, it's a lot easier to prove a federal wiretap violation than almost anything else that a cracker can do most crackers, one of the first things they do is put the sniffer in in order to get other passwords and logins and that's the kind of thing we don't have to prove much only that it was an interception that occurred so that's, if there's any other advice I can have you take away from this is don't do that here comes the mic oh she want to talk real, stand up and talk real loud practice your public speaking I sort of have a question on the extent of liability and insurance and I'm going to use an analogy here if I leave my car in the red light district with the keys in it running the door's unlocked law's not going to do a whole lot to protect me if I really like silence a steal this car how far is the law going to go to protect systems that are basically set up wide open welcome out on saying come on in you just couldn't help yourself are you going to be able to sell that to the blue-haired old women and retirees that are sitting on that jury I'm not a lawyer so I wouldn't be able to do that yeah now I think I'd have a better chance of selling them than I'm a male model than that story think about it you don't have to persuade me because you're paying me I'll believe you you're going to have to persuade the people who are sitting on that jury that I just couldn't help myself and it's not going to fly that argument actually comes up in some of the criminal cases too the argument that you didn't exceed authorized access if the system was so poorly configured that you know pretty much anybody with some basic skill can walk in and it gets back to the old concepts of right and wrong if you know that you're doing wrong then you probably are and we can argue all day about whether people deserve what happens to them I mean just because I forget to lock my apartment one day does that give a neighbor the right to open the door and take my stereo I don't think so I mean who would say that that's okay I mean if I happen to leave my shades open at night and somebody you know creeps through the bushes climbs a tree to take a peek at me in my in the nude I mean you know the peeping Tom thing who thinks that's good I mean that kind of obtaining information you know that it's wrong I mean exactly nobody but you don't know what you're getting until you peek in the window you see me you go shit I climbed the tree for nothing Jennifer had the tree cut down outside her house there's no trees outside my house that's where we have a crossing point because both sides are damaged shit the gentleman in the house send the mic back to the man with the hat let's get the mic somebody stole her mic oh sorry the other guy with the hat somebody should play Jerry Springer and kind of roll you around should I do it? that's an extension to the previous question there how does implied access I mean if you go to a government website you know it's public there's no passwords you are you generally assume they're allowed to go in look around to anything that's linked there so they're giving you access to certain files within a certain directory structure in the system now if on that same box they had an open net bioshare with no password there's nothing in the front gate telling you don't enter but because there's nothing it's just like your website you're giving them access but in this case it happens to be to the entire drive how does that really differ on the civil side we're basically looking at a trespass analysis here if I invite you in if I'm a bar owner and I invite you in to come in and have a good time that's fine and Jennifer will be there Jennifer will be there longer than she should but if you then sit at my bar and reach across the bar and grab my margarita bowl and throw up into it or something you have exceeded your access and your authorization and you have caused me damage and you know when you're talking about government websites I do a lot of legal research on government websites because they're excellent but you know usually once again in the civil arena you're talking about reasonableness and when you're talking about reasonableness you're talking about what can I persuade a jury with that preponderance of the evidence standard and it's not that hard if there are hard cases those are the ones that get litigated and make law but we don't have that law right now but you know I actually look at what the end result is if you get access because the system has been so poorly configured the website is so bad that you get into even into places you shouldn't and then you just start deleting files at that point there's no question that you know what you're doing is wrong and I'm with you I'm all over you if all you've done is look around and go I'll look at these idiots is there a potential federal misdemeanor there maybe do I care no this is that invitation part of this well what's the invitation I mean if simply by having your system poorly configured it's not the same thing as having a welcome sign it's not saying steal this car it's not about the invitation it's not the action that results from that's what he's saying if you have delicious intent does it get someone's walk around with a taste or does it need to go out this is an excellent point I mean as a practical matter what I see is that often to prove unauthorized access all they have to do is they bring in the victim to say like I didn't mean he could get in there and it's you have to understand that especially I think this is difficult for technical people that permissions is not the same as permission just because it's set up so you can doesn't mean that you are allowed but as a practical matter the way that law enforcement looks at it is they don't really care kind of if you get in it is sort of what you do after because that's what captures their imagination and makes it a great case when does yes me know when does what? when does yes me know that's a very funny joke I would say yes means no when what you do in there is something that pisses off the prosecutor and I had a case just like this where a web server released software that displayed the master password file and it wasn't so much that the guy got the master password file that pissed off my prosecutor but it was that he ran crack against it and then printed the de-encrypted version on IRC that's what she got mad at so you know sometimes it's the thing that happens after the fact that really gets you in the hot water trafficking and passwords is a federal offense it's 10, not 29 there was a guy over there who had a question what kind of arrangements are made with the United States and Canada if a Canadian citizen breaks a law in the United States concerning assistant penetration hypothetically oh yeah well I'm retired hypothetically eh okay as you know I'm not I cannot talk about any pending cases and of course my responses are my own and not the official response of the United States unless there are a number of ways that we deal with international events Canada is actually an easy example because we have a we have a treaty with Canada that allows for mutual legal assistance we have an extradition treaty with Canada that is actually used fairly extensively we share some culture with Canada so that we have an easier time talking to each other about matters of import we have the ability and have regularly provided Canadians with information that we've obtained that they've used to prosecute their own citizens so Canada is an easy example there are countries however in which it's much more difficult even with even with treaties to gain the type of cooperation that we want where there's been international destructive behavior can you address that same issue for the opposite it's actually the same when we have these treaties like with Canada China we do not have an extradition treaty with so if an American hacking from their own room breaks into a Chinese computer system and causes all kinds of damage it would create some international diplomatic discussions but the American would be it would be completely free from prosecution unless they happen to travel to China or a country familiar with China or a country friendly with China now China hack back now then we get into some very interesting situations does China hack back as the government of China or does it hack back as an individual who is hacked once we get into areas of warfare of informational warfare and suddenly we've gone from a very simple hey I got into a China system cool next thing you know we have the Chinese government responding it's almost like a movie and the answer to it is one that would the American ultimately get prosecuted probably not because the Chinese government probably would not give us enough information to amounts to successful prosecution if they did would we prosecute yes I'm sorry would we send it to China for prison for no just for reeducation I had a question on due diligence on the on the places that are the jump point you talked about civil litigation there what sort of matrix or guide do you have to go by to show that they haven't performed due diligence well this is a trick where is that established at well you're talking about you're talking about a negligence standard and a negligence standard you have to have a duty there has to be a breach of that duty and then damage there's absolutely the case law and this doesn't exist it's it's basically what you have with the civil the civil field is legislatures it's not sexy to pass civil civil laws that you know civil penalties you can't go out and get re-elected saying I'm tough on torts so what you have here is you have basically the common law developing through the courts and you have the courts trying to jam technology issues that evolve daily into pigeon-holed tort theories that go back to Blackstone now go back a hundred years right now we have cases about unauthorized access basically trespass to chattel cases the eBay eBay case Jennifer discussed earlier that's kind of a trespass analysis there's a misrepresentation analysis that goes on with regard to using pass codes basically you're representing that you are the rightful owner of that pass code and the machine is seen as the agent of the victim accepting the pass codes as being genuine there's some real strained analysis going on here with regard to the jump point analysis you're looking at a negligence you're basically the analysis would be was I reasonable in the efforts that I took to protect the system from this kind of event happening and the evidence that you would put on as well we came to DEF CON and hired the best security guy I could find at the bar you have to sit there and you're going to have to show that you took reasonable steps to secure your system if you have security consultants or in-house people who keep up to date on the latest exploits and you can document that you can show that they've gone to conferences we have training programs we've done this we've done everything we can you've made a strong case for for not being negligent like I said before you may do everything you can but there may just be someone out there smarter than you and you're not convinced as it comes down to you know convincing a jury or you know settling a case if you don't want bad publicity I've been on the other I've been on the other side of that one no I I've done like I say this is a developing area of the law my background I've done a lot of a lot of consulting and basically similar similar type areas going I've gone into auto body shops and given speeches on sexual harassment why you can't post the hustler centerfold up on the wall you know it's a similar type analysis with what you have now you know you'd be surprised but it's a similar type analysis to what's going on now you basically have to take the analysis that I just described and say well you know we've entered into contracts you also have to you know what have I represented to my clients that I can do you may have a fraud misrepresentation claim coming your way if you've puffed and puffed and built yourself up to say you can do more than you can do see I'm going to need a translator for that we'll see that's a case where I think you have to in your contract if you're you've got to disclose make a full decision if you're if you're telling them that I'm giving you a system that we're monitoring and we're doing this and that and you're not providing that you've basically defrauded your client if you tell them here here's what you're getting it's as is it may be good enough may not you buy it at your own risk and you can get somebody to sign off on that good for you that's one of the other things if you buy it somebody to get them to give you the money voluntarily that's also the kind of thing that interests us because it's simple and we're not that smart this gentleman right here and then we'll go back over there yes we have to prove that the evidence that we're using is authentic and there are ways to do that that's why it's important one of the things that we went back to earlier is if a company or a victim gets logs or at least at the point we have them we know they're genuine the issue is whether they're genuine when we come in is a difficult one and when we have to look at on a case by case basis we have been successful 100% of the time in getting logs in we have overcome several challenges of because it's digital it's ephemeral and it could be changed and modified and then you get to look at who these people are and what's their motivation what is your motivation in faking these logs so it's a practical matter it's not a real issue for us although it comes up in every case here we go start over many of the things come down to at 10.30 with authorization and the ability to grant access if a person is on like pound hack and they say I own this box here have an account in the password they're committing fraud in one way but they're also using the form of communication common in the culture on that channel in fact a case where a person that didn't break into the box is now using that box and doing their own work and deleting files against the use of the box is not something that generally concerns me I mean it may be some unauthorized access there but I don't care we look at the result if after you get access to the box whether you've got it because you've been lied to or got it legitimately or got it illegitimately once you start doing damage to the box and we look at what it is what kind of damage you cause whether you did it intentionally recklessly or negligently that's when we start looking at it we look at the conduct after the fact the mere fact of illegal access while it is an offense is not the kind of thing that gets us going okay if a person who claims to own the box gives you access and says go ahead and trash it and you do the easy answer for me is that the person who comes in if they if they reasonably believe that person did own the box and gave them permission they lacked criminal intent and there would not be a criminal case okay officially rather time but the next speaker says we can keep with questions for a few more minutes and by the way I just got an announcement they have finally fixed the air conditioning once it exceeds $5,000 that's it's right in the statute and we talked about how to get there it has to be reasonable but once you exceed $5,000 in costs of repair and costs of analysis then there's federal jurisdiction if the computer was a federally protected computer say again the question is does every case in which the damage exceeds $5,000 get investigated the question is it depends where it occurs in some federal districts the the resources are such that the federal the guidelines before which the case will be investigated or prosecuted gets ratcheted up in some districts like mine if it exceeds $5,000 I'm looking at it so stay out of San Diego if someone breaks into your box and then proceeds to install an IRC client and use it are there any legal problems with logging all of their actions and communications that originate from your box that they've broken into Jennifer talked about yesterday as assist manager you have the right to monitor use and disclose and protect the rights that you have in the system so you can monitor to your heart's content that's that's not an issue for me and I'm allowed to some extent to surf behind you if you're assist manager and you're monitoring using a disclosure to protect yourself I'm allowed to use that information so long as you're acting in your best interest and not in the interest of the government once your interest shift and the reason you're monitoring and disclosing is to get somebody prosecuted and not to protect and not to get a wiretap order unless your system is banner that is that anybody who enters your system sees something that says you have no right to privacy in this system we're going to monitor use and disclose to our heart's content beware then I can as a prosecutor I can use everything that the assist manager does in logging and this goes back to the question that was asked earlier from the civil side if you're aware of this kind of activity as for law enforcement purposes you want to have the ability to preserve evidence that you may use in a suit against someone else or maybe used against you you want to have that talk to your in-house counsel or your lawyer and notify the authorities and preserve that evidence for your own purposes okay here's another scenario a distributor denial of service tool is used to take an e-tailer offline for an hour that e-tailer generates $5,000 worth of income every hour they're online is that a prosecutable case if you can identify the source of the denial of service if you can identify well you say the source you mean the person who triggered it absolutely love to do it just curious that's all can use of bandwidth be used as a damage tool probably not well we do that you can get federal jurisdiction is if you enter a system without authority exceed authorized access and steal computer time and if that time exceeds $1,000 then we have the ability to prosecute but mere bandwidth mere use of bandwidth I don't know how you you can put a monetary use on it the denial of service cases are easy because if you've shut down an e-commerce site they can actually count or extrapolate what they've lost if all you've done is steal some bandwidth most part Mr. Dapster then it's not the kind of thing that we'd be looking at unless there's some other intention of damage going on how do you value time oh there's a lot of companies some of the some of their computer centers actually build their departments for use of some of their mainframes and we would just use those figures to determine the loss and yeah providers charge for bandwidth as well we would use their costs back here it's my understanding that existing supreme court case law limits the actually calls on constitutional statutes which are so vague and get prosecutors so much discretion in the prosecution that a potential offender would not necessarily know if they were subject to prosecution how does that how does that jive with the amount of discretion you're saying that you have when you're choosing to prosecute well the courts have also recognized the prosecutors can decide what to prosecute and what not there's a difference between prosecutorial discretion and whether the law is so vague that you don't know what you're doing is illegal the law is actually pretty clear we can debate what is unauthorized access and what the permissions mean in that regard but unauthorized access if you're an assistant that you know you don't have rights to it's pretty easy to prosecute that's something firmly committed to the executive branch and it doesn't raise the same issue we can argue about whether I exercise my discretion appropriately but it's a practical matter that they're two separate ideas I think we're just about out of time now since people have filled into the room for the next talk so any final thoughts as such and let me get some vitamin followings or bar or something I'll buy Jennifer her first drink after this session that won't thank you Jennifer is a fun drunk thank you but that won't be my first


So I am currently a care and support broker but I started off as the placement and sourcing assistant which was a purchasing assistant so that involves using our system to purchase the care and support in a timely manner whether it's a council managed budget or a direct payment and it involves looking at the support plans which are done by the care and support brokers, checking all the details in there and then making sure you're doing the purchase accurately. It also involves processing invoices for payment again checking those against support plans, the case notes. There's also a very wide variety of emails and queries that come in for purchasing to look into. You sort of investigate the journey that that's been on and payment queries and what's gone on through that. So that's a really good starting point for working in West Sussex as well. I started doing that and then I have since progressed in March 2020 when we created the team. I moved over to the placements team and did all of the sourcing for the placements. I'd done some work on the triage so I've worked on quite a few teams since joining West Sussex and everyone is really lovely in all of the teams. There's a lot of support and help so when I started off as the placement and sourcing assistant I had a buddy who showed me everything that needed to be done. For me what was really helpful for training. I had a lot of sort of list of tasks which I needed to learn and it's like a checklist which really helps me. I since joining West Sussex I've been here for about two years. It's been a lot of change of variety which you know I've learned that I really enjoy the change in work. You know no day is the same, no customer, the case that you're working on is the same. Some things can be quite simple to work on and then other things it's very complex and you sort of have to dig in to find a solution and every team that I've worked in has had sort of an open chat on teams so that we can pop a little question in there and someone's usually always about to answer you so it's really good communicating when we're working from home at the minute. The teams are just really supportive. I've struggled with learning the direct payments a little bit but I've had a buddy to help me through that and we've worked together to sort of really structure how you learn that. A big difference I started I came from working in a bakery so I've learned having worked in all the different teams a good overview of how the service works really. I've worked on the triage inbox looking at each different pathway from hospital so I've learned the pathway that people come through where Sussex from that hospital discharge referral we receive to then trying to source that care. I've built a good foundation with a lot of providers, care homes, communicating with the hospital. I've built a good relationship with all of those different people, the social workers that you work with on a day to day and they're all really helpful. I've worked with some of CHC team members to try and work together to get someone home if we can do it through our West Sussex providers. There's so many people that work to support getting customers home from hospital getting the right support in place in the community. It's not just those hospital discharges we've got cases from the community as well.