Okay, folks, hello. Welcome to Ned's Seminar. Today's speaker is Richard Moche, we came all the way from the UK and he wrote there any weather as well with him. So Richard is a fellow at the University of Notham in the Department of Computer Science and he's a member of the network systems group. His work has been focusing on the intersection of HCI and network systems and the challenges are rising when you have to deal with humans besides pipettes. Before going to the University of Notham, he was working at Microsoft Research at Cambridge. She got his PhD from the University of Cambridge and his BS was well and he also spent time here in the Bay in split clubs and also he found with his own company. So, hi. As I said, that's where I'm from. As well as being in the School of Computer Science, I work with an interdisciplinary research institute called Horizon. So what I'm going to talk about is a project that we've had for about three years. It's just about coming to an end, which has been looking at home now, working from kind of the point of view of the user and what we can do to help the user make more sense about it. Starting out, I'm just trying to understand what the user wants to try and what to do. So with quite a strong HCI and not the element going through to developing and then deploying some technology, some interfaces to try and see what they make. Possibly, I don't need to say all this, but I will anyway, just to make sure it's on the same page. So, the reason that there's a kind of problem with home networking, home networking is now a kind of Monday. It's sort of expected in large parts of the world that you will have some kind of net access at home. Some of these figures are probably a couple of years old now, so they're probably even greater penetration. At the same time, home networking gear I think remains one of the most returned, not the most returned piece of consumer electronics. Often on open, I mean the boxes come back on open, so people don't understand how to make this stuff work. So although we've got a lot of deployment going on, people have signed up, they're using broadband, a lot of people make some use of it. The software that is around that, to help them make use of broadband, to help them understand what's going on, to help them get their devices connected, to use the network, is still fairly, it's not as good as it could be, perhaps. So there's still a sort of emphasis in network management, I think, in terms of protocols rather than the activities and services people carry out. So some of the technical, for example, showed that people don't think anymore of browsing where this is not an activity, anybody does. People shop, they bank, they do watch videos, watch films and so on, that the net is just another way of delivering that. It's not something they care about, particularly this has all become very mundane. So I think kind of the claim, sort of, to give away the end of the talk here, the kind of claim we make is that this technology is not quite appropriate for use in this context, for it's not being presented in an appropriate way. It's all the technology that's used, all the protocols that are used, all the way that we manage them, kind of still derives from this idea that you've got an expert CIS happening, a net happening there, to sort of help make sense of this and get things connected, and able to debug things when they don't work. And this is not the case in most households, this is managing the network to ensure that there's another piece of housework that has to be done. You mean uninterested, right? Not like impartial, right? Householders? So yes, I guess, neither uninterested nor disinterested is quite the right word, because they are interested in having it work, they want it to work. So uninterested, I guess, but they don't want to have to do network management. This is a tool for them. So they definitely are reluctant that people would do this in most cases. And I think one of the claims we make is that it would be beneficial to enable these kind of top-to-bottom connections to be made so that people could link up what they think of as a network and the activities they try to carry out in the network with what actually is happening down underneath everything with what the technology is doing. And the key word that I kind of, one of the key words I learned while doing some of this was from the HRP was it's about making it intelligible, not intelligent. We don't want the network to be doing things by magic, because then when it does the wrong thing, and it will do eventually, it's impossible to fix, nobody can understand what's going on, certainly normal people can't understand what's going on. But if you make it so that they can get a handle on what's happening, so they can control it, then they're much more likely to be able to continue controlling it and to be able to fix it when it doesn't do quite what they expected. So to sort of set the scene, this is the kind of conception, I think, that was being pushed as what home networking would be like. So it's all quite clean and tidy. There's lots of things connected, stuff works, you've got video conferencing at home and delivery of content and all that, and it's all great. This is the sort of scenario that I guess people have constructed for the homes of the future. I don't know if anybody's home actually looks like that. I suspect anybody with children doesn't have a home looks like that. Very, very tiny. Perhaps this is a more normal sort of experience that people have with their network configuration. I would guess in a room full of network experts, you wouldn't get that kind of thing with cables everywhere. Absolutely. You certainly wouldn't have cables being, like, taped to the stage, I think, that clearly never happens. It's all much more like this, where it's all nice and neat and tidy. So this is actually from, there's at least one company that makes bespoke furniture for putting networking gear in for home networks, because people don't like seeing this stuff out in the show. They make cabinets and things and close it away. And this never happens. This is certainly not the case. So there's a kind of thing here about how the reality is quite messy. We tend to try and concede that these things have been quite tidy and ordered, and in reality they're not messy, and people poke around with them and change them and do all kinds of things. In the same way, there's sort of more abstract conception of these things. So this is a home network. This actually won a prize from Cisco for the best home network. I've seen this diagram. So it's a nice diagram. It's a good network. One of the things that was on in this project is we had some ethnographers working with us. They did some work and they went to talk to some people. So this was a diagram that one of the households we worked with produced that described their network. So this was a guy who, I think I don't even know who's a guy, this was the person in the household who was kind of the main network admin person. So it's kind of reasonable. They've got cables and they know USB versus MIDI. They've got print server and a fan screen or something. So they've got a pretty good picture of what's going on here. They know about the names of the different sheets and so on. I believe it was their partner who produced this diagram of what was going on. So this is much more about physical things in the home, the bed, the desk. You probably can't read them, but some of these comments that have been written about, particularly wireless network things, I think this is crazy energy ways to communicate with the computers, and I think maybe use our frontal lobe. So there's clearly quite a different conception of what's happening here. This is not a detailed thing in terms of machines and links and so on. This is about things we use in a particular place. That's kind of it. And then you contrast that with the sort of network engineer perspective which shows you things like this. These things don't match up. These are the exceptions of what's going on that don't match up. This is kind of the problem. So there's been cases where this was a BBC report from... This is a while ago, 2007. This is a while ago. So the state of home networking is pretty visible. So let's go on there from the camera. And it's the complexity of getting all these things to work together. The camera will read you down here. So the complexity. So this is a sort of presentation of information, maybe. We think it's a bit more than that. So these are the kind of interfaces that you get currently on home routers commonly. And these are not explaining anything to anybody. I wouldn't have said this one. So this is the sort of... The world that people are doing this kind of thing. This is where all the networks seem to be. So what did we do then in the projects to try and start to address this? So we did some ethnographic studies. When we said we, I didn't do this. The project did this. So ethnographic studies for about 24 homes. So we went in, the ethnographers went in. They did technology tours. They got people to talk through what was in their homes. There's some semi-structured interviews. They got kind of anecdotes and war stories. And just finding out what people try and do in a fairly rich sense. So the idea here is not to be sort of statistically sampling the population. This is just to get some idea of kind of richness and detail that's felt in here. Around how people use and manage our networks. So the sort of things that they found as part of sort of coming out of this. So in people's homes, in home networks nowadays, there's very hefty units collection of devices. Sort of 5 to 15 devices seem to be the rough kind of range of things people are connecting. UK average that's claimed is about 4.6. So it's kind of within scope but probably a bit bigger. There's lots and lots of different things being connected. These are not computers. It's not laptops and desktops connected to the network now. This is all kind of devices around the home. Ownership and access rights are really very fluid. It's not simple enough to say that's that's computer. That's the voice laptop and that's it. These things are passed around. They change who's allowed to use them at different points in time based on where you are and who's there and all this kind of stuff. And again this fact that the digital housework is one of these unremarkable things now. Somebody in the house just has to do it and make sure everything's still working and if it goes down they're going to give it a kick to make sure they're trying to get it back online in this kind of thing. So what sort of came out of this? So I guess what came out of this was we picked up various themes from these kind of interviews and there were sort of four kind of challenges that were around consumption. So I'm trying to understand bandwidth use in the home. Monitoring, so trying to see what the performance of the network was going on right now. Being able to prioritise network activities this turned out to be something that people seemed to want to do and being able to police the network and police activity of the network and it's kind of important there is policing activities not network level behaviour. So the sort of to give you some snippets, some examples of this. So commentary from picked up the first point there. So they want to see this particular household want to see what's a person's household want to see a historical record of bandwidth use. So the context of this was that they had just got a lodger and the lodger was causing to exceed their monthly bandwidth cap. And the belief was that this was because the lodger was doing a lot of accessing a lot of video from the Far East because that's where the lodger came from. But they didn't have any evidence to present them that and say look can you start contributing a bit every month to this because we're not able to use the network now. So the second point, monitoring. So there were a couple of households where sort of the kids picked up with a lot of bit torrenting. And this was getting more by other activities. And again it was this thing of being able to just see what was going on so that the household, the people in the household that had time could then kind of negotiate what was supposed to happen there rather than being able to say we never want to see bit torrent on this network.  It's what to do with what's going on right now and what time of day it is, what day of the week it is, what are people trying to do. Prioritisation. So this came up in a couple of cases where people would work from home. And so there's definitely this quite strong opinion that if you're working from home then you've got priority on the network because you're working and bringing in the money to pay for the network essentially so the kids can get out of the way while you're doing stuff. It wasn't that you wanted to stop them doing what they were doing but you just never wanted your activity to get delayed or caused for any reason. Maybe the other ones doing the work and bringing in the income. Nowadays this is made more likely. This kind of goes back to the perception of activities on that as well. So it was one case where the family kind of had this policy where if the dad was online doing his banking, doing his banking activity online to his bank, everybody else had to stop using the network. They didn't want anything to get in the way and kind of affect that. So for him this was because banking is an important activity. You don't want that to be affected by other activities. And then the final thing was policing. So this was policing activity. Again, not in a way of saying you must not do this ever but in this much more negotiated, nuanced way to avoid the endocrine use. So in this case it was student doing, has to do homework. And she was spending too much time on Facebook. So there was a negotiation that had to go on about getting on Facebook or not. Was it an appropriate time to go on Facebook? Had you finished your homework then you could go on Facebook. That's my house. I tell you there's a fortune to be made. There is no good tool available that allows you to solve that problem. And there must be tens of millions of people. Facebook has been resisting having anyone solve it. I'll attempt to come back to that later. So another example there which was quite interesting was the parent did not want to ask another parent for the password to the other parent's wireless network to give to their child when her child went out to visit because the other parent's partner worked from home and they didn't want their child doing things they shouldn't do on the network or is somehow getting in the way. So she was actually going to buy a 3G dongle for their child so that when her child went to visit his friend he could get on the network using the 3G dongle rather than trying to get access to the wireless network in the island. Because that was going to be too difficult a social situation to deal with. So these things are quite, as I said, quite subtle in some ways. So what did we do to address this? So we built a thing that we could try and deploy to get some idea of what we could do to affect these things. So we looked basically at the home router as being the gateway point. One of the criteria was we didn't want to have to go around and change any of the devices that were connected. So really all the stuff we did was going to be in the home router. So we misspelled infrastructure. So we built some software that sat on an EEPC that he could use to start with. Running all the standard Linux host APV kind of stack. And built some APIs against that using OpenFlow and a custom pop-up system which a partner at Glasgow built. His name is Steve, a head of school that used coding while he was having to be head of school. And so we then used OpenFlow and Knox as the kind of control interface to that because it seemed to be both a reasonably straightforward way of getting it back in control. Flow level management of this seemed to make sense. And it also left open the possibility in the future that you could imagine a kind of three-way negotiation or two-way negotiation going on here between the ISP and the home, the people in the home about how they wanted their network to be controlled what was the appropriate thing to happen right now. And they also managed the policy probably come back to the policy end. So the regulatory system diagram for this. So this was how the software was constructed. So the database sits at the top, HWDB. That provides all the monitoring capability. And that feeds into Knox and the policy engine. And then we've got a control API, DHCP implementation, a DNS interceptor. This kind of gave us the control that you exercise that our home network has been used. Simple web API against this. So based on those challenges that we kind of brought out from the things that what people were talking about when they were interviewed for this, it was really sort of levels of control. Ideally, these control would be based on users and activities. That was a step that we haven't managed to take yet. So in fact, we're basing it all on devices as an approximation to that. I have a PhD student currently who's started to analyse some of the log data from this to see if we can work out ways of automatically inferring the network layer, what the activities are and particularly who the users are that are doing them, whether there's some, any sort of signal you can pull out there. And it's, I think, possibly differently to the way that a lot of HCI takes place. This was building a technology platform on which lots of different interfaces could be explored rather than going and doing the kind of the minimal let's try this interface thing out and just sticking with one thing. So this was something that was an interesting interaction within the project, because all the HCI people were kind of going, come on with like two weeks, three weeks between the prototype out there, we want to get in there fast to see whether this works or not. And we were kind of saying, no, this is the home router, it really does have to work because the first time the internet breaks after we put this thing in, they're going to throw out the window and turn the old stuff back on again, and they'll never talk to us again. So it was kind of a time scale issue doing the research there, which was quite interesting. And it took us some working through. So this was the context. So basically we have, we've got a route today, presents this API, and then we set up a bunch of different interfaces on tablets and smartphones or the things around the house. So I tend to talk much faster by the way, so if there aren't any questions you can stop if you want to just get faster than that. I'll talk through some of the interfaces, some of the things we've built, and test it to different degrees with the households involved. So the first one was sort of measurement and interaction. So these were fairly simplistic interfaces on iPod touches, these were not deployed as. So this was the database, the HWV database, capturing, tracking in real time. And you can post user actions into that as well from the different control interfaces. And then there's a notification service that you can use to allow users to be informed about what was happening. So you can set the notification service to say tweak the following accounts or do a push notification, whatever, to say that something's happened or some trigger's been exceeded. So this kind of thing was pretty useful for some of the households because they could now start to see what was happening. So just as kind of instantaneous thing of being able to say why is the network slow now, look at the thing of laptop is going mental, a bit torrid or something. So this kind of, it's scratch valid for them anyway. Obviously there are weaknesses here. Significant weaknesses, particularly being things like this. This is just a protocol level, pool based, mainly in traffic, which is not really very meaningful and possibly not correct. But it did enough to give people a feel for it, to start with. It's clearly better that we can do that. An interesting feature of doing things in the home is that there's quite a lot of physical access control around your home, usually. See the intent or something. So this kind of physical access and physical devices, physical interfaces are kind of the norm here. So this was a, I tend to call it the network mode, which is not perhaps a very good name. Basically this thing about this big with a bunch of LEDs and an R we know in it interfaces the APIs on the router and was then able to display things like signal strength at the point this device was sitting in the house corner. It could also show you DHCP leases being acquired. So devices coming along, a lot of network potentially. So you could kind of, the idea was that this was going to be a thing of interest on the coffee table.  Without being able to be too engaged with detail of it. And if you were finding that for example signal strength was too variable you weren't getting a good connection in your bedroom or something. You could take this upstairs and sit it down, put it in the right moment and have a look and see what you were getting what was happening. There was actually an interesting thing about that. You might think that if you didn't get wireless coverage into certain rooms in the house that would be considered a problem in most households. You don't get signal on the kids' bedrooms is an issue. So at least one household where they said we don't get signal wireless going into the children's bedrooms. And this is a great thing. We love it. Because it means they can't go online in their bedrooms without being able to see what's going on and see what they're actually doing. So it's always clear that increasing coverage is a good thing. The other thing we did, and it was that sort of physical thing which I don't have a photo of, is using USB keys as a way of controlling access. So we used the USB... You never interface on the next. So we can have a USB key being plugged in with certain files in in the root console on that. And that would permit certain devices to be connected or commit certain devices to do certain DNS resolutions, for example. So this was one way we thought addressing this kids can't do Facebook until they finish their homework. The idea would be that mother would have a USB key and until that key was plugged in Facebook was not available for those devices of the children. And this kind of interaction between control interfaces and these virtual things was sort of interesting. We didn't have to deploy that one to test it. But anyway, there's a bunch of things you can do physically which make sense if things come down. I'll go into a bit more detail now in the physical model of this model of security. So this was a way that we were trying to address the problem of association. So getting devices associated to a home network is one of the key pain points that people seem to have. Particularly when these are not devices with keyboards. So I'm getting a game console or a TV connected where you've got like a one-dimensional interface and you've got a scroll through a full alphanumeric thing to try and input a 20-character passphrase or something. It's really kind of annoying by the time it's even fiddling. So we're kind of wondering what we could do about that. So the way we tried to address that was to kind of invert the problem. So instead of trying to get the credentials from the network into every client device we put the credentials for the client device onto the client device in the form of the QR code. And then we can have a control which has previously been securely associated to scan the QR code and instruct the router to create a virtual access point with the correct credentials for that device. So we've got the router supporting essentially a virtual access point per device and then when the device is on it just automatically connects. It's already got the credentials to create the store. The idea would be that when we do a manufacture time or you take along to good guys and say please give this a thing so we can get on the network and print out the QR code and not stick it on the outside. So that's the kind of sort of interaction there. We did some usability trials of this usability study. So the test condition was that users were asked to construct a network with these three devices. So HP Printer, Sweezebox and Laptop and compare that to using WPS Direct sort of how usable that they find this. In order to make this fair comparison we rewrote the instructions in both cases to try and make the instructions for what they had to do roughly comparable. This was particularly important with the Printer. The PhD student around this test also works as an assessment for the University and I think it took him to work between 20 and 40 minutes to get the Printer connected the first time around and tried it with the instruction booklet in front of him. It was like 17 clicks down through the menu system to get to the point where you can push the button for WPS. So we ran a bunch of people through this 10 with the home network admin of the 16 12 that never used WPS, 6 that never used QR codes. And the home networks that these people have again were in that kind of range in this case 3 to 15 but sort of reasonable sized networks. So when they ran through this test to see how long it took them to get these devices on the network the configuration times were fairly convincing. So with WPS a couple of minutes in the case of the Printer usually following these instructions not much less in the case of the sweetbox with the multi-net system just scanning this QR code it was relatively straightforward tended to get a thing on the network and watch fast. The other thing that was a nice side effect of this is because the interaction remains the same between each of the devices instead of having to go and learn a new menu system you scan the QR code devices on the network and so people stopped having to read the instructions much more quickly as they went through this test as a flavor of users. The other things that we've done have been tested through a more long term deployment of users. So another kind of thing we tried another intervention we tried was to get people into the protocols a bit more. So rather than trying to set things up so they happen automatically you might have an enterprise network where you've got a certain scale deployment you want to manage, you try and simplify things you've got a few experts to run it for tens or hundreds of thousands of people these are much smaller networks and a lot of the decisions that have to be taken are harder to encode in a database or some other kind of backend system. So we amended the DHP server we wrote the DHP server that sat inside NOx so that it would rather than just saying yes when somebody requested an address it pops up and then in this case an HTML5 interface running on a tablet I think that's what it was when we deployed it with the device as a requesting permission so the middle column is where the device is currently trying to set the DHP request ends up sitting. And then if you want to put that device on the network you just drag it across to the committed column if you want to take it off the network or you drag it over to the denied not allowed column. This gives you quite a nice sort of interaction so you can get a bit of metadata about the device as well so you can get the person who's committing it or denying it, but in particular committing it to give you some information about it so what's its name, who owns it which kind of helps with some of the other things that need not to be built around this. And then you can put this as a situation display so anybody walking past is allowed to do this or is able to do this. And that probably makes sense in the house because if they've got in through the front door then they're sort of trusted to sort of step up so they can't get this to be in control. This one also seemed to go down fairly well with people. So part of the things they liked was just that they could see what was around them and they could see how many devices they were or this world that was previously on loan from economics or something. And we took this as a demo of Seacop a couple of years ago we were surprised to see that there were like 3,000 devices available available there in the hotel so that was a surprise for us. They could also see how to get people off the network which was something that turns out a number of the households wanted to do so they'd have guests come around the children would have friends come around and they'd want to have network access while they were in the house but the parents weren't entirely happy with the idea of leaving them on the network sort of forever more. So the ability to have them all when they were visiting then take them off as they left and get rid of the idea of at the same time they didn't always want to have to do that so this kind of a, again just giving people back this degree of control so they can make a decision as to whether devices are always going to be missed off the network in the future or whether this device is only on for the next couple of hours then it's going to go home. It's about giving the control back. The final sort of control interface then was with the DNS intervention the DNS server and this was another way of trying to release access to different services on the network so this was a DNS intercept a proxy that sat in the middle in the router and would deny resolution of certain names based on various information such as a calendar so you can set all the times when the kids were allowed to get on Facebook for example and you can say well they're allowed to do the weekend and they're allowed to do it for an hour in the evenings but they're not allowed to do it because kids access to Facebook seems to be a common driver for some of this stuff and again using some of the other infrastructure it wasn't just about denying it I'll get on to that in a moment with policy stuff but it wasn't just about denying access it was also about calling or notifying of access so that's sometimes more useful than simply saying yes or no again coming back to this intelligible this is an intelligent one so the final interface I'll talk about then is the policy interface we constructed so this was kind of how to set things up so you could give information to users so they could control the network from the interface side of this this was about being deliberately non-technical playful with the interface so it wasn't going to be a standard policy based thing and the idea was to explore sequential art or comics as a way of setting things up which you could then personalise with individual families so this was the kind of interface so this is a policy interface I think this was implemented on iPad you'd have a bunch of these comics that would contain the policies you have for your network for your network so this particular one you can flip through each of these panels so you can say where mom is, where dad is, when the kids are etc on the computer, the laptop, the phone in this case it's being used between particular time ranges you can also set it once it's achieved once it's used a certain amount of bandwidth and when it's being used on certain days when it's accessing certain sites then you can take an action and the action can be blocked in particular device which need not be the device that's being monitored the action might be to notify an individual through a certain channel so there are different things you can set up with this so it's a way of presenting event-condition action policies to people who really don't think in those terms normally and are not really interested in this kind of thing this may seem like why would you ever use that but it turns out we've had two of the deployed households it's been used total of three times to set up rules set up policies on the network and it was the two of them accessing Facebook example that was set so in one household it was set to block the computer so this thing was actually done and so if the daughter's computer accessed Facebook then that computer would be denied access to the network completely which would then lead the daughter having to get hold of the parents to say I don't know what was going on there but something must have happened please get back on the network I need to finish my homework so it was kind of used as a trigger for negotiation even though that was a fairly firm trigger I guess I think one of the houses the notification service was used in a similar kind of way where instead of denying access it was just the model was going to get direct message or SMS or something when certain actions happened on that one so it turns out that if you give people the tools here that they can kind of understand to some extent they do sometimes start to try and use them I think I mean some of the other feedback we got from one of the other employed households was yeah this is not for us I think one of the student households was a shared house they didn't feel comfortable doing this because it would mean that somebody was setting the policy so somebody was kind of taking a chart in charge role and that wasn't appropriate for them so they were much more about using some of the ad hoc interfaces where you could make decisions on a much more kind of live time scale so they're not appropriate everywhere but it turned out this didn't work quite well in a couple of cases what I'm going to do now is for completeness I'll point to some of the theoretical work that went on within the project this was not work I did don't ask me any hard questions if you're interested so one of the things that was that was addressed within this as well how can we how can we try and automatically reason about some of the state going on here and make decisions to help again help the household to understand when things have gone wrong when things aren't being done the way they expect so the approach that was taken by a couple of people in Glasgow was to take a modelling approach from Robin Millman called BiGraphs and try and apply this to the model, the hub network the state of the hub network so BiGraphs basically you can look at spatial and temporal relationships simultaneously so you can look at representing locality and connectivity within this and then one of the neat things about them is you've got visual and algebraic representations which are entirely equivalent so you can reason about these automatically using the algebraic representation but you can also automatically generate and present displays of what the state of the network is currently to the user so the kind of flow that was implemented here was that you've got events coming in, or traffic coming into the network turning into events which get fed into this thing which encodes it into a biograph, analyzes it and then you can both feedback the network to make some control decisions to say for example, this device has joined the network this policy needs to be applied to it it's not been applied yet and you can also log things and basically show for the user this is what's happened this is a display of what's occurred and that's kind of what I just described I guess so that one of the nice things here is this display between the policies that are being applied in a kind of real-time event that are occurring as the network is in use so you can interpret policies as properties the system has to satisfy and then when events invalidate this so you start out with a policy applying a state that says all the machines are blocked then you add a machine that machine is not blocked at that point and the biograph analysis can tell you if this state occurred this is the following thing that needs to happen and they implement a prototype of this and it turns out because of this again because the size of this kind of network and the scope of this kind of problem you actually do this reasonably quickly so this kind of thing could be running live in the background and keeping an update picture of which policies have been applied which ones have, what was in violation what the state of the system was this is a sort of example of the automatic which represents that state where you've got a router and a machine both have wireless content to each other and that is completely equivalent to the algebraic representation in use inside the code so just to kind of summarise the deployments that we did and the results we got from those deployments so the deployments we did for the router and these spaces were fairly long in fact in comparison to a number of these studies that were put in other studies in this area so they were in the households for something like 4 to 6 months so the initial thing about some of the traffic displays was you get this same effect you get with home energy displays I don't know what the state is here in the US but in Europe this is quite an active topic of home energy monitoring we get a novelty effect so when it first goes in people are looking at it all the time like every 10 minutes or something they want to see what they're doing about the first week or two and nobody cares maybe until there's a particular reason to get it out so the case of our module for example that would give you a reason for wanting to know that again but it's not something you really care about enough to keep doing one of the things that we did see though was when you surface what's going on to people and start to expose the network to them you get what they refer to as domestic discourse so it turns out that again this thing about the network the network being quite mundane something that's just past everyday life means that it gets intertwined with the way things are in the home anyways the sort of relationships people have with each other and what they're trying to do and the network is just the other thing that they use in day to day life so when you start showing people how it's being used and when it's being used and for what it's being used this can lead to people basically arguing about what it should be being useful what it's not being useful who's getting in my way intense there was another interesting angle that came out of this which was that privacy starts to be an issue even within the household so one of the things we can obviously do with the outreach is you can record a lot of information about how that was being used certainly once using it, potentially who's using it this kind of thing is already happening in many houses so it was I think at least a few cases the relationship between the parents and the children was would have the parents say well, I want to know what you've been doing I'm going to come and look at your browser history and we'll know what you've been accessing over the last week and this was a normal thing a reasonable thing if you move the recording into the router it completely changes that relationship one of the things you do when you go and ask to look at the browser history is you have to go and ask the person involved to get access to that device when you put it simply on the router you can now do it surreptitiously if you were to choose to do that and so the access to that and how you allow that relationship to allow that activity to happen becomes quite subtle because you have to have trust still happening within the house trust each other and so managing that becomes quite an interesting problem which is not solved yet and it all comes back to this thing of when you're managing a network inside a home network inside a house it's part of managing the household it's part of what happens anyway that people have certain roles from the house they do sort of things it turns out that people in the household want to have some control over this they want to have some involvement in this because it is just part of what they did part of living their lives so to sum up home networks have become mundane what happens there's a sort of I guess the business and the economic incentive here is that there's more and more things that are being predicated on the idea that home networking exists and people can get connected all the time they can do more and more things online there's something in the UK that's coming along in the next year or so the government is moving to what's called digital by default so one of the benefits system is going to be online by default if you're using it online it's no longer going to be a piece of hate being delivered or at least it's going to be a much fewer piece of hate being delivered to people's houses and so there's a real need for people to get online to make use of this certainly not a need very strong incentive to get online and these things still just don't work that well in many cases they don't provide the right to use the control in the right places so it turns out that it is about making the network intelligent it's about allowing people to understand and control this rather than trying to hide it away and making it automatic so some of the phrases that seem to describe this are things like getting interaction so you actually have people controlling it and using it in the way they want it to work and this is not something that the HCI community really seems to do very much of they tend to focus more on the interactions themselves not so much on how you will change the infrastructure to commit different interactions or make different interactions possible so it's stuff like when we monitor traffic it still tends to be these kind of levels you're monitoring protocols, you're monitoring services you're not monitoring the activities that have taken place above that there's a lot of vocabulary that you make available to people who are designing interfaces to deal with that sort of final slide I guess was some of the interesting things that I certainly get to bring me out of this was that networking system group it really didn't need all these different angles and explaining to ethnographers about what you can and can't do from a technical point of view an interesting experience and having to learn some of this vocabulary like affordance HCI terms that I came into contact with for the first time was a sort of interesting thing in the reverse sense and I think this is going to become increasingly important when technology gets deployed out to people who really don't want to be expert and they still want to use it like it's obviously already important and it's going to become more so I don't think there's any questions Can you go back to the previous slide I'm surprised because I would think machine learning would be one of the first to go up here right after HCI in the sense that if you're trying to present someone an understandable description of what's going on so they can manage bandwidth that you would want to build a giant model of what is that traffic or use all the features you have like what was the DNS resolution maybe IP self-inclusive information the amount of traffic, the time you've been doing it all those things can contribute to a much more understandable a model of what someone is doing so that if you're managing the network you go okay this is shopping, this is downloading let me separate the two so I think at some level yes it wasn't something we did within this project although I have a student who now started to take some of the box that we've collected here and begin to do some analysis to try and figure out what the features are and what tools like machine learning can be applied to do some of them as I understand it from colleagues who work at this it might be the case of machine learning it's not actually the right way to go to set the techniques because as I understand it those techniques tend to produce quite a black box that sort of results so you get a model that works quite well but you don't really get any indication of how it's working or what it's using to do that and so something I work with is quite deposit logic that kind of an angle of computational intelligence might be more appropriate because you get more explainability of what's happened but yes it did there's a big gap between the raw data you can collect and the inputs you need to make decisions based on that some kind of processing like that is going to take place given that children are usually much better than adults in opening child proof models locks etc is the idea of Facebook walking Facebook a sane or possible idea sort of so on the one hand people will have free access on their smartphone that's the other thing even relevant if it really is smart that's another question but again it's about putting controls where they can be used within the house so it might not be so much that you want to stop this access and it's a binary if they get access we've failed the situation it's just you want to have a way of enforcing the idea that it's important they don't do it now and the homework is important so you don't make it easy for them to access for you if they try and do it easily at least you get told this is what happened in these cases it wasn't so much that it was the blocking that was important it was that the action took place and that caused a negotiation to happen it caused the parents to have the conversation about how important homework was and how it's doing its pain yeah well that's the thing I wouldn't do it until maybe these exams really do matter and so it wasn't this case it's not like doing network management in large networks where if it fails it fails and that's really bad it was just this thing of trying to service it and make people aware of it so that they could sort it out on themselves it was dry land to see how much difference there is between the internet access network yeah I don't know we went into the domestic network because that was where a bunch of previous work had already taken place they did not necessarily I think it would be very interesting to try this in small businesses small office scenario, guest houses that kind of situation I think they would also be potentially good customers at the start of the talk I was actually thinking that one of the things that you might get into is trying to figure out how to correlate the low-level networking information with the activities for example a kid assessing Facebook can you actually know that a kid actually assess Facebook because he has to be locked in there should be some way of figuring out that password should be sent to Facebook somehow but it does seem like that is an instinctually difficult problem because everything goes through CDN this and nothing so there's a lot to be done there I'm wondering if you guys took a poke at it or not so what we did try there was the DNS thing so it's a combination of DNS and URL monitoring maybe give to use some of that so the DNS proxy that we have there basically doesn't permit anything through unless it's seen a resolution of that article so I guess some of it would be so it doesn't reverse resolution at least I've never seen otherwise it intercepts the DNS request and then the response that comes back and then makes a decision and that probably gets you some of the way there it's not quite as easy as that because there are cases like where you've got virtual it's a virtual web server stuff so use WBCCO UK remap WBCCO.com you've got to catch quite a lot of different names there to actually prevent access to that service if that's what you want to do that seems to be the most promising way we can do something how do you distinguish between accessing some site like Twitter versus every site on the website on the web that has a Twitter button and you know every time I load a web page it loads about 60 other social networking Facebook, Twitter, Google plus one etc how do you like distinguish that or is that even possible I guess it's URL matching but it's pretty hard probably because they're constantly updated yeah so that would be again the combination of URL matching and the resolution that occurs whether the object in the page is fetched at the moment probably the experience that's delivered to the user if we do that sucks to a certain amount because they're going to get lots of crosses on the page when it's still going to be loaded so doing something better then would be better we haven't gotten that far down the route yet so one of the use cases we've talked about is this game who goes to a strange house and he's got a lot of access to it so again some of the policies that the parent has installed for that game should follow so how do you power spot how that can be done so we've started looking at how you can integrate this with mobility and kind of have a home access point with policies that we haven't implemented or employed but yes I think there is you know and if we could figure out where taking that and then applying it to the devices near the networks that are not home wise networks but I think they might sort of find where that would be interesting that almost certainly the class of unrights would be involved with all software so most of the use cases that you've shown they assume like a static model where the user has to understand what happens and then based on this knowledge he can configure some policies for his network have you looked at and you said that the user knows the context that the network behaves and the things that you know the user just said something goes wrong or you know I have a problem in my network now and then you're trying to infer from the current context what's going on so at least in one version of the route we deployed there was a kind of if you hit this button you hit this button when things have gone wrong and there's going to be a lot of the last couple of hours or something which will then get dumped basically it didn't get used so we didn't get that around to it but that would be a thing that's interesting I think some of that sort of facility would be interesting in the context of some of the bi-graph things because that gives you this running picture of what's happening you have a more abstract model than having to dump a couple of them on the Mexican log and then go and analyse it you've got this kind of running model of what the state is and then for the user to say the state is bad at the moment you can then do anything on that you can apply the genome and then think it's gone wrong we have them it seems to me that there's two types of things that you're trying to control one of them are plumbing related who can use this shared link who has the permission to use the shared link and then there's the social engineering part which is I don't want my kids doing more two categories and it seems as though it's the right place to solve the first problem because you're sitting in the path of this shared link but the second one it seems that it's a losing proposition because with a number of 3G connected or 4G connected devices in a home or the ability to connect a next door neighbour or the ability to just walk down the street and use the coffee shop or if you're actually particularly interested in that prevention of particular activities whether it's kids or any type of activity or I don't want someone accessing the bank from this particular device because it's not secure or it's an iPad that might get stolen it seems as though that's a function of the device that's a property that you want of the device that's something that you want to say and there are fairly rudimentary things that do this kind of thing already on particular mobile devices just seems like that's where it belongs because it belongs closer to the application you know who the user is, you know what the things are that they're doing it seems as though this can't continue to sit so even if you figure out how to place it in this home device through which all of the traffic is going as soon as a large fraction of that traffic isn't going through that then it becomes impractical I think and the semi-sophisticated teenager can always create a proxy or a tunnel and so it's fairly easily it's fairly easily got around but if it's something that you've denied essentially at the application level of the device it's probably breakable but it's where it belongs I think that that's I think that it is certainly arguable that a lot of the application level stuff should be done in the application that's really where you've got the context if you want to do it automatically and if you need to have it work one of the things that I kind of realised doing this was that a lot of the time it's not the case that something gets around it the threat moment is not that you want to stop people and that's it and if they get through you've failed the whole thing has broken or give people an impetus to think about this or to talk about it or to negotiate it so it's not the case that if the semi-sophisticated teenager puts a tunnel through VPNs out or something the whole thing has broken it's in the same way that if you ground somebody and they manage to get out for the night it's not the end of the world but this does mean when you find out that's happening you're going to have a fairly serious conversation about their behaviour or about getting some of this information into the normal processes that people have in their families at home so I think it's thinking that it's if somebody gets around some of these mechanisms it's broken it's not the right way to think about it So I still argue that anything that's to do with the application belongs to the end device it's going to be an easier interface now there is no current way to farm or collect that information so it's either setable centrally or measurable centrally or made visible in a central location however I would love to have a simple or love to see it simple to understand the way of representing how much of my home network is carrying Netflix as opposed to Dropbox so 90% of the problems are probably caused by simple things like that but they likely be performance related largely to do with access control it just seems like a funny the wrong place to put the access control and in that particular location I care about why doesn't my network work and having there are more indications of well it doesn't work because of these particular colliding applications or you're doing too much stuff or you've got three people so I certainly I don't think and that's all present there that's all present at that point in that particular network that's broken so I think if I've given you the impression that this is the final solution then that's the right impression I didn't mean to give that impression I think it's part of the solution though I think having some of the software roots does make sense because there are at the moment some of the channels for delivering for example applications that could do monitoring control to a lot of these devices are just not good channels right it's too hard to install a new software in TV but that completely makes sense so the idea that you try and get this thing installed on your smart phone for example there'll be something running on there that would give you the information that would be a condition of them having a phone that would make sense hopefully that would be able to be done in such a way that it would feed into this sort of infrastructure so there's a collaboration that could occur between these different devices different monitoring points this seemed like a good place to start given the state of the art from the moment where's the best place to get enough coverage to start doing some of this and that seems to be a routine rather than trying to go after really different things there are definitely things you can do with chrome plugins and ground plugins that will get you along the way there for some of those applications of a specific application you had a really good I thought the pictures that people draw in the network were really fantastic that's sort of the state before and he mentioned that you want to make the sort of change people's mental model of the network and so I wonder what's the after picture or how did you change people's perception of the network or what pictures would they draw now do you have any results for that unfortunately no, I don't it would be interesting to see what people are under by having them in the service we don't have that do you want the person saying there's a network there I don't know where they're going I don't even know anyone even know it's that the richness of the way people can see the service but it's all the intelligibility like how are people and how is their mental model changed now that they have the system the other thing to take on this kind of management how does he ask about the network so many people are using it the two commercial partners of the project were Microsoft Research Cambridge and BT absolutely provide the whole side of all the DSL they're very interested in some of this stuff from a point of view of support call costs they would very much like to have figures of people's homes that allow them to reduce support call costs because that's a significant cost and having some of this information available and being able to have their people do not even necessarily diagnosis but just to see what's going on and they just go stop a bit of time before the network is busy so we've tried to talk to them about how we can go out getting some of this for the employee but it's one of the problems there seems to be that the cost structures there don't really permit this so easily because the division that makes the home hub is focused on reducing costs and that and if other divisions want to reduce their costs but this costs so it seems like there's this contiguous argument I don't really have any information about and I'd like to get information out of the hood as to whether the home should be really thin and all the stuff uploaded or functionality also whether the home router should be fairly fat and able to do stuff and retrieve information and give measurements back what actually is the best place to put that functionality in order to reduce costs overall we'd like to follow up with some of what we've not actually taken yet my guess would be that in 10 or 15 years that device that we have at home would be would look more like a Marraki device which is essentially a dumb access point in this case an uplink to a very reliable uplink in other words high degree of connectivity as done from cloud that was the entire amount of almost everything that's going on in the home I think that ultimately the home user just does not want to know that stuff the way that you have the control here they want to be able to do that through a web browser they obviously don't care where that's actually sitting and then for allow for more operability or the improvement of an operative that service over time and have it completely outsourced I think I would imagine that would be the way that we were going to discuss this before lunch with Janice so I think that I don't really have a strong view on where that functionality goes but I think that some of the things it has to provide are it has to work when the link goes down because there's more people doing things like media streaming within the home and they want that to work even when the internet's gone away but I think that we tend to think today that a link that connects to our house and as soon as it rains my cable connection seems different but maybe I'm overly optimistic but that's not a sustainable position because it's only going to be mandated by law and through regulation that you have a telephone systems having in the past whether they're monopolies or it'll just be through competition that this is the way in which my friend has more reliable service and where I've got a choice I'll just choose to use that because it's becoming so important to the home that this will be a link that we will think like you know I don't have a backup generator for when the electricity goes down because it's so reliable now I think we are getting to a point where it will just be that we will just assume it's there and it will be really only in a big crisis it's a big crisis I'm primarily concerned about either physically getting out or phoning someone with my 3G or 4G or 5M 5G or my phone which doesn't depend on my own infrastructure at all so I think actually I would buy that it becomes this thing like my driveway which is something I assume is there except for that way we don't get that so I don't have a strong view on it I don't understand where the costs are now I can easily believe that you'd be right that in 10 years time that's the kind of position we're in I think when we do that though it will be important to make sure that you still have ways of getting the information out to the home to make the decisions appropriately so some of the contextual information about what's trying to be done right now it's important in making some of the decisions that have to be taken in controlling that device so there's a thing in the UK where the government are going to say they're going to mandate that on detailed policing of home network use in terms of what websites are being accessed and doing that in a kind of blanket fashion seem likely to always be pretty suboptimal so we had a case a few weeks ago where visiting a relative and we wanted to book a table at the local pub and my wife tried to look at the brewery website where the details were covered and she couldn't do that on the phone because she hadn't proved that she was on the phone and they wouldn't allow it to make that resolution to access that website and it's kind of firstly I think that's pretty excessive to start with but I could see that being reasonable in the case where it genuinely is somebody who is on the phone but they didn't have the information at that point to make that decision appropriately and the only way to get around that is to for those who are looking to say give them credit card details and say I really am over a team it's okay you can do this and then that's the kind of blanket thing and these kind of controls are too course great I think for making this decision so if the control does end up in the cloud it doesn't mean that it's not customized even though it is customized as you want it it's in the cloud but it's not statically customized it's got to be kind of dynamically customized based on the other things in the context of the time they would agree but it can be far more sophisticated possibly you get less benefits from doing things and having people collaborating looking at what other people around you are doing at the same time I can believe that I would imagine that in that time frame for the start of most people will actually have into the home a good 3G 4G alternative and so for they will essentially have a backup plan and just as we then not to think about how our 3G I mean most people probably have a lot of bogus idea when it comes to space stationings to where the home is they don't even think about that as an infrastructure that they will never work will probably look it will have the same feel which is I am actually unaware of the existence of this just as most people don't know where the fuse box is in the house and it will be the same thing just don't even think about it and so therefore it's something that is somebody else's problem like reading my meter is now somebody else's problem I think in some ways it might be even more than that so we've had as part of another project we were looking at trying to do some work to help with digital exclusion problems we got some questions into one of the survey the counter problems just to be clear, so that's the lack of accessibility that some people have so the digitally excluded in the UK apparently the last survey that I've seen or thought of suggested that nationwide there's around 9 million people in the UK who have never used the internet never and there are a bunch of different reasons why that's the case and I'm probably simplifying this too much but I think it's roughly that people over a certain age don't see the point and people over a certain age can't see it but these are the two big drivers as I understand it they're the same 9 million that determine our presidential election so one of the questions that went up on this survey, one of the things is the information that's gathered there is often countered in terms of home broadband wire, fixed wireline broadband and people who are accessing the internet on smart devices and pay as you go similar data plans and all this kind of stuff it's not clear how much of that is included in that and part of the reason it's not clear is that this was just answers in the survey where there's actually interview material that was done but a bunch of people like relatively young people 15 to 25 sort of range I think can't remember the details where they're basically saying oh yeah no I don't have internet access I don't use the internet oh okay so how do you talk to your friends how do you communicate with your friends oh I use Facebook I don't use the internet and so it's this again it's this thing of the conceptions about the service and the activities and all that this one like there's a question if you put everything in the cloud who who is the owner of the pieces of the cloud the service provider that's a question of policy it's not a question of technology that may look different in 10 years the technology may look the same but the law may be different so that's something it is important it is important it's an important policy question and I think that we shouldn't be scared of technology for fear of policy I think that we should separate the two and say you would allow the use of the technology and then you would have a policy that would allow for it to make sense in that context I mean if we took that out we wouldn't have the internet we wouldn't have cell phones because of the data that other people are holding on our behalf we wouldn't have online shopping we wouldn't have there's all sorts of things but I think you have to keep those things hold those things separate but it's easy when you give up control yourself when you have control yourself at least you control your local area you're not necessarily doing up control because someone can present to you the means to configure that however you agree as part of your service so you can imagine the service which is hey I don't want to know anything about my network you can manage it for me or you can imagine the hey I want to control my network give me a web interface that will allow me to do that and give me the phone brain control again that's not precluded by where that control is I think it's just a question of where is more scalable where is more likely to be cheaper because I would guess that the vast majority of people just don't care about that kind of control and hopefully in 10 or 15 years there's enough bandwidth going into the home there's only a certain number of video channels I can be watching where I probably am not worrying too much about the use of it anyway it's a it's probably a less controlled than I'm more worried about the part in why people are not caring about control as they're not aware of what is control and what is happening I'll show you a fear-configuring in fact I would put myself in a paranoid camp but I think that we often make this mistake of confusing the technology with policy I mean I think the reason that Facebook is so annoying to the same people in terms of the data in a sense it's not I would say it's not Facebook's problem, it's not a technology problem it's the fact that legally that there is no there's no attempt to push back against that in a sense they're doing what they're what they're allowed to do and there's a fear of having the many attempts to have regulation the right way to do that is actually you can't do that there's not something to give up that information that's better handled it's not a question of technology I mean I would agree with you in terms of fear I would share the fear but I think they're saying therefore we shouldn't take that technology step it's probably the wrong way to do it I'd ask that by thinking that although it is important to consider them separately in a sense the distinguish between them I think that if you make them completely independent you have the building of things that you want I would agree definitely agree thank you very much


First of all before we dive in I would like to express my gratitude to all of you for joining to this section in this later hour. And today I would take you on a journey about the intersection of AI technologies and Geospatial World. And today I would like to show you how AI based technologies makes a revolution in interpretation and analysis in valuable special data. Today we will speak about the point clouds. Do you know what is a point clouds? Cool. Our platform this is software as a service solution that is provided different AI models for performing of processing point clouds. Engineers who are working with point clouds definitely know that using of traditional approaches and manual tools for extraction data from the point clouds it's extremely time-consuming challenge and it spends a lot of efforts from the engineers in this. And at the same time today's digital world suggests a lot of AI based technologies and some approaches like using of deep learning, machine learning and I assume almost everyone asks yourself how can I use this in your activities. And our main focus this is classification of the point clouds. While point clouds contains enormous amount of information, raw point cloud this is still like having a book with jumbled letters. Classification at the same time allows you to organize all this data making interpretation of this data and further analysis much easier. For classification our platform suggests currently four standard classifiers which are purposed for different types of surveys. You may use these for extraction objects in different landscapes. For example the most general solution this is standard classifier for airborne laser scans. You may use these for extraction, building, classes, trees, vegetation, vehicles, ground. This classifier is using intensity channel. Apart from the general solutions we also suggest more specified classifiers like classifier for highways. This classifier is able to extract objects that is really one for highways. It may be fences, objects of road infrastructure around the highway, road signs and so on. Apart from the four classifiers that already is available for using at our platform our team currently working on the two more new solutions that is also very general but will be very useful. These solutions will be delivered in the first quarter on the 20-24 and one of them is purr-sport for extraction of objects of road and street infrastructure like road signs, fences, traffic lights, catenary pools and one more solution that we are suggesting for extraction of electric lines. This will be a separate solution. Despite these classifiers can cover a lot of general use cases. There are a lot of cases where you need to solve very very specific tasks. For these cases we can suggest custom solutions. It may be custom classification, future extraction, vectorization and so on. The next example illustrates this very well. In collaboration with Dibinets from Deutsche Bahn we made a specific classifier for extraction of objects around the railways. SMSE, this is platforms, catenary pools, equipment for electric lines, etc. Coming back to time-consuming question, how many time probably you may ask yourself, requires for processing of these point clouds and the next example illustrates this very well. Together with City of Munich we made a classification of scans of whole City of Munich. It was more than high for a hundred squad meters of area. The data set contained 2,000 point clouds with some amount of 84 billions of points and we spent just four days for processing all of this enormous data set. But what about another use cases, not only classification? The next example shows you how AI can be used for extraction of features from the point clouds. At this model that we implemented for performing of three inventories, we performed instant segmentation of the forest and extracted features from each segment. SMSE on the screen, this is type of tree and other parameters. In the end as a result we have map layer which contains information about all the trees inside of forest and information about them. And probably after three days at this exhibition you already know the most challengeable thing in extraction of data in point clouds, this is vectorization. In collaboration with the Autobahn we made AA-based model for extraction of road outlines. At the screen you may see green lines. This is original line that replaced manually. Red lines, this is lines extracted in a fully automatic way using AI. And as you might see there are a lot of cases in processing of point clouds that can be solved using of AI technologies and all of them all of them can be solved potentially at our platform because we provide so wide amount of solutions for different cases. But definitively if you know how works neural networks you have a question where you can get the data for extraction of these features from the point clouds because any AI model firstly requires get training data set. We also can suggest solution for you. Our platform contains some tools for performing of manual classifications. At this demonstration you may see a user can use applied segmentation and selection of different set of points inside of the point cloud and then can assign these selections to different classes. As an outcome of this process you have fully classified point cloud that can be used in your further actions or for training of automatic models. The same is really one for vectorization. Recently we delivered our new features allows you to make vector objects around the point cloud manually. You may draw polylines, place point features or create pelagons. You can combine this manual approach for placement vector objects with classification. This is very efficient way because when you're placing the object on the rail cloud you may skip some details or make mistakes but when you use reclassified cloud for making manual vectorization it increases your efficiency a lot. Recently I would like to have small announcement. We started working on the general solution for extraction of vector data in a fully automated way. It will be AI model purpose for extraction of road outlines and road markings from scans of highways. It will deliver it in the next year and that's why I invite you to follow to our updates and then try. In the end of my presentation I would invite you to sign up at our platform and try to use all these features on your own in your activities and estimate how you can increase your potential using AI, how you can increase your performance. Thank you.