But not in a pervy way. Chris will be disappointed. Anyway, we're going to be talking about pointers and interactions, a topic which is very close to my heart. Sounds like a dating website, doesn't it? Anyway, so I'm going to start this by just introducing the panel. We've got over here the lovely Patrick Lauker. I always get that name wrong. Have I got that name wrong? Patrick Lauker? I'll do. York Tangdela at the end there. York is author of HammerJS, which is a library that I used very recently. It's wonderful for dealing with touch events and now pointer events as well more recently. He builds mobile apps for a living, loves to experiment with new techniques. Patrick, I didn't tell you what you actually did. You used to work for opera, didn't you? Being a dev rail guy. He's now an accessibility consultant and a member of the pointer events working group and WC three touch events community group. We have Pete Smart over here. Pete, very much UX focused. He's done a lot of incredible work with Viz Cities recently with his partner. I was going to say with partner, I didn't know what to say. Not partner. Rob Hawkes, incredible 3D visualisation. She should absolutely check it out. He's also the author of 50 Problems in 50 Days, which describes my last 50 days organising this portion of the event. Finally, we have the wonderful Steve Workman. I've known Steve for an awful long time and incredible web developer. He's a very insightful young man, I find. Next, we're going to pass over to Rick Byers. Want to introduce me first? No, I don't need it. Rick Byers is from Google. He's a wonderful man. He's going to be talking about pointer interactions. Thank you very much, Rick. Thanks. Let me just get my slides displaying here. Sorry about this. Okay. Yeah, we did it on purpose, sorry. All right. I'm here to talk about pointer interactions. To me, this is about how do we involve input on the web? First of all, even just looking backwards and saying how do we catch up to native mobile? I think there's a lot of ways that input is better on native today that we really need to catch up on on the web, but also looking forward and saying what are the things that are coming and how can we prepare the web to make sure we're ready for all sorts of new ways of interacting with content and applications. I'm going to talk about this from my biased view on the Chrome team.  But first and foremost, this probably comes as no surprise. Our top priority on our team is performance. If your site doesn't perform well to your input, it doesn't stick to your finger. If things don't respond immediately when you touch them, or even on a laptop, if things are janky when you're scrolling, that's a real problem for user engagement. So this has got to be the top priority. But we've also got a big problem with richness. Today, it's possible to build really rich interactive user interfaces, and frankly, it's easier to do that on other platforms. We've got some problems on the web where certain types of UI are really hard to do well without re-implementing all sorts of browser features yourself. Last but not least, we've also got a problem with rationality. This is just the idea of the pit of success, the idea that the obvious thing should ideally be the correct thing most of the time. You don't want to have a million different foot guns where every time you try to do something that seems obvious, you shoot yourself in the foot and something breaks. Like I said, I think we're not here on the web for any of these things yet. I think we've got to fix all of this. But what's really worse is that there's these trade-offs between them, and we keep dancing around. For example, over the last couple of years, with all the folks on mobile, I think we've really put a lot of emphasis on performance, and I think we've lost some richness and rationality as a result by not focusing on it at the same time. So I'm going to talk about a big problem for each of these areas and just talk about how it's affected performance, richness, and rationality. So first one's been talked about a lot to death. I'm not going to spend too long on it, but touch latency is definitely too high. On a touch device, we're trying to present a physical illusion that you're manipulating a physical object. As soon as the latency goes up, so it's not sticking to your finger, or if it's variable latency, if you've got latency jank, then it destroys that illusion, and that's a real problem. It really causes engagement to collapse. There's been a ton of improvements here over the last several years. Probably the biggest architectural change to the platform, even though it's not exposed to the apps, is that most browsers now will scroll on a completely separate thread to try to insulate the scrolling from what else is happening in the browser. There's been a ton of talk about the 300 millisecond click delay. We've talked that to death, but the one thing I'm most excited about here is that we now have a standard way of turning off the click delay on individual parts of the page without turning off anything else. You just turn off double-tap zoom. We've got a new CSS property called Touch Action. That shipping in Chrome 35, Firefox is going to have it soon. Touch Action does much more than this, but this is one of the things I'm excited about, that Touch Action lets you do. We continuously have been making tooling better. I think we still have a long way to go. On my team, we've been working, for example, on trying to accurately correlate input to painting. It's not good enough to just look at your timeline view and say, oh, I'm getting stuff at 60 frames a second if that painting is happening a second behind when the input came in. We've done a lot of work to plum latency tracking data down through Chrome all the way from the input events coming in from a kernel down to when we tell the GPU, hey, we want to display this. I'm hoping that eventually we'll get that exposed in a more friendly way through DevTools. For now, it's there on Chrome tracing if anyone wants to play with it. In terms of richness, this is where I'm starting to focus a lot more. I think we've neglected this space for a long time. As we've done all these fast paths, we've taken control away from developers in the name of performance. Frankly, I think when we look at it really closely, it's not necessary. There's ways that we can give developers control, but still maintain high performance, or at least high performance by default. This comes back to the extensible web manifesto that I think has been mentioned before. As a community, we're never going to evolve the web to be the platform that we want it to be to compete with all other platforms if we keep kind of spending years focusing on these high-level features and trying to agree between browsers and the high-level features. Instead, I think we really need to focus on what are the core primitives, what's the kernel that all browsers are going to expose so that we can enable jQuery in different frameworks and all of you to innovate like crazy and try new things without having to wait for the long drawn-out standards process that really bothers me here. Scrolling is so critical in this experience, but if you want to customize it, for example, there's a great library called iScroll that customizes scrolling in various ways, you've got to reimplement everything yourself that the browser was already doing and you're prevented from doing what the browser is doing. You don't have access to this separate thread I was telling you about. Everything's going to be blocked on your main thread, it's going to make it jankier. Depending on the browser, you may not have access to the precise physical pixel locations that the browser is, because on mobile devices anyway, one CSS pixel is more than one hardware pixel and only the browser knows in many cases where exactly the input or the scroll offsets are. And you might also not have as accurate information about velocity that the browser has. I think we need to fix all these low-level primitives so that somebody like iScroll can go and build scrolling that's just as good as what the browser does with some additional features. Even more immediately than that, there's a whole bunch of UIs that are really popular effects like pull the refresh where when we've made these trade-offs in the browsers and we've said scrolling performance overall isles, whoops. That's right. I guess that means I'm talking too long on this slide. Scrolling performance overall isles means that we've taken control away from you. You can't say, well I want to scroll but now I want to switch to drawing my own little custom effect that the scroll normally has and I want to keep that lock step with the scroll that's happening and I want you to tell me when the user lists their fingers so I can change that effect. Similarly, Microsoft has the snap points feature in IE that is a great feature but we should be talking about how do we enable the web so that people can innovate on features like that without always having to come to the browser, always having to add every new UI feature at the top level of the browser. One of the ones that really bothers me here when we said that scrolling is happening on another thread it's a problem for rationality because you're not used to having the reason about multi-threaded behavior in your UI but it's also a problem for richness and the kind of UIs you want to present. Everything from on a mobile app you never see checker boarding, right? Why does a web page always have to have checker boarding? Maybe for your app you might want to trade off and say, you know what, I would rather have scroll jank than checker boarding or I would rather have the ability to have perfect parallax where things are updating exactly with the scroll position rather than doing the scroll asynchronously to those sorts of things. I think it's been naive of us as browser vendors to say, we know the best default for all scenarios. We're going to trade off performance and richness for you and not give you any say in the matter. I think we really got to start giving control of that to developers and also how exactly should touch behave while scrolling. We're making a big change here to this in Chrome 35 I won't go into the details now but you can read all about it. As soon as you start scrolling, we stop sending you events entirely and that's terrible for richness. It means you can't build UIs like pull the refresh. Okay, now in terms of rationality, one of the topics near and dear to my heart that people bring up a lot is why should we have to build, why should we have to write different UIs for different types of input? We should be able to make it easy to have a single set of APIs that work for all sorts of different types of input. IE has an API called pointer events that we've heard about. We've been working on standardizing it for a long time and when we should be talking is this the API that we want the entire web to move to? In doing things like that, how are we going to make it so that when we're exposing these low level input APIs to web developers, how do we make sure that they're thinking at a high enough level of abstraction that their sites are still accessible, that they're not just targeting screen readers and all sorts of assistive technology as well? And then looking further out, there's all these exciting types of input on the horizon. Even old school stuff like directional pads on TVs, the web doesn't handle very well, but Microsoft has perceptive pixel, these gigantic touch screens where you can have multiple people touching them at once and track 100 different touch points. How should the web interact with that? Or what if you've got multiple users each with their own Wii remote voice input, depth cameras, Google Glass heads up displays, touch screens like the Galaxy S4 that can measure, can tell when your fingers above the screen, haptics. What if your touch screens can start to give you physical feedback? What worries me is if we block all of these new technologies on, let's wait until we have a consensus, let's wait until we can all agree in standards bodies how this should be exposed to the web, we've got a chicken and egg problem and we need to be talking about how do we, what else is next and how do we prepare the web so that we can innovate without having to have consensus first really. And part of this is just discussion, that's what Edge is all about, I'm looking forward to this discussion, but there's also plenty of places online that we should be discussing this. We've just spun up the touch events community group, which is a group within the W3C for people that are having, have questions or issues with touch events and want to see touch events get improved. That's it. Wonderful. Thank you very much. I thought I was going to hit you over the head with that bottle. I'm really sorry for that introduction. I should explain to myself. The reason I didn't really give you a very good introduction is because I wrote it by hand and when I got up here I couldn't read it. Rick is an engineer working on touch screens a point in crime, a member of the point is events working group and the touch events community group. Just to clear that one up. So first up we have someone like Andrew Betts with a question. Is Andrew there? So web apps that use touch gestures can have problems with iframes swallowing touch events. Is the web developer hamstrung by their inability to exert complete control over user interaction in ways that native app developers can? I guess we do give over a lot of power to third party control, third party websites in iframes to the browser. What do people think about that kind of giving over that control? Should developers have more control? Start with maybe Patrick. It's a difficult one. Developers will always want more control. Be it with CSS stuff. I want to control exactly how my users will experience the concept I'm creating and the same way with inputs as well. So output and input. There is an argument to be made that at certain points we should limit what the developer can do to mess up the conventions of the platform, for instance. You don't want every single website to just behave in a slightly different way and the user has to relearn it just because the developer thought, A, this is really cool. I can do scroll jacking but now extending it to touch gestures and everything else as well. There should be some kind of convention, but on the other hand the topic we touched on at the start is that we have a lot of these cheap laughs, I promise. We do want to innovate, so we can't at the same time also say, I'll say we even though I'm not in that fold anymore, we as browser developers know what's the best interaction model. Developers will want to experiment with new things. So it's trying to strike the balance but there will be situations where probably for security reasons you don't want to have websites that completely take over the interface and start showing things that the user thinks they're doing one thing but it's actually doing something else in the background. It's a tough balancing act to make, I would say. Jorric, you're developing hammer. Do you want to give more control to developers? How does that work? I would like to see a lower level API when a new device comes up like the Myo or the Wii modes. I just want to get that information in my browser. So I can build my own gestures and don't have to wait for four years or something. But when it arrives in the browser and for everyone. I'm obviously a huge fan of giving developers more control but the scenario they mentioned in this question was about iframes and we actually, I think it's fixed in all our stable releases now for a while, actually quite a long while where the touch of any API will tell you what other fingers are currently down and so it was actually possible for an ad or something running in an iframe if you touched on it to receive information about where your other fingers were and that was a potential leak of privacy and a security concern and security is kind of the thing that I'm saying we shouldn't focus as much on performance and we should be trying to maintain our performance but give the developer more control of security is the thing that trumps all of that. I suppose if the users don't have confidence and it's the unique strength of the web. Will you take the opinion there that with gestures that you should be giving developers kind of more control. You said that in the talk as well. You want to be giving lower-life APIs. What are your thoughts in terms of UX around web design and web development? Do you think we should, is there a benefit to be had for gestures which are universally across an operating system, across a device?  Interesting. The conversations that I have with developers often go like this. Guys, it would be great if we could come up with this really innovative new feature. Why don't we try scroll-jacking for example? Why don't we go into that realm and see what's possible with it and how we can surprise the user and get them to reconsider the experience that they're looking at. I think the common response that I get is to arrive at the natural default of the browser. We don't want to confuse people. After many arguments, I would probably side with the developers on this particular front. Although I want to innovate, I want to create experiences which people would find themselves immersed in, surprised by and excited by, I think there must be conventions which exist across different platforms, across different browsers which are therefore expected because not everyone wants to be surprised. Not everyone wants to have this kind of immersive experience. Most importantly, people don't want to be confused and I think that is the bedrock really.  To me, this represents a fundamental lack of layering in our platform. Many other platforms, for example, I know on Android, you can replace little classes or hook into the process but you have to re-implement everything yourself so you can get the native look and feel but also customize it slightly. I think we failed at that in the web. I think one good example where potentially there has been some improvements just with some little personal experiments maybe is, I don't know if you guys experienced this but you're scrolling through a web page and you suddenly get to a full screen Google map and you're on your phone and suddenly you're no longer scrolling the page and you're suddenly inside the map and that's an area which we've tried to innovate in around that which allows you to declare when you would like to interact with the map and actually when you are still scrolling and I think there should be space to allow developers, designers to innovate in those areas and try and break away from the iframe suction of doom that you go into but there's got to be a limit I think convention is also really important. You're quite a good mixed bag of dev and design. What's your thoughts? With things going into with all our touch events going into iframes one of my concerns is if I were to make say a web component talked about today if I was that web component developer I would obviously want all the touches to be sucked into to what I was doing because I said ok I really know what this component needs to do so I should have control at this point why are you trying to do the rest of the thing and the more this goes on with web components and with other other areas advertising of course iframes maps is a great example the more this happens the more important that actually having that kind of override is going to be so that when you do as a developer know better and quite often you do know better then you can override that and you can actually make a difference to the web application and improve the user experience. Andrew if you if you have a mic has anyone got any thoughts on this at the front row because you look like you were doing something else there Andrew pay attention so it's kind of touching on your point and you actually the idea is actually a lot of times the interface the developer is building it knows best and for example you're a tangible example about like the Google Maps we actually give them about the exact same thing there's a response website, there's a big old map but what we did is actually once you went into one layer and snapped downwards we actually changed the Google Maps so it's no longer dragable, the interface gets changed it's now double tap the zoom but I think that would only know you can only really do that at the developer level you can't really do that automatically I think in the browser there's too many assumptions because there could be other pointers where I don't want that to happen for whatever reason maybe we would want an immersive experience there's a notice of the developer making it to make decisions about that absolutely you have to design around what you're doing I did the Met Office Met Office Weather Observation website and the mobile view for that and most of that is a big Google Map and the really important thing was on the different browser sizes actually giving you space for your fingers down the side of the map so that you could still interact with the map but you could also still get out because otherwise you're just literally you are stuck next question we've got Andres Brovins at the front here a nice simple microphone there This is a question about gestures Is there a case for custom gestures that users would be unfamiliar with rather than standardising a set of gestures that are semantically well understood? Your library is a lot about gestures What do you think? You should I guess you should be able to write custom gestures but it also makes sense to just use the system default gestures because the user is expecting like you can swipe and it acts the same on that page like the other page but if you want to write a custom gesture like swipe with with I don't know you still should be able to handle those things It's a really hard writing gestures It's really complex I've not done it so much with touch but definitely with things like connect trying to understand what someone's doing and how it follows through especially if it's really complicated to do maybe not for the uninitiated Is that your experience? It can be complex What we often find is that the UX is actually harder than the engineering We actually had a bunch of touch gestures when we first introduced the Chromebook Pixel and we put a bunch of work into the engineering because users didn't know about them they were hard to discover it was hard to train people to use them and ultimately we said we should stick with the simple things That said I think it's essential we can't innovate we're never going to come up with what the new interaction modes are unless we give people the power there's going to be some the next viral app that's going to have some cool thing that you do in your game to make it do something different it's going to become the standard It's an interesting point but our developers are necessarily the best people to make decisions about gestures generally they're probably not and if you give people really I don't mean to offend anyone but I am a developer myself so I think that it's very difficult to write gestures which are well understood across the system It's definitely more of a UX question I would say because from the technology point of view I mean not in point of events as such specifically about the Microsoft implementation but there is a separate part that Microsoft has in IE that isn't specced the W3C which is all about the gestures and how to actually write programmatically what a gesture is picking out say two fingers that you've put on the screen and then following any changes between them the angle etc so technically I think that is not an issue we can and we do with also with libraries like Hammer write our own gesture code and it really is more a case of are there standards I mean Luke W has documented a lot of the standard gestures that you get on a variety of platforms there are similarities but also quite fundamental differences in some cases if you go from iOS to Android for instance there are a lot of pinch zoom that kind of stuff nowadays we know about it it's become common knowledge but when it was first introduced nobody had that understanding that that is there so UX wise there is the argument of how do you teach your users to that there are gestures how do you hint at gestures without doing a big before you can use the app here is a ten minute tutorial on how you move how you shoot how you go into your inventory kind of stuff it's more of a softer issue I would say so from the technical point of view gestures are here we can create gestures we can hijack pointers we can hijack finger movements but should we it's usually more the fundamental question Steve I know you're quite involved in the London sort of web standards meetups and groups do you think we need more standardisation in this whole world of gestures there's an awful lot of proprietary technology in this is there maybe room for more collaboration I think so definitely so there's a lot of work going on in the different standards organisations in kind of different routes going through this obviously Microsoft's implementation is one another is one from Apple so we haven't talked about enough yet today who are doing this spec called I think it's Mondo UI we talked about it last night Indie UI which is not a gesture thing specifically but relates to actually system wide commands so like you were to do undo it would also trigger an undo kind of action or a named undo action throughout the web as well and that kind of thing also then expands into gestures at the same time so Apple is going a completely different way of this from Microsoft and in the end these people need to talk to each other and we need to standardise something like this otherwise we are going to get to completely different implementations that are probably going to be incompatible with each other but is there a case for custom gestures I would say definitely yes I would say that especially when you stop considering traditional point and click interfaces and touch interfaces and start thinking about the new input devices that are coming onto the market at the moment things like the Myo which reads the electronic signals in your arm for example things like the leap motion when you are interacting in 3D space I think these give us fantastic opportunities to start exploring custom gestures some of the work that I have been doing with very talented Rob Hawkes should probably be sat here rather than me looking at how to do gestures in 3D space and you can't just simply re-create and touch gestures in those spaces where we are trying to create 3D cities simplicity for real life how you extrapolate out a city and how you break it down and see different parts so we are talking about gestures like this where you are able to pull apart things like being able to pull apart a watch that famous diagram that we all see I think there is a real case especially as we start to look at new forms of input to look at gestures which are more suited rather than replicating two dimensional gestures that we have now Give developers power, standardise more talk more We are I have a shabba Don't have a second name My question is about browsing the web on devices like Google Glass or Auto 360 the watch is a very different experience as they do not have the traditional methods of input how do we develop or adapt our web pages to better support such devices would a device information API be a good idea You were talking about new devices I was talking about new devices I would come at that question from a high level I am sure these guys when we talk about the technicalities but I would probably start with for me what is the most important thing is the user need or the task that that particular person is trying to achieve on the device that they are using and we might not even know the task for example so do we need to be agnostic I think we also need to consider things like with Glass for example that we have to display information it is very very small and the input as well we are looking at non traditional forms of input we are talking about voice in that particular case so if we are talking about browsing the web on something like Google Glass what should we be looking to do I think those three things come into play at that point You were talking originally you were talking about this I don't know about the UX side I need someone like Pete to tell me what the UX should be but then what worries me is how do we make sure that people can experiment with these things we can't wait for whatever Glass events W3C specifications and maybe we can do this with low level APIs if we can address the security issues maybe we can give some limited raw USB access to pages or maybe we need something like into UI events just a higher level semantic and just be able to say someone is manipulating this object by rotating it and scaling it it can be completely different from how they do it on a touch screen and the browser and even the page doesn't need to know there's also this chicken and egg problem you've got to design for compatibility most websites out there are not going to be designed for your new watch so you've got to come up with something so that you can do a really good job for existing content will still allow some incremental extension to do a little bit better for sites that are really designed for your special type of input I agree we obviously talked about this over the last day or so as well and on the pointer events mailing list while we're working on the standard and on the one hand it's really good for a developer that we're now working on these low level APIs I want to get access to is it a finger or a stylus or is it three fingers and what are the X and Y coordinates and that's really great that I can do that as a developer but in most cases if I'm developing an application generally I don't really care how the user is touching the touch screen with a nose or with a finger or any other body part or any other device or if they're using voice what I really want is usually the intent the user intends to activate this button the user intends to get more information or manipulate so even though pointer events are a step in the right direction I think there might already be a little step too late because now we already have touch and stylus and mouse and everything else there's all these new interfaces and really we should be looking more at high level stuff Indie UI is probably a good step in the right direction it's been a bit sidelined because it falls under the main its accessibility so it's just for blind people and stuff so we're not going to care about it so I would say have a look at Indie UI stuff it actually abstracts a lot of these things that in most cases unless you are trying to create something that has a custom gesture or that really takes advantage of something that can only be done with say elite motion if you really just care about the user wants to open this document or manipulate this thing or re-sort this table you want the intent you don't want the actual raw bits and bobs of which key number was pressed which key code was pressed because you're just going to end up in a situation later on as a new device comes out that you're going to have to reinvent or re-implement a whole new model which is what we've seen with touch events it was a great idea but all of a sudden then something new comes along and instead of which was a really wise decision I would say for Microsoft instead of then saying ok we're going to have stylus touch and whatever else touch connect touch they decided to abstract at least anything that's pointer like but it should really go a step further it should have included keyboard in my opinion and just be more device agnostic in general and yeah Indie UI is probably not perfect but it does move towards more that idealised goal of just looking at intent rather than raw gobbins so we've got a question I think Chris was with his hands up simulation makes a lot of sense simulation makes a lot of sense as well for example I've written a few things for leap motion and instead of doing my own handlers I just fired a click event after that so if somebody wants to build a new UI that takes this in all they have to listen for click events and click events are great cos they're also keyboard events so instead of reinventing and putting more and more event handlers on our interface for all the possible things just firing or generating an event that is already listened to is a simpler way in than having another library around that absolutely it's like focus blur and click are probably the ones that if you want to do something today that will still work in a few years time with whatever other device is there they're high level enough and they're abstracted enough from the actual what is the type of input that it will work the reason that things like touch events and even pointer events have to fire other mouse compatibility events on mouse over on mouse out and everything else is mainly because I think we shot ourselves in the foot at the time when we started inventing these very device specific event handlers and that's why now just to make sure that the web as it has been written and 99% of the web that's already out there to make it actually work on any new device any new input needs to at the UA level fire these compatibility events which is a shame and we don't want to end up perpetuating this that in five years time we're going to have to end up when we're using our minority report until your arms fall off kind of interactions that all of a sudden they have to emulate from that it needs to emulate pointer events which then need to emulate mouse events just to make sure that you keep cascading back to the old technologies just quick questions and quick comments sorry for the run it's kind of a question rather than a comment I might maybe a caveman but the pointer events and touch events I get they apply to PCs and smartphones that have browsers in them quite often made by the companies that put the browser vendor on the device this watch can get a connection to the internet but there's no standards on that device and the leap motion to get it to talk to a web page requires something in the middle to get the connection to talk to anything on the web requires something in the middle and I might be missing this but what devices are actually talking web languages like web compatible languages and if we're talking about these pointer events for these kind of or any kind of standardised event for these kind of big we're talking about minority type minority report type interactions I don't I might be missing this but where are the devices TVs aren't getting there they're not TVs are going to fire pointer events so if I'm kick me if I'm dominating this we'll get on to this later I think we'll go to a new question we've got Matt Andrew there's a very similar question coming up to you boring sorry laughing at him not this is from Patrick Law touch events are a very simple mechanism for touch whereas pointer events are a far more detailed abstraction should a browser vendor ideally implement one or the other offer both separately or some kind of combined as is my own question I'm not going to answer what do you think there must be a problem for browser vendors because you've got to support touch events because that's what everyone knows and it's so easy to tighten to what already happens with clicks and we can't stop supporting something we've already supported and it's not even as easy as saying well on the page we'll decide one or the other because you've got to make sure there's a transition path for libraries that operate within an existing document so google maps for example right if we said the page is either sending pointer events or touch events then google maps could never use pointer events the whole page would be broken so there's this difficult transition path I think firefox is planning on supporting both touch events and pointer events I'm really optimistic about that I'm hopeful that at some point we'll do that in Chrome as well we really want to make sure that we're only going to support APIs that really last that are lasting APIs that eventually all browsers are going to support and as long as the juries kind of still out on whether or not all browsers are going to support it we want to be a little bit careful to make sure we're not introducing something new that's largely redundant that ends up not standing the test of time so we're really looking for feedback from developers to make sure it's something that developers really want and intend to use for a long time and then we'll support both but we can never get rid of the stuff we've supported previously so that's probably exactly my problem so the uilleadonyell.com big website if we break something it's going to break hard so if I were to try and convince my boss to say okay I want to implement pointer interactions it's going to say okay that sounds great what browsers is it supported on I'll go IE 10 and 11 and I'll probably stop there because right now that's all that supports it Chrome will get there which is great Firefox will get there which is great Apple which is Apple Safari which is let's make about 25% of our user base they probably aren't going to implement this spec and so if I'm going to spend what will take me a couple of weeks to completely tear out our interaction model and put pointer events in there instead even pointer events within all the back polyfills in order to make it work on Safari that's going to be a massive amount of engineering effort for something which probably isn't going to give me any business benefit whatsoever except that it might get rid of 300 millisecond click time and that's going away with something else anyway so as from an implementation standpoint and a purely business standpoint actually selling pointer events is quite difficult unless you're making a Windows 8 app in which case go for it and if you are making Windows 8 speak to me at the end of the event I mean you're doing with hand gestures you have this problem you must be dealing with all the time well when you implemented pointer events about having to deal with both at the same time yeah it's really yeah it's like when you use a touch event it sends touch and mouse to the next to each other and sometimes in a different order in other browsers so you can't really find out when it's like a touch or a mouse and then you have the pointer events but it only works like in the latest internet explorer versions and it's hard to I was looking at the triage of all the different events that happen when a touch happens on the screen I mean you must live this stuff happens you have to put it in there because of legacy behaviours and so forth it's extraordinarily complex how do you how do you work around that sort of complexity who's heads it in so first of all we've got to do better to document it I think there's a question coming up about this I think the touch event standard didn't really document any of this stuff on retroactively to try to take the majority of what the existing implementations had done and you know what when I realized two years ago that touch event standard didn't have anyone from Google on it that I realized there was a problem there it was all retroactively trying to document what had been done without trying to real point of standards is to get the rationality in there from the start so hopefully I think we're doing a better job with pointer events we just defined pretty precisely and then Patrick came along and said let me give you an example and show here's the exact things you should expect I think that's the way forward we've got to do some of this stuff for compatibility but hopefully we can rely on layers on top to make it simpler so your UI framework hopefully we'll say on this UI framework you just got to deal with the events it generates and you don't need to worry about different browsers and their old compatibility modes and what the differences are between IE and Safari and hopefully we can smooth a lot of this over with frameworks I think a lot of the complexity they get nowadays is coming from the fact that we've made bad design design decisions in the past mentioned already on mouse over on mouse out and everything else which is the reason why touch events need to fire mouse compatibility events because we as developers were told years ago to do a hover effect you just do an on mouse over and we've all been taking it and cargo culting it around and now that content is there it's set in stone and these new devices need to use that if we could start fresh we change it but the reality is we are living with both that legacy content and we're moving towards a multi input world I mean already now we've got devices a few years ago it was unheard of to think oh you might have a laptop that also has a touch screen touch surely means it's a mobile or then it was or it's a tablet and now it means the fact that you get touch screen event there is a touch screen it doesn't say anything further and that's the fire alarm test there is a fire alarm test which we couldn't avoid today attention the public address and fire alarm systems are about to be tested okay do we have your attention will sound first followed by the evacuation message right okay should we just run around and panic do not take any further action no action why if a fire breaks out while there is a test no one start a fire we didn't start the fire it was always burning attention an incident has been reported with isn't a building I repeat it hasn't been announcements right it's like space invaders this is good though attention please we have an emergency an emergency double plus good it's not just an incident anymore this is escalating test is now complete yes last bit at the end Martin that last bit at the end was brilliant if you have problems hearing this please alert someone accessibility we'll talk about that next I can't remember where I was we are in a multi import world if you can't remember let's just go to a new topic Danny Croft do you have any topic because we have no idea where we are now are pointer events over complicating touch interactions no alright I'd like to echo no we've got seven minutes of this guys no no they I'm aware of that and it probably shows up quite a good idea also really really tell the people which got the fire alarms please don't do it anyway so yeah but I mean are they over complicating there is an argument though from a developer point of view playing devil's advocate here obviously not as a real developer from a developer's point of view it is quite a complex thing but is it better to kind of with touch events you kind of we papered over the cracks of we needed touch we kind of got it in there I think with pointer events we took a step back and we looked at well what are all the different types of interactions that could happen on a device realistic interactions that could happen on a device I mean is that are we making too much of it though because it's all good on paper this abstraction but is it too much of a complicated abstraction I think if you think of devices strictly like you've got tablets and you've got laptops then maybe you could make an argument and say pointer events is over complicating it but Microsoft believed that that's not the only two kind of devices we're going to have and we believe that at Google as well and that we need to be prepared for devices that are a hybrid and like there's a continuum between them and then the complexity of having touch events and mouse events on these kind of devices is really high like you probably don't realise the extent to which touch events implicitly capture to where you start touching and mouse events don't capture at all and the implication that has for when you remove a DOM element that is in your event chain somewhere you know there's very subtle bugs that result kind of from these differences where pointer events unifies it all and says there's one model if you can you don't need to worry about any of these old things just target this new model absolutely the complication comes from when you try and add things on top of pointer events so if you say we're to look at the TVs it's trying to add the D-pad into that which it currently doesn't do then you're trying to fire two models and that's where you kind of get into much more complicated stuff if you're still thinking about the polyfills and everything, HammerJS solves pretty much all of that for you so it's just when you're adding more and more things on top and as we've touched on already if you're trying to invoice onto the top as well you're trying to have a UI that reacts to probably far too many things and that's when it gets pretty much overly complicated Well I think with pointer events the nice thing is no it doesn't over complicate it it actually simplifies it in most situations because already with touch events if you wanted to properly separate it you'd have to listen for the normal mouse events which work in a certain way and then you'd have to also listen to touch events which work in a slightly different way because Apple couldn't be asked to just do something that's a bit standardised and had to invent their own little things and then not standardise it and then threaten people with patterns about it and instead with yeah okay anybody for that for you whereas with pointer events the sanity is kind of a return it extends mouse events so any of your code that you wrote already for mouse just works in 90% of the cases and again as a developer if you don't really care how the user pressed on something activated a button whether it is with a finger or a stylus or a mouse or whatever if it's Xbox One whether they use the joy pads or the connect kind of touch thing or even voice I believe it would just fire the same event that says this button was activated so it unifies it however if you want to know explicitly this was actually caused by a finger or a stylus or a mouse there is a way of reading the attribute of that event that was passed on so that's what's been extended so it gives you the best of both worlds you couldn't write completely to a certain extent completely agnostic input agnostic code and just forget about it and hopefully if new types of devices come along that also use pointer events they will just work rather than I've only got mouse and touch now I need to have X and if you do want to do something very specific it still allows you to do that so I think it's a good compromise it's not perfect as I said keyboard for instance you still have to handle that separately or just go for high level events again focus blur and click but it's at least a step in the right direction and it's more sane than inventing something similar to touch events but now just for stylus so we have a question for the floor so Apple's Touch API has a really really nice property that the DOM element that's received touch start will always see the touch end event and pointer events seem to have regressed to the previous mouse down mouse up situation where it's very easy to miss the mouse up if the pointer just ends up being outside the element is there a plan to have an easier less complicated way for pointer events for catching the up event so first of all the property you're talking about touch events is simple as you alluded to I think the way you worded it is actually incorrect it's not true that the event that received the touch start will receive the touch end the element that you touched on received the touch start you might have an ancestor that's actually listening for the event handler and receives the event and then if the DOM tree underneath that ancestor gets moved the ancestor will never see the touch end and he'll be surprised and I've seen this bug in practice and people are really surprised by it and it's one of the disadvantages of this implicit capture model is that the programmer hasn't told the browser to capture the events to the browser just says well you started touching here so clearly that's the element that wants it even though it's really your handler further up that really cares about the event stream so pointer events has an explicit API called set pointer capture that lets you say if what you want is to track the finger no matter what's underneath it then you call set pointer capture and tell it here's the element that I want to receive all the events for as long as this fingers down and so it is one more step but it's at least explicit because it's the exact same model it builds on top of the exact same model we're used to with most you still get pointer enter pointer leave events that you can still say well if the user draped their finger off you just have to remember to watch for leaving as well as ending I've got one question and there's texture Rooney so you guys have mentioned a couple of these things so it's a sort of overall question about what might become mainstream so D-pads, TV screens and multiple users interacting on a single screen are all scenarios we currently consider non-traditional which of these will become mainstream most quickly and why do you think that? This is all of them we always talk about these new devices and these modern monority report devices but they haven't got great market share which ones do you think could go become mainstream? If I knew I think our standard idea would be so much easier isn't it? Awkward I'm allowed now I don't work for a browser anymore But it's because we don't know that we have to rely on the community to innovate and experiment and try new things and see what becomes popular I really want massive touch screens to be popular I think there's a real possibility when you have 80 into touch screens that's really cool the collaborative web experience I mean, leap motion What do you think about leap motion? Do you think it's ever become mainstream? The problem I have with leap motion is it doesn't really solve an issue If I'm like 10 centimetres away from the screen I can touch it but with a connector you're 10 metres away you're solving a problem We've got some waves I'm going to debate the validity of the leap motion I think it's got a lot of great things that you can't do with touch like depth for example and being able to kind of zoom in and out of things which I think are unique to that particular type of input What else would you like me to say on that matter? What other devices do you think can come out? Is there anything which you think is actually genuine has become mainstream? Maybe voice, that's not really it's kind of mainstream on native devices Voice is getting there but the way of voice is actually getting somewhere is the Google Now stuff so if you're seeing the Google Wear stuff this week and if you've ever played with Google Glass the voice commands are by far the best part of it and now that's starting to make its way on to native Android hardware through the whole OK Google thing that's starting to work really well and with the Xbox One as well it's getting there, it's not quite right yet but it will get there so actually the voice input is going to start being important on the web you're just trying to use the same kind of APIs underneath but you don't really want to talk to your computer it's more of a user experience thing So I've got a question just from the So with regards to actually I was going to try and interject with what Remi brought up earlier with regards to where these kind of devices with regards to the required custom gestures and things like that I'm actually quite in a unique position because I actually know exactly where they're on and for those of you who aren't aware it's basically any digital display you see there's an advert out on the street and the underground things like that that's an industry digital at home I'm experienced with it and there's a huge amount of money going into that and that's actually a symptom of how capable Chrome is Chrome is actually and to a lesser extent the other browsers but for some reason Chrome's managed to capture is the actual platform that's driving a lot of that basically a lot of those displays if they're ever interactive and they're all with new gestures so that's things like where the hammer dress comes in because as a tangible example when you have a device that you're swiping left to right with if it's a display on your phone or actually a small device you swipe left to right with one finger if you actually have a large touch screen and you walk up to it you swipe with two fingers we've actually noticed that people do it they don't realise they're doing it but they're actually right there because it's out in the world and if you're ever doing things with our installations and things like kiosks and you have them a lot in trade shows where you have digital displays they have games, they're capturing weird interaction events they're all powered by the browser and the advertising industry that's today, that's not a weird thing that's coming I'm not kidding, every single month we have things like that going on and then we're kind of building these things We're talking like kiosks though and kind of installations that's not mainstream, that's not consumer technology in the web sense really I mean we're using it definitely as consumers That's the thing that technology's moved into this non-web scenario but if you take for example the crosswell TV ad network which is like the visual displays you get on the underground some of those now are actually wifi mobile enabled with your phone you can actually get out and through the web medium interact with the display you're using web technologies but you're not on the web but actually you're now passively interacting with it using gestures on your phone to control a big display with that, I think there's an awful lot to learn in terms of we should perhaps talk a lot more We should talk a lot more Let's take over the next session I think it's really good actually in this point there's now a lot of collaboration which is happening in such events groups and so forth and there's a lot to learn, I don't think anyone really knows there's the answer to a lot of the questions that we're asking some of these we're going to figure out as new devices which will become mainstream one day Ram Never So I want to just thank my panel and thank all you for listening Thank you


Live from Boston, Massachusetts, it's The Cube at the HP Vertica Big Data Conference 2014. Brought to you by HP with your hosts, John Furrier and Dave Vellante. Okay, welcome back everyone. We are here live in Boston, Massachusetts. This is The Cube, our flagship program. We go out to the events and extract the seeds from the noise. I'm John Furrier, my co-host Dave Vellante. Our next guest is Cube alumni, Brian Weiss, who's the VP and Global Head, subject matter experts at HP Autonomy. Welcome back to The Cube. Thanks a lot to be here. Love to read that title, subject matter experts. Yes, a lot of syllables in there. This is great conversation to have. Just kick it off with subject matter experts. I wish we had a crowd chat up on the main stage during the keynote, which is kind of a flash mob group discussion, kind of thought leadership going on. But subject matter experts in this new era of social is interesting. So being it's a big data conversation, is that a big part of how things get done now? What does the subject matter experts mean to you? Obviously there's two things you look at. One is people love to talk to experts about stuff. Two, with metadata and big data, curating data modeling requires experts, ontologies, whatnot, right? So what does it mean? So what does that mean subject matter experts to you? Yeah, that's a good question, right? So when you think about subject matter experts and the value they can bring to solving a business problem, right? And if the specific one is either it's in big data, it might be health care, it might be electronic discovery and compliance. What my group does really is I go look for those people who have deep depth and experience in what they do to help then solve those problems for our customers, right? So you're talking multiple 10 plus years in their domain. To your specific question around big data, I think there's something happening right now in this industry, and maybe five years from now, we'll be able to come back and check if we're right about this. But right now everybody's talking about making, getting insight out of information, getting value out of data. And frankly there's still a lot of people work involved in that. And I think where everybody's hyped up about data scientists and the era of the data scientists and everybody's trying to tune their resume, so it sounds like a joke about what's the difference between a data scientist and a statistician? Salary, right? But so we've got this era where there are tools. Got that John, right? Yeah, it's going in the crowd chat right now. Compostal service this morning, that is a very good point. But this is the age of the data scientist, right? It's in the same way that, so people who can understand data, make sense out of it, help their line of business get value out of or becoming valuable right now in the marketplace. And for me, what that means is there's, I think a short window of that where you're gonna use all kinds of different tools, whether it's structured or unstructured or Hadoop, et cetera, there's this huge learning curve to get up there. And so there's gonna be a lot of people in there and then over the next five years, someone will say, why am I spending $500,000 on people to do this work when I can really do it with software? So in the same way you see software start to do what people are doing that you increasingly, companies like HP and otherwise, are figuring out how to automate that, how to automate that process. But we are still at the very early stage of it. There's so much exciting work to do. So in answer to your question, I'll look for people who are very deep in their domain and as experts to bring them to bear on customer problems. I mean, I just think it's a really key thing and I bring it up only because it was in your title, but Dave and I have been talking about this data first, born and born on the big data market and subject matter experts is where the sales leads are converting. Literature is gone with social. You have also automation happening and orchestration with artificial intelligence and some of the things that autonomy in Vertica does is high performance based stuff. So I see this metadata thing happening and certainly the Snowden thing highlights it at a global level. What do your customers think? I mean, where are they in that spectrum? Honestly, we're just more advanced conversation, but like, at the end of the day, I just want to get close to my business and I want to orchestrate things, make money, solve my customer problems. Yeah, I think, you know, as we get into these conversations with companies and who have big data projects, right, and who know they can get value out of their information and can get competitive advantage, et cetera, et cetera, a lot of times they simply don't know where to start and how to get there. And so the idea that I want to get closer to my business, for us, it's how do I take the infrastructure? How do I take the stuff between the data I've got and the insight I need to get, right? And that was our conversation. Jeff was here talking about idle on demand, right? The fastest way to get to that is not have to set up idle in just a bunch of data, figure out what I want questions. How about we host that for you and you ask the questions? All right, so how do I take out the stuff in between the insight and the data itself and there's a lot of technology in there and go and speed that up? And so what we ask people is, what are you trying to do and why? What's the point of this big data? Actually, are you playing with Hadoop because you think it's a good thing to do and because you've got some budget and everybody says it's cool? Or do you have a business reason for executing on this? And at that point, what are the right tools? Let's bring that up, because this is something Dave pointed out earlier with Jeff Kelly, his study is that there's a big schism between IT and the business users. And that is that, hey, I want to see the meat on the bone in terms of the project. IT guys think they're successful, but yet 18% in his survey said it's not. So there's a mismatch. IT's like just, they're geeking out on big data, but they're checking the boxes. They go, great, we're done. But wait a minute, the business guys are saying, wait, where's the beef? So what's the point? Yeah, so this is where, this is an interesting conversation. You know, we talk about retargeting, short-term, long-term gain. These are the challenges. What looked good in the short-term might not be the outcome. Well that, you know, the interesting, what I like about this conference, and so we're here, it was a year ago, right? When we sat down and we were talking about, you know, the explosive interest in big data. And at that time, I would say, I'm hearing more conversations that are being driven by what the business outcome needs to be as opposed to, let me go play with the tools. I would say a year ago, and I don't know if you're having the same experience talking to people here at this conference, but the customers who are coming here are not, they're beyond the, I want to mess around with technologies that can host more, that can manage more information and distribute compute across it. They're coming with a business problem. Yeah. They're saying I need to look at, you know, my customers and understand what they're going to send and be able to drive really more sales out of it. Not just say, okay, they're interested in X, Y, and Z. Yeah, two years ago there was a lot of kick in the tires last year, a lot of proof of concepts and now it's like real business value. I wanted to ask you, so we think about autonomy's ascendancy. Things like the Federal Rules of Civil Procedure certainly helped. Yeah. And at the time, as you guys were, you know, exploding, the General Counsel was really driving the bus. It was information was a liability. Yep. Now we're talking about as information as an asset, as value, so who's in charge these days? Is it still the GC? Is it the CMO? Yeah. Where are we? That's a great question. So you throw the CMO in there, which is a bit of a wild card in it, but you got to break it down into two different polls. One of which is cost and risk, right? And you've got a whole group of people who look at information from a cost and risk perspective, and that's where your GC sits, right? And then there's a whole group of people saying there's value in this business value and money in that data, right? I can learn something about it. So the pendulum is either cost and they're actually fighting. Yeah, I was just saying, does value trump risk or no? No. It doesn't. No, it depends on the industry, I guess. It depends on the industry, but see, here we get back to the point. So great value, how much value, right? How much value is it to not get sued or to win that case, right? So historically, a lot of what we're seeing is you get the folks who are really, really good at mining data and understanding, funny enough is these people are doing the risk and cost management. They're the ones who are looking for fraud and communications around stock trades. They're the ones that are using advanced tagging and categorization and what we call automated coding for electronic discovery. The funny thing is those guys are really, really good at finding a needle in a haystack cheaply, okay? Which is the same business proposition as trying to figure out what's going on in information so I can capitalize on the value. So you say you stop fighting. So what we are seeing when we get together, I go to these discovery conferences and the lawyers are there talking about the value of the data. So the beautiful thing is where projects to delve deeply into, say, categorization. It's a great example. Instead of reacting to the information once I've been sued, help me go find the data and do it cheaper. Organizations are moving toward the middle and say help me categorize my data ahead of the fact. Think about that. How are you gonna categorize your data? You've got a half a petabyte of data and I want to know what each one means. I want you to automate that for me. I want you to automate that for me. And the reason I want you to automate that for me, one side of the aisle says is because there's cost and risk in there. I wanna delete it, right?  And then there's the other side, or I wanna know what's on legal hold, right? I wanna know, I want the potential of smoking. And I don't have time to read it, let the computer do the work. And then there's the other side, it says I'll fund that project because I know there's business value in there. I can see the patterns of behavior in my workforce. I can see what they're writing about. I can see what they're saying. I can mind that. And so we're seeing these beautiful conversations of people who wouldn't ordinarily get together. Yeah, but I buy the thesis there and I want to add a complication to it and this is the interesting one. So I agree with you that if I'm looking at my risk cost to store the data and take more of a compliance view versus the upside that's not coming home yet, quote, crossing the finish line in value, which is what you're saying. That's okay, I buy that. However, let's throw in another caveat. Let's just change the game in there. So if you change it to, I will pay HP or IBM or other vendor for their pre-package approved global platform versus open source, which is more of a lagging time-to-value equation which you're pointing to. Now that complicates it. So now I can buy Vertica or Autonomy or other solutions, get the time-to-value faster. So do you optimize for time-to-value or do you optimize for compliance risk management? That is going to be ultimately, open source, I can see that lagging, but when you say, okay, got Vertica, Autonomy. Yeah, I mean, is your question whether or not there's, whether people will do it themselves with open source faster than they will to take a productized approach to it because folks like HP are the ones that we're going to productize and bulletproof this stuff and we're going to build solutions around it that actually a CIO will buy. Well, I'm bullish on open source. My point is that open source by itself is longer time-to-value. That's kind of what's being, people agreeing on that. And it's free, you buy, you get what you pay for and the security is filling the holes and global issues. But if I buy Vertica, I don't have to save time with more of the standardization, the Ferrari and that yet still got open source support. So I have that luxury. But now I'm buying a time-to-value equation. So I have a time-to-value equation that's faster with buying versus free open source. So that's kind of what we're hearing from customers. And that's a separate dimension that you're saying, which is risk versus the upside. So if I can buy a faster time-to-value with HP, what do I optimize for? So now it's all about the focus. So what am I optimizing for? I'm program management policy. Am I optimizing for risk management compliance? Or am I optimizing time-to-value? It's just, I don't know the answer. It's interesting to hear. I think you end up, we're in this kind of lovely place. We're almost doing both. So we see people wanting to do legacy data cleanup, whether it's structured data or unstructured data, they want to just get rid of systems and all that. But you can't because you don't know what the information means. This is the unstructured stuff. I can always do a structured analysis and say, it hasn't been touched for three years, it's three years old and it's a word file. Metadata. I can do metadata management. But what's it about? What's the actual document about? Is it a lunch menu or is it a critical document for my business? So being able to categorize and tag things is the same technology that I would need to do a big data conversation with my CMO than that I would to be able to, with my compliance folks. So basically you get two pots of money to dig into. The compliance guy comes to the table and says, I'm gonna buy this technology to be able to figure out what I shouldn't, shouldn't keep and what might be on legal hold. And by the way, you guys on the big data side, look what you're gonna get for my dollar. So but there are differences, because the risks of getting it wrong are higher. Yeah, absolutely. On the legal side than they are on the big data side. I mean, you can iterate in the big data side and say, oops. But I suppose you can iterate on the legal side as well. Yep. So where are we in terms of being able to auto classify, auto categorize data at scale?  Yeah. You know, the heart and soul. Yeah. But in some case, I mean, you're kind of using search as a blunt instrument, right? And so I still need to pull a lot of data in as the data grows. If I'm paying lawyers to do searches, it's expensive. So if you could auto categorize, might save me some dough and then search on a small- Well, that's really the secret sauce of what we're doing with unstructured information. And it's having the computer automated insight, right? So what you might ordinarily require a person to do, the machine can do a lot of that work. It can tell you what the document's about. It can, here's a phrase, find similar. So if I hand you a document and say, go get me things that are like this. One, you have to make a decision about what it's about. And maybe it's not a two sentence tweet, it's a 40-page document. So you have to read it, find out, figure out what matters and what doesn't matter and figure out what's similar. And you don't just go look for things that have the same word in it. It's conceptual. And so that underlying technology ability to do almost to get insight out of that in human ways is what really what's autonomy is about. And we're using it broadly powered, you know, the big data proposition. How do you address the fact that that data is distributed in nature now with mobile more so than ever? I don't want to put it all until a single repository, right, that doesn't work. That's kind of been the, you know, I think the 1.0 version of that has tripped over that. Really, and the way we think about it and talk about it is you need connected intelligence. You need intelligence, you need connectivity. You got to solve the connectivity problem. Nobody's going to move it into a big bucket in order to make sense out of it. It just doesn't, there's nobody's going to do that in their business process. You need to be able to connect to data source, you need to be able to understand the variety of information, the video, the audio, right? They're both, that's both unstructured as well as free text. And how do I look at all that in the same context and ask or interrogate that same set of data with one intelligent view? And by the way, I got to do it securely. So I have to still honor the underlying security mode or the underlying security framework or the documents and the repository. So I got to be able to do all of that in the enterprise at scale and give you human insight across it. And this is, there's, I think I mentioned we're moving from tire kicking in the, in the mode where I think the business problems are becoming front and center in these big data conversations. We've got some really exciting ones in there. This afternoon, we're going to be talking about our healthcare initiative. And this is a great use case of where you get, you take an example of where there's an awful lot of very rich information. And the rich information is structured, semi-structured and frankly, really unstructured. And that's patient information, all right? So when a doctor makes a diagnosis and they make notes about you and coming in, they take those notes and sometimes those notes have structured information and they don't and they're dictations and there's all kinds of, it's very difficult to make sense out of that information. And historically, what has happened in the medical industry is that all gets put down to one code, which is the billing code. So all the things that happen in this information, it's like the, it's the travesty of, you know, ETL in the medical world is that at the end of everything I've learned about you and what might be, and I have to say, and I'm going to bill it to the following code. So read a 500 page book and give me one code to describe it. Now, what if I could take all that other information, what you said, how you said it, and I can take those descriptions and correlate it with other types of information, whether it's structured or unstructured, right? There's all these data systems that are out there. If I can connect to those and I can also analyze the notes itself and understand we can look at patterns of not only how people are talking about what's happening, but what's actually happening. We've got a project with Stanford Children's Hospital we've been running for about two and a half years now and they're doing exactly this and they're able to look at, you know, increasing their standard of care, increasing their efficiencies. It's really a tremendous project and, you know, it's not on TV, but we'll be talking about it later this afternoon. Well, you're going to go on, you're going to speak to this audience with Dr. John Palma of Stanford, right? So, can you take us through more of that case study? I mean, what are they actually doing? Well, first of all, it's a backdrop because, you know, I'm in Palo Alto, Stanford. I know they're doing a ton of big data across all their disciplines. It's pretty much been infiltrated across, you know, from cancer research to all kinds of stuff. So they are really doing some great work. So go ahead. They're going through all of the records they have that are associated with patient care and they're looking at the way in which the system is describing, right? The tag that's been associated with that. But they're digging deeper and looking at the way in which any of the free text fields which are associated with that, so then the way the customer described, the customer, see here, I'm talking about customer, the way the client might have described what they were, what's happening to them, the way the doctor may have described what's happening to them and they're able to draw correlations out of that information, which are very different, perhaps, from the way that the information was coded, right? Because they don't talk about things in a similar way, right? So difficulty of breathing might be abbreviated DOB, which is date of birth, right? All of the problems around understanding context and semantics around, you know, he denies having any chest pain. Well, if I'm just looking for chest pain and I haven't understood that it's in the context of denying something, right? So the way people talk about things is not always coded into rows and columns and so they're pulling this data out and able to run really lovely analytics on what's more likely happening than the way things have been coded. So their standard of care and what's beautiful about it is that they're getting to a point where if they can really do this across, not only their data, but other healthcare data, right? So it's not just Stanford's hospital's data, it's if you could do this writ large, you start to do really proactive medicine in a different way. All right, so I gotta ask the question. I mean, it sounds eerily similar to IBM Watson in a way. What's similar, what's different? Well, there's significant technology differences, right? You know, what Watson does and what it can do at scale, you know, Idle has a whole different set of capabilities. It's more based on a probability model than a natural language processing. Although when you look at healthcare, you kind of have to do all of it, right? So you have to do not only what Idle will do with probability modeling, but natural language processing as well as any kind of ontology and lexicons from snowmen and the dictionaries, et cetera. So it's a combination of everything that, you know, we have, including some of the coming from HP Labs, using Idle, using autonomy, and then as this progresses out, we'll get into, you know, much deeper, larger data lakes of information. So it's really apples and oranges, but it's, you know, to me it's a very exciting place where we really could potentially change the way we do medicine. And in a place, we had a question from the audience today, when are we going to hear from, you know, big data not getting people to click on ads and drive revenue, but to change the world and improve the human condition? And how would you answer that question? Are we seeing it already? Is it, are we in the cusp of that? Well, I think we're at the beginning of it. To be honest, I think we're at the very beginning of that process. And innovators like Stanford Children's, who clearly see that, I mean, we have, you know, if I ask you to describe one piece with one field, it doesn't, it's not right. There's so much rich information out there. And if we could tap it in a way that we could make use of it for every clinician that's out there, we could really change things. And so there are innovators out there, Stanford, and other ones who see this very clearly. Yeah. It's human information. It's human information. It's human information. It's real value ads. It changes your life. So big data changing the world, helping people live longer. Trends like Fitbit and these things are here. Well, you add that to Internet of Things and you add all the data we're pulling off of people. And whether they checked into Stanford Children's or not, but that all becomes a data source and a data set that's worldwide and you start to, it's very tantalizing. If you free the data, good things can happen. That's what we've got to do, unleash the data. You know, medicine's really interesting right now because it's moved from, the pressure on it is not about fee for service, it's about fee for performance. And also the personalization of medicine means that we really need to be proactive and manage somebody before they get to the emergency room. And both of those things drive analytics. The only way you're going to achieve those is by getting better information out of the data that you have and adjusting your systems accordingly. Brian, thanks for coming on the key. We really appreciate it. Always my pleasure to talk. You guys doing some great work. Obviously, human information to business, commercial and to prize all the above Internet of Things, all happening. Great to be here, great to hear all those stories and loves the epic tweet, thrusting the statistician and the data scientist. Salary, great job. Thanks for that quote. We'll be right back. It wasn't mine, it came up this morning. We'll be right back with our next guest after this short break. Take care.


Link in the description box. All right guys, welcome back to the channel. If you're new Manus Bobby guys, today we're going to react again to my favorite and your most hated video topic. We're going to react to Sufism today with a video on the supreme identity by Emir Abdul Qadr on the absolutely mind blowing YouTube channel Let's Talk Religion. But to the Sufi haters here, guys, don't tune out just yet, look into the biography of Emir Abdul Qadr. Emir Abdul Qadr was an Algerian religious and military leader who led a struggle against the French colonial invasion of Algeria in the early 19th century. As an Islamic scholar and Sufi who unexpectedly found himself leading a military campaign, he built up a collection of Algerian tribesmen that for many years successfully held out against one of the most advanced armies in Europe. His consistent regard for what would now be called human rights, especially as regards his Christian opponents, drew widespread admiration and crucial intervention to save the Christian community of Damascus from a massacre in 1860 brought honors and awards from around the world. Within Algeria, his efforts to unite the country against French invaders saw him hailed as the modern yugurta and his ability to combine religious and political authority has led to him being acclaimed as the saint among the princes, the prince among the saints. So as you can see, this man was not just a Sufi philosopher meditating all day, but he found himself fighting the French army. Guys, I'm super excited for today's video. With no further ado, let's have a look. Emir Abdul Qadr Al-Jazari was an Algerian Muslim military commander, writer and spiritual teacher who is mostly famous for resisting the French colonial invasion of Algeria in the 19th century. His chivalric behavior as well as an incident where he shielded Christians in his home in Damascus during a riot. He was also a Sufi master who followed the philosophical mystical school of the Andalusian master Ibn al-Arabi and was buried next to him in Damascus. I absolutely love Ibn al-Arabi. In his book, Kitab al-Muafik Abdul Qadr explains the metaphysical doctrine of the unity of being, Wohdad al-Wujud, strongly associated with Ibn al-Arabi and his school. The following is an excerpt from that book, tackling the God-slash-creation relationship from this perspective and taking the form of a poetic conversation between God and the author on the supreme identity. God, may he be exalted, said to me, do you know who you are? I replied, yes, I am the non-being which is manifested by your manifestation. I am the darkness which your light illuminates. Then he said to me, since you know this, persevere firmly in this knowledge and be careful not to claim that which does not belong to you, for the deposit must be returned to its owner and the loan must be paid back. The name contingent being has always belonged to you and always will belong to you. He said to me again, do you know who you are? I replied, yes, I am in reality God, but metaphorically and in relation to the way I am creature. I am a contingent being with respect to my form, but I cannot not be the necessary being. It is the divine name on Haqq which belongs to me by original right. The name of creature is only a borrowed name and a formula of differentiation. He said to me, veil this symbol and let the wall fall down and bury this treasure in order that only he will be able to extract it who has put his soul through severe trials and looked his death in the face. All right, there's of course a lot to unpack here. Guys, disclaimer first, I'm not a religious scholar, let alone a Christian or Muslim scholar. I'm not talking as an authority on Islam here. I want to make that very, very clear. Please understand and this is why this channel is called Bobby's Perspective. All I am contributing here is my perspective. I'm talking about my experience with spirituality, with certain medicinal plans, with shamanism and other mystical states of altered consciousness. Moreover, I'm not making truth claims either. Please keep that in mind whilst listening to me. I'm simply sharing what has happened and I'm not indicating that therefore it must be true. All of that being said, I can absolutely relate to those statements made here. However, when you read them in your everyday consciousness, they seem almost blasphemous of course, because here you can see the creature, the human, claiming to be God. So how can that even be? Before that, they talked about the illumination of light and that that creature, the darkness, cannot exist without the light from the necessary being. So to put this truly into simple words and explain it is actually impossible. This can only be experienced. However, nevertheless, I'm still going to try to explain this phenomena. What he is describing here is that God is al-Hak, that God is the ultimate truth and that he is the only truth, that he is the necessary being. What does that mean? It means that God is the original existence. So God is existence in himself. That means if we experience existence, we only experience it based upon him. If he wouldn't be there, if he wouldn't exist, existence wouldn't exist at all. And therefore keeping this in mind, it means that he, God, is the substrate, the life giver to everything you see around here, almost, and this, of course, a downgraded example, as air. If there was no air, we would not function. Therefore, in a way, we are oxygen as well. Of course, we are not oxygen at the same time. However, oxygen is found within our bodies and without oxygen we would not live. Therefore, we are contingent upon oxygen. Of course, this is an extremely downgraded, simplified example. However, I'm using this example to paint the picture of what it means when the creature says it is God. It is not blasphemous if you understand the underlying meaning here, even though it appears as such at first. All the creature is realizing is that the creature itself is non-existent, that the creature itself is the ego, that the creature itself is the darkness, that only God exists. He is the ultimate reality and through him we experience existence. We are ultimately like a vessel being filled by God's existence. Without him, we wouldn't even exist. I am the necessary being who epiphanizes himself. My necessity proceeds from the necessity of your essence and my eternity from the eternity of your knowledge and your attributes. With respect to me, I am pure non-being who has never breathed the perfume of existence, the adventitious being who remains non-existence. It is adventitiousness. I only possess being so long as I am present with you and for you. Left to myself and absent from you, I am one who is not, even while he is. And yet again I'm gonna try to put this into simple words, I'm gonna fail anyways. But the point of the story is that he is distinguishing here between the true self, the higher self if you will, and his ego. He is describing that the ego itself dies with the body. When we are dying, all of this dies as well. I'm not taking my beard into the afterlife nor am I taking Bobby into the afterlife either. This is a name that has been given to me. Look at it, I can change my name here if I wanted to. Those are temporal things. They are bound to this time and space and we're not taking anything with us. If I lose my hand right now, I'm still Bobby, am I not? But I'm certainly not taking my hand into the afterlife nor am I taking my body, nor am I taking this mind, nor am I taking this ego. And therefore he understands that the ego, this self, is absolute non-existence, non-being. And this is him in regards to his own ego. However, if he goes within and finds his true self, he understands that he and God is absolutely inseparable because this is the true reality. And in the true reality we have a union, we have only one and if we have only one there is no separation and therefore there is no other. Then he said to me, and who am I? I replied, you are the being necessary through yourself, alone perfect in its essence and its attributes. Better yet you transcend by the perfection of your essence, the perfection of your attributes. You are the perfect in every state, the transcendent with respect to everything which can come to the mind. Exactly right. And now he is describing God in his essence, the absolute perfection that even transcends perfection itself. There are absolutely no words that you can attest to God. He is the absolute transcendent. So this is a dualistic understanding. In this life here, there is of course a distinction between creator and creation and of course creator and creature. He understands that he is the opposite of that perfection. He is flawed. He is in a separated state which can come to the mind. He replied to me, you do not know me. I said to him without fear of being disrespectful, you are he in whose likeness all contingent creatures are. You are Lord and the servant, the near and the far. You are the one and the multiple, the sublime and the lowly, the rich and the poor, the worshipper and the worshipped, the contemplator and the contemplated. In you are joined the contrary and the opposite. For you are the apparent and the hidden, the voyager and the sedentary. He who sows and he who cultivates. You are he who plays, who tricks and who deceives.  You are creature and I am creature. You are neither this nor that and I am neither this nor that. He said to me, that is enough. You know me. Hide me from those who know me not. For lordship has a secret and if it were revealed, lordship would be annihilated. And servitude also has a secret and if it were revealed, servitude would be annihilated. Praise us for what we have taught you about ourselves. For you cannot know us by any other than us. Nothing leads to us except ourselves. Alright, this is it. The video finishes here. That being said, the last passage is of course mind-blowingly deep. In this dialogue slash monologue with God, he comes to the conclusion that this is a secret that has to be kept and therefore ultimately, even though I'm sitting here trying to frame this somehow, put it into simplistic words, I really believe that those mystical experiences are not for everyone, obviously. And there is a beautiful passage here in the end. Praise us for what we have taught you about ourselves. For you cannot know us by any other than us. And this of course could lead us back to the Quran to simply worship God the way that it is displayed within Islam here, for example. Because as it is so beautifully put here, for Lordship has a secret and if it were revealed, Lordship would be annihilated. And servitude also has a secret and if it were revealed, servitude would be annihilated. So this is really something that I experienced within the New Age movement. People go out into the world and they're having all kinds of spiritual psychedelic experiences and ultimately they start worshiping themselves or they start worshiping the universe, life and whatnot. They don't understand the value of submitting your will to God and they don't understand the value of the dichotomy between a servant and a Lord. They don't understand why they would worship something greater than them and like that they suffer. This reminds me of the Surah in the Quran which states, and whoever turns away from my remembrance, indeed he will have a depressed life. And this truly can happen when you start inhaling all kinds of different concepts out of Hinduism, Buddhism, drug culture and whatnot.  This is of course lost as well when people simply read those texts with no understanding. Those texts are metaphorical descriptions of the supreme reality or as he likes to call it the supreme identity and cannot be understood in a literal sense. And therefore I'm gonna cut off this video here because ultimately it makes no sense talking about it. Anyways guys this is it. If you liked the video leave it a thumbs up. If you haven't subscribed already guys please do so and if you want to support this channel via Patreon for example all the links are in the description box below. Thank you so much for your ongoing support guys and as always may God bless you all. Much love and peace.