device play it back and this is something that's interesting and unique to the web not all devices perform the same way ultimately you want to give the video the user the highest possible video but the 1 gigahertz system on the left probably doesn't have the same kind of hardware decode support as the device on the right this is important when av1 comes out it's also important right now quite frankly for different video playback capabilities so let's look at vp9 if i want to detect whether a device can play vp9 today i'll use the can play type function i'll pass in web mvp9 it'll say can you do that and it will say probably and you'll say okay here's some vp9 please go play it now what that doesn't really tell you is are you going to drain the battery are you going to play this back at high quality or are you going to be dropping frames so there's a new api coming out to address this called the media capabilities api and this allows you to fine tune the video playback experience for the user on the top again a progressive feature you say if we have this media capabilities api i'm going to pass in the resolution and bit rate and the codec of the video i want to play ask for the decoding information and then it will tell you number one is it supported number two will the playback be smooth and fluid and number three will it be power efficient now you can use this information to make decisions about not only what type of video you want to send but also what resolution of video you want to send to optimize the viewer experience based upon your specific use case but we haven't talked about what i think personally is the area of greatest innovation and growth in video and audio that we are seeing right now particularly over the last few years my daughter's ability to make mobile music videos on her phone and her ability to put things on her face and tell and communicate with other people is all social it's all about sharing it's all about communication here this it's hard to remember this was not mainstream as recently as five or six years ago but it has quickly become very mainstream the whole premise here with social is you want people to get in get out and be able to do things quickly which is great for the web so let's talk about the most personal social media video communications web real-time communications web rtc is not new but what is kind of interesting over the last year or two has been the rapid adoption both by the browser manufacturers as well as the app platform there are sites now that are using it on the scale of tens and approaching a hundred million users on the site so what is new is that progressive web apps make web rtc really interesting on the mobile web you have the ability to provide people with a website that uses web rtc on the phone and let them jump in communicate jump out add to the home screen they can come back so there is a next step here as far as peer-to-peer personal communication and that's fine for one-to-one let's talk about another type of social event which is live streaming how is this social well if you're sending video to 10 people they probably want to communicate with you if you're sending video to a million people they probably want to communicate with each other it really depends on the event and what's really neat about this is that where you look at all of the platforms to support live streaming today the web has a pretty unique advantage in terms of the feature set as well as the way people use it if you want to share something you simply send somebody a URL they don't have to download a link an app in order to watch it because this event is happening now don't make users wait this is where the web can really come into play now the truth with live streaming is that there's a whole infrastructure challenge underneath the hood and what i mean by that is if you're deploying to a million people and you want latency a low latency rather you might take a different approach using files that are getting placed out onto the cdn or even files that you're reading while they're growing as they're placed out on the cdn or you might use data channel web rtc and do peer to peer cds to deliver it i am not going to go into all the details on that because that would use all our time but what i will let you know is that the web does support live streaming both in terms of web rtc as well as the ability to put out the video segments in the form of files and play them back and in fact shaka has the ability to do this by reading dynamic presentation descriptions and then as the segments become available in the network knowing the appropriate seek range as well as the end so that it knows what to play back and all of that's great but there's the last piece of social media that we want to talk about and that's creation if you look at the way that something where you are creating video and sharing it with your friends works it's really a beautiful cycle it starts with somebody finding something and watching it and getting inspired and then they say now i want to make something and then they capture and they create and they encode and they upload and they share it with their friends and then their friends say ooh that's very cool and they get inspired and they want to capture and so this really only works if it's easy to discover it's low friction to watch and then also low friction to get into capture scenario that this cycle can go very very fast as long as the whole path is very easy the web has a lot to bring here and to show you an example of what's coming in this space i'm going to take you to french war thank you john i'd like to share with you a simple web app i've been working on and use it to showcase some awesome media capabilities coming to the web may we switch to the android device please so this web app is called moustache and you will understand why soon i have previously added moustache to my home screen and specified in the web manifest that i wanted to run it in standalone mode so that the brother ui does not show up it is quite hard almost impossible to tell that this is a web app so now you can see me with moustaches and wearing a funny hat not that there is no lag here it is perfectly smooth may i say creamy okay i said it let's press the record button and that's all at the bottom right of my screen i have a preview video of myself that i can share now click the chevron and let's do it for instance boop or not please and that's all let me describe what you've just seen and may not have realized by walking you through all the api i have been using for that let's start with the basic this is how you get access to a camera video stream on the web today set the video attribute a chassis object to the asynchronous result of navigator dot media devices dot get user media and you're good to go as soon as the video plays not that the only media constraint i pass there is video true but my moustache app actually asks for more the ideal video width height frame rate and facing mode the browser will do its best to accommodate your request so it doesn't hurt to ask my custom draw function is going to be called is going to be called sorry every time the browser asks for a frame to be animated thanks to the request animation from function here since i'm looking for the smoothest experience there that means the draw function is going to be called approximately 60 times per second and each time it's called i'm going to draw a live video frame on my canvas at that point we have our face on the screen and that's not bad but let's go further now using the experimental shape detection web api i was able to draw some moustaches and a hat on my moving face while keeping the app running smoothly at 60 frames per second as a matter of fact that wasn't the case last week so thank you miguel for working hard to get this demo ready for today as you can see this is a pretty simple and easy to use api as hardware accelerated detectors may potentially allocate in whole significant resources i recommend you reuse the same face detector object when doing several detection the fast mode option here is a hint for the browser to prioritize speed over accuracy which is exactly what i'm looking for in that case let's look at what is happening now in the draw function calling face detector the data canvas return asynchronously an array of detected faces in the canvas note that the processing is all done on device so there is no internet connection required there when the face is dated i use its position x y width and height to draw some element on top of a video frame one best practice is to use a simple Boolean like is detecting faces to avoid overrunning the api as the draw function is called all the time pretty cool right this api along with the barcode qr code and text detection api will be available to everyone later this summer for testing purposes in chrome's table note that you can already play with it today by enabling the experimental web platform feature flag in chrome by well we love feedback so if you want more features such as eye or mass detection that's a bad example we have it but if you want something else something more or if you simply stop a number of bugs please let us know now how about media recording is that how it is not like really this is the full code i've used in my moustache app grab the stream from the canvas by calling canvas dot capture stream and use it to instantiate a new media recorder the mime tab option tells the browser which connect to use for the recording it can be h264 or vp9 for instance if you leave it and specify though the browser will choose the best one which usually boils down to the hardware accelerated one if any you can also pass the bits per second option to customize the video quality of the recording recorder start actually start recording the media and each time the recorder delivers some media data we store them in an array object i call chunks when the user stops recording we call recorded stop which fires a stop event allowing us to create a new blob object containing all chunks we recorded so far and upload that dvd develop to our backend i could also have chosen to directly stream these chunks to an RTC per connection by the way the media recording api has been in chrome for a year now so it is pretty stable has proved some popular chrome extension use it intensively to record up to one hour of content such as video and tutorials when i click the share button did you notice that the native android share ui showed up and included all my native app that supports sharing text on my device i was able to do that with the experimental web share api which enables sharing data such as text and soon images from the web to an installed app of user choosing as you can see this is as simple as calling navigator dot share and pass it a title some text and the video ur and that is pretty much it these four lines of code enable seamless sharing and i personally can't wait for this api to be available to everyone later this year thank you thank you fred swath that's pretty awesome right there's a couple of things in there one is a media recorder api which i want to highlight because what is great about that is the web creating media for the web that is something that we see over the next few years is becoming more and more common but what's really great about this is i love the way that with in a plug-in free world you can connect these items together like nodes meet and pipe them together using media streams so camera to canvas to media recorder to upload microphone to web audio to rtc peer connection there are a lot of different things you can do with these tools if you want to access the mustache demo here again to remind everybody this is a relatively new api it's still in development so you do need to turn on the web experimental features flag if you want this to work and we would ask that you would use chrome canary for that so that was a lot i told you at the beginning we were going to cover a lot so what did we see today we showed you a couple of demos and we talked through a lot of apis many of which are available today so in the biograph demo which was really about playback service workers were used heavily shaka player was used for media playback and then there was a long list of supporting cast of apis including the media session api the full screen api the screen orientation api and a variety of others you don't have to remember all of them fortunately because friends who has written a wonderful article on best practices for mobile web video playback which is available at the link at the bottom and if you're looking to do your own progressive web app doing media playback please do take a look at the sample code provided by biograph on the mustache side media stream recording api as well as media capture and streams these are both available today and at the bottom is the link so that you can access a mustache demo again chrome canary on your mobile device and make sure to enable the experimental web features which brings me to arguably the most important point we want feedback some of what we showed you today the title of the talk was the future three of the apis in particular background fetch to allow you to do downloading of media even while the user navigates away from the page or closes the browser it'll resume when they come back the shape detection api the ability to look for qr codes texts faces and other objects and the web share api the ability to share items socially all of these are being developed this is your opportunity this is a great thing that's coming in the future but frankly it'll be better if somebody in this audience looks at and says i would like this and our developer team and the other developers on the web working on the apis say that's a pretty good idea please do help us make this a reality by going to these sites and providing feedback look at the apis and tell us what you'd like to see or you can see us in person we will be in the sandbox behind the stage and it's a lot of great apis we're happy to talk to you about them and frankly we are just really excited if you look at the pace of innovation over the last few years the web is coming to play the mobile web is coming to the media game and we just cannot wait to see what happens next thank you everybody so you've built an amazing mobile app that your users are going to love but you want to get into people's hands and let them see just how awesome it is well adwords helps you do this putting ads for your app in front of billions of people that use search youtube google play and more you can quickly set up an ad campaign to reach the type of users that might be interested in your app you only pay if the user clicks on that ad and you can set the budget and acquisition costs that you're comfortable with but how do you know you're reaching the right users maybe some will install your app and forget about it while others will make it part of their daily lives firebase analytics helps you do this you can define events that happen in your app that you consider to be important such as reaching the first level of your game purchasing a fancy new pair of sunglasses or returning every morning to check out new products you can tell adwords which of these events are most important to you then adwords will display ads to people who are more likely to complete these important actions in the future you can also build audiences which are specific segments of users and have adwords display your app to them for example imagine that you have a group of users who are very active have added a product to their cart but haven't purchased yet well you can use firebase to create an audience of just these people and then use adwords to give them specific ads and encourage them to come back to your app and take action understanding your users and engaging with them at just the right time and in the right way will help you build loyal users for your app firebase and adwords working together to help you grow your user base get started today your new users are waiting android instant apps make it possible for users to access your app without having to install it first imagine users opening your app just by clicking on a link in an email or a text message we've recently made android instant apps available to all android developers to take full advantage of this we have some best practices to help you make your instant apps user experience as great as that in your installed app or maybe even better you can find all this and more at the url in description below it's important to keep in mind that by enabling your app to run instantly without installation you're not creating another additional app we're thinking of instant apps is another way to use the app your users already know and love it's the same app just without installation by adding the ability to access your app directly from a link a search result or another app it's much easier for users to engage with your app and get excited about it if they decide to keep your app on their device permanently they can then install it right from within the instant app the ability to launch an app without having to install it provides an enormous opportunity for a long time app developers have focused on the number of app installations as a proxy for the metrics their business really cares about without installation users simply weren't able to engage with the developer's offerings at all removing this barrier to entry enables you to think about the metrics your business really cares about your audience is now just one tap away from engaging with your service your instant app is just another mode your app can run in so don't branch your UI and make any unnecessary changes regarding the layout interface design or experience of your instant app the transition from instant to installed mode after installation should be as smooth and seamless as possible your users should have a rich and full app experience even if they haven't installed your app rather than thinking of instant apps as a limiting factor to what your audience can do think of it as an opportunity to get them to your functionality quicker and a way to foster your relationship with them avoid prompting your users to install the app when they're in the middle of a task there'll be much more inclined to place your app onto their device permanently after it has had the opportunity to prove its usefulness refrain from bouncing them back and forth between your instant app and your mobile web offerings you can probably tell by now instant apps are all about removing friction for your users and getting them closer to your functionality think about ways you can remove further barriers for your users for example wait until users can see the benefit of making an account and signing in until the value of doing so becomes apparent asking users to create an account after installation seems like a small additional ask when they've already gone through the app installation flow and are only just getting started however when they're coming from a link looking for specific content or functionality being asked to register can feel very disruptive additionally make sure to use available apis to make your and your users life easier using google smartlock for example make signing up and signing in a much simpler and straightforward experience in summary we really think that instant apps will unlock a lot of opportunity to engage your audience more directly users will be able to focus on what it is they want to accomplish rather than having to spend time maintaining and updating apps on their phone we're super excited to see what you come up with everything i talked about here and much more you can find on g.co slash instant apps thanks for watching firebase makes sending messages to your users easy with firebase cloud messaging it's a free service that allows you to send messages to your users apps across a variety of platforms messages can be addressed to single devices groups of devices or even topics building and notifications or messaging system with firebase cloud messaging is easy first you register your users app instance with the firebase cloud messaging servers then on your server you write code that allows you to address these devices by id group or topic and which tells the firebase cloud messaging server to send the messages for you it's powerful it's scalable delivering hundreds of billions of messages per day with 95 percent of messages being delivered within 250 milliseconds to connected devices across many different platforms it's that easy it's that powerful firebase cloud messaging people love to use different mobile devices in different ways that suit their situation and lifestyle michael uses a phone to play games on the go while tony enjoys using a large tablet as he relaxes on the sofa and jen carries a small tablet in her purse for reading on the bus but they all want to use your app on the devices they prefer so you'll want to make sure they each have a great experience regardless of screen size os version and the features of the app they use the most it can be taxing to test each one of these situations so that all your users can be happy we know you'd rather not have to buy in store stacks of devices and test your app in all these circumstances that's why we built firebase test lab for android to make it easy and affordable for you to test your app with a variety of devices and be sure it works great for all your users our device lab hosted in the cloud offers a variety of physical devices ready to test your app the selection of devices is always growing so your tests will stay current with the latest hardware and operating systems the easiest way to use firebase test lab is to run a robo test this is an intelligent automated test that crawls your app to discover and exercise its features you won't need to write any additional code to make use of a robo test for more advanced testing you can also script the interactions with your app to simulate specific use cases and verify that everything works as expected test results include a detailed report for each device used including screenshots device logs and any crashes that may have occurred during the test this allows you to verify that your app is working correctly on the variety of devices and configurations you selected it's easy to make firebase test lab a part of your daily development routine and we have multiple ways to help you test regularly and spot errors early first you can use the firebase console to upload and test your app there is also a command line interface for testing with continuous integration servers so you can automatically test every build during android development you can deploy your app directly to firebase test lab using android studio 2.0 and finally in the play store developer console there is a special automated launch test that will run for android apps published to an alpha or beta channel to get started using firebase test lab and learn how to regularly test your app on different devices and configurations you can start with the documentation available right here happy testing when i was 10 years old my dad got me a book in pasco and just a few chapters in i immediately realized i can create some amazing things all by myself my name is shruti sarath and i'm a founder and game developer at blacklight i always remember my mom playing solitaire on a computer but when moving to mobile she struggled to find a good solitaire for her phone so she insisted that being a game developer plus her daughter i should develop solitaire for her so i took the idea to sumit i'm sumit sony and i'm co-founder at blacklight i'm 32 years old and i have 25 years of game experience playing them and now designing them we played solitaire for hours we even bought a deck of cards real cards to understand the game better 60 percent of our users are mommies and crannies so we designed solitaire keeping them in mind big bowl cards soothing colors and minimal design taking a lot of cues from google material design guidelines it took us six months of research and design to make solitaire look great and be really fun to play and then we noticed a lot of visitors are visiting our page but very few of them downloaded the game so we thought maybe the logo is the problem we experimented on google play with a modern design and a very old-fashioned one and wow the old-fashioned logo had 54 percent more downloads and after we adopted it our total downloads went up by 90 percent plus we moved to android studio which allowed us to integrate google play game services and emulators helped us design user interface for different screen sizes and varied pixel density the feeling that something you have created is adding fun and value to so many people around the world most of them from places have never been true can be quite profound shruti's love for coding and her mom's love for solitaire led to this today we have five more games in the pipeline and we had five more android developers to work on that it definitely wasn't a smooth journey but through hard work and great use of feedback we are where we are today and we can't wait to see what we do next android 2.0 is the most significant update to android since its launch in 2014 i'm hoi lam and i'm here to take you through three highlights of android 2.0 first material design has arrived on android the new android interface uses a darker color palette which makes the watch plenty more in socials environments it also helps save battery by older displays we also suggest that developer adopts vertical layouts and reserve horizontal swipe for activity dismissal in our research this makes user interface easier to understand to help developer implement these new design patterns we have introduced new user interface components such as the wearable drawer layout and wearable recycler view check out the material design for android site for more details the second highlight for android 2.0 is watch face complications before android 2.0 if a watch face wanted to display data from another app they need a one-to-one agreement in both business and technical terms to alleviate this complexity with android 2.0 we have introduced the complications api complications are a traditional watch making term for areas of the watch face that displays information other than the time such as the date or phases of the moon we have extended this concept to smartwatches so any app can publish data for watch faces to consume and watch faces can display this data in a style that fits into their unique designs for app developers this helps increase user engagement and mind share for watch face makers this adds utility to the watch face users can now choose whichever watch face they want as well as getting the data that they need check out the watch face and complications sections of the developer documentation for more details last but not least standalone functionality previously android 1.0 apps require a phone app to communicate with the cloud with android 2.0 wear apps can access the internet directly without the need for a corresponding android phone app installed this means the android app can access the internet even if the user has an iphone to help with app distribution we have also put the google play store on the watch users can now download apps directly on their wrist apart from these three areas of improvement material design complications and standalone apps there are numerous other enhancements with android 2.0 check out the android developer site for more details i'm hollam happy coding have you played app ship 3000 no but i'm ready to and i'm standing here with ape who helped build this fantastic experience ape tell me all about it yes this is app ship 3000 this is a collaborative firebase trivia game where three people work together to launch their app ship and hit their moonshot it's pretty crazy played with arcade controllers out here in this very loud situation where it makes you yell to each other and get all hyped up it's a lot of fun awesome so we're gonna play it and maybe i'll give you the microphone and you can sort of narrate what's happening and maybe talk a little bit about the tech great so uh for my fellow team players we've got dug over here dug take a bow and todd over here who you may know from such things as everything todd does all right i'm gonna give you the mic yep everyone's already logged in and they're sitting with their rocket primed for ignition in order to take off everyone has a specific color they need to hit so let's give it a countdown in three two one take off there we go there we go now in the game of app ship 3000 the very first thing that happens is you will get prompted to ask have you played before everyone here has is good and we'll dive right into the game the very first question who has it there you go there you go those blinking things up at the top are asteroids which will absolutely destroy you good good good good i'm amazed you knew that good job how many google engineers does it take good call good call yeah you cannot be a true not be a true fire baser if you think uh pineapples are ugly all right while this game's running i'll talk a little bit about what's happening but it's a little hard since they're all shouting all the time somebody picked something you're running out of time all right so while they're playing this game every movement they make is being thrown off into the firebase real-time database and the real-time database is taking that doing some analysis updating high scores and things like that so absolutely every move they make is getting thrown off into google cloud and firebase for analysis oh you got to watch out for those asteroids guys no it is not it's definitely not it up or we're low on fuel low on fuel if you miss this next question you are done oh you're so close that is the end of a 754 feet that is an okay score that is supremely okay good job everyone thanks for playing all right Abe thanks so much for that absolutely anytime so we're actually going to play another round so uh bye all right good morning google i o i'm noah fidel and i'm going to speak with you today about how to go from research to production with tensorflow serving so i'm going to start by sharing a few stories uh from my past and observing industry so the first one show hands who's taken a commercial flight in the last 10 years all right everyone's awake and you've taken flights so this is good you might be shocked to know that circa 2005 i was working on the flight planning software that plans the majority of the world's flights this does things like compute how much fuel to take on the plane that you probably care about but before 2005 that system did not use source control and for those of you who know that means that's pretty scary uh it does now uh and i raise this just to show how even though we know what best practices are and we publish them and blog about them and talk about them it sometimes takes a while for them to be adopted across industry fast forward to 2010 i was at a startup and we had a mobile photo sharing app and we tried to do all the right things we had source control continuous integration using jankins uh in one day we decided to add machine learning to our app uh we trained up a model that could do face detection auto crop to faces this is really great our users loved it uh our investors really loved it in 2010 um but for machine learning what is source control and continuous integration these best practices don't really exist so i trained that model on my workstation and if i got sick or if my workstation crashed no more training for us we'd have to start over so here we are in 2017 and at talks like this and other conferences around the world we have all kinds of great uh tools and best practices for machine learning uh so we have TensorFlow and a whole ecosystem around it and many other tools but there are many areas still to be defined uh so just as one example of many uh what's a continuous integration test look like for machine learning so you're deploying models every day you might uh when you start doing this you might make a test that runs some sample data from today through your model and it does that all the time what happens when your users behavior changes what happens if they look at bigger objects or different kinds of objects or the distribution changes so there are many of these things we're figuring out and i'm going to make a bold statement that i think machine learning is 10 to 20 years behind the state of the art in software engineering and so we have a ways to go one of the ways and there are many where TensorFlow serving is helping is by making it easy to safely and robustly push out multiple versions of your models over time as well as the ability to roll back and so this seems pretty simple but we've seen you know people inside and outside google that don't have that capability there are many other things uh that are part of this as well all right so here's the agenda for today's talk for starters TensorFlow serving is a flexible high-performance serving system for machine learn models and it's designed for your production environments and before i dive into the details of TensorFlow serving i'm going to describe what is serving for those of you who might not be familiar it's pretty simple serving is how you apply a machine learning model after you've trained it many of the talks on machine learning both in academia and industry are focused on the training side but serving side is kind of left as an exercise to the reader so that's where we come in all right so on the right side of this slide you're hopefully all familiar with the orange boxes here if you had a TensorFlow talk you know you should be pretty familiar with these you have a pile of data you have somebody doing the role of a data scientist and you're training a model and you'll use something like TensorFlow to do that now you have your application on the right side and for the sake of example let's say that you are ranking videos so you have an app that users come home from work and they want to look at some fun videos and just relax so your application has a list of candidate videos and your model is sitting over there on disk how are you actually going to apply it so the really straightforward and common answer is you're going to have an rpc server so this server is going to take your model it's going to load it off of disk and it's going to make it queryable by your application so your application can give a list of videos along with their features and the server will reply back with maybe the probability of each of those videos being clicked on all right so moving on to some goals for serving and in particular for serving machine learn models so the first one and this is where serving differs quite a bit from what we're used to when talking about training of ml is that requests are coming in all the time asynchronously from users you can't control them you're not reading them off of disk but whenever a user sits down on their couch and loads your app that's when you're going to get a request so we want to answer these requests online and at very low latency and consistently low latency for all of your users the next ones are also subtle departures from training so the first is that you might want to have multiple models served at the same time so when many groups start with machine learning they'll have one model that might as this example goes serve rankings for videos but now what happens when you know week two comes along you have a model it's working well in production but now you want to launch a new one you're not sure if it's going to work you might run an experiment so now you want to run two models and maybe you have a couple different experiments you want to run so that's really common as well and lastly who's familiar with mini batching okay just a few of us so I'm going to describe it so we're familiar with this because it's an important part of both training and serving so when you take a neural network graph and you process data through it there's some overhead that happens with all the nodes of the graph and scheduling the work across them and to get good efficiency almost all of the training libraries out there will produce what's called a mini batch so instead of you're putting one video through your trainer at a time it'll come up with batches of 32 64 128 push them through the graph together and you get massive throughput improvements by doing that in particular on CPU on GPUs and TPUs and so on so we aim to achieve that efficiency of mini batching but at serve time when all the requests are coming in asynchronously and you don't have nice neat 32 size batches off of disk and you know the multiple model support in efficiency you know they're neat challenges on their own but what makes them particularly interesting is doing those while also maintaining those standards for low latency all the time all right so just wanted to say again and throw this up there for you all to read that TensorFlow serving is a flexible high-performance serving system for machine learn models and it's really designed for your production environments TensorFlow serving has about three major pillars the first one is a set of C++ libraries these include standard support for the saving and loading of TensorFlow models pretty straightforward something you'd want to do with a serving system we also have a generic core platform and this is truly generic it's not tied to TensorFlow although TensorFlow has first-class plugins for everything you might want to do so let's say you have a legacy ML system in-house and you might want to mix and match for some period of time with say TensorFlow you can do that you can write adapters that plug in different ML platforms and run them in TensorFlow serving building on top of our libraries we have binaries and these incorporate all of the best practices that we've learned out of the box so they make it easy we have reference Docker containers and tutorials and collabs for running on Kubernetes with auto scaling and so on and the third pillar is a hosted service with Google Cloud ML as well as our internal instance that many Google products are using lastly I want to highlight that across across all three of these different offerings your models are portable so we worked really hard to make sure that one model format will work in any of these environments so you can take your model you know try it on a binary you want to do something custom try it in a library and so on so you can seamlessly migrate back and forth all right so super excited to show for the first time that we're used very very broadly across Google so these are just some of our key customers inside Google and for the first time I can say that we have over 700 projects inside Google using TensorFlow serving so whoo thank you all right so I'm going to dive in a little bit to the libraries so I mentioned the generic core platform I wanted to mention also that the components are all a card and this means that you can mix and match them to suit your needs so if you're doing something really advanced and we have some you know incredibly advanced internal customers you can mix and match these components and use just the ones that accomplish your needs you don't have to buy the entire set of libraries you can take the ones you want let's see there's the batcher for inference performance this gives us that mini batching performance but at serve time and lastly you can plug in different sources if you have different storage so maybe you have a cloud storage of your choice you can write a plug-in for that all right so we're going to go through a pretty big slide here I'll try and make this digestible for you so let's imagine and this is zooming into the libraries as they would exist inside our server in the binary so say your server is up and running and it's serving your video ranking model version one and it's cruising along everything's working well and say you've trained version two and you want to deploy it so I'm going to walk you through how that would work so on the bottom right of the slide here you'll see the green source API with the yellow orange file system plug-in so it's a really straightforward plug-in it simply monitors a file system observes the new versions of your model in this case version two and the source is going to emit a loader in this case it's going to emit a loader of a TensorFlow saved model it's important to note that the loader doesn't immediately load that TensorFlow graph it keeps the track of the metadata it can estimate the RAM requirements and other resources used by the model and then it can load the model when it's asked to and this is really an important differentiator between a straightforward model server that you might build yourselves so this loader that's very lightweight is emitted over to the manager now the manager actually knows the state of the server how much RAM is available and other resources such as GPU or TPU memory and only when it's safe to do so is it going to ask the loader to go ahead and load version two of the model so let's say that the manager has plenty of memory it goes ahead and loads version two the model you might think that the first thing it's going to do immediately is let's unload version one we don't need it anymore but there's another important detail here I see some smiles so you probably know this is say you have client request threads on the top left here and they're actually still processing some queries on model one so you actually have to do is keep track in this case we have a handle on top of that TensorFlow saved model and it keeps track with a high performance ref counted pointer mechanism of exactly how many clients are still outstanding processing requests only when all of those requests have queues then the manager will go ahead and unload version one all right so I'm going to cover some strengths of the libraries I'll just highlight a few aspects of this slide so the one I'm going to highlight here is the fast model loading on server startup so this is another really subtle detail but it's really helped a lot of our users so let's say you're starting up a server there's a couple of reasons you might want to do this let's say you're running an experiment on your workstation you don't want to wait a long period of time you want that to load as quickly as you can another example where it's even more critical is autoscaling so say your users all get home at six o'clock they all sit on their couch pull out your application and send you a big traffic spike a video ranking request using a system like Kubernetes you're going to want to autoscale as quickly as possible and you want those servers to start up quickly so at the same time most of the time with servers you only want maybe you know one or two of your threads to do model loading you want all of your threads and CPUs to be performing inference for your users but say on a 16 CPU machine why not use all of those cores and all of those threads for model loading so it's just a small optimization and one of many that we do to make it easier to autoscale and run your actual models in production let's see we use the read copy update pattern which is a high performance pattern for doing concurrent memory access of these models I mentioned the ref counted pointers and the simple plug-in interfaces so it's really easy to extend to support your own data store cloud storage or even a database of your models all right another show hands who has used TensorFlow servings libraries okay a few advanced users so especially for you folks but also for anybody who's thinking that you might want to use our libraries definitely take a look at server core so what we observed was that our libraries were low-level very powerful and very configurable but for most people you really wanted most of the sane and sensible defaults you wanted to load some set of models you wanted to load new versions over time and so we made the class server core which does this for you and it's configuration driven so you can probably if you move to it from our libraries you can remove about 300 lines of code and just have a nice little config so give that a try all right so moving on to our binaries and this is what we recommend for most users so I mentioned a few times there are things like sensible defaults how many threads to use and so on so the binaries come out of the box with those enabled so you only need to specify the minimum amount of configuration they're very easy to configure I'll show you on a coming soon slide how easy it is to launch these and they're based on gRPC which is Google's open source high-performance rpc framework all right so hopefully you can read these code samples so the line at the top this is how you build the model server it's a one liner with basil the second line is how you're going to run the server for a single model in just a one line command no config files needed just three flags so you're going to specify the port the name of your model and the path to the model on disk and this is we call it a base path because you might have multiple versions over time so over time you can just drop more and more versions in that directory and they'll be loaded and lastly we have the command to run the model server with a config file and this could have as many models as you want as many as will fit on your server all right so this is great we have a server how do we actually talk to it so I'm going to speak a little bit about our inference apis that are supported by the model server so the first one is called predict it's very very flexible and powerful so it's tensor oriented you can specify one or many input tensors and output tensors and so you can do basically anything you can do if you're familiar with a tensorflow session if you've been playing with that in say python you can do just about anything there with the predict api now moving up a layer of abstraction we've observed that the vast majority of machine learning used in production is actually doing classification and regression so we have two apis regress and classify and these use tensorflow's standard and structured input format called tensorflow example it's a protocol message it's feature oriented so you can have different values for different features and a nice thing about these apis is you actually don't have to think about tensors at all so if you're new to machine learning you just want to try something out try a codelab or even if you want to deploy in production which we see you know many production users inside google uh using these apis you know go ahead and try the high level apis first next up and I'll talk about multi inference a bit later in the talk uh multi inference allows you to combine multiple regressions and classifications into a single rpc and that has some really cool benefits all right here's a total toy model for you and this is just to show you know what does the syntax look like you know I've talked about these apis if you haven't seen them before so we have a model spec and this specifies the name of the model and this is because your server could have multiple models running at the same time we'll have one feature uh in this case the key for it is measurements we have three floating point values and we have a structured result uh so again you don't have to inspect tensors you'll actually get a structured result called regressions which has one score so just an example all right so I'm going to move on to some of our key challenges that uh for the most part we've solved but we still have quite a ways to go okay so this is a story of isolation and latency so from anybody who's ever looked at a latency graph before you can probably tell that those spikes are really bad you don't want to have those right so uh those spikes were happening pretty much Monday through Friday around 12 or one o'clock so does anybody have an idea just shout it out uh what caused those latency spikes lunchtime strongly correlated so uh we had this fantastic engineer we were working with on one of our largest client teams they were serving multiple models these were large multi gigabyte models and they were serving several hundred thousand queries per second on a fleet of servers and what was happening is this great engineer uh he really a close collaborator um he would uh go to lunch but right before he went to lunch he would push a new version of the model so you can probably pretty quickly figure out what's going on here so uh every time a new model a particularly large one about five gigabytes was loading inference would slow down at the upper end of percentile so you might wonder why is this going on once we figured out the cause is pretty easy to figure out so for most of machine learning you have one model in a server at a time whether you're training or serving until now and so most systems are optimized around throughput not latency and so all available threads are available for any computation that's ready to run in this case model loading is uh parallelizable in tensorflow and so tensorflow would gladly use all the threads available and go load that new model starving inference of uh threads so here's the after slide and definitely note that the uh the right side of this slide the axis uh just dropped by 10x so we went from over a second in terms of our latency spikes to about a tenth of a second this fully met the needs of the customer they're serving sLA they're happy and the way that we achieve this was we added an option to tensorflow that lets you explicitly specify the number of threads to use for any call to your graph and in this case we wired them up so there's one thread pool for model loading and one for inference so pretty straightforward by default since all of our users uh want to use all the threads for inference all the time we actually only need users to configure the number of threads for loading so usually specify one or two and then you're good to go all right so this is going to be uh moving on to batching several slides I mentioned earlier in the talk and this is a really key challenge to get great performance and throughput um and so you know let's look at the right side of the slide you have these touches blocks that are falling down so imagine these are requests from your users maybe those blocks represent uh individual videos that are being ranked by your server and their different uh height represents the time at which they arrive at your server so what we do is we wait for a very small and tunable period of time and we wait for a few queries we aggregate them together and stitch them into one set of tensors that we feed into the tensor flow graph so this enables you to get very efficient use out of uh GPUs and TPUs that have really good batch parallelism uh one external paper the spin paper saw that moving to batched inference on GPU led to a 25 time speed up I'm also uh excited to share that on CPU we've seen for models up to a 50 percent improvement in throughput so it's a tiny little tuning knob that can save you you know quite a bit of your inference costs another thing is that uh once you're actually uh doing batching on top of custom hardware hardware accelerators and GPUs you'll typically only want to have one unit of work being done on that chip or device at a time and so one thing we noticed is when you're going to have multiple models and multiple versions but perhaps one piece of hardware you're going to need to schedule the work carefully across those models and so we have a shared batch scheduler that lets you do just that lastly the batching capabilities are all available in both library form as well as they're configurable in the binary so you can access them very easily uh in either approach now we're going to dive into what happens inside one of these session run calls all right so hopefully you can parse this utilization graph so there are two different kinds of utilization i'm showing here so the first is the blue dash line this is the utilization of your cpu and you'll notice it's not used through the whole period of this request the orange line represents the gpu or tpu utilization so walking through uh we'll start with some single threaded cpu bound work like say parsing of your input it's really common pretty much all models are going to do this then you might do something else that single threaded like do a vocab lookup or an embedding lookup okay you're done pre-processing your data now you're going to shift over uh and do computation on the gpu or tpu so your cpu basically goes to idle your tpu or tpu is maxed out for some period of time it returns and then again you're going to do some post processing on cpu and the reason i show this is that naively if you just ran one inference at a time on this set of cpu and gpu you're going to be vastly under utilizing both pieces of hardware so one of the ways that we solve this is by having multiple threads you know very very common in serving systems and what you can do is you can have you know a good number of threads and limit how many are running at the same time and you can make it so that you're constantly using your gpu and tpu and that most of the time you're using your cpu but the key takeaway from this slide is that there's this queuing time that happens at the beginning of each request so once you enable batching i mentioned earlier there's this tunable window of time where you're going to wait for a few requests to come together and then you'll process them this totally makes sense for the gpu and tpu work but for the work done on cpu there's no point in waiting all this does is add some latency to that work all right so we have some challenges here there's you know the scheduling and utilization of both the hardware accelerator and cpu there's the issue of saturating one resource so maybe you've tuned your model it's working really great you're using all of your cpu but only half of your gpu that's not ideal and vice versa i just mentioned queuing latency so you really don't want cpu single threaded work to wait to be batched together all that does is add some latency and the last one for sequence models where you have input data of variable length and the computation will have variable cost it's really challenging to batch these pieces of work together one of the common approaches is add padding to the shorter sequences they all match but this means that your gpu and tpu will be doing work over padding which is wasteful so there's you know challenging tradeoffs there so we think we know the way forward i'm going to share with you some work that we're currently doing is ongoing but it's still in the experimental phase we are moving batching inside the graph and we think this is going to be huge for throughput in performance in particular for challenging models moving batching inside the graph enables you to surgically batch the specific subgraphs that benefit the most from your custom hardware your cpu's and so on for sequence models where you have things like a while loop with a single step of code being run inside the while loop you can actually run the batching only inside that while loop so it's really neat you're never doing batch inference or batch computation over padding and you know early results look super promising here another area is for complex models that might have for example a sequence to sequence model maybe it has an in-code phase and a decode phase that have different costs this would let you batch those subgraphs separately and tune them separately all right next up i'm going to share some new technology that we recently released as part of tensorflow 1.0 so the main piece of tech here is save model so save model is the serialization format for tensorflow models it's available for very broad use it's not specific to serving you can use it for training you can use it for offline evals and you can use it for other interesting areas i'll get into save model has two new features to tensorflow so the first is that you can in one save model have multiple metagraphs that share the same assets such as vocabs and embeddings as well as the same set of trained weights you might ask what i want that for i've just been training models well it turns out that the metagraph for serving is very very commonly different than the one for training and now let's say you want to have a you know third metagraph that's going to be used on gpu that's also something you would want to customize so the multiple metagraph support lets you have you know a metagraph for training one for serving on cpu one for serving on gpu all right the second major feature of save model is signature def so a signature def defines the signature of a computation supported by a tensor flow graph you can think of it like a function in any programming language it has named and typed inputs and outputs so as humans we're really good at looking at that graph you can probably all guess what the nodes in that graph are i guess if we pulled us you know all of us would figure out that the middle node on the left side of this graph is where you feed the input but if you want to take your model and have a whole ecosystem of tools such as serving systems that can interpret those graphs that they had never seen before you would need to annotate that graph in some way that's exactly what signature def does so in this case the middle node on the left is where you feed your inputs the top right node outputs a set of classes and the bottom right node outputs the scores all right so building on top of signature def is multi-headed inference this is also known as multi-task and multi-tower inference so another show hands who's familiar with multi-headed multi-task inference oh wow okay so this is pretty new there's some emerging research and great work going on here i think you'll be really excited about this so let's take that example earlier you're serving video rankings your users are using your app you get lots of content creators you're successful now and so users are drawn to your platform and maybe some not so good people are drawn to your platform maybe they're uploading clickbait we've seen this in many places so your first model the orange model computes the click-through rate and you know these bad guys are you know training models optimized to get clicks so you might decide well let's define a new metric and train on that let's call it conversion rate and it's going to track users watching the majority of a video so we'll call that cvr and now we're going to go train a new model for this but wait a second all of the careful feature engineering and other things that we did to get our data ready for the first model apply to the second model it's very very likely and in many cases it's a certainty that those features are usable in both contexts so you can actually train one model for multiple objectives in this case ctr and cvr so i've listed some infrastructure benefits on this slide and there's some really really big infrastructure benefits your inputs are only sent once over the network so that's already a pretty big one you only have to parse the data in your model once you only have to compute your hidden layers one time so this is going to save you bandwidth cpu latency and even ram overhead on your servers this is great i could stop right there there's another really important attribute of these uh why they're uh becoming more and more exciting and the research that's going on so you're likely in this example to have many more clicks then you'll have conversions so when you go to train these models you have much more labeled training data for your ctr objective than for your conversion objective so what happens when you actually train one model for multiple objectives so we've actually seen some early promising results where in one case we were able to see a 20 improvement in a key quality metric by moving an existing separate model into a multi-objective model with another objective so there's wins here on infrastructure and on quality multi-headed inference is available in the tensorflow estimator api so you can train models with multiple objectives today and you can serve them as well so we're really excited about this all right i wanted to show you know really quickly what does a multi-headed inference request look like so it's pretty simple you can specify one or more inference tasks they each have the name of a model as well as which signature to use so really simple okay uh all of this you know power and functionality sounds really great uh from early adopters we also heard you know this is challenging to get right i might want to do things like inspect a model and see you know what's actually going on with it run a sample query look at the metadata and so on so we're introducing the save model command line tool and it lets you do all these things with the model so the first query you can do with the save model uh client you can look at what metagraphs are in the model so pretty simple so in this case we have a save model it has two metagraphs one is for serving and one is for serving on gpu now let's say we wanted to look at the serving metagraph and see its metadata in this case the metagraph contains uh two signatures one is called classify x to y the other one is called serving default so serving default is a documented key and it's a constant uh in tensorflow and what this does is it says if the user has not specified which signature to use just use that one and in many cases for people getting started you have just one uh signature so it's really easy to get going all right now say you wanted to actually look at the input and output tensors in the serving default signature so you add on one more flag to the client and you'll see the inputs and I skipped the outputs for the slide here you'll see there's one input tensor called x it's of type float note the method name on the bottom so the method name here TensorFlow serving predict this is kind of like a type in a programming language um it's informing another system in this case TensorFlow serving that this signature was intended for use in the predict API we have similar ones for TensorFlow serving regress and classify but you can override these so maybe you have an in-house offline evaluation system you can make your own method names and check for them in your models all right and lastly let's say you would like to run a graph you have some input data maybe you're debugging the model or you just want to try it out for fun so uh I'll highlight here all of the prams are the same except instead of show we've asked to run the model we also have two ways of expressing input that you can mix and match so maybe you have a numpy file you can actually specify the path to the numpy file and it'll just be read you can also specify a python expression and it'll be interpreted all right so in closing wanted to say that collaboration is very very welcome on this project uh we sync our internal repository with github about weekly we have a developer on call rotation that includes facilitating your pull requests answering questions and so on uh we have lots of open-ended research problems you know so feedback is encouraged on apis techniques anything I've talked about and any challenges you have as well you can reach us at discuss at tensorflow.org for some links on how to get started uh just search for TensorFlow serving very easy to find uh we have a great kubernetes tutorial this will let you uh you know launch an inception model server uh and it's gonna run an autoscale for you we have a google cloud ml uh as well as I mentioned our mailing list and you can also use our uh hashtag on stack overflow uh hashtag tensorflow all right let's uh go to questions thanks very much hi there ios developer interested in getting started with the firebase platform in your app well you've come to the right place there are two main parts to getting the firebase platform up and running adding your app to the firebase console and installing the sdk let's go over these one at a time for starters let's go to the firebase console at this url here depending on when you're watching this video the ui might look slightly different but the general concept should remain the same now depending on your situation you might see a blank create a new project screen or you might see a list of existing projects oh before we go further let me take a moment to explain the difference between projects and apps a project consists of one or more apps now all the apps in the same project use the same firebase database backend and if you want you can use features like firebase cloud messaging to talk to all of them at once you don't have to but you can which is sometimes convenient so if you're a developer that has a cross platform app you generally want to put the ios and android versions of your app in the same project now that'll give you some nice cross platform benefits your user can access the same data if they switch back and forth between the ios and android versions of your app things like dynamic links will work across both platforms you can send one notification to all versions of your app and so on on the other hand completely different apps you should put those in completely different projects there's nothing gained by cramming them into the same project except tiers and heartache i guess so if you're working on a cross platform app and your android or web team has already created a firebase project you should probably select that project and connect your ios app in there otherwise if you're the first one to be adding firebase e capabilities to your app you can be the one to create the new project in my case i'm the first app associated with the project so i'm going to create a new project here i'll give it a name and there we go once you've selected or created a project you're going to want to connect your client app i'm going to select the ios button here and i'll give it my apps bundle id you are eventually going to need to add your app store id here if you want features like firebase invites or dynamic links to work but you can leave this blank for now and change it later now when you click continue your browser should automatically download this google services info dot plist file for you note that it needs to be named this exactly so if you get that little one in parentheses after the name like i just did you're going to need to do a little bit of renaming and find her okay next up drag the file into your xcode project like so and uh let me go back to the console and hit continue here and it's telling us that this would be a good time to install the firebase coco pods now i'm assuming you know something about firebase coco pods but if you don't here's a little video for you to check out it's fun so i'm going to jump into my project directory here and do a little pot in it we'll open up the file and i'm going to uncomment this line because i am using swift and uh this line because my app happens to have a base sdk of eight dot oh although at the time of this recording firebase is supported as far back as seven dot oh next up let's add some pods now it's important to remember that to keep your app nice and spelt you should only install the coco pods for features that you need in fact there is no all-encompassing uber firebase coco pod that installs everything for you you're going to need to pot install each individual feature and you can find a full list of the coco pods and the features they correspond to over here now for starters i'm just going to add firebase slash core which includes everything needed to get the basics up and running and also enables firebase analytics so now i'll make sure my project is closed and i'll run pot install and then let's open up the generator workspace all right we'll build it and make sure everything compiles okay and it does so we can move on to the next step okay looks like the last step here is to add some initialization code i recommend putting this in your app delegate did finish launching method uh first things first let's import firebase in my app delegate uh note that this is usually the only thing i'll ever need to import no matter what we've installed we're doing some pretty nifty work behind the scenes to make sure this works properly and i know this sounds like the exact opposite of my only pod install what you need advice from like two minutes ago but trust me here this makes development a whole lot easier all right so next up we'll add the line for app dot configure to make sure firebase gets set up properly and uh that's actually all you need to do this configure method will take a look at what libraries you have installed initialize them grabbing all the appropriate constants from that google services file that you dragged in earlier into your xcode project so we'll give it a quick run and if everything is set up and working correctly you should see a few lines in your console about how firebase analytics is up and running all right so congratulations you are now up and running with firebase um so there are a lot of places you can go from here you can add sign in using firebase auth or get your app talking to the real-time database or start tracking more of your user's usage with firebase analytics you can check out these links to get started and have a little fun what's up everybody david here and today i have a quick and easy firecast for you we're gonna get up and running with firebase and the web and this is actually going to be the first of many screencasts in a series so make sure you subscribe to get notified of tutorials on authentication storage hosting and web push notifications with firebase cloud messaging also if you're a fan of javascript frameworks i'm going to be dropping videos for angular one and two polymer react and ember so you better subscribe because you don't want to miss those but today we're going to start with the very basics i'm going to show you some mad copy and pasting skills by getting the project initialization code from the firebase console and then we're going to set up a small web app so let's go and dive in so i'm in the firebase console at console dot firebase dot google dot com you can see i'm logged in as myself up here just smiling at you but to get started i'm going to create a new project so i'm going to click create new project i'm going to call it web quick start and then we'll create it my project is now created so i'm going to click add firebase to your web app and this brings up a little model with all the initialization code i need to get started it has things like my api key off domain database url and storage bucket so now i can go to the bottom right and then i can click copy and that's all the code i need to get started but just as a little fyi you can access all of this information by clicking off and then going up to the top right where there's web setup but now to the editor so here on my editor i'm going to get crazy i'm going to create this web page from scratch so i'll start with my basic html boiler plates give it a title and now i can just paste in all the code from the console and this is all you need to get started and just to prove that it works i'm going to use the database as a little tiny demo so i'm going to create an h1 and give it an id and every single time the value changes in the real-time database i'm going to sync it to this h1 so the first thing i need to do is get that h1 by its id and then now i'm going to create a database reference using firebase.database.ref and then create a child location to the text location and now i can synchronize any changes using the on function and then using es2015 arrow functions i can just do it all in one line so to the left right here i have my project in the firebase console and to the right is just my blank page to use the database i'm going to remove all security so i'm going to click rules and then i'm going to say read is true and write is true and click publish and you should totally know that you only do this while you're developing because that means anyone can read or write to your database so now i'm going to give my browser a refresh and then i'm going to add a text location and it synchronizes to the browser and so i can change it and then it changes as well so keep in mind that the real-time database is just one of the many features firebase offers for the web you can also use authentication storage hosting and even firebase cloud messaging so that's all it takes to get started with firebase in the web and if you want to go and learn more then check out the link in the description for our official documentation and if you're super excited to learn more about firebase on the web then please subscribe to our channel because we're gonna have tons of more content so that's all for this time and i will see you all later we started is on a living room couch and we really started because the problem that we had which was asking the same question to our closest friends where are you what are you doing we were baffled by the fact that there wasn't a solution that solved this problem and we felt like we could build one that was better the value that is drives for all users is knowing which of your friends are nearby so if you look around where we are right now an arena how many times have people gone to a basketball game hockey game or a concert and found out the next day that they had friends were at the same event and think about all those moments that are missed because they didn't know they had friends there so what we're solving is letting people know who's nearby and making those moments matter my name is diesel pelts and i'm the founder and CEO of is i'm mark french co-founder of is we felt there was no reason users should manually go fetch data but i get a text message there's no reason for me to tap refresh and we felt why should it be different from anything else and firebase let us solve that firebase really allowed us to enhance user experience by making it real time simplify the ui by not having a fresh button and cut down and develop in time like any startup the most valuable asset that you have is your team and your time and what firebase has allowed us to do is save 50 percent in terms of time by moving that much quicker with a product it's a game changer we're using eight features from firebase right now they're analytics remote config dynamic links the real time database and more traditionally that would have been in eight different places and now we go to one place through the firebase console we're eager to launch this product in a big way we're seeing how people are using the product and how they're inviting more and more friends that we're concerned we're growing very very quickly so we sleep a lot easier at night knowing that we got firebase that's really there to build that infrastructure if your developer use it we love it and it's enabled us to focus on developing the user experience and not have to worry about the things in the background that should be and with npr one we are reimagining what a listening experience could be outside of the radio it's the radio but better it has all of the great stuff that we've spent 40 years perfecting with npr one we see the opportunity of reaching a more diverse audience that have a device in a pocket all the time my name is might safe lahi and the lead mobile developer for npr digital media my name is nick to pray and i'm the innovation accountant and npr my name is tejas mystery and i'm the senior product manager of npr one there's some of the biggest challenges in any mobile app are that first impression when the user first installs the app you've got a very limited amount of time to convince them to keep the app and to get engaged in the experience trying to figure out how we can get users into the content as quickly as possible was the real focus of integrating firebase and dynamic links using dynamic links we are able to shorten the number of interactions it takes for a user installing the app to get from the promoted content to the content from 20 to 3 so that user is able to get right into the content we're driving more and more listening for a user every week it's really astounding creating playlists of content that are configured by the podcaster or by a member station or by us internally and with firebase we have that at our hands having the analytics product interact with things like dynamic links remote configuration cloud messaging it adds a real multiplier effect and the integration with the broader firebase suite i don't have to go outside the platform to figure out what's working so it's not just about shipping the product faster it's about analyzing the results faster and with the integration with all the other firebase products we're really excited about all the things we can learn from it raised labs is a company that is focused on building excellence in software technology and design we do that through our work on mobile applications and websites and technologies in general my name is Gregory raise i'm the CEO and founder of raise labs we really want to understand the human problem and oftentimes the hard problems in software aren't just the technology problems the api the how do you connect these things but really getting at the heart of what people are trying to accomplish and do in their day to day my name is ben johnson i'm the managing director at raise labs in boston we decided to put our hat in the ring for the google certified agency program the first leg is just having access to a lot of what google is doing today so there's access to design reviews invitations to events and that's sort of the base level and i think that's hugely rewarding even in and of itself having google review your app from a design perspective is amazingly helpful so that's sort of the first year the second tier comes with certified status you know there's a long application process for that and once you have it it's something that you can really say to your clients that gives them comfort that we're a reputable firm that we're building great software in a way that google believes in the certification is a higher bar for us to really differentiate ourselves from many of the other companies out there it required us to really dig into what that means to be truly world-class and we wanted to set that bar for ourselves as well my name is john green i'm a vp creative at raise labs for the google developer agency program allowed us to have access to engineers with the map team the design team to figure out how can we actually do some of these things and we could reach out to them when we needed and also allowed us to set up and say we can make this success they might look closer at this app because we're part of this program which has actually been super helpful some of the challenges in building the six flags app and which touched on some of these are certainly mapping technology and payment technology material design or the api's having access to the google team to really ascertain how we're approaching certain software and ensuring that we're building technologies the right way makes for a smooth development process we set off to build the six flags app with a pretty lofty ambition and it was to bring in park navigation and commerce to the app and the comfort of knowing that google is there to help us understand where they are heading as an organization and that we are along for that ride is it a really helpful thing to know and as a business we know that going forward we're going to be at the cutting edge of whatever google is doing through access to programs through you know the collaboration with their teams it's really helpful for us to know that six months nine months down the road will still be a part of that process and we'll still be working with them to figure out what's next it's going great this is brad abrams brad is a product manager on assistant that's right yep how's the going timothy it's going great um now that you're here and i'm here i think we should talk about the assistant what do you think i think that sounds like a great idea because the only thing i know to talk about so perfect all right let's start with square one okay what is assistant and why is it cool for developers well the assistant is really a conversational interface to google it's uh the kind of smart brain in your google home or on your android phone and actually at this event we just launched it on ios as well uh and you can have really a conversation with google to help you get things done that's awesome i think so what are the different uh surfaces for developers to integrate with oh great so assist there's a couple of different ways one you can extend the assistant by adding your own custom apps to the assistant um and then you can also host the assistant in different devices so if you're building your own hardware or whatever you can host your assistant there and that's the difference between actions on google and the assistant sdk right exactly so the assistant sdk is where you host it with your hardware and actions on google is extending the assistant okay well we're at io and i know some new stuff has been announced why don't you tell us about some of your favorites oh man it's hard to pick there's so many uh i'm really excited about actions on google coming to phones to android and ios so uh up until this event you can only build actions on google for home for ice eyes only experience or ice ice free experience um and now we have actions on google for the phone as well so we have suggestions ships lists image carousels they can really give you a deep multimodal experience oh would you mean like seeing and hearing that's right you can see and hear yeah we also launched uh transactions so you can buy things via the assistant i think that'll help users a lot i know i want to buy a pizza uh on you know from my phone i can just say okay google i want a pizza and uh it can connect me with somebody to go do that and and for developers it also gives uh you know a financial incentive which i hear some developers like to get paid yes so indeed they do uh and then of course deep uh for me is developer experience and we've done a lot of work we launched the platform first in december and we did a lot of work on developer experience so we launched a new version of the developer console we call the actions console uh developers can use that to work better in teams together to get deep analytics about their application as well as it integrates really well with the firebase and google cloud consoles awesome and what can i do one more absolutely see i can just add them so one more uh would be discoverability i think that's been in the whole kind of chat bot space generally having people be able to discover the them has been a problem and so we've done a lot of work in that regard uh the first is that we have a new directory for our apps and it's right there on the assistant surface just one tap away and you can see a category view of all the assistant apps that are there uh we also gave you a way to get a web URL into that so if you wanted to promote your app via your enormous twitter followers you could totally go do that which is awesome and of course people who love your assistant app can share it with their friends via the deep link um and finally we did some work to help discover when to invoke your uh assistant app from information in the directory other information provided by the developer we can just respond to queries like uh play a game and we can actually find the right uh assistant apps to go trigger in that case that's totally awesome yeah it's really fun it's really it's kind of a big release for us here at io yeah i mean there's a lot of different stuff yes it certainly certainly is but i think it comes together in a really cohesive package though um because you know if you're going to do transactions you're going to buy things a lot of times having a visual surface to see the see the different options see the card and what now it really helps with that um and then of course if you're going to have a financial interest there being able to have people discover your assistant apps uh really helps awesome okay so let's take a step back and i would love to hear your thoughts just on conversational interfaces in general yeah i mean i if you think about this evolution that's been happening you know like when i started using computers it was like mouse and keyboard it was very indirect right you move wait you had a mouse i did have a mouse you don't think i had a mouse well i mean you know you start with like the common or right yeah yeah okay before you had a keyboard and then we added a mouse but these are indirect like you you do something with your hands and something happens on the screen and then you know we got capacitive touch right so you could actually touch the screen and like my kid when he was two knew how to touch the screen right like that was very intuitive and i think we're at the verge of making the next step and that is rather than just being able to like touch the button what if i could just express what i want why do i have to dig through a bunch of menus find the right button to press not really sure what's going to happen when i press that button but what if i could just express my idea this is what i want to happen i want to play this game i want to order this pizza i want to do whatever and that's what i think this conversational interface is enabling awesome um i and i do want to say that i think one of the things that's been holding us as developers back from that is the right tooling to do so you know you know as i described that i think the developers are going like wow sure but how do i do that when the user how do you think about the huge number of ways people could say they want a pizza order toppings on the pizza how can they turn the huge vagaries of human language into something a developer can understand and that's where tools like api today i can really come into play api today is our own conversation building tool and what it does is you let's you as developers define intents just by saying giving us some example phrases of what a user would say and then we can do the um pull out the entities from that and do the natural language understanding so that we can do slot filling and actually figure out what users are saying so you might give an example like i want a large pepperoni pizza with tomato sauce as your example we know it's large is a entity pepperoni is an entity um and tomato sauce is an entity we can actually pull those out and then recognize other queries like i want a small pizza with tomatoes and green olives uh with you know no sauce and and that uh because we can do that it makes it much more tractable for developers to actually understand human language seriously how many times have you ordered a pizza with no sauce well you know that's pretty rare but you know i actually i'm for extra sauce you know just because it dribbles down your oh okay that's that's that's good all right is there anything else you want to say about assistant or any of the developer tools before we go uh no i think we pretty much covered it all right brad thanks so much all right thank you is this the clicker everyone welcome and thank you for coming to our talk today we are members of the android ux team and we're excited today to talk to you about the new notifications features in oh from a ux perspective so i'm julie i'm a researcher on the android team and today i'll first tell you about the research that inspired the new notifications features in oh so in november of 2015 our team went on what we call a ux expedition to learn more about how people experience notifications so our team of researchers designers engineers pm's we all spent a week in new york and new jersey speaking to and observing users we approached the topic as attention management to include interruptions from devices other people around us ourselves and everything in your environment we didn't want to limit our scope to just on-screen smartphone notifications we used multiple methods in the research we started by interviewing people but then we also observed people because notifications are so contextual and in the moment we wanted to observe people receive notifications to see what they would do and how they would react to them we also had group discussions amongst friends who did design exercises together and lastly we synthesized the data together as a team and we held a workshop later and came up with ideas that turned into the features in oh so what did we learn the most striking observation we made is that there was this addiction to phones people were bound to their phones all day because they were afraid they might miss something important phone notifications were a major source of stress people were very hyper vigilant to any kind of signal they could be getting from their phones afraid to miss something important and most notifications were actually not important to them only a few were necessary we heard lots of things like this from our participants denise said if i don't have my phone for two seconds i freak out what if i miss something and she was a busy lawyer and i observed her in her house the whole day constantly going like this where's my phone where's my phone and she'd leave the room and come back looking for her phone just really anxious all day just in case her phone wasn't with her and then there's Alice who's a professor and she said i wish i could have something like an amber alert for my parents ex-husband and son's school so that those would override everything else she gets lots of notifications from different sources but the only ones she cares about are these and she wishes these would always override the others we observed different coping strategies from users to block out all of this noise the first one is to physically put the phone away so people would force themselves to put their phone in a purse put it in another room someone even put it in a box in another room just anything to prevent yourself from checking it the other strategy was to not let the device make noise so let it just vibrate or don't even let it vibrate at all another strategy was to not let sources contact you that means to block all notifications from an app and the last one is to do nothing but be constantly annoyed every day and all day Stephen he mentioned a particular social networking app sent me way too many notifications so i deleted the app so he got fed up and deleted the app and actually every day he uses the service in a mobile browser multiple times a day purely just so we wouldn't receive notifications in fact the pattern that we're hearing now is when people get so annoyed with notifications whenever they download a new app by default they'll just block it all block all notifications without even giving it a try this is obviously something we don't we all don't want so we need to really be careful about this cumulative effect of all of our apps notifications people often want only some notifications from an app but not all of them so even if they want just one type of notification they still receive a bunch of unwanted ones and many people don't adjust their settings even if they know they can so even if they know there's a way to do it they might not know how to do it or it might feel like a hassle so they just live with the problem Angie said notifications are like junk mail you can take the time to set up filters or just hit delete i just deal with them just deals with it every day Stella said if you allow push notifications i know there's a way to go into settings and adjust it but i don't know how to do that so what else do we learn there were only two types of notifications that people needed to know the first one above all notifications from other people are most important so these include sms messaging apps emails from people phone calls etc anything that came from a person on the other end these were most valued Flora said notifications from other people are most important i always respond those are the only ones that she pays attention to and then there are more important people so notifications from the inner circle matter even more so these are people like your spouse significant other parents kids closest friends they can reach us at any time and we keep channels open and reserved for them andrew said work emails and texts for my wife are the only things that matter basically things i need to tend to so i don't get in trouble so those are the two types that he cares about above everything else the other kind of notification that's important to people are reminders for things that they need to do so we observe people having things they needed to remember throughout their day like picking up your dry cleaning calling a credit card company to follow up with an issue cleaning a chinchilla cage if that applies to you setting up a kids play date making new keys for the car things like that right we all have things like this in our lives and people need to remember these things david said sometimes i get a notification for something that i actually want to do but just not at that moment and then i forget about it later because i don't see it again and shelly says i know i have to pick up my dry cleaning but telling me right now is not going to help me it's like nagging tell me when i'm near the dry cleaner so when a notification isn't addressed right away it's very easy to forget it out of sight out of mind we realize that we need to make it easier for people to remember these tasks so while this research as i mentioned was conducted only in new york and new jersey we've observed similar patterns in many other studies across many locations since then whatever topic we're studying from productivity to wearables to driving we keep hearing over and over again from people about the stress and the burden of all the notifications are being bombarded with so here's some other locations that we've done where we've heard similar patterns across the us and many cities and internationally so based on this research our team felt it's important for us to respect our user's attention and to allow them to have more control so this inspired some major changes in oh this year now here's rachel to give you more details about the features hi i'm rachel garb and i'm an interaction designer and manager for the android ux team the first new feature i want to talk about is channels a show of hands from smartphone users in the room who here has ever wished that they could get some of the notifications from an app but not all of them that's that's almost all of you yep so knowing that we definitely wanted to add support in the android system so that users could have more granular control over an app's notifications in oh we're introducing channels and here's how it works you define a channel for each notification type that you want to send and then your users are able to make decisions about notifications at the channel level let's look at an example suppose that we are all the designers of an app for crane air a fuss free corporate airline that caters to the frequent flyer and here's all our existing notifications we've got a pretty sweet program for our frequent flyers or as we like to call them executives we have a set of notifications for our customers about the points they've earned and their status in the program now all of these notifications you see here have to do with the executive program so let's make a channel and call it executive program now we occasionally also offer discounts and deals to incentivize our customers to book more flights so let's group these notifications together in a channel and call it discounts and deals now when we categorize notifications into channels we want to look for similar subject matter and similar importance to users here we have a number of notifications that could be sent in the hours and minutes leading up to the flight so they're pretty important let's create a channel for them called flight updates and now with this example in mind let's shift over to how users experience channels the ui design for channels is optimized for the number one most important use case which is i don't want this type of notification from this app now without channels any support for this use case has to come from within the app with settings to block notifications by type now how does a user find out if an app provides this they have to open the app and look at the settings and it's not like all apps are going to do it the same way but a lot of the time there won't be any way to block notifications by type and at that point the user has to decide what they're going to do they can ban the app from sending any notifications at all they could uninstall the app or just live with it and none of these are ideal but with channels it's a different story one that's straightforward and consistent let's check out the ui now we at crane air know that a segment of our users love to celebrate when they earn points or reach a new status so whenever they land we notify them about how many points they earned on the flight and how they close how close they are to the next status level but maybe not all users want to be notified like this some might not want any notifications except the ones that are essential to functioning in their daily lives and they know that they can get this information in their monthly statement they're fine waiting for it so to stop this type of notification all a user has to do is touch and hold it which lets them see the channel and then they turn off this switch and tap done no more executive program notifications from crane air but they'll still keep getting the others like those handy flight updates now our aim with channels is to make it possible for users to get most if not all of their notification control needs met in line or one step away in settings now in settings users can decide to block all notifications from an app and new for oh an app can show a dot on its icon when it has notifications waiting for a user but if the user doesn't ever want to see a dot for that app they can prevent the app from showing it users can see all of the channels at a glance and decide which ones they want they can view and edit settings for a given channel more on that to come and if the app wants to provide other settings beyond what's here the app can have a link for the users back to a screen in the app this approach bringing more notification controls into the system makes the user experience more convenient and consistent so that is an overview of channels and next i'd like to talk about the new behavior model for notifications in oh and i'll start with the research that informed it some people are putting up with unwanted notification behavior because they're reluctant to use settings you heard about this from julie so the opportunity here was to make it easier for users to understand and control notification behavior now let's compare the current behavior model with the new one for oh in the current model the app developer uses a field called priority and sets a priority level for each notification that level maps to a set of behaviors and some of them are customizable by the app for example if you choose high if you choose priority level high as the app developer you can have your notification make sound or not you can have it vibrate or not you can have a peak or not it's a lot for a user to follow in the new model priority is deprecated and instead the app developer sets an importance level for each of their channels rather than for individual notifications and there's fewer customizations that the app can make on that behavior here you see the inner workings of priority levels i won't read this out to you but let's compare it with importance levels as you can see we've gone down from five to four levels and reduced what can be customized we are intentionally trading flexibility for simplicity so that users have a better shot at understanding what the heck is going on we're letting go of allowing each notification to behave differently so that we can guarantee users that all notifications from the same channel behave the same way in the current model a user can control an app's behavior but has no insight whatsoever into the app's intentions as they're making decisions but in the new model the user gets a clear picture of app behavior as they make decisions so it's the difference between i can make sure that this app won't be able to do x versus i can see this channel does x but i'm going to have it do y instead i'm going to show you what i mean by comparing the UI in the current UI if show silently here is turned on then it's predictable but otherwise the user has no clue how notifications from the app will behave in the new UI the user just needs to look at the important setting to see how the channel will behave and if they don't like it they can choose a different setting in this new model android creates a setting screen for every channel from every app this is the place where users go to see and control its behavior you the main settings are to block the channel and change importance if the importance level is high enough the user has sound and vibration options too and there's more advanced options now you can also optionally provide a brief description of what the channel does if it's not obvious so that users can make more informed decisions if your app already offers channel like behavior please leverage android solution so that users don't run into the situation where there's discrepancy between an app and system settings you can link to the system settings screen either at the channel or the app level and if you want to offer channel controls beyond what the system offers for example letting users specify notification and email preferences in the same place in your app you can have a link back to your app settings and that's an overview of the new importance model next I'll turn it over to Justin to talk about other new things for oh thanks Rachel I'm Justin Barber and I'm a visual designer on android and I have to start with an apology because many of you probably showed up for this session hoping for Justin Bieber concert and it's not happening but I saw some pretty interesting stuff to talk about so visual hierarchy is the focus of oh and if you remember last year in n we completely redesigned notification templates to become more flexible modern much cleaner and the new features in oh that I'll be covering we're really built on the foundation that we set in n and also on Julie's amazing research so that's where I'll start with the insight research sorry with the research insights that really led to what our approach in oh so the first insight was that people are overloaded with notifications and it's hard to find the important ones I'm pretty sure we've all been here before and in Julia's research we uncovered that users were building associations think to determine what was important for them so if the if the phone made a certain vibration tone or they saw a certain color of light blinking or they'd see a color in their notification they were making associations to determine whether or not this was important enough for them to spend time on but that can be frustrating misleading and above all time consuming so our goal for oh is to present notifications in a way that helps users sift through the noise not just to build more associations but to build a consistency and a trust you should know where the important stuff's going to be before you have to go looking for it the second most important insight that led to our approach was that people are the VIP notifications in absolutely all scenarios who is more important than what or when so the opportunity here is pretty simple make people notifications VIPs of the drawer people matter the most and notifications is really communications hub so we really want to emphasize that in this space so as I mentioned before and focused on individual notification templates and then oh we've zoomed out to kind of see the bigger picture and how they all fit together in one experience specifically by improving the hierarchy of notifications that appear in the drawer we did this by designated four different types of notifications and but four different buckets for them represented by these yellow blocks before I get into the detail of each different bucket I'll give you the the high level overview so the first bucket is for major ongoing this is things like a phone call the next is people to people like a message from a friend the third is a general notifications the best of which are timely reminders and the fourth are by the way or a BTW and these are the four buckets of notification types and how they're organized within the drawer and the purpose of this is to ensure that notifications most critical to read first are shown first it also creates consistency when you pull down the drawer to find that specific notification look you're looking for you'll already know which area to be looking at because you'll become familiar with these sections so we're enabling users to build associations through these through this ordering and lastly we're going to be ranking notifications within each section this means if you have two message notifications they're not going to be separated by something else in the middle they'll always appear together so it'll be a lot easier to scan the notification drawer so now you have a big picture let's drill down in the details of each bucket starting with the first one major ongoing notifications are really for time sensitive content users should actively be aware of this is like phone call driving directions timer music and as a developer to send this type of notification you need to be running a foreground service with importance higher than men or using a media style with media session for music notification and this type of content is for content that's using device resources that directly impacts real-life context and the things that match that is a really really short list the type of notification will rarely be appearing in the user's drawer because the context that requires it is a very rare situation so in end this is kind of what these types of notifications looks like right they're not very visually distinguished they don't necessarily always appear at the top of the drawer they can be lost amidst the other notifications and if you wanted the proper prominence to really gain that association for the user you had to go with customization which led to inconsistency with the rest of the notifications so an oh what we're doing is we're pulling that color from the app and filling the background with it and guaranteeing a top spot in the notification drawer so these will always appear at the very top but again because they're so prominent misusing this type of notification for something it's not meant for will be very obvious to users and with the channels that Rachel just described it'll be really easy for them to block this type of notification if it doesn't align with what it's really meant to be falling into this category is also music so in end you can see that the templates are the music template the media templates the most different template that we have compared to all the others but it still feels really generic and it doesn't really look different from other notifications so an oh we've integrated the album artwork into the notification and drawn out the most prominent colors for the background and as an accent the result is a beautiful lively notification that really celebrates the artwork and the artists behind the music that we really love and I found myself pulling down the notification drawers I listen to music just to see what kind of colors that song generates now for the next section people to people this is really for content from a person like a message or a miss call and as Julie mentioned people are the VIP notifications so we're going to put them right at the top of the drawer in the number one position since major ongoing is going to be pretty rare to get this type of notification you should be set you should be using messaging style with direct reply or attaching a contact to it so as it stands today a message notification is really not that valuable and until you interact with it there's like one line of the message so you don't know if it's worth actually expanding and reading and then you start to make the decision if you should reply to it and you may not be able to reply to it directly within the drawer you have to jump out of the drawer so it's it's really just not that valuable so in oh we've made one really subtle change that has actually made a really big impact on the messaging experience and that is that we're exposing more lines of content in the collapse state of the notification so now you can read up to three lines of the message without having to spend the time expanding the notification and deciding what you want to do with it and we're continuing to invest heavily in this area of focus and we're looking forward to making even more improvements especially around direct reply coming soon this next bucket is for general notifications and the best of which as we documented in material guidelines and Julia's research backs up is really meant for well-timed and informative task reminders and in oh there's no behavior or visual change in the representation of these notifications but it does play into the next bucket of BTW and today there are a lot of notifications being sent in this general bucket that really should be sent in oh as a BTW BTWs by the way notifications are meant for contextual or informative but non-urgent content we heard a lot of stories from Julia's research about how people felt so overwhelmed if content in the drawer isn't urgent it should be sent as a BTW things like weather updates or traffic or suggestions and promotions so earlier in this presentation Rachel used crane air as an example of how to organize your apps notifications into channels if you remember there are three different channels executive program flight updates and discounts and deals and that last category of discounts and deals is an excellent example of what should be sent as a BTW notification and it's pretty simple to send one you just need to set the importance of the notification to min so in n the visual representation of these notifications is pretty equal with all the other notifications the only difference was a slightly different colored background it's a pretty heavy weight presentation for content that's actually very lightweight it's bothersome to users it takes up a lot of space so what we've done in oh is we've taken the app name and the first line of the notification and simplified that down into a single line so the new presentation is really meant to feel lightweight so that users don't feel bothered by them they don't need to manage them because they're out of the way and it's okay if they can glance at and don't want to take action on it right now you can also set your notification to automatically expire so that users also don't have that burden of having to manage them as well the advantage of this is that instead of risking annoying your users by sending a notification in that general bucket you can send it as a BTW instead this way you don't have to hurt users trust by risking something you can just put it in the BTW section also the full functionality still remains so if the user taps on this BTW it expands the full notification you get all the color and if the notification has actions with it those will appear as well so when you put it all together it comes it's comes together really nicely you can actually tell a very intentional order within the drawer and when we compare that with n that's when you can really start the difference you can really see the differences come out if you squint your eyes and look at n which is on the left in the black phone frame you can't really tell which notification is which at a glance if you look at oh it's really clear you can see the major ongoing at the top you can see the BTWs at the bottom you can even kind of tell it's a for people to people a message notification because it's taller than the rest now you can also see an n how when there's an overflow of notifications those notifications got stacked behind the ones that were visible this doesn't match up super well with the spatial model of the full width notifications so an oh we've improved that spatial model and made it feel much more unified of a really beautiful animation and the point of this is to correlate the notifications of icons of how they appear in the status bar with the notifications that they are in the drawer and again yeah you think that was pretty cool right this is something we're really really proud of and again it wouldn't have been possible without those template changes that we made an n right so we're taking steps to unify the whole experience you can also see out instead of stacking at the bottom overflow notifications fit into a shelf and we have a really beautiful experience going on I'll just let it play one more time whoo all right so this is the last feature I'll be covering pretty briefly and that's snoozing and this feature came directly out of user research the insight was that people need to remember the things they need to do and today you only have two options if you can't deal with the notification right now you can either just miss this notification and count on your memory which if you're me never works ask my wife or you can leave the notification in the shade but it can get lost and it's a ton of other notifications right it's it's either or so our opportunity was to give users more ways to do this in the drawer so let's say you're in a meeting and you get a text from a friend asking you about dinner now you're in the meeting so you can't reply right away but you know as soon as you leave the meeting your mind's already going to be on to the next task and you're not going to remember to reply so instead of letting the notification linger in the shade linger in the drawer or dismissing it all together you can snooze it and this is how you can do a partial swipe which reveals two different icons you'll notice it's an additional icon to the gear that we introduce an n and this is the button for snooze so if you tap this icon the notification will automatically be snooze and you'll get a confirmation of this snooze with an option to undo it if you need to so snoozing the notification will set it to come back in that set duration with the exact same interrupted qualities that are sent with as before so if it made a vibration tone when it came in the first time and you snooze it it'll vibrate again the second time when it comes back so if you want a different time than one hour you can just tap that arrow and you'll get a few different options to change that duration and the option that you select then will then be set as the default duration the next time you snooze a notification pretty simple that's what we have to show you for the new notifications in oh and if you're interested in finding out even more especially from a more detailed engineering perspective there's an earlier talk what's new and notifications launcher icons and shortcuts that we'd encourage you to check out we're really excited about these improvements and believe that these features are not only beneficial to users but will also increase the quality of engagement their apps thanks so much and we have a few minutes left for questions how do you expect inheritance for existing hello i'm timothy jordan and this is your update about the coolest developer news from google in the last week the advanced android app development online course has been updated improved and extended with it you can build a portfolio of apps as you improve your android dev skills the course is linked from the post in the description below we recently launched a i y projects do it yourself artificial intelligence for makers with it makers can use artificial intelligence to make human-to-machine interaction more like human-to-human interactions we'll be releasing a series of reference kits starting with voice recognition more details and links are on the post chrome 59 beta is now available with headless chromium native notifications on mac os service worker navigation preload and more all the details are on the post google cloud launcher has more google maintained containers including kassandra elastic search jenkins mysql and more google container solutions are managed by google engineers and since we're maintaining the images the containers available on google cloud launcher will be current with the latest application and security updates there are two announcements from google cloud next london that i wanted to tell you about first google cloud natural language api is adding support for new languages and entity sentiment analysis and second cloud spanner is now generally available check out the details of both announcements on the post google hyo is just around the corner and if you're like me you like to go in prepared which is why we have an android ios and web app to help you customize your ios schedule and get around the developer festival check out the screenshots and find the download links on the post in the description below please subscribe and share i'm timothy jordan and i'll see you next week as a lot of developers know there's more to having an app succeed than just building a great app you want your app to be dynamic and responsive by delivering fresh content to users and quickly reacting to their changing needs you want to test out major decisions to make sure you're doing the right thing before you push them to your entire audience and ideally you want to provide a tailored experience for each user so your vip's feel like well vip's but let's be honest that can be a lot of work and if you're a developer without a ton of resources that's time you'd rather spend on other things like building your app that's where firebase remote config comes in firebase remote config is a simple key value store that lives in the cloud but don't let that simplicity fool you because it lives in the cloud it means you're able to deploy changes that your app can read within a matter of minutes for instance say you've just pushed your app out to the world and you suddenly discover that your swedish text contains some offensive language how are you supposed to know you don't speak swedish i don't blame you but fixing that text the old-fashioned way would mean creating a new build and going through the entire publishing process again that's something that could take days which is an awfully long time to have 9.2 million people cursing your name but if your app uses firebase remote config you could change that text in the cloud through the firebase console kind of like this the next time your users fire up their app remote config will grab the latest values update your app's text and just like that you've averted a major international crisis or let's say you've got a puzzle game and you're hearing complaints from your players that level five is too hard if you've configured your app using remote config you could tweak those settings to give your players a few more turns and push out that change to the world but hang on are you sure that's the right thing to do what if the silent majority of your users actually enjoy the challenge of a more difficult level and by making it easier you're going to turn away your most hardcore and potentially highest paying customers how could you test whether or not this change is a good one sounds like you need an ab test that's where remote configs audience segmentation feature comes in this allows you to deliver different configurations to different groups of users simultaneously so you can try out your new level settings with half your users while keeping the old settings with the other half but audience segmentation isn't just great for ab testing maybe you've got a feature change that could have a major impact on your in-app economy or maybe you just want to double check that some new networking code isn't going to set your servers on fire you can use firebase remote config to gradually roll out these changes trying them first with a small percentage of your users before pushing them out to your entire audience remote config can also deliver different configuration sets to your users based on all sorts of different factors from device type or locale to any audience segment you've defined in firebase analytics so you can send out one welcome message to your new zealand customers and another to your australian ones or only show your review this app button to people who use your app every day or you can change your home screen experience for your customers who have spent large amounts of money on in-app purchases so they feel special remote config is backed by a client library on ios and android that handles important tasks like caching dealing with flaky connections and keeping network requests lightweight which is always a good thing to give remote config a try check out our documentation here we can't wait to see what you build and with npr one we are reimagining what a listening experience could be outside of the radio it's the radio but better it has all of the great stuff that we've spent 40 years perfecting with npr one we see the opportunity of reaching a more diverse audience that have a device in their pocket all the time my name is might safe lahi and the lead mobile developer for npr digital media my name is nick de pare and i'm the innovation accountant and npr my name is tejas mystery and i'm the senior product manager of npr one there's some of the biggest challenges in any mobile app are that first impression when the user first installs the app you've got a very limited amount of time to convince them to keep the app and to get engaged in the experience trying to figure out how we can get users into the content as quickly as possible was the real focus of integrating firebase and dynamic links using dynamic links we are able to shorten the number of interactions it takes for a user installing the app to get from the promoted content to the content from 20 to 3 so that user is able to get right into the content we're driving more and more listening for a user every week it's really astounding creating playlists of content that are configured by the podcaster or by a member station or by us internally and with firebase we have that at our hands having the analytics product interact with things like dynamic links remote configuration cloud messaging it adds a real multiplier effect and the integration with the broader firebase suite i don't have to go outside the platform to figure out what's working so it's not just about shipping the product faster it's about analyzing the results faster and with the integration with all the other firebase products we're really excited about all the things we can learn from it we all know from experience that people love to share things about themselves such as photos videos and gifs that express their feelings so what do you do to let them store and share these files through your app that's where firebase storage can help our storage api lets you upload your users files to our cloud so they can be shared with anyone else and if you have specific rules for sharing files with certain users you can protect this content for users logged in with firebase authentication security of course is our first concern all transfers are performed over a secure connection also all transfers with our api are robust and will automatically resume in case the connection is broken this is essential for transferring large files over slow or unreliable mobile connections and finally our storage backed by google cloud storage scales to petabytes that's billions of photos to meet your app's needs so you will never be out of space when you need it so give your users space to share their lives with firebase storage available right now for ios android and web applications and to learn more about firebase storage check out the documentation available right here right let's have a little chat about a brand new api that's just coming out on the platform called the media session api um i heard about it when i started doing this app it was mentioned to me by a couple of the chrome engineers but france war who's on my team there's a brilliant google web developers update where he explains in complete detail about how to set up the media sessions api so check the notes below we're going to pop in a link for that but i will just show you briefly what it actually looks like in the context of the app on the phone here in fact because you probably can't see it let's switch across to the direct scream cam there you go and that's what it looks like on the phone so as i start playing a video like this you see that if i swipe down from the top we actually get a notification which has an icon here and it has play and rewind and fast forward buttons and you get to configure those yourself in fact let me go into another one of these videos where i think i've actually set up to load some custom album art as well so there's me and jake and you see here now we've got custom album art which is the picture of jake and the previous and next the fast forward and rewind buttons are actually set to be skip forward 30 and go back 30 so you'll be able to tap that and go forward 30 which you can see there oh it's just skipped it right to the end whoops a daisy but i can replay the video don't worry the other thing that it actually does which i really like is if i turn off the screen then on again you can actually see that the picture the album art is actually the picture for my phone in the background that is very exciting isn't it so let me show you a little bit as well since we're here let me show you a little bit of the code uh it's very straightforward we have a quick check whether we support the media session api uh which is basically looking for let me show you actually it is just simply looking for media session in navigator and if we have media session in navigator then we consider ourselves having the media session api and what we do is we say media navigator dot media session dot metadata and then we create one of these new media metadata uh thingamajigs very exciting uh eslyn doesn't like it it doesn't think it's a real thing it is a real thing uh you can totally use it and you give it things like the title the album the artwork i've only set the 512 and the 256 but very much like your manifest files for progressive web apps you can set as many of these as you need and the user agent will