more about immersive mode. What does that mean? Immersive mode is a term that's just been borrowed from native, it's a full screen mode and basically it removes all of the operating system decoration, so no status bar, no navigation bar and this is great for games or other media, basically when you wanna be able to address every single pixel on the screen. So I could start a video player in full screen from Dicon at home screen, nice. And what about app shortcuts? Sure, app shortcuts are a way to provide quick access to important functions in the app directly from the app icon. So for example, on Android you might be familiar with a long press on the app icon on the launch screen or on the home screen, so if you were to long press on a home screen icon for say a mail application you might see compose functionality directly in the menu that appears when you long press on the mail client. App shortcuts also work on desktop operating systems and that support will be arriving in Chrome 85. Interesting, so that's like deep linking to parts of my PWA directly from the icon that's on the home screen. Exactly. Switching gears a little bit, we launched Trusted Web Activity last year's IO and since then we have many feature requests and questions from developers and I wanted to go through some of those with you today. First, developers have pointed out that the way permissions work in the browser and in native is different and that makes their users a little bit confused. As an example, native apps get the notification permission by default while web apps need the users to accept the permission first. How are we planning to solve those inconsistencies? So let us just start by just sharing my perspective on the philosophy here of I don't think that users should need to think about what technology was used to build the application that they're using. Users should really just have a job, users just have a job to get done and we want to help developers help users as easily as possible. So wherever it makes sense, I think an installed web application should use the operating systems typical UI for things like managing permissions, launching and switching between applications just to match the user's expectations for how things should behave on the device that they're using. So we've introduced this concept of notification delegation into installed PWAs and that means that when a PWAs installed, it will delegate the notification permission into the native setting area on Android. So another difference between an app installed from Play and a web app, for example, is that an app installed from Play automatically receives notification permission. So we want that experience to be the same for Android applications that were built using a PWA. And that's why we've delegated the web notification permission to the notification settings panel in Android and these apps can be configured to auto enroll users and notifications so that they just look and feel and behave exactly like any other Android app and the user doesn't even need to know that this application was built using web technology. In the future, we're going to be adding locations of the settings panel too. Of course, there won't be any auto enrollment for location because users are not auto enrolled in location for location permission on in Android native apps either. So we're just going to match what the behavior is between for a native application, for apps that are installed from say the Play Store or from any other distribution channel where the user may or may not be aware that this application is a web application. We're going to continue as well to delegate more permissions and match OS preferences over the next few releases. Got it, so this will make the experience more seamless to users regardless of the technology that the developer used to build the app. Another frequent request, developers sometimes feel that when they make an Android app using PWA and tracer web activity, they should have a communication channel between the native application and the web app. This would allow them to use native platform capabilities where an equivalent on the web doesn't exist already. Is this something that is being considered? I'm really glad you asked this question because this is the exactly the kind of product question that I really love. And I'd like to hear from our audience on this one. So today you can pass parameters into a tracer web activity when you launch it and you can use intents to leave the tracer web activity and pass some information into another activity inside of your app. And we're considering adding support for a message bus. For example, we could extend post message to enable a bus with this functionality. However, I don't have use cases from developers on exactly what they need here. Most capabilities are already in the web platform or are planned as part of the Fugu effort to add web capabilities to the web platform. So if the web platform has missing capabilities, I'd really rather add that capability to the web then create a message bus to native because if a capability is part of the web platform then it's gonna work everywhere and developers only need to have one code base, which is simpler and that code base will work in multiple browsers, whether the app is installed or not installed, et cetera. So I'd like to turn this just to do a question for our audience. What do you need to do in native code that you can't do today in the web platform and are there things we could do to improve the web platform so you wouldn't need to do that in native or perhaps it's something that could only be done in native and you really need that message bus. I'd really like to hear from you about this. Cool, I'm also looking forward to hear from folks on Twitter or if you're watching this live on the live chat. Many developers have a native application and prompting users to install a PWA in the browser when they already have the native app installed can lead some confusion. Is there a way to prevent the prompts from showing when the native app is already installed? So that's a great question. It's also probably one of the top concerns that I hear from teams that are implementing a progressive web app. It's called channel conflict and it arises where you're not sure which experience is gonna be best for the user. So I think the most important thing for developers to know is that you do have full control over the promotion of PWA install. So you don't need to worry about the browser promoting your PWA if you don't want it to. There are ways you can prevent the browser from promoting the installation of your PWA if, for example, you have a native app. So let's talk about how that works. First, in the web app manifest, there's a couple of fields you should know about. One is called the related applications field and this is where you can list native apps for Android and iOS. And then there's another Boolean field called prefer related apps. And if this is set to true, the browser is not gonna promote the install of your PWA. Secondly, there's an event that fires when a PWA passes the installability check in the browser and when this happens, developers can call a prevent default method and that's gonna block any promotion of the PWA install in browser UI. Finally, there's a new JavaScript API just landed earlier this year called get installed related apps and this is gonna let developers inspect if the user has native apps installed on the device that are associated with the origin that the user is currently on. And just to be clear, this isn't gonna let you see any app that the user happens to have installed on the device. The app has to be associated with the origin that the user is on. This API does allow for a lot of programmatic flexibility for developers. So it means you have full control. Which experience do you wanna promote to the user? You could use, for example, user behavior. You could provide the user with preferences in your HTML. It's really up to you how you wanna use this control but it does give you, as the developer, a lot of control over what you promote to the user and when it happens. So this means I can use this API to check if my data application is installed and make decisions like showing that sound screen prompt or not. What if my native app is using trusted web activity? Will these APIs to return? So yes, absolutely. This does mean that you can use the API to check if your native app is installed. A trusted web activity is really just an Android app. So it will show up just like any other Android app and this API will return it. It will also return if the app is a, it has been installed from the web browser. So a PWA installed from the web browser will also get returned if I get installed related apps. Cool. On the developer experience side, we've been working on tools like bubble wrap to help developers build their project using trusted web activity. But many developers as you wonder if it wouldn't be easier if they could just copy paste a new role into the Play Store. That's a great question. So the reason why we've been focusing our efforts on making this easier for developers on bubble wrap is that we could create a much more powerful, much more flexible system for developer, building Android apps using a command line utility like bubble wrap than what we could do in a web interface. So app stores are really a different ecosystem and they have different requirements and policies. And we believe that developers should have powerful tools that they need to rethink the experience of their applications for this environment. And we also want to avoid giving developers a perception that they could just drop a website into the store without thinking about that experience. So there are all kinds of design decisions that should go into building an Android app, whether or not it's a PWA powering that app. For example, what's the splash screen gonna look like? When are you gonna hide it? When are you gonna show content? Do you need notifications? There wouldn't be this kind of flexibility for configuring these options with a web interface. That being said, we want it to be as easy as possible. So we've worked to streamline it to the maximum extent that we can with bubble wrap and make it a powerful but flexible and easy to use developer tool. What about place or policies? Do Android apps that use PWA inside a tricetable activity still need to comply to those policies? I think you said the magic word there, which is Android app. It doesn't matter how your Android app is built, whether it's built using PWAs and web technology or whether it's built using Java or Kotlin. Play policies apply to all Android apps distributed in the Play Store and therefore they also apply to Android apps built using progressive web apps. Gotcha, so same store policies apply. What about applications designed for children? Can developers use web technology in those apps? If the target audience for your application is children, you need to comply with the Play Family Policy requirements and these requirements are intended to help keep miners safe from inappropriate content. Unfortunately, that's really difficult for the review teams to evaluate with web apps where the content can change and not only can the first-party content change but it's really easy to include third-party content inside of a web application which can be unintentionally inappropriate. Let's imagine that you are using an advertising network or something else where you're loading content in from a third-party site. You might not be able to verify yourself for sure whether or not that content is always going to be appropriate. So for the time being, it's not possible to build Android apps using trusted web activity that comply with Play Family Policy. We are working on ways to make this possible in the future. Got it, so it seems PWAs are crossing over ecosystems and developers need to adjust some of their expectations from that. Developers using trusted web activities are also expected to fulfill a quality criteria. Is that why that criteria exists? Yeah, nobody wants an app store that's going to be cluttered with low-quality apps. And we also want developers to succeed with their apps in the Play Store. Keep in mind something that's really different about distributing through the Play Store than it is to just sort of build a PWA on your own site is that these apps have user ratings and reviews. And we want to make sure that developers are set up for success to get good ratings and reviews for their app. So at a minimum, users expect apps that they install from the Play Store to look and feel app-like, to be fast, to work offline. And that's why we have quality criteria for PWAs in the app store. And this is exactly what progressive web app criteria has been intended to do from the very beginning. So first I want to share that there are certain types of events that can happen to a web application that are effectively a crash. And these are things like a 404 happening. These are things like failing an offline check when the user goes offline and returning to Chrome Dino. This is effectively a crash. Failing the digital asset links verification, which is something you need to do with a trusted web activity to verify that you are the owner of the content and the owner of the application in the app store. So if any of these things happen, starting in Chrome 86 in October, we are going to be mapping those crashes into Android vitals crash events. And you will see Android apps begin to crash if users are running into 404s in your application, et cetera. So that's just something that developers should be aware of and we're making another announcement with more details about this. The second thing is on the performance side. Right now, don't have a date to announce with respect to enforcement of performance, but developers should be aware that the criteria is a full batch PWA and a light has performance score of 80 or better. And you can use webpagetest.org slash easy or PageSpeed Insights against your start URL as the fastest way to check whether or not you're meeting this criteria because having the app launch quickly is an essential part of having a great application experience for your users. We will be providing a lot of notice as to when enforcement will begin. It will be in 2021, it won't be in 2020 for the performance and full batch PWA scores. So expect to hear more about this later in the year. So last one. Twitter has recently launched their PWA on the ChromeOS Play Store, which was quite exciting. Can you tell us more about how they did it and when this is becoming generally available? I'm really excited about this. Right now, this is an early access program and it requires manual intervention from our team members. So it's not something that we can extend to everyone in the community, but we're working on getting it to general availability and when that rolls out, it'll be possible for everyone to just do this themselves. I'll give you a hint, it uses trusted web activity. So if you're building a great trusted web activity, it'll be really easy to make your progressive web app available in the ChromeOS Play Store in the future. And I hope we can share more about this in the second half of the year. So wow, trusted web activities are coming to ChromeOS, nice. Well, I think we covered a lot. I wish we had more time to discuss. I do too. So for those of you who are watching, please do reach out to us on Twitter if you wanna give feedback. And if you're watching this live, please join us on the live chat. We'll see you there and thanks so much for watching. Yes, thanks for watching. Hi everyone, thanks for joining. I'm PJ, I'm a product manager on the Chrome web platform team responsible for progressive web apps, notifications and permissions. Today's talk is about quieter notification permission prompts and how recent changes to how Chrome handles notification permission requests can make browsing for the web a little better for everyone. There's also important information in here for developers who use notifications to help you improve your user experience, improve your notification accept rates, and to tell you about upcoming changes that will detect and flag abuse abuse of notification prompts or content. If you're not familiar with them, web notifications are a channel for communicating timely and contextually relevant information to the user. Mostly these work just like push notifications in mobile apps, except they can also work on desktop, on Windows, Mac, as well as smartphones. On Android, for example, web notifications appear in the notification drawer and on desktop, they typically appear in the top right corner of the screen. In some cases, notifications aren't just helpful. They are almost essential to the app's functionality. For example, if you had an incoming call from a communication app like Google Duo or Chat, that's not something you want to know about later. You need to know about it right away. Of course, not everyone uses apps that require notifications and not all websites are putting the needs of their users first. That means that we are seeing a lot of websites out there that are misusing notifications in ways that are annoying or could be abusive. Before we get into that though, I want to talk about how users get enrolled in notifications. To receive web notifications, a user needs to accept a notification permission request. When websites prompt users out of context for a notification, such as when a user first arrives on the website, it can be a pretty annoying distraction both from the browsing experience itself as well as from the website's content. Worse, some abusive websites look for ways to trick users into accepting notifications that are then used to promote malware or undesired content. I want to cover why we have notifications in the web platform in the first place in a little bit more depth. The web platform is there to enable web developers to create powerful applications and web notifications are part of that toolkit. Without notification support, there would be entire types of apps that would be simply impossible to build using web technology. So for example, messaging apps, calendars, e-commerce for food delivery notifiers, taxier ride sharing apps, all dependent notifications to provide a timely tap on the shoulder to the user. And while you can imagine that some of these apps might be usable without notifications, you could see that most of the time you're probably gonna want one. We've also all equally experienced some of the misuse of notifications though. That includes things like unwanted marketing, promotions or content that just isn't very important or relevant to us at a given moment. To address this problem starting in Chrome 80 that was released in January of 2020, we started making changes to how these notifications requests work to help make browsing the web safer and less interruptive. We're gonna get into that new UI in the next slide. In Chrome 80, we added quiet notifications UI. Quiet notification UI is less interruptive, but it still lets the users know that the request has been made. There's a little bit of animation to catch the eye, but on desktop, the dialogue is in the omni box. So it doesn't actually cover any part of the web content. On mobile, the notification prompt used to be a modal in normal UI, but in quiet UI, it's an easily dismissed info bar at the bottom of the screen. Quiet notification UI aims to reduce the visual priority and interruptiveness of notification requests. On desktop, which you're seeing in this example on the left, you'll notice the bell icon initially animates with text, indicating that notifications are blocked on the site. In mobile, the quiet UI is now an info bar. In both of these cases, in product help explains to the user why notifications were blocked on the site. Quiet notifications UI was created specifically to address the concerns I mentioned earlier in this talk. We receive frequent complaints from users in Chrome feedback about disruptive notification permission prompts or unwanted notifications. That being said, there are services with tens or hundreds of millions of users like messaging apps and calendars that are depending on timely web notifications every single day. Let's talk for a moment about how users get enrolled in quiet notification UI. There's several ways that this can happen. First, users could just enroll themselves manually by changing their preferences in Chrome settings. Second, sites that have very low accept rates for notification permission requests will be automatically enrolled in quiet notifications UI. And this is currently sites that have the lowest few percentile of notification accept rates. So the absolute rate needed for quiet notification UI does change over time because we are using percentiles. We'll also periodically increase the accept rate percentile that's needed to preserve normal notification UI. We always keep a control group of users that are in normal notification permission UI so that if a site's accept rates improve, we can remove it from quiet UI enforcement. Third, there are some users who almost never accept any notification permission request. These users simply don't want notifications. And for these users, we adaptively enable quiet notification mode on their behalf in Chrome settings if they repeatedly block notification requests. As sites improve their behavior and use of notification permission requests, we expect that there will be fewer and fewer users who are adaptively placed in quiet notification UI mode. Finally, and this is starting in Chrome 84, which is coming up soon, we're gonna begin enforcing against abusive notification prompts that try to mislead users or phishing for private information or promoting malware. In this case, in addition to quiet notifications UI, the user is going to be advised in the notification prompt that the site may be trying to trick them. So what should you do to make sure your website is not enrolled in quiet notification UI? Well, first and foremost, if you're prompting users to enroll notifications as soon as they arrive on your website, please stop doing that. This is the easiest way to improve your notification accept rate. Very few users will accept a notification from a site they're visiting for the first time. And if you think about it, why would they? We're all experiencing information overload. Wait until you know your user better and you know you can add value for your user before you prompt them. You can and should prompt your user to accept notification when there's a clear user benefit and in the context of the user's journey in your application, websites that ask for notification permission in the context where the benefit is clear to the user have 80% accept rates or higher. That should be your goal. Even if you do the best possible job with your notification prompt UX, it's possible that some of your users may be in quiet notification UI mode. The first thing you wanna check here is to make sure that the accept rates on your site are what you really expect them to be. Notification accept rate data is in the Chrome User Experience Report, which is a public database containing important information about real world Chrome metrics for popular destinations on the web. There's a minimum number of users and decisions that are required for data to be available in the Chrome User Experience Report and that's to help with preserving visitor privacy. So if your site doesn't have data in the Chrome User Experience Report, you may need to get that information from somewhere else. For example, most notification service providers will have this instrumented so that you can check your accept rates or if you've rolled your own notification implementation, you may need to add your own instrumentation. It's also a good idea to look at the notification accept rates of sites that are like yours in the Chrome User Experience Report so you can get a sense for what the benchmark accept rates are and aim to be better than those. The article linked here in the slide will give you more details about how to use the Chrome User Experience Data Set to learn more about user's notification permission prompt decisions on your website. So the second thing you need to think about is how can you make your notification requests more in context? I know I mentioned this before, but it bears repeating. You wanna make absolutely sure that you're asking for notification permission at a moment in the user's journey that makes sense to them. In this example, we're showing a notification request the first time the user receives a response to their first chat. This is a perfect moment. Even with quiet notification UI, it should be obvious to the user why they would want to accept notifications. The activity and motion in the web app combined with the motion of the browser prompt should be sufficient cues for the user to enroll in notifications if they want them. If your user doesn't accept notifications with this in context pattern, there's a pretty good chance that they just don't want notifications and you should respect that decision. Before I finish, I wanna share a little bit about what's coming next for notifications. First, we're planning to increase the accept rate percentile that's needed to have normal notification prompts. Since this is a percentile as well, something to keep in mind is that if other sites are improving their notification UX and you're not, your site may be slipping into a lower percentile and quiet notifications may be activated for your site. If your site has a notification accept rate of over 50%, you're in safe territory, but we recommend aiming for 80% or better. Second, Chrome places a high priority on user privacy and security and we intend to take more steps to protect users from abusive notification in the future. That includes more protections from abusive notification content as well as retroactive action to help users who may have already enrolled in notifications from abusive sites prior to the release of Chrome 84. Most important, as we improve the signal to noise ratio of the web notification ecosystem, we hope users will come to view notifications as being more helpful. If this happens, it means we're doing a good job protecting users from unwanted notification prompts and unwanted notification content. Ultimately, this will help developers who use notifications for key functionality in their apps as users are more likely to accept notifications when they have less reason to be worried about spammy or abusive notifications. Thanks again for joining today, everyone. Have a great day. True or false, IndexedDB is limited to 25 megs. False, gone are the days of tiny storage quotas. True or false, local storage should be avoided. True, it's synchronous and may cause performance issues by blocking the main thread. All right, here's another one. True or false, cookies are a great way to store data. False, they've got their uses, but should never be used for storage. How about this one? AppCache is a great way to make your app work offline. Yeah, trick question, absolutely false. AppCache is awful and it's going away soon, thankfully. So how should we be storing data and caching our critical app resources on the client? How much can we store? How does the browser deal with eviction? And be sure to stick around to the end and I'll tell you how you can start Chrome with only a tiny storage limit so you can test what happens when you exceed your storage quota. I'm Pete LaPage. Let's dive in to storage on the web. Modern storage makes it possible to store more than just small chunks of data on the user's device. Even in perfect wireless environments, caching and other storage techniques can substantially improve performance, reliability and most importantly, the user experience. With the Cache Storage API, you can cache your static app resources like HTML, JavaScript, CSS, ensuring that they're always instantly available. And with IndexedDB, you can store all kinds of data, article content, users documents, settings and more. IndexedDB and the Cache Storage API are supported in every modern browser. They're both asynchronous and will not block the main thread. They're accessible from the window object, web workers and service workers, making it easy to use them anywhere in your code. There are several other storage mechanisms that are available in the browser, but they've got limited use and may cause significant performance issues. If you're concerned about storing large amounts of data on the client, don't be. Unless you're trying to store several gigs, modern browsers typically won't even bat an eye. And even then, it really comes down to the amount of disk space available on the device. Of course, implementations vary by browsers. Firefox allows an origin to store up to two gigs. Safari allows an origin to use up to one gig and when that limit is reached, Safari is currently the only browser that'll prompt the user to increase that limit. And Chrome, well, look, it's a little complex, but stick with me here. Chrome and most other Chromium-based browsers limit storage to 80% of the total disk space and each origin can only use 75% of that. For example, if you had a 10 gig hard disk, Chrome would limit its storage to eight gigs. Then each origin would be limited to six gigs. Essentially, each origin will be allowed to use up to 60% of the total disk space. It sounds complex, but there's an easy way to see what's available. In many browsers, you can use the Storage Manager API to determine the amount of storage that's available to the origin and how much storage that you're already using. It reports the total number of bytes used and makes it possible to calculate the approximate bytes remaining. Unfortunately, the Storage Manager API isn't implemented in all browsers yet, so you must use feature detection before using it. But even when it is available, you still need to catch over quota errors. In some cases, and I'm looking at you, Chrome, it's possible for the available quota to exceed the actual amount of storage available. Most Chromium-based browsers factor in the amount of free space when reporting the available quota. Chrome does not, though, and it will always report 60% of the actual disk size. This helps to reduce the ability to determine the size of stored cross-origin resources. So what should you do when you go over quota? Most importantly, you should always catch and handle right errors, whether it's a quota exceeded error or something else. Then, depending on your app design, decide how to handle it. For example, delete content that hasn't been accessed in a long time or remove data based on its size or provide a way for users to choose what they want to delete. Both IndexedDB and the Cache API throw a DOM error named QuotaExceededError when you've exceeded the quota available. For IndexedDB, the transactions on a board handler will be called passing an event. That event will include a DOM exception in the error property, and if you check the name of the error, it'll return QuotaExceededError. For the Cache API, writes will reject with a QuotaExceededError DOM exception. Data stored in the browser can be cleared in a couple of ways. It's most commonly initiated by the user choosing to clear data in the browser's site setting panel. But it can also happen when faced with storage pressure like low disk space. In this case, browsers typically automatically delete data from the least recently used origins and continue to delete that until the storage pressure has been relieved. If the app hasn't synced data with the server, it will cause data loss and means that the app won't have the resources needed to run, both of which can lead to a negative user experience. Thankfully, research by the Chrome team shows that this doesn't happen very often and it's far more common for users to manually clear storage. Thus, if a user visits your site often, the chances are small that data will be deleted. Let's take a look at a specific example of how automatic eviction might happen in Chrome. Origin A is the least recently visited site. Origin B is the next least recently visited site and so on. Origin E and Origin K are getting close to their quota limits, but they haven't reached it yet and the overall usage is less than the total quota, so nothing is gonna be evicted. Origin B has a star next to it because it was granted persistent storage, meaning that it can only be deleted by the user. Check out my article on web.dev for more info about persistent storage, when you should be using it, and how to request it. Now, let's say the user visits Origin N again, which happens to be a music playing site. The user saves a few more songs for offline listening. Now, each origin is still within its quota limit, but Chrome has exceeded the overall limit. To get back under the overall limit, Chrome will start evicting stored data from the least recently used origin first and continue until it's back under the total limit. Firefox and other Chromium-based browsers work in essentially the same way. Safari is a little different. When it's out of storage, it will prevent anything new from being written, but they recently implemented a new seven-day cap on all writable storage, including IndexedDB, Service Worker Registrations, and the Cache API. This means that after using Safari for seven days and not interacting with the site, it will evict all content for that site. This eviction policy does not apply to progressive web apps that have been added to the home screen, essentially installed PWAs. Check out the WebKit blog linked in the description for complete details. Modern computers typically have large hard drives, which makes it hard to test the over quota failures. So here's a little pro tip. Create a small RAM disk. Here, I've created a 500 meg RAM disk on my Mac. Then, start Chrome using the user data dir flag. This tells Chrome to store the user profile and user data on the RAM disk. Chrome now thinks my disk is only 500 megs, thus is gonna limit my storage quota to only 300 megs, which I can quickly fill. This makes it much easier to verify that my code behaves properly when it hits those quota exceeded errors. Chrome DevTools also have helpful features for understanding what's going on with the data that you've stored. In the application panel, the clear storage panel will show you how much storage you're using for the origin and makes it easy to clear some or all of that data that you've got stored. The storage panel lets you see what's in local and session storage, as well as what's in IndexedDB, including the actual databases and even the individual entries. And the cache storage panel shows you what's stored in cache storage. Gone are the days of limited storage and prompting the user to store more and more data. Using the cache storage API and IndexedDB, you can effectively store all the resources that your app needs to run. Be sure to check out my article, Storage for the Web, where I've got additional details and info on some of the not-so-good storage mechanisms. Then, check out my article on persistent storage to learn how you can protect your data from being blown away, even when the device is facing storage pressure. See you soon. Hi, everyone. My name is Thomas, and today I want to talk to you about the explorations that we've been doing with the Zoom team over the past few months and some of the specific advanced APIs that we've been exploring together. As you've all probably seen, Zoom has become a stable in many homes, and so it's critical that we're able to provide a good experience directly through the browser. Zoom does have a web version today, but compared to its native client, it's missing some features and sometimes misses the mark in performance. We wanted to change this, and so even before COVID hit, we started working with the Zoom team to understand exactly what changes they would need and what new things in Chrome they would want to create a truly great experience. Now, Zoom is, of course, a video conferencing application. You can't talk about video conferencing on the web without talking about WebRTC. WebRTC is a really great full stack solution that provides a complete package for achieving video conferencing on the web. WebRTC was built and standardized about 10 years ago and now ships in all major browsers. This makes it the best choice if you want a complete solution with broad support across browsers.  Zoom would rather want a set of more simple, low-level APIs that they can then build their own architecture and system on top of themselves. And the three specific ones that we've been exploring and that I want to talk to you a little bit about today are WebAssembly SIMD, WebTransport, and WebCodex. I'll mention from the start that all of these are fairly cutting edge and most of them are in active development. So while they're all in a place where you can start to play with them, these aren't shipping APIs just yet. But hopefully this presentation will cover some of the early parts of it. And by the time you're watching this, it might be in the future and you'll be able to actually use these directly and the lock already shipped. So first of all, I want to talk about WebAssembly SIMD and how it can provide really highly performance code. Most of you have probably heard about WebAssembly already, but as a recap, WebAssembly is a new low-level binary format for the web that is compiled from other languages and offers maximized performance. This means that you can take something like C++ or Rust and then compile it into WebAssembly before shipping it to the client. WebAssembly has been out for a while and has been shipping in all major browsers for a while, but we're continuing to expand it with functionalities such as SIMD, which stands for Single Instruction Multiple Data. To explain SIMD, let's look at this incredibly simple loop that just adds two arrays together. Without SIMD, the CPU would go through this loop and add their different elements together one by one, requiring four full steps. But with SIMD, the CPU is able to vectorize these elements and then take just a single CPU operation to add them. The best part is that because compilers are so smart, they can automatically detect these optimizations and do them for you. In Inscriptin, you just need to pass the dash M-SIMD 128 argument to EMCC and for Rust, you can pass dash C target feature equals plus SIMD 128. This will cause the compilers to automatically find and use SIMD where possible. Sometimes you also wanna have more explicit control and this is where you'll wanna use SIMD Intrinsics, which let you use the SIMD instructions directly. This is more detailed than I can cover here, but if you're interested, I highly encourage you to go and check out these links. SIMD can be used for a huge variety of things, including highly performant ML models such as this hand tracking, a real life invisibility cloak, and real time automated background removal. And this last use case is just one of the things that Zoom is excited about using SIMD for. They have an awesome feature where you're able to automatically remove the background so that people in conferences can't see all the random stuff that you have in your background and then replace it with fun videos or animations. If you're interested in diving into WebAssembly and SIMD, here are some of the links that should help you get started. WebAssembly SIMD is doing an origin trial in Chrome M84, which will start rolling out to users in July 14th. If you aren't familiar with origin trials, it's basically a mechanism for you to test out features with production users while we may still be making some changes to the API. You can read more about those origin trials at this link as well. So the next API I wanna get into is WebTransport, which is a next generation networking API for client to server communication. Let's look at the definition of WebTransport. WebTransport provides bidirectional transport through both unreliable datagrams and reliable streams-based mechanisms. That's a mouthful, but let's see if we can't break it down and understand it a bit better. First, bidirectional means that it enables easy two-way communication. With something like HTTP, the connection has to be initiated by the client and you have to send all of the requests at once and then wait for a response. With WebTransport, you don't have these limitations and so you can enable a much more interactive session. Looking at the two different mechanisms, unreliable datagrams are one of the mechanisms for sending data through WebTransport. These datagrams are similar to UDP datagrams in that they are packets of information that gets sent, but with no guarantees about delivery or ordering. Reliable streams, in contrast, are similar to TCP streams and provide reliable and ordered data communication. So now that we have an understanding of the definition of WebTransport, let's understand what you might actually use it for. Firstly, WebTransport will be the only mechanism to do unreliable data communication without leveraging WebRTC. And this is exactly why Zoom is interested in looking into WebTransport because it'll allow them to simplify their deployment and put it a little more in line with the other platforms that they support. It's important to note, though, that WebTransport won't be just a pure UDP sockets API since it does have some requirements around encryption and congestion control. It does offer an alternative to WebSockets and to understand exactly how it compares to WebSockets and WebRTC, let's look at this chart. So to understand the differences, let's dig into each of these pieces. First, WebTransport and WebRTC offer both reliable and unreliable while WebSockets only offer reliable delivery. WebTransport is an in-development API while both WebSockets and WebRTC are widely available. While WebRTC provides a fairly high-level complete solution to the problem of video conferencing, WebTransport and WebSockets are both much lower-level APIs that doesn't solve everything for you but gives you more of that basic access. WebTransport also enables multiple cancelable streams whereas WebSockets can only do a single stream and WebRTC can also do multiple streams but they aren't cancelable. So here is a quick example setup for how you can actually use WebTransport. In this part of the code, we really just set up our new QuickTransport which is specific subtype of WebTransport and create that object passing in the URL that we wanna connect to. Then we just set up some simple logging function and await the transport being ready for us to use. Then we can simply grab the writer from the sendDataGrams function of our transport object which we can then use to send data at any point. Remember that this data that we send does not have any guarantees of delivery or the order that it will be delivered in. Next, let's look at how you can actually read data from the server. Here you see a simple example where we get the reader from the getReader function and then in a classic whileTrue loop we just read things from that reader and then detect them when we're done and console log out the actual values that we're able to read. WebTransport is still very much in development but we do have a blog post already published about how you can use it and you can find that and more information at these various links. So now it's time for us to jump in to our last and exciting API, the WebCodex API which aims to offer direct codec access on the web. But first let's back up and remind ourselves what exactly a codec is. A codec is a device or computer program which encodes and decodes a digital stream or signal. While many of us have not worked directly with codecs we've all seen common examples like mp3, vp9, h264 and many others. Codecs are actually used in tons of places throughout Chrome such as the audio and video tags, web audio, webRTC and the media recorder API. However, in all of these places where it's used you can't really configure and get pure access to just the codec part. For example, web audio allows for decoding a media file but needs to work on the complete file and doesn't support a streaming based approach. Media recorder has some controls but they're very high level and you can't really configure it to support extremely low latency use cases. As mentioned previously webRTC does give you a lot of this control but it needs you to bring the whole package of webRTC along and without doing that it's hard to get access to just the encoding and decoding parts that you want. As a result of this lack of configuration some apps have started compiling these codecs to JavaScript and WebAssembly. Some of you may remember that this is how the awesome application Squoosh lets you resize and re-encode images. This approach is really cool and workable today but has some specific drawbacks. Specifically it increases your bundle size, lowers the performance, causes slower startup time and reduces the power efficiency. Really what you want is to avoid shipping these codecs all together and just get the direct access that you need through the codecs that are already shipping as part of the browser. And that's exactly what the goal of WebCodecs is and in their own words the goal of WebCodecs is to provide web apps with efficient access to built-in both software and hardware, media encoders and decoders for encoding and decoding media. WebCodecs main advantage is that it lets you get the direct access that you need to again build your own systems on top of the basic codec access. This completely unlocks some use cases like video editing since you really need that frame-by-frame access and faster than real-time encoding and decoding to do this properly, something that's currently completely impossible on the web platform, except for maybe shipping codecs with WebAssembly. Additionally, many existing things that are possible to stay on the web but only if you use WebRTC, things like cloud gaming, live streaming and video conferencing will get more flexibility about how they can interact with these codecs. Zoom for example is looking into using this API in conjunction with the WebTransport API. They're hoping that they'll be able to take encoded video frames and send them up to the server using WebTransport at the same time that they'll be fetching down encoded frames and then decoding them to show to the client, providing a really smooth integrated experience. Next, let's look at some of the simple examples for how you can use the decoder part of this. Here in this canvas setup part, we're really just grabbing a canvas's context and then making this very simple function to paint a video frame to that canvas from converting it to an image bitmap. Now, when you want to set up the decoder part, you just call this new video decoder element and you set up the output function that we defined previously, as well as just constant logging out any errors. Then you configure it with the codec that you want to use and then you have this incredibly simple function that you just pass in your encoded chunk and call the decode function from your video decoder and then it does the rest of the work for you. WebCodex is still extremely new, but for those of you who are curious, you can go and check out the explainer to see what the team is currently working on. We will also be doing a web.dev post about WebCodex, so if you're seeing this in the future, be sure to go and check out that. And that brings us to our overview of these three new exciting APIs that we've been exploring with Zoom. You've hopefully gotten a better understanding of some of these new and advanced APIs and hopefully an understanding of how they will be bringing all of us closer together in the future. Thank you so much for your time and I hope you enjoy the rest of the sessions. Goodbye.