I am Anshal Mathur and I am working as a lead engineer in the Test Automation Department at QA in Kotec, Noida. My topic for the day is an automated item to test accessibility. As the name suggests, I am going to talk about a recent integration that we have made at in-house at our organization regarding executing the accessibility test in an automated fashion. So, it essentially is about an automation framework which is designed in a way to integrate the functional tests alongside the accessibility tests and execute them together at one go. Now in the present era, we are imparting accessibility into a web application as not only seen as a social or a moral obligation but it is also seen as a business opportunity because a lot of specially-abled people are now willing to access the web and consume the web with it. So, it has become all the more important for us being testers to do those accessibility testing in a more formalized and organized manner. At QA in Kotec, we do have a dedicated accessibility testing team, you know, with specialized engineers and our research team have been closely working with them to find out the methodologies that they, you know, follow and of course the reports that they generate, the output of accessibility testing. So, as to understand the requirements and then provide a solution that assists the accessibility testing to be, you know, to happen in a more efficient and less time-consuming scenario. So, the output of that effort is this automation framework that I am going to demonstrate today. So, on this note, let me move on to the agenda item. First of all, I will go over the available approaches to accessibility testing. We will see a few tools which are already being used and the value that they are adding. The problems that these tools possess and of course offer are also, you know, under scrutiny. I will go over those problems one by one and analyze their impact on the actual testing process. Then, those problems will lead us to the solution that I am trying to make, the proposed framework. What I am going to do is I am going to give a working demonstration of the framework that we have built alongside the architectural details of the framework which will help all of us to understand, you know, the framework as a whole from end to end. Then, we will move on to the advantages that the framework has and it has a lot of them. Moving on, I will quickly put forward a case study that I have, you know, we are working with a lot of clients. We have used this framework for them. So, there are some numbers that would assist, you know, my proof of concept. Essentially, I will do a conclusion with some key points and takeaways. Moving on, here are a lot of tools and all of you must be, you know, in one way or the other familiar with these. A couple of examples as you see are Synthiasis and Tenant. So, these are highly, you know, reliable and very popular accessibility testing tools which are, you know, very effective. The effectiveness is to the extent that they get integrated within the developer's code itself. As and when there is a single line of code which is implemented by the developer, the accessibility feedback is given right there and then. But the problem with these tools are they are paid and they are very highly paid. Now, on the other hand, let's talk about a few open source tools which do not have any free, you know, license cost associated with them. A few important ones or I would say popular ones are Wave and X. You must have heard about these two. So, I have a demo, a video basically. I'll show how they work, you know, and we'll try to figure out what the problem comes out to be. Here I have got the Wave video ready. I'll just play the video. Before playing it, let me give you a brief about what the tool is all about. It's an online resource. It's a third-party server. What it does, it takes the web pages URL, you know, scans all the HTML from A to Z, and then provides you the results in the form of all the errors, the accessibility violations. Of course, those violations are against the WCAG 2.0 guidelines, which comes under the 508 compliance of accessibility. And then it also gives you the structural elements and all the alerts that are required. You know, when you're testing accessibility. Let's go over it and see how it works. And then in the later part of the video, I'll show what potential problems this tool has. So, Wave is an open source, a very free open source tool. And X is another one. Yeah, I'm sorry for the delay. So, showing you the web, it's the home page of the Wave tool. Let's go to the website. What I'm trying to do is copy the URL of the Google sign-in page, paste it here, and hit the Go button. You'll very soon see the screen with the left panel that tells you the details about the Wave finding. It is, you know, very descriptive in terms of what the errors are, what the structural elements are, you know, which is, again, very important when it comes to accessibility. Right?  Sure. Now, in this first part of the video, what we saw is it's very easy to use this. All you need to do is put the URL, get this done. But now, think about a scenario where you want to test a page that has a user speciality in word. Or, for that matter, an authentication that you need to somewhat bypass, or maybe not, but get those cookies into your URL so that your web pages which are dynamic in nature are also tested. So, in this part of the video, what I'm trying to do is I'm trying to get into the inbox page of my Gmail account and then paste the URL into the Wave tool and find out if it gives me the correct results or not. Just copy-paste, copying the URL of the inbox page and taking the Hit button. Sorry, hitting the Go button. So, it tracks you down back to the login page because it could not log in. Because it did not have any user session information on that. Now, having said that this tool is pre-open source, reliable to an extent, but then it doesn't work with this dynamic page. Now, solve this problem. Now, I see this as a problem, of course. There are a lot of other problems as well while you test accessibility as a whole. Let me put some focus on those. Let's get down to problem analysis. First of all, if you need to test accessibility, you need to have a dedicated team. Now, that dedicated team means you need to have people with usual abilities alongside people with special abilities to test the accessibility from end to end. You know, this is, again, a very costly effort because you need to maintain the team over a longer period of time. And, of course, building a team, again, you will need to find skilled engineers who are specially able, who are willing to do this kind of a testing, you know, given all their physical and the natural conditions that they are in. The second problem is, even if we are able to build such a team, are all these guidelines, are all those standards met while they are testing? Because there are a lot of those. As you know, there are different kinds of impairments like visual impairments, hearing impairments, motor skills, and all those problems. So, while you are doing this accessibility testing from end to end, are you covering all those standards? That's another challenge that you need to take care of. This essentially is the time and the cost that you are putting into build the team and, of course, maintain the test. And now that every cycle, you are doing the accessibility testing over and over, even if you are doing it manually or if you are using the tools, doing it over and over. And, of course, even if you have got, you know, the best tool in the world, it has its own shortcomings. So, some are paid, that's the shortcoming, I think. Others, in the case of Wave, it just works with static pages. The bigger problem in the case of Wave is, you are sending your web pages HTML to a third party server. It may pose the security threat to your web pages as well, right? Now, having said a lot about these problems, let's see what automation can do to solve these problems, right? So, first of all, I'll begin with a working demo of the framework that I'm, you know, trying to demonstrate. Obviously, on the screen is, you know, it's a Maven command that I'm trying to trigger. It'll run my test. So, the idea behind building this framework was to have your functional test scripts, which are already ready and there, you know, get your accessibility test included within the same framework so that you get a single report that gives you the details about the functional pass and fail and, of course, accessibility while you can do that, all right? So, this is doing nothing but triggering the command, which is a Maven command. Let's see how this works. It's running my Selenium test. So, essentially, it's the Selenium on which I've built the functional automation and then I'm trying to make my accessibility test work with them as well. The same example I'm just trying to do it in an automated fashion. I'm just trying to log in, go to my URL, my inbox page. This is intentional. Just, you know, diluted my inbox so that nobody can see it. Now, what I'll do is just the functional test was to log in. Now that my Selenium is on my inbox page, all I would do is I would trigger my accessibility test on this page. So, essentially, what I'm trying to do is everywhere my Selenium goes, my accessibility is going right with it, you know. I'll just forward the video a bit because it takes, you know, a few seconds to scan the accessibility from end to end, right? So, what it did was just close my browser after scanning for accessibility. Here you see that my Maven command is completed. Now, I'll show you a report that would, you know, tell us the effectiveness of this framework. Just open up the HTML report that is generated by the Maven surefire, as all of us know. Right now, I'll just pause it. I'll try to pause it so that everybody can able to see it. What I have right now here is I have created two tests. I'm not really sure if you're able to view it, but I'll just tell you, just to spell those. It's this step one which is a login test and it's a functional test, right? You're providing a username, providing a password, trying to log in and verifying if you're on the home page or not. The second test, step two, is an accessibility test on the inbox page, right? Now, my functionality test passed with my accessibility test failed. I can see both of them in the same HTML report all the time. Now, let's focus down on accessibility violations, how they are being reported by this framework, right? Again, if I am able to... I don't think that I would be able to zoom this in because it's a video, but I'll just try to, again, spell it, spell everything that's possible. What I'm doing is I'm checking the accessibility issues on the inbox page. It gives me a list of violations. The violations are in the form of the HTML element. Because essentially, all of us, you know, all of us are aware about how accessibility is being tested. So it's more towards the HTML DOM structure and how it has been created. So the focus of this tool, or the framework that we have built, is towards the HTML only. So from the DOM structure, it tries to find out all those web elements which do not comply to the accessibility guidelines. They're formed by, again, the WCAG 2.0. It comes under 508 compliance. Issues would be like, area attribute must confirm to the valid value. Basically, on this URL, there is one element, a web element which has the CSS. Again, sorry, but you can see this. It's a CSS which it points out, and it says that the CSS element should have an area attribute. Similarly, you can cover all the color contrast tests, all your tests which says an image should have an all text. All those tests that say you know, the text would be present and it should be meaningful. All those tests which say you should have captioned for your videos for people who are hearing impairments. All those tests are covered by this framework itself. And the biggest point is, again, I would repeat, it is like functional plus accessibility both together at the same time. The video now further gets us down to the more details of this report. Let me play the video back. Scroll down, so one of the section was about my functional tests, all my messages are printed there. Now, these are all the lists of violations that are there, so all you need to do is just give this report to the developers. They know what the functionality is, and, of course, they would know how to fix the accessibility. Now, one very good thing about this framework about the tool that we are using for building this framework is that it gives the suggestions to fix these problems as well. Let's say if you have a problem, it points out the problem in the HTML and then alongside it also gives fix any of the following area. So it essentially gives you the best way to resolve the problem. Of course, you cannot do it, but the developers can look at this report and find out that, yes, this is what the solution is. If that doesn't work, they are the champions of the course. They can do it the other way, they want to do it. So this is essentially the framework how it works. Now, let's get down to the technicalities. Something wrong. Again, sorry for that. I think this should work. So let's talk about the requirements of how this framework has been built. The major ingredient, of course, alongside my Selenium test is the X API. It's a JavaScript library. It's a simple JavaScript library that has, that's of course an open source, coming from an open source community. So you have no license sauce associated. And then it covers all the WCAG 2.0 guidelines. Right? Then there are a couple more reasons that why we are using this particular API to test the accessibility. Number one is cross browser support. Just as a Selenium is working, you know, across browsers, doing a compatibility testing for the functional point of view, in conjunction to your Selenium, your X is also working across browser to test the accessibility. And now in terms of accessibility, it is really imperative to test across browsers as well. You know, because the rendering engines are different, the CSS might be different, and all that. It doesn't require a third-party server communication. So it's a JAR. It's a JS file, sorry. What you need to do is what we have done in our case, we have built a Java client that consumes these JavaScript APIs from the JavaScript file, you know, and then provide it to our test cases for associates. That's simple. So there's no third-party communication. There's no web communication going on. That saves your web applications from any security test that could might have been the case if you were using Wave or any other online source. Eventually, you'd end up testing the entire document even if there are multiple frames, there are frames within frames and all that, because there are some tools that do not work well with the frame. But in the case of it, it does. It does, it works really well with whatever structuring you have in your edge table. The data might be coming from ten different frames, but your accessibility would work. The accessibility test would work. The next important requirement is and maybe I would say the only requirement is execution of JavaScript commands. Because AX is built in JavaScript, if you want to consume it, your automation script would be able to execute JavaScript on the ground. With Selenium, it's really simple. All of us know that Selenium can execute JavaScript, you know, and then you need them. So it was really simple for us to integrate this AX API with the Selenium automation frame, and that is what it's done. Let's look down at the architecture of the framework. It'll give us, again, a more clearer picture of how this integration works. We do have a resources package where we keep all of the utilities, you know, the data, the data handlers, our jar files, maybe if we're not using a build or a dependency management tool. We keep this minified version of AX API within the resources folder itself. So this is how we have built it. Then we, of course, have a keyword package where all our Selenium code base is right there, you know, your actions, your UI. So what we've done is we've created an export Java which I just mentioned. It's a Java client that consumes the JavaScript API. All right. So these keywords are then being called then being called in the test packet. So everyone, you know, all of us create our automation framework in a similar fashion. We keep it modular. So we've got our test package where it separates on the keywords. The assertions that we need to apply on the, you know, the X result, we do it in our test package. What X actually does is it passes the HTML document and eventually returns your JSON object. The JSON object has a list of violations on that part of the page, right? So what my test does is just reach that JSON and assert the JSON object, find out the violation and print it in my HTML report that I'm printing. So essentially, this is the structure of the framework when I execute this as I did in my video using the build tool, I'll just hit my web application go, now, as in when Selenium is on a web page, my accessibility tests are fired, right? Eventually, you have two sets of tests. You have two sets of results. One, the functional tests other, the accessibility tests. So you've got two reports at the end of each execution. What our framework does and, you know, the way it has been designed is that we integrate both of them together into a single edge tumor which I showed, right? So it's a final report that is what is worth sharing with our clients. That is what we do. And that makes more sense for any development procedure. Eventually, our framework, again, has an additional parameter at the end which is emailing the reports right to the client using Java API itself. We're not dependent on any CI server or email doing it from the Java code itself. This is essentially how automation architecture looks like. All right, so after the technicalities, let's switch down to the business. So what are the advantages or let's say what is the value out of this framework? Dynamic and complete automation tools. As I said, your accessibility tests are running in conjunction with Selenium.  Number two, the dynamism is right there. So Selenium is taking care of your all-user actions. It's taking care of all your navigations. It's taking care of all your user authentic agents. Accessibility is just being tested everywhere where Selenium lands you. That's it, right? Fast and cost-effective. So it's fast because it's automation, number one. Then it's cost-effective because it's an open-source tool. So it need not pay any additional cost to have this integrated in our framework. Let me make a point here. So XAPI is freely available right there on the web. You would definitely find it on the GitHub. So the idea here was to get that integrated with your functional tests. And then you can do it in any other language as well. We did it with Java because we were already having a very comprehensive application in Java. But yes, you can do it in other languages too. It's just that you need to build your client to have that JavaScript embedded inside your code base. Extensible and flexible. So every open-source tool, you know, X is just like every other open-source tool which is extensible. You can get the source code updated the way you want. Customize it the way you want. The feature of X is flexibility. Now, flexibility here means the kind of results that you want. Let's say someday you don't want to execute an extensible test on any of the images. The ING tags in the HTML. You can exclude them. If someday you want to execute only on the ING tag, you can just include them and exclude everything here. So the results could be flexible. Right? The most important advantage of having accessibility into automation is doing it more often, doing it early. Because, you know, right now, while everybody is thinking about test automation, functional test automation, accessibility generally takes, you know, a backseat in the testing process. You generally do not have that much time to fix the accessibility issue because it's almost the release time. Right? Because you do not start off as soon as the product development is dead. While on the other hand, automation, you know, right now in Agile we do start automation right in parallel to product development. So with this framework, what we are trying to achieve is early or accessibility testing, which is given alongside functional testing. So these results can be used to give the accessibility feedback of a developer so that they are aware of the guidelines. Of course they are. And then the next set of code that they write they always keep those guidelines in mind so that they do not break more guidelines, you know. And then maybe at the end of it they come back and fix the existing problem. This is a practical situation that I'm trying to portray. Let's say you have an accessible, an automation test framework that runs on a daily basis. You have got your CIs ready.  But those results are being sent to the developer to whatever means and they are, you know, tracking those down. If I have my accessibility tests already integrated within the framework it would look something like this. So the number of failed tests in the first three days could be, you know, same because either the guidelines were followed knowingly or unknowingly right. Or maybe even if they were not followed, there was no such extremely development that broke it, you know, implicitly that broke the accessibility guidelines implicitly. Now on the third day, developers push a new build and you see a lot of accessibility training. Now that you are getting those results automatically, you cannot avoid this. Number one. Number two if the number is increasing every day, you get your focus back on accessibility imparting the accessibility.  That is what the importance of automation is and that is why we want accessibility within automation. Let's talk about a case study. It's essentially what we did was we used this framework for a pilot project with our existing client for whom we were already doing accessibility testing manually. We wanted to test this out that if this framework is effective or not if yes, to what extent it is right. How is it helping the manual accessibility engineer? How is it doing quicking, you know how fast is this giving the result? So this is what the data has come out. So they were overall 34 accessibility issues that were found on that application. That includes the manual testing. So our specialized engineers, the engineers with usual abilities, all of them did an exploratory sort of inaccessibility testing on an application and they found about 34 issues on a single page. When we executed the same automation framework, you know the Selenium test alongside our XAPI, now a single framework, so it yielded 20 issues. All those 20 were a subset of the 34 issues that were found by the manual team. That ensured that the reliability of this particular framework could be about 60 percent. I mean it's just one case and this may of course vary from case to case but then this is a very good number that we show to our client. Now every time we do this kind of an automation, we report these kind of issues to our client and they are really happy. Because again, they are getting it without any cause. You are doing it, you are doing the automation and then you are doing accessibility at the top of the page. So without involving any specialized engineers from the accessibility team, oh I have reached the conclusion slide but it was very quick. So let's read this out. There are three very important presentations. The first being accessibility is tested early in the cycle. As I said, if you do it early you have more time to fix it. Now with accessibility getting into the automation, you get the feedback more frequently very early in the cycle so that you have enough time for a developer to think about it. Or maybe your business owners actually decide whether they should go for accessibility imparting the accessibility or not. Now the other important takeaway is you are empowering the automation engineers to take on the accessibility. That is more important. Now we being automation engineers have started thinking about accessibility scenario, respecting the way it has been tested and of course it is adding value to the overall delivery that we are doing to our client. I may not be an accessibility expert but still being an automation engineer I can deliver the defects that are right there for accessibility. The final point is quick and continuous accessibility results. So again if you are using an automation you will definitely get into the DevOps and CI servers.  Maybe more frequently than that. But then essentially at the end of it you will get accessibility to a greater extent as compared to what you might have done with only manual testing. I think that is it from my side. Okay. Yeah. Thank you so much open for questions. Yes definitely. I will not show that. So again as you know that in the automation we customize the reports. We can use our test frameworks and given at the end of the report. So we might not or we may remove those tests from there from the home page or do it only in one of our test classes. We divide our workflow into multiple test classes. So what I can think of right now is just keep this accessibility test on the home page in only one of the test class. Just keep on training only in that particular report. As simple as that. Yes. Yes. Absolutely do it. So that is what I said. So I have my keywords you know the X Java the Java implementation my keywords package. Wherever I want to use those methods I can use that in any of my test class. If I don't want to do that I may you know skip that. Exactly. Right. So you are absolutely correct. So and that is being done by a lot of other tools as well. That is what I said initially that we have tools like Tenon that gets integrated at that level only if the developer has not tried that. Again. Yeah. So the answer would be it's an API. It's a JavaScript API. That's it. You can integrate it with any other JavaScript or Java code right and get your answer. So you know that question would be valid in case if I'm doing accessibility everything which I'm not because I have got my framework ready to test accessibility. So on which page I need to call my accessibility test. So my functional test scripts are ready. It's my choice. It's my choice again to have my accessibility test within my testing test or let's say my JUnit test. So it is not that tight coupling it's about we are having a utility that we may use during a certain test to get the test to be done. Okay. Yeah. I can definitely give you some. I totally understand. So let me go to the inbox page which I'm using for my demo. Just quickly show. And then to add a point here the X API that we are using there's also an extension the same name X so it works with different kind of browser especially for Chrome and Pipo I'll just try to use that. I must have got that installed. So let me just find that out from HTML. Now what I'm trying to say you know and to answer the question there has to be an element just this image. Right. So here is an image tag that doesn't have an alt you know attribute. Now for accessibility there is one major accessibility guideline that says that every image tag should have an alternate tag so that a visually impaired person actually understand what that image is all about because he cannot see it. Now if I test this inbox page I would see that this particular image doesn't have an alt tag. My automation script offer that matter any of the other tools that I'm using it would find this particular element and report that this will be wrong. Now the tool that I'm offering you know the tool that I'm proposing or the framework that I'm proposing it gives you the text the hot fit for that particular problem. So it will show there is no alt text and of course in this case it will show you can fix this by adding an alt tag with a meaningful text message alongside. Now is what was your question in this context? Have I answered it already or is it still pending? I'm saying yes. So for any visually impaired person he would be able to understand what image it is all about. So you use screen readers for you use screen readers for people who are visually impaired they use screen readers to read the text from left to right. And if there is no text they would not be able to understand what the image is all about. I'm really sorry I didn't get you. To write for that particular case that was actually automation limited. Why? There are a few areas which you cannot cover with automation in particular with this framework. So let's say there are problems like incorrect labels. Let's say there is a button which says cancel or the button says thumb it. Clicking on it it takes you to the cancel it cancels the form. That is incorrect labeling. Or there is a label called so the visually impaired and specially able people would think there should be a text box but then in actual there are radio buttons. That's a problem. So those are the areas which the framework automation cannot judge whether these really exist or not. Another example is a keyboard trap. So there are multiple frames on the page you're trying to use to review the fact that the accessibility engineers they use keyboard tab button to go over the entire page to access all the elements. Now there are some frames from which they cannot come out ever. If your tab goes inside that frame it would never come out. It can only be tested by a manual testing engineer. The automation cannot do that. So when it's about audio let's say and there's a video. So the accessibility guidelines say that there should be a caption alongside so that the person can read. Let's say the person is not able to read either. So in that case it will, so if the person cannot see but can listen maybe your question is into that context. So listening on it's again something that goes out of the automation scope. Because you cannot automate the sound. That's the answer. Now number one I generally don't know it because I'm not an accessibility expert but yes to your point it's just like that automation in this case is done at the HTML level. Every time you do it at the HTML level so accessibility for such things are you would have some attribute so that you can understand that what this video or the audio needs to convey. And your test would essentially pass into the HTML. That is the kind of test that you would do it using automation. So if not at the particular client level but you know we have gone through a few surveys so there are about if there are 100 users on the web there are about 20 or especially that is a survey conducted by I think IWSE International Accessibility Organization Association they did it and we gave a webinar on this topic you know two months back it was a 45 minute session in that I quoted that particular survey but due to lack of time I did not not really because we are not just going to defame any not really maybe the dedicated team would have done but we are trying to do this with our existing clients and unfortunately Amazon is not ours now we are trying to figure out if we build this and then give webinars and propagate it to the level that they know we do it exactly that is what we are here for and we do webinars regularly we have done this webinar and last maybe in April and we have got a few interested folks coming to us understanding how this works how can we help them we go to them with these details the data that we have the case studies and all that they slide and help them right that is a screen reader that is a page to no no no because there is a problem that is one thing but I also like to add a point here this is a reading sequence associated with every web page it is you know either from left to right or maybe you would see it is just littered from between and it is like first you have to read out the left side and then you have to go to the right side and then you would also have seen some tables you know if there is a table on the web page it would let's say it is an employee table it is serial number it is name it is ID it is phone number now when I am reading it from the left to right the screen reader like Jaws it reads it this way and it reads Anshal Mathur 1 it does not tell me what is what right so this these are the kind of problems that even the tools can't find out and that is when it is comprehended by the manual testing team so we do keep you know we do keep that expertise of having a manual team on board but then it is just to find out the basic problems that you know they would do it just at looking just by looking at the experiment so all those things are already covered by the framework so it is an initial step it is the first step to do it just an assistive tool right and one more you know one very important thing so personally I started respecting accessibility initially I had no clue about how it is being done now that if I can do something programmatically why not right yes and every accessibility guideline is implemented in the HTML itself and your system and DX head top to the HTML so all guidelines can one way or the other be covered using this so you know I do not want to overcome it or false commit but then that is why I presented a case study that it was 60% it might vary but then there are other clients you know we are working with and you know it gives you that initial edge over others that you are doing it automated in an automated fashion so there are no accessibility defects after let's say two cycles it reduces the eventual effort as well now when it comes to the percentage it could be about that 50% area because I do not want to you know the manual testing at all and then there are limitations too we have to take that into account 50% 50% and 60% is my point of view any other question yes


Welcome back to theCUBE's live coverage of AWS re-invent friends. It's good to see you. Lisa Martin here with Dave Vellante. This is our fourth day of CUBE wall-to-wall coverage, Dave. I can't believe it. And the expo hall is still going incredibly strong. Yeah, it is. It feels like the biggest re-invent ever. I'm told it's almost as big as 2019. I don't know, maybe I was half asleep at 2019. That's very possible. But I'm excited because in 2017, Andy Jassy came on theCUBE, and he said if Amazon had to do it all over again, if it knew then what it had now, we would have done the whole thing in containers, using Lambda, using Serverless, and using containers. Didn't have that opportunity back then. And I'm excited because RAFE Systems is something we've worked with a lot. And it's an innovator in this space, so. Yep, and we're going to be talking with RAFE again. I think it's your 10th time, Haseeb, on the show. Like once or twice. And a great customer who's going to talk about their Serverless journey. Haseeb Ujjani joins us once again, the CEO of RAFE. Great to see you. Rakesh Singh is here as well, the head of Cloud and DevOps at Regeneron. Guys, it's great to have you on the program. How are you feeling on day four of re-invent? Excitement is as high as ever, basically ever. Isn't it amazing? That's true. I just need some sleep. Oh my God, so many parties, so many meetings. But the great thing is, Haseeb, that people want to engage with you. They're loving what RAFE is doing. You guys are a great testament to that, which we're going to uncover on the show. What are some of the things that you're hearing in the booth from customers? What's been some of the feedback? So, firstly, as you said, it feels like the biggest one ever. I've been coming to re-invent a long time, and I know the numbers say it's not, but oh my God, this is a lot of people. Every time we've spoken over the last year, and the point I always make to you, and we've spoken at a time about this, is that enterprises are truly adopting this idea of Kubernetes containers, Serverless, et cetera. And they're all trying to figure out what is the enterprise strategy for these things. They're thinking beyond technology and thinking operationalization of these technologies. And that's not the same thing. There's a toy, and then there's the real thing. And that's not the same thing. And that's the gap that every enterprise customer I talk to, and the booth traffic has been just amazing. But coming here, I was thinking, my God, this is really expensive. And I'm thinking, wow, this is a great investment. Because we met such amazing companies who all essentially are saying exactly the same thing. Which is, as we go and productize and bring our high value applications to the modern infrastructure space, like Kubernetes, Lambda, et cetera, solving for the automation governance is really, really hard. Because, well, at one point, I guess when the economy was doing crazy well, I could keep hiring people, but I can't do that anymore either. So they're out looking for automation strategies that allow them to do more with the teams they have. And that's exactly what Rafi is here for. Yeah, you know, at least Adam, it's a lipstick in his keynote. I love the, he said, if you want to save money, the cloud's the place to do it. Exactly, yep. Let's talk about Regeneron. Everyone knows it's a household word, especially over the last couple of years, but talk about Rakesh Regeneron as a technology company that delivers life-saving pharmaceuticals and where does cloud and Rafi fit into your strategy? So cloud has been a backbone of our compute strategy within Regeneron for a very long time now. The evolution from a traditional compute structure to more serverless compute has been growing at a rapid pace. And I would say, like, we are seeing exponential growth within the adaption of the compute within containers and Kubernetes world. So we've been on this journey for a long time and I think it's not stopping anytime soon. So we have more and more workload which is being running on Kubernetes containers and we are looking forward to our partnership with Rafi to further enhance it. As Seed mentioned that the efficiency is the key. We need to do more with less. Resourcing is critical and cloud is based, is evolved from that journey that do more things in a more efficient manner. That was the original catalyst is we've got to help our development team be more productive. That's correct. Eliminate the heavy lifting and then you started presumably doing some of the less heavy but still heavy lifting and we talked off camera and then you're increasingly moving toward serverless. That's correct. Can you describe that journey, what that's like? So I think like with the whole adoption that things are taking a much faster pace. Basically we are putting more compute onto containers and the DevOps journey is increasingly getting more faster. So I want to understand where Rafi sits in this whole equation. I'm not a developer, but I was talking to developers today trying to really understand the benefits of containers and serverless. I said take me through what you have to do and when you're using containers is I've got to build a container image then I've got to deploy an EC2 instance where I've got to choose it. I've got to allocate memory, I've got to fence the app in a VM then I've got to run the computing instance against the app and then oh by the way I've got to pay because all that EC2 that whole time. Depending on how you approach serverless you're going to eliminate a lot of those steps. And so that is correct. Like so what we do is basically like in a traditional sense the computer sitting idle at quite a lot basically. But you're paying. And you're still paying for that. Serverless technologies allows us to use the compute as needed basically. So whenever you need it it is available you run your workload on that and after that it shuts down or goes to a minimal state and you don't need to pay as much as you're paying.  Look serverless as a paradigm. If you step back from the idea of containers versus Lambda or whatever function. The idea should be that the list you just read out of what developers have to do. Here's what they really should do. They should write their code, they should check it in and they never have to think about it again. That should be the piece. If they want to debug their application there should be a nice front end where they go and they interact with their application and that's it. What is Kubernetes? I don't care. That's the right answer. And we did not start this journey as an industry there because usually the initial adopters are developers who do the heavy lifting. Developers want to learn, they want to solve these problems but then eventually the expectation is that the platform organization and enterprise is going to own this platform for me so I can go back to doing my job which is writing code. And that's where Rakesh's team comes in. So Rakesh's team is building the standard at Regeneron. Whether you're writing a long lasting app which is going to run in a container or you're going to write an event-driven application which is going to be a function, whatever. You write your app, we will give you the necessary tooling and plumbing to take care of all these things and this is my problem, my being Rakesh. Rakesh is my customer. He has his customers. Right? B, B is Rafa. A, we have to make Rakesh's team successful because we have to give them the right automation to do all these things so that he can service 100, in his case, thousands and thousands of different individuals. But then collectively, we have to make sure that the developer experience is optimal so that truly they just write their code and EC2, they don't want to deal with this. In fact, on Monday evening in the Kubernetes keynote by Barry Cooks, one of the things he said was that in a CIO survey they did, CIO said 80% of the time of developers is wasted on infrastructure stuff and not on innovation. We need to bring that 80% back so that 100% of the work is on innovation and today it's not. And that's what you do? That's what we do. So in your world, as a developer, I only have to worry about writing my code and what functions I'm going to call. That is correct and it is important because the efficiencies of a developer need to be focused on doing the things which business is asking for. The 80% of the work like to make sure that things are secure, they're done the right way, the standards are followed, scanning part of it, that work if we can offload to a platform which, for example, RAFE, saves a lot of work cycles from the developers perspective. Thank you for that. That's a little tutorial on sort of the benefits. Absolutely. So you transform the developer experience. That's correct. How does that impact Regeneron's overall business? We up level that. Give me that view. So with that, what happens the key thing is the developer's productivity increases. We are able to do more with less and that is the key thing to our strategy that with the increase in business demand, with the increase in a lot of compute things which we are doing, we need to do, and hiring resources is getting more difficult than ever and we need to make sure that we are leveraging platforms and tools basically to enable our developers to focus on key business activity rather than doing redundant things and things which we can leverage some other tooling and platform for that. Is this something, in terms of improving the developer experience and their productivity, faster time to market, is this accelerating? That's correct. Is this even like accelerating drug discovery in some cases? So COVID is a great example for that. We were able to fast track our drug discovery and we were able to turn it into an experience where we were able to discover new drugs and get it to the market in a much faster pace. That whole process was expedited using these tools and processes basically so we are very proud of that. So my understanding is you're running Rafay with the EKS, right? A lot of choices out there. Why? Why did you choose to go in that direction? So Regent Ron has heavily invested in cloud recently over the years basically and then we are focusing on hybrid cloud now. We're like again these multiple cloud providers or platforms which are coming in are strategies to focus on hybrid cloud and Rafay is a big leader in that particular space where we felt that we need to engage or partner with Rafay to enable those capabilities not just on AWS, but across the board. One single tool, one single process, one single knowledge base helps us achieve more efficiencies. Less chaos, less complexity. That's correct. Let's say when you're in customer conversations I know you've had many this week but you probably do that all the time. This is, Regeneron is a great use case for Rafay. It's so tangible. Life sciences, we all get that especially coming out of the pandemic. What do you say to customers are the top three differentiators of Rafay and why they should go Rafay on top of EKS? So, what's really interesting about these conversations is that look, we have some pretty cool features in our product. Obviously we must have something interesting otherwise nobody will buy our product. We have access management and zero trust models and cluster provisioning, all these very nice things. But it always comes down to exactly the same thing which is every large enterprise that started a journey independent of Rafay because they didn't know who we were. It's fine. Last year we were a young company, now we are a larger company. And they all are basically building towards a roadmap which Rafay truly understands and in my opinion, and I'm confident when I say this, we understand their life, their journey, better than any other company in the market. The reason why we have the flurry of customers we have, the reason why the product has the capacity that it does is because for whatever reason, look, it's scale, lock, that's for the history books. But we have complete clarity on what a pharmaceutical company or a financial service company or a high tech company, the journey they will take to the cloud and automation for modern infrastructure, we get it, right? And what I'm telling them is the why, not the what. There's a lot of great answers for the what. What do we do? Rakesh doesn't care, right? I mean, he's trying to solve a bigger problem. He's trying to get his researchers to go faster. So then when they want to run a model, they should be able to do it right now. That's what he cares about. Then he looks for a tool to solve the business problem and we figured out how to have that conversation and then explain why Rafay helps him essentially multiply the bandwidth that he has in his own organization. And to, of course, to that end, we have some great technology, right? But that's a secondary issue. The first, to me, the why is more important than the what. And then we talk about the how, which is he has to pay us money, right? That's the how, but yeah, we get there too. But look, this is the important thing, right? Every enterprise is on exactly the same journey, Lisa, right? And that, if you think about it from just purely economic efficiencies perspective, that is not a good investment for our industry. If everybody's solving the same problem, that's a waste of resources. Let's find a way to do, what's the point of the cloud? We used to all build data centers. That was not efficient. We all went to the cloud because it's more efficient to have somebody else AWS solve this problem for us so we can now focus on the next level problem. And then Rafay's solving that problem so that he can focus on his drug discovery. Not on Kubernetes. That's correct. It's all about efficiencies, right? Like doing things, learn from each other's experience and build upon it. So the things have been solved one way. You need to leverage that. We use it. So the principles are the same. So then what's next? You've done an amazing job transforming the company. You're facilitating drug discovery faster than ever before. From a cloud, from an infrastructure perspective, what's next on your journey? So right now the roadmap what we have is basically talking about making sure that the workload are running more efficient. They are more secure. As we go into these expandable serverless technology, there are more challenging opportunities for us to solve those challenges that are coming up. We need to make sure that with the new, the world order we are living in, we are more securely doing stuff what we were doing previously. More efficiencies is also the key. And more distributed. Like if we can leverage the power of cloud in doing more things on demand is on our roadmap. And I think that is where we are all driving. And when you said hybrid, you're talking about connecting to your own prem tools and data, how about cross cloud? We are invested in multiple cloud platform itself and we are looking forward to leveraging a technology which is truly cloud native and we can leverage things together on that. And I presume you're helping with that, obviously. Last question for both of you. We're making an Instagram reel. I think if this is a sizzle reel, like a 30 second elevator pitch. Question, first one goes to you, Rakesh. If you had a bumper sticker and you put it on, I don't know, say a DeLorean, I hear those are coming back. What would it say about Regeneron as a technology company that's delivering therapeutics? It's a tough question, but I would try my best. The bumper sticker would say discover drug more faster, more efficient. Perfect, Haseeb, question about Rafay. What's the bumper sticker? If you had a billboard on Highway 101 in Redwood City about Rafay and what it's an abling organization's enterprises across the globe to achieve, what would it say? I'll tell you what our customers say. So our customers call us the vCenter for Kubernetes. And we all know what a vCenter is. We all know why vCenter is so amazingly successful because it takes IT engineers and gives them superpowers. You can run a data center. What is the vCenter for this new world? It's us. So vCenter is obviously a trademark with our friends at VMware, so that's why, you know, but our customers truly call us the vCenter for Kubernetes. And I think that's an incredible moniker because that truly codifies our roadmap. It codifies what we are selling today. There's nothing more powerful and potent in the voice of the customer. Thank you both for coming on. Thank you for sharing your Regeneron story. Great to have you back on, Haseeb. You need a pin for the number of times you've been on theCUBE. At least a gold star, right? We'll work on that. Guys, thank you. We appreciate your time. Thank you very much. For our guests and for Dave Vellante, I'm Lisa Martin. You're watching theCUBE, the leader in live enterprise and emerging tech coverage.


Excellencies, distinguished colleagues, guests from all around the world, ladies and gentlemen, it is a very great pleasure for all wider staff, myself included, to wish you a warm welcome to the September 2016 UNU-Wider Development Conference on Responding to Crisis. We tried to put together a stimulating program for this two-day conference, which includes an important keynote by Elizabeth Rehn, former Minister of Defense, Minister of Equality Affairs and Undersecretary-General of the United Nations. When we early on discussed the topic for this conference, we held a series of meetings and discussions with the Crisis Management Initiative, CMI, FinChurch Aid, the University of Sussex, the University of Copenhagen, and the International Organization for Migration IOM, which is now a member of the UN as a related organization. I'm pleased and indeed grateful that we managed together to arrange a subset of the sessions to be held at this conference. I'd therefore like to extend a warm thanks to CMI, FinChurch Aid, Sussex, Copenhagen and IOM for excellent cooperation. Our appreciation also goes to our regular donors, the Ministry of Foreign Affairs of Finland, DFID, CEDA and DENIDA for their sustained collaboration and financial support, without which we would not have been able to bring you all here. It's not quite as sunny today as we had hoped, but the weather forecast for the rest of the day is very good and tomorrow should be excellent. I hasten to add that our donor group and we at WIDA would warmly welcome new members. The topic responding to crisis could, as I see it, not be more timely. And to get us going, the Minister of Finance of South Africa, Praveen Gordon, a close teammate in the struggle against apartheid, would like to make some welcoming remarks. Praveen Gordon knows more than most what it means to respond to crisis and importantly, he looks to think stands such as wider for advice and interaction on how the bigger policy issues of the day should be faced and addressed. I invited him some time back to be here with us today, but urgent matters kept him in South Africa, but he kindly agreed to do a video welcome message that we will now play for you to get us going. Good morning to colleagues, friends and I imagine comrades who are in Helsinki for the WIDA conference on responding to crisis. Firstly, thank you very much for this privilege of addressing you as part of the welcome to this conference on behalf of WIDA. I certainly would have preferred to have done it personally, but regrettably the pressure of work in South Africa prevented me from doing so. I want to congratulate Fintab and his team and of course all of the participants for organizing and attending a conference like this that is very time-yous, very important and one which policymakers like myself around the world who have a progressive orientation will be looking forward to answers for. So what is the crisis about? What's caused the crisis? What are the underpinning currents within this crisis? And where do the solutions lie? I imagine we'll be amongst the questions that you will address and whose answers we would certainly look forward to. The concept of responding to crisis also behoves us to ask some tough questions of ourselves as a global community in trying to understand the crisis itself. And as we said today, many want to know what does inclusive growth mean? And what are the standards or parameters and the dynamics by which each of us would be judged in terms of whether we are inclusive enough, so to speak, in terms of the growth models, the quality of growth, the extent to which ordinary citizens who are becoming increasingly active in virtually every part of the world, in expressing either their frustrations or their aspirations and expectations, certainly of policymakers, but I imagine of your community as well. In demanding answers to the question, how do I benefit from the kind of growth that we are actually seeing at the moment? And so above all, in addition to the benefits question, the challenge for us is how do we increase the standard of living and well-being of 7 billion people on the globe? And what are the different types of dynamics or systems that will assist us in answering that particular question? Of course, as we all know, the global economy is going through a particularly difficult period. It seems that that period is going to last for a few years to come, a period in which growth is low, where investment is low, where financial flows still dominate possible disruptions to particularly the emerging world, where trade has been diminishing, trade finance is a challenge. And the all-rounded development of society in terms of the kind of expectations that the sustainable development goal sets for us is today a demanding proposition for governments and organizations to actually meet. Equally, citizens, as I said, are becoming increasingly dissatisfied with their state of affairs. And the fact that a group of elites, and they can be defined differently in different contexts and in different parts of the world, seem to be appropriating for themselves the fruits of the work that ordinary people put into their different environments and the kind of outcomes of growth patterns and investment patterns as well. This is leading to challenges, as I said about what inclusivity means. It's leading to challenges about how we find very different but answers nonetheless, which are consistent with the objectives of creating a more socially just world for ourselves and for the majority of the people. But social justice will then mean greater inclusivity, less inequality, a more advanced way in which, particularly in the developed world, the billions of people who live in that part of the world can actually improve their standard of living, their standard of education, the standard of health, amongst many other factors. All of these factors are also giving rise to serious questioning, as you do in your program, of the phenomenon of globalization. Some are talking about de-globalization. Others are questioning the benefits of globalization. And yet others are talking about reshaping globalization so that if you like, we can correct some of the aberrations of globalization and unintended consequences if one can talk in those terms that globalization has resulted in. But equally important as you meet in Europe is the kind of populism that we see rising in Europe, but in other parts of the world as well, amongst politicians and political systems and the populations who are frustrated with the range of things that I have mentioned and which you are very familiar with and can't find the right answers to, one, express themselves clearly, two, have themselves listened to clearly, and three, more importantly, have themselves responded to in a way in which meets their expectations both for now and the future. This populism is giving rise to the wrong kind of, if one can call it ideologies in the 21st century, the wrong kind of political developments and political formation developments, all of which could result in very negative outcomes which could impact upon the kind of progressive, for example, social security systems and safeguards that working classes over decades have created as their mechanisms for well-being in different parts of the world. So clearly in all of this, the financial sector has been increasingly questioned about its role, about where it actually benefits the rest of the economy and more importantly, the population as a whole, the kind of this great recession that it gave rise to as a result of Invertecoma's Wall Street banking practices. It was fascinating a day ago in South Africa to view the CEO of Wells Fargo being questioned. I think it was in the US Senate in a television program coming from one of these 24-hour channels. And the blatant lack of willingness by the CEO to take responsibility for the kind of damage that was caused and the tendency to repeatedly pass on the problem to people that had actually reported to him. So the great recession of 2007-2009 has clearly had a massive impact both in the developed world and the developing world. It's created problems around sovereign debt crises in different parts of Europe. It's raised questions about banking regulation and from a developing country point of view, some of those banking regulations that have been developed over the past few years by the FSB under the aegis of the G20 have in fact potentially created problems for developing countries and their financial systems and their stability as well. But most importantly, these developments have resulted in some serious questions about where do jobs come from in the future? What is the role of technology in the diminishing number of jobs that will be available? What does it mean for how human beings across the globe are to be educated, trained, retrained and retrained in order to fit in with a very different kind of the world? In that sense, the world economy today is at a critical juncture. It cannot be held to ransom by either political inaction or by political populism. And clearly conferences like your own that you're attending can help us to demystify many of these processes and give us some guidelines as policymakers as I said earlier on as to what could be propositions that we could begin to consider in developing a more inclusive growth path but one that takes into account some of the developments that we have seen around the globe and I'm sure you are going to see well into the future as well. We all familiar with the fact that if you like some of the centre of gravity of growth globally speaking has shifted from the advanced economies to the large emerging markets, China and India in particular but equally we know that the last two years has seen countries like Brazil and Russia find themselves in a recession and in countries like South Africa we stay just above 0% growth for this year and slightly better prospects for growth next year and on the African continent all we see for 2016 in terms of the IMF estimates is a growth of 1.6% and better prospects of about 3% as we go into next year but at the same time have we actually dealt with all of the factors that led to the Great Recession have we dealt with adequately the way the banking sector and financial sector more generally behaves have we dealt with the search for short-term yields and the kind of unmanageable financial flows that are happening once interest rates change in one place or negative sentiments emerge in the second place or positive sentiments emerge as well which begins to attack that capital have we actually come up with answers about how the capital senders can find new ways of ensuring that these kinds of disruptions to exchange rates and other phenomena in emerging markets which might or might not lead to so-called currency wars as we've called them in the past are actually averted rather than reinforced how do we avoid the kind of observations that Raghuram Rajan, Robert Schiller and others have made about the negative results which could emerge from credit dependence by consumers and the willingness of financial institutions in different parts to actually lend in an inordinate and sometimes unsecured way on the other hand the financial system itself requires some inquiry in the post recession period how stable is the financial system in different parts of the world have governments in particularly most of the advanced economies all done enough to in fact stabilize their banking systems and ensure that they don't become the source of a problem is the link between the status of the financial system and sovereign debt problems being adequately resolved as we go forward and some of these issues are no longer concentrated in the advanced economies is beginning to impact certain parts of the emerging economy world as well excuse me clearly as I conclude the FinTAP and his colleagues together with their collaborators have taken a fairly wide view of the world crisis as it obtains to our world today including issues of climate change, trade, refugees the sharing of the use of the Nile is a fascinating topic and if you like the state of states or governments at this point in time and the kind of ability or inability they have to influence the kind of environment in which they find themselves so let me repeat the crisis is certainly one which is economic in character at one level but there's no doubt that the other factors that are in your program including the political, the refugee crisis the crisis of uneven development across the world because the development status of different parts of the world is as important as the developments that take place in a particular country the politics that actually influence decision making so political economy becomes some parts of your program indicate a very important topic in our current situation and above all geopolitics is becoming a prominent factor both in terms of being the cause of crisis but also in terms of its inability to resolve these crises in an inadequate way but we all seek the world where there is social justice where there is stability where there is even development where all of the sustainable development goals are addressed in an even way across the globe so that all seven billion people can lead a life of well-being and increasingly see that their lives are being transformed by the things government do, by the things they do and by the thing other actors do and therefore an increasing standard of living begins to emerge what is critical and becoming more widely accepted today is that unless we give content to what inclusive growth means and focus on the real causes of some of the crisis factors because we can't address all of them together nonetheless we cannot begin to understand them as you are attempting to do at your conference we are not going to create a better world we are looking down a tunnel where the world might remain in the current state of what some might call secular stagnation I'm sure you'll have a fascinating debate on that what others might call a low growth scenario for the foreseeable future which is something that the billions of people who look forward to a better life can't actually afford at this point in time so finally thank you very much for this privilege of addressing you as part of the welcome to the conference may I wish you well in all of your deliberations and certainly we in South Africa will be awaiting your outcomes and hope that you could elucidate a clear path to a better set of propositions about how we improve the lives through economic, political and other means of the billions of people around the globe but particularly on the African continent and in our own very young democracy thank you very much thank you very much I hope you will allow me to extend the gratitude of all of us to Praveen for his welcoming remarks it's certainly nice to touch upon the major themes of the UNU wider ongoing 2014 to 2018 work program on transformation, inclusion and sustainability and you also pointed to the broad way in which we have approached the underlying themes for this conference after all, wider is wider