Welcome everyone for today's session by Anand. The topic of today would be automating the real user scenarios across multiple apps and multiple devices. Without further ado, I would like to give to Anand. Thank you Shri Krishna. Hi everyone. Good evening or good afternoon. Good morning from where you are joining. Thank you for joining Agile India. It's a great pleasure to be back in Agile India. And today I will be sharing with you my experiences and my approach of how can you automate the real user scenarios across multiple apps, multiple devices as part of your functional test automation. Little information about myself. I've been in the quality space for more than 20 years now. And I've played various different roles along the way to help build a better quality product, whether it is as part of product organizations, services organizations, or as a consultant that I am being since the past four or five years now. I'm also a contributor on the Selenium project. I help out and volunteer on Selenium conference, APM conference Agile India and various other conferences as well. And this is a way for me to learn more from others experiences and try and get better. You can reach out to me on Twitter or LinkedIn. And without further ado, let's get started with the core topic of today. How do we automate real user scenarios across multiple apps and multiple devices. If you have any questions, please do keep on asking it in the Q&A section in the conference and I'll try to address it as we go. Otherwise, we'll definitely catch up on the hangout table and discuss further. So before we get into the solution of how do we automate the real user scenarios, it's important to understand why do we automate the functional tests. And that is of course to simulate the user actions and behavior so that we get better sense of quality of a product before we do the release to production. It is important to remember that these scenarios, they include delays and waits to inspect and then proceed with the next action. Users actually do this more than what our test to users will wait for the screen read the information and then proceed but they will not keep waiting forever. At the same time, our test should not keep on waiting forever, but we should have sufficient delays the right type of delays to make sure the product is rendered correctly the functionalities rendered correctly and proceed. We should not accept any random arbitrary delays over here. It has to be realistic. How much will you as a user wait for the screen to load before your attention gets diverted to something else. You have to always put on that hot process when you're automating. Now, to do functional automation, you need to understand your application under test. This application could be in form of a web app. It could be a mobile web app. It could be available as a native application on your Android or iOS devices or tablets as well. But once you understand your application under test, you would create a set of flow charts or scenarios or mind maps and prioritize then what should be automated, what is required and important to be automated for your product. And now once you identify these scenarios, then it's actually a solve problem. How do you automate this? Automation is not new. There are many options that you would have today that you can choose from and this is a very small set of options that I have listed on the screen, how you could select to do the automation. The functional automation is what I'm talking about. What I want to present to you is one additional option that I have helped create, which is called TestWiz. It's an open source framework that I've created that will help do the real user automation along with the type of scenarios that we are used to. How does TestWiz solve this problem? It's an opinionated framework. It is guiding you. It is telling you how you should be doing your automation and this is based on my experiences in automation, what has worked well, what has not worked well. And what I have gained as experience from there to say how you should build a framework that is going to be long lasting, it is going to be maintainable, scalable. New users can come and continue contributing to it in a very easy passion while getting the type of value that you expect from automation. So how does TestWiz solve this problem? It uses Cucumber JVM as a BDD tool to specify your test intent and in this test intent, as you can see, it is very easy to understand if you define a structure, which set of steps are being implemented using API. The annotations determine which platform the test is going to run on. So if your functionality is available as an Android app or as a web app, then you could choose to run the test across any of them simply by providing the annotation or the parameter, which platform you want to run. And because we are using a BDD tool, it is very easy to specify the business logic that you're trying to automate at the top layer of the pyramid. These are tests that sit on the top layer of the pyramid. These are end-to-end tests. These are not test cases, granular test cases that you want to test. So if you focus on the business logic, define your steps in a declarative form, then you will be able to provide a very clear, concise set of actions in form of business rules that can be executed. That can be executed by the user to implement the functionality. Now, this is again a simple standard solve problem. Okay. If you run this test using testvis, let's say if I have to run it in Android, here's a test. I can simply run this. I have emulators running on my machine over here. And I can simply run this test by giving the right configuration parameters on command line and execute the test. The test is going to execute. You will see the results also as soon as they are complete. And it takes care of managing the driver, managing the browsers, managing the apps, installations, and so on. All of those aspects are taken care of by the framework automatically for you. Okay. So this is a very standard test that you'd be able to write using any tool and testvis is no different in that regard. But there is a different category of complexity because of which I really had to think about creating testvis. This complexity comes from the fact that we live in a hyper-connected world. No thanks to the pandemic. We have got used to being in front of the computer almost 24-7. We are available 24-7, unfortunately. And we are collaborating with people, our colleagues, our team members across the world, and they could be collaborating with you on a completely different set of device than what you might be using. You could be on a Windows laptop. They could be on a Mac or a Linux or on tablets, anything that might be there, or you could also be doing some online screen sharing and live coding over there, for example. The point over here is there's a lot of collaboration happening, a lot of interactions happening. If you are testing a product which is a collaborative platform, an interaction-based platform where multiple users are coming together to achieve a certain objective, achieve a certain functionality, the question really is how would you automate such scenarios? And that is again where testvis comes into the picture. So let's take some specific examples and let's start off with thinking about multi-users. In this particular case, let's see how testvis solves this problem. The scenario over here is, if you notice from the earlier demo, it was a video conferencing platform like Zoom. This was a GeoMeet use case. In this particular example, what we are saying is as a guest who's coming from Android device and a host who is also on an Android device, when they are in a meeting, they can send files to each other over chat communication as part of the meeting. So in this case, the test is saying the host is going to sign up using API. So we are creating data on the fly and then the host is going to log into the Android app and start an instant meeting using the credentials that were just created for it. Then the guest is going to come and join the meeting from another Android device and only when there are more than one participant in the meeting, then chat option is enabled. Hence we are saying then host should be able to get to the chat window before that chat is disabled. And when you can get to the chat window, then you can send a file in the chat message and the guest should be able to receive it. So in this case, the step indicates who is doing the action and where that user is. But the interesting thing over here is in this particular orchestration, there are two users coming on two different devices and they will be interacting with each other. As part of the same test orchestration. So let's see how test with solves that particular problem. Interesting, the video is not wanting to play over here. It's not seen rather apologies for that. Let me start it again. There we go. So now in this case, I'm going to run only a small snippet of the test where there are two users on Android devices. And again, it's the same two emulators running on my machine using a command line argument. I'm starting off the test and we see that now test has started off. The first device is picked up. The host is going to sign in over here. We can see sign in option is coming up. You're signing in. Once you sign in, the host is going to start an instant meeting when the meeting is started. Host saves the implementer is capturing the meeting ID from here and saving it in the framework and that same meeting ID is passed to the next user. Who's going to now, which is a guest was going to use a second device start the application. But in this case, instead of signing in, the guest user is going to join to the meeting that the host has created. And when the guest joins the meeting, we will see that on the emulators, we are able to see now both the users are available over there. And because they are available, the chat option is also enabled. Now the next set of functionality can be executed. Okay. So this way, in a very easy fashion, where all we did was specified the personas and the platforms where the users are coming from, and we are able to orchestrate the interaction between them across multiple devices. But this is not sufficient. Our users might not be only on one platform. What if they are coming from different platforms, where someone is on web, someone is on Android and so on. So the way Tesla solves this problem is, again, it's exactly the same thing. The only difference over here really is we have the persona and where we are doing the action we are specifying a different platform. So in this case, the host is coming in from the web browser, the guest is coming from an Android device, and that is how they are going to communicate and collaborate in this particular scenario. So let's take a look at an example of such a scenario. So over here, again, host is coming on Android, guest is coming on web. We run the test. I still have both the emulators over here. We see that first the app is installed, the host has come over here on Android, sign in is done, meeting is started. And now, instead of the guest coming on Android, in this case, a browser is opened up and the guest is going to join the meeting from the browser. When the guest joins a meeting, we can see there are two participants in the meeting and we are able to now continue the orchestration of what should happen in a meeting between these two participants. So this is how you will be able to create multi-user, multi-platform scenarios to have a different level of interaction. Is that all? No. There is something even more interesting that is possible. Let's look at different scenarios what can be still implemented. This could be a multi-app scenario. Let's take an example. I'm ordering something as a consumer. I'm ordering something from an Android app or it could be from a web or base anything, but the app is for a consumer to place an order. When the order is placed, the order eventually gets to the warehouse to say, yes, the order has been processed, payment is processed, ship the order to the customer. The warehouse will do the processing using a completely different set of applications. And from there, when the order is finally reaching the last delivery station, the delivery person will pick up the items and go to the delivery address to the customer and deliver the order. Now in this case, it's a highly simplified e-commerce flow, but the minimum what we see is three set of applications over here. What if you want to orchestrate the end-to-end scenario? This is the topmost tip of the top of the pyramid. These are the type of scenarios where you're doing pure end-to-end across different systems and applications as well. You would have extremely low number of scenarios for this type of validation, but this type of validation is still important and you still need a way to automate it instead of having to do this manually. So in this particular case, you have multiple applications being interacting with each other as part of an orchestration and you should be able to automate that. There could be another scenario as well where you might be interested to know what if there are different versions of the application and can you collaborate with each other using those different versions. A classic example could be one version of Zoom, participants are on different version of Zoom, post and participants, and you should be able to collaborate between those. How can you run an automated test that will allow you to do that? And this is where again, TESVIS solves this problem in a very interesting way where the step over here now is indicating who is doing the action. And you can have as many number of personas, number of users interacting in that orchestration. The only limitation is about your machine where the test is executing or the number of devices or browsers you have available that can be executed on that infrastructure. That is really the only limitation. TESVIS can support whatever scale you really want. So in this case, we have the step indicating who is doing the action. We are also specifying over which application is being used by that persona to do the actions or do the orchestra and be part of that orchestration. And of course, which platform are they coming from. So in this particular case, we are saying there are three different applications that are really being used. Host is coming from the latest version of the GeoMeet application. The first guest is coming on an older version of the GeoMeet application. Both of these are on Android. And the second guest is actually joining from the web. And we are able to do this orchestration as well. So let's take a look at an example of how this would work. So here's an example of what we spoke about. In this case, again, we are just going to run this from command line. We have two emulators running. The host is starting off on first device, logging in, start a meeting. Meeting started, host is the only person over here. Now the first guest is starting, but in this case, though the application looks the same, it's actually using a different version of the application to install on that device. And the guest is going to connect from that. So in this particular case, the example is a little trivial because you cannot really see the difference in the applications. But believe me, and the score is actually there on GitHub. You could see these actually different applications, versions of applications used to run the test. So now the guest has joined the meeting that the host has created. And now we see there are two participants in the meeting over here. And now the guest is number two. The second guest is coming from the browser where the browser has launched connecting to the same meeting. And now we see there are three participants in the meeting and that orchestration can continue again from here. So this way you are able to implement real user scenarios across multiple devices, multiple applications, multiple platforms as part of the same test orchestration and achieve that end-to-end validation, what you would want to do in an automated way instead of doing this manually for every release that is happening. All the code that I ran is run from command line using simple properties provided as environment variables or directly at command line. And TestWiz just takes these and runs with it directly. You do not need to make any code change for this. Let's look at what else is there. We saw how the test can run, but just running the test is not sufficient. We need to be able to see what has happened in the test, what kind of reports are generated and what kind of value we can get from the executed test. Because just running the test is not important. Knowing the status of the test and what next do you need to do, that is an important aspect. So for this, TestWiz integrates with three different tools from a reporting perspective or coverage perspective. One is Report Portal, which gives you a holistic view of your test execution. It's a one-place stop for all your test execution. TestWiz also integrates with AptiTools visual AI and the AptiTools ultra-fast grid to get visual testing included as part of your test coverage. And the ultra-fast grid gives you increased seamless scalability across all different browsers and devices for your web-based execution. So it automatically increases the test coverage. You do not have to write as many validations for it. And another important aspect, a question which I get asked very often, what is the code coverage from your functional tests? We do not do code coverage. I strongly believe you do not get enough value or any value by capturing code coverage from your functional tests. Instead, what is important is to capture the feature coverage or the functional coverage. And let's see how TestWiz solves these problems. First is Report Portal. The way we define the scenario in your cucumber scenarios in given when then, we see exactly the same representation of the test executed in Report Portal. And you can drill down into these reports and see screenshots attached and impact visual test results attached and any artifacts that might be there, additional artifacts, log files. Everything is attached with the same test scenario. So when you come over here, if the test has failed, hopefully 99.9% of the time is what I would like to believe to do not need to run your tests again to figure out why it failed. We have captured the device logs, the browser logs. We have visual test reports over here. Everything is integrated in one place or is linked from this one place. So this is the only one link that you need to bookmark and use for your test result evaluations. In addition, Report Portal has an amazing capability for result analysis and trend analysis. So you can create widgets in the Report Portal dashboard, which can give you insights like which of your tests are flaky, how much time is the test execution taking, what is the reason of the test failures. So if you start taking decision on the fail test and marking them as a product issue or automation issue, data issue, environment issue, flaky test, whatever the categorization might be. As you keep doing this diligently, Report Portal will automatically do the analysis going forward and give you analysis of what is happening with your test execution in a very seamless way. So this becomes a very popular, powerful way of knowing what your test execution results look like and a trend of your execution over a period of time as well, which makes it very important to understand how your product quality is evolving. Apply tools for visual testing. Now, because we've integrated visual testing with this, I have to write less assertions in my code. The only assertions that I probably need to write in my code are business assertions. I should have seen the quantity as X instead of whatever right because that's a business rule validation that you're really doing. But all of your functional and UI based validation user experience based validation will happen from Apple tools and by simply choosing the right type of algorithms that you can use in your implementation itself. You can implement your tests in the correct fashion get the visual results automatically in that way. And of course, you can provide the ultra fast grid configuration in your test implementation itself. And with that, you'll be able to get as part of the same test execution, get increased coverage from all of your different browsers and devices automatically. Now, what this means is your test may have passed functionally, you're able to complete all the actions the way you want to. But there might have been visual differences user experience based issues, or your design is not conforming to what your baseline is, and you're able to do that validation as part of the same test execution. And this of course works across for native apps as well as browsers. In fact, it also works for desktop applications which I'll talk about shortly. The last part of the reporting is a feature coverage or a functional coverage. Because we are writing a test in cucumber JVM, we can add as many annotations as we want as part of our test, and these annotations are something that I recommend you add as functional components, or modules of your application. And when you run your suite of tests, your smoke, sanity regression, when you run all these tests, this type of report is generated at the end of the test, where it will give you a heat map of the tags that you have used in your test execution. And that way, you'll be able to say if I am a banking product, adding a pay might be one set of tests, transferring money from one account to the other might be another set of tests, viewing balance might be another set of tests. Now, adding a pay might be less important compared to doing a transfer between accounts. So with the heat map, you'll be able to see how many tests have used the tag for execution of account balance versus add a pay versus amount transfer. And if you see this is not correctly prioritized, the number of tests for add a pay is, for example, more than the others, you know your tests are not prioritized correctly, you are implementing and running tests, which are not more important to your business. So this is how you can actually get your feature and functional coverage visible from your executed tests. And hopefully that gives you insights into saying where you need to focus on more as part of your implementation. Okay. The next part is understanding what makes test was unique and why you should use it, consider using it. The unique capabilities that we have is of course, this is open source, and you can use it to automate real user scenarios and real user scenarios across for multiple user based applications multi device. Multi applications as well. It also has various cloud device farm integrations. So you do not have to run your test locally itself. You should be able to run it in device farms are also very easily. In terms of what type of support it has, it supports all the popular web browsers, mobile web browsers, Android apps, iOS apps and Windows desktop applications as well. The tech stack that is used over here is cucumber JVM. This is how we write the intent of the test. We use APM test distribution, which manages the Android iOS and the windows applications using APM. APM of course to actually do the last mile interaction with these devices. Selenium web driver to interact with the browsers. Report portal from a reporting perspective, a central reporting server. Visual testing using apti tools, visual AI and seamless scaling as a modern way of doing cross browser testing using apti tools ultra fast grid. And the build tool is gradle. In how do you run the test? We saw the command line that I used to run the test earlier. The command line is a way to override the defaults that you have provided that test with provides. In fact, there is a three layers. There are three layers, how the properties or arguments can be provided to test with in terms of what should be executed. First is the defaults that exist within test with itself. There are certain defaults that are assumed, but test with does not assume much. I started off saying test with is an opinionated framework or somewhat an opinionated framework. We do not do magic. We do not take decisions automatically. If there is something that is not expected, not provided explicitly, we would rather fail the test make the error very visible to you. So you are in more control of how you want to execute the test. So there are some set of defaults and you would provide the additional information required in form of property files and some JSON files as well for capabilities that are required for device interaction. And though I have property files, I could override that easily by providing environment variables for the same name and test with will automatically use the overridden values for execution. And this I believe is very important because let's say the testing team is implementing these tests, but you want your developers to also use this, but they do not want to run the full suite. They want to run only a few set of tests, which are important based on the kind of changes they have done. So just by telling them, okay, clone this repo, here is a command that you would run to run a subset of tests, they could easily run that as well. Or for that matter, if you want to run only a specific set of tests for investigation or whatever other reason that might be, you could very easily do that by just passing these arguments at command line or as environment variables. And that will override the property file values and you'll be able to execute it accordingly without making a single code change that is required. Okay, so this is what testvis is about. How can you proceed from here? There are two ways how you can proceed from here. One, you could be a consumer of testvis. You like what you see. You think it will add value to you. So the easiest way to get started is go to getting started with testvis. And this is a Kickstarter project. It has sample codes. It has the same sample test that I actually ran and showed in the demo. You'll be able to run the test, just clone the repo, run the test. You'll see how testvis works. And you could just start making changes and adding your own test over there instead of the sample test. And you're using testvis automatically for your project. The other way is help make testvis better. As I said, testvis is there on GitHub. So you could go to testvis, the GitHub site, and I have listed out over here a different set of enhancements or additional capabilities that I think will be important from a testvis perspective as features, as enhancements, as fixing certain things which might not be working as well over here. If you think you're interested in contributing to testvis, go over here. Let's collaborate and figure out how to implement some of these features and get it merged in testvis for your own use as well as for others. Of course, it's a age of social media. What is a use if we do not have more stars or more likes? The least you could do if you like this approach, like this thought process, go to GitHub, give a start to testvis and help make it a little bit more popular and give me and the other contributors of testvis more incentive to try and make additional capabilities available for you. With that, I would like to say thank you. And I hope to have more conversations with you, what additional things you would like to see over here and make it better. So with that, let's move to some questions. Pradeep is asking, you touched upon automation heat map. I hope this is the coverage report. If we use the right tag, will this tool automatically pick and show the coverage? Yes, Pradeep, this shows the coverage from the functional automation perspective. This is not the code coverage. Again, just to be very clear, this is not the code coverage. And if I go back to showing some of these, one of the slides, it's really whatever you have implemented as tags on the test. That is what is going to be used in terms of creating the heat map. Now, testvis also provides a very easy way for you to say which tags I do not want to show up. So for example, login might be a common module across 90, 95% of your test probably. You don't want to see that in the heat map. So you could very easily exclude that as well. Okay, so this is functional coverage. Absolutely spot on. This is functional coverage. It completely depends on how you have added these tags. Testvis will just create a heat map out of it based on the execution. Of course, what this means is that test has to run in order for it to create the heat map. If a test is excluded, then that tag will not be included in the heat map. Just clarifying on that aspect. Okay, so I hope that answered the question, Pradeep. The next question Saurabh is asking, does this framework have the capability to run the test in parallel? Yes, it does. So I strongly believe on certain principles, certain practices to be adopted in the framework to make automation framework usable, reusable, usable, maintainable, scalable. And one of the aspects is of course a clear test intent. If I don't know what is being executed, then everything else is futile to me. To get fast feedback, it has to run in parallel. So it supports it as an inbuilt functionality. You just have to change the parallel count, the default, you can set in the property value. You could change that value on command line if you want as well. So parallel test is there, visual testing is there. And because it is running from command line, you can easily integrate this as part of your CI pipelines and run it as often with any combinations as you want automatically without having to make different type of TestNG files or anything. There is no such manual configuration in TestWiz. So Saurabh, I hope that answers the question. Yes, it can run the test in parallel as well. And that is for web as well as mobile apps for that matter. Amai is asking, test data is very critical for end-to-end testing. How is that automated in TestWiz? Well, TestWiz does not do any automation. It's a framework that will give you the capability of doing what you want to do. So it supports making API calls. You will have to implement those API calls to say, how am I going to create data or query data from your application and use it in your test. TestWiz makes it very easy to define these aspects as part of your framework. So for example, if I go to the browser over here and I will open the Getting Started with TestWiz project, over here you will be able to see in source, test, resources. There is an environment station file. So your test should be able to run against multiple environments. Dev, QA, pre-prod, prod. So you can specify the environment data as a separate aspect. And your test data for each environment can also be specified over here that you will be able to use. Okay. The parameter that you are going to pass into TestWiz is going to be over here in configs. You would say, for example, if I'm running a local test for GeoMeet, which is my target environment, I'm using a prod environment. And again, you could override from command line to say, let me run the same test in Dev and see what happens. So the target environment determines what environment data is used and what test data is used for running the test. Okay. I hope that clarifies our test data. TestWiz does not create it. It gives you the capability to create it and keep it separate for each environment so that your test can run seamlessly across any environment that you want. Next question is from Prashant. We already have Cypress-based tests written. Can I integrate TestWiz so I can test across devices, measure coverage? TestWiz uses internally APM and Selenium WebDriver to run the tests. I don't see any easy way for you to run Cypress-based tests over here. But maybe that can be a capability that could be added to say instead of using Selenium, run it with something else. And this is just the driver that we are interacting with. How do you manage the driver? I frankly do not know how Cypress manages the driver instances. But maybe it's a conversation Prashant that we should have separately on the hangout table or afterwards to understand more. But assuming your test can run with Cypress, the rest of the infrastructure is still the same. As long as you create JUnit reports from your Cypress test, you will be able to create the heat map and measure coverage as well out of it. Pradeep is asking a question. Is this same facility available in plain Jenkins? I'm not an automation test expert. Excuse me if I ask the wrong question. There's no wrong question Pradeep. All questions are good questions. So don't hesitate. Any additional questions, please ask. Jenkins, in Jenkins you would create a job. A job, the way you would run the test is you would give a set of commands. What is it that you want to execute? That command is what I ran from command line. So yes, a very simple answer is yes. It can work with plain Jenkins as well. I have a lot of teams using TestWiz in a very large enterprise using Azure Pipelines, a build as well as a release pipeline. And they're using it for web as well as mobile on device farms as well as local devices. So this works fine across any CI tool because any CI tool, the only requirement they would have is give me a command that I need to run on a particular agent and the test should run from there. So yes, it will work. Okay. So I believe. Oh, there are some more questions and I think we still have time. So I will answer that over here. Does this framework spin up pods real time on GK? I'm guessing GCP instead of GKP or AWS for parallel execution. No, this is not going to spin up any pods or anything. Parallel execution is handled on the local machine where your tests are running. If you want to distribute your test executions, you would use a device farm where on the local you don't need any additional resources. A device farm will be able to provide a device to you, run the test over there or provide a browser to you, run the test on that. So that would work fine in that way. Okay. So it does not spin up any pods or anything. It's not required to do that. Next question, can we run test with against our application? How different is it from Selenium automation tool, which is already automated for our test cases? It is nothing very different because internally, TestVis is still using Selenium WebDriver. It is still using APM for native app automation. What TestVis is doing for you, it is abstracting away all the pain that you have in terms of managing the browsers and devices, managing the reporting in a seamless fashion, doing visual testing in a seamless fashion. You do not need to build any of that. You just focus on implementing your test, you have an approach for how data should be specified, how environment configuration should be specified. Just provide the data and you focus on implementing your test, getting maximum value of automation for your application. You do not need to worry about anything else. That is what TestVis provides for you. You could also very easily take inspiration from TestVis, the features capability it has, enhance your own framework to do the same thing. The difference is you have to build and maintain that infrastructure and that framework logic versus with TestVis. It is going to take care of all of that. Okay. So I hope that answers that question as well. And with that, I think we have a few more questions coming up. Interesting. Thank you. Can I use TestVis to test an end-to-end scenario which involves a desktop and an enterprise application? Of course, yes. The criteria only is going to be wherever you are running the test from. That needs to have connectivity to the application that you're talking about. So yes, you can do that. But if you want to have more concrete answer, we would need to talk more specific in person or over chat or Twitter or LinkedIn. And I'll be able to guide you how to make that happen. But yes, you can use it. Can we get the usage of it using any video or docs? Yes, I am building a lot of documentation. That is an area which is slightly weak right now. The existing set of functionality is working pretty well. But getting started, I am working on the documentation. I'm looking for a use case who wants to use it and I want to take help to create documentation based on actual usage by someone. And I would love to get participation from you and create documentation while helping you get started. That becomes a reusable asset as well. Okay. So it is possible. But the best way to go is right now is go to the TestVis GitHub. And over there, there is ReadMe. ReadMe. I'm trying to beef up as much as possible. I will add more videos of this as well to make it easier for others. Okay. Yes, we are done with the questions. Is there anything in chat? Okay, the same question from chat was there in Q&A as well. So I guess we have covered all the questions over here. So thank you very much for this opportunity. The questions indicate that you do see some value from TestVis. I would love to collaborate more with you. And I'll be on the hangout table so we can continue the conversation over there as well. Thank you again very much for the opportunity and look forward to working with you in the future as well. Have a great rest of the conference. Thanks, Anand, for sharing your experience about the automation tool, the open source automation tool TestVis.


So, hi. My name is Jose Evertse. That's the pronunciation. I'm a DevOps engineer. I'm very excited to be here. It's great to be able to travel here and to see what San Francisco is all about. I already had an interesting day with all kinds of presentations. I hear a lot of similarities with the story that I'm going to tell. I live in the west of the Netherlands, near the sea, near the Hague, with my wife and two daughters. I'm in my 40s. I've been in IT since 1998. I've worked at startups and also at several banks and telecom organizations. During 2018, I wrote a book about GitLab 12, Mastering GitLab 12, and how to use it in your organization. I'm currently in the process of starting a CICD branding company called CI Academy with a former colleague from ING because we think there's much to be trained at big companies about CICD still. So, while I've worked at startups, I also worked in enterprise organizations. And I want to tell you today about how we experienced scaling GitLab in a big organization, ING Bank. You've probably heard of it. I want to show you the difference between DoomNet and a small company. So, first, let me tell you how we skilled GitLab at App7, where I first worked with GitLab. Imagine Cloud7, a telecom services company based in the Netherlands. In 2014, I was part of the engineering team. This company started in the 90s and was looking for new markets. They started in the 90s with the management of telecom networks and private bronze exchanges. And we already switched to supporting managing mobile devices in 2014. But now we want to go into the app market. The owners of Cloud7 decided to start a subsidiary company called App7. What's the name? The goal was to begin a real app development agency and to do it as fast and cheap as possible. We had to choose a technology. We chose web apps with the React framework. At Cloud7, we were used to using CVS and SVN as a versioning tool. We liked the command line. I'm a newness guy. But we hired new front-end developers. And those guys, they really hated that. So, they wanted the newest stuff. So, we started to look for a Git server-based solution. So, we wanted it to be possible to use it on premise. It had to be open source. And it had to cater to all kinds of users. So, GitLab was our natural choice. It was the best option for us because, of course, you can install it on premise. It has a web interface that most people can learn to use. It's quite easy. And it's open source. It's free to use. So, we could use it for our business. It took a couple of minutes to make this decision to use GitLab. And also, a couple of minutes to install the Omnibus package. It's very easy. So, I'll give you another example of scaling GitLab in a small company. This is how we use it at CI CD Academy, the company, the training company that I'm starting. We started out organizing small courses for developers. Using the portable desktop you see here. We took it along. And we used Raspberry Pi's. You know them as well probably. We used them for the practical exercises for developers. It works very well for a handful of people, but to handle bigger groups of students, we needed to scale. So, we created an automated learning environment using Kubernetes. So, I mentioned Kubernetes in the school.  And these examples show how it is really great to start small and use cloud-native technology to scale up your business. And now, for a different ball game, a GitLab on an enterprise scale with thousands of users. We already heard various comments about that today. My story will focus on how we as a GitLab team at ING experienced the growth of GitLab from 2,000 to 12,000 users. But first, some info about ING. Just the biggest bank in the Netherlands. There's an annual balance sheet of 845 billion euros. It has presence all over the world, also in the U.S. In 2015, ING started an agile transformation, where they focused on creating an engineering culture. They want to be the Google of banking. How do you scale this culture? They introduced a Spotify model in the organization. It gave more autonomy to teams and introduced all kinds of other interesting ideas. Teams are being called squads, for instance. I was in the code squad, that was the name of our GitLab team. But you can feel several presentations about this subject alone. I won't do that. I will focus on CI CD, because it's part of the vision. And it included the use of GitLab for all developers. In 2017, I started a temporary assignment at ING again. This time, it's a CI CD engineer at the department that was specifically set up to provide CI CD services to all developers within ING. The department was called CDAS. It's an abbreviation for continuous delivery as a service. Here you see the basic overview of the CDAS pipeline that was offered and that existed in 2017. There have been some changes, but the concept has not changed. You see a Java Linux flavor, you see the penguin, and a Windows flavor with Visual Studio team services. There are different phases, code, build, deploy, etc., and various products are used. GitLab was chosen for the code phase as a code repository and collaboration tool. Then you see a stateless build server with Jenkins to build the code, and the artifacts are stored in the artifact. The deployment, test, and release phase used to be handled by Nolio, but in 2018, they switched to a combination of Ansible Tower and Visual Studio team services as an orchestrator. In 2015, when CDAS started, they had to choose a product for the code phase. There were three options, GitLab, GitHub Enterprise, and Bitbucket for code versioning. But GitLab was chosen. Why? Because feature richness. It had much more features than other products, and it's actually still the case right now, I think. The authorization model that could be used inside GitLab, mapped very nicely on what ING wanted. Very important for us was that the on-premise possibility, also here, we could install GitLab in the data center of ING, and the other products could not. Well, GitHub Enterprise had an option to install an appliance, but it was a black box, and ING didn't want a black box in their network. So, let me tell you a bit about how our teams at ING used to work with building software. I'm taking you back to around 2001, when I worked at ING as a Unix system administrator. My team was responsible for Giltel online, that's Internet banking in America, and mobile, and banking. This was before agile. It was way before DevOps, the term was coined, and you see here that silos existed. Everybody did double work. Communication was terrible. In the end, it made it all very costly and projects took a long time. This does not scale well. Fast forward to 2015, when CEDA introduced CEDA's Linux and Windows pipeline. It does not take much imagination to see that this is much more efficient. The most important thing I want to show here is that the lines from the teams come together at the pipeline. It means that collaboration was getting better and breaking down silos. GitLab simulates this kind of work, and what happened was the team started in the source and components. We heard a lot about this in other presentations as well. They shared code and components. It meant that inside of IG, there was more open source software within the company. Communications between developers shifted from email and big systems to a matter-most instance that was installed alongside GitLab. Another effect was that developers started to help each other use our pipeline, and that was a very beneficial effect. Also, the shared pipeline had APIs that we built for gluing different tools together, and all developers started contributing things like monitoring dashboards that used that APIs, and also other automations like the requests for SSL certificates, for instance, or the renewal. There was a presentation at GitLab Commit London by Fabio Hoeser from Siemens, maybe you've seen it, and that shows also this picture. They had the same evolution, so to speak, at code.Siemens.com. Today, I heard several stories as well, but this picture could be used for that as well. To give you an idea of how we plan to scale GitLab as a team, let's look at some diagrams. These were actually used by my team in sprint planning. It used to be talked about and made the planning. We had a simple GitLab setup in place in 2017, just a reverse proxy and application server and a backend database server. The next step was that we created a copy of this configuration in another data center, so we created a DR setup. We used rsync to replicate files and repositories, and we set up a streaming world application to copy data from the production postgres database to the DR site. The DR site was called standby mode, and we had a switch script that we could flip to the other data center in case of an emergency. We planned to scale further in 2018 to the situation you see here with multiple database clusters, load balancers, Redis clusters, shared file systems. We started with pulling Redis out of the picture and creating a Redis cluster that worked perfectly. We did it in two weeks, I think, and nobody noticed. It got a bit faster, maybe. We then started to test cluster FS as a shared file system, and we used a service offered by the infrastructure teams that was called, that was cluster FS, and we were the first internal customer to use it. The initial design and calculations indicated that performance was comparable to what we had already, so we said, okay, that's cool. But unfortunately, in the summer of 2018, when the whole new environment was set up by those guys from our teams, we noticed that the performance was not good enough. It was worse, and that was impossible, so we could not migrate to this new cluster FS solution. So we celebrated our failure with a barbecue of my place. This is my team here, you see, and my barbecue as well. But you got to eat, right? So I'm getting a bit hungry right now, actually, seeing this. After that magnificent event, we had to change to a plan B, and that was that we decided to migrate all of our machines to the ING private cloud. What is that ING private cloud? Well, it looks a bit like AWS or public cloud of your choice, but you can get computing, storage, and network resources with self-service provisioning so that it looks like that. It had been on our list list for a long time, but we didn't do that earlier because management was not really in, yeah, they didn't like that because there were some stability issues in the past with the new private cloud offering. As a team, earlier felt that we could not migrate to this system because our automation tools were not yet good enough. We thought. So we were out of options because cluster FS was not going to cut it, and FS also not, and other options also. So we decided to go for it, to migrate to this environment in August 2018. We managed to finish the migration in the fall quite fast, actually, and the big difference with the former situation is that we had all installations done with Ansible, everything, and all the infrastructure components are defined in playbooks, in GitLab, by the way. And it was proven that it worked because in the end of 2018, we had to use, we had to scale up a bit more CPU memory RAM, and we had to get much more GitLab runners there, you see in the picture. We only changed some parameters in Ansible variables. We ran the deployment scripts with Ansible, and then we had more components in ING private cloud working. I left ING in the beginning of 2019. So for me, this infrastructure is the end state. But I heard that a month ago that they were already scaling up again more runners. The amount of runners is exploding. So what are the hurdles for pushing GitLab company-wide that we saw? Ironically, GitLab was too successful. If you look at this user growth diagram, we started with under 2,000 users. And when we started growing users, we had issues with the hardware that GitLab was running on. We couldn't add more CPUs, more RAM, or more storage to it. So that was a problem. The growth of users also meant that we had more support work. We were a three-man team, so remember that. Actually, we started with a two-man team, even. But we got more support work as well with more users. And also, because it's a global service, it was 24-7. When we found out that support was going above our heads, we created a first-line support team in Poland with the colleagues from ING in Poland. So that relieved some pressure of first-line support issues. But still, a lot of the second and third-line, we had to do that. We were responsible as an autonomous team to manage that. Another effect of being too successful is that, and also more users, is that it was getting harder to secure maintenance windows. There was change management in place in the bank, so we had to get sign-off approval for all kinds of changes. And more users and more departments, it was getting harder to get maintenance windows. So that was for us quite a hurdle. It's politics. There were a lot of reorganizations in the last couple of years at ING, which are not per se bad. But what happens with these organizations, we had a lot of overhead tasks, we had to do for that. Also, that got on our backlog as well.  So we got some knowledge dispersion around different tool teams within the department, not so smart. Also, we had new people coming into the GitLab team, and then we had to train them with all the sysadmin talks around GitLab and how to use it as well. So that was not very handy. And very important, rules and regulations. Of course, IG is a bank, and a bank is a regulated industry, but that really is actually at odds with innovating and growing. So what does it mean scaling in a heavy regulated environment? Why is it so hard? Well, in 2017 and 2018, at ING, there was still cloud paranoia, even though cloud infrastructure had been around for some time. And you have safe hardware agreements, and you have all kinds of technical solutions to keep your data safe and your trust code. You have to guarantee that your customer data is not being seen by anyone else. You have to approve it. It was the reason why we did not migrate to the public cloud for scaling in 2018, but to the ING private cloud. Our backlogs could have been full of cool integrations and new features for our department pipeline, but also we had to do proper risk management. We also had stories on risk and innovation. So there was a trade-off between innovating and being 100% compliant. We actually had to spend 30% of our time officially on risk and compliance work. And in practice, it was even more. And then if you look at the support work that also came on that, you have a very little time for innovating about the tools. I've heard that before today as well, tool fatigue. But you see a lot of different tools and integrations in regulated industries. At banks, there are a lot of legacy applications. Much was invested to build them and to keep them regulatory compliant. It's not easy to scale them in a modern way. And replacing them is also not an option. This is partly because of the law of the handicap of a head start. Does anyone know what that is? I don't think so. I think it's maybe a Dutch thing. It's from a paper from a Dutch historian. And it means that when you innovate and when you create a head start, then it's very hard to make that even better again. The example is the mobile banking industry in Kenya. Maybe you've heard of it, M-Pesa. It's been around for some time. And there they could pay with mobile phones 15 years ago with SMS. And they totally skipped the internet banking. And they just could pay with a phone with a very simple mobile phone already where we could not actually in the West. I've seen that personally myself also in Zambia. I'm not sure if you heard of that country. It's also in Africa. I worked there as a consultant for the Zanaco Bank. That was really cool. And there I witnessed that I had a look at their backend systems. And they skipped the mainframe totally. So they only have a box, a Java application server, running all core banking services for internet banking, credit cards, everything. And they do it with FlexCube. That's a Java application server that was built in India. And now actually, I think it's part of Oracle Bot. Oracle has that as a service offering. So that's an explanation, a bit of the law of the handicap of that start. If you have all things built, it's very hard to throw it away and start over again. So what does this all boil down to? Enhancing scalability in enterprise. Can we say something general about this? Well, what are the factors that positively influence the scalability? You need executive sponsorship. I've heard this many times today. It's the most important thing. An example of this in our situation was that before 2018, our management didn't want us to migrate to the IG private cloud. So it immediately was not an option anymore for us. But when they backed us, we had the green light. We could fill our backlog with all this stuff needed to migrate. And we could also get help from other departments. Corporate culture. So we changed plans after the Gloucester VS failure. But that was okay. It's okay to fill. We experimented and we shifted. If you are afraid to change, you can't scale. You cannot innovate. Long-term thinking may be contrary to agile practices. But organizations, also IG, still move more slowly. Until this changes, don't reorganize every month. Use your involvement. At IG, developers are the biggest advocates for GitLab. We actually, we use them for support. Because we are only a three-man team. But when somebody had a problem and they posted it on the Mattermost support channel, our channel, other people, other developers started helping as well. Even before maybe I saw it, actually. And they helped us migrate to the private, IG private cloud as well. The guys from the infrastructure department. And they were GitLab users also. Training programs. Very important. You have to keep them up to date and change them. Because if you grow your, scale your software, you also get different kinds of users. So you have to change your training program accordingly. It's very essential. It's the basis of our new company, actually. Automation. Very important. This is the way, I'll also read this in the former presentation again. A lot of themes. That automation can get the problems with risk compliance out of the way. If you automate all risk compliance stuff, you have, in our case, 30% more production. You can do more production things. Uber communicate. Well, also, I think that's obvious. Because your users have to know, you have to know what you're doing. You have to prepare changes and communicate with them. We use service now and we wrote blogs internally. We emailed every stakeholder about what the changes were that we were going to do. And also, afterwards, we evaluated them. And for communication with developers, we used the MetaMOS channel to announce all kinds of things, changes, handle support, experiments with the exclamation sign. In our case, we only did one experiment. And I encourage you to do more than one experiment, because you see the clusterFS experiment failed for us. And we didn't have the time to do more experiment. I really wanted to do more NFS experiments, but there was no time. So, experiments. In my book, I talk more about several scaling scenarios which could be useful in enterprise, especially using Ansible, which I have used at ING, and also then combined with Terraform and AWS Cloud. But it's also applicable for other clouds. Well, this concludes my presentation. And, yeah, thank you.


I think something that the Navy Marine Corps team and with our Coast Guard folks embarked as well, have been very good at over the years, is responding to crises or events, any place in the world at any time. Whether that means we've had two years to prepare for it, or in the case of SP MAGTF ROOT, two months, we will find a way to accomplish the mission. The logistic combat element, they have the bulk of capabilities that would really alleviate any sort of disaster or provide that humanitarian assistance. The other part is going to be capabilities with our Peruvian partners. Essentially, we are the people who make the trains run on time. In a disaster, we can provide electrical support in purified water. We have a medical capability from our sailors, our corpsmen who we have with the unit. Our ability to move supplies and equipment via helicopter with an external load. It's in our DNA as Marines to accomplish the mission.


Most about sunny gray. I mean, that's that's that's a peak right there at what what he can be when he's at his best. I mean, he was on the attack all night. You know, when he did need a big pitch, he was able to get it. Whether whether it was with the fastball he mixed in some of his splits and the breaking ball was was dynamic. So I thought his stuff was great tonight. Just just a pace and an energy on the mound and just just really on the attack. And, you know, we weren't going to go much more than 75 pitches price.   Do you think that aggressive approach that attack mentality, the bullpen helped him get there a little bit? You know, who knows?  You know, it's not the worst thing to go through and you learn a lot of things about yourself. And as I've always said, you know, he's a guy that is in the prime of his career and has the equipment to be a really good pitcher. And we've seen glimpses of it. And we saw a glimpse of it tonight. Reminds that on the field, he thinks that's the best he's ever seen his slider from your vantage point. Did you see the same thing? Yeah, I even walked over to Robbie once early in the game. I said, that's when you throw your really good one, it kind of looks like that from the side. You know, it just goes forward from a long for a long time. It's late, it's sharp, it's big, and you see a lot of funny swings on it. The records are double headers are not easy to see. So no, get this and yeah.  It is huge for us and and hopefully we can finish off a great series tomorrow. There are great numbers are so different between home and away. Do you think he puts too much pressure on himself at home or it can relax? I don't know, I mean, look, I think there's probably been some of that that's that's gone on a little bit this year. But you know, I think there's there's been some ups and downs for him that, you know, may have coincided with being at home. And so I know that's become a big story. But the bottom line is when he's going out there on the attack, he's capable of nights like tonight. Anything to if you just put him back in the bullpen. Yeah, yeah, he'll probably go back back into the bullpen.  And we and that's one of the things we talked to him about, you know, when we first put him in the pen is, you know, this there could be a lot of different situations you find yourself in. We saw him in a high levered spot and extra innings. We saw him, you know, trying to give us innings in a game where we were down. Tonight we needed a big start. We got it. So I'm sure it's a role that maybe continues to evolve. Anything to him when he came off when you took the ball from him? Nothing too poignant. Good job way to go. I mean, pitch great. He was he was terrific because of everything he has been through the ups and downs. This is in the innovation, a little more meaningful for him as he walked on. You know what? I don't know. I noticed it. I actually turned around on the mound and saw it. And I was I even said something to the guys out there like that's, you know, that's pretty cool right there with with all he's been through. And just, you know, to see the support he got. That was hopefully a nice moment for him.


Meat Boy is back! Today we're making a carnivore version of a chicken fried steak and traditionally you pound out a steak really thin, you bread it with some breadcrumbs which is wheat which is a problem then you have a milk and egg batter which is actually carnivore back into the breadcrumb and then after you fry off the steaks you take the fond what accumulates in the pan from the frying and you add a little bit of milk put some flour in that and then you have a nice gravy so we're gonna use a couple culinary techniques and substitutions to make this carnivore overall I think it's a delicious healthy approachable recipe that doesn't use too many ingredients. The steaks we're using today are some of the sirloin steaks from Frankie's Suranged Meat these are four to five ounces and really nicely portioned out this is gonna be perfect for us to pound thin of course we're seasoning with salt this is some of our finlandia grass-fed butter we have on Frankie's Suranged Meat by all means I would just get local raw butter if you have access to it for the breading component we have some raw milk from a local farm some farm eggs and some Parmesan cheese that we're gonna use to substitute the breadcrumbs very nutritious and this is where a lot of the fat soluble vitamins in the meal are coming from of course when it fries it's gonna absorb some of the butter which has additional fat soluble vitamins and just in general you know steak great source of protein B vitamins amino acids and certain minerals for the gravy since we don't have flour to thicken it I'm actually going to use cream as opposed to the traditional milk and I'm just gonna add a tiny tiny bit of beef broth to add a little bit of flavor of course this is raw from a local farm make your own if you can I'm just not making beef broth right now it's pretty labor-intensive and this is some colatura we have on Frankie's Suranged Meat it's basically garum fish sauce this can add a little bit of depth of flavor to the gravy if you so choose so let's get these steaks pound it out let's get this cheese grated and then we'll be ready to cook so I padded the steaks dry with some paper towels in hindsight I should have grabbed some filet mignon for Frankie's Suranged Meat it's a bit more tender easier to pound out but sirloin is great ribeye is great really just any steak that is already you know fairly thin and properly sized so I'm gonna use parchment paper and we're going to pound this out a little thinner you want to go for a quarter inch thickness traditionally but if you like your meat rare medium rare you might not even want to pound this out much at all yeah and part of this isn't really getting this that thin it's more about making sure the meat is even you see how this is a bit lower than this over here all right that looks good we got our steaks pounded out to about half inch thickness for the Parmesan cheese you want to grate this on the finest grating possible you could do it you know I guess on a thicker one if you really wanted to and it wouldn't make that much of a difference but for presentation and getting the breading to stick go as fine as possible most of our prep work is done all we have to do now is make the wet batter for the chicken fried steak so we're gonna take our one cup of milk I'm gonna crack two of the farm eggs in here and then I'm going to put about maybe a teaspoon of salt in that because this is gonna be how I'm seasoning the meat I'm not putting salt on the meat I'm not putting salt on the breading we're gonna put it in the wet mixture whisk this up you could add garlic powder onion powder paprika black pepper a bunch of different seasonings to this if you'd like and it would definitely be more approachable more traditional to the average person maybe two tablespoons of butter in here medium medium high heat and we're gonna take the steak put in the parmesan really try to cover the surface here and it wouldn't really be unhealthy if you use like high quality organic breadcrumbs by no means but this certainly more nutritious and less inflammatory for a lot of people that are having gut issues or are trying to avoid wheat so from the parmesan we go into the egg mixture trying to be gentle to not get too much of that parmesan off and then back into the parm for a second dip and most recipes call for breading this twice but I'm sure you could go you know three or four times if you wanted to this looks like it's gonna be plenty of cheese the difficult part is gonna be getting this cheese to stick to the steak as it cooks you know with a lot of these carnivore recipes when you're substituting ingredients the real main thing you have to do is just be gentle with the product because the texture and ease of making a traditional recipe isn't there and the parmesan tends to clump up so between steaks you just want to break it up a little bit with your fingers we'll do the same thing first coating nice and light don't really try to pack anything on it even shake it off a little bit nice and gentle into the egg again and then when we go in this time we really want to cover it you know I was in Whole Foods the other day and they had some organic breadcrumbs and organic panko breadcrum so that's probably a decent option you know I mean I was looking at the ingredient list and they do have a lot of additives so if it was like homemade breadcrumbs with your own bread that's one thing but that's a little labor intensive and difficult to do so we want to kind of pat the parmesan cheese into the steak so it stays our pants heated up the butter is actually browning a little bit we'll put this one in nice and gentle and we'll put this one in as well I actually want to turn the heat down a little bit because the cheese will burn very easily maybe even take this off the heat for a second now I'm not too worried about the internal temp on these steaks the main thing I'm worried about is getting the crust at the bottom to set without over cooking it and you guys can even see it looks almost like the cheese is kind of sticking in the pan a little bit so we'll see if we could get this to stick on the steak what I'm gonna do is I'm gonna leave it at a medium heat for a minute or two I'm gonna really try to get under there and scrape it to make sure it doesn't stick alright so when I shake the pan it's kind of stuck which I'm a little afraid about but let's let's see how this goes that's actually not bad you can see it was like peeling off a little bit there so you definitely want to be really careful with this crust so we'll take these out so now we have all of these drippings and caramelized pieces in the pan if you want to cook more steaks like you have the other steak we have to cook you want to refresh the pan because if you put that steak in this pan now it's just not going to cook properly there's too much crust and bits it's it's not going to crisp up evenly so what we want to do now in this pan is make the gravy put a little bit of beef stock in here maybe about quarter cup scrape everything up and I want to boil off that beef stock okay that looks good now we're gonna put in a little bit of heavy cream maybe a quarter cup don't have that much steak here so this is basically your gravy you can reduce this to whatever thickness you would like that's the difficult part of not using flour you can and use up quite a bit of liquid to get this thick enough all that really means is an extra you know five minutes on the stove gravy has thickened up a little bit that's how we'll serve it I'm sorry my taste tester Gina is not available today but Frankie boy will have to do so let's check the temperature we got on the steak and this is pretty much a perfect rare in the middle so it could have definitely got a little bit thinner on the steaks if you wanted to I mean it tastes like you would expect like you fried a bunch of cheese and put it on a steak and it's actually pretty good I would say this isn't actually that close to a traditional chicken fried steak recipe and flavor or texture mainly because you have so many strong flavors from that Parmesan cheese it's still delicious the gravy is really nice but again if you don't like Parmesan cheese you probably want to substitute a milder tasting cheese or maybe just opt for some organic breadcrumbs or something of the like so as we showed earlier all of the ingredients for this are actually on Frankie's range meat you have the butter the Parmesan cheese as well as the steaks but I hope you guys try this out and let me know how you like it so thank you guys for joining me today if you do want to support me you can check out everything available on frank-dafano.com we will be doing a live stream later at around 3 p.m. Eastern time on my second channel Frank to Fano so I'll see you guys there


President you've asked you have the Sultan of Al Maan You have a paper If you and I went over in front of the fireplace We'd change our papers and we would get in and then You were happy with all the coming You would have a picture Please to have you back here I'm very pleased with the relations that we have. I hope we can continue to work together to press the United Nations and see if resolution 598 cannot be followed by a second resolution. I'll declare an embargo. Thank you very much. Thank you. I think if they were between us, it would be much better. Mr. President, the Ambassador Panama. Mr. Ambassador. Mr. President, it's an honor to meet you. It's an honor to see you. It's my pleasure. And Mrs. Sosa? Pleasure. And Lisa? Lisa? What do you want to say to everything? Go after Mr. President. Pleasure to meet you.  It's here and then you will come over and join us for a picture. Thank you. President Del Valle? Thank you. I know our relations have been a little strange lately, but I hope that we can get back to the solid friendship that we've always had. I'm sure we will, Mr. President. The reason I'm here is because I feel that we should take all the steps necessary to strengthen our relations. We have a long tradition of friendship and understanding that we have the will to continue doing so. Between us, we created one of the world's great undertakings. Yes, sir. Cut out. Well, that's wonderful.  Right here. Can I just get a look at me? Do you want to look at me? Lisa? Okay. There you go. Thank you very much. Pleasure, Mr. President. President, Ambassador Columbia. Mr. Ambassador. Senor President, it's an honor to be here. It's an honor to meet you, Mr. President. We're honored to have you here. Thank you. We are honored, Senor President. Thank you very much. It's an honor. And all of the family. Come in. Thank you very much. Water. Nice. Sun. Well, hello there. I'd like to go over there. See you. Hello. Hello. Hello. Mr. President. Hello there. How are you? Good to see you. Thank you, sir. How's that? Good to see you. That's everyone. Well, I think we... I think we... I think we... I think we... I think we... I think we... Hello, Mr. President. Excuse me, let me get out of your way.  Thank you. All right. So Mr. President, you're miscarriages. All right. Thank you, Mr. President. And I want to say I admire all of you in your courage and our mutual value in progress. We're going to continue from our end trying to remove the market for them. It's true, Mr. President. We've made tremendous sacrifices in the fight against the joint traffic. Many lines have been lost. Some of them lies against the illustrious people, ministers, magistrates, judges, high-ranking military police employees, and other public officials. Often, those efforts that we have carried out have not been acknowledged properly. At times, the position of Colombia and its government has been misconstrued. We will continue in that struggle with all determination, and we will try for Colombia and its justice to act quickly and effectively. And we also determined to comply with our commitments at the international level. Even though the Supreme Court of Justice of Colombia declared the law approving the treaty entered into the United States in 1979 and constitutional, but there are other treaties, and Colombia understands that treaties make it possible for an independent country to be reached in this way at a given time. It would be possible to carry out the extradition of people who have committed crimes in the United States living in Colombia. In this regard, Mr. President, so as not to use up too much of your time, I would like to leave you with a summary of the main points of view of the conflict regarding the matter of drug trafficking. Thank you. I'm pleased to answer. And as I say, we admire your courage, know how very important this all is, and what a threat it constitutes for your country. And we're going to continue to cooperate. Thank you, Mr. President. We understand that drug trafficking is not only a threat to the United States and to Colombia, but to all of mankind, and we feel that in that struggle we have to involve all the countries of the world. It's not enough for there to be extradition treaties. We also need to make a tremendous effort so as to control consumption. Yes, yes. It's not enough to attack those selling the drugs. We have to remove the market. I am sure of that. All right.  It could be a great honor and a great pleasure to meet you. My pleasure. Thank you, sir. Thank you. Thank you. Without you, it would have been very quiet. Bye, Mr. President. Goodbye, sir. Bye. Mr. President, Ambassador of the Federal Republic of Germany. Mr. President, thank you very much for giving me this welcome. Thank you so much.  Thank you. Thank you. Thank you. Two letters recalling me and my speech. Mr. President. All right. Well, we have a fine relationship, our two countries. Thank you very much. Within NATO and all, and I must say that you have been most helpful with the Intermediate Range, our INF Treaty. We're most grateful. Thank you, Mr. President. Thank you, Mr. President, and the Federal Chancellor. And they attach greatest importance to our relations and they saw me before and instructed me to really do my utmost and look after it well. Which I pleasantly do. I've been a full white scholar here and I look very much forward to working with them. Well, and please return my greetings to them. Thank you very much. I think we're going to get an active, active year. Mr. Gancho is planning to come as the chairman of the 12 year chair of the European Community. And the Federal Chancellor, I think, told you he would be coming in February. So they could keep the embassy busy. All right. Yes. Thank you very much. Thank you so much. Mr. President, the Ambassador of the People's Republic of the Congo. Mr. Ambassador, welcome. Thank you. Please have your seat. Hello. And the children. All of the children. Hello. Hello. Well, why don't you and I go over there from the fireplace and use your papers. And then the family will join us for a photo. So Mr. Ambassador, you and I, we're going to go in front of the fireplace, exchange our letters, and then the family will join us. We admire very much what your President is doing with regard to the economic changes and so far as we wish him well. Mr. President, I'm honored to be able to return the letter to President Sassoune-Greçois, my credit to your excellent quality of Ambassador XA, an exponential of the Congo. Mr. President, I have the honor of giving you the letters of credence by which my President, Mr. Sassoune-Greçois, credits me to you as the Ambassador of the People's Republic of the Congo. So it credits me to you as the ambassador for this opportunity to take an opportunity of this occasion to transmit his best wishes for the year health and the prosperity of the American people and the year happiness. Well, thank you and please give him my regards and my thanks. I'll be your faithful interpreter, Mr. President. Thank you very much and welcome. And as I say, I look forward to having a closer relationship with American countries. It's a great honor for me, Mr. President, to be the representative of my country next to your great country. We are pleased to have you here. Now I think if you were in the middle here, maybe one of us on either side. We're going to run this show to make it's outlaw a bit. And I would be pleased to be able to work with you.


Hey, did you know there are three major things that if you're able to fix this will greatly improve your chances of Getting your disability claim picked up. That's right There's three things that most people get wrong. I'm gonna share those with you. My name is Tony Vieira I'm an attorney with 22 years of disability experience And I'm gonna share with you these things that if you are able to fix them will greatly improve your chances of Getting picked up for disability benefits number one make sure you are applying for the correct program The social security disability program has two major programs that qualify you for big payments Number one is based on your work record. The other one is SSI that is a needs-based program The problem is if you apply for the wrong program and you don't have the work credits It doesn't matter how injured you are or how ill you are you will not qualify Number two make sure you thoroughly complete your application It's real important you give social security the information they require in a way that helps them understand Not just what your conditions are, but how they affect your ability to work Number three the most important thing if you are denied it is important to know This is not the end of the road most disability claims are denied You have the right to appeal that decision instead of just starting over again You can even appeal it to have a judge sit down speak with you and hopefully make a much better Decision if you need any help with a disability claim if you want somebody to talk to and evaluate your case Or if you know anyone that would benefit from watching this video, please give us a call and feel free to share. Thanks