Hello, and welcome to our workshop about running the Beacon Chain Explorer on your own. We will go through how we came to the idea to make the Beacon Chain Explorer, and then we will explain how you can actually run the Beacon Chain Explorer. We, this is Stefan, my colleague, and me, Patrick. We both work for the company BigFly. So yeah, first we all, Stefan will introduce you to our company, and how we actually came to the idea to make an Explorer for the next stage of the Ethereum project. Then we go briefly through the architecture of the project, and finally we will show how you can run the Explorer on your own. But before we start with anything, everyone who wants to participate, like, interactively, we need your, ooh, sorry, that wasn't the wrong laptop. Yeah. We will go first through how we came to do the Beacon Chain Explorer, and what the architecture of the project looks like, and finally we will show you how you run the Explorer. But before we go to anything, we need your, anyone who wants to participate, we need you to download a few things. So are there anyone who wants to participate on his laptop? Do we have installed Docker in Docker Compose? Sounds fine. Yeah. So can you basically try and see if you can access the repository, if we made it public or not? Yeah, then I'll quickly, we'll jump back to that, and I'll quickly introduce kind of who we are a bit more in detail. So the company Bitfly was founded in 2017, and we've done a range of products. We've been basically active since the beginning of Ethereum in the space. We actually created one of the first block explorers, but kind of let it slack a bit. So that was etherchain.org. It's one of the older ones. And then later on we kind of regretted not putting more effort into that explorer, and then we kind of decided now that there's going to be Ethereum 2.0 or basically the merge, we decided to create another explorer, specifically for phase zero. And that explorer kind of ended up being the beacon chain explorer, and it started out where everyone could basically see all the information about the beacon chain. Maybe a show of hands who has used the explorer before, maybe just to kind of get a feel. So about half of the people have used the explorer before. So maybe if you want to show of hands who's running a validator. Okay, quite a few of the people that are using it. So basically we wanted to create a nice place for you to understand what's actually going on with the validator, to see the different states of the validator to kind of make sure you don't get slashed. And we also operated one of the bigger mining pools called ethermine.org that was now retired since the merge, and we kind of shifted away from mining a bit and focusing, we're focusing a bit more on staking. So kind of to shill a bit our other products, we have staking.ethermine.org now, where you can stake with less than 32 ETH, and we have ETH pool.  And then a brief history, I kind of already touched on that. So it started out in 2019, end of 2019 with the first test networks. So even before the genesis of the beacon chain, here you can see a small or kind of nice picture of the genesis event. We kind of created like a slot view. So each one of those rows, if you don't know, is an epoch and in one epoch there are 32 slots and here you can kind of see the green part is when it was proposed, the red one is when it's missed, and the yellow is if it's orphaned. And that's kind of a nice overview of the first few epochs when the genesis happened. And we also have like a checklist below how much finalization we have, a participant patient we have, and if the epoch is justified and finalized. And a nice rocket for like the countdown when genesis started, which is also pretty nice. So that's kind of how this whole project began. And it kind of exploded a bit more and more and with the merge we tried to add more information, not just the phase zero information and it kind of grew and grew. And this presentation will also be kind of about how we kind of handled the scale and also give a kind of a workshop where you can help us with kind of finding ways to scale better. Yeah, so let's go a bit to the architecture. So in the beginning this is kind of a very simple view what the Beacon Chain Explorer looked like. I also presented this, a similar slide. I think it was ETHCC a year ago where we had just the Prism node, we had Infura node and the Funeral node was mainly to get the deposit data from ETH1 and then we had an exporter which basically grew out everything into Postgres database and that worked really well in the beginning, it was really easy to work on and then we just had a Golang front end with some templates that kind of served everything to the end user. And even when I presented it back then we kind of can see that Prism already needed like 576, 67 gigabytes of data, Ergonode already 1.4 terabytes and Postgres already 2 terabytes of data and I looked at these numbers again, how it's running today and kind of a reason why we had to scale a bit more. So the numbers now that we switched to Lighthouse and we have a 32-slot sync so there's different kind of sync versions that you can choose and depending on what amount of slots you choose the more information is stored but the quicker you can retrieve information and we basically want to store everything and that's already a 5 terabytes disk that you need for that Lighthouse Ergon is 2.1 and our Postgres database is huge so kind of the tables and the indexes that we have in the Postgres database they just don't scale that well so it's already 10 terabytes big and we don't only run the mainnet we also run the testnet and the testnet is just that amount and even a bit more for different testnets so it's a huge amount of data and it's getting a bit expensive running that in the cloud so we're going to talk about how we kind of started migrating away from that and basically the scaling challenges during the merge so this is kind of an analytics view that we had so we had concurrent sessions of 2,267 people that were looking during the merge because we had like this nice slot view that you saw before where people could see okay during the transition what was happening it was a nice visual way of kind of tracking it and we had a lot of people we kind of were kind of prepared so you have this Super Mario mushroom that should power you up but that mushroom was kind of overpowered by the people looking at it because the way the architecture was designed it just didn't scale well we have frontend instances that kind of have to query the same stuff like if we have 5 frontend instances they do 5 times the same queries because we don't just have like one layer that updates our cache we just have every frontend instance that has its own cache so these are things we kind of try to try to improve and then again we have a very expensive indexing as well in big table and big table that's going to get to later in Postgres and Postgres also has to get a lot of the table into memory to be able to query stuff which is a lot of the time it's not very efficient especially with huge amount of data and so what we try to do is we started out with one big binary where everything runs and we try to strip away things from that big binary and kind of isolate that in microservices that not every binary does everything and if we duplicate one binary we don't do unnecessary work and kind of the migration is kind of a bit challenging for us because then we have to kind of make sure that we manage a lot of the technical depth that we kind of have and are maybe adding through changing technologies could someone maybe say if they can access the repository? Yeah, there's a docker file and a docker compose file in it and if you just run docker compose pull it will pull all the images and when we later get to the interactive part everything will be done we have all the things ready Okay great I hope the internet is good let's pray a bit because when we tried it or tested it before it took a while so here you can see I kind of touched on that the scaling and large table so in Postgres in the beginning we didn't have any partitioning at all so later on when you can just maybe hold I can hold it up Okay Again, lots of assumptions that we made without knowing too much of the spec or your specific use case let's go so cloud first I think still makes sense especially for like the DOS or whatever you guys might need that for in terms of front end I don't know if this is a bottleneck but we mentioned thought about clustering or some autoscale for this or even if we're going to the web3 ethos of let's put this on IPFS or like distribute it somehow and merely make it undeniable service scaling that way or it could be just some autoscaling surface on cloud run or GCP or whatever we might use for that so I don't think this would be the bottleneck anyways the front end itself I think that and if depending on what you have here this could be heavily statically generated so it might be something that you wouldn't be a problem then second layer we're talking about the cache so I imagine most people are our team imagines that most people are looking at past data so you probably don't have to have direct access to the database you don't have to do real queries against it you probably cache heavily previous data that was already from previous blocks that will never change so things that could be pretty much frozen there because it's not, it's really static and stale day let's call it like that so this depending I guess there are probably lots of ways of doing this cache layer this could be like a DB level this could be some like Redis cache level or something like that but yeah really something before the database itself then on database our friend I forgot his name but our friend there gave an idea of maybe we could start sharding this maybe some sharding based on terms like hash or something like that which same thing since this is mostly data that will live forever and not touch it again we probably have a part that's only read only so it's there very high output for reads not that much in terms of rights and a part which will be the right part which this part here would have more direct access to the actual nodes and then the nodes probably would make sense to have multiple archive nodes if the RPC calls might be a problem I don't know if it would make sense to have lots of different machines running those instead of infura infura for us in our company is always a problem if there's anyone from these companies talk to me please but yeah maybe just run your own archive nodes for these not sure also if the actual RPC calls are a bottleneck for you guys but it's good to have a backup okay so at least yeah maybe to like archive nodes you guys are going to run for that and then this part here would just be like the heavy right part which everything that you check that's in you you would update database based on that thanks so much okay thanks for participating I'm gonna hang that up this is the solution we came up with so with which part kind of the solution we have right now yeah this is the solution we have right now with which we parts of the of the data set we now store in big table instead of in postgres so we thought like it's not really necessary to have a relational database for this kind of data set and big table scales really well so I think that we were able to to index all of the current 8.1 transactions in 5 hours so it's like really it scales really well big table so we now store the blocks the balances of the validators every transaction all of the stations in big table and this resulted in a really much more stable index time so now you can see it takes us like really stable 30 seconds to export all the data of one epoch yeah this is where we come to the interactive part of this of this workshop for everyone who wants to participate so he's just resetting everything he's done to kind of start fresh with you guys so the first thing we want to do is initialize the databases so we turn on postgres where there's a big table and initialize the postgres tables and the big table what is it called tables database so the next thing we want to do is to start at 1 indexer we called it that because we're not really creative it indexes like it first gets the blocks from the execution clients and encodes it in protobuf and stores it into big table encoded in protobuf and then after that we indexed the blocks with all the fields and values and the transactions as well so one big thing is like we're actually not using big table from Google Cloud we're using big table emulator from the company Bitly it just emulates the big table API and stores it in a square lite database and I think like for just storing and then analyzing the data it's not like it will not scale for a thousand requests per second but for analyzing the data it's a really good solution in our opinion so the thing here is we are not able to reach the nodes we think before the workshop in the cloud somehow we don't get access so we are now running the nodes on our laptops and that is where you see some errors because the nodes are not synced and it's not like it doesn't work well but so now we have the databases and there is one index around me the next thing we want to run is the exporter the exporter is like just the exporter we had before the merge but now it will index the blocks from the from the big table that we just exported in the one indexer again we have some errors because the nodes are not synced the next thing is like the statistics module and this is also a really important and integral part of the whole project because the indexer will aggregate the data of one day and store the aggregated data of one big engine in the database and so all the queries become much faster and the last thing we start is the frontend it will get started with the cache update and the frontend itself the go lang web server which just queries all the databases so all the things are running now and we can browse the big engine explorer on localhost 8080 the thing is since the nodes are not synced it will take some time until any data will show up this one some extra scripts we wrote into the main script the thing is like we understand this repository not as we want to make the big engine explorer available for everyone who wants to explore the big engine not so you don't have to trust us big engine big engine big engine big engine .in you can run it yourself it will take some time until it's like synced and we will try to make it more easy but the thing is we really we stand by the decision to make this an open source so we will try to make it available for everyone it's not like we made the decision to go into big table from Google to run it in GCP just to make the scaling issue like gone but that doesn't mean it's not available for everyone thanks to such amazing open source project like from Bitly the GCP emulator so the last script is now you have like a Postgres database where you can think with the data we made just some the script is not up to date but now it is yeah okay I don't know if I want to do that is anyone following along is it working okay it's working yeah okay yeah so the thing is like with this repository we show really how you can set up the extra and we know it's not like there's a lot of technical depth that is like we need to solve it but it's working for everyone not just for those who have really GCP accounts and everything I think we can go to the next slide so this is probably yeah okay so for anyone using our mobile app or the website we've got a discount code bogota50 if you want to use it you can get our premium app services if you have a lot of validators it's nice among other things or if you want to use our api services you can use the code over the website not directly over the mobile and yeah if you're running a validator and you don't know about the mobile app it's pretty nice let's you look at your validators on the go let's you get notifications when you propose a block and kind of see if you've gotten those 30 ETH block reward or something like that yeah okay then that's a lot of features yeah if you want to contribute here's kind of the overview of kind of the resources so we have a get coin grant that's linked below you can tweet at beacon chain if you see any issues or something create issues on the github repository that's also linked there and oh did anyone still want to take a picture oh no thanks yeah that's basically it from us thanks so much for participating and listening if you have any questions yours right now that you... it's called so it's one of the testnet it's a bit smaller so it should be able to sink in a reasonable amount of time the wifi isn't the best here so here it could take a while but it's a lot smaller than prata or which is girly or mainnet which are pretty huge right now any other questions not sure if I should answer that but it's quite a lot it's quite a lot so it's what is it five figures yeah we looked especially at silo db but we looked also at other db's but in the end we took something that also fits in our team like we don't have so much resources I was like so much take so much man hours to get into a new technology and for us this looked at the best fit and right now we are really happy with it so yeah we basically went from an export time of 30 plus seconds for attestation assignments to like 5 seconds which is a lot better and there was less contention between different queries on big table now that we moved but that can change if we have everything on big table we're just starting to migrate so I hope it stays good yeah and layer 2 is coming so okay anything else thank you very much for listening and for participating again


I am Ganapathy Raman, aged 80 years. I belong to Tamil Nadu. I have been suffering from right leg pain since 2 years. A number of Ayurvedic and other things were done and it was no use. And we recated this Manipal hospital and Dr. Marli Nath through a relative of mine and we landed here. And we consulted first and the date was fixed but because of corona it was postponed. Then again when during the corona was eased they contacted us and we came here. Got preliminary tested and Dr. Marli Nath operated and I am feeling now absolutely comfortable. There is no difficulty at all. I am able to walk with a walker and he said after about 2-3 weeks you can walk normally and absolutely normal but for certain physical things. And my experience here in spite of the restrictions imposed by the government regarding the cleanliness and hygiene and all that. The hospital management has been excellently doing things. Right from cleaning staff and nursing staff they take particular care in all such matters. Cleaning then very often and nurses attend precisely at exact timings and now I am absolutely comfortable in walking.


I used to teach in the men's dating industry. And one of the things that you need to know is that in the men's dating industry, unanimously all over the men's dating industry, what they teach to men is that men need to tell women that they're just looking to hang out and have fun. Because as soon as a man starts talking about being serious and being in a relationship, what ends up happening is a switch flips in a woman's mind. And she goes from just hanging out and having fun to all of a sudden being like, okay, is this guy a guy that I want to have in a relationship with me? And she starts looking at his traits and she starts trying to figure out if he's really a good guy for her or not, right? And so she's less likely to be willing to hook up with him and have fun and do all the things that make it so that it's easier for him to have whatever it is that he wants. And so you have to remember that. However, if you talk to a guy and you ask him what he's ultimately looking for and he's like, yeah, well, you know, right now I'm just kind of meeting girls casually and I'm just looking to have fun. What you need to say is, well, are you open to something more than that? And if he says yes, then say what is it that you're open to having, right? And ask him what would create that? Why would you want that? And make sure that it's something that he wants, like he wants to have this real relationship before you even think about doing something sexually with him. That's my suggestion. If you want to protect yourself and you want to make sure that you're getting yourself into a good situation, make sure you're only doing it with guys who are open to that because if he's not, he's probably not going to fix whatever the problem is mentally and emotionally that he needs to fix to set himself up so that he actually wants a real relationship because usually guys don't want real relationships because they've been hurt or because of things that have happened with other women in their past. And so he needs to fix things first.


Um, first before I'm going to start, who was here at the awesome talk of Angie? Because she stole my thunder. I only got one slide for you left for the people who were already here. So, yeah, but now you know the advantages you can have with building or kind of like doing all the images compare image comparison. I'm here to tell you how you can create your own module.  But first, let's take a look at the slides. Am I working? Where's mine? Yes, today is not about me, but first of all, I need to give you a little bit of context. So, yeah, let's, this is a slide for everybody who was in the previous session. I'm a solution architect at Sauce Labs. You might also know me as the blue guy because of my picture in the open source community. You might have seen me last year at the first Appium conference where I explained the research I did for a customer I was working for. What I should use when I needed to automate a React Native application in the end, yeah, we came up with Appium. Otherwise, I was not invited.  Here also in Bangalore where I explained all the best practices about how to speed up your test cases with Appium and React Native and kind of like getting the coverage you really wanted to have. And this talk is going to be part three. And I hope in comparison to movies that my sequels get better each time. So, let's start. Start with this story. I'm going to throw away my clicker. I started at Tele2. It was in 2017 and I was in the team with developers with me as an automation geek and we had one manual tester. And even though all my tests were passing, my colleague was still finding some issues in our app and basically they were layout issues. And yeah, we got more and more features also in our app and he was kind of like stressed out. Kind of like the guy you see here doing all the tests and it was really, really stressed out. So I thought is there a way that I can help my colleague and especially with automation because yeah, I was the automation geek in the team. So I pitched the idea for doing visual comparison and got some time to do the research. So I started my research and normally when I do a research I do that based on some requirements. So I put down some requirements on paper which we kind of like discovered together with the product owner and the developer. And one of the things that at least our visual comparison solution should have, it should have native app support. It should also be a cross-platform solution and the reason for that we were using React Native. We were writing something with JavaScript and in the end it was compiled to a native iOS and a native Android app. We also wanted it to integrate with WebDriver.io because that was the tool we were using and already written all our tests in. So it would be very nice, kind of like very useful if we could reuse the code we already had to get to some specific stages also in our app. If we wanted to do the research and we saw some paid tools, yeah, we also wanted to have kind of like a trial first. In our case we said, okay, we wanted it to support blockouts and to support element screenshots less bounding boxes. And you might be wondering why should an image comparison tool need to have that? Well, basically for these three reasons. Initially when you take a screenshot of a native application you will get a full page screenshot or a full view screenshot where you have your status bar, you have your tool bar if you're having an Android application and if you compare that to a web screenshot you will not get that. You will only get your viewport. So you want to focus on the parts, and Angie already explained it before. You want to focus on the parts that really matter. Secondly, if you would look at animations, if you're taking a web application it's quite easy to disable animations. It's harder to do that with native screenshots, with native applications. Last but not least, we also had the dynamic text. For example, with a build numera version, app version, that kind of stuff. On a web application you can easily kind of like put static text in that and then take a screenshot, put it back. Well, cannot do that with a native app. So now that we know where we need to look for it, so let's take a look at the research we did for all the tools. And basically one requirement we had, it needed to support native apps. And then we just did the research. The research was quite hard and the reason for that is that there are not a lot of tools out there to support visual regression for native apps. We found these tools. EpicTools is there. We've got screenshot tests for Android, FizzWays, Nicole, a lot of tools. What we now, what we did after that is kind of like create a list. And based on that list, we looked at our requirements. And one of the first requirements we had, it needed to be a cross-platform solution for a native app. So we could already narrow the list down because the three tools that got across were only or for iOS or only for Android. Then in the end, we also wanted the tool to be maintained that if we found something, it should actively be maintained because if there was an issue and it was an open source tool, for example, we wanted to use, we wanted to be able to add a PR and hopefully get it fixed as soon as possible. But yeah, Nicole's end of life is not maintained anymore. So in the end, we had FizzWays and ApplyTools and FizzWays already draw our attention because it supported WebDriver.io. But in the end, it did not support the blockouts. So in the end, we had ApplyTools and we're like, okay, this is awesome, but we had one challenge. And the challenge was that ApplyTools did not support WebDriver.io for native application at the time we did the research. So we contacted ApplyTools to support and we got the option to use just the JavaScript Appian bindings and kind of like created ourselves. So we're wondering what should we do? And you might also be wondering, what have we done? I think you might already know the answer, but I'm just going to explain it. In the end, we did what every JavaScript developer currently is doing. We build it our own. And the reason for that was that initially we said we wanted to have something that we could reuse our code, our WebDriver code already. And if we would then compare the amount of effort we needed to put into creating a new framework, implementing parallelism, it was kind of like the same time we needed to put in creating a new module. And basically what we also wanted to do from a Teletube perspective, we wanted to create something that was open source that everybody was able to use. Because if we were able to use, it was awesome if more people could use it. So we said, okay, let's do that. And I got some time to do that. So, yeah, you need to start somewhere. So where do you start? I thought, okay, if I'm going to start coding, I'm trying to or maybe I'm going to get a complete mess. I first need to know which requirements I have when I need to create my module. I wanted to create something for WebDriver.io. So I divided logically the module into two parts, kind of like the module itself that only needed to kind of retrieve some information and a comparison engine. And a comparison engine would retrieve the information, do the comparison for me, and give me the results back. Basically, this would kind of like create the module we wanted to have. And then, yeah, we've got some requirements here also. If we would look, and especially for an open source comparison library, it should also be actively maintained. Yeah, also the blockouts should have been supported. But what was also very important for us, because we wanted it to be an open source module, you could easily install that we did not have any environmental dependencies that people who were using Node, for example, in their Jenkins instance or whatsoever. Did not need to install extra dependencies. And last but not least, it would also be nice if we had some comparison options, if we could kind of like change some stuff. So in the end, I did the research. And like I said before, it looks like JavaScript developers like to develop everything for themselves. Because as you can see, we've got a lot of libraries here. They're all pixel by pixel libraries. And if you would then look at the requirements we had with one of the first things we said, okay, it needed to be actively maintained. That means that we can already put across to four libraries. We're not going to select them or do a proof of concept with them. Then we also said we needed to have something that should support blockouts. Again, to narrow the list down. And as said, we did not want it to have extra environmental dependencies. So in the end, we did not look into Node OpenCV, even though it's really awesome. But as said, it should just be a module you should install like you do that with WebDriver.io. So it should just work out of the box. So in the end, we came up with PixelDiv and with Resomble.js. And I've been using PixelDiv in the past. It's a very nice library, but in the end I found out it was not accurate enough when it did the pixel by pixel comparison. So in the end, I ended up with using Resomble.js. In the end, as said, I started using Resomble.js. And if you would then look, we've got WebDriver.io, we've got Appium, and we've got Resomble.js. And you would now wonder why are the images not aligned here? Well, we're missing one important ingredient here, and that's some blue magic. Because yeah, we need to stitch them all together. So if you would then look at the module, again, I'm not coding here yet. But if you would think about a module, what should the module do? First of all, it should be able to create a screenshot. Secondly, it should be able to compare those screenshots with each other. Then we should be able to do a save element. And last but not least, we should also be able to do the compare element. So now let's dive into each method, again, before we start coding, and just check what do we need to do. This one was the hardest part. Save screenshot. Because yeah, what should it do? It should save a screenshot. Let's go to the next one. Compare screens. I think this was kind of like the most difficult part. Because as said, if you're going to create a screenshot, you get everything. So where could it go wrong? Well, basically on these three points, those three points can cause flakiness. And I'll kind of like dive into them and show you how you can dynamically fix that. Because it would be nice if we can do that dynamically. We might know the device we're going to test, but yeah, there are different devices, different screen sizes. So first of all, if we would look at the status bar. For the status bar, if you, for example, compare Android and iOS with each other, they can already differ in height. But if you would then also look at iOS versions or Android versions, they can also differ in height. And what we just wanted to have is a cross-platform solution that our module was able to determine the height and width, the position of the status bar itself, so we could just block that one out during the comparison. So let's use one of my favorite tools, and that's Appium Desktop. Because Appium Desktop is kind of like the way to show how the UI hierarchy looks like, visualized with a screenshot next to it. As you can already see here, this is on an iPhone X. We can determine the position, which is quite easy, 00, but we can determine the height and the width here of the status bar. That's awesome, but if you're going to do a research, you should not rely on one phone, you should test multiple phones. So I'm not going to bore you here with all the phones. Just going to show you a smaller phone from Apple. That's the iPhone SE. It's a smaller screen, and as you can also see here, we have the option here to determine the status bar based on an element and also based, we can get back the width and the height. So for Apple, this time it's very easy to use Apple and to determine at least the status bar. So now let's take a look at Android. And I'm already taking a breath. This is cool. With Android, this is on a Samsung S10. We already got the status bar here. Awesome, we can determine the position. So as said, we need to do a research, we need to check multiple phones. So I checked the Google Pixel. And as you can see here, you cannot find a status bar element. You cannot find a navigation bar element. So what should we do here? Because it's quite hard now to dynamically determine the position at least of the status bar. And I think the following research costed me some precious minutes of my life because I needed to know what the difference was between the XE UI test driver and at least the UI Automator 2 driver. And during my research, I stumbled upon the Settings API. And I think this is the best kept secrets of Appian. Who here knows the Settings API, by the way? Ah, guys who work with Appian. The non-contributors or the core contributors to Appian. Who know it? Well, basically, we can all read it. Basically, it can help you over, kind of like, it specifies the behavior of the Appian server. And if you would dive into the documentation, you would also see that there's a difference, for example, between the previous UI Automator driver, UI Automator 2 and also with the XE UI test driver. Because it does not show you all the elements. It does not give you back all the elements. Kind of like it's smart enough, call it AI, call it machine learning or not. It's only going to give you the elements that is, or relevant, or are on the screen. So how should we solve this now? Should we use the Settings API? Yes or no? Well, I think I needed to use it. So what I did, and I'm going to show you. I'm going to stop the presentation. We're going to dive into a little bit of the Appian server. It's here. What I'm now going to show you is not only for my module. I think it's also something you can use when you're going to do automation. So please keep paying attention. It might help you. First of all, what I wanted to show you is the status bar and the toolbar. But just let's go to a nice form. And in this form we have a button. We're not seeing when we're opening the screen here. Let's refresh Appian desktop. And in our UI hierarchy, as we can see here, we can search elements. We are not finding something that looks like a status bar or something. But also, if we wouldn't look at the buttons, we would only now see a text view here. We do not see the buttons. But for example, I would now refresh my page because I've got the elements in the view now. Keep paying attention. And yes, we've got the elements now here. This is basically what you also might face when you need to automate your native app. You cannot see if your element is there. You need to scroll it into view. Okay. Now let's do some magic. First, get this back. I'm going to hide the elements again. And yeah, they're gone. There's also something new in this screen. They released it with, I think, Appian desktop version 1.12. Does anybody see the new awesome feature here? No? Okay. It's this one. It's the Action tab. And if you click on the Action tab, you've got some options here, for example, to select an action. You can kind of like do something on the device and on the session. Well, we're going to demo the Settings app here. So we're going to do it on the session. And again, we've got some options here. And let's take a look at the settings. And what you've got here, you've got a button called Update Settings. And if you click on this one, you get the possibility to kind of like post adjacent object to the Appian server.  It's a demo, but we would hope to get more information. So let's start doing that. And one of the commands would be allow in visible elements. By default, it's set to false. We're now going to set it to true. And we're going to execute the action. If I would have made an error in my JSON object I was posting, I would have got an error here, but I did it quite good, I think. If we would now go back to the Source tab, we would still not see our element because we need to get the page source again. It needs to be refreshed. And if we're doing that now, I'm now going to make the screen a little bit bigger. Ah, we already now see our status background. And we also see our navigation bar. So now, if we go back to the module, I got the option to dynamically determine the position, the height, and the width, at least at the status bar. But also get back to what I think would also be useful for your automation to see, yes, even though the buttons are not visible, they're now visible in the UI hierarchy. And even with an option, they're displayed false because they're not in the view. So maybe you can also use this when you're doing your automation for Android with UI Automator 2. Okay, having said that, let's go back also to the module. And I'm going back here. Please go back here. Now I've already shown you that you have the possibility to at least determine the status bar. But there's also something extra. We've got the toolbar. The toolbar itself from Android can also cause some flakiness. And I've seen this with the latest Galaxy phones where there's kind of like a gradient in the software navigation bar of Android or Samsung. And it could also cause some flakiness. So I also want that kind of like to be blocked out automatically. And as you can also see here, the status bar itself or the navigation bar itself can also differ in height. And you can also have some devices that do not have a status navigation bar. Take for example, what is it? I think the Samsung and Galaxy S10. It has a hardware navigation bar. So if we would now kind of like use the API again and would now look for example at the Samsung S10. Initially we did not have the navigation bar with the Samsung. Now we have it if we're going to use that one. So this is quite easy now to make the module itself dynamical. But there's more because as I said, we've got the navigation bar. We've got the status bar, but we also got the handle, the home handle bar from iOS. And yeah, this is also something that can cause some flakiness. And I don't know if you ever used an iPhone X series. When you're watching for example a YouTube movie, you're going to kind of like put your phone from portrait to landscape. You would also see that the handle bar kind of like fades away. So the behavior, kind of like the fading could also cause flakiness if we're doing the comparison. So we also need to find something here, a way to block out the handle bar. And I think we have a second challenge here because there's no place here in the UI hierarchy to determine where we can find the handle bar. No element here to determine what the position or the width is. And yeah, I think if you would now think about this finding a solution, I think we again can be grateful that we are automating Apple. Because Apple is quite limited in the amount of devices. So we can now just figure out a different solution for blocking this out. And one of the ways is determine on the width of the screen. We can determine if it's an iPhone X series, even though we've got two different phones, we've got the iPhone X and the Max. But then we can still determine the width and if it's an iPhone X series, yes or no. So now just with the static block, we can just block it out because we can tell is this an iPhone X and if it's a large phone, yes or no. So I think we now covered at least three points that could cause the flakiness. So the image comparison for the screenshot itself, I think it's done. We can kind of like build it. But then we also have the safe element. It would also be nice if we can do that. And if you would now look at the WC3, W3C, I'm always missing that word up. If you're now look at the W3C specs, you would see that we initially have the screenshot, but we can also take an element screenshot, which makes it quite easy for us. We do not get the noise. But not bad. You can get something like this. This is kind of like a login form. But what if, for example, a user of the module is using an old version of Appian that might not support this? Or if you want to do something extra, there's a second option, how you can create an element screenshot. I think it's basically almost the same as what I did with determining at least the position of the status bar. We can use a method also from the W3C specs to get the element rectangles. And when we create a large screenshot, we know the position of the element on that screenshot, and we can use the elements rectangles just to crop it. It's quite easy. So we kind of like also have a fallback here. But when you're going to use that, you can also provide some extra powers. And extra powers is like saying, okay, if I have an element, I want it maybe bigger, provide some margins, or I want to be smaller, provide negative margins. And that will give you, for example, the two results you're seeing here. So you've got some more options here to create larger or smaller element screenshots. And then last but not least, we've got the compare element. What else you could do, yeah, basically the same as what we had with comparing screenshots. So it's not that difficult here. There's only one thing that I found out when I wanted to kind of like block out elements on the element screenshots itself. I've chosen not to do it because it makes the logic more complex. I didn't have time enough to do that. And normally if you have a lot of complex code, it could cause flakiness in the code itself. I just need to have a vacation to fix it or something. But for now, it's not in the module. But it's, yeah, I think you've now seen at least how you can determine it. Let's take a look at the use case because it's nice to know now how to build it. But yeah, it's also nice to see what you can do with it. And I want to get back to Tella 2 and get back to the real use case where we were developing, why we were developing this module. We were at a point where we needed to migrate, we needed to upgrade React Native. We were working with React Native version 0.54 and we needed to migrate to version 0.57. Wow, that's a very large migration. I executed all my functional test cases and again everything was successful. But as you might remember, we had our colleague who was always checking all the layouts. He was kind of like freaking out. I was like, okay, let's start using this module. And I don't know if you already see a difference here. Well, the difference is here. This was just an indentation issue. It was a little font issue. But this simple test case already proved for me that the module was working. So I just released it on all the test cases I had. At least specifically for the image comparison. And we found multiple issues in the end. Issues that could cause flakiness for the customer or unexpected behavior. We had issues, for example, also with a login form that certain elements or certain characters, especially for the password field, were just shown. We needed to fix that. It's kind of like a security thing. We had issues with our radio buttons as well. These were kind of like the things we had with the image comparison module and also we found during the upgrade. So for us, it was really useful to use this within our pipeline. And now you've seen this all. You might think, okay, this is really, really awesome to use. And I must agree. But there's also a disclaimer. And I want to start at least with the cons and then start with the good things of the module. Well, first of all, as you might have also heard this morning, this is just a basic pixel by pixel comparison. So it might can give you some results that might not be an issue. Yes or no. On the other hand, it's open source. It can also have an impact on your project size. And the reason for that is this module does not have a dashboard. It does not use a cloud service. So you need to store the images somewhere. Basically try to store them in your project. There are some limited blockouts, as I said already. And yeah, we do not have the dashboard. And I must say it's really awesome to have a dashboard because it makes your life a little bit easier. But yeah, there are some pros. First of all, it's open source. So you can just use it. You can use it with WebDriver. It's only a few lines of code and you can already use it. The module logic, as you've seen, is not that hard. And if you want to create it your own in Java and Ruby or Python or whatsoever, I think just take a look at it. It might help you creating your own module. And if you're going to do that, please share it with the open source community. But what I think is kind of like the power also of my own module is that it has an easy cross-platform device blockout logic. And I'm just going to show you a simple part of piece of code. And what I mean with this is normally when I need to use something where I have my validations, my checks not in my code itself, but I need to visit something else like a dashboard. And I read my test. I do not see what's happening in the test.  So it makes it easier for me at least to see what my test is doing. I like to see that. So I hope you now got a very good impression of what you might need to do if you want to create your own module or what you should not do or why you should not do it. I also showed you that even you might not see your elements in your UI hierarchy with Android. There is an option to kind of like find your elements. And yeah, if you want to know more also about this module, well, the link is there. I already shared the PDFs, the slides, so you can check the links also. Check the logic if you can use it or just check if you want to use it. There's a specific version for version 4 of WebDriver.io and there's also a version 5 of this module. And just to give you a heads up, I just released a new version of this where the blockouts in the previous screen you saw with the Tella2 app, there was a black line. I thought, yeah, you just want to see what's happening behind it, what was there. So I just created a new way of blocking out the elements. So you could now at least see what's blocked out, yes or no. For now, I think we have a few minutes left also for Q&A. So are there any questions? Meaning that you can block out a specific part of that element? Yes. If I understand it correctly, there is an option, for example, with this module to determine or to provide extra margins. So if you find the element, it has kind of like the height and the width, but you can also say just add 50 pixels on top of it or a negative margin. I hope that answers your question. Okay, so my question is related to graphs. It's not only we do most of the time image comparison. So what we do, many times we provide some random data and generate graphs to test it out. So how can I make sure the graph which is generated for me is correct on the X and Y coordinates? Yeah, you're already kind of like if you're doing the image comparison, you're comparing it with the baseline. But if you're baseline, if you're kind of like the data you're constantly putting in it is fluctuating, yeah, with this module, it's quite hard to do the proper image comparison. Resumable JS provides something called mismatch percentage. Yes. So how do you determine what should be the mismatch percentage so that I can pass my test? Because I always see that there is a mismatch percentage even if the screenshots are similar. Yeah, and what I would always advise you is not to accept a mismatch. And the reason for that is if you're going to do that unattended, you're going to run it at night. What will be 1% mismatch? Is that allowed or not allowed? Because you do not know which element gave the mismatch. So I would always advise not to use a threshold with the mismatch percentage. Always just accept zero percentage because otherwise you might, especially if you're going to look at screenshots for iOS, they have a, you have a higher resolution, meaning you've got more pixels. And then 1% of what is it, 300,000 pixels is still a lot, but you do not know where the mismatch is. So you might be accepting things that might not go to production. So I would advise not to use it. Yeah, as said, that's kind of like one of the downsides of pixel by pixel. Yeah, you're doing kind of like all the calculation on your server. So I must say you need to have kind of like a good server where you're running this on and not like the most slowest one because it will slow down in the end your comparison and also your test execution. I do not have real data on if you're going to run things in parallel. You have 100 instances and you're going to do the comparison how it will take. As said, I do not have those numbers. Yeah, you're mentioning memory leaks with it. I don't know what the parallelization amount is that you're using. I'm running this. I'm using Resomal.js in multiple modules. And I also run at least weekly a check build with what is it? I think 12 or 13 concurrent sessions. And I do not have any issues with that. But yeah, it could be that if you're running 20, 30 or 50, yeah, it's consuming a lot of memory, which might lead to memory leaks. That was it. I think one question. And then let's get to lunch. Yeah, so I'm not that familiar with Resomal.js. Okay, but my query is like when you're working on an Android device, as per your flow, you would be taking a screenshot, then taking the screenshot of like saving the elements, right? So one time if you take the screenshots, will that be applicable across any of the Android devices? Or for each of the device execution, the flow would be saving the screenshot elements and go on? It depends on how you set it up. If I understand it correctly, you need to create a screenshot of each device. Each device. Yeah, each device. What I have kind of like your requirement is that I need to have Android 8.1, 7, and 9. But I also need to execute that on five different devices. And that means that you have five times three is 15 screenshots only of one element, because you cannot reuse them, at least not with this module. I don't know how fancy all the rest of the paid tools are. But basically if you want to do proper image comparison. Across all devices, you'll have to take the screenshot. Okay, thank you. Okay, but then I would like to thank you for being here, for listening to me, and hopefully it was useful for you, that you can also use this or the module or the logic, or even the new, not so new, the secret API of Appium for UI Automator 2. Thank you.