Welcome, everyone, to the Observability Meets a Flaky Test by João Proenzo. And without further delay, over to you, João. Thanks, everyone. Let us begin. So, welcome to SeleniumConf. So, this is my talk, Observability Meets a Flaky Test. My name is João Proenzo. I am a quality engineer that works at Aether Health, a company that works in the health space. I come from Portugal, from Lisbon, where I live, and I've always lived my life. Before I proceed with the story that I want to tell you today, I need to say thank you to an ex-colleague of mine, who is Lisa Crispin. I was fortunate enough to work with Lisa for a year or so, a while back. And when we were working together, we paired a lot on observability topics at the company that we were working on at the time. Just trying to figure out between the two of us what observability meant for that team, and also how we could improve observability inside the company in its various facets. So, it was out of those conversations that I have with Lisa, those prior sessions that the idea for this talk came from. And so, I need to thank her because she contributed to the content that you're about to see. For folks that don't know Lisa Crispin, she is a very well-known person inside of the testing community. She has written books about HL testing and is a leading practitioner that a lot of people follow and admire, such as myself. So, without further ado, let's see what we're going to cover today. I'll start by clarifying a little bit of what observability is. Some of the key, let's say fundamentals of observability. Then, with those fundamentals, I will tell the story that illustrates hopefully the need for observability in the testing context. And at the end, I'll share some ideas of what observability in the story that I've just told you in that context, what it means. So, before we, as a starter, let's talk the difference between monitoring and observability. And a lot of times, when I speak to people about observability, some people will tell me, well, I sometimes get the idea that we just gave monitoring a new name, a new fancy name, but actually we're just doing the things that we used to do before. And I don't think that's true. So, the difference between monitoring and observability is that with monitoring, you're usually dealing with known unknowns. Whereas with observability, you're trying to use data and visualizing data to deal with unknown unknowns. What exactly does this mean? So, with known unknowns in the monitoring side, usually you know something may go wrong. And because of that, there's questions that you know you'll want to answer in the future. And you know that you're going to need data to answer those questions. And so you build dashboards and set alarms around that. So, you know that maybe a specific service may go down and you want to have information about that service that's running right now in production. You want to know every second if the service is up or not. And so you have a dashboard that tells you that. But these are the things that you can predict that may go wrong. With observability, when you walk into the unknown space, what you're basically acknowledging is that some things may go wrong and you don't know about them yet. So, the data that you have already, maybe you'll want to ask new questions of that data. And you have no idea exactly what data will be relevant in the future when you have these new questions that will come up and you need to look at them. Also, you can't exactly build a dashboard today that will answer those questions in the future because you don't know exactly which questions you'll want to answer. So, the whole idea with observability is that you want to have a very rich set of data that will tell you exactly that will allow you to ask new questions in the future, questions that you can predict, questions about what's happening in production or is happening with your systems without having to ship new code. And that's like the key idea of observability. You know you have an observable system when you are able to answer new questions from your data without having to ship new code because you need new data being produced or new ways to access that data. So, let us proceed. So, with these fundamentals in minds, Lisa and I were having these pairing sessions about what observability meant for us, for our system in production, we have a lot of things that we're missing in our context at the time, but then we started looking a little more inward. What about if we don't think about just production but we think about other places than production? And then hypotheses appeared in our heads. And the hypothesis here was that we lacked observability in our CI-CD pipelines. And it's hard to tackle unknown unknowns there. And when we're talking about not just the pipelines themselves, but the automated tests that we are running in those pipelines, whenever an automated test failed, we felt that understanding why that automated test failed was really hard just by looking at the data that we had at hand. But this was just an hypothesis of ours that we couldn't quite prove or we didn't have a good example to showcase to everyone. Until one day a story happened. And that's the story that I want to tell. But before I tell you the details of that story, let me just give you a little bit of context which will be really important in order for you to understand the constraints that we were operating with and even the no constraints that we were operating with. You need to understand that at the time, Lisa and I were working in a product software company that offered just one product, which was a big monolith. We had very complex CI-CD pipelines validating that big monolith. And you also need to understand the scale at which we were operating. So let's go to each one of these topics before I proceed with the story. What do I mean about the monolith? What this essentially meant was that our product, our software, was being offered as a big binary that contained the whole thing inside. It was a 20-year-old product with millions of lines of code, hundreds of thousands of automated tests running over that code. Whenever we produced a build, that build was a single binary that could be deployed in different contexts, could be deployed on premise, could be deployed to the cloud. We still have a lot of things to solve regarding the fact that we have a monolith and start breaking it down into separate decoupled services. But at the time, it was a monolith and that had a big, big impact in the way we were operating. And not just the fact that we had a monitoring with a large code base, but also we had a lot of engineers. We had around a little bit more than 100 engineers working at the same time in that code base, committing code and running automated tests over that code and that huge build process that we had. Now, because we were working with the monolith and we had the engineers, a lot of engineers working, trying to validate the product every single time whenever they committed new code into the product, we had a very complex CICD pipeline. It comprised of several stages. At the beginning, we would have what we call the dev stage where we would run mostly unit tests and whatnot that was really fast to give us a lot of feedback at the beginning. But once that was done, every single code change, every single commit from an engineer would set up all of this and would start firing up different stages of the pipeline that even ran in parallel with different types of tests like performance tests, contract tests, security tests, you name it, N2N tests or UI tests. So the complexity was really, really high regarding our pipeline and the amount of stages that we had and the different types of automated tests that comprise those stages. Now, given this context and given the scale at which we were operating, one of the things that was really, really hard was getting to a green. And when I mean getting to a green, having a build that ran all of the tests, and I'm talking about, if I remember correctly, around 300,000 tests being run for each run. And given that scale, if one of the tests failed, you wouldn't have a stable green build, so you couldn't use that build. So getting to the green, as we called it, was really hard at the time. And we had a lot of processes in place, and we took the pipelines and the automated tests really seriously because we wanted everyone coming together to make sure that all of the tests were passing so that we could have a green, really simple build. Now, I'm sorry, but I'm having a little bit of a hard time. Okay, sorry about that. I was having a little bit of a hard time with the slide, but now I can see it. So one of the things that you need to take into place is that we, because we had so many people working on top of the code base and with these really large builds, it was really hard to have just one run per each engineer. So imagine this scenario. If you have like an engineer that comes up in the morning and no one is really working on the code base yet, she would make a commit of code. Maybe that would fire up a build and the test run with all of those different stages. If one of those tests failed, we knew that it was the code change introduced by the engineer that would have made that test fail, so that engineer could just go on top of that and right away start looking at the failure. However, most of the cases that we weren't in that ideal scenario, because we had so many people committing to the same code base, a lot of times you could have another scenario where like you have three engineers that commit codes maybe to the same part of the code base, maybe to separate parts of the code base of the whole monolith, but those three commits were almost simultaneous. And because of that, that would trigger a pipeline run that would aggregate the three commits. So if one test or two tests or three tests failed that weren't failing before, all of a sudden the only thing that we couldn't say for sure is that these three engineers made the code change that made that test failed. So those three engineers, the three of them would need to go on top of that test to figure out who made the test fail and then fix the failure. Now this creates a lot of different, let's say, social dynamics that we need to handle. And so we created what we called like the rules of engagement, stating that whenever tests fail, that should become top priority for the engineers involved, because we didn't want any bias coming to place like the bystander effect where all of a sudden three people are looking at a problem and each person is waiting for the next person to take it. We would have rules in place that would say one of the three people is elected as the person who needs to figure out which of the code changes made that test fail. And then he should pass that problem on to one of the engineers in the group of three or group of four or group of five once they deem that person made a test fail. We even have had an integration with Slack, so our whole CI CD pipeline had this system that we built that basically detected, hey, we have a new test that is failing. We know that the code changes that made this test fail are this aggregate of code changes and it would automatically open a Slack channel in Slack that would send a message to this group of three engineers and say, hey, we have the test that is failing. Person A will be elected as the person that needs to triage the failure. Please figure out as soon as possible who is responsible for the failure and please fix the problem as soon as possible. This made it much better for the whole team to know tests are failing. Who are the people that need to go after that? Once again, that should be top priority for everyone because we're all trying to work together to get to the green. Now, this was how things work.  Flaky tests. Why? Well, if you think about that previous scenario, what happens is if you have the group of three people that committed code and all of a sudden a test fails not because they changed the code or because of any of the changes that any of the three people did, but just because the test is flaky, so it's just like it's passing, passing, passing, it fails, but in that run, the people that are flagged with that failure are the people that triggered the run. Then you have a big problem because you're basically addressing a group of three people and saying, hey, this test failed, you are the person responsible, but then people are like, but I have nothing to do with that failure. That's in a completely different part of the code base. It's just that the test is flaky. Okay. In that situation, we also had some rules of engagement that told us that if the group of people that are looking at a test, if they found out, find out that the test failure most likely is not related to any of the code changes, and if they can detect that the test is probably flaky and the way people did this was looking at our test reporting system that we have that will tell you just for a specific test, how the test runs had been and what the history for the test runs was, and looking at that, people could tell, hey, this test is clearly flaky because it's passing, then it's failing, then it's passing, then it's failing, so you can see that there's no stability to it. If they could come up to these two conclusions, then they had the right to quarantine the test immediately and open a ticket to the team that owned the test because the different tests were owned by different teams and saying, hey, this is a flaky test. We can't have flaky tests in our system even though you'll always have flaky tests, but please deal with the flakiness before you take it out of the quarantine and include it again in our pipelines. And so this was the way we were operating and this was the standard operating procedure in order for us to deal with the flakiness and given the big impact it gave us in getting to the green. Okay, so this was the essentials of the context in which we were operating. I hope that it was clear to everyone and now I will proceed to the rest of the story that I want to tell you. Any questions about the context? Just in terms of if it was clear or not, if you want, just write something in the chat that I would guess not. Okay, cool. Let us proceed. So here's what happened one day. One day, thank you, Palavi, it's clear. Cool. One day I arrived to my desk, I looked at our dashboard with all of the tests that are failing, and all of a sudden I noticed that there was a test user, test user providers that set up admin user and I could see that this was a really strange message. Fail to obtain test suites could not find tests in the module. Now, this test, I didn't make the code change that made this test fail. It really seemed like flakiness and this test was using a framework, a testing framework that I created myself, so I was really interested in it, so I told everyone, hey, I need to, I'll take this test because this test is flaky, and I need to figure out exactly why it is failing. I tried reproducing the failure in my own machine. I have never seen this sort of thing before, and these tests tended to be really stable in the past, so I didn't know what was going on. So I kind of went against the rules and made the test stay in our pipelines in order to figure out exactly why it was failing just on our pipeline machines. But a few months later, the test failed again with the same message, and all of a sudden I had a test architecture to the right and a tech lead that were getting really angry at the fact that we had this flaky test running around, and they were like, another flaky, failing flaky test, we can't have flaky tests around, and then the tech lead was like, it's blocking your beam, we need to tackle flaky tests, so I'm going to quarantine this for now, which was just like, you know, part of the standard operating procedure. And then how you appear, this is supposed to be here, here on the right, so I was like, I'm sorry about the flakiness test, people, I'm trying to fix the flakiness around these tests, but we've been trying to catch the test failing because we can't reproduce the test, so that's why I'm running the test a few times in our CI CD pipelines, even though it's failing. And then the architect was really angry at this, he was like, do we really need to have this enable to analyze it? We can't reach a green like that. And I was like, well, I can't know what's going on without accessing the only machine where it fails. And of course, the tech lead was like, she was being empathetic, and she was like, I can understand how that's tricky, just make it quick so that we can quarantine the test, you know, and the architect was already thinking, there's got to be a way for us to improve this somehow, but I don't know how just yet. And so they allow me to have the test running a few days more, but a few days later, the test was still failing. And the architect was now really, really angry, like, the test is still failing, this is not acceptable anymore. It's the end of the line, please take the test away. And I was really frustrating because I was waiting to see the test failing before my eyes. But for some reason, the test only wanted to fail at during the night, like at 11pm, when everyone was already away, and just problems with just running on a night rebuild or something like that. And then they were like, all of a sudden, I was like, hey, wait a minute, the test just failed. And this was like during the morning, I can finally check the log. So this is exactly what I wanted. But the architect was like, we need to shorten feedback loops. Why didn't you access the logs before? And this was really important. I had a problem, which was the machines where the tests failed, they were being fired up during the morning, they ran tests throughout the day, but then at the end, because of costs, we would tear down the machines also because we didn't want any garbage being left in the machines in case some tests were being bad, bad behavior. And so with the machines being deleted, we also lost the logs that allow us to look at the test failures. And let's say the specifics of the test failures. And this will be something very important that I'll come back to later on. And then the architect said something to the likes of maybe you should output the relevant logs in the test report itself. This is a very, very interesting observation that you will also be coming back to later on in the story. But okay, let's aggravation around. Aside, I put the test in quarantine. No one was angry at me anymore. I had the logs, the tests that failed, and I could finally look at the logs and figure out what was going on. Now let me just tell you exactly what I had to do in order to figure out what was going on with that failing test. So the failure message wasn't telling me much, just hinting that something was wrong with the framework that I had created. So, okay, do you have any logs about the test, like the main specific logs about the test? So I went into this web console where basically I looked at a number of logs. The logs were not there because the test module had been deleted. But I knew that even though the console showed me a lot of logs, it wouldn't show logs for test modules that had been deleted. But the logs would be in the database. So I did a remote desktop connection into the machine that ran the tests. I hopped on to another application to get some configuration so that I could use the beaver to access the MySQL database where that was running and check the logs. And all of a sudden, whoa, I have lots of logs there, way more logs than I can see through the web console. So how am I going to filter this out so that I can look logs that are relevant for the test that was running? Okay, what time did the test fail? So I go to the CD, which is the CI CD pipelines technology that we were using. And now I have a timestamp that tells me whenever the test failed. So I can use that timestamp to go back to the beaver and just filter out the logs for that timeframe so that I can look at less data than looking at a lot, a lot, a lot of logs. Okay, cool. I just found a error message that tells me that the test framework failed or crashed when the test was running. Cool. But in what situations would this happen? I mean, the test framework has been stable up until now. It's open source, a lot of people use it. Let's look at the code. So I fired up my ID, started looking at code. And then I found out the only situation where this may happen is when a setup timer has not run. And that is like an asynchronous process that usually takes place as soon as you publish the test frameworks to start using it. It shouldn't happen, but it's possible that if things happen really fast, you have a concurrency problem and then all of a sudden tests failed because of this problem. The only way I can check this out is did this timer, this asynchronous process fail or did it run after the test? So I went back to the beaver, looked at different things, and the timer ran after the test. Okay. So now I knew why this was happening. Basically, we had this asynchronous, a synchronous procedure that was needed. We needed to run it before any test run. But because things were happening in a very special sequence, tests would try to run tests before the asynchronous procedure ran. And because of that, the test would fail with this message. And this only happened in a few of the runs that we did. Okay. If what I've just explained to you in this really stressed and really fast way was hard to keep up with, let me just tell you, it doesn't really matter. The only thing given this whole double saga I had to go through to figure out why the test is failing. The only thing I want you to tell me is looking at all of the things I had to do, what was one of the things I had to do a lot of times in order to figure out what was going on? Does anyone have an idea? Please write in the chat if you have any idea. And I'll wait for a few ideas from you. Add to look at the database a lot of times. More ideas. Then just give you a few moments to see if I can get any other idea about things that I had to do a lot of times while scrambling around to figure out exactly what was going on. Okay. Gather logs. Yeah. Looking for situations. Checking logs in call service logs in one place. You're getting there. You're getting there almost trying different situations. Sure. But the thing I'm trying to get at folks and all of those are important things was one of the things I had to do a lot was hop, hop, hop, hop, hop, hop, hop. I had to hop between different technologies and different tools in order to look at different data, different pieces of data at different places. I had to actually hop one, two, three, four, five, six, seven, eight times in order to get to the conclusion that I needed. And to be honest, I was on a mission. I was on, like, this was my test framework. And I needed to figure out what the hell was going on because I wasn't letting my test framework be a source of flakiness in our pipelines. But most of the engineers at our organization, they wouldn't go through all of this trouble. After the first hop, sure, they would go hopping from the test failure into the web console for the logs. Maybe they would try to go into the database to check a few things more, but they would stop there. They wouldn't go through all of these hops to figure out what was going on. It was just easier to just hit the rerun button and see if the test would pass on the next run. You know why? Because hopping means friction. Every time you need to hop from A2 to another, that's just friction that you're pulling into. You're putting into your own process in order to achieve your goal. And that causes a lot of problems from a developer experience or from a troubleshooting experience that then doesn't allow you to figure out quickly what's going on. Now, imagine that the story could have gone completely different. Imagine that we just had these different things. All logs in one place, and I see people in the chat that's already hinting at that. Of course, that's really important, having all of these different logs in just one place. And I'm not just talking about the logs from the domain-specific logs, but even like the CI CD logs for the runs or even the failure messages, all of those different things. Imagine that you had data longevity, and that was a big problem in this example, that we weren't keeping the data around for longer than just one day because the machines were being deleted. Imagine that we had tracing information that allowed, that made the data much richer and allowed us to correlate better what was going on at the time when the test was running because that was really important. We had to look at something that was happening on the side to figure out why the test was failing, actually. Imagine that we had a way to explore and navigate this data way better than just different dashboards and different tools just to run some SQL queries. Probably what would happen would be like the tech lead would be like, hey, this test looks really flaky in the past week. I would, okay, let me look at the logs. I would navigate them. And a few moments later, I would just say, oh, I know what's going on. I'll fix it. Basically, I just need to make sure that no test run before this setup is run beforehand. The architect wouldn't even be involved. No one would be aggravated, and it would be really, really easier to deal with the flakiness itself. And if we go back to the conversation, there are a few things with which I want you to think about which are really important. Remember when the architect said, maybe you should output the relevant logs in the test report itself? This is monitoring. This is not observability. When you say that you're going to output the relevant logs in the test report itself, you're saying beforehand while creating or changing the test that you already know all of the data that's going to be relevant for that test. But we were dealing with unknown unknowns here. No one had thought of the problem that the framework has an asynchronous process that maybe is not running at the right time, and we didn't detect that that could happen. I had to go into logs I haven't predicted before that I would need in order to figure out why the test was failing. So this whole idea that, hey, it doesn't matter if logs are available or not, you need to do your homework and include the logs in the test itself. That is wrong for the observability perspective because you can't predict all of the data that will be useful for you in the future. Also, you shouldn't need to debug tests in CI environments. This was also an idea that was behind all of this way of thinking, but to be honest, that's only true if you can look at your test failures in another environment, which a lot of times is not possible because when you're dealing with flakiness or even with legitimate failure, sometimes they only fail in specific contexts that only happen in CI, and or you either are able to reproduce problems in another machine or you have all of the information that you need. You're practicing true observability. You have a true observable system, and then, yes, you can kind of like borrow people from debugging stuff directly in production in CI. Proper automated test observability eliminates the need for reproducing failures, but we can't ignore all of the things that we need in order to reach this automated test. So let's do a small retro about the things that we learned from that story. A flaky test, you don't expect it to fail. It's not like we're building our tests, our automated tests, and then saying, hey, this test is flakies, but I'm just going to get them to my pipelines. Usually in most situations, you build a test, you're confident that it's pretty sturdy, but then you start learning from CI that it's flaky and you didn't expect it initially. And usually at the beginning, you have no idea where the flakiness is coming from. So the ability to ask questions that you have not thought of before is very, very important. In our specific case, we couldn't ask questions because we didn't have the data for the day before. So a lot of times we would see ourselves in situations where I need the data and I don't even have the most basic data about our my test failures. Data longevity was a big problem and that was one of the first things that we need to fix. Also, the fact that the data is scattered just makes it really hard to go after these problems because a lot of people don't know what data is available. They don't know how to access it. And even if they do know, the process of going after it is so cumbersome that we start just thinking if it's worth it or not. You saw me going through all of those different steps because I had a lot of knowledge over the whole system. Not everyone would have been able to do that because not everyone would know all of those different details about where the data was scattered. And hopping means friction, as I've said before. So what did we end up doing or what are the things that we should do? Of course, we should send all of the data to the centralized location and we started doing that. We started seeing all of the data to the data like so that we could look at it. Not just the test results, but also the go CD logs because that would be very important for us to have timestamps and understand which tests run before, which tests run after. Does that play a part in this? Also the main specific logs from the test machines themselves. So get all of those different different log levels into just one place so that you can correlate. Introduce tools to properly navigate that data in the centralized location. Think about that exploration is really important here. Not just having a predefined dashboard that you believe will tell you everything beforehand. And then, and this was the part that was harder that at the time I was in that company, we didn't get to that. We also needed to instrument the tests and product codes to get richer information to explore. That would allow us to barricade it to the use case of understanding what's happening in the machine while the test is running. And circling back to the fundamentals of observability, if you think about the story, I think that it relates nicely into some of the things that I've learned about observability in the industry. Whenever I want to learn, let's say, I'm not like the biggest expert in observability in the world, but the things that I've learned about observability came mostly from two sources, from two people, from two persons. One of them is Charity Majors, and she's the CEO of Honeycomb, which is one of the leading technologies for practicing observability in production. Sorry, failing voice there. The other person is Abby Bangzer, and she's a friend of mine. Abby came from a testing context and an automated test, and they're slowly progressing to observability, and she has been talking to the world and to the testing community about observability in different ways. And there are two key pieces of content that I want you to take home with you, which is one of them is observability, a three-year retrospective, which is a blog post from Charity Majors. And then the other is a talk from Abby, a journey to truly understanding observability. And in both, they talk about what are the fundamental characteristics of an observable system. Now, these fundamental characteristics, there are a lot of them that you can group in three big buckets, which are the ones I want to focus on. The first one is the only way to ask new questions is to keep the original raw data available and queryable. And that ties in nicely to the story I've just told you. You can only ask new questions if you keep the original logs that you have for the tests, and you can still query those logs. This means, in their view, having raw events and not pre-aggregating data have the raw data available right from the beginning and keeping it around for a while so that you can go back to it and ask new questions. The other key characteristic is empowering creative and shared exploration based on business context. In our case, the context here is testing, and the creative and shared exploration ties into having those tools that allow you to explore the data in different ways rather than having static dashboards that tell you everything from right off the bat because it's basically impossible to do that when you're trying to practice observability. And there are also small things around this, oriented around the life cycle of the request, which makes sense in our case as well, and batching up the context like having access to the context of when things are happening. Finally, it's also really important to make data easy to add details to and easy to query. Now, I didn't go too much into this, but this was actually something that was also a pain for us. What this means essentially is that if you see yourself in the situation from a development perspective, a developer experience perspective, that you want to have more data to the system because you believe it will be useful for the future and future questions, adding that data should be a frictionless experience as well. So you should have structured data. You should make that data as wide as possible. You should be able to add as many details as you want to that data. You shouldn't have really strong schemas that just tell you that you can only have these types of data in these places. Now, you should have unstructured data that allows you to add new details and new structure if you want to, and make the data just really, really rich. And just something that we identified we needed to work on, not just from a test automation perspective, but also from a production perspective when it came to our observability capabilities. And if you look at all of these three different G aspects, this is actually a way that the yellow one, the blue one, and the purple one, these are actually ways or these are actually fundamentals that you can apply to your own context. When you think about your test automation and what you have to go through whenever an automated test fails, how much of these three different things are possible in your test automation system? And if any of them are being constructed or limited by the constraints that you have, what are the next steps that you can take in order to have better data, more data, data around and ways to explore that data that will allow you and your organization to be much faster and much more effective in figuring out why your test fails or why flakiness happens, for instance. So as a key takeaway, observability, it's not just for production. And for observability, we need explorability. We need one data store where all of the data ends up in multiple views over that data store and ways to achieve that. This is a direct quote from Abby Bangzer, which I've just introduced there. And to analyze test failures faster and learn more about the changes we're making in our code, observability is the future. I truly believe it's a big part of the future of test automation, and that's why it's really important in that context. Sorry about that. This slide shouldn't be here. Thank you so much. I'm from Proviso Quality Engineer, and I'll be taking questions now if you want me to. Thank you so much, Joao. It was a really nice talk, nice presentation. So I see a few questions in Q&A sections. So first is we normally have future exceptions, stack trace, and screenshot log when we run automation. What else we can improve on that? So I see from Harshal that he's talking about filler exceptions, and stack traces, and screenshots. Log, I think that data usually tells you what happened from a testing perspective. About the software that we're, let's say, testing itself, I think that we should also have visibility over data over that. So you don't want to just have logs about the test themselves, but also about the code that is being tested directly. So having observability over the code that's being tested, I think that's valuable as well. Also, I'm thinking that, of course, that we have screenshots, filler exceptions, stack trace. There's a lot of information that we can also collect about the test themselves. Like what context did they run in? Were their tests running before? Are their tests running after? Could it be possible that tests before left the environment where we are running these tests in the state that causes new problems? Or maybe, like I showed in that case, what other things are happening in the system at the same time that a specific test was running? So having the filler exception stack trace is really useful, and the screenshots as well, but definitely can go way beyond that. Also, having historical data about the tests themselves, which maybe Arshul has that, but it's important to mention, it's really important for us to figure out exactly if a test is like, if it's not, looking at the history usually tells us a lot about the nature of the test. I hope that I answered what Arshul was looking for. Next question, I think this will be the last question we'll be taking, and then we will move to the handout's table. So this is an anonymous attendee, I think. I faced a scenario where my tests are passing on local machine every time, but failing on pipeline. How can we deal with such scenarios? So are passing on every time that's failing on pipelines? How can we deal with such scenarios? So that was like, that was like the that was basically the situation where I was at at the beginning of the story, if you think about it. It's like, if they fail in pipelines, I mean, I think that you need, when you don't have observability yet, I think that you need to look at the test failing, see what information you have available over that test failing, which usually if you're in that situation, that information won't be won't be enough. However, you can think, okay, I don't have this information, but what are the next pieces of information that I would like to have about this? And maybe you want to have more data into the testing itself, or you want to have more data into the code being tested itself. And then you just need to follow those breadcrumbs and say, hey, when this test fails again in CI pipelines, I would like to have this data available so that I know what's going on. This, of course, if you can't go directly into those machines or those environments and run your tests in the same context, so that you can check firsthand what's going on. But that's usually the way I go into it. If I don't have enough data coming from CI that tells me what's going on, then I just do whatever necessary or talk to whoever necessary to make sure that the next pieces of the breadcrumbs that I that I'd like the questions I'd like answering, the data that I need for those questions becomes available in the code so that the next time it fails in CI, I can look at that data and start kind of like solving the mystery. Okay, great. So I see a couple of more questions, but we are out of time, so we'll have to end this. And thanks, Joao, for sharing your experience with us today.


So prior to the current pause on student loan repayments, the White House chief of staff, Ron Klain, said that they would either choose to cancel student debt or they'd extend the current pause at the time. And so they chose to extend the current pause seemingly in lieu of cancelling student debt. So at that point, once we learned about the current pause until August, I thought, okay, so I guess that cancelling student debt is off the table. But apparently I'm pleasantly surprised to be wrong here because the Biden administration told the Congressional Hispanic Caucus that he has some big plans with regard to student loan debt. CBS News reports Biden tells Congressional Hispanic Caucus he's looking at forgiving some federal student loan debt. Now, before we dive into the article, I just want to point out that there are other conflicting reports saying that he's considering canceling all student debt. But regardless, what we're getting, for the most part, is the signal that he's going to cancel a substantial amount of student debt. How much? I don't know, but I think that this is a really positive thing that I want to celebrate, not too preemptively, but if this does indeed happen, it would have big implications, not just for this election, but I think for an entire generation going forward. So the article explains, President Biden is looking at different options to forgive an unspecified but substantial amount of federal student loan debt, a move that would throw some of his most loyal supporters and financially strapped students nationwide, but as a departure from campaign pledges to provide limited relief. The president shared his plans during a 90 minute White House meeting Monday with members of the Congressional Hispanic Caucus. Participants in the exchange tell CBS News. The move could affect more than 43 million borrowers who hold more than $1.6 trillion in federal student loan debt. The second largest debt held by Americans behind mortgages. Representative Tony Cardenas, Democrat of California, who attended the meeting, said the president is open to forgiving debt for college students regardless of whether they've attended a public or private institution. The president never mentioned an amount, nor did the president say he was going to wipe out all student debt. Cardenas said he did a dialogue with us about the differential between young people who went to public schools or private schools. And we, Congressional Hispanic Caucus members, said he should focus on both and he said, OK, good to know. Cardenas said he reiterated to Mr. Biden that the Hispanic Caucus supports executive action that would forgive at least $10,000 in college debt if Congress can't pass legislation doing so. In response, Cardenas said, the president smiled and said, you're going to like what I do on that. I'm looking to do something on that. And I think you're going to like what I do. So first and foremost, I've got to point out that Cardenas and Democrats in general need to stop lowballing themselves. You don't tell the president, well, at a minimum, you should cancel $10,000. You tell the president, you should cancel all student debt. But if you're unwilling to do that, then at a minimum, cancel 50,000. But they always like lowball themselves. So I mean, I wish that they'd stop doing that. But that's neither here nor there. This would be something that would be huge. This isn't just the president of the United States unilaterally doing a good thing. This is the president of the United States, fundamentally changing millions of people's lives. And I don't think that people fully understand the implications of this. This means that my entire generation and Gen Z years for the first time might be able to purchase a car, might be able to move out of their parents' houses, put down a down payment on a home. I mean, this would change their lives. And so if Biden were to do this, I believe this is really the only fundamentally good thing that's massive that a president has ever done in my lifetime. It would just automatically make Biden the best president in my lifetime, just with this one thing, because it's so transformative. And that's not to say that, you know, this isn't Biden writing the wrong since when he was the senator, he made it almost impossible for us to discharge our debt, but for him to right that wrong, to provide people with relief at a time when we desperately need it, people are going to remember this. Now, I'm not going to say that if he were to cancel all student debt, this would definitely save Democrats in the midterms. I don't know what is going to be the electoral outcome. But what I can tell you, just speaking from my own experience, if Biden were to do this, it would fundamentally change the way that I think of him for him to do something good. So I don't know what's going to happen. But I've got to say, I'm expecting at least ten thousand dollars to be canceled because you can't just keep stringing people along saying, maybe I will, maybe I won't. And then you don't like that's just horrible politics. And it's certainly a possibility, right? I'm not going to rule that out. But I think that at a minimum, he's going to cancel ten thousand dollars. Now, if he did that, it'd be good. It's better than nothing. But I mean, you're really only wiping out a few years' worth of interest. And it's not going to fundamentally change people's lives. What we really need to do is eradicate all student debt. That's where you have the most profound impact. And so, you know, maybe it's fifty thousand. That's certainly that's that's huge. But it's not canceling all student debt. So if he did this, this would be incredibly popular. And Republicans know this, which is why just the mere prospect of Biden actually signing an executive order, canceling student debt is making them mauled. So Mitt Romney took to Twitter to denounce his cancellation of student debt, saying desperate polls call for desperate measures. Dems consider forgiving trillions in student loans, other bribe suggestions, forgive auto loans, forgive credit card debt, forgive mortgages and put a wealth tax on the super rich to pay for it all. What could possibly go wrong? Actually, yeah, we should do some of those things. We should at least forgive all medical debt in this country. But sure, let's go further. I mean, this is somebody who literally has an elevator in his mansion just for his cars. So this multimillionaire can't fathom why students want their debt canceled. Mitt Romney's generation, even though he's super wealthy, but his generation did not have to deal with crippling debt after they put themselves through school.  I know this because my college professors have told me that it was a lot easier for them to put themselves through college than it is for my generation. So it's incredibly unfair that my generation gets strapped with student debt that Mitt Romney's generation did not have. So I'm sorry that you don't like that Joe Biden might do something popular and you see it as a bribe. But good, I don't care if you think this is a bribe. Doing good things for Americans is why we put you there in the first place. But this goes to show you that Republicans, they don't like what would happen if Democrats actually got their heads out of their asses and started doing good things. Because then they couldn't just focus on the focus on the cultural issue stuff. They couldn't just focus on Disney going woke or M&M's, you know, redesigning their candy mascots to be less sexy. They'd actually have to find some way to distract us from the fact that Biden just gave us a new life. So they don't know how politically they'd be able to compete with that. So that's why they're melting down.  Republicans must fight this with every ounce of our energy and power. To which David Dahl responded, saying, ah, yes, all the rich people have loads of student debt. You are very intelligent. Exactly.   So don't do it. OK, well, what's your solution? What's your solution? What do you do?  And regardless of how much you hate these students who decided to try to better their lives because you all told us that we need to go to school in order to succeed economically.  Do you just let that bubble pop? Do you let millions of people default on their loans? What do you do?  That's the mentality from Jim Jordan and Republicans. They don't want this to happen because they know that it's a good thing that will make Joe Biden very popular and that scares them. I mean, look, you all could do good things as well, right? Trump could have canceled student debt. Republicans can propose a bill to cancel student debt, but they don't care about ordinary Americans.  So, you know, I love watching them mauled. Just them freaking out about this tells me that Biden really is considering this. Now, the only way that this can go wrong is if Biden all of a sudden says, you know what, I'm proposing canceling all student debt. Let's do it legislatively because then that's just nothing's going to come up. So he has to follow through with his pen specifically. And if he does, you'll see a bunch of Republicans squirm because they won't know how to compete with that. They won't be able to compete with something that is incredibly popular.  In fact, this person responded to Jim Jordan saying, I borrowed around $40,000. Interest brought me over $100,000 in debt over the years. I have more than repaid the original $40,000. I still owe over $100,000. We are not freeloaders. We have been scammed by the banks right this wrong. Similarly, Michael Sonato shared his balance. He originally took out $73,000, paid back nearly $97,000, but he owes $168,000. So this is an issue that isn't just going to go away on its own. This entire generation, two generations now with Gen Z disproportionately who strapped with student debt, we're not all of a sudden going to have enough money to pay off our student loans.  It's a scam. It is a complete scam. So if there isn't executive action, then there's no other way to actually solve this crisis. Now, some other responses that I wanted to share. This is from Brian Tyler Cohen, who writes, hundreds of billions in PPP loans have been forgiven, including for big corporations. Republicans had no issue, but the moment it seems like the president Biden might forgive some student loans to help younger Americans and our economy, Republicans are screaming about handouts. And that's exactly it. Ida Bellwell says, I came from a working class family. My parents took out plus loans and desperate to make something of myself. I took out tens of thousands in student loans after repaying many multiples of my original debt at 45.  I'd wish that on no one coming behind me. Exactly. I haven't been able to pay off my student loans. I'm nowhere near paying off my student loans. But if I were lucky enough and privileged enough to be able to pay off my student loans, I wouldn't just think, oh, well, I paid off my loan so nobody else should be able to because being able to achieve that is statistically unlikely. I mean, most of us will have this debt until the day we die unless Biden cancels it. So if Biden were to do this, I mean, this entire generation, two generations would sing his praises.  But apparently he's supposed to make an announcement about this before August. It'd be nice if we knew as soon as possible, but I understand that he's trying to maximize the impact of this and, you know, schedule this closer to the election. OK, whatever, I get it. Play politics off of this. But I mean, just do it. It's a good thing. Objectively speaking, it would save a lot of us. It'd be a boost to the economy automatically. You have millions of Americans with a lot of purchasing power. All of a sudden, more income every single month that they didn't previously have. And we already know that this isn't going to impact the economy in a negative way because with COVID-19, we haven't been making our repayments. So you can't even use that argument. There's no reason to not cancel student debt. So Biden needs to do it. And all of the screeching around him, the fact that Republicans want to stop him from doing this tells me that he might actually do it. But I'm not going to get my hopes up. You know, I'll believe it when I see it. But do this and we will sing your praises basically forever, Joe Biden. It doesn't mean that all of a sudden you're a deity, but it does mean that we will remember this in election times. And you set a new standard for Democrats. You can prove to other Democrats that doing good things for people, giving them more money is popular. And Republicans know this, hence why they are freaking out.   Don't contemplate any longer, do it, and this will pay dividends for the Democratic Party for quite some time, I'd imagine.


Good evening, Geeks. Yeah, myself, I'm Ramu. Working as an architect with Imaginea. I'm here to share some of the insights that we learned while developing products and tools for some of our customers, not that the examples I show in the slides are representation of the customers. So this is the agenda that I want to cover today. I'll introduce you to the problem, and then what's the benefit of deep links, and what are the various scenarios in which you can apply deep links, and then give you with the conclusion, and what are the further resources that might interest you. To set the context now, yeah, how many of you are familiar with this screen? Where you got a mail from LinkedIn, and you click on it, you're taken to a web app where you're asked to sign in. And yeah, most of us sound familiar, right? We have LinkedIn app already signed in, but we land up here. And I'm sure many of us would have gone back to the home screen after this, right? So in the world of web, we are all used to moving from one link of one website to another link of another website seamlessly. Did you any time wonder why that is not possible in the world of apps on mobile? So that's what I call as a broken user experience on mobile, which obviously translates to a lot of potential business opportunity loss. So that's where I introduce deep linking as the bridge or the solution with which we can actually fix this broken link and then enhance the user experience. So technically, what deep linking is? Deep linking is all about tagging or linking any specific location of an app. And how you do that is with the URI as we do any web page. Today, there are no set standards, but typically we follow this URI scheme. So in a nutshell, what deep linking allows? So deep linking allows you to get the user back to your app from multiple sources. So be it you click a link on an SMS or an email or an ad or whatnot. Anywhere on the mobile, you're back to your app, which means that you are getting the user back to your app again and again from multiple sources. This is just a small glimpse of what are the possible scenarios. So when you're getting the user to your app from multiple scenarios, obviously the advantages are going to be manifold. Your experience has increased drastically. And the user engagement, the amount of time the user is going to spend on your app, shoots up drastically. And your app is going to be discovered again and again.  Deep linking helps there, what do you say? Very often. So if you're bringing your user again, it means that your monetary chances are high and your campaigns are more effective. So I've listed a lot of advantages. So if that motivates you enough, now let's understand how we are going to wire up deep linking or how it's going to work. So I'll split this working part of deep linking into two scenarios. One is like, how I can apply deep linking on the apps that are already installed on my phone? Versus, how I can apply deep linking on the apps that are yet to be installed? So you can apply deep linking in both the scenarios. So starting with the first use case where deep linking can be applied for installed apps. Okay, consider this use case. You have a lot of, I mean, you want to buy a product and since we are in DroidCon, I thought I'll try buying a Moto 360 watch, okay? And I went and then Googled for all the links. Then I got a list of all the search results and say I clicked the link and I'm landed up on this page where I have installed this app already. Now I'm given two options. Either I can continue shopping using a web app or I should download again. You got me. This experience isn't it suboptimal? Right. So now enter the scenario where the app has enabled Google app indexing, which is one means of deep linking. So down in the same search results, here Google is saying, Google search in fact, not Google. Google search is saying, hey, you can open this link with an app that is already available on your phone. You saw the magic Google in its search is able to figure out that an existing app on your phone can actually is, will be able to open the link. And the moment you click on it, you're taken to an app and the user is more happy now to land on this page and probably, I mean, if it's keen on, the conversion is going to be much more seamless. So, yeah, we are geeks, right? So now if we want to do that, how we are going to do that. So the first and fundamental step of enabling deep link starts from your app. So I should deep link my app. So all I have to do is, given you are right to each and every activity that I'm going to wire up in my app and then give a URI scheme. So once this URI scheme is declared in your app, you have to share it with Google, right? So that is where, when you are pushing your app onto Google Play Store, there's an option where you allow Google Play to actually crawl your app content. So you have to opt in for that. That means Google is going to crawl your app and then see what are all the activities that are there. Now, once that is done, you should connect your backend with the content in your app, right? So that step is done in two ways. One is updating your site map. So in your site map, you're going to tell for each and every page in your website, what's the corresponding app link? And it should follow this specific URI format for at least Google to determine that, okay, this particular page on the web is similar to the same activity, this activity on my app. So this is one way or the other way is, you go and then update each and every page of your website and in the header of that page, you say, hey, this is what is my URI for the app. So that way your Google bot is going to crawl your website, map it with the content in your app and when it's going to do a search, it's going to map both of it and then show you this result, where you can actually see the app in the Google search results. So if you think this is too many steps, yeah? Can we take the questions at the end? Yeah, it's just because it's a short talk. I'm reserving time, yeah. So if you think it is too much of steps to be actually done, we have a very simple method of again deep linking, okay? So let's get to that example. Okay, this is one more shopping site, okay? Which is called Locracy, where you again search for some watches, okay?  Android system detects that, hey, this link can be opened either again in the browser or there's an app that is installed on your phone which can handle the same URI. So isn't it easy and then the user will be happy to open the app, right? Because it's already right there and it's going to get a best experience. So doing this is a bit easy as compared with Google search. So all you have to do is deep link your app, which is like you just have to wire up each and every activity with the corresponding URI that maps to the same URI on your webpage. So here you don't need to actually tell Google search, yeah, do crawling and all this stuff because the URI schemes of your website and your app matches. And that's why you saw this dialogue saying that, hey, either it can be opened and the browser are in the app. So now that we have seen the implementation, do you want to see what's the advantages of deep linking? So apps that have enabled deep linking, yeah, apps that have enabled deep linking are actually getting, I mean, they got a jump of 2.5 times more than those apps which are not deep linked. So you can see this graph which also indicates those apps with deep links are actually users are spending more time here as compared with apps without deep links. Okay. So now let's get onto the second scenario where you can, how you can enable deep links for apps which are not yet installed, which where you are actually driving the user to go and then install your app. So this technique we call it as deferred deep linking.  The moment you do that, you're directed to the Play Store or Google Play and then you download and then install the app, what do you see? Your home screen, the coupon is not there. So many of times would have faced this, right? And you're left for searching the coupon. Again, a bit of suboptimal user experience. What if I open the app and I'm able to see the coupon? So then the conversion and then the chance of the user redeeming the coupon increases many fold, right? And to see it exactly, this is what Airbnb has done. So if it is a normal download, they take you to the default homepage. But if it is from a referral campaign, the moment you open the app for the first time, all you get is, hey, you're going to get a discount of this much, do you want to sign up? That means the chances are more that the user is going to sign up in this process. So how we do this deferred deep linking? Again, with Google Play, you have something called as install referral parameter. So with every coupon link, you can send along with the Google Play link, you have to give an install referral parameter. And once the user installs this app, downloads the app from the Google Play, after installation is complete, Google Play is going to send you a broadcast with the install referral parameter. So along with this install parameter, you're going to get the coupon link. So you can handle in your app on receive of your broadcast receiver, say that, hey, because of this, I'm going to take you to this corresponding page. So you want more details about this? There's this github link where a guy has posted complete sample code as how this install referrer is going to work completely. So what magic has this done for Airbnb? It got 300% increase in user signups just because of this small change. And it got a 25% increase in bookings. So that's what deep linking does. So you make your customers happy, you get your users back to your app again and again. Yeah, so that's the advantage of deep linking. And on a concluding note, while I say deep linking is getting your users back to your app again and again, I just want to conclude with some of the points that you might have to take care of while implementing deep linking. So we are saying deep linking is about wiring up your backend and then your mobile app, right? So it means your mobile team and then backend team should probably be working very tightly. And there are a lot of open source deep linking frameworks available already. You might have to take a look at it. And since you're saying as an app, I'm going to support a lot of URLs. There should be a robust error handling that should be handled in your app. Otherwise, chances are that you might, there's no possibility that like a browser which can say for not for link not found, you probably have to wire that up. And in case you're using, releasing an app on multiple platforms, you probably have to check what is the best scheme for you to enable it on multiple platforms. Yeah, that's it. I'll leave the rest of the slides for you to actually go ahead and then see as what are the deep linking landscaping is all about. And there's a lot of startup ecosystem who are actually focusing a lot on deep links. So any one of them can be a good resource for you to start off with. Yeah, that's it guys for me. So in case you have any questions, yeah, you had some questions. So on that slide where you had that site map for the deep linking. Right. If you can quickly please open that. Yeah, exactly this one. So I'm assuming you mean LOC slash HTTP colon slash slash, because in site map you will have HTTP links. This one? Yes. No, this is the URI scheme of your app. It's typically for Google. We say if your app package name is com.example.android, the URI scheme starts with the scheme. If you can see the scheme of this is actually like this. You can see the scheme is starting with example and then the host is gizmos. So that is why we say the same thing. So this site map is my website site map? Yeah, you have to specify in your site map which page of your website and it has an alternate content on the mobile app at this URI. Okay, got it. Thank you. Yeah, I want to know if deep linking is only means that it's for Google, correct? Like, so deep linking means when someone search, like make a Google search result, like not any other, even not any other thing on browser. So the Google search result will only drive you to your app. No, I gave you another example, right? Where in respect to your Google search you can as well do deep linking. Where I told if you can map up, yeah, this example where the URI scheme is the same as your web page. Here Google doesn't come into play. Okay, so you're saying for any page? Yeah. Like any URI on any browser it will. Exactly. As long as your app is able to handle this URI, the same URI as your web page, then you're good to go. Okay, thanks. Please download the talk funnel app to ask some more questions to Ramu. So up next are the flash talks. Yeah, thanks a lot guys.


The fiery horse with the speed of light, the cloud of dust, and a hearty hyosilver, the lone ranger. During the years of unrest that followed the Civil War, a powerful secret organization called the Legion of the Black Arrow sprang up in the western United States. Its members were to be found everywhere, defying the law or using the law for their own purposes, working toward the ultimate goal of revolt and the foundation of a despotic empire. It was the masked rider of the planes who led the fight against this band of outlaws and traitors. And for once, his great strength and courage, his daring and resourcefulness were taxed to the utmost in the cause of democracy. Return with us now to those thrilling days when the West was young. From out of the past come the thundering hoofbeats of a great horse silver. The lone ranger rides again. There is untold worry and misery in Calabash. That was the way the note began. It was the latest of many messages that the lone ranger had received from the same mysterious source. He knew but two things about the rider of the notes. The rider was a girl, a small girl to judge by her footprints, and the information she gave the masked man had proved reliable. We're nearly there, Tunnel. Good place to leave horse in Cottonwood. Right. Steady silver. Oh, there's still lights in some of the buildings in Calabash. Beyond the cafe, that's where you to go. That's right, Kimosabi. I'm to go there wearing my mask. I don't like ID. I don't like this evening's business either. It seems to me that it's just about the most unusual request this unknown girl has made. That's right. If you go masked and alone to the cafe, I'll do it. Oh, Kimosabi. Yes, Tunnel. Maybe sometime someone learned about girl who send note. Well, maybe send take note. Note that take you to trap. I know the girl's handwriting. Oh, maybe someone write same way. We'll take that chance. You'll always be on hand if there's any trouble. Oh, even though you don't trust this girl too far, she's been a big help in running down crooks. There now, Silver, old fella. You stay here with Tunnel for a little while. We'd be ready for fast run when you come. You'd better be. I'm likely to be just a jump ahead of bullets. Hunter not like plan for tonight. I'll be back in a little while. And maybe we'll find out just what this letter is leading up to. What's the matter, Sheriff? You aren't drinking tonight? No. Maybe you'd like me to fix you one of them fancy drinks for the liquor that comes in bottles instead of cans. I don't want anything, Bart. Where's Maisie tonight? Didn't she's singing in your cafe anymore? Sure, sure thing. She'll be singing a little while now. And I'll wait to see her. See her? To hear her, you mean, don't you? I said I'd wait to see her. Look here, Bart. Have you noticed anything odd around town in the past couple of weeks? No, I can't say as I have. Yes, sure. Of course, I'm sure. What should I see around town that isn't the usual thing? Sakes alive, you're acting doggone mysterious tonight, Sheriff. You seen anything at Grand Dundee during the past week? Sure. Only yesterday I saw him. He was over to the land office talking that little half-pine-sized army that handles the real estate. How'd he look to you, Bart? Well, it tell you the truth, Sheriff. He didn't look like he was enjoying his vitals. He had a sort of a tired look about him like he hadn't slept much. And he was selling his land. That's what I heard. Can't figure it out, neither. He's willing to sell his land for almost nothing. Anchors to leave these parts and take his wife along with him. So he says, did you hear that he ever needed cash money? No, I should say not. He's always had a whopping pile of cash. Oh, there's Maisie now. Sure. And isn't she the handsome girl? I'm downright lucky to have a girl like her singing in my cafe. Downright lucky. I'm going to speak to her. Hello there, Maisie. Your name? I'm Maisie Hustrix. How's that handsome gentleman friend Maisie? Sit here, Maisie. I'll be with you by the bar boys. A little errand first. Maisie, I'd like to speak to you a second. Oh. Hello, Chef Collins. What are you looking so scared for?  No, I know. I'll see you by and by, Sheriff. Well, hold on. I want to ask you a couple of questions. Please let me go. I got to sing in a couple of minutes. Well, the plan is that way, not out the front door. I want a little rest here. Rest here. That's all. I'll be right back. Let me go. You bother me. Mm-hmm. Mighty nervous. Now, why is it? Let go of me. I know I am. Are you waiting for me? I didn't mean to startle you. Your name is Maisie, isn't it? Here, take it. Take it. In the name of mercy, leave me alone. All right. I'll have a little watch in there. The Sheriff. You're covered, Mr. and I'm going to talk to you. Give me that package. Hurry, stranger. Get away from here. No, Sheriff, don't. Let me get him. Let me get him. Maisie, let go of my arm. No, no. You mustn't stop him. Let him go. It's all right, Sheriff. It's none of your business. I'm making it my aspect. What is this? Eyes that drop the gun and keep your back toward me. If you don't, I'll shoot. I'm not fooling. Well, I was on nerve. After you, Maisie, you'll come with me. I mean that, too. Who are you? That's what I'd like to know. Come on, Maisie. If you do what I say, you'll be back here in time to sing your song. Don't turn, Sheriff. My gun's pointed right at your shoulder. I'd hate to put you out of business for a few weeks. Well, where should I go? There's a horse for you back of the building. Get in the saddle and I'll join you. We're going to take a ride in the moonlight and talk. The Lone Ranger returned to the spot where Tonto was waiting on the edge of town. He carried the package still unopened in his hand. I don't like it, Tonto. You do what letters say? Yes, I took the package from a girl who was almost paralyzed with fear. And the Sheriff came from the cafe. He'd have taken a shot at me if that girl hadn't shown her weight on his shooting hand. What in package? Open it up and find out. Maybe we'll account for what I had to do. No one come after you. Tonto, watch. And we can stay here for the time being. At least long enough to see what Maisie gave me. And he's wrapping on the package. Yes. Silver, I'll be glad when this is over. I guess you will, too, won't you, old fella? You must hobby. Look. Money, huh? Oh, much money. No message of any sort. Nothing but money. Tonto, there must be thousands of dollars here. That's right. Now, what do we do with money? Why did Maisie hand it to me? Why did she stop the Sheriff when he tried to stop me? Where'd she get money? Tonto, there's a mystery here that might lead almost any place. We'll stop the horses here and talk. If you want to, Maisie, you'll be able to get back to the cafe in time to do your singing. The Sheriff doesn't pay us down and call you to account. Who are you anyway? My name wouldn't mean a thing to you. Well, what do you want? Just a few facts from you. Where did you get all that money you gave away tonight? None of your business. Who else has had to make a big payment like that? I'm not telling you a thing. Who ordered you to go on the porch of the cafe and hand the money to the man who came up to you? Go on, girlie. Keep asking questions. I'm not answering any. You get tired of talking for nothing. Maybe you'll let me get back to work. Maisie, I'm going to tell you something that'll scare you. Get ready for it. Scare me. All right, then. Listen to this. The man you gave that money to is generally known as the Lone Ranger. The Lone Ranger? No. No, he couldn't be. He was. It was the wildest story I've ever heard. I've heard all about the Lone Ranger. He's not a... Not a what, Maisie? I'm not talking, Fee. I'm not saying a word. Then I'll talk some more. I got wind of the fact that you were going to hand over the cash tonight. I sent the Lone Ranger a note. Why? I asked him to be there to take the cash. Now the man you were really supposed to hand it to will think you failed him. He'll think you defied him. Dared him to... Well, to keep his word. Listen, is this the truth? It's as true as I'm sitting on this horse. You heard the voice of that man. You heard the Lone Ranger as a voice that doesn't sound like anyone around here. He was masked, wasn't he? Yes, I told him to come on foot because I thought he might be identified if he came on that white horse. He's been pretty active around this part of the country. I didn't want him to be stopped before he took the cash from you. So he did come on foot. He got the cash and got away with it. I wonder now just what'll happen to you. Wait. Do you know what you've done to me? In the name of Mercy, what'll I do now?  The same as they told Mrs. Grant Dundee. Who? You're not the only one, Maisie. Mrs. Dundee was made to pay. Grant Dundee had to sell his land to raise the cash. He sold it for a third of its value. Then there are others? Yes. Why didn't you pick up one of them? Why didn't you take someone else? Why did it have to be me? Wait a minute, Maisie. You needn't be afraid. You said you heard of the Lone Ranger. Well, he's on our side, on your side. Can't you understand that? What can he do? He can do a lot. If you'll help. Nobody knows who... who these fiends are. You've got to help, don't you see? If you don't help get this gang, I'm... Gang? I'm sure that more than one man is involved. If you don't help get them, they'll... Well, you know what'll happen. Yes, I... I know. All right, then. Now come back to town and sing your songs as if nothing happened. How can I?  You can down some cold water in your eyes and gone. Nothing'll happen to you while you're in the cafe. And when you leave, the Lone Ranger will meet you. How do you know? How can you be sure? He'll be there. And you? Who are you? Have you been threatened, too? Where can I find you? How can... You may not see me again, Maisie. I'm going to get word of the Lone Ranger and for the sake of other women, for your own sake, tell him all you can, and trust him. No argument about it, Tullo. We've got to wait here. Oh, me, not like it. We can't right away and take this money with us. We've got to do something with it. Give it to the owner. Don't wait. My Tullo, you're a prejudiced against everything, this mysterious girl request. She said in her note that if we wanted to help her, we should remain here unless we were chased. She'll probably get word to us in some way. Well, what we do then? I don't know. This amount of money is involved, though. We can be sure there's something big going on. You think maybe girl will come here? I doubt it. She'll probably send someone with a note for us. Oh. She's taken every precaution to make sure we don't see her. Oh. I wonder who she is, what she looks like. Where does she get her information? How does she learn so much about us? Not us. I learned plenty about you. It's the same thing, Tullo. I... Sir, if we give one, someone come near. Tullo, it might be. Maybe it's a message. Hello, sir. Just a minute. I'll be there.  I'm here, sir. Are you right and catch girls? She said read the note. She's depending on us, Tullo. I'm heard. Right. You write hard. Maybe catch her. I can't do that. She wouldn't have come here herself if it wasn't urgent. I stopped to read the note. There'd be no chance to catch her. Here's the rock. Ah. The man knows. All right. There's still enough moonlight to read it. As the masked man read the letter, explaining what the girl had found out, his lean face hardened. His eyes behind the mask of his jaw reflected steel determination. Tullo, we've got to do something. To find out who's behind the crimes that are threatened. Come on up. Where we go? Down to Calabash. You've got to have a talk with that singer at the cafe before it's too late. Huh? You ready? We're riding into danger, Kimosavi. Ah. Get up, scoundrel! Still there! You have just heard the first act of our Lone Ranger story. Before the next exciting scenes, please permit us to pause for just a few moments. Now to continue our story. As the Lone Ranger and Tullo rode into the town, Maisie was talking with the tall, dark man in the cafe. But, Jack, you've got to let me leave here, Lone. I can't explain now. You don't want to meet someone. Ain't that it? Well, maybe. But, honest, Jack, I can't tell you anymore now. It's about time we told folks around here we're married. It's a good time the way all the men want to dance with you and buy you meals and presents. That's just it. I can't let our marriage become known. Bart wouldn't let me sing here anymore. He'd say our wife's place was in the hall. What do I care? I say the same thing. I've got lots of money. My gold mine pays rich. Oh, you never will, understand, Jack. I want to be a great singer. You? Oh, I know I ain't. I haven't got the education, but someday maybe. I'm studying all the time. Now let me go. What's the matter with you? You paid that cash, didn't you? You ain't got nothing to worry about from those scheming crooks. I paid it? Yes, I paid it, Jack. Right at the time I was supposed to. And what's hailing you? Why you got to leave here alone? Who you meeting? Jack, I'll tell you all about it later. Honest, I will. I'm not even sure that I'll see him. But I've got to leave here alone. And now. All right, Maisie. Go ahead.  I'll just follow her alone. She meets a man. Well, I can throw a knife as straight and true as any man alive. As Jack followed Maisie from the cafe, keeping to the shadows, he clutched a knife in his right hand. A little singer hurried on toward the appointed meeting place and found the lone ranger waiting for her. You are the man who took that money? Yes, you remember me? Why did you do it? It will be returned to you. I want you to come to Grant Dundee's house with me. Why? His wife was threatened with torture just as you were. Maybe we can learn something if we talk it over. Oh, there's nothing I can tell. We'll go there anyway. By comparing notes, we may be able to find a clue. At least you'll be safe from Dundee home. At Tutto, you can walk the horses. Ah, how to do it. But wait. Maisie, you've got to be guarded, you know. Oh, that's right. Don't you know anything about the man who demanded all this money? No. Didn't he write you a warning? All right. I had to burn it. I kept it for a couple of days, but he knew I hadn't burned it. He threatened me with awful things if I didn't obey. Oh, how did he threaten you? Well, he spoke to me one night in the darkness when I was going from the cafe to the place where I boarded. He spoke to you? Yes. He came up beside me. It was terribly dark. Couldn't see his face. He had it covered with a cloth. How tall was he? I don't know. As tall as I am? I can't remember. He seemed huge in the darkness. He was much taller than I am. But I think he was taller even than you are. And heavy. Fat. He was fat, you say, Maisie? Yes, that's right. Don't you know of anyone in town like that? No. I thought at the time that he must be a stranger in town. Well, you see, Maisie, you've already given me some information that may help. So? He'll be at Dundee's in just a moment. He'll then... Look out! What? You! Jack, you!  Let go of me. Let me go! Not just yet. I want to know who you are and why you threw a knife at me. Jack, have you lost your wit? This is a lone rain. I don't care who it is. He's the one that got that cash, my cash. Your cash? The truth may as well come out now as ever. If your friend hadn't jumped me out of gotcha, that's the way I wanted it. Now the law will have to take you instead. You're the one that threatened Maisie, and she's my wife. Your wife? Maisie, is that true? Yes. We're married. We had to keep it secret. I live in the Roman house and Jack in the hotel. Jack, this is a lone ranger. He got the cash, but he'll give it back. He's trying to help. A likely story. You come along with us, Jack. We'll all sit down and grant Dundee's house and have a conference. His wife was threatened too. What if I don't want to go? Bring him along, Tonto. I'll Tonto do it. Oh, let go of me. I'll go with you. Dundee and his wife, Maisie and Jack, and the lone ranger and Tonto, sat about a table in the Dundee home. The mask man said, We hope to get something definite to work on, Mrs. Dundee. I don't know how I can help. Maisie didn't think she could help either, but she gave me some very important information. What was it? Well, she received a letter telling what she had to pay and what would happen if she didn't pay it. Same as my wife did. I paid it. I'd do anything before I'd let something happen to Molly. Were you instructed to burn the note, Mrs. Dundee? Yes, me too. Maisie didn't burn the note at once. Somehow the crook or crooks knew that she didn't burn it. When she was going from the cafe to her rooming house, a man approached in the darkness and told her she'd better get rid of the note. No one said anything to you did the Molly. No. Didn't you burn the note either? I kept it. I put it away. I paid the cash, but I aimed to try and find the crew that got it. Maybe that note will help. I still think that man's the crook. What's that, Jack? Maisie said the man that approached her was big and fat. Sharks, this genie, fatty. He could wear a lot of padding under clothes. It was too big for him. He could look taller in the dark. He's plenty tall enough as it is. The fact still stands. He got the cash from Maisie. Jack, this is a lone range. You know why he got the cash. You'll get it back, Jack. Your dog gone right, I will. Mrs. Dundee, are you sure your first note was the only one you got? Oh, yes. Just the one. I put it away in my strongbox. Dundee, I'd like to see the note if you don't mind. I don't mind at all. I'll get it for you. I keep it in the strongbox over here. You have to keep cash here, too. When I had the cash, I had this. I'll lock it for you. I don't see as this is getting us anywhere. You'll hand over my cash. I'll go home. Stay with us for a while, Jack. Pretty high-handed way of doing things. Yeah, there we are. There's a lot of papers here. I keep most everything here. Ink. Going stuff spilled out a while back. I keep her here with a pen, so it'll be handy when I got papers to sign. And let me see. Deeds of land and mortgage. I'll get that note here somewhere. You let me have that box, and I'll try and clean up things where the ink spilled. No, it's old right now. Here we are. Here's the note. I don't know if it'll help much, but you're the judge of that. Have you seen handwriting like it anywhere? Nope. Macy, you look at the note and see if it's the same writing as the one you had. All right. While you're doing that, I want to talk to Tonneau outside. Come along, Tonneau. Hey, you know I sure like the style of that masked man's talking. I don't know why it is, Grant, but somehow I feel that with him on our side, everything will work out all right. You mean maybe I'll find these terrible criminals that are robbin' us? I'm hopin' so mad. I never will. Well, I won't nobody be safe until he does. Who rode off, Richkin? Go on and errand for me, Mr. Dundee. We have nothing more to talk about until he returns. Now, let me go home. I'm tired. Sit down. But listen here. Sit down. All right, all right. I don't know why you're so dead-setting the little Ranger, Jack. Oh, you don't, huh? Nope.  You're sore because he's the one that's helping me and not you. Hey, who paid that cash? Who did? But you didn't have any ideas of trying to get the cook. And risk your life by doin' it? Not me. I'm just afraid this little Ranger is gonna cause all the women that have been threatened to really meet up with trouble. These crooks are smart, mighty smart. They'll know what's goin' on, and you can mark my words, they'll do somethin' about it. We'll take that chance, Jack. Maybe it would be best to let things stand, Missy. I have an idea that when Tonneau comes back, we'll know who the criminals are. Huh? And we'll just sit quietly and wait for Tonneau. Well, you might return my cash in the meantime. We'll wait till later for that. You plan to keep it? No, I don't. Why don't you return it, Missy? We'll wait. But a minute ago, you said... A lot has happened in the last few minutes to change my mind. We'll wait. Tonneau returned to the Dundee house. He dismounted hurriedly and burst through the door. Then he gave a signal to the lone Ranger, and the masked man rose to his feet. All right. Now it's a showdown. Come with me, all of you. Me, too. You, Dundee, your wife, Maisie, and Jack. Tonneau, you get the sheriff and the deputy to take charge of the criminals. You made it. You really got him? We know who he is. Come on. Where are you going now? The hotel, Jack. To one of the rooms in the hotel. Hey, this is my room. That's right, Jack. Sheriff, have you told your deputy that he should be ready for trouble? Yeah. And I'll tell her what I know. Maisie. Huh? Oh, I'm so confused. Nice you handed me a bundle of money, didn't you? Yeah. What did you get the money? Jack gave it to me. Where do you get all of your money, Jack? I've got a gold mine. You didn't get it from anyone in town, then? No. Dundee, pay attention to me. Oh, sure. What is it?  That's right. You kept your cash there? Some of it. What I used to have cash.  You gave it to the man who sent the note of warning. The note had no ink on it. It must have been put in the box after the ink was spilled. But the cash you put in the box, some of it, had been there for some time. You paid that cash to the extortionist. Left it where the note demanded. Tell me, then, Jack, how it happens that you gave your wife those same bills. How did you get Dundee's money? You mean you accused me? I don't think I'm here to search your room. Tonto, show us what you found. Uh, here big clothes, here padding. That would make you look bigger, Jack. Why are you... Go on, Tonto. Um, here high heel boots. To make you look taller. Jack, you... All right. Stand where you are, all of you. I'll shoot to kill. Look out, Grant. He'll shoot. You're smart. I wondered if you caught the sign on my wrist when my sleeve slipped up. Jack, I did. I suppose you had to raise a lot of cash for the black arrow. You sent your own wife a note so you could pass the word around how dangerous it was to cross the orders that were sent in those threats. So that's how you knew I didn't burn the note while your two-faced double dealer...  I'm leaving here now. I'm leaving and I'll shoot the first one... Jack, before you go, let Tonto show what else he found. Uh, here all cash that crook collect. I will be returned. You can take charge of it, Sheriff. Yeah, I will. Here's the cash you paid to Jack, Mr. Dundee. It wasn't extinct at all. I got it from the landowners. I know. I used that to make Jack admit the crime and show his hand. You won't leave us, Jack. The sheriff has deputies outside the window. Too thin. They ain't turning my head. You needn't. We'll tell you that we're here. What's that? Oh! Is that good, isn't it? Take charge, Sheriff. Tonto and I will be on our way. Hey, stop. Oh, go on. They went out that window before we got a chance to thank him. Sure, that was a long range here. I had you to promote me, I sure like the way he works. Tonto! Hey! The story you have just heard is a copyrighted feature of the Lone Ranger Incorporated.


Walks tonight, but did you feel like you were dialed in for the most part all night? I felt like I was getting better as the game was going on They were to throw more of the pitches for strikes and that was a big help, but you know, luckily On the hit and run row through them out and then we got that double-play ball as well. So a Couple big plays to get out of a few of those walks It's going through your mind when you retire 10 straight 11 of your last 12 Trying to keep it going trying to pound the zone You know definitely trying to avoid those walks, but I think as the game went on Was using more and getting ahead and that was key remind complimented your ability to really use both sides of the plate When you're able to do that. Do you feel like that's when you're at your best? Yeah, it gives me a lot of confidence You know to use multiple pitches there. So that's always the goal don't always have that but you know tonight Especially later in the game. I felt like it did Hitchers in it now they're kind of a little bang-up and guys are struggling But does that make does that put more pressure on you? Or how do you how do you look at that when the rest of the rotation is having a harder time? I think it's just part of the team aspect. I mean you were always trying to pick each other up no matter what the situation Yeah, I came to be a part of it and trying to help any way I can But a good team win for sure tonight You know to come to a team in a pennant race It's motivating to pitch in Yankee Stadium. It's motivating to be on this team being a pennant race and Heck yeah, I mean this is fun


The study compares the cost-effectiveness of enzyme replacement therapy, ERT, bone marrow transplantation, BMT, and supportive therapy, ST, the mild to moderate alpha manacidosis using a discrete event simulation model from the perspective of the Serbian Health Insurance Fund. The results show that ERT is not cost-effective compared with ST or BMT, while BMT can be cost-effective compared to ST but only if the willingness to pay threshold is increased. To make alpha manacidosis therapy widely accessible, criteria for assessing the cost-effectiveness of orphan drugs should include other aspects like equity weightings of QAL-wise, risk-sharing, reimbursement of severe forms of a disease only, or availability of dedicated funding. This article was authored by Anna Antonasković, Ivana Stedik, Reflect Gojek, and others.