Good afternoon everyone. Thank you for having me here. This afternoon we're going to be talking about testing a bit. I'm assuming for this talk that everyone in this room, two things about everyone in this room. First off, that you love testing at least in theory. And second off, that when it comes to your actual tests, the ones you're actually dealing with on a day-to-day basis, you maybe love those a little less. Everyone values testing in the Rails community now, in the Ruby community. We've kind of won that battle and this is great. Even people who maybe don't find as much value in test room development or in other specific testing practices still agree that you need some kind of testing. We fight about testing even though we collectively value it because pretty much no one loves testing all the time. Even died in the world to DDD fanatics like me. It's okay to admit this. The testing police are not in fact coming for you. Today I'm going to help you fix that mess. Today we are going to learn why we hate our tests and why we fight about them. We're also going to learn about some ways that we accidentally make our tests worse by enhancing and making them a little too interesting. All is not lost, though. We are also going to learn about how to fix the ways we accidentally make our tests worse. While we're at it, we'll learn about how we can make our application code better at the same time.  In order to do this, first we're going to talk generally about some underlying causes of test hate and fights about testing. Then we're going to talk through three specific ways that our tests can get annoying and learn how to fix their root causes. So, why do we hate our tests? The simple answer is that tests are code and code is terrible and therefore tests are terrible. But getting a little bit less snarky and a little more real, we write our tests with imperfect knowledge. We also write our application code with imperfect knowledge. We can make educated guesses about what we'll need in the future, but that's all they are, is guesses. We can never get away from our assumptions or from our current mental model when we write application code.  Now, whenever we've made some incorrect assumptions in our code base, and this has made our tests or application code or both really painful to work with, there's always going to be some died-in-the-wool TD fanatic, like me, who insists on making a total jerk of themselves. They will say, it is not a problem. Why don't you just try listening to your tests?  But why is listening to your tests? Why is just listening to your tests such a jerky and annoying thing to say? You're presenting your listener with this.  And, like, I used to do this. And so I know that what people who say just try listening to your tests mean to say is, I see something wrong and I kind of know how to fix it. I think please just give me a chance. And so in this talk, I want to fill in the question marks a bit.  Before we get started, everyone take a deep breath. Bad code happens and bad tests happen. This is fine. Totally fine. Code is terrible because the world is complex. And while it's good to be honest with ourselves about when complexity is useful or necessary, complexity happens. We are paid a lot of money because complexity happens. Complexity happening to you does not mean that you are a bad developer or a bad human. It just means that this is a hard job. Also, sometimes you actually need an esoteric testing technique. None of these techniques they're about to talk about as they test smells are inherently bad. They just can be overused. Finally, I have done every single thing I have about to show you in production, especially the parts I say are bad. Every talk I give is me lecturing my past self about a mistake I have learned better from. And this talk is no exception. So with that out of the way, let's begin. The first test smell we are going to look at today is testing private methods. When you see this line of code in the test, chances are someone is using it to test a private method. Now, I have heard a lot of different justifications for testing private methods directly, instead of indirectly through public methods. They boil down to three things. First, the method might have weird indirect results that were hard to introspect. Otherwise, maybe the public methods were really expensive. Maybe you instead also wanted to isolate the side effects. These reasons are valid reasons that are hard to argue with. Even if you're feeling uncomfortable about testing a private method. So I suggest that we not try to confront these arguments and instead sidestep them. Instead, you can just make the method public, which is jokey but underrated. If you're feeling a need to hit something directly in test, it's probably important. It might be important enough that you need to hit in your application code someday. Maybe from an admin interface or something. You can also use this as an opportunity to build more introspection logic. If it's hard to check on whether a process completed in test, then when you're debugging later on, it's also going to be hard to check on that then. Do your future self that favor. Finally, you can also extract a simple class. There's a quick and dirty way to do this. You use a shim. This is kind of like what was talked about earlier in the functional programming talk. Where you create a simple class. The difference is you're not different. The thing you're doing here is literally just copying and pasting the code from the old method into the new class. Refactoring downwards at your ledger. This is quick. This is cheap. There's nothing wrong with that. Maybe there are other private methods you need to pull in, which says that's actually super awesome because you're figuring out a new domain concept. This shim technique might get you someplace useful quickly. It might not. If it doesn't, maybe you've pulled on a thread and gotten a tangle and it's not productive to fix that tangle right now. If you can't get someplace with this technique quickly, it's okay to fix it later. It's okay to fix it later. It's more than okay. It's better because the thing you are learning when you go, oh, this is harder than I thought, is that you don't know enough yet. In the meantime, it's fine to practice harm reduction. It's fine to leave the code deliberately ugly in test and in your application code as well, because that's a signal of the future that there's something wrong here. Maybe leave a to do comment. That way, when you do know more, you can come back and fix it later. The next smell we're going to be covering this afternoon is duplicate tests. So for this one, imagine, if you will, that we're running a donation service that works with nonprofits. And our clients, the nonprofits we serve, each have a primary contact phone. We can look up whether the phones are mobile or not, and here's some tests that do that. We set the client's phone number to a predetermined phone that we know is mobile, and then check the result of that mobile phone method. Then we do the same for a known landline phone. I'm using many tests here in these code samples as least common denominator, but I want to reassure y'all that everything I'm saying in this talk applies to our client's phone number. The syntax of these two tests is different, but the semantics are the same. There's very little meaningful difference in how we write tests between deaf test mobile phones, and it can have a mobile phone do. Both are bound in test cases where we set a client phone number and then verify that a method returns the thing we expect. Anyway, so you've got these clients, you need to know whether they're using a mobile phone or a landline phone, and unsurprisingly, you also need to know this about your donors who also have phone numbers. So we test this the same way, and now we've got some test duplication. I don't know about you, but anytime I see code duplication in app code or in test, it makes my brain feel a little weird. It makes me go, there's something here, what do I do with it? So how do we deal with this? Now there's a wrong way that's very tempting at first, which is shared example groups. There's a special DSL for these in our spec. In many tests, we just share code between the two test classes like in our application code, namely looking at the module. So we can sling all this shared code into a module and then include that module in our test classes. While this is a valid testing technique in some circumstances, I am calling it out as a mistake here because it should almost never be your first code to when you see test duplication. When we look at the application code, we can see why. Our tests, as is very common, are basically just a one-to-one mirror of our application code, and that means that our tests are making the exact same assumptions that our application code is making. Over time, that will lock us into these assumptions. When we go to refactor the code later, we'll be changing the application code, but not the tests, or maybe we'll be frustrated by feature requests that conflict with test assumptions, and therefore require us to change a lot of test code as well as add new application code. We could also try reducing duplication by testing the module directly.  And again, this is a great technique when you actually want to have a module, but it ties you to the assumption that the code needs to go into a module, and that assumption might not hold true. If we turn the module into a class instead and test that class directly, we reduce duplication in much the same way that we reduced it when we extracted the shared example group, or tested the module directly. But in addition to reducing that duplication, we're also improving our application code at the same time. By listening to the awkward duplication in our test, we are able to see a cluster of shared functionality and improve our application code by properly encapsulating the single responsibility of phone number lookup. This is a much more sustainable solution in the way the shared example group doubled down on the module architecture and locked us into it long term. I model this example after a lot of times, I've seen API clients and started as modules, but it can apply any time you've got a module that really wants to grow up into a real class. The last test smell I'm going to cover this afternoon is inventing the universe. There's a particular kind of test where to paraphrase Carl Sagan, you are trying to bake an apple pie seriously from scratch. And so you need to invent the universe first. You need to plant some trees. You need to water them and pick the apples and harvest the flower and, and, and finally, you can test the pie came out tasty.  This hurts. Our job as developers is organizing complexity. And so we see, when we see complexity that is not organized, it sets up this itch at the back of our skulls. But boring tests make boring code. If we made any new test abstractions now, we'd be doomed the same mirroring nonsense that we just learned how to refactor away from. This is because we only have a little bit of domain knowledge so far. We know 12 lines of code about a domain. We understand one possible path for creating one kind of pie. And we know that one possible path is required for application. We do not know anything else. Anything we did to abstract this further would be total guesswork. We would be using our current understanding of an assumption about the problem domain to guess it is abstractions. You know we've also been using our current knowledge and our current assumptions to create abstractions? Yeah, that's right. Our application code. When you try to make new test abstractions in advance of test duplication, you risk making parallel abstractions to your application code. Just like we saw in the module with extraction example. Once we've got multiple test cases, we actually have enough information that we can start to tell where our abstractions are. Once we've got multiple test cases, we can actually start thinking about techniques we can use to encode those abstractions. Now, just like in the last section, I'm going to show you two ways that I've seen people make mistakes by trying to encapsulate abstractions within test-only code. Then I'm going to show you an application code-only approach that I recommend as an alternative. Note also that in this section things start to get a little less clear-cut. The test approaches I'm going to show you. While they can bite you if you use them inappropriately, they do have a lot of legitimate uses. The first test approach I'm going to tell you about is shared contexts. Our spec has a DSL for this again, but in many tests what you need to do is write a method and include it in your setup block. And what our spec does is metaphorically equivalent to this. Maybe you can even include the setup method in the module to share it between test classes. What you're doing with a shared context can also often be pretty benign. Methodically reinventing the universe from scratch every time you write a test is painful and oftentimes it is not a useful pain. For example, if your application has a database it is okay to extract database cleaning logic to a shared context or a helper file. There isn't much of anything you can or should do in your application code about something that's test specific. In production it is kind of unlikely that ordinary day-to-day procedures would require you to drop or otherwise clean a database.  So this is an easy case of telling something that's test specific from something that isn't, but sometimes the things are going to be a little less clear-cut. We stripped out all of the inessential lines of the test, all the things that don't matter in our production code, and we still have an extended test setup phase. It's a sign that an object has a lot of dependencies. It is always wise to examine whether these dependencies are necessary, but sometimes they are so the conversation shouldn't stop there. A pie without a filling is pretty sad. And so while I'm going to flag shared context here as a warning sign, I don't want to talk down to you all here by implying the domain complexities of your production apps are the product of unnecessary or easy to remove dependencies. My experience is that that is never the case. Luckily though the mitigations for necessary domain setup complexities and for unnecessary complexities you don't quite know how to remove yet are pretty similar. We'll get to them in a bit, but first I want to cover one more false trail. I should emphasize first, but this is not a rant against factory girl. I used it in many projects. It is a great DSL for describing test and seed data when used effectively. It just has a few traps. Factory girl enables and encourages us to make factories, dedicated test setup helpers such as line factory girl dot create pie makes a pie with a filling and the line factory girl dot create filling creates a filling with a pie. I could say that like shared context the problem here is hidden dependencies that we are hiding the dependency of the pies have on fillings. But in both cases if I stopped there we would go back to the magic and particularly arrogant L's definition of listening to your tests. The real problem is never that something hides dependencies. Dependencies means code I am not looking at right now. 95% of the time that is on purpose. I want to be able to look at my dependencies when they misbehave but the rest of the time I put them in those other files I didn't need to think about them right now and I want them to stay there and I want to stay not thinking about them. The problem with tests that hide dependencies comes in when the dependency that they're hiding is the domain logic. As a maintenance developer I want to know we make fillings in the course of making pies that's actually very important information for me to have. So I'd like to suggest that when your test stops get complex you start thinking about how to move the important things that your test setups tell you about your domain into your application. Now there are a few different ways to do this. You can pull more of the setup logic into the constructor for example. You can also make factory methods on class. Maybe these are shorthand for invoking the constructor with certain configuration. You can even go full Java and build out honest to God factory classes. There is nothing wrong with going full Java if your application needs it. We usually hide this logic in factory girlish girls developers but when we decide to dig it out instead what we are doing is saying that this piece of business logic is too important to delegate to a framework. It's like when we're startup we maybe want to get an agency to build a prototype but then later you want to say thanks we need to bring development in house. Getting core business logic under your own control rather than the control of your framework is important in itself even if your factories stay test only. If all factory objects do is make logic flow clear to developers then they are pulling their own weight. So let's look at some more foreign afters make this more concrete. This is how our invent the universe test looked before we started extracting some of its logic into factory methods. And here's where we can get to by using application code factories judiciously. Note just how much code we were able to move into the application where later developers can discover and understand it better. And because we've lost enough lines to actually set this on our screen in a way that isn't a scare tactic let's victory lap I'm going to bump the font size back up. So we're all done now right we've gotten through everything we've wanted to cover. Not quite yet. And incidentally we're not actually just doing this last bit because it let me make a DC flag but I could not resist this pretty moment of hometown pride here. So we've just performed magic trick by insisting that our tests be as boring as humanly possible. We have isolated the domain concept that we can use to also make our application code more readable. By insisting on boring test code we make our application code more boring. But now we come to the trick part of magic trick. This is totally subjective. This is a whole series of subjective value judgments. The problem lies in how we define boring. Valuing boring code over clever code is a really popular idea and since we can all nod along to that popular idea we sometimes miss that it's a value judgment. But if we don't define boring code very specifically all we are doing when we say clever code is bad code is creating another way to call code bad. In order to convince you that I'm on the side of boring code this will talk I've been deliberately sticking to uncontroversial territory. Everyone likes removing duplications but when I talk about what I consider boring it's actually pretty hard to stay on that safe ground. When I say that factory girl dot create pie is the literal opposite of boring is a single line that should strike fear into the hearts of any developer new to a code base and most developers old to it. What I'm reacting to is a history of seeing that one line imply thousands of lines of code that I suddenly need to care about but have no easy way of searching for. But the developer who wrote that line they don't think so maybe they've worked on the same code base for six or seven years maybe that developer has a good intuitive feel for the thousands of lines implicit domain knowledge embedded in that one line of factory girl they don't need pointers how to refresh them around specifics and so they might say the Java factory version I'm recommending as useless verbosity for the sake of verbosity and this brings us to one of the central questions that defines our day to day as software developers this right here is the hard part it's the hard part because the answer changes people come up here on these stages they give keynotes and they say here I promise you can solve all your problems with this new kind of you object I am giving you or maybe they say you can solve all your problems by throwing patterns away and embracing tightly coupled monoliths whatever the technical solutions change in these talks but the attitude that there is one true solution somehow never does I could stop by saying that this attitude turns us all into that jerk who asked well why don't you just whatever use this new kind of you object but that would be lying because what's underneath that self righteousness is the unchangeable fear of how hard this all is boring is subjective because boring depends on the background of a team and the life cycle stage that product is in boring is subjective and boring is hard to get right and we are afraid of not getting it right and we are afraid of what right is changing on us so that even if we get it right now it suddenly becomes wrong six months down the line but if code doesn't stay right doesn't that just mean our product is grown I think that's something to celebrate to me the magic of test room design of learning to hear what your tests are saying is that it gives us a way to figure out what the correct code is for right now it gives us a path through not just everyday simple decisions like should I make this method private but also harder decisions that people might disagree on more loudly to talk about these harder decisions I need to talk about some controversial things that's right I'm going to need to talk about mocks it gets worse I'm also going to need to talk about controller tests I am even are you ready going to need to talk about micro services now when I think about boring code this rails controller method here is basically my platonic ideal I want to update a blog post so I find the post in the database try to update it and render a response that reflects whether the updates succeeded or not there have been a few changes to method names and security improvements etc in the past three rails versions but the me of nearly a decade ago who was learning rails 2.3.5 would look at this method and not find anything weird to me that's boring as hell and yet controller tests are very hard and as a community we have a lot of opinions about them here's a pretty standard happy path test for this controller method even this simple controller method the happy path test we need to do some complicated stuff before the test even runs there's an invisible setup phase against database transactions we can clean the database quickly later we then create a post using active record which takes only one line but can potentially invoke a lot of other code then we indirectly invoke the controller method by using rack test to trigger the full action dispatch stack finally we test both response and also for side effects whether the post body has been updated or not this is a lot of stuff this a lot of stuff can take a lot of time to run and so on the rails community for a while there was a fashion for cutting out a lot of this time with test doubles a full discussion of test doubles is outside the scope of this talk and that's a really fun thing to say but briefly for those who aren't familiar mocks are completely fake objects so if we needed to post someplace but didn't want to go to the trouble of building one out using active record we could create a mock post that just returned a test for the post body we can also create partial mocks by taking existing objects and altering them so that a method always returns a stubbed value finally we can have either a full or a partial mock expect that a message will be received and this means that unless the method is called during the course of that test the test will fail now here's what controller tests written during that fad usually looked like looking at this you can kind of see why it fell out of style this test is much longer it's kind of ugly and it's super brittle if you change anything about the way that finders are being invoked in the controller all of a sudden everything will shatter into red it'll shatter into red even if the code still actually does the thing you want it to do and so a lot of fights about controller tests that I've seen are repeating people who look at this and ask but why would you even do that and people look at this and say okay this is ugly but our mocks are clearly telling us something about the design of the underlying application code if you're doing test-driven design you might instead write a test that looks like this begin with the interface you want to see in the world just one line of mocking so nice so simple so pretty and yet when we look at the application code that it implies how many of you actually do this in the real world how many of you are have luck convincing your co-workers to do this in the real world the code doesn't quite seem boring anymore it's not very racy and these are hand-wavy ways of talking about something that's very real we have made the project less discoverable we have made the project less accessible to newer developers and when I say newer developers please remember I am not talking about juniors I am talking about people newer to a project junior developers are entirely capable of comprehending these patterns non-standard characters have their place but discovery and accessibility are real values too and we condescend to people who don't like non-standard patterns by implying that if they don't like them they just need to learn to code better we are portraying a rhetorical trick we are erasing the very real and very valid values match that's going on by telling ourselves it's a skills mismatch instead and that since we are smarter we must be right fact is in the real world it is okay to ignore most of the buried complexity on a controller test a lot of projects have gone back to this style of controller testing and they are fine it's predictable it's easy to read we can ignore the complexity buried in this test because it's framework complexity that we can trust will stay buried slowness has become an issue sure but if slowness becomes an issue then maybe the answer isn't doing complex things in the test it's believing the test if a test isn't providing you with a lot of value then why should it be there it's just done here it's traveled up and then back down and then back up the realism continuum when you're dealing with test complexity one of the things you're always doing is trading off a feeling of realism against the ability to focus on exactly one thing at a time and when I say the feeling of realism that's very important because realism is not the goal here even if it feels like a nice security blanket when we exercise a lot of lines of code realism is a proxy for the ability to change the boundaries of our code arbitrarily in case we've gotten our assumptions wrong and so in this world a test with dynamically persisted objects that exercises everything is risky because it tests a lot of things accidentally that aren't the thing we're necessarily looking at but also lets us arbitrarily change the object structure around a test that uses test doubles extensively on the other hand is going to be working with a lot of assumptions that might be wrong but it's going to test a much smaller set of things and be more targeted what's interesting about this is that we be faced with pretty much the same set of concerns if instead of looking at the database directly we're retrieving posts and comments from some kind of blogging engine microservice what our code look like in this case how do we change this controller maybe it would look like this more to the point how would it change our tests here's our test again using this very realistic boundary ability to change preserving style and active record so if we're going to maintain the spirit of full realism we'd want to create things on the external service directly right full end-to-end test totally okay and we change the test itself for this be very simple very easy to write and yet no one does this and no one does this because in those cases everything surrounding the test is totally horrible connectivity issues are the thing that everyone brings up here but even worse than connectivity issues is the fact that all of a sudden we need to create all of this test set up and test tear down ourselves Rails does a lot of work for us when we're dealing with the database of the external service we're dealing with to handle that set up and tear down and it's very easy to lean on that invisible 10 years of work by open source people I don't know about you but when I move into microservice landia where there's a little less support code I don't want to suddenly take on the burden of that 10 years and become like personally responsible for writing a duplicate and so in this world mocks start to look very attractive to me you basically can't operate without them okay so but we're still in denial about that and so one thing we can do instead of going for a fully mocked test is to preserve some of the quote unquote realism by which we actually mean the flexibility to change boundaries within the code later that and full end to end test would give us by using the web mock library web mock allows you to set mock responses to achieve requests made to any given URL this greatly reduces test set up and test tear down it's relatively resilient against change but I personally find when I use this method for external service testing I find myself needing to care a lot about service details that aren't necessarily relevant to the test I'm writing and that itself is a test awkwardness that I feel the desire to listen to in that spirit let's go back to full mocking as a valid technique let's look at how this test this test right here that is a copy of the one we saw earlier as the fully mocked repository test how this test might evolve if we use mocks heavily in a microservice world there's barely any change the interesting thing about this is that if we'd followed the heavy mock style of testing all along we would have been prodded toward the repository pattern for our posts early on that would have helped guide our decision to extract a blog service when we needed to there would have been this big repository clash somewhere shouting extract me and it would have been so easy whether you like mocks or not I think that the objects you extract when you listen to complex controller test setups are a pretty fascinating guide to the actual overall shape of your application whether your team monolith or team microservice or team I don't care does the code work having that level of structure helps manage the necessary complexities that all older and larger code bases accumulate listening to your tests isn't the only way to arrive at that architecture but since these are hard problems I think we can all use as many tools as we can get regardless of what boring application code looks like to you or boring test code the thing I want you to take away from this talk is that whenever you are writing tests you are writing them about your current understanding of your code duplication is cheaper than the wrong abstraction applies to test code too when you do introduce test abstractions they are likely to mirror the assumptions you are making in your application code because humans are fascinatingly self-deceptive creatures application code influences test code and test code influences application code and so whatever boring does mean to you sometimes the easiest way to make our tests more boring to hate our tests less is to just make them more boring we do in fact have that power we can then trust that our application code will fall into line that's all that listening to your test means it means the places where you hate your tests are opportunities to refactor so that you hate both your tests and your application code less so I'm Betsy Habel I go by Betsy and Muffin on the Twitter's but if you follow me you will get a lot more cat photos and us politics than tech talk um and you can also find me on the internet at BetsyHable.com and GitHub.com slash Behable do you have time for questions yes we do have time for questions does anyone would everyone like to volunteer question hey so you show this example where you moved where you introduced the repository and then in the controller test you just stopped or walked this repository now then you need the test for the repository itself right and what would you do there would you use mox or would you just use the direct connection to the database and if if you use mox in that case then what is the real advantage you're just moving the mox from the controller test to the repository test you're introducing additional concept and then then you just split it into two files but what's the real advantage there well that's a straw man because I would just use active record methods there when you tie yourself to any one specific method and say this is the correct method of all time you lose the ability to make trade-offs and whether to use mox or not is incredibly situational if I were to use mox in a repository test or a query object test or any test where what I'm doing is testing how these very specific queries that I'm using active record to write play out in the context of the database then I would be tying myself very hard to my implementation and I never want to be doing that I would be losing the ability to maybe optimize the query by hand rolling SQL later I'd be losing the ability or to do that easily rather I would be not really testing anything I whenever you see a very long long sequence of mox that all deal with the same object the chances that your test is doing something that's worth doing gets very low and sometimes the feedback you need is turkey factor but sometimes the feedback you can take from that is that this test you need a different strategy for or perhaps don't need thank you for the talk I agreed with basically everything except for the last and one of the last slides where you do the mocking or I don't know stopping I still don't know the difference when you run the post test against the controller and you test if the service object receives post underscore update like if the method is called and I didn't understand why you do that why don't you just test why don't you just run the controller and then test if the like I mean you still have to find out the idea of the updated post but you why don't you just test if that really got updated with all the knowledge about the service object well it's very nice that you would mostly agree with me that's very validating and I needed that sorry are you being are you being sarcastic I'm totally being sarcastic but to answer the part of that that was actually a question I think what that demonstrates is the fundamental futility of a lot of controller tests I think that when a test basically tests whether a method is being called or not then oh no I can do that by looking at the code I can do that by running a more end-to-end test higher up the tech the testing pyramid there's a dozens of ways to test that controller tests have their place but controller tests have their place when the thing you're testing is more complex stuff like whether your authorizations are working correctly and in those cases you say you test if the service object gets run in the controller test and you test the service object itself in a separate test yes oh okay so but then okay but then basically I would basically ask the same question like how do you test the service object okay all good thank you cool anymore okay so just now look at the controller test you made I think the like very elegant solution and very short it actually removed lots of redundancy in the current controller testing but also a few the original version the more robust version it's kind of feel like an integration testing so I have a feeling that you're making controller test a unit test instead of an integration test so I'm just wondering what's your opinion on the integration testing which is kind of miss out I think that trying to divide clean lines between unit test and integration test is almost always mistake or not almost always I think that you want to have some higher level tests that exercise more of the stack and to follow the testing pyramid but I think that as you get lower and lower level you don't necessarily want very fine-grained unit test coverage of everything because some of your units mostly exist as coordinating units and again I think that you're not necessarily getting much value a lot of the time from testing that that coordination happens that isn't also being captured by tests that are higher in the stack and so I think that when you make a lot of to do about is this a proper unit test is this a proper mid-level integration test etc then what you're doing is forcing yourself into the current assumptions assumptions of your object structure again in a way that makes that object structure much harder to change later when it's no longer serving you and so I think that tests that are more integration even though a lot of people think they're less technically correct from like a test purism standard I think that the reason that they keep on emerging in real world practice is because underneath pragmatically people are drawn to the idea of something that they think is going to easy to change later and I think that's valid yeah yeah yeah I agree I think that makes a point that you're seeing integration testing is kind of unnecessary as if all your unit has passed it should just work and you shouldn't just you should not need to test the coordination between different units and also because you don't test the actual underlying integration it's easier to make changing your actual nitty gritty of your coding is that your take on the issue I'm sorry if you repeat that is like by removing the very detailed world boss integration testing but focusing on unit testing and assuming the unit's coordination is correct is actually making the changes easier in the future because without boiled down to the actual detail you can change stuff without affecting your test yeah I mean ultimately a lot if you have too many full integration tests or full end-to-end tests you're doing a lot of duplicate work all the time and you're not necessarily getting much marginal value from each individual test and even though you're not doing that as a human you're still making your ci run that and the level to which a ci that takes an hour to run versus level ci affect the lci that takes five minutes to run is very different the level effort or the level of attention that people pay when ci starts to take four hours to run is virtually nil and you need to keep that in mind but it's more than that it's that that past a certain point trying to multiply end-to-end tests is a fool's game there's a really great point I saw recently on Twitter that a game developer was making and this game developer had worked on a game where you had to do a lot of combinatorial simulation and so she was all okay we can't actually QA this game fully because QA in this game fully we have 500 options for this multiplied by 500 other options for this by 500 options by 500 options and all of a sudden you're getting to a point where forget a human tester not even a computer computer tester can meaningfully exercise all of that and that's fine what you do is you look for this you look for a few simple cases of smoke tests and you look for the esoteric cases that are likely to give you bugs and you pick and choose what's actually useful to run yeah yeah I agree thanks cool I think we have time for one more question if there is any any more questions no okay thank you very much Betsy