File name in newtwsied? I can have a demo so there moment at 15 minutes Welcome to the talk, and I am talking about transactions for theounge Today I am going to talk about micro services translation is appropriate for the running of micro services in dressing up environments Yn dwl 3 maen nhw ddim ychydig yn c pleadidio o gweithio tuning wedi gwneud ynnydd ser achi sydd o bod pan gofaws répers. Os yna, wrth gwrs ddigon nhw'n emrwy'n Sound Stewart yn un arworldgell iawn. ac mae'n gweithio'r ffordd yma yng Ngheilio APEI'r Ffwyrdd i'n gwybod i'r parw sydd i'r parw sydd yn ei wneud i'r newid yn ymwneud o'r cyhoedd yma o'r ffordd ar hirio ar LRA. Mae'n gweithio'r ffordd o'r llwyddoedd hynny. Mae'n sgrifau. Mae'n gweithio ar gyfer, ac rwy'n gweithio'n gweithio. Mae'n gweithio. Mae'n gweithio'n gweithio ar gyfer. A'r hyffordd yma yw'r hyffordd? Mae'n gweithio ar gyfer. ..eith yw'r dda'r cyffredin? No, mae hwn yn dweud, hefyd. OK. Yr eich gwasanaeth ar y cyffredin. OK, yn ystod? Efallai yna'n mynd i? Efallai, mae'n gweithio'n cyffredin. Ie, mae'n gweithio'n cyffredin. Fel gyfer. Efallai, efallai, efallai yna'n mynd i? Efallai. OK, dwi'n mynd i'n meddwl ymlaen, mae'r cyffredin a'r gyfer y system yn cael eu cyfle. Mae'r cyffredin a'r methu ynghyd, mae'n ddweud yn ymdegu yn buddwl. Felly, mae'n dod o'r hyn, mae'n defnyddio'r ddweud yn y ddweud yn hyn ymddeg, mae'n ffordd yn rhywbeth, y dyfodol, y dyfodol, bod yn fan ymwybod diddod. I yw'r gweithio rwy'n amser yn maes ei ddweud, ond yn y digitalr hon yn ymgyrch. Fy hoffwn ymddeg ar gyd, fyddwn yn fyddi, ymdeg yn ddweud. Fyddo'r gwylltion. I wnaeth ymdeg ar gyd, mae'n gynyddu mae'n gallu gweithio cael ei chael yn ymweld, i wneud ychydig yn gyllidol yr horddiad. Yn ystod, mae'n cael ei gweithio, rydyn ni'n gweld, ymweld, fe, mae'n ddigon y system cyllidol, a'n gweithio'r rhaid o'r gwybod, a'r gweithio'r ddweud ac yn dweud, a'n gweithio'r gweithio'r gwybod, a'n gweithio'r cyllidol yn ei wneud. Ymweld yma am ysbytiau yn ei wneud. Ac yn ymweld, mae'n gweithio'r cyllidol. Yn ystafell sydd y dystiolaeth, mae'n gweithio'r problemol yw'r ysgol yn ymgyrch yn ymdwyaf. Mae'r micro-surfadau sy'n dweud y dystiolaeth. Mae'r cydnoddau yma, yn ymddangos chi'n gwneud, mae'r cydnoddau yn cael ei fod yn siarad, yn cydnoddau'r cydnoddau. y phunwyr yn oed y fathodol rym ni'n gweithio mecanallwydau o'r ffaenonon eifryd o gael mwerth y llwyd gan y tîm yn rhyw fan. Mae'r cyflawnsion gwahanol, mae'n cyffredinolaeth i'w dweud o gael y ffaenonon i'r dda. Ond ydych chi'n gallu nhw'n cael ei wath dependr, yn ei wathwn i'n cynhyrch a chanol, y syl passwords, mae'n anodd wneud o ceisio i un pwysigol ar gyfer y chaillol sy'n mynd i'r gorffwnol o'r maen nhw'n wneud. You've got tim Aus. You can do load balancing. If you detected that one of your loads is running more slowly in scale up and down, depending on if you think that things are becoming fullacting or you've got latency in your system. So we think that transactions is really part of this ethos and that should be able to be part of your tool case. It's not the solution, but it should be one of the solutions. You can apply it to building your systems. Ac y cwestiynau eěn meddwl, y wybodau y diwrnod i ddim yn gynnwys, ac mae'n meddwl ychydig yw'n bwysig hyn? Mae'n meddwl. Felly, mae'n meddwl mewn gwahanol i gyd,. Cefyddiadu gynnwys yma, o'r ganddo ar feddwl y shawdd y dyma, ai'n i'n mynd i'r gwahanol i gyd. A'n meddwl, y dyma eich ganddo â'r ganddo y dyma, nad ynddech chi fod os y gallai dechzefnu.ressing this guarantees. So Ifeles weld it provides like an all-or nohting compliqué so you like if you move the system and so thyn Gridio gwaslawr at the end of the transaction you want to be able to revert things. So there's only 1 model of transactions there are many different ones so you've probably heard of the GTA model which is like built based on an old ex open of A'r model JTA yma, mae'r sleif yn cael ei amser yn ar gyflogodd. Mae'r model yn cael ei amser, mae'r model yn cael Saga'u, mae'r bwrdd amser cyntaf amser. Mae'r busnes ac yn ei gweithio ar y dyfodol, ac mae'r rhai gwneud o'r bwysig o'r protocol. A bod eich bod yn dweud, ydych chi'n oes y transasktion model? Le'r modd Llywodraeth wedi bod oedd y modd fynlluniaeth neu'r modd Llywodraeth, fel'r modd Llywodraeth yn ymddorol gan мол Apparaturi. Oherwydd mae'r llyfr yn ymddors fawr ac'r modd Llywodraeth wedi'i amser gan y gynhyrchu Llywodraeth. Byddhaf i'r newidau ddechrau yr un iddyntau ar y modd Llywodraeth, rydyn ni'n enw i'r ffynlluniaeth gwahanol o'r modd Llywodraeth a'r rhaid i'r moddd Llywodraeth. I'r modd Llywodraeth yn ymddors fawr. Bydd y cyfle其实a daeth eich sefydliadau a fyddwch chi'n gofio'r cyffredin iawn. Mae hyn yn mynd i bod yn bobl cynnig iawn maen nhw'n aethom. Felly, yna, y gallwn i'w gweithio'r cyffredin iawn, neu'r cyffredin iawn, y gallwn i'w cwchio'r cyffredin iawn. Felly, dyna'n gweld hynny sy'n gwelch chi'n hoffio'r cyffredin iawn, felly dyna'n gweithio'm yn cael ei bwlad o'r model ei fawr.. Felly, mae'r model eich bod ynwn i'r ddweud allan o ddweud o'r ffordd mewn ddweud o'r ffordd i'r ddweud o'r ffordd o'r ddweud o'r ffordd, ynnyddio'r ddweud o'r model jta. Felly, i'r ddweud hynny, roedd hynny'n gwneud o'r ddweud o'r ddweud. Felly, atamysgwyd o'r ddweud o'r rhan o'r oedd o'r prifsfyddiad, mae'n rhan o'r prifsfyddiad o'r rhan o'r sengl uned yw rhaid. Nodd hynny, when you finished on a set of operations, you want all the operations to complete or none of them to complete. So that would be full atomicity. Second property is consistency. So that's an application developer's point of view. So he's building his application and his business application provides certain guarantees and certain invariance. Before you start running the transaction there's an invariance that you can rely on and at the end of the transaction is moved to the states moves to another invariant of the system. And that's saying that the system is consistent from the application's point of view. So in between the start and end of that transaction then you can have inconsistent states. So that's why we have to introduce a third protocol, isolation. So you want to be able to isolate that inconsistency from other transactions running the system. So two transactions cannot see what each of them are doing in between the transaction. i gychwyn eu cyfrifiadau sy'n gallu ei chygingrifio i'r cyfrifiadau gyda gwpaeth. I fawr, yn yataeth, lle wedi'i hen i gwybod y cyfrifiadau cyfrifiadau yna yn ymwneud eich cyfrifiadau gyfrifiadau y gydag ar eich cyfrifiadau. Mae yna dwych i'r cyfrifiadau, 어� i gydig i'r cyfrifiadau, ond mae'r ei wneud am y cyfrifiadau i'r cyfrifiadau, a bwg yn gweithio ac dyna fydd y pwynt sy'n oes chi'n gweld â'r cychwyn. Felly mae hynny sy'n mawr ysgwrs cychwyn swyddfaen, yn cymdeithas y fawr i bobl ei fawr yn dweud. Felly faint o'r hyn yn oed, mae fan hynny'n meddwl eich bod adeiladol yn chi gweithio a w abandon i bwg yma? Maebell a'r pwynt hanes o golygu ar gyfer rhai ffARS, mae'r cychwyn cychwyn cychwyn cofwilion wrth gwrdd y log yn cyd-deithas hynny rhywfydd. mae'r dŵr ymlaen i fyng o'r rhaid oherwydd mae'n bai yn ychydig o'r pethau sydd yna'n gyfrins. Felly mae'r gwrthodd neu'r ddweud o'r ddafodd, ddim ddweud o'r ddweud o'r ddweud o'r ddweud a phobl iawn i'r ddweud, ac yn ddweud o'r ddweud o'r ddweud o'r eu ddweud o'r ddweud. Felly mae'r ddweud o'r ddweud o'r ddweud o'r ddweud o'r ddweud o'r ddweud. Felly, mae'n ffwrdd dros agnod o dollars i chi gynnwys a'w'r pethau sydd yn fawr ac i chi gynnwys pan ddoch. Felly, mae'n gyrfaen â'r f hyn mae'n cyflymau fawr i gyd yn dwylo'n amser dyma sydd yn rwyf yn gweld bwrdd syd. Felly dwi'n gyrfaen â'r ffordd teiemol iawn. Felly, eich ffordd ffyrdd ymweld yn dweud o'r peth sydd yn dweud â gwmwys gyda chi'n y gwirchai o'r mae'r wlog gofyn. felly mae'r ysgolwyr mae'r ysgolwyr wedi'i gweithio'r ysgolwyr, dwi'n rhoi'r ffordd o'r ffordd o'r ffordd i'r Lymbr. A wnaeth i'r ffordd o'r cyfnodd, o'r gwneud o'r ffordd o'r acedau acedau, mae'n rhaid i'r ffordd i'r gweithio'r data ac yn rhai o'r log. Felly mae'n cael ei fod yn ei wneud. Felly y ffordd o'r gweithio'r ffordd i'r wneud o'r log o'r ddweud o'r ddweud o'r gweithio'r data. Felly, mae'r ffordd i'r gweithio'r ddweud o'r ffordd i gweithio i ddeudio hynny,worda, mae'n d possessedio o explanation, i'n gweld sesi's Pressing to have interior of the ar gyfer y partid yn ymdod. So that's... So what about using transactions for microservices. So microservice interactions typically what you do is you start with a monolith. You've got a monolith that's running inside an application server and you want to break that monolith up into different services a goes on run those in separate machines in the network in the cloud. For a start, you've got many different parties involved, so that increases the complexity. But once the system's actually broken up, when you're talking to Microsoft's environment, them will be a strong propensity toつ and do use plug in other people's APIs, like them so you might like, you have to only concentrate on your core strengths, ond wrth gwrs y tro rydych chi ar gyfer y sylwb i gyd沒關係 ac yn gyf feelingwyr yn fwy o'i oed i'ch gynghyd i'r lleidio a'r lleidio'r lleidio. Mae hyn yn gobeithio, a'r hynny'n gobeithio, oherwydd roedd gan ffacolig sydd y migrosirfysg ac yn cyfnodd y migrosirfysg sy'n byw bepysig, dyna i siarad yn cael ei gydag i'r gwell llunodol, ond holl o'r tanig iddyn nhw. O cael ei wneud yma yw i ddweud o ddych chi i'r gwas iawn i'r ond, you need something to be able to co-ordinate all these different activities. Otherwise, the system's going to be like that because you're going to have to have a lot of requirements and a lot of responsibility on other services because of his trust balance that you don't really have any influence over. So you need some kind of thing to co-ordinate all of that. So the question really boils down to can we use full asset transactions to achieve that correctness. Felly, mae'n debyg arall oherwydd mae'n byw gyda'r syniadau dystrybu'r system yw'r 60s a 70s, ond mae efallai i'r wneud o'r modd yn cymrydau yn cael peth. The problem with full acid is clearly that two phase commit protocol is a blocking protocol. Also to achieve the isolation typically you have to provide locks on data although typically you like your locks on data zone if you use optimistic locking during the transaction. Then at the final two phase commit phase part of the protocol that's when you do the locking. pan oedd eich cymdeithas yn y pwg yn ei ddweud. Yn dweud o ddweud, yw'r cyffredinol sydd wedi ariedd o ddweud y cyfan llwyddi, ond mae'r cyfeirio'r cyfeirio'r cyfan llwyddi yn y pethau yw'r cyffredinol. Mae'r cyfrannu cyfnodol yn ariadur. Mae'r cyfrannu cynnwys i ddweud sydd wedi cael y set ymgyrch, mae'r cyfrannu cyfrannu, If you've read one bit of data during this transaction and you read it again, it's guaranteed to be the same value. You've got to be committed and read uncommitted data etc. So the highest level is serialisable. So that's serialisable, that would make effective meaning that if you run two transactions it would effectively look as though they've been run one after the other consecutively. And that's the one that impacts availability the greatest. There's also a piece of light, a theorem from an academic literature called the CAP theorem. So the CAP theorem concerns about consistency availability in the presence of partitions. So if two services in the environment if they lose connectivity because of a broken network or because the machine has gone down or because the service itself is running solely, then that's what's called those two services become partitioned. So if two services become partitioned then the CAP theorem states that you can't guarantee full consistency and full availability. So in this kind of system you have to be able to relax one of those if you want to be able to make progress. And in a microservice environment availability is key because that's why you're going for the microservice to get the scale and to get the responsiveness of your systems. So really it's like down to like looking at how you can like what you can do with respect to consistency. So full assid is clearly it's not an optimal solution in these kind of loosely coupled systems where you've got long duration activities. So if you can't use full assid what the option is is to start looking at those set of four properties that characterize all transaction models and not thinking about which ones are those you can start relaxing or even do without in some cases. So as I go through the four assid properties to make it concrete I'll refer to a simple case of where you might be trying to book a seat on a plane and some travel insurance and you want to do those in the side of transaction. A conventional two phases in asset full asset transaction you would have to book those two things as a one atomic operation. So either the both booked or neither would be booked. So if you want to relax acid a bit atomicity after that all or nothing guarantees then you might want to need transaction. You might want to like cancel some work while allowing other work to continue. So in the example we've got the travel insurance example you might go away you might book the flight and then you don't want to like unbook the flight if you can't get your travel insurance because you might better get your travel insurance some later time down the line. So that's a case where you want to break out atomicity. So the more conventional approach to handling that kind of break of atomicity is with nested transactions. So you don't get nested transaction with JTA but you do with a lot of other models of say like the object transaction service from CORE for example they had nested transactions. So that's where you can do some of the work inside the nested transaction and then if you decide you don't want to do that nested work you can cancel it and it won't affect the top level transaction which will continue running. So if you book your flight and the top level transaction that's still there and then you just like cancel the work you've done with the insurance. The second possible property to relax is about consistency. So you might have heard in terms of when people talk about transactions for the cloud and for microservices environments they talk about the idea of eventual consistency. So as long as the system eventually comes to some kind of agreed consistent state then that's often satisfactory for a lot of applications. But eventual consistency is a very weak guarantee. So when the system and the data will converge on a consistent value that's indeterminate. It can happen at all times in the future. Also once it's converged it can quickly start diverging again. But that can be mitigated with application knowledge so typically you would combine it with timeouts. So if you're waiting for some state that you want to be consistent and it hasn't happened within a certain period of time then you can time out that work and go and try a different approach. Try a different strategy for 15 minutes or abort it completely. So the next one is isolation. So when you start. So as we said earlier I think into isolation you have to be able to relax isolation because isolation requires a lot of light work on behalf of the application developer to isolate work from the other applications. So you might want to commit work early. I'll skip through that one. And also the durability aspect. So typically you want your work to be good even to be durable and to pierce after you've completed your transaction. But STM software transactional memory that is a good example of where you would like to relax durability. So in a software transactional memory system that has like the acid properties that has like atomicity consistency in isolation. But it doesn't have the durability and typically you'd use that for like high scale concurrent systems. Object based systems. So you've got your objects and you're making changes to your objects with many concurrent threads operating on these objects updating state. And then when you come to commit at the end of your transaction you find that there's been a problem. Then you can just abort it and start again. So in that case you wouldn't necessarily want to do durability. So how do we go up route choosing which set of properties to use? So we approach the micro profile community. Disgusting with them came with some of the odd use cases. So we knew that scalability was the main thing to look at. So full acid wasn't proposed. We needed to remove locking so we need to like something that's where different business activities can see what each other are doing. So you need to drop isolation. Basically you want to make sure you get as much work done and much forward progress as possible before backing out. We knew we wanted nested transactions. We knew we wanted to be a time bound. Time bound the operations. Time bound the period of which the undue compensation activities can be guaranteed to run. And also composition of these transactions called long run actions. And also to be able to compose them and run the compensations within long run actions as well. So the result was a draft specification. So we submit that to micro profile community. We have a weekly meeting where we go through all the different stakeholders, go through the different issues and different problems with the spec. So it's still open. It's still available for people to make contributions to it. Micro profile is the CDI first environment. So we define a set of CDI annotations by which you can start these long run actions and by which you can register callbacks for the compensation completion activities. There's also a pure Java API for people that don't use CDI annotations. And also we've got a group that have their own implementation of a saga-like model that uses GRPC and Google prototype buffers. So we've separated out the transport aspects from the CDI annotations that define the model. So this is a sequence diagram of our Narayan implementation of it. So the actual specification is defined in terms of like a set of CDI annotations, as I said, and the map of the transport. So this is how we implemented it. So I'm going to skip over that one and go straight to the demos. So in the demo we used various pieces of technology. So we've used the OpenShift platform, OpenShift, to build on top of Kubernetes. It fried some extra management functions on top of Kubernetes and also the build and CDI deployment pipeline. We've also used the Narayana. Narayana is the Transaction Manager Toolkit, and we use that to build the prototype implementation of this specification. And also the Wildfly Swarm, which is now called ThornTail, but they haven't got a logo for it yet. So ThornTail is like a way to get a cut-down version of Wildfly, the Wildfly application server. So it's cut-down, it has just what you need inside.  That you can run in a JVM. So this is what the demo is. So the demo is booking a hotel and booking a flight. So the initial state is to book a hotel. To book the hotel, we run that in a top-level route lowering the action. And then the flight bookings, we want to have two strategies you want to book. So we've come to book the flight and we can't find an economy one. So okay, we'll have a first-class flight, but we might want to later go back and cancel that and go back to an economy class. So we've run that in a nested, in a nested lowering action. So the idea there is that you can go and cancel the first-class flight and then have another go at booking an economy flight. So you're not going to lose your hotel booking. And then after we've done that, we're going to cancel one of the flights and then close the LRA. And the thing that's managing these LRAs will then make sure it calls into all the different services that are registered with the LRA. So for example, if you cancel the whole operation, then the transaction manager will ensure that it goes back and calls all the different callbacks to tell them to compensate for the work they did. So the hotel booking could be to refund the customer. So there's a deep dive there. If you do want to add these slides online, if you want to have a look at what we've done. So I'll just have a look at running the demo. How long we've got now? Nine minutes. Nine minutes. Okay, so quickly. So yeah, what I've got there, this is the set of annotations. So for example, to start an LRA, you would annotate one of your methods with an at LRA annotation. So there you've got things like required, requires new. If you're familiar with JTA, it's a model on JTA. So with those annotations, so for example, if you put a mandatory LRA annotation on a method and there's no LRA context available when that method is called, then it'll throw an exception, for example. Okay, so let's have a look. So we've got... Yeah, I'm running a balance show on time. So I think it's probably best if I just run the demo. Right, but first I'll say that so the three services running is the trip service. So the trip service is one that does the booking, that coordinates the booking of the hotel and also the flight services. So then there's one for the hotel service and there's another one for the flight service as well. So if we start, the service is running. So this one is the hotel service. This one's the flight service running. So this is what I mean by the flat jar.  It's like a cut-down version of a wildfire flight. And that's got the bundle inside it. It's the various services that we're running. We also need a coordinator. So when these microservices register compensation activities with the LRA, that they might want to be compensated for if the transaction is subsequently rolled back, closed, cancelled, sorry, it needs to register this information with the coordinator. So we've got two coordinators running in the same environment. That's the bottom right one, that's the one for the sub-transactions and then we've got another coordinator. So the coordinators can be federated in the system. They don't all have to be running in the same JVM because they can talk to each other. And then this is the main coordinator. So when you do a trip booking, the trip booking is going to go to the main coordinator and register with that one and then the flight service that does it in a nested LRA. That one will go and register with the subordinates, with the subordinate coordinator. OK, so that's the services all running. So what we've got now is... So this shell script, so this shell script is going to... is going to go and make a call to the trip service microservice and it's going to ask it to go and book a hotel and book the flight. So if we run that. OK, so that's gone and done the booking and that URL I've printed out there, that's the URL that you have to give to tell the trip microservices to complete the booking and what the trip microservice will then do is it'll then go and confirm the long-running action. So if I type a copy of that, run that one. So that's gone then and ran it and that's gone and closed the LRA and then the LRA has then closed all the completion callbacks for the various microservices that were registered with this long-running action. And you can see the aggregate booking, they're all in the status confirmed. So you can also do one with Cancel as well. You can do the first class too. Yes, you've got to go to get economy if it's available. And Simly, if you decide you want to abort the whole trip then you'd ask the trip controller to cancel the, cancel the whole trip and that would then just close the LRA and then the LRA would do all the right callbacks. So the trip manager, the trip manager, microservice doesn't have to cope with all the complexities of cancelling everything and shutting everything down because it's already registered all the logic with the coordinator. Effectively, in this environment, it's a REST-based system which is in JAXRS so they're just all the endpoints that it's registered. There is a second part of the demo which is running on Minishift. So Minishift is an open-shift environment that you can run locally. Let me make sure. Let me start the console. So I have already deployed these three microservices and also the coordinators into Minishift and this is showing them all running. So there's the flight coordinator, there's the hotel coordinator, LRA coordinator. So now we can interact with the open-shift environment with these services running in that environment. So I've pre-canned some of the commands. So this is going to the trip microservice so that's the endpoint in which it's lifting it, listening. So if I run that one, so with Kubernetes you have service-based URLs which then you have the actual pods that implement the services running behind that service so that's what that Minishift IP, et cetera, is doing. So that's created a booking ID. So the booking IDs I'm using are actually the IDs for the longer-in-the-actions so that's like you could have a mapping between the longer-in-the-actions, the booking IDs and the longer-in-the-actions, but for simplicity, booking IDs are the same as these longer-in-the-action IDs. So that's the booking ID that we've got back from starting the booking and then to finish the booking you would put a request to the trip service and then that will go and close the long-running-action that's running in the coordinator that I've also deployed to Minishift and that's showing them all confirmed. You can... two minutes. Another example would be to show that you can balance the coordinator so you can start that running. So what I've done there is I've started a booking going. The long-running-action is in progress and then what I can do is go to the coordinator. So he's a coordinator. So with OpenShift I can scale down. So if I scale that one down so when that scales down to zero that means there's no coordinator running environment so this is just going to show that there's some resilience in the system so the whole point of transactions and models is you have reliable guarantees so when things crash and things fail you can always come back up and continue where you left off so that's scaled down. So if I scale that back up again then once it's back up again I should still be able to complete that trip booking. So... So I'll balance the coordinator and then I can ask the trip microservice to complete the booking and there it has completed the booking even so it demonstrates that the coordinator does maintain state and it does remember how that is and which of the long running actions it's responsible for. OK. So it's good because that's where we are. We have a break between now and there's a break now between now and the next session I think it's about 25 minutes so if you want to ask him some questions he'll outline out of the session and we'll show you what this session is about. And if you take a look at the slides there's various links of where you can find out more information about what we're doing and a project which implements the transaction manager and presentations there as well and also the information about the microphone for our weekly hangouts we have. So that's the place you go if you want to contribute any input about what you might want to have changed. So for example like I said before we had some guys from China they're working on a SARG project and they did contribute changes to make sure that we can support things other than just rest-based systems. There's no time for questions. Right, offline questions then. There are enough time. There's no, is there? It just goes by like this. I don't feel bad. 35 minutes is quite a short time. There's normally 50 minutes.


certain reasons to care about but more also how we could go about caring about code quality. This time to ask a question or make a comment is when you have it, so please don't wait till the end. Anytime you have a question just raise your hands or just attract my attention and I'll more than be happy to yield to you and listen to you. So the first question is why why care about code quality? What's a big deal about it? Now certainly we want to be agile right? So what does agile mean? Agile means we're gonna get feedback and based on the feedback we're gonna go back and respond to those changes and then go to the customer and say here are the things we did last time you wanted us to make these changes so here you go and take a look at it review it let's move on forward but as you are sitting there with the customers and asking them what kind of change you need to make you begin to realize that the particular change they want you to make if you were going to make it you would go you would have to touch that piece of code remember the one that I'm talking about right that piece of code that you touched the last time and you could not go home that weekend you're gonna quietly tell the customer you know what that's not a good change after all because you don't want to spend yet another weekend at work so in other words one of the main reasons to be to really care about code quality is as simple as that you cannot be agile if your code sucks and so it's important for us to care about code quality now this doesn't mean just because you have a good quality code you're agile I'm not I'm not saying that right but it's really hard to be agile if your code sucks you got to have other things but this is important thing as well but really focus on code quality that's the reason but as Albus and Sussman talk about this in their book on software in the SICP book programs we return for people to read and only incidentally for machines to execute so we all write code and we focus so much on writing code but more than writing code we have to make sure that the code we write is actually readable now why is it important for us to be able to read somebody's code in our team and the reason is we write code just once but we have to maintain it continuously and if you're not going to maintain the code what's the point in writing that and and any code that you write has to be maintained if somebody tells you that they wrote a code once and they never changed the design or the code what they're telling you is that the project got cancelled right so any code that's relevant to the marketplace has to evolve and change so that's very important so it's important for us to be able to read the code and understand it now of course right of the bat I will ask you how do you write readable code now I've worked with several people somebody would tell you my code is readable they say how do you know your code is readable and they would say my code is readable because I wrote it that doesn't count right because that person who wrote it may be able to read it but that person may be the only one who is able to read it you probably have worked in companies where the boss tells you whatever you do don't touch that piece of code because the guy who wrote it no longer works here and nobody understands it right and it's really hard when you have code like that or you work with people who write code they can understand it but when you come back from vacation they cannot understand it of course that's easy to fix right no vacations for you but what if this person dies God forbid right our quits then somebody else has to maintain it still that becomes really hard so what do we do about it so in my opinion there's only one way I know off to make code readable it is to read it so if you write a piece of code give it to somebody else and let them kick and scream and then they will give you the input and then eventually the code becomes readable so it's important for us to read the code but that again and the reason simply is that a software project as we create it has to evolve and the evolution comes from changing requirements so depending on the size of the project as the bigger the project is the more the opportunity for the requirement change and we're going to change the requirement that eventually is going to come and impact and change the code so if you know it when I used to work in software years ago on traditional projects I would hear words like this we froze the requirement another word for being stupid right there's no way you can you know freeze requirements it's you know a lot of things will keep changing we could that's not called freezing the requirement that the right term is we're in denial of change in requirements that's what the real term is but they don't want to tell you that way right so you cannot freeze the requirements it's going to change whether you want to change and accept it or not is the question and we do have to change the requirements you have to come and change the code if the code is incredibly hard to make change then you're not able to respond to those change and it becomes really hard so we have to be able to respond to this in other situation the cost of finding and fixing defects increases as time goes on if you were to write a piece of code and let's say within about two minutes or less immediately you know that the code is poor and it's it's messed with something you can fix it right away what if you get to know the next day yeah you can still fix it what if you get to know about three weeks from now you have to first stop what you're doing and then go back and do this fix that takes away time from what you're working on right second what how much of this do you remember in three weeks after three weeks if I was writing the code I would even question if it's my project that's how short term my memory is right so you got a context which and and pick up what you left on and if it's a year after you wrote the code it's even more difficult and sometimes you get these request after a couple of years and you have to gain huge amount of context which is more expensive not only that when you get that far it's not one person who is involved it's several people who are involved in fixing the bug or the error or are changing so it's important for us to be doing this very quickly for us to get a good effective you know to be economical if you will so there is a way to think about you know how do you you know detect defect and how do you reduce defect we're not talking about fixing defects how do you reduce it there's a top 10 list by Bohm he's talks about this and the first thing he says is finding and fixing problem in production code is hundred times more expensive than during requirements and design phase so you want to definitely do it as soon as possible and and then he goes on to say that forty to fifty percent of error on projects is avoidable rework you know where we can go back and keep changing things if we can avoid some of these things talk about 80% of available rework comes from 20% defects and of course 80% of defects come from 20% modules but what's really interesting about this is that 90% of downtime on projects come from 10% of defects so do we want the software to be completely defect-free I think that's a myth it's almost impossible to create a defect-free software but there are only two things I aspire to do one is I don't want to reintroduce defects you found a defect you fix it I don't want it to creep back in right I want to just quash it forever and I don't I want a good regression so I don't have to re keep refixing the defects the second thing is you you want to really look at the ones that have the major impact there's no point in sweating on small stuff so it's important to focus on real things that we can fix going further they talk about peer review catch is 60% of defects now don't quote me wrong don't tweet this wrong but given all context if you ask me given a choice between writing unit test and doing code reviews given choice between these two I strongly prefer code reviews I've done this on projects and I cannot believe anything that gives enormously better results than code reviews now what is the reason to do code reviews code review gives an opportunity for other people to look at your code that means they are reading your code and I mentioned earlier the only easy way to create readable code is to read it so when somebody reviews your code they're doing code review already if they cannot understand it they're going to give you the feedback they also look for various conditions another pair of eyes another brain looking through it and I have very strong belief in code review but having said that it's extremely important who is doing the code review now if in your organization lowly programmers write code and then the so-called architects who don't code anymore review code it is a dangerous situation to be in what you want is the person reviewing the code should be a fellow programmer who writes code along with you and in other words don't go to the so-called priesthood based code review these are people who sit there and look at the wall all day and then they say they can review the code for you right and they have not written any production code in a long time these guys are dangerous if you're working in a company where the review only happens by these priesthood you know architect my recommendation is do not walk away from those companies run there's no point working in those organizations right you want real guys on the project reviewing the code and giving you the feedback perspective based reviews are really critical because they catch 35% more defect than non-director reviews that's why you want these people to be active reviewers on code the worst kind of reviews you can get is where the curly bracket should be we all know where the curly bracket should be right you don't need somebody to review that for you what you want is they we want them to review to and say you know what kind of you know things you have forgotten what kind of algorithms you could use and so on and again quality of code is it readable that's very important and as people are reviewing this code I get the opportunity to work with some really brilliant people from time to time depending on the project and a lot of times I would review the code I would say this code is really great but my review back to them would be hey you forgot to write these few test cases so not only do we review code we also review the test cases that go with it as well this also ensures that people actually are writing test cases for Lee well and discipline personal practices can reduce defect introduction rate by 75% let's note that down we're not saying discipline reduces defect no we are saying deep discipline reduces defect introduction right that's even better than reducing defects so if you have discipline and if you practice this is hard work we can start reducing defect introduction quite a bit and it costs 50% more per source instruction to develop high dependable software product let's not kid ourselves it costs money it costs time it costs effort if somebody says we don't want to spend money but we want to get results quickly that's called gambling right they need to go buy lotto's not develop software it takes time it takes money it takes effort I was I was at a client site a couple of years ago and and they said hey it's all great you come and talk about quality to us this is fantastic but how do we know that any of these is going to change anything in our company and my first response to them is it's your problem not mine right and then eventually I said yes you are right if you want these to change I'm going to leave right you guys are the ones who are going to be sitting there and doing stuff and and if you want to make change then you need to have somebody who takes responsibility and spends the time and effort doing that so in other words my recommendation to them was for every software project they wanted to improve quality on I asked them to go find one person a technical person who would be a champion on the project not a project manager you see but a technical guy not a team leader not certainly an architect there is no other term I fear more than architect right so not at somebody who's technical but who is eager passionate interested dedicated discipline and say you are a champion in this project and your job is to come up with initiatives to improve the software quality tell me what you're going to do in a month tell me what you're going to do in three months in six months come up with a plan of things you're going to do and evolve this plan along the way but have a champion and this worked really well for them because they did find these champions in the company and they work really hard to improve the quality of their product but unless somebody is going to be their responsible and spends time and effort don't expect quality to improve all of a sudden magically there's only works when people devote time and effort to do this 40 to 50% of user programs have non-trivial defects so these are the top 10 list that they talk about but but you all know this right we all know this we have read this in books we have read this everywhere we have heard this but then what happens when you go back to work and it's so funny when you see this happen right the evidence is overwhelming but still companies don't want to do this what what do we see there we never seem to have time to do it but we always have time to redo it you tell hey this is going to take time oh no we got to ship it no time for it then we create a massive mess now the company is willing to spend all the time and money required to fix this nonsense right so they never have time to do it but they always have time to redo it so this is something we have to break the cycle this is the industry-wide hypocrisy Einstein said the definition of insanity is to do the same thing over and over and over and suddenly expect a different result and what do we have we have projects that fail continuously capers don't talk about software projects failure rate can somebody guess throw a number don't worry about being wrong how many percent of software projects actually succeed 20% how and his number was more like 10% 10% of projects succeed of course his definition of success was he's delivered fairly on time within budget works like you wanted it to work and customers are happy to use and I've heard people say oh my god that's too restrictive right now think about this 10% of projects actually are successful here you're going to buy a house and the builders says come on over we're ready to sell a house for you you say tell me what you have done and they show you around and there are 90 houses fallen down crashed down 10 of them barely standing and they say you could be one of those lucky ones to have those houses standing build with us all the flight attendants is buckled up your seatbelt we got 90 flights missing today but don't worry we're gonna get you there safely right and yet we develop software projects and what do we do differently we have hope and we say this time it's going to be different right I can only think of two professions where we have such a low low success rate only two professions where we keep doing this one is programming on the other is politics we know these guys are no good yet we elect them and we put our faith over and over you know what there's not a whole lot I can do about our interest with politics but I hope people in this room can do something about programming right so let's at least make change where we can so there's definitely a case for doing something better but one of the things we need to think about is technical debt so what is a technical debt now technical debt simply is the things that we accrue along the way a little shortcut we took is it really wrong to accrue technical debt absolutely not it is actually a pragmatic thing to accrue technical debt you if you don't accrue technical debt you are becoming dogmatic you're saying I will make everything perfect but you see the problem is this what is perfect I don't know what perfect is because I may spend 10 days doing something to perfect it and you come along and say you idiot why did you waste time doing this after we don't even need this in the current release so you want to get something fully quickly done but not a quick fix where you ignore what you know but get something up and say you know based on what we know right now we're going to build this there is these things we don't know quite so we don't want to invest time on it right now but that may incur a certain debt but that's okay we are ready to fix it but okay we're going to acquire a bit of a technical debt but the first thing I would do is don't avoid technical debt but have a good sense of what your debt is and once you know what your debt is you say we're going to fix these debt along the way so here's what I would ask you to think about you are scheduling your iterations or sprint whatever we want to call it I'll use the terms interchangeably here sprint and iteration you're scheduling your iteration and what are you saying you're saying I'm going to spend this much amount of time implementing the features but when you're doing your sprint planning or iteration planning you need to also say explicitly here is a certain amount of time we're going to dedicate to fix our technical debt you need to schedule time to fix technical debts if you don't schedule time to fix technical debts where do you spend the time on doing it and if you don't you keep accruing technical debt now what happens if you accrue technical debt the technical debt is like financial debt right I go around and charge my credit card and every month I charge my credit card at the end of the month I get a bill it says bank it you you you are supposed to pay this and I say you know what why do we want to pay the stupid credit card bill when I have the money and I can buy more things today because I have ego to go buy stuff so I go buy stuff and I send the minimum balance to the credit card company and say here have it and the next month I have more bill I send the minimum amount still and after a few months my balance has exceeded to a great level but what happens now the credit card company sends me a letter and they tell me that I've exceeded my credit limit but you see I'm a valued customer so they increase my credit limit because the credit card company really cares about me right and then what happens I continue to charge and one day I realize my credit card bill is much more than my yearly salary oh dear how do I do this what do I do it and I quickly think of one option I can grow beard change my name and move to another state just for the sake of record that's not the reason I left Texas and so the point really is that there would be a point when we no longer can pay it and what do we do we have to declare bankruptcy a number of projects do this they declare a technical bankruptcy also known as people quietly quit they say this is totally unmaintenable I'm gonna go work somewhere else so we have to develop the sense and say we need to pay this off by the way this is also a professional responsibility right as you're developing application you have to have a sense of what you're creating so that you don't accrue these debts and create poor quality how does it mean to create poor quality now this reminds me of an experience and please do tell me you were exactly like this to otherwise I'll feel really bad remember the time you were in college well I was in college and we rented an apartment there were like three or four of us having this apartment we would just live in this apartment but we would never clean it we would never vacuum it we'll just completely trash this place and one day this becomes very unbearable but it was very easy to fix we'll just move and we'll go to another apartment and we would get a brand new apartment beautifully furnished clean well painted and we would be there for about six months until we'd make a mess of it and then we'll move to another apartment you guys were like that too right thank you that's good and when I joined work I thought that's how life was like you work on a project you trash it and you then leave to another project but then when I did the first time I realized the project you learn you go into they didn't have the cleaning crew you are the cleaning crew because somebody else has left before and then you realize life is not like that on in the industry right that you got to make and you have to inherit code and you got to maintain the code so if you say you received very bad code there's a name for it the reason you got really bad code to work with is called karma you cannot change your past but you cannot say you guys gave me bad code I'm gonna leave a worst code on your hands before I leave that only will you know decrease on your karma bank you don't want to mess with your karma bank right so you got to fix this along the way that's very critical so it's very important for us to pay this technical data along the way that is extremely important so we need to measure how well the software is designed and implemented but it's a very qualitative approach there are some ways to make it quantitative we'll see that a little bit later so how do you measure this it's very hard to measure but we need some metrics here is a metric that you should definitely avoid doing if you are doing it it's got the line of code there's nothing more stupid than that because if you take this measure either way it is wrong if you say you have more code you're good then programmers get together you know the smart programmers they don't write more code that a program that spit more code right because after if you want more code I can produce that but overnight you come back while you worked all night long because we see more code in the system now having fewer lines of code is better that is a nonsense too because sometimes when we have fewer lines of code it may not be concise code it may be a terse code what is a terse code a terse code is a code which is fewer lines of code and you have no clue what it's doing it's whatever pack and you cannot understand what it's doing right so the point is you cannot measure quality based on lines of code so don't do it and if you're doing it that quit doing it or quit from the company that's doing it now it's very highly subjective right what is quality here you can say this is quality but but how do you measure it it's there's no real good standard to do that but we can try a few options it is very qualitative if a code is readable or is it understandable that is something you want to evaluate but again like I said how do you evaluate that have somebody read it somebody who's knowledgeable on the project read it you don't want to go find a guy layman on the street and say can you read my code right the person doesn't understand it and is the code verbose is the variable names and method names meaningful how many times have you seen code where a variable name is single letter have you yeah all of us have right I was teaching a course in Virginia and this client said I was telling about code quality and I was talking about variable names and this guy said we're a big software company how dare you come to us and talk about you know naming variables I said well I'm sure I've seen really bad code I thought you guys do that too and this guy said I was just giving you trouble and I was preaching them that you shouldn't have single letter variables about six months later I was back in the company teaching another course and this guy came to me and said hey remember we met last time yes I said yes he said I want you to know you made a big difference in our company last time you came and you said don't use single variables I was very proud I said yeah he said now we use only two letters right it's extremely difficult to break this practice right and I'm not kidding with you I was looking at a code I probably have that later and and people write code which is like really why can't you name these variables better right and and that is the real thing it takes effort to put good variable names we have to take it takes time to do that create simple code that works now what is a simple code by the way by the way there are a couple of different problems most of us don't know what simple is simple is hard simple is hard to identify Einstein said any internet and fool can create something violent and dangerous but it takes that's a genius to create something simple so it requires more time and effort to create simple things but I think there is a problem though fundamentally programmers not that they cannot create simple code they don't want to create simple code I mean you go to somebody and say here's a code I don't take a look at it the person looks and says that understood that's simple you feel let down you say what I mean I just showed you the code and you understood it give it back to me I'll be back tomorrow then you go back with another version and say here take a look at this version now and this person is scratching his head right until the hair falls off says I don't get it and you're beaming with pride like yeah I took a few hours to write it you better take a few hours to write it right so fundamentally we want complexity right why because this code is complex they say you know what we could let go let go people let people go but don't touch Joe over there because nobody knows what he is doing right that's job security right nobody understands the code you're writing but the problem is at one point you get to the time where that code is so messy you don't want that job right and that's where it becomes problem for the company that you work for and do you have test cases and if you have test cases what is the coverage of these test cases how much of code has been covered so these are some of the things we can look for to measure the quality but the very first thing I would recommend is if you're interested in quality there's a very good time to do it it is very early take a look at a little graph right I could say here is a graph of a time there's a timeline on any given project there is a time when you can turn the project towards better whatever the better is but that time very rarely comes towards the end so if you start early you have a lot of time to make things better and if you don't start early one day somebody says hey we found a tool we could use you turn on the tool and it gives you a million errors and warnings the first thing you do is right we don't want to see it because nobody has the time to fix all this mess right now don't compromise and say we want quality we're gonna work towards quality don't take quick shortcuts and say you know what let's understand this problem and fix it schedule a time to lower your technical debt every iteration or sprint you say hey here is a time you're gonna allocate for you know cleaning up tickling your debt how much time can we allocate that depends on how much time you can provide based on the technical debt that's been accrued I'm a huge fan of this particular recommendation make it work make it better real soon not in seven years make it work make it better don't try to become perfect right now make it work first once it works immediately refactor and make it better this requires monitoring but more important than monitoring it requires change in behavior and you go to a team and say hey guys I've got news for you you can improve quality if they say that is great as long as I don't have to do anything different that doesn't work right and you go to companies and say we gotta make change and everybody says that's a great idea let me know what everybody is going to do no we're talking about you what are you gonna change then we worry about other people around you changing stuff so it requires change in behavior if we are not interested in changing in behavior no waste your time thinking about this be willing to help and be helped this is a great thing you write something you go to somebody and say hey I have something here could you tell me how I can improve it rather than saying I wrote something you better not look at it and you're defining and holding a fort even I look at good quality good people meaning good programmers what do I see they write something they come to you and say tell me where I am stupid and I look at their code I'm like wow this is really nice and I go to guys who say I don't want to show you my code because I'm really good at what I do and look at that I think you know stings and so the more you get your code in open and have people look at it its quality can improve that's a great way to find better quality code so device lightweight non bureaucratic measures to do this so I'm gonna talk about what you can do as an individual and what you can do as a as a company to do this as an individual what can you do first of all care about design of your code pay attention to good design qualities first of all very simple things is your variable name and method names good I'll be very honest about this this is very difficult for me and I've really something over the years by the way I like to write code I write a lot of code but I realized over time that I can never ever write good quality code I'll be honest about this you can sit next to me and say Venkat write some code I would write code and he would say gosh that's all you can do and it'll be poor quality I'll tell you right away I cannot write good quality code but I'm extremely good in finding fault at your code I can look at your code I'll kick and scream and tell you how it sucks so over the years I realized rather than pretending that I can write great code I'll write quickly code and the minute I finish writing the code I will ask him to review it and while he is reviewing the code I will go review somebody else's code at the end of the day we both have better quality code how about that rather than we sit alone and pretend that we have written good quality code now we have both arrived at a better quality code so go ahead and I have other people review it are your methods method short are your classes small how many of you have think how many of you think long methods are a great idea not a single percent right I thought you were raising the hand but you're scratching your face that's fine absolutely not how many of you have seen long methods in their code just look around right this is called cognitive dissonance we know it's wrong to do but yet we see it why is that code so long because somebody in your office thought it was a bright idea and you know what the problem is you guys didn't do it obviously I know this look at the bright phases here you just didn't do it those people in the office did it but the problem is those girls are still at work today making those methods longer as we speak there is no way to get out of this right so yeah we have to take effort to make these methods shorter but one way to really write better quality code is to read better quality code you see one of the problems is if you were sitting in a bus and next to you in the seat if good quality code was sitting you want to know it you can't look at your glance away you didn't even know this was oh you are good quality code aren't you it's sitting next to you in the bus most of us won't even recognize it if we're sitting next to us right so one way to write better quality code is to read better quality code how do you do that go read some good books go read some good code if somebody says hey I was looking at this code I really like it and when you do code reviews I never even say Naresha's code I say you know what hey good job buddy you've done a really good job well don't keep it quiet go to the rest of the guys and say hey I was looking at this code I thought this was really good and I think we should do more of this so go ahead and show people what a good code is if you find it especially in other people's code they don't really believe you if you say this is my code look how beautiful it is like you are right but if you say this is his the code he wrote look how beautiful it is now people are interested in listening to you right absolutely and keep it simple and what is simple simple is all the unnecessary things that craft that goes away all the clutter all the things you don't need to do and it's to bear minimum does one thing and one thing well it's simple single responsibility right and write test with high coverage run all your tests before you check in your code as many as you can as fast as you can and check in frequently I go to companies I ask them how frequently do you check in your code once in three weeks how could you check in code once in three weeks we got to make sure it's correct what is correct this is insane you got a check in code as soon as a test passes and people say oh my gosh how could you check in code as soon as the test passes this is not complete when was your software complete why would pretend about all these constraints we draw upon ourselves what will happen if you check in the code right away others will use it what happened if they use it they give you feedback oh my gosh I don't want to see it but then you're worried about what you're writing right if you put it out and other people give you feedback it only improves your code what if I break it why would you break it you ran your test before you checked in why would it break be brave right most of us are covered Lee we are brave oh but I won't check in code I'm worried I would break it right I was at a client side I told them check in code oh no I will not why because something may break I said look I guarantee you I promise to you if you check in now if you check in the code now I guarantee you nobody would die tonight seriously you're not enough you know in a continuous integration mode where this is going to go fire up and launch it and launch the missiles tonight it's not going to happen I know what you guys do why are you so read about it nobody's going to die because you check in the code only we can learn and get better right so if you don't check in code frequently what is frequently every minute a check test passes check it in I check in code hundreds of times a day hundreds of times maybe thousands of time depending on what I'm doing I don't want the code to be sitting on my machine as soon as something works I want to check it in why the reason is extremely simple it's important to ask why though why because I modify a code I've not checked it in I continue to modify it and I spent four hours modifying this code and I've not checked it in and you come along and say here writing all this looks complex when you make this change oh don't touch it I've not checked this in yet I don't want to lose it you're worried about making the change or even worse what do you do you start zipping into multiple directories right this is the worst way to do it you gotta think of professionalism if the guy who comes to fix your plumbing does stupid things would you hire him again so we gotta watch ourselves are we doing stupid things would you zip your file in 20 places and keep it is that the way to develop software say that again that's even worse right then you gotta go look at all these shelving why don't we just do the right things right we're trying to hide behind things of course if you're using something like get it kind of gives you the shelving and that you keep checking in locally and then you can decide when to push it right but I think don't overdo it it's okay to get the software out there and get the feedback really quickly and then once you do that you can go back and improve it so one of the reasons I check in stuff is I can go wild and make changes in fact if I'm working on something and I say you know what I think there is a way to improve it you know what's the first thing I do I check it in because I would try to improve it and after about 10 minutes I realize and I will simply kick out the change I made right throw it away and just get from the version control I'm back on my feet to try my next brilliant idea and so checking code frequently gives you agility and fearlessness in making change to your software so that is why insist on checking in very frequently go ahead please great question do I check in before I do code reviews let me answer that question the answer is yes but let me ask how if I what the answer is if I work with clients and they are you know the pathway to hell is paid with good intentions you go to them and say why are you doing this we had good reasons for it but you landed in hell then don't do it right they had good intentions I go to them and they say code reviews are great you told us that and we read about it yes so we got a code review before we check in then I go around and say when did you check in last two weeks ago why because I'm waiting for code reviews quit doing it right so check in code keep it continuous I checked in Cody is reviewing it hey can I come back and change it absolutely we want to improve it right so that is why I don't wait until the code review for check-in in fact I would actually ask the other person to review based on what's being checked in on the same note I don't check in without having the test pass what's a bad what's the good news you're not reviewing code and saying this code won't even compile why am I wasting my time reviewing this code and so compile it run the test review it great so check in frequent learn your language I can't tell you how many times I can run across programmers and they'd say we don't want to do it why and they do all kinds of stuff from seven years ago when the language was there and because of why are you doing all of this when you do this other feature which came two years ago and they're not kept up with the language learn your language don't blame the language for what you cannot do if you're switching languages or you're a Java programmer learning C-Shop C-Shop programmer learning Java learn the idioms learn the differences you cannot apply the same thing the worst kind of guys are the guys I mean I cannot tell you how many times I go I go to companies and I meet architects they secretly tell me hey I'm an architect on the product and the last time I wrote code was six years ago in fact some of them are proud of the fact it's scary we have a name for those guys we call them the PowerPoint architects very dangerous people they draw pictures and in PowerPoint and they show it to you and before anything serious happens they quit leaving a mess behind so don't don't encourage PowerPoint architects very dangerous guys in your project right let them write code let them then be the architects right so this is very dangerous about the cargo cult programming what is cargo cult you look at somebody else and you do what they are doing hey why are you doing it you work for them such a work for us but why did that work for them ask understand why that's really applicable and I can court feedback and criticism criticism is a good thing because you can improve and when you give criticism don't attack the person instead motivate them to change and tell them how this could be better like that is what criticism is not not go going and bashing people keep it simple don't build a root Goldberg machine right keep it simple so you can understand what this is and maintain it don't divide something very complex and people look at this and say wow how does this work and all this happened before this one thing happens right so don't build it we all have built this in software you know what's the worst thing we build it and we are so proud of it is it isn't that cool right absolutely got to change it so keep it simple Tony Horace talked about there are two ways of constructing a software design one way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complex that there are no obvious deficiencies actually he goes on to say the first one is more difficult to do so we want to make sure the code is actually easier to understand so we can actually reason the code if it's so convoluted and confusing I don't even know what it's doing how do you know if it's correct or what the bug is right it's very difficult to go back and fix it so it's important for us to be able to create simple code so it's easier to reason easy to understand and easier to maintain it that is very important so what do you do as a team first avoid shortcuts why are we taking shortcuts because we can get this in quickly speed is important but that is a crashing speed that doesn't help take collective ownership team should own the code by the way I don't want Venkat's code and Raju's code that's that's a you want to avoid that right whose code is it it's the team's code anybody who has the desire to change the code can change it as long as they think the responsibility to fix it and make sure it's working anybody can come and change the code why should I do this if you're a project manager you want speed so I got a brilliant idea you said over there I'm gonna write GUI code from today and you said I'm gonna write database code from today you have been designated for XML processing sorry and every day I come to work these three guys do exactly the same thing he writes the GUI and nothing else he writes the database code nothing else he's still processing XML nothing else now are these guys gonna be really good at what they do as they do it absolutely so as a manager I say well look at that these guys are producing code really really fast and one day he says I quit this is no fun anymore because on the way to work he writes all the code in his head and all he has to do is type it and he's got himself good at it he types it and the compiler doesn't even give an error anymore and he says this is not challenging for me and I want to do this work anymore so he quits now we tell her and say okay he's gone who's gonna do it now and he comes over and says hey nobody here knows what he did ah you spoke so you're gonna maintain that code right and he opens the code and he says oh dear God what has he done in the code nobody in the project understands it how long this is gonna take to get back and knowing very long if you're lucky if not I worked on project where they say we poured concrete over it this is what I love in our field they would do stupid things but they'll give a nice term for it so you don't feel so bad about it right these are called euphemism report concrete over it meaning we suck and don't look over there and now what happened we come to a grinding halt this is called a truck factor or bus factor what does that mean the number of guys in your team who should be run over by a truck or quit for your project to fail how many what does that count on your project usually one one guy I've seen this in large enterprises they got a hundred thousand people working on it this guy says I'm the only guy who knows how to do this I said what if you quit this is the project will fail he spines when he says that right how fun what fun is that so it reduces your truck factor everybody you know hey you all will collectively work what's the speed slow why is it slow because everybody is trying to come up to speed six months goes by everybody is quietly working because they are familiar he says I'm going to go quit well thanks for working with us and the rest of the team organizers we bring one other guy we pull him up we all have understanding of the code right and and so promote collective ownership promote a positive in positive interaction give good positive feedback tell them hey this is how we can improve and provide constructive feedback yes please see but what you're trying to do right now is you're saying we got to be perfect in what we do it is it is competency of the team right and if you say I was working with one guy he said I will not write you I code I told him there's one thing you don't know how to rewrite you I code you and I can pair up and you will learn but if you will not write you I code I cannot change you I can maybe tell the boss to come and you know slap you around but it is a willingness and how did you you're good at doing something you do today how did you become good at it by trying it's mostly the lack of willingness I don't want to do it where says you know what I'm really eager to do this I'm not good at it today but I will be because I'm smart I'm professional I learned this way if I can learn this I can learn that and where is idealism in there it's pragmatism in fact because we want to elevate everybody and not you know seclude them and say segregate them you will go only do that and you know what great developers quit when you do that I work with those guys they come to the boss and say you know what I'm quitting I can't come and do the same thing because my career is at risk because if I want to go look for other job they ask me what else can you do no this is the only thing I can do you know how many people I know lost job and cannot find another job because that's the only thing they can do and nobody else wants it it's your career and we kind of restrict ourselves and then we say this is idealistic well how could it be idealistic when there are so many positive things about it right why do you care about any of these it is for us to improve so don't say everybody says that I perfect we already know it's not happening because one guy sits there and writes the code it's not perfect already not that this guy was the master of everything right I mean I've worked in companies where PhD sit there and write code and say these guys can rock and the guy quits and nobody can ever maintain the code anymore the company is sitting there holding binaries you could might have this is the source code is lost at least it's easier to convince the company that's gone the source code is here but nobody can touch it and we know the coherence is the contrary right so we have to kind of say how could we get out of that rut that's the whole point we are assuming that this one person writing code actually is marvelous the reality is totally not true I cannot write great code I've not seen anybody who can write great code everybody's code needs to be evolved and reviewed and the sooner we get to it the better it is I've seen companies lose products because of this because the guy who think they are experts end up leaving code that nobody else can maintain and that is the real bottom line so and constant code reviews and I am emphasized this even earlier but how do you really do constant code reviews one way is to do you know I say you say great code reviews are you crazy about code reviews you say the last thing we did code review we were in trouble you go to the department and say hey we got a code review on Thursday show up at two o'clock and then he says wait wait wait you guys are gonna be going to review my code on Thursday I'm gonna take the day a sick day that day I mean you're gonna come and slap me around in the work I might as well stay home sick and then and she says I'm already behind my project I don't want to waste my time looking at the code he wrote and the project manager says wait a minute code reviews the guy is the time when you all get into a room and you know post the code on the monitor and review it yeah he says no no no last time we did that there was a fight one guy quit we had to call the cops no more code review for you right it becomes political we do code reviews all the time but never go into a room to do it we do tactical code reviews I write a piece of code he reviews it next time I write a piece of code he reviews it the third time I write a piece of code she reviews it I rotate it among these three people and at the same time the code he writes one of us review and we keep rotating it and we don't go you know we don't give a review to them and follow them home sit next to them and say I'm gonna make sure you're changing it hey it's a professionalism right you wanted my input I gave it to you take it or leave it and you move on right and you tell them that and say here are the reasons I would change you decide but I have reviewed the code and you know my comments about it and this worked really well another way to do code review is to pair up pay a program with somebody but if you're pairing up with somebody it already reduces the cost the truck factor a little bit right you got another person working with the code but don't let two people pair up forever these two people you know it's dangerous these two guys pair up and they pair up for seven months and they really get together really well he says I quit you think he's gonna quit and be quiet he's gonna go there and say hey would you want to come and work they leave us a pair now your truck factor is still a tough factor I work in companies like that one guy quits within six months his entire team is over there right because they all like working together they all leave birds of feathers stick together right so rotate your pairs rotate your pairs so they can cross-pollinate and learn from each other I'm a huge fan of pairing I pair almost everything I do I do pair presentations from time to time I like pairing there are other benefits to pairing if you're sitting and pairing what are you doing you are focused on something and if you're stuck with something the other person comes in and helps you out and he has a book on pragmatic thinking and learning he talks about the left brain versus right brain he talks about these things there is another benefit often companies say oh wait a minute if I put two programmers alone they can write code faster but if two people pair up they produce only half the amount of code these people think programmers are stupid typist who can just sit there and type away that's not programming I'm not you're not dictating I'm not typing it is it is design it's innovation and when I'm doing it if it's really typing is fat what do you need you don't need programmers you say we're looking for programmers and use that typing speed will require you to have don't ask do you know Java do you know design how fast can you type you're hired why because you can type faster than the other guy we're not hiring typist right so what happens you are sitting there and programming I mean how many so how is productivity gain you're helping each other out you're producing more code but there's another thing honestly have you been coding and when you're coding do you ever get distracted all the time the ones were not raising either are perfect or in denial what are the chances we get distracted all the time you are there sitting and working and there's an email that pops we have to respond to the email today's worst enemy is Twitter you gotta keep tweeting what do you get work done if you're tweeting all day long so sitting there and programming tweeting checking your emails and SMS and then somebody else has to call you and then we kept distracted you know when I work I don't want to be distracted meaning I don't want to go to other stuff I'll turn off email or turn off Twitter I'll turn off everything that's not essential and I want to really have a focused time working on the problem if you come and talk to me I'll be very angry but if you and I appearing up that's great because we're working on the problem together now you're working on something there's this term called shaving the yak what is shaving the yak the Google for shaving the yak if you're not familiar with it shaving the yak is you do something totally unrelated to what you're working on you are working on writing code you're not sure what function you would call what's a good way to find what this function is real quick Google for it or was it Google for it so you click on your browser and the browser says hey there's an update available would you like to install yeah why not click on it then you get a proxy error huh what's this proxy error about I mean you could have used the old version of browser but no you cannot do that right I gotta check for this proxy error and then you look for the proxy error and you're trying to fix that for that you had to call this other guy you called him and he said yeah come and help you have proxy error but can you come and help me with this you get up and try to help it and suddenly you find you're sitting in a corner and shaving a yak they come around and say dude what are you doing to this yak over here I'm shaving it why are you shaving the yak in the middle of a day and how did this yak even come into the office and you're like oh I remember I wanted this good API I went to Google I better get back to writing that code now but what happens when you're pairing you're sitting there and pairing and what is this API Google for it you open the browser would you like another to do install a new update and that guy creates the third no not right not right now I have never seen do you guys sit together and shave a yak that never happens very very rare right and on top of that you guys should sit together and shave a yak that is why I love paid programming right you got no productivity because of that so code reviews are very useful but like I said go try the paid programming and do a tactical code review be unemotional about code reviews don't get really antsy about it right and so when you're doing code review don't try to criticize people instead give them constructive comments how do you improve this that is very important and don't be very emotional right so you want to seek feedback and provide feedback be constructive and so do tactical code reviews so what is a tactical code review like I said go around and have people do smaller you know little little tactical things don't try to boil the ocean and let him review the code today let her review the code tomorrow and then so on and rotate it I heard this somebody say code reviews make me a hero or make me smarter what a great way to think about it he said code reviews make me smarter most of the time because I give it to somebody and they come back and say how to improve it and very rarely they come back and say well I did not you could do I did not know you could do that and now you're smarter because you showed them something that they didn't know while they were reviewing the code right that's very important so again rigorous code review and take you know is very valuable another way to improve code quality treat warnings as errors warnings in code are evil I worked on a project and they hired me to get stuff done so I sat down I compiled the code it had 150 warnings I told myself Venkat you are there to write code for them don't dig in and start preaching them why they should fix it alright and a few minutes later but I want to see what those warnings are you're curious right so I kind of go in and look at this and the warning was if Boolean variable single equals true this is in production code right single equals not do equals and you're like okay you are setting the value why are you putting it if around it the ad spot never was ever reached now I'm curious I want to see what the next warning could be it was worse than that and I found out there were several bugs hidden in the warnings so I call around and said guys come on let's fix this first I was in another company they said you know I said hey you got to fix all your warnings treat warnings as errors they all laughed at me they're all laughing yeah that's kind of stupid recommendation said why because we got a lot of warnings I said oh really how many warnings do you have there was silence in the room they said how about just leaving it as a lot I said how about thousand warnings they said amp how about ten thousand warnings they said you don't understand the word lot do you and I said okay we won't go there and then I asked them do you have more warnings today than you did six months ago they said of course we do what do you do about it this what can we do about it this is the first thing about solving the problem is to contain it so right a little tool let it go into continuous integration and measure the warning number of warnings okay now this warning is X whatever that X is the big number we won't talk about right and the tool says if your warning number is X or less I'll keep quiet if it is X plus one it will fail the build so what does it mean you will never be more than X and you can configure the tool that the minute one programmer takes an act of kindness and fixes one warning it's no longer X but X minus one immediately lower that and in six months you have fewer warnings than you do today so it is not all of nothing I mentioned this to a company this was in 2007 timeframe I was back in their office a year ago and then they came to me and said for the very first time they got fewer warnings in their system than they ever did that is pragmatic in my opinion right how you fix it so take warnings as errors and I was working on a project I mentioned this I said turn on the flag in the compiler in the in the tool in the IDE and one of the guys said and nope I don't want to do it I said what why don't you want to do it he's not a good idea he said no it's a bad idea I said why he said I'm gonna go walk over to the build the machine and I'm gonna set it up on that machine so nobody can check in code with warnings I went and hugged him this is great and that is where you want to catch coverage and complexity we have code coverage can you get a feel for how much of code is touched what's the complexity of the code what's the code size what are the duplication measure some of these things along the way and there are tools for these by the way right and I'm gonna skip through the tools now I'm not gonna go through the tools but there is tools for code coverage you can Google and find these things you can look at cyclomatic complexity values there's Tom Thomas McCabe index but one thing I want to point out is this you can look at the complexity you can look at some of these metrics right don't duplicate code that's another thing you can do but this is a thing to think about if I tell you my code has low complexity that's a good thing right but I also have no test our very low test that's a risk isn't it absolutely I have no complexity but very high amount of coverage this is the world you want to be right that's the ideal world you want to be in great what about high complexity and low coverage this is higher than high this is really messed up beyond repair but how do you tell the people where you are so there is actually a very interesting tool for this it's called change a risk analysis prediction known as the crap metric the crap metric basically says how crappy the code really is it looks for a threshold value if the code the value is 30 so what do they do they measure the test coverage and they measure the complexity and then they put together a complex equation to find this and beyond a certain level the code has been crossed over to a crappy territory and you cannot fix it anymore so that's a tool you could use there's a tool called crap for a J for example so we have to also keep track of quality of code how do bugs creep in how do we deal with it we have to be very careful about it we got to think about code smells code smells come in various forms how do we deal with code smell what are some of the code smells we can identify long methods poor variable names inheritance when we shouldn't use it there are so many smells we could identify right the list goes on and on and on and we could collectively work together and find these code smells and the reason is to not blame it so basically the idea is that you want to really get quick feedback and don't try to boil the ocean right give the feedback so the idea really is that you really want to you know respond based on the feedback right that's the whole idea so be incremental about it so you want to deal with code smell you want to finish up technical debt those are extremely important so essentially continuously evaluate your technical debt continuous look at code quality continues to see how you can improve what you do right and again treat very very strongly take exceptions how do you deal with exceptions don't don't don't you know close exceptions and quietly tuck it away propagate it properly those things are important so to wrap up what we did I've got to skip a few things here so essentially the idea really is this you really want to continuously evaluate the quality of your code so a few recommendations to wrap up the first is continuously practice pure evaluation like I said I cannot think of anything more important than peer evaluations right and consider untested code as unfinished code testing is so important to do make your code coverage and metrics very visible when others see it there's a peer pressure to improve it right and don't tolerate anyone trashing your code take it very serious be angry if they