Hello everyone, and welcome back. Our next EDW session is called Data Driven Agile, Embedding Data Quality in Software Development. We're gonna be joined by two representatives of JPMorgan Chase for the presentation. Victor Bosco and Shula is Data Quality Manager and Jennifer Epoliti is the Legal Chief Data Officer. I'll remind you that all audience members are muted during the sessions, but the Q&A section on the right-hand side is a good place to leave your questions. So let's just jump right into it. Victor, Jennifer, please take it away. All right, thanks, Tony, and thanks for joining everyone. I'm Jennifer Epoliti. I'll let my bio speak for itself. I'm joined today with Victor, my Data Quality Lead, and we're gonna talk to you about how we have embedded data quality in the software development lifecycle of JPMorgan Chase legal department and are hoping to make everyone a data steward, so to speak. So Victor will walk us through the agenda and then we'll get on with it. All right, hello, my name is Victor. I work as a Data Quality Lead at JPMorgan Chase. Essentially my work consists of running different projects or initiatives to identify poor data quality and then work with the different teams to remediate that. So moving to the agenda, what we're going to cover today is four main sections. We will start with an overview of software development frameworks. Anyways, we are not going to spend a lot of time there on what the frameworks are, how they work. We will focus more on what are the challenges that due to the nature of these frameworks, the shell frameworks, like SP or Scrum, due to the shortened development cycles, it's often happened that data quality is left behind. So we will work on those challenges. Next, we will explain how we are dealing with that and we introduce our data champion role and our data-driven asset model. Second, after that, we will cover what are the concepts that we train our data champions? How do we prepare them for running this model? What data governance concept we explain? What are the design principles that we train them on? So they can better produce user stories, accept criteria, in general, improve the data quality from the beginning. And lastly, what are the benefits of doing this data-driven shell model? Now, Jennifer is going to cover an overview of this model. Okay, so I want to tell a quick story that goes back to five years ago at Enterprise Data World. I think it was in Boston. And of course, back then before COVID, we were in person and a bunch of us were talking in the hallway about the challenges of agile and how everyone's gone agile. And data just got left by the wayside. The agile process moved so quickly. It was all about quickly turning out a widget this week and a widget two weeks later and another widget. And nobody had time to think about data quality or even data management at all. And I felt that the data quality was beginning to suffer as a result in the industry in general. And we all were kind of moaning and complaining about, what do we do about this? And how do we make agile data-driven? Well, it took me five years and the right team, but I think we finally cracked it at JPMorgan Chase and Legal. We've been working in this mode with two of our applications very successfully. And we are now in the process of rolling this model out to the entire legal department, legal technology and our product owners that are working in the agile environment. So the key to this method is the data champion. We're gonna tell you a lot more about the data champions who they are, how we picked them, what they do. But the key is that they're not part of my team. These are people out there in the application teams who have been selected by their teams to be the data advocate, if you will, for their respective applications. So they're the ones who are looking after the data quality and the data governance of their respective applications. And Victor and his team at the center are more advisory coaching, stepping in when they have issues. And so it's absolutely vital for this model to work to have that data champion role. And then finally, on top of that, stopping data quality issues from happening in the first place with preventative design principles. That's something that we've developed training classes on that are mandatory for all of our developers that really talk to in things like data modeling in UI design, and even as something as simple as picking a slider versus a radio button, these are things that we train our teams in that can keep data quality issues from creeping into applications. And we're gonna share some of that with you today as well. Back to you, Victor. Right. So we're development frameworks. We have the two most known, Scrum and Extreme Programming, XP, and not going to cover all that, but the question here is, we have different roles in those teams, like Scrum Master, for the owners, developers, et cetera. But that comes to the question, who is responsible for the data quality within the application? And the answer, even though it's simple, that is, we all are, it's something missing there, but at least it was for us. We need to have one key contact or one person that is our partner or another set of eyes in all the teams, because essentially we cannot be everywhere. So we need to train those partners to have a data mindset and look for the things that we as data practitioners, we will be looking at. But we need to train those people to do that. And that's how we introduce the data champion role that Jennifer will cover now. So first and foremost, we think of the data champion as almost a mini chief data officer for an application, which means anything that I'm worried about, anything that keeps me up at night, should be keeping them up at night for their applications. So data sourcing, data use, data classification, privacy, quality, all of that is the responsibility of the data champion, but the primary focus is the data quality. So before I get into what these people do, let me talk a little bit about where they came from. We're fortunate in LIDL in that we have a legal technology team and their parallel universe in the Office of the General Counsel who are all people who are basically product owners. So they're the ones that face off with the technology and the business and bring the business requirements to technology and work on prioritization and user experience and so on. So we have basically business and technology representation for every application.  We ask that that person be either, they don't need to have the ability to do things like write a SQL query and pull data themselves, but if they can't do that themselves, they need to be able to ask someone to do it for them and to know enough about data and data modeling and data structures to know what to ask for. So that we let them choose, do you want your data champion to be a technology, a developer, or do you want it to be somebody from the product team? Once that person is identified, we had already trained the entire team, the whole group of the product owners and the technology teams on the data, embedding data quality into the application lifecycle. That was a training that Victor developed and trained everyone on. So once we had the data champions identified, that's where you start to look at these broader responsibilities. The first one is just to know what is a data project or a data story and what is thought. So everything that the team is doing, that the developers are doing needs to be assessed. Is it data or not? Is data heavy or not? So if data is being created, consumed, supplied, modified in that story, that's a data story. We also asked them to be aware of any data privacy concerns, any data use concerns that we might need to take to our forums or councils. And then we wanna make sure that the requirements coming to business are clear. And in some cases, we asked them to fill out a request template, which is a simple Excel spreadsheet, which asks questions about any new data that's being requested to be added to the application or modifications. So things like, is there a set of valid values we can use for it? Are there data quality checks that we should be doing on it? Where should we be getting it? What should the label be over the field and so on? We then asked them to be responsible for what we call enhanced data governance in the application. So that's for our higher risk or higher criticality data. We look at where are they getting the data? If they're creating data, is that necessary or is it already available somewhere else? Again, are they following those preventative data quality measures in the application design? Is the data sourced from the right place? If the data is new, do we know what the data quality requirements are for that data? And generally just being responsible for elevating the level of data quality in the application, identifying those data stories, making sure that the data stories are given an extra layer of testing and data validations that wouldn't normally be done beyond just UAT. And then making sure that those data stories have acceptance criteria specifically around data. So the acceptance criteria can't just be, the field is populated, it has to be the field is populated with one of the valid values or what have you. So that's the approach that we took. We're now gonna get into a sample use case, how we put this into practice. All right, in here what we can see is a sample process flow that can help you to understand and see how the data champion is interacting with the different groups or different parts of the process. So it always start with the spring planning. As the future team is working on determining what should be included in the spring or upcoming sprint, part of the data champion role is to review those stories and make sure that the right questions are being asked. Challenge the requirement to make sure that nothing is forgot about data impacts or something that can lead to data problems in the future. So essentially data champion is reviewing the stories, challenging what is there and then ensuring that there is a data source evaluation and make sure that the data is coming from the right place and there is an operating model for that as well because it's very easy for the end user to request data. Okay, we need to add this new field or this something else but how is that going to be updated? Who is going to be being accountable for that and taking the responsibility to keep the data updated and make sure that it's good. So all that is part of the data champion. Make sure that the data management, governance aspect of that requirement is covered. Make sure that the data consumers are engaged. Who is going to use this data in the future? How is it going to be used? What kind of validations are done? And lastly, make sure that our enterprise data catalogs or the documentation is updated. We are going to cover a little bit more about what tools we have and we train for data lineage and all that kind of things but that's part of the data champion. Once we start on the screen review, let's say that the requirements are ready to be tested in QA or UAT, the data champion needs to make sure that specific validations are done in the data like data profiling, validating data loads. And this is also not replacing unit testing or normal QA, it's adding an extra validation, extra layer of validation. And you will see that it's helpful because sometimes in UAT, you don't have maybe good enough data or it's productive or it's not covering all the edge cases and some things are missing. So part of the data champion is to make sure that those things are covered. Going to the findings, is there are things that are identified in those phases. Part of the data champion role is to work with the product team to understand the impact of those and evaluate if that needs to be fixed right away or if it can wait for next print but we consider that very important because in addition to the functional sign off for the UAT before production, we also consider data sign off. Make sure that this will not have any negative impact in the existing data or in the data that we are adding. Once we are in the production release, right after data champion need to make sure that validations that are applicable are also executed in production to make sure that there are no any surprises or findings there. And of course, if they are, again, the part of the data champion responsibility is to bring that up to the product team and the stakeholders, this is what we found and evaluate if that is critical or not and if it can wait or it can be fixed right away but that's essential for us because we have visibility of data quality issues and the stakeholders know about them before they find that in production. So, and lastly, when we have the ongoing data quality monitor, if applicable, of course that it's always ideal to prevent data quality issues from the first place in the user interface, adding checks or control, but sometimes it's not possible or that will be added in later sprint, more complex validation or check with an API in another service, et cetera, it's key to create some kind of mechanism or tools to monitor that in an ongoing basis, identify some data quality rules, report findings and make sure that this is also feeding the application backlog for future enhancements. I mean, if we find something in data quality monitoring, let's see how we can prevent that and work with the product team to retrofit everything and continuously improve the application data quality. All right, so that's kind of a quick overview of a process that this is what we follow on every sprint. Now transitioning to how do we train the data champion and on what concepts do we train them? These are the key concepts that we have training for them. For the data champions, product owners, application leads, for us, and to follow this model, it's key that they know about data governance, data use, data lineage. So we train them in all these concepts, including reference data, and lastly, design principles to prevent data quality, all that Jennifer was mentioning. What kind of even small things as selecting a dropdown versus a radio button, et cetera. All that kind of things is we train them and we have them to think about it. That's part of the data champion to challenge the decisions of maybe a user interface that maybe it's really nice and looks modern, but it's not preventing issues. It will cause issues in the data later. So all that we are going to cover more those design principles. Some of them are basic, but we consider this very important to refresh every so often. That's something we're going to cover later in life. All right, I just want to mention we are starting to get some great questions in the chat. I'm just going to knock off one question because it's a really quick, easy answer. We're on two weeks sprint cycle and we'll cover the rest of the questions toward the end in the Q and A period. So fit for purpose, this is one of the really important concepts that we train our teams on. What does it mean for data to be fit for purpose? And to put that into context for you, imagine that you're pulling data from some transactional reporting system, maybe a number of widgets sold and you're really interested in how many widgets have we sold and is the pace of widgets sales going up? Is the price increasing? And there happens to be a sale price and there happens to be a sale currency, but that's not really important to you. You're really looking at the volume of sales. Well, if that transactional system then sends data to a reporting warehouse that does a report on how much money are we making selling widgets in Yen? And there's been an error with the currency codes and you've got three different spellings of Yen. You're going to have a really inaccurate and maybe even catastrophic report. So that's what I mean by fit for purpose, that data, that currency data that was fit for the one purpose of looking at your sales is completely garbage when you're looking at your profit. So we ask our data champions to really think about data consumers. If your application is producing data then somebody is consuming it and when you're designing new features or new modifications, you really need to be thinking about where is the data going? Not just on your screen, but after it leaves your screen, who's using it? What are they going to do with it? And are they likely to have different purposes with different fitness and different data quality requirements than your original set of users? So that's one really important concept that we share with the teams. Another thing I wanted to share here is helpful tools for good governance. We're very fortunate at JP Morgan Chase to have records of our systems of record and our authoritative data sources firm wide. We've also spent a lot of time documenting not only what is the application but what are they the system of record or the ADS for what type of data, what kind of data. And we've also created a set of responsibilities around being an ADS or a SOAR, including data quality requirements. That may be something that you view in your data lineage tool if you have one, but if not, that's a really, really valuable tool to have in your arsenal. Another is an enterprise data catalog, which is a single source of knowledge for at least your data definitions, but maybe also additional classifications like privacy, confidentiality, other data related questions that you may want answered. So we use our data catalog to confirm when we're not sure what something is or what it means or which field to pull or how it needs to be protective, we can get that information from the data catalog. If you don't have a data catalog, a lot of that information can also reside in the data dictionary. Let's go to the next slide. The dreaded default value. So what is default value, first of all? A default value is when you have something like a dropdown or a radio button, and it has defaulted to a value for you. Somebody decided in the design process that that was the value that people were most likely to pick. That could be something mildly annoying, like if you're going to a travel website, you might be spontaneous and you were hoping that that date of departure defaults to today or this weekend, or you might be somebody who plans ahead and you're thinking, well, why doesn't that start a month out? Now I have to change it. So that's not a terribly critical example of a default value. Something important might be like is a vendor approved for business. If the person's filling out new vendors and approved for business is defaulting TS and somebody gets sloppy and forgets to click that radio button, then they might be saying and a vendor is approved for business that isn't. Another place where that can get really hazardous is in web design, where if you refresh the page, the default values reset to their defaults, well, people might not realize that and might leave them and their default values. So there are two risks around default values. One is just the sort of laziness factor where if you give someone a default value, they may not look at it, they may not bother to correct it and set it to the correct value. And you're gonna end up with incorrect data. That could even lead to legal risks and all sorts of downstream consequences. For example, if a default value has a dependency, if one value, if selected leads to additional follow up questions, well, you can miss those. So imagine, for example, a date of death field that defaults to today's date or imagine a diversity field that defaults to white, which by the way has potentially inherent racism to it, but then let's imagine that if you skip that field, there was a whole bunch of follow up diversity questions around diverse status that you never got to because you never clicked off of non-white. So that's some of the things that we ask people to think about in the default value space. All right, I'm going to cover free text fields. We know that free text fields are kind of the, one of the biggest enemies of good data quality. Assuming that we have a free text input, we can type anything there, copy, paste, add blank spaces, leading or trailing the white spaces, or even for the more technical audience on these calls, it can also have SQL injection and can cause a security risk. So how we can prevent this type of problem, right? The first one will be always include input sanitization. Make sure that we don't save the data as it is, that it has some kind of cleaning. And here it's going back to the data champion, challenge the need of the free text fields and avoid that when there is a risk that they will embed key information. Because sometimes what happened is that free text fields, they are used as a substitute for requesting new fields just to give you one example. We have information about our vendors and there is a common field and we did that kind of analysis using some techniques, text mining, et cetera, to identify what the users were using the comment for. And to our surprise, there was a lot of really good insights there. For instance, if the vendor was suspended, they just typed the suspended reason in the comments because there was no suspend reason attribute. They just changed the vendor status but they didn't have where to put why. So they added in the comments or another example just with that, when they want to add who is the relationship manager of that vendor, they only can indicate one relationship manager. But if they have more than one or even other roles, like who is the chief data officer or who is the, I don't know, billing contact, et cetera, they were putting all that in the comments instead of having that information in the proper place and being used for other things. So that's why free texts are dangerous. Of course that sometimes they are needed and there is no other way but challenge that so there can be another way if possible and try to keep the field size at minimum because the bigger the field is, it is easy to overload. Suppose that we have a zip code and the zip code side is 100 characters. That's unnecessary and it can introduce a lot of risk. And also evaluate what they are going to enter in the free text. Maybe there are just a list of possible values. So it can be a dropdown or free text or something that it's not needed to do a free input. And lastly, if we need to really use free text, evaluate if there are any kind of mechanism to validate that, maybe regular expressions or something like API or services that can be done to evaluate more complex information. Just like an address in Amazon, for instance, it auto corrects the address. We also have that internally, a type of modern chase where we can use our own services to validate address, make sure that it's the right city, et cetera. So always having that kind of centric tools to provide that for all the applications. So we have a reference data for a geography and we can leverage that across all the applications. Moving to data validations. In this part, we are going to introduce that, like I said before, always try to put in the UI as much control as possible to avoid any kind of issues. This is similar to the free text, but also considering a range of values like H, for example, or just to give one recent example for COVID vaccines. There was a man that he applied for the COVID vaccine and he was selected, but he didn't understand why because he was not matching the criteria because one of the parameters to be eligible for the COVID vaccine was the body mass index. And what happened is that when they entered the information, instead of using another unit, and basically that's why he was eligible for the vaccine because the calculation was making a different score. So using that kind of range validations for known values, making sure that units are expressed in the UI so they know what type of data they are using. And lastly, try to leverage mandatory fields where practical. If we are going to rely on some information that is input, we will use that for doing reports or group buy, and we really need that, make sure that it's mandatory. We are going now moving to drop downs. Here we have one simple example that we have Stelica preferred fruit and you have a different type of options. And then you have a chart and it says that Apple is the number one. But here it comes to the question, are they defaulting to the first option? Is it true? So here we have some recommendations for drop downs. One of the first ones that is a really, really bad idea is to allow the users to edit the drop downs. Let's say that I come here, I don't find the fruit and I add it. That's a really, really bad idea because they can maybe don't see the option and they enter Apple without S or in a different way. So that ends up being a nightmare for the values. So it's always important to have some kind of governance and control of the drop down options. And as usual and always sourcing for the right data store. In addition to that, if the options, the drop downs are a lot or not clear, it's always useful to use questions, type ahead, conditional or some kind of cascading mechanism to guide the user on the right selection. So they don't just pick anything. Well, like I said, sort of an adaptive data sources. And if the drop down have a few options or too many, try consider other options, maybe slicers, play your buttons, cascading. There are really nice UI designs that can replace this and they can lead the user to select the right data. See, we talk about authority data sources. Now, Jennifer is going to cover ADS. Right, so first and foremost, the obvious use for an authoritative data source is to source your authoritative data from it. But you can also use it to validate. You can use it as a lookup or to check in data quality validations against a set of valid values that are in your authoritative data source. Another thing that we ask our data champions to do is to really think about when you're sourcing data for new build, what's the exact data set that you want? For example, if your business users tell you that they want a list of workers or employees, ask them why they want it. What are they going to do with it? Because if they're using it to organize a party and they wanna know how many cupcakes to buy, that's one thing. But if they're using it to determine salary raises for employees and they don't need to see the contractors, then you're probably gonna give them the wrong data set or maybe you're using that data set to restack your cubes after COVID and you need to know who's got a permanent work from home assignment going forward and who doesn't, right? So making sure that you've thought through exactly what filters need to be applied or not applied when you're pulling your data, that's really important. And that's something that we see all the time. And I'm sure those of you who run data quality teams have as well where someone says the data is no good and actually it's like, no, you asked for the wrong data. Right, another thing of consistency is trying to use consistent data labels that isn't always practical because different user groups call the same thing different things and you want them to work in the vocabulary that they're comfortable with. But to the extent possible, your data dictionary should say, what is the right label for this term or maybe alternate acceptable labels and then try to keep those labels to a minimum so that when people look across applications and see the same data, they recognize that it's the same data. We're in a process right now of taking some data that our third party oversight team primarily uses and making it available to all the attorneys in legal. And what we're learning is that there's a lot of third party oversight jargon that is the absolute bread and butter to that team, but the attorneys have no idea what it means. So you need to either translate it into their language or do a tooltip or something to make that label meaningful so people don't get confused. And the same is true on the backend. Even when you're coding or if you're designing a database or a data model, striking that balance between using the standards, try to be consistent wherever possible but don't go so far as to lose the meaning of the data. All right, we have a couple more design principles that Victor's gonna cover and then we're going to share a success story. All right, shall I close this section of the design principles? Just all the recommendations that we have. The first one is that do not assume the user will do things correctly or that they will know how to enter data or that they even know about data. If a process is seen as a checkbox, they will just check the box. So if we are collecting data, we need to guide them through the right process and make sure that they don't make mistakes either intentionally or by honest mistakes. And part of that, it's all comes with the design of the application and also ensuring that there are a negative testing and see how the application will react to unwanted inputs or unexpected behavior of the user. Another recommendation is have bulk loads. In general, if they are a bad idea, of course that they are needed and bulk loads are there. But here we are talking about bulk loads that are done by the users. So this means allowing the end users to do bulk uploads on their own. Maybe loading a spreadsheet in the application or database. So in here, we're recommending that typically we should avoid that unless there is a very solid process because opening that door of propulating databases with spreadsheets, CSV, it can lead to a lot of errors and data quality issues. If there is no other option, make sure that the process is solid, also do some kind of validation of the file and prepare for the unexpected because we have all sort of problems when we do these kinds of things like allowing the end users to do the bulk uploads. And lastly, make sure that as part of the application, there is some kind of test automation for data management, test data management. So having all the different edge cases even with synthetic data or be able to reproduce all that automatically and make sure that as part of the testing, there are no other issues that are found in regression testing or some kind of problems that are introduced by new features or by changing something in the database or in the application. All these are just to give you an idea of what kind of things we are asking the data champion to think about it, to challenge the need of the users, work with the application team to keep all this in mind. And for us, it has been helpful and we have prevented a lot of problems doing this. So that's why I'm now I'm going to cover another section that is a success story with some numbers. This is one of our applications where we introduce this model. Essentially here, what we have is the number of release, how many stories, how many stories require an additional data quality review. Again, not only the unit testing, unit testing, et cetera, additional validations. How many data quality checks were executing, the number of findings in UAT and the number of findings in production. You can see that there were findings in UAT, but the numbers in production slightly decreased. You can see that what happened is that almost 80% of the data quality findings from UAT of doing all this model with the data champion, they did not impact production. And the other, they were considered low impact, but they gained visibility and they were fixed later. But the good thing about it is that the stakeholders, they know about those issues because we are the ones informing them and not them finding them in production. So this is just some numbers of how we track all this. We do track other things as well, but in general, this is how we have been keeping track of all we do. As you can see for us, it has been resulting very well and we are working on expanding this to all the other applications in the department. Okay, moving to a summary of the benefits of doing this. One of the key things for us is avoid costly data cleanups. Here, the idea is that most of the data quality issues, they can be prevented from early stages in the application design. If all those things are considered in advance, we could have avoid a lot of problems. And keep in mind that cleaning the data once the application needs life, it's in production, it's very expensive. It's time consuming in each of those ways. And also typically people don't want to change something while it's in production. It's reluctant to do it. No, let's better not change that or do not touch it or be careful. And maybe something that could have fixed in a couple of hours in the development, it takes months until it's decided to finally go in production and clean it or define a plan, a lot of meetings to define the impact, et cetera. So it's very expensive to clean up after. And all those UI validations in application screen can prevent all that. And lastly, the user experience and reputation. Especially when we are building new applications, we are launching a new application and it has already data quality issues in place. That's something in our department we have been introducing a lot of new apps. And for us, it's key that the user have a good experience because as soon as they see bad data, they start losing the trust in our apps. Second, we have data-centric development. All these having this mindset in the data champion in the teams, it helps us to prevent all this by thinking how data is going to be used, what data do we have, what do we need, going back to the fitness for purpose. That's often a hard thing because they say, okay, how do I make sure that data is fitness for purpose if I don't know what other purposes do I have? So that's why it's key to work with data architects that they can have a view or they can design a database in a way that is extensible and not just for that specific purpose. And lastly, all these data-centric help us to make sure that issues are visible and they are acknowledged. So we know how to prioritize them, we know what's there, we have full visibility of our data problems. All right, some additional considerations. The benefits of high quality data coming from your authoritative data sources or your systems of record. Now, obviously not every single data quality issue can be prevented by sourcing from an ADS because of the fit for purpose issue, right? But in an organization like ours where there are expectations around the application teams that own authoritative data sources and one of those expectations is an elevated level of data quality, you should be able to expect that out of the box you're gonna have less data quality issues than you would if you were building that data set yourself from scratch. Another consideration is when you don't have authoritative data sources you end up with data stored in multiple places that maybe get slightly different over time, create synchronization issues and confusion for users. Even something as simple as a timing issue. Years ago I had a data quality complaint from the prime brokerage that some of our trade prices were wrong and it turns out it was just as simple as they were using an overnight batch and everybody else who they were comparing with was in a real time update. So of course the data was out of sync because the data was different data. And then lastly, using that trusted reference data that you get from the ADS instead of creating your own means that updates will flow in automatically which actually will save time for your developers in the long run. And I know that's a point of contention. Oh, we only need two countries why don't we just make our own table instead of building connectivity to the system of record for the country for the country codes, right? But down the line when things change when you get that new consumer who has different data requirements that's when it becomes a problem that you just made your own table. Or God forbid you're the person that made their own table right before the Euro came along, right? And you had to rip everything out and start over. So those are some considerations on the ADS side. And then benefits of having this additional layer of validations that we've added to the agile process. First of all, putting that additional layer of data testing on top you find other things. You find things that your users wouldn't have found. You have people sitting and thinking about edge cases that maybe you would never would have come up with if you're just sitting and staring at a screen during UAT and saying, oh, the data looks all right to me, right? For example, you're unlikely to find duplicates unless you happen to be sorting in let's say alphabetical order. But if you have somebody thinking about all the possible different kinds of duplicates and testing the data for first name, last name reversed as an example, you're much more likely to find those duplicates. Another one is that having this data-driven agile model really does make it so that data quality is everybody's job because every team is gonna have that data champion going, hang on, we need to put a data quality story in this sprint because we need to fix that low priority issue that made it into production last time because we don't want it to linger and get worse, right? Scripting and test automation, again, the benefit here is especially with larger data sets, it's not a human trying to test all that, it's a script, it's automated, you can really go through everything instead of just doing spot checks and hoping that you catch a problem before it goes live. Another benefit is your business users will start to think about data quality because an agile they're involved in UAT, they're gonna be right alongside with the data quality team and they're gonna be seeing those errors that issues be uncovered and resolved. Just one thought on data-oriented testing, we talked earlier about bulk uploads and we're dealing with a really hairy one as early as this morning, which is one of these edge cases that you never would have guessed, right? But we let a manual upload happen for expediency and some of the users overwrote some IDs, they copied and pasted and we didn't catch it and then the next time the data came in from the authoritative data source, it took that correct ID and it overwrote the good data from the bulk upload. So having everyone sitting and thinking and that's a rush example, right? That wouldn't have happened if we weren't rushing, but having people sitting and thinking with each story, what can go wrong with the data is really invaluable. So I'm excited to get to the questions because we have some really good ones. I'll just wrap up by saying that we have this model, like I said, up and running the two applications. It's been fantastically successful and we're really excited to roll it out to the rest of the legal department and I hope some of you will take this on as well because I'm confident that it'll improve your data quality and increase the number of people who are data stewards in your organization. So let's go to the questions. All right, maybe I can read some of them. There is one that say, are these each of the sprints, the spring teams or at the project level? Well, we are doing that on every sprint and the thing is that some sprints, maybe they have less stories that are data intensive. So it really depends on each sprint. Sometimes it happens that there are sprints that are mostly UI or some kind of other changes that they didn't require all these. So it really comes to what's included in the sprint. I hope that answered that one. Right, I'll take another one. This is a great question. What kind of resistance did you get from the product engineering teams to assign data champions and how did you overcome that resistance and demonstrate value? That's such a great question. We definitely did get some resistance up front because this just feels like more work, right? We're just dumping more work on you. So you do really have to sell them on the benefits. I think the training helped with that, showing the success stories to the extent that you can get end users involved in your sales process, that really helps as well. And when the end users are the ones saying this is a worthwhile process and we're glad we're doing it, there's also the carrot and stick approach that says, look, this is something that we're rolling out firm wide and so do you wanna be part of the solution or part of the problem? And I think also it's an opportunity to sell people on the value of learning a new skill set. Data quality is a very in-demand skill set. High quality applications are in demand and so positioning that as you're going to learn a really valuable skill and get recognition for it helps as well. It's critical that this be part of people's goals and objectives so that when they're successful at it, it becomes part of their overall success. I don't know if you wanna add anything, Victor, to that question. The other thing is that we typically work with them when they're doing it for the first time, maybe for a few months, we help them, we are there with, it's not that we say, okay, we need to do this and they are alone, it's that we kind of work with their teams that we embed ourselves in their team and we slowly start doing that thing with them. And another question it says, is this focus on application development only? Is there any connection with data engineering teams? Well, the same concept can be applied on that. In fact, one of the application teams that we have is essentially a data warehouse team. This is a team that is building our department warehouse. So yes, it can be done on that thing. Of course, that's some recommendations that they are applied to UIs, maybe they don't apply to a data pipeline, but there are concepts that they apply as well. So the same concept applies for that. Great, and I would add that we also work very closely with our chief information architecture team and make sure that data modeling is part of major sprints as well. And data modeling is one of the best ways to get people thinking about fit for purpose. The data maybe needs to be a certain way now, but what does it need to be like in six months? What could it be like in the future? And data modelers are greater saying, well, what about this? You think about Victor's example with the comments where there are multiple relationship managers and the extra ones are stuck in the comments. A data modeler probably would have, when that application was being designed, would have said, well, are you sure you only wanna have one relationship manager? What about other roles? What about a one to many relationship here where you assign a role and then put a person in the role? If you'd modeled the application like that from the beginning, you wouldn't have ever needed to have people sticking their additional relationship managers for other regions in the comments. Here's another good one. What process do you have for agreeing or approving the use of an ADS to avoid unexpected performance impact? It's basically mandatory for us. It's part of our firm-wide data management policy that you must use authoritative data sources for critical data sets. That being said, we do give people a pretty long window so that we're not saying if you're not compliant, you have to fix it overnight and have a performance crisis. But we do ask that application teams provide a timeline and a project plan to become compliant with the use of the ADS. I think a couple of minutes left. Is there anything you would do differently next time? Well, that's a great question. I guess we must be perfect. No, I will say, I wish I had engaged some of the product owners a little earlier in the process to make it a bit more collaborative so it didn't feel like my way or the highway. We're doing that now, but I definitely thought about this as something that we needed to just figure out and solve this problem. And so we went in a room and figured it out and came back and said, here's the solution. I think we could have called on some of our colleagues to help us with that process. Pete had a follow-up, Pete Rivett had a follow-up. My question was related to the impact on the ADS of increased access. Ah, okay. I don't know that that's been an issue. A lot of our ADSs, especially for reference data, are fairly robust so they'll either have a message, a messaging layer or something that will protect them from too many calls. Some of them use APIs, but I'm not really the right person to answer the question, but I would say that it hasn't been an issue so that being fairly robust, I believe is part of the requirements to be an ADS in the first place. Okay, we will need to wrap it up there, but Victor, Jennifer, thank you so much for sharing your story. This has been a really wonderful presentation and I can tell from everybody's response here that they got a lot out of it. So thank you for that. For everybody else, appreciate you joining us. We will be starting up again in 10 minutes for our next round of sessions. Please stop by, connect with some folks from this room or speakers or the sponsors. And we look forward to seeing you. We've got two more conference rounds this afternoon, so hope to see you again. Thanks very much, everybody. Bye-bye. Thanks everyone.


The study found that babies born via cesarean section are more likely to have higher body mass index, BMI, age set scores compared to those born vaginally. This suggests that cesarean deliveries may lead to an increased risk of childhood obesity. However, this association was attenuated when macrosomia was taken into account.


Good morning everyone. I am Srikanth Patnaik. So first session will consist of version control. Before we start version control I want to show you some repositories which eventually lead you to motivate how why to use version control. So let me go back to browser. So this is one of the version control hosting website and what we will do is we will see some of the good projects which which are hosted on this website. So let us see most of you know about the Linux kernel. So we will just search for Linux. So in the first link you can see it says torbles slash linux. Torbles is linux torbles who we have started the Linux kernel project. So you can see this is the entire Linux kernel source code which linux torbles and he authored a day ago. You can see he has made modifications one day before. So you can see he has updated a day before. So what is all this? Why why why this website and why this linux version control? Why these many files? What is this read me? Let we get let us get through it before that just keep in mind then these files get changed day by day every day people download people contribute it is a large community. So it is a collaborative effort in which many people contribute to this project. So let us come back to our version control topic. So we will cover these many things version control using git is our main topic today. So there are many other tools beside git that of course we are not going to cover but they are also important but git is the most widely used and most fast available version control tool which we know. So let us start with what is not a version control before that let us know what is a version control and official definition. So what is a version control when you write any document in Microsoft Word or open office you save it right again you write something again you save it. So that is also a version control let us take a simple example this is my linux terminal emulator what I will do I will open a vi editor vi is a command line editor and say I will write hello dot txt. So this will open an empty file hello dot txt and if I write hello world and I save it correct. So is there any version control involved here till now not exactly again what I decide later no it is not hello world let it be hello India then again I save it. So far all good there is no version control practically but there is a revision control you can say. So hello India is the new revision of this file how if I undo this I get hello world. So this is the previous version of this file if I redo this I get hello India. So this is the latest version of this file correct fine. So this is a revision control this we do all the time in open office in Microsoft Word in emails we save our emails in graphs etc. But this is not good enough for our source code to be managed with you cannot write your source code save and undo it to make a separate version. What happens if you deal your source code like this or your hello India or hello world program with this if I exit this editor vi editor by pressing shift z z or call and wq write and quit if I open it again it will show hello India this is the latest version. And if I try to undo it is not possible it says already at the oldest change that means whatever previous changes to this document I made just before they all get lost why because I just close the document and everything get lost. So do we want this kind of revision control for our source code to get managed we will come to that not exactly we will come to that fine. So let me remove this hello.txt and let me come back to our slides. So version control is a also known as revision control or source control is the management of changes to documents computer programs large websites and other collection of information. So it clearly states that not only source code you can manage your any kind of document. So let us see how to do it means we do not use version control by default means this should be a default practice from now onwards because this is very easy to use tool of course it is very scalable you can scale it to any extent we will see just basic of that which will get used going. So normally people get confused with version control with software versions say software version of GIMP GIMP 2.6 it is not a version control it is a version of that software. So similarly the version of open office it is 3.6 point something now. So that is not the version control it is the version of the software and that numbers 3.6 does not signify any amount of source code management done to that am I making sense. So it is not a version control. So what is a version control? Version control is a source code management tool which manage the changes to the document or source code etcetera. So I think I made very clear what is a version control and what is not a version control of course without knowing why we need to learn the software it is just wastage of your time and mind too. First of all for anything we must be motivated to learn the topic. So I think this paragraph will tell you why we need a version control. First of all with our hello dot txt example what happened suppose I remove the hello dot txt what happen if in your source code you accidentally remove a source file does it mean that your project is means over you have you have launched a critical source data if this file would have been under version control you can retrieve the deleted file also is not that good. So you can go to previous versions you can even retrieve the deleted files also which are of course under version control then only. So this paragraph will tell you why we should get motivated to learn this particular tool and use in our day to day life the first one says reversibility. So it says that you can get back to the previous state if you discover some modification you did is a mistake or a bad idea of course suppose in our case in hello dot txt we did a mistake with hello world and we just get reverted to hello India etcetera etcetera. But with the version control you can easily get the previous version even if you close the file also concurrency its ability to many people modifying the same set of files or same source code and history means you can browse and navigate the history of the changes which you made. So what we will do we will just focus on a particular version control tool known as get version control which is fast robust and most popular tool around. So if you just search for get in your Google it will give you tons of results you have lot of online documentation you have very good support you have YouTube videos to support of course it is written by Linus Torvels who is the maintainer of linux kernel which is the single largest project in the world means it has more than 10 million lines of code still adding. So if this version control tool can manage linux kernel efficiently you can think of what it can do with your simple projects. So I think these three reversibility concurrency and history in a single word or in a single statement I can say you can able to revert your changes you can go to your previous version of changes you can retrieve some lost files and you can collaborate many people to work on the same project. So this is very important third statement is very important because none of the projects big projects or small projects done by a single person even the linux kernel it is initiated by Linus Torvel but it is carried out by thousands of developer around the world. So unless a version control tool allows you collaborate with people it is not worth using okay I think this is enough to motivate you why you should go for version control and why you should go for get version control especially of course I have not given you any examples of other version control tools let me give you a few. So there is a version control tool known as bazaar and there is known as mercurial. So there are very good version control tools but they have their limitations. So we are not going to discuss those limitations here. So our first topic for today is how to make a simple hello world project with get version control okay so this will take hardly five minutes because I am explaining also that makes it maybe 15 minutes but believe me initiating a program initiating version control tool for a particular project mainly takes two to three minutes that is it.  So what we will do come back to terminal I want to clear the terminal so control L will clear the terminal for me so it says record at the red del what I will do I will just change this name and replace it with dollar sign like this again control L to clear the screen. So now we have clear Linux terminal emulator let me do LS first to know what all files are here though it is not required let me delete unnecessary files clear the screen MKDIR what is the project team hello world hello iPhone world. So this will create a hello world directory in my home so where I am PWD will tell me in which directory I am it says slash home slash Srikant so this is the home for user Srikant so you can see clearly there is a hello world directory CD to hello world. So now our present working directory says we are in Srikant hello world home Srikant hello world let me clear the screen for you so what we have done those we started a simple hello world project we created a directory we CD into the directory now as the name suggests hello world we will this project indirectly says that there will be some source code inside this project which will print hello world finally so this directory itself communicate to user that what kind of source code or what kind of project is going to be this is also a good practice to have a directory named after the project. So before starting before writing the actual code it is also a very good practice and well recommended by many experts that you start with a read me file so read me file means the read me file should state about the project that what that project is going to do what kind of application is going to host etc. So what we will do we will create a read me file for those who are not comfortable with vi text editor vi is a command line text editor what I will do I will use g edit just for just to make you comfortable so g edit read me dot rst so g edit is a graphical text editor remember sorry I forgot to tell you these tools you can use it on your windows operating system also so this will open g edit so I have a read me dot rst and I already have written a small dot rst file read me dot rst file to save time so what this read me dot rst file says let me copy first if you have terminal with you please you also practice with me it is not showing any syntax highlights let me discuss this because I want to show you the syntax also so let me get back to vi editor it is if we get time I will tell you about basics of vi later though it is covered in the android advanced debugging so vi read me dot rst I for insert so I am in edit mode now fine so let me so escape will turn off the edit mode and if I press I or insert on the keyboard it will again put me into edit mode escape get back to the command mode so you can see there is a neat syntax here so this is a rst file known as restructured text which is the markup lightweight markup language which will deal in the end of this session from 2 to 230 so this read me is significant though it is not nothing to do with the git version control but you can see the syntax of read me why this is important is I have not written a single line of code but I know what I am going to do this is very helpful for my collaborators because suppose I have added multiple people to this project then they might want to know what exactly is going through going in my mind and what exactly what I am looking for so this will say that okay there is a project known as what is this project for this project will create a binary executable file which when invoked will print hello world on terminal we will get back to the syntax of rst file later so basically this double dots will make the header this will be subheading and double star will make this text bold take bold and prerequisite says what it requires it requires a GNUC compiler that is it how to use compile the source file and generate binary executable name as hello how to do that like this save and exit done now our simple hello world project we have our read me file in place to let me initiate the git repository inside this directory before that let me clear the screen pwd I am in hello world project let me do ls-la so this will list all the hidden files also so it says no hidden files this dot means present working directory this double dot means previous directory parent directory and only one file read me dot rst so just initialize the repository here single command git init git init will initialize empty git repository in this directory then you have to do git add you might think that how I know these commands because first of all I should tell you right means these commands are listed in such and such directory etc. If you get lost anywhere just do man git this is the inner star world documentation you will find many funny lines here so this will list all the commands and their use and to get information for individual commands you can also invoke like man git init just now we did git init to initialize empty repository in this directory so let us see what is the manual entry for that create an empty git repository or re-initialize an existing one q to quit this manual page control l to clear the screen let me do ls let me do ls-la so now you can see there is a directory known as dot git which will host all your future repository files in a non readable non editable format okay so never ever try to edit these this directory this will corrupt your repository or of course you will not delete this directory for sure now the repository in place I have to tell the repository I have to tell git to start tracking the file so how many files we have in this repository only one read me dot rst so what I will do git add read me dot rst this will add read me dot rst to git repository so git will start tracking this file suppose you have multiple files in this directory it won't start tracking all the files by default you have to tell git explicitly that I want to track this file then only git will start tracking that file suppose just out of context I will make two or three files here touch is a command that will create an empty file so touch one two three will create three empty files in this repository so you can see one two three and read me dot rst is there but I have not added file one two and three I have ordered added only read me dot rst so if I do git status it won't say anything about these three files one two three you can see them it says untracked files what does it mean it means git has no tracking information git has been not approached to track these files so it also suggest you if you have forgot to add please add these files git add and file name clear and it says there are some changes in this read me dot rst which has been recently added control l to clear the screen I will remove all these temporary files which are nothing to do with our program project coming back to my documentation okay so our read me dot rst file is there so how do I tell git to make a version of it so far what I did I asked git to add this file in the repository by git add command so git add will add this read me dot rst file into the repository now I want to say because I have completed finishing finishing this documentation read me dot rst so I know this is the right time to commit this changes what I mean by commit changes means I want this version of file to be saved somewhere somewhere means in the git repository only so what I want I want to tell git that okay whatever the version of read me dot rst file is there please commit to that so git commit so now I am committing that whatever read me dot rst file is there is a version a stable version appropriate version to come back from future if required so it will open an GNU nano command line editor so it is asking for a commit message so if I want to come back to this previous version somewhere from future then I must have some text to it text associated with so that it will remind me okay this commit refers to this this text message refer to this come in that means okay I can revert back to this because this commit miss this commits will be logged as a encrypted form you won't know you will not know the content of the file unless you revert to that version so it is recommended and recommended to add a commit message so that it will tell you when you come back from the future so this commit message will look like this so you can add anything I have added a statement like added read me dot rst what else I with instruction yes your commit message should be small precise and it should save what exactly you want to convey yourself when you return to this so I want to convey myself that okay this version means this so I added this read me dot rst file with instructions to compile and execute that's it it's basically done so what we have done now we have written a commit message for this particular commit now you might be wondering why this GNU nano what is this again you don't know vi editor you again introduced with a new editor forget about these editors you can modify this editor preferences later in your git config file this is a separate topic but for time being just just accept this changes that somewhere it will open a graphical editor for you in windows or in Linux also which where you can enter these things so control O will write the file and control X will exit so it clearly states on command line that one file changed 22 insertions in the in that file plus means insertions and created a mode for that rst file that's it we have successfully created a git repository for our hello world project isn't it simple what we have done so far we have done only these three steps we have done git init to initialize a empty git repository in our hello world program let me clear it first first command we did was git init then if I do git init it will again re-initialize it so I won't do then we did git add read me dot rst again we did after that we did git commit and we have given a commit message that's it we have created a success we have created a empty git repository but is it enough like we have seen before that git has main advantage with collaborative project means unless you invite people to work on your project there is no use of a single hello world project lying somewhere in someone's computer so let's let's add people to this okay so there is a note here if you want to remove any file from git you can use git rm that file name because that is not part of this tutorial so I prefer to add it in a separate note so likewise you added git add file name similarly you can do git rm file name everything is documented don't worry you will not get lost so the more practical approach will be doing this on some server so collaborating with people doesn't mean that you will share your files via email or you will create some tar file and put it in some pen drive and give it to your friend that is not a collaboration what does git offer you when we say collaboration git offers you a centralized server on which you can upload your files in technical term git says git push and collaborators can also push to that server it's like this diagram so there will be something cloud cloud means there will be some web website on which there will be repository which we will push and this is suppose my repository working directory I will push to this server and someone else may be someone else here will also push to this repository someone else here will also push to this repository and we will have collaborative coding effort means we will be assigned multiple tasks which we decide among ourselves or our read me will communicate there is there is no particular reason that collaborators should know each other suppose I am here and my collaborator is somewhere in US and we never met we don't know each other this happens because suppose I have this project hello world for example I have something better project say I have created something a programmer 8051 programmer someone in some other part of the world has appreciated it and you want to contribute to that he don't know me it doesn't know me then what you will do he will just ask me for the request he will ask me request that please add me to this project I want also to contribute to this so I will add him see if he do something bad to that project or if we remove something I can revert back to my previous version I have saved copy so every collaborator has a complete copy of the repository so let's not move around let's concentrate our work now it is best to have a server based approach rather than having a local repository so these are the benefits you can see you can access your file from anywhere you can share with people people can communicate with you people can see your files they can comment on your files etc etc and of course if you don't like any contributor as an administrator you can manage that also so you remember I shown you one web page github.com this is one of the web page one of the website which allows you to host git projects there are many such like gittorias or bit bucket or source force.net there are many such projects but most of the open source people choose github to host their project you can see the best example is we have linux linux project hosted on this linux torbal has hosted his project on this so this gives us some confidence that's okay this website might have some value so what we will do we will create a account here I assume that you don't have account on github I do so what I will do is I will just go to my account so it will save what all repositories I have so if you don't have an account it's free just create an account initial process is to create a new repository by clicking first option next to username this is my username and there is a first option is it says create a new repo click on it even this web page is part of this documentation you can go through this also I have previously created a hello world project just to make this documentation so it says create a repository with Srikanth Patnaik and repository name remember this is important because we have created our repository locally as hello world and git github.com doesn't know about it so you can't give any other name you should give only the only possible name allowed here is hello world the name should be same suppose if I started my project from github itself that is also possible you can directly come to github.com create a new repository then it will give you some instruction that download that repository and that will create a local copy of it so that is also possible so both ways you can do so I choose to do the previous first way so this is a hello world and there is a simple hello world project you should give some description meaningful description it says is it a public or private we are open source people initialize this repository with a readme not required because we are smart people we have already created a readme created repository that's it so it says Srikanth Patnaik slash hello world and this is the link for a repository okay so now github.com shows me few options it says if you have not created locally which is which I want to convey now then it says that create a file like this get init git add do these steps push an existing repository from command line this is important because ours is an existing repository isn't it we have already created a hello world project so all we have to do is we have to add this line so what does our documentation says so a new page with two option will be displayed as we seen so just add this git remote add origin why because this will tell my local repository this command let me copy this come back to terminal pwd I am still in my hello world project this give me a confidence that I am doing right I will paste this command git remote add remote means a remote server add origin means the url you can add multiple origins https github.com Srikanth Patnaik hello world.git this is the recommended thing to do that's it I have added my remote now what I will do so we have added our server information in our local repository we have created a repository in the server with the same name we have copied the url of the repository in our local repository then what we have to do next of course we want to push our repository to server so a simple one line command is git push will do the work but as we are doing it for first time you should tell which origin you want to push so we have only added one url you can host your code on different website suppose you have created hello world project on github.com and source force dot net also now you want to push to both the servers simultaneously you can do that so in our case so we have added git remote git push let me clear the screen control l git push hyphen you origin master so it is asking me username so I know my username it is asking me password star star star star star star that's it so it says that a new branch is created master and that's it so you want to verify yes you should just reload this page control r that's it so here is my hello world project and this is my password that's it my read me file and you can see Srikant has authored 14 minutes ago so it is a time when I committed it is showing the commit time it is not showing when I pushed to the server server does not have any intelligence whatever I commit that is local to me whatever my collaborators commit that will local to them whenever they push to server then already they get merged okay if there is a some conflict between the files suppose we will come later so you can see that read me that rst file is now rendered beautifully as this file so this is my heading this is my subheadings this hello world turn to be bold isn't it good yeah one more thing you need not to open github web page every time to push changes this this is a common practice people do they the thing that unless you get online on github.com you can't push on push your repository to get this is not true is nothing to do with browser you have added your git remote add URL then you push it it will push no need to get login into github every time though it seems seems stupid to mention it but it happens okay so far what we have done we have created our hello world project locally we committed a sing we have done single commit we pushed our repository to Srikanth Patna slash hello world now I am interested to add collaborators to it someone has approached me and he want to collaborate to this project you want to contribute to this project then what I will do I will go to github.com and in the settings tab in the same repository remember in the in the collaborators tab so I know my collaborators who collaborator whose name is akash m h rd this is a temporary account I created to just demonstrate this project add so what does this signifies this signifies that my solo project now become a multi user project so I have another user who will contribute to it and mostly this user m h rd akash m h rd will create the source code for this project so my task is to create read me file and I asked my collaborator to create source code logic that is it now splitting the task and collaborator side so what I will do so you can see who am I is a command which will tell who am I means which user is says it says Srikanth so if I download the project and again try to push git will push from my user account only because it is saved here locally but I do not want that what I will do control shift t this will open a new terminal and what I will do I will switch user to akash I have created a user akash on my machine so in linux gnu linux you can have multiple users logged in that same time password for akash is I do not know so you can see the font terminal color everything changed because it takes from local bash rc bash so if I type cd it will put me into akash at the red del so you can see here pwd who am I Srikanth and here who am I is akash that means there are 2 separate users presently logged in this machine so we are not doing any mess up so let me clear it and let me change this prompt from dollar to or what I will do is let me exit from this and let me create a new terminal and I will keep both the terminal terminal side by side so it will be easy to understand ps1 equals to let me resize Srikanth's window also now we have 2 terminals side by side control l will clear my both terminals so this one is Srikanth and this one is akash so what I am going to do is I am going to create what I will do Srikanth terminal already has this read me dot rst it already of course Srikanth is admin so has this project this akash is a collaborator he does not have this files one way is I can email these files to him ok old way the other way is he can download from github so they are multiple way to download he can click on read me dot file he can copy the content of read me dot file he can save that file in his local repository as read me dot file is it that is also possible another way is he can download this entire repository as zip that also he can do but as a collaborator it is not recommended what he has to do is he has to clone this repository cloning means getting the entire commit messages logs whatever the previous collaborators have done everything will be carried out to the new collaborator so cloning is the right way to get a repository so control c to copy akash user now control v so git clone so this will clone the repository for whom for our collaborator akash enter so it will throw some error do not worry now you see ls so now you can see a directory hello world in the user akash control l will clear the screen cd hello tap completion press tap to get complete ls so he has a copy of read me dot rst ls hyphen la will also show that he has this dot git directory to that means he has the commit information which Srikant the admin has done fine so let me see what does this read me dot rst contains it basically contains the same correct so as a collaborator I go through this read me dot rst I know my job I have to create a source file and the function of source file should be it should create a binary which eventually print hello world so this is the source file let me copy this so vi hello underscore world dot c i for insert okay so this is the simple program which will print hello world on terminal so as a collaborator I have completed my work now my job is to push this changes to server back to server okay that is what the meaning of collaboration so what all steps control l to clear again what all steps we should do do we need to do git init again no because git init will create a repository again because this contain already initialize repository we need not to create again so the what we will do git add hello world dot c so this will tell local git to track hello world dot c file so now git will track hello world dot c you can check my git status so it says our master changes to be committed in hello world dot c git push either you can say hyphen you origin master or as this information is already there you can simply say git push now it will ask for username so username of whom Srikanth Patlaik no username of collaborator whose name is Akash and his password authentication failed let me try again yes thank you username is Akash mh rd is it mh rd okay it says everything is up to date let me check the server control r to reload let me check the username of collaborator Akash mh rd correct git status so this is a lesson now so git commit will open a editor mt okay now let me check the user name when a new user comes in he has to configure git for first time okay so git suggest me how to configure that so all you need to do this thing in fact this is also mentioned in your document in document also I intendedly make this error so that when you do it on first time you will get to know how to do this so git config global username you have to do like this so I am just copying this paste which say some see this username and email address need not to be same of github this is nothing to do with github this is this will help you this will help git to create a encrypted file with your username and username and email address it will create unique non-editable encrypted files this is nothing to do with github account or source for account so what I will do I will give some say some email address and name I will give Akash so now I have successfully configured my git to do git commit so let me clear the screen control l now git commit so what should be my commit message anything sign which signifies added hello sufficient enough control o to write control x to exit now the collaborator Akash mh rd has created this file and created hello world dot c file he has committed to it so now in total to this repository we have two commits one commit made by administrator who has made commit to read me dot rst the second commit is made by collaborator who has made commit to hello world dot c so git push will push this changes to github fine so done because administrator has allowed me to push back the changes so I can push now let us verify again let us click on hello world program hello world repository you can see there is a new file hello world dot c you can click on that file that will open in local repository it says Akash has committed two minutes ago he has this is the commit message of Akash it says added hello world c program that is it fine now think of a situation where I have means as an admin Srikanth Patnaik who has initiated this project has decided to modify the read me okay most likely condition I do not know what my collaborator did in the back end I do not go to website all the time and I do not check whether he has pushed or not etc etc so what I decided to do I decided to edit my read me dot rst file and I want to add this last line to it so this is Srikanth again admin of the project what all he contains he contain only read me dot file read me dot rst and what is there with collaborator he has read me dot rst as well as hello world dot c now I decided to edit this file and I want to append this line at the end thanking my users what will be my process user process get add read me dot rst there is a shortcut to it see we have not actually added a file we have modified a file so you can do directly get commit by command line flags like add all files which are modified this hyphen m will tell you will allow you to add your commit message in line so if you type only get commit what will happen it will open a text editor and it will ask to enter your commit message but I want to add all the modified files hyphen a means add all the modified files I know there is only one file which has been modified and m means I want to add my commit message in line within double quotes modified read me read me dot rst simply that is it so this is my third commit for this repository first two commits made by me one commit made by my collaborator now I will do git push so this should push my this changes to server okay user name it says error what is the error fail to push some reference to server very well why failed same thing is documented here so so this says that there are some changes in the server which are not there in my local repository so I cannot push it so git is intelligent it says I will not push it unless you both means the collaborator and the admin have the same copy of the repository only one updated file can be there in the server clear so my collaborator has pushed hello world dot c file with read me dot rst file that is commit number two the copy of that I do not have and I did not check the server and I decided to push the file what happened it rejected so now this diagram will make some sense so this is the github server so you can see the first commit is made by me admin one signifies commit number it says read me dot rst I committed to means second step git clone done by who akas mh rd third step akas mh rd pushed read me dot rst plus hello world dot c good enough then shrikan modified read me dot rst and he also try to push unaware of that there is an updated version there now what happens it get rejected now what shrikan will do what any collaborator anyone will do he will do git pull then he will do git push so git pull will pull all the changes from server and it will merge if at all any merge is required and if ok it will merge simply merge do not get into more detail so what I will do I will do git pull that is what my document also suggest do a git pull so git pull from shrikan side and it pulled and while pulling see we had three commits right the git pull will merge the both files one which if it one with shrikan pattern that hello world and the commit means the modified version of hello world which shrikan has committed that commit will get merged with the collaborators commit so there will be a new commit so git will not modify the previous commits it will not say that ok your collaborator has committed before so he has the priority so you have to merge with something like that no when you pull anything from server and you have local commits it will merge those commits and it will create a new commit so if you remember this diagram git pull will what it will do it will fetch the it will internally it will execute two commands git fetch which will fetch the changes from server and git merge it will merge the changes whatever it get from the server so it will create a new commit so the commit message is already written so if I want to add something to this commit message I can so what I will say I will say my mistake I forgot to pull first of course so this will create a new commit which is not very relevant commit though the relevant commits are what which have some changes relevant changes this does not have any relevant change this is just a merge why because of my mistake I forgot to pull control o save control x to exit now I will do git push now if you see ls now the who am I I am streak on the of course admin so I had only read me dot rst but after git pull I get the changes of my collaborator also who has created this hello world dot c and of course my modified read me dot rst also the git push that is it so now the latest version on github.com is my merge commit commit number four does collaborator has this copy no you understand collaborator does not know about my merge does not know about my modification done to read me dot rst what the collaborator should do he should pull the changes so let us go to collaborator account and let us see what all he has he has the read me dot rst with with no greetings at the end then it is a good practice to do git pull all the time when you start doing some meaningful job so git pull will pull the changes and the changes signifies clearly that there is a commit and there is a merge so one file has been changed one insertion the insertion is the last line that is modified by admin now if you see what is read me dot rst thanks for using our hello world program so collaborator also has the updated copy so now we can say collaborator administrator and the github server all have the same copy right so the second aspect of version control we will see now which is reset as I mentioned earlier you can revert back revert to previous version if you have made any changes that is the beauty of version control like we have seen in our first file in that hello dot txt once you save and exit the file you cannot undo the changes but with version control you can easily get back to your previous state of the file or if you are in a previous state if you want to go to the next state that also possible you will just see that to know all the commit history there is a command known as git log hyphen all hyphen hyphen all space hyphen hyphen graph this graph is not much significant to us this graph will show a dot means this will show a plot so these action means this git reset I can demonstrate on either way on admin side or on collaborator side because both have the same copy now this we know so control L will clear the screen who am I I am Akash so what I what I will do git log hyphen hyphen all so if you do not know how this flag came what you can do git log hyphen hyphen help this will display all the help commands associated with git log or optionary you can do man git log this will also point to the same help content q 2 quit this page now let us do our git log hyphen hyphen all without adding a graph attribute let me unzoom a bit so it will display all the commits so it would be better if I could place all the commits in a single page so basically git this is my command so the top most entry will be the latest commit so the commit hash you can see this is a hash of that commit which is created from the user name and user name an email address provided by Akash while configuring the git not the git have user name password but the git local git configuration so latest commit shows that there is a merge of this hash and this hash so these are 7 digits you can count 1 2 3 4 5 6 7 so only first 7 digits are sufficient enough to point a particular commit you can you can point a particular commit by referring to the entire entire hash but it is sufficient to mention only 7 digits of each commit hash to point to that commit that is not relevant immediately but we will come to that so this is my latest commit I have selected it you can see it says merge author shrikan because shrikan has pulled and merged it date and this is my commit message shrikan's commit message so it would be better if I select up to here so it says by mistake I forgot to pull first this was my latest commit if you remember so before that there is a commit by shrikan only that was third commit which is a modification to readme.rst so this git log will show all the logs all the commits which is there in the repository this could be thousand this could be 2000 also fine so let us get back to our document so it says git log hyphen hyphen all hyphen hyphen graph let us see what does hyphen hyphen graph will show you graph will make a small text based graphical interpretation that who has done what you can see it says that this is a merge here and this is a commit done by akash and this is a separate this is a master branch this is a main branch on which administrator is working admin is working and someone has cloned here who is someone someone is collaborator who has cloned from here and he had made a commit what is his commit he added a hello world dot c file then his commit got merged here and so this is the latest which is common to all collaborator as well as administrator so this is a graphical pictorial representation of all commits and all branches etc q to quit now we will use the feature of git to get back to a previous version so which version we want to reset so this hash will not be present in this file because this I created two days back and that time my file and my content probably little bit different so just replace this with the commit log hash which we have now so to simply move our head means the latest commit technical term is this a head to simply move your head to commit corresponding to read me dot rst that is our first commit remember our first commit is read me dot rst which admin did if you want to move to that what you have to do git reset hyphen hyphen hard and that commit hash that commit hash you will find from git log so let me do git log again so you can see added read me dot rst with instruction now the commit message make some sense isn't it so can you remember out of nowhere if you have two thousand commits in your repository and simply point to what you want to mean without knowing the this commit message is it possible difficult so I can recognize my commit with this commit message now so what is my first commit message added read me dot rst with instructions to compile and execute this make sense right so I will take the corresponding hash entry it is not necessary that you copy exactly seven characters okay you can take some like this not an issue it should be minimum seven copy q to quit now I will do git reset let me clear the screen so that git reset hyphen hyphen hard and my commit hash so it should take my version back to first version it should discard everything else that means there should be no hello world dot c file in the directory it's not that means it should not have the modified version of read me which I did in commit three that's verify you can do cat also see it doesn't have that what does it signifies signifies that if a user in future thinks that whatever changes he has done to his source code or project they are not going well he want to get back he want to choose a previous version which is more meaningful he want to start from here not from some other version he can just do git hyphen hyphen git reset hyphen hyphen hard and that commit log hash and you can get back here okay this this has been done where on akash akash user so does administrator know about this no unless akash user push this modified version to github server the administrator won't know okay so to push the changes to server what you can do is okay you can you can check the git log again so it says there is no commit changes right we haven't done any changes to commit message instead of all all will show the all commit messages so far but we want to know what is my present state of log message for that let me clear the screen git log so git log will show the present state of the commits not all the commits from the entire repository from day 0 so it is showing me only one commit is true because I revert to this version by telling it that do a hard reset and come back here so if I know what is there now it shows me only this commit okay what about the other commits suppose later I realize no no no again I did mistake I want to get back to that version in which my collaborator just had added hello world dot c I don't want my modifications I want that version in which my collaborator has added hello world dot c so how to get back to that version simple you will do git log hyphen hyphen all because you know all the list of commits so this is the commit I was looking for this says author akash added hello world c program now I am where now my commit because I did a hard reset I revert to this version now I am here I am this version now I am here means once one version back now I want to go one version ahead of me by because I want to know I want to have hello world dot c also what I will do I will just copy this randomly more than seven has is control I will clear the screen up arrow will show you the previous commands so git reset hard instead of this I will paste my relevant commit hash now it says head is now at added hello world dot c how to verify git log git log will show you the head is at head means the latest commit latest commit is hello world dot c now how to verify visually you can see your hello world dot c file here right so suggested reading comes here so this is the official git hyphen scm documentation you can browse this there are many there are really good tutorials and examples here there is a book free book this book is pro git and if you want to practice git online there is a try dot github.com which allows you to practice these commands online also so that is all about git we will discuss about the about your questions during our discussion session.