The title of my talk is Auto Database at Sanity. Another title is Syntax Error, because there is no such comment in Postgres, and I will be talking about preventing disasters. Now, Gabriel from Second Quadrant delivered a very nice talk on Barman and how you can recover from disasters using Barman. But my talk will be how do you prevent disasters because you don't want them to happen altogether. And I work for Zalanda. Zalanda is a big company in Europe. We probably sell more shoes than any other company in Europe, at least most of the companies. And our marketing department is very sensitive about the numbers of customers we have, but I will just say that we have millions of transactions and Postgres is the system that supports most of them. So we have a big technology stack, and really a lot of technologies are involved in making our customers happy and making us happy as well and sleep at night. So all our components are interconnected. Somehow, unfortunately, they are interconnected through the database. And this requires additional attention from us, because we want to make sure that no single change will break the database and the shop should run without any down times. And our customers are really crazy. Sometimes they shop at 2 AM, sometimes they shop at during the day, during the holidays, 24-7. So we really have to make sure that our database is also run 24-7, and we don't have a possibility for down times. We don't have this luxury. So I want to talk to you about how do we change the databases. So Valentin is here. He was talking about how Postgres is easy. But to make Postgres really easy, you have to think about your process of development, and not only the software development, but the database development. And I would first talk about the changes in your database, about what possible problems one can encounter by changing the data in the database. And I will also cover some tools that makes this process more easy. And I will talk how in Zalando we use those and other open-source tools to make sure that our databases are always available. So let's cover the database changes. Usually there are three of them, the model change, which changes the scheme of your databases. This is what you do when you develop your application. And for instance, add columns or add tables and add foreign keys. All these involves model changes. And these changes are pretty... So applications are usually very sensitive to these changes. Because if the application expects a certain table and the table is not there, then the application just breaks. So you have to make sure that changes are tracked. The others one are not really less important. The data changes, usually it's data migrations. For instance, you may decide to populate your table after you edit it, for instance, with customers. Or you may decide to update your table to correct some error. And the table may be 10 gigabytes. And it might take quite a lot of time for you to do this. And the third level, third type of changes, is the changes to your code and to database functions. If you use database functions. And this also involves some issues, which I will cover. So there are different types of changes. And yeah, so this guy is expert for changes. Changes that work for you. But sometimes they don't work in production. So I want to talk shortly about some horror stories. And these horror stories are based on real events. But all the table names and code names are changed. So one of them starts like this. You get an internal error. And you see developers running to you and saying some loud stuff. And they are definitely not happy. So you start looking at the logs to see what caused this error. And you get to the application. You get to the Postgres work and you see this query. This is just a select. So you check the table is there. But you see that the column that is trying to use is not there. And you go to the logs a little bit upward and you see that this column was dropped by the developer. So some developer decided it's a good idea because the column is obsolete. And why would you keep this? We can save the space in the database and it just went to the database where it's equal and dropped this column. And now we have an application that is not running. For sure, once we diagnose this problem, it may be easy to fix or it may not depend on the content of this column. But what we want to do is not to fix the problem afterwards. We want to prevent such kinds of problems. And for this, we need to be able to track what statements are executed against the live database. And better, we want to actually test these statements before on some staging systems before running them directly on production. So I want to also cover one thing that is related to the logs. Unfortunately, it's a little bit too dark. But we have a tool that allows you to show the running processes on the database. It is similar to the top, but specific to Postgres. It shows you some system statistics and it also shows you what kind of queries are executed currently. Basically, the contents of PgStartActivity at a given time. And it's updates in real time. So what we can see here is that AutoTable took a look on the database and forced all other processes to wait on this log. And the developer thought this AutoTable is very lightweight because the table it tries to alter is almost an empty table. It has 8 kilobytes. So what can possibly go wrong? The 8 kilobytes table, AutoTable, run in an instance. But what actually happened is that someone was doing select from this table before. And how Postgres works? So the AutoTable acquires a very heavyweight log, the access-exclusive log. It conflicts with other horror story that involves sprocks. And also show one of our tools, the open source tool called PgObserver. Yeah, it doesn't have a light mode, unfortunately, it's only dark mode. We like darkness in our database team. But what you can see is the development is the running time for the sprock, the stored procedure that was recently deployed. And the red line delimits the running time of the stored procedure before the deployment and after the deployment. And you can immediately see that the time after the deployment just skyrocketed. So we jump 10 times or even more. And this resulted in a lot of timeouts. And the rest of the story, you don't want to be this guy. So the sprocks also has to be tracked. The sprocks has to be tested. And one has to be aware which sprocks are running on your database. So what I want to take are some techniques that are usually used for the source code of the application and transfer these techniques to the database development. So these are, for instance, version controlling. How many of you have your application under version control? How many of you have your database called under version control? Okay, quite a lot of people, I can say. And unit testing? How many of you do unit testing for your database changes? Valentin, raise your hand. You do it. Okay, okay. Yeah. So also, how many of you have staging environments for your databases? Yes, this is very good. Awesome. And the last thing is the special deployments for your database changes. So you may be aware that at the given time, your database changes, and it's not like your database is changing constantly because someone's going and dropping your columns and then your application stops working. So this is things we will have tooling for. And I want to mention that Postgres is really very good for doing structural changes and for supporting these kinds of tools. Insane and great for same database changes because it's fully ACID compliant, meaning that you always have your, once you edit your data to the database, you'll have your data there. Who actually knows how ACID is translated? No one? Okay, atomic, consistent, acceleration, very good. And the last one? Durability, exactly. So Postgres is durable. Once you put the data there, the data stays there. This is very good because you cannot test the database, it just doesn't retain the data you put there. It's not a database. The one thing that is very important is transactional DDLs. It's amazingly important for testing because you can actually say something like auto database, auto table, add column, and you roll by this transaction and the column is not added. The database is in the state like before the transaction. And you can write a lot of tests, like adding a column, putting some data into the column, adding a foreign key, seeing if it works or not. So all of this is possible with transactional DDLs. Postgres is quite unique in this regard. There are a lot of other commercial databases that doesn't support this feature. So you can write stored procedures in different languages and this encourages you to use the stored procedures instead of writing your code inside, writing your SQL inside your code. And the last thing I will cover is this also allows you to select which function, which stored procedure you run via this mechanism called the search path. So all of these makes writing tools to have structural deployment and change management in Postgres quite easy. And the first tool I want to cover called Sketch is the tool that is written by David Wheeler from Portland. He's a Postgres hacker that usually goes by the name of Theory. And he wrote quite a lot of amazing tools and Sketch is one of the tool I learned and actually evaluated quite extensively. So you use this tool to create atomic database changes step by step. He says about this as agile database development. So we don't throw all the schema at once, but you create table by table, column by column in small change sets. And it actually integrates with your version control system, for instance, like Git. So it has support for merging different changes from different branches and resolving the conflicts that may occur during this merging. This tool is also, so it has a explicit change plan that allows you to write changes and always know in which direction, in which order these changes will be applied to the database. The process that you use with this tool is deploy, verify, deploy, reward, verify process. So you write the deployment scripts. You write the scripts to roll back your deployments in case something is wrong or in case you just want to reward your tested environment to the initial stage. And you also write the verify scripts and this is very interesting concept. So you write scripts to test your changes and I will show you, right, how these scripts work. So, and one of the ideas here is you run your change sets. You then deploy it on your testing environment, on your staging database, which might even run on your local laptop. Then you make sure that these changes actually work and then you put them into production. So how it works, it's very easy to convert your existing code into Sketch. You basically, so here I just initialize the new Git repository, but it's not necessary to be Git. Then I just said, okay, give me the engine postgres, give me the project name pgconf.us and give me the URI, which allows you to have actually multiple projects and have interdependencies between projects. So then it creates some files. It creates the configuration, which allows you to specify the connection screen for the database among other things. The sketch plan, which shows the order of execution of your change sets and these three directories that will contain the actual change sets. And so let's create our first change set here. I will add the application role that will control which role will run my application. And it creates three files for me and says that it added this change to the sketch plan. So this is how the change looks like. It's always inside the transaction because if it's inside the transaction, you can guarantee because of this atomic function of databases that you either apply it completely or nothing is applied. So this is a transaction and there there is a simple statement that just creates a role for me. And sketch is also adding some commands to the beginning of this file. It says which deployment is it. So the verify script is also quite easy. What we have to verify is this role that this role exists. And what we do is we just check that postgres has this role and we have usage rights on this role. So if we run this statement and the role is not there, then it will throw an error and sketch will actually work with you and process this error as a failed test and report that you cannot deploy your change because the test has failed. And this one has a role back at the end because sometimes we don't want the tests to affect our databases. And how do we deploy these things is very easy. You create the data, I created test database. I say deploy with a given database stream. This database stream can actually be hardcoded inside, not hardcoded, but added to the sketch conf. So we don't have to specify it all the time. And then it will just apply my change to database and says, okay, this change is applied. I can add another change. And in the second change, I say that this change depends on the first one. So I want to create a schema for my database and to create a schema. I want to have a user in place because I will grant the schema to the user. So I do this and this basically the same command, just minus r thing that says it's a dependency. And it adds these new files for the change. Each change has its own file. And it also adds a string to the sketch plan. So this is how the change with the dependency works looks like. It's basically the new string in the command that says it requires a pro. It requires the previous dependency. And you won't be able to deploy this script if you didn't deploy the app schema first. Now I want to notice, note that sketch does not try to resolve the dependencies for you. This is still your job. What it tries to do is to make sure that you don't deploy your dependency. That you have all the dependent exact change names, the timestamps and the author of these changes and the commands that you specified when you created your changes. And you can also see that for the app schema, it has a dependency in the brackets for the app row. So you can actually take a look at the plan or you can even edit this plan manually if you don't like how the changes are structured for you. And then a sketch will instantly catch a top end around the changes in your desired order. So sketch also supports development in branches. So you might actually have a branch for each of your database change, then do something in this branch and then might merge this branch back to the master branch. And the problem with the merge happens because the plan file is the single file in the project and every change adds something to this plan file. So if two different changes in two different branches modify your plan file then some of the changes, then during the merge file, you'll get a conflict and you'll get a conflict like this. So one change added the talk table to our PGCon project, another change added the speaker table and since both of them try to modify the plan file, the merge just failed and we have to do something to resolve it. But happily we have the support for resolving such conflicts in Git. This, you can specify what kind of merge you use in Git and you can say that you want a union merge and the union merge instructs Git to actually take non-matching lines from both files and combine them together. So at the end, you get a file which contains changes that works everything. Even your database is in an empty state because you applied some changes and then we got the changes. Everything is locked in SketchVoc so you can always see who applied the change to the broker application or you can always use it for audit purposes. So how does it do this? How does it track these changes? Sketch revise on the metadata that is in your database. There is a special schema that is called Sketch and in this schema there are tables to track the changes, to track the, so it's a very easy release cycle. You test it on staging, you bundle, you tag it, you bundle it and then you roll out your bundle on production environment and you get exactly the changes that you put on the testing environment but running on production. So one interesting feature that I also want to cover is rebuking. Usually when you create a new change in Sketch you do it in the separate file but sometimes it's not convenient. For instance, if you create a new version of your function and you only changed a couple of lines in this function you don't want a second copy of your function in your Git tree. You actually want to use, to leverage the tools you have in Git to see the history and to track your changes to the functions. You always want to see who changed what. Git is very good for this. So there is a concept on in place changes in Sketch and these changes, so instead of writing new files for the change the revoc command copies the existing files and then the new files are named with the tag that comes between the changes and this tag actually marks the original change and this leaves you free to edit the original file. So this is a little bit complicated but I want to show you it and is an example. Suppose I have a function that says set talk and that sets the information for this talk and I want to revoc this function so I said sketch revoc and what it does is adds the original file to the plan but it actually renames this file and the new name of this file contains a tag that I said before. So this tag distinguishes between the original file and the renamed one and the new files that I have with the new that I want that I will change are those files that are shown below. So this set talk in the deployer about and verify you are actually free to change them further and they will implicitly depend on the previous version of these changes which are marked with this tag. So if I do git status you will see this untracked files these are the files with original changes that were added by a sketch and there are also changes that are there are also changes in the scripts. So what this revoc also does is that it puts the script that deploys the changes as a script that rewards the changes. So that you can reward the new changes to this structure that was before to the structure of version 1.0 of your file. So if you want to revoke, to revoke your revoked changes sketch wants to make sure that you will revote it to the original version. And this actually only works for some parts of changes. It works very good for functions because you can do create replace functions multiple times and it will always bring you the version of the function that you want and probably won't work for things like alter table at column because you cannot do alter table, sorry, not alter table at column, but if you just put the new, if you add the column to the table in the table definition and by doing create table and you try to apply it over the existing table in the database it just won't work. So there are only certain types of changes that you can revoke and these changes should be important. You should be able to apply it multiple times. So this is sketch. It's quite a powerful tool. So the concepts are you have a deploy, verify and reward cycle and you can instantly bundle your changes and put it in the production and you can also do in place, in place modifications for some of the changes. The next tool that I want to cover is a little bit more simple than sketch. It's actually a very simple. It's called versioning and for me all the tools that I see for managing database changes has very awkward names. This is probably the most confusing name I have for the tool, what tool did you use? I use versioning, what, versioning? What kind of versioning? Yeah, but this is the name of this tool. It's written by Depeche. It's capitalized. It's capitalized, yes. It's written by Depeche, the hacker from Poland and the idea of this tool is that instead you try to isolate your changes in the simple transactions, simple atomic transactions. And so instead of making changes on development and finding differences between production and development, you just isolate your changes in atomic transactions and then roll out your changes on development and then roll out the same changes on transactions. So the concept is very similar to what we have in Sketch but as I said, it's much, much easier. So it has also the notion of changes as atomic files contain the transactions. It can track already applied files. It allows you to write rollback changes and it also can maintain dependencies for you. So you can always find out which change has to go before the, before which one. So how the patch looks in this versioning tool, it's very easy. You get the transaction and inside this transaction you has to register this patch and you have to supply the unique name for this patch. It should be unique among the databases.  So once you do this, you can apply this patch to your database and you can see if you try to apply it second time, then the register patch function will catch that you already have this file in your database and will not allow you to do this. So every change can be applied only once and not more than once. And also has the notion of rollbacks. You can write your rollback diff and this rollback diff should cancel what you did in the forward diff. Like here we drop the table and instead of register patch, it does the code function that is called unregisterpatch. And this function removes the patch from the system table that tracks which patches are currently applied. So you can apply this patch for instance once again. So the rollback patches are especially recommended for some mission critical applications. If you're not sure that your change will work with your software and you want to quickly revert this change, then you just write the rollback patch and if something happens, you quickly apply it. So it has the concept of dependencies as I said. You can set in the register patch, you can supply the dependencies and it will make sure that the dependencies are already in the database before it tries to apply your page patch and it also, one patch can depend on multiple other patches. So it also have a concept of conflicts. I don't throw it here, but it works pretty much the same. You can say that a single patch conflicts with some other one and if some other one is applied to the database, then your patch will never be applied until you revert it. And this it, it's a very simple tool. It basically couple of screens written in PLPG SQL. It's very easy to audit. It's very easy to see what it does. It's very easy to modify if you like. And it has only one table in the database and the special schema, which has the name of each patch, the timestamp when it's applied, the name of the person who applied it and the requires and conflicts to track the dependencies which patches are required by the given one and which patches conflicts with the given one. So it's really very lightweight. You can really start using it without much preparations. Just roll out a single SQL file on your database and start writing your divs in this encode in transactions files in encode, patches encode in transactions. So these are very similar, but different tools. One is very simple and it doesn't say, it's very, I want to also mention some other tools. First is PG type, it's also written by theory. And this tool is designed to create unit tests inside your database. So if you saw the tests that we wrote here, these tests are pretty crude. So we use the functions that are supplied by Postgres in order to check that, for instance, the schema is there, but PG type is much more powerful to also get to see, for instance, if the table is there, if the table has the structure that you expect it to have, if the table has the proper defaults, if the table has foreign keys, if the function, for instance, that you defined has a certain string in the body, if the function is a volatile or security defined or and so on. So basically almost every feature that Postgres has is covered by one of the functions that you have in PG type. So it's very easy to write unit tests with PG type and it's also very easy to combine the sketch with PG type and write your full scale unit tests for your changes. The other honorable mention is the schema evolution manager made by Guild Group. It's similar to two tools that I already observed. It's a little bit different. For instance, it doesn't have an ocean of rollbacks because this guy said that if you drop a column that add a column is not a proper rollback because you already lost the data, but it may work, I may not work for you. It's written on Ruby. So if you like Ruby, you may check this out. Another tool I want to cover and to mention only briefly is the FDIF tool. As I said, all these tools have very awkward names. And this tool is designed to make sure that the modifications of your database functions stay in sync with what you have in your version control system. And basically it prevents... So you get a snapshot of your functions in your database and once you apply it to the new function, I can always check that the function that in the database after you applied it is the same that the function in your version control system. So it's basically designed for the environments when multiple people can modify your database concurrently at the same time and it might not be or might not be a very good idea, but if you have such an environment, you can check this FDIF tool. So this is a tooling and I want to show you the actual example of how do we use some of these tools in production and we end on the very... So we like the process that we have for our databases. Sorry. And this process consists of three parts. We have database diffs, basically the schema changes, which we use version info. And then we have the schema-based API deployment and we also have some tooling to make data migrations to make sure that one running transactions won't kill our databases. So first, how do we access the data? All our data access comes through these tool procedures and we use the Java layer, which codes the special middle layer, called SproKrapper. And this SproKrapper makes all the hard work of translating possible types to Java types and back and forth. And it also creates the proper function inside the database from Java's standpoint. You only write a certain class, so you give some decorators and then your function is magically translated to the function inside the database. This thing is open source. If you go to the GitHub for Zalanda, then you will find all these tools in SproKrapper as well. And all the Java layer talks to the Postgres layer, talks to the data layer, the stored procedures. We usually don't allow access via simple SQL statements, so we can control who actually access our databases, what operations are performed on the databases, so that we can always review the functions that we execute before they come into production. So our deployment procedures are very agile. We have a weekly deployment cycle for our code changes and this also covers the database changes. And this means that every week, basically, we receive sometimes tens and up to hundreds of changes to our databases. So we really want to track these changes. We really want to make sure that we know what we put inside our database, what schema changes we do, so that it won't break the production. Because as I said, we have to have a maintained 24-7 environment. And we use the Git to track our code and we also put the database files in Git as well and we put change sets in the Git as well. And we have the notion of release branches, so once we have a deployment, the deployment always goes from the release branch and this release branch is the branch that was tested and this testing and reviewed and tested and this review and testing includes the changes in the database. And as I said, we don't allow any downtime and we don't have any downtime during our deployments. So this is our Git structure looks like. Basically, our database files reside in the directory under the database and the data directory contains full definitions for our schema, for instance, such things as create table, the complete statements or create type or create function and also have the API and the notion of API. This is the functions that interact directly with the application code in Java. So it goes to the separate directory and we also have these dbdfs, the change sets that are produced by the versioning and these change sets are actually patches that we apply on top of our existing database schema that is running on production. So if you want to add a column in the database, you write the change set that says auto table at column and we put it into the dbdfs and then we have a special tooling that can find out which dbdfs we have to roll out depending on the current week number because we are from Germany, we use numbering for everywhere, for everything, including our releases and our releases are marked with weeks. So we find the proper week, we get the dbdfs from there and then we apply it first on staging and then on production. So yeah, I think I already covered this. I want to talk now about the database dfs, these change sets. We use the versioning system that I was talking before and this changes what is important. Our development team is very big. I mean, hundreds of people, more than 500 I think and we want to actually get it twice as big as currently but the number of database people are, database people are, we only have exclusive works and we have special tricks to do this. For instance, here is the dbdfs that actually runs the transaction that adds the column and adds the new column with the not now default value. Basically, if you do this in a single statement, alter table at column, not no default something, it will lock your table for the duration until it rewrites it completely and for big tables, it might take quite a long time and during this time, as I was throwing on the previous slides, everything that tries to access this table is blocked. So basically we can just use the normal alter table form in our diffs. So what we do is actually inside the transaction, we only add the column without the not now statement and we set a default value and this thing doesn't lead to table rewrite. This thing is very fast. Basically, it's just the change in the system catalog and all the heavyweight part that does the rewrite of the table that updates every row from the table, we put outside of the transaction and as a final stage after we made sure that all the rows are updated, we set it to not now. So at the end, the locks that are acquired on this table are very minimal. We only acquire one lock for the alter table inside the transaction, which is very small because it's a catalog change and the second one is we acquire the lock after the update statement and it's also quite fast because it doesn't involve rewriting of the table. So, and this process also runs for other diffs that has to change the schema and has to acquire heavyweight work. We try to avoid this. So we try to do not to avoid duplicating the definitions of our database files and for this Pseq can work with us because there is a notion of adding the data so inserting other scripts in Pseq. Basically here, we have the patch and in this patch, we just add the new table and what we could do is to just put create table inside this dbdif, but then we had to do it twice. We had to put create table inside this dbdif and we also had to put the create table in the directory which holds all our schema definitions in the data directory that. So basically, here in the data directory, we would also put the same create table statement. So we want to avoid the duplication so we just use the I command which inserts the full contents of the given file inside the database when we run this diff. So each dbdif proceeds, we proceed with dbdif with a certain, in a certain life cycle. First, it's tested locally with the feature developers. We have a background box and we have a virtual box that allows them to run postgres on their machines. Then afterwards, we run these things on the release staging and on the release staging, our QA teams goes there and tests the changes to the database as a part of the overall testing and after we make sure that on release staging everything works, the patch is applied to the patch staging, which is a special environment we use to apply hot fixes to the production before they actually apply to production and it's also applied to the live database. So generally, before it goes to live database, it's applied on at least three databases and reviewed by at least two database engineers. So we try really hard to structure the process in such a way that no diff is left unattended and no change is left unnoticed. And also have some tooling to actually review these diffs so you have a patch and you can put the comments here and you can say that either your patch is okay or it's marked is rejected and if it's rejected then the developer gets the email saying, okay, you have to work on this patch more. Here's the suggestions of how to fix it and then they fix it and then we review it once again. So this makes the process a little bit more streamlined. And we also have some naming conventions for our diffs. So as I said, this versioning tool is very simple, very easy, just one screen of functions. And what the power of this tool is that you can put some additional policies on top of this versioning and customize it for your specific process and for your specific company. So in our case, we use, for instance, Jira and for in the name of our deployment, in the name of our versioning scripts, we have the Jira version of the ticket that is, we can actually check before applying the diff that the ticket that is related to this diff is in the proper state. So it's not in the state of still working on it or it's not in the state QA didn't work on it anymore but it should be in the state like completed ready for production. And we also have some names of the databases inside our diffs that allows us to write tools to actually locate these diffs. So if I say something like, okay, so this is about the schemas but what about the functions? Since I said that we have all the data access gone through the functions, we also have to make sure that the functions are properly tracked and deployed. And for the functions, we use the trick that is called the search path. The search path is the feature of Postgres which allows you to switch the function that you are running instantly and in runtime. So here in the example, if the search path is set to first one then you'll get one function and to the second one you get another one and the function name is absolutely the same. So we use this trick in our deployment to switch between different versions of APIs. As I said, we have one API per week. So each week we deploy the new version of API and in order to select which APIs we use, we just switch the search path inside our application. Just one statement, very easy. And if something goes wrong, we can always switch it back. So we can instantly roll out the previous version of our database code or we can just test the new version of our database code before the production application will pick it up which is also quite powerful and we use it. And our API deployments look like this. So suppose we have two instances of the application running Java and they are trying to, so they connect to the Postgres database and initially they use the old version of the application and when the deployment, we just roll out the next version of stored functions, the next version of stored procedures basically by executing these SQLs against the database. And now we have the two versions of API, two versions of our stored procedures running concurrently in the database. But the application is only using the old one. The new one is just in the database. It's not used yet and the QA can test it and we can take a look and see that it's rolled out correctly and that's the version of the database is really the version that we want to. And afterwards, after we make sure that this is the proper version, we just switch one instance where the load balancer, so sorry, we are not the load balancer, we just switch one instance where the search path. So in one instance, we say, okay, search path to the new API. And this instance picks up the new API and transfers the new API. The old instance is still running with the old one and what is important that we maintain the compatibility in the schema between the old and the new version. So every dbd that we try to roll out has to make sure that it does not break the schema. It does not change the schema to be incompatible with the old API. We always say that the new changes to the schema has to be compatible with the still running API in the database. And by giving this, we can achieve this step. We can run multiple versions in parallel and actually use multiple versions in parallel. And once we tested this, once we made sure that this thing is working correctly, we can switch the whole application or other instances to the new API. And suddenly we run the new API and this deployment happened without any downtime and without any troubles from the database part. So the last part I want to cover is how do we bootstrap the databases? These are very easy since the databases are running in Git and we actually don't use something like plan files because we have very many changes and these changes are produced by a lot of developers concurrently. So we don't want them constantly resolve the conflicts when modifying the single plan file. So we just use this versioning and we write the changes in such a way that, so we number our database scripts basically, the most simple process. And when we try to roll out the database on staging or on production, we just use the find command to find our SQL files or them by the numbers in the files and then just apply them to production and do it in one transaction. So if everything happens, you still have the database that is in the same and in the consistent state. And by doing this, I mean by having these SQL files, we can actually make sure that the current code can always be rolled out on our testing database with a very simple command. So one thing that we also do is the data migrations and since we have tables that are multiple tens of gigabytes in size, we cannot just run update on these tables because this update will probably block other processes trying to modify the data in this table. So what we do instead is that we split updates in chunks and we have a special Python script that can run these updates chunk by chunk and it also can run them concurrently on different shards because we also use sharding. And what is important when you run the updates and what we will come across multiple times is that when you run a big transaction on your database, you may not only block the existing processes, but you also, for instance, may run out of storage for your right-hand walk and then your database will just crash or you may even bump up the load on the server so much that it will be unusable for every other process. So we have these things covered in our tool and we try to control the load and we try to control the wall storage and we also think that we have to run vacuums after a certain number of chunks because otherwise we will just lose this space and we run vacuums and this is also configured. So these tools are not yet open source but we are just trying to eliminate our internal dependencies on some other tools that we have open-source eventually. So the idea is that you use the select function to get you the IDs of the rows that you want to change and then the second statement just changes these rows and you can set the number of chunks that you use for this and you can also configure the vacuum. Actually, in 9.5, it will be very easy. You may roll out your own schema migration without such a tool because it has a new feature called skip locked which allows you to lock to select some rows for update from your table and skip the rows that are already locked. So you may combine this in a common table expression with an update statement to actually get your rows selected, then update these rows and then return the count of updated rows and run this command until you have zero updated rows which means that you updated all the table. So in 9.5, it will be much easier but so far we have to rely on our tools and some things that we only want to implement is the testing of our changes. So we use versioning. Versioning doesn't have an ocean of testing but we actually make changes in two different places. We write dbdfs for the changes and they also make changes in the schema files, for instance, in the create table. If we want to add a column, we write a dbdfs that adds the column and they also modify the table definition itself to add this column. So the idea is to combine these two sources and to test that the dbdfs actually put the database into the state that is in our version control system but so far we are not yet implemented this because it's, git is a little bit problematic on how to find the path in the git, how to find the branches in the git, I mean how to find path in the git from a certain commit to a certain commit. So we are working on the schema comparison tools that allows you to find all the difference between several environments which is quite important if you have a staging environment. You want to make sure that the staging environment is the same as a production and you can use something like PgTab for this or you can write your own tool to do this. And this is what we are doing. Yeah, and one thing that we also want to do is to handle the dependencies between the schema changes and the migrations. If you want to add a column in the table and then populate a column with a big update statement then you don't want to fire update statement before the column is added. So this is also something in the future and we are constantly updating our open source tools and we are constantly, we are trying to present about them in the conferences so stay tuned. We are going to roll out new things. Yes. So now do you have any questions to me? Yes? Can you go back to the slide with the links to the tools? Yeah, actually those are links to the, yeah, here. This one. Yeah, I will put these slides. To the, somewhere, I mean in Postgres Wiki for instance and I will also add the links to some other talks that covers the same topic. So I want to have some comprehensive repository of these practices so that you can choose between them and whatever you want. Yes? We also forgot to mention is that we are not only checking the, reviewing all the changes that developers write, but we also provide the training for developers that they know programs that can happen when they... Yeah, yeah, we do this. Yes, when the developers come aboard, we make sure that they can write SQL, they know what Postgres is, they know how Postgres works and they are aware of our infrastructure and this is part of the developer training so we want to trust our developers. We don't want to be a bottleneck in the development team. We just want developers to do everything by and themselves so we can focus on other tasks like helping the community for instance or writing open source tools. Yes, this is a good addition. Yes? How do you check the performance of the queries? So how do we check the performance of the queries? This is a very good question. First of all, we have this, sorry, that was too fast. Yeah, we have this PG View tool that allows you to view the currently running queries and for how long are they running and also some other metrics. I think it's somewhere in the beginning. Yes, yes, here. Yeah, it's also open source in our GitHub and when it comes to testing, we just, we usually test, try to test big data, select big statements that extract data on the same data set that we use in production so we try to, we don't have some profiling tools. I mean, there is a profiling tools for PLPG SQL but we only recently started using it so it's not very, very widespread in our company but we try to provide developers with the data sets so we periodically get the data from the production and populate the staging environment with it, upfuscating it a little bit because of the regulations and then the developers can test their queries on the real data sets and this is how we make sure that the queries perform adequately. And we also have this PLPG SQL provider as I mentioned. Okay, any more questions? Okay, then I will be here and if you have any questions about our process, about these tools or if you have any suggestions on these, on other tools that you use, then I will be happy to talk to you afterwards. So thank you very much. Thank you.


This is a video on how to calculate binomial probabilities. Family wants to have six children. Assuming that the probability of a child being a girl is 0.5, find the probability that the family has at least four girls. So that would be greater than or equal to four. Exactly four girls. So that's equal to four. And at most four girls. So that would have to be less than or equal to four. Now, it's clear that the behavior we're observing here is a child being a girl. So a success for a binomial experiment in this case is the child is a girl. So that means that the probability of a success P is equal to 0.5. Okay, so the Google Sheets input for each of these, at least four girls will first off the number of trials or children being born as six probability of a success is 0.5. This stays the same for every single scenario. This never changes. What does vary slightly though would be your lower bound and upper bound that you put into Google Sheets. So if you're interested in at least four girls, this would mean four, five, all the way up through six. You can't have seven because you only have six children. You can never exceed the number of trials. So the lower bound is four. Upper bound is six. Exactly four girls to lower and upper bound would be looking at just four. Lower bound upper bound is exactly the same. At most four girls that means zero, one, two, three and even includes four. So the lower bound in this case is zero and your upper bound would be four. This is all the information you put into Google Sheets. If you're not there already make sure you're in the compute tab. The number of trials would be six and then the probability of success is 0.5. For my first part my lower bound was six and upper bound was four. This gave me a probability of 0.3438 rounded the four decimal places. And the second part where we did exactly for my lower bounds for my upper bounds for. So this is about 0.2344 if you're on the four decimal places. In the last part where we did at most four, the lower bound is zero, upper bound is four. And you would get 0.8906. Those are the three answers to this question. So part A, your answer was 0.3438. Part B, your answer was 0.2344. And part C, your answer was 0.8906. So those are the probabilities for each of the parts in this question.


When I have your wounded five words dust off medevac members at forward operating base Salerno live by every day I get to provide care to soldiers and injured personnel on their absolute worst day of their life The name dust off refers to the army units responsible for evacuating casualties from combat A little bit more on duty 24 7 we don't have the luxury of being able to stay down for two or three days To talk for a Patient's lives depend on this team's ability to act fast and provide medical care no matter the circumstances We have to be ready to be able to do any sort of mission.  We will get to the mission We will accomplish the mission, but we have to be prepared for everything that could possibly come It's a challenging environment to work in we land in some pretty remote areas Put patients on board the aircraft and have to do a lot of sort of advanced medical procedures in the back of a moving loud hot Aircraft once they get the call this team is up in the air in less than five minutes Which gives their patients a better chance of coming back home alive my thing with dust off this It's my chance to make a difference in our guys lives when they need us the most We're there. We will land. We will pick up your patients. We will have your mood It gives me a chance to help out my fellow soldiers and get them back home back to their families So it's it's real rewarding


So, these appear, and then it's, yeah, of course, R, and sorry, okay, excellent, oh. It's a meaning, since everything is E, A. And then, end by tensoring, and one more thing. So, consider the map A to A, R, A, so this is a generically flat. That means, so, localizing by one thing, so adding, so, A inverse, we can assume, also, yeah, yeah, R, A is flat over A, okay. And so, we call this R, A, a model, model of R. And then, to take any maximal ideal of A. Then, of course, A over mu is a field, finite regenerated over Z. That means, this is a finite field. And, of course, characteristic is some P. Going to consider some property, so P, which is preserved under a basic extension field. I don't say explicitly a rational, a pure, et cetera, is preserved under a basic extension field. But, that's, and so, this, yeah, okay, so, the property P has P does not depend so on, so, so, individual at only P. So, choosing some A, so, we, for some mu, that makes models very simple, okay. Okay, so, and we have two, so, we say, so, for some P, open P type, except for finite, R has dense, sorry, I forgot if P type, you can, for infinite, okay, so, for example, so, this R has, yeah, R P is a rational, except for P equal to E5. So, this is, this R is a rational type. It is same thing as regular type. So, but not R, okay, R equals, so, so, we have seen that R P is, if I'm going to leave, yeah, P is equivalent to 1 model P. So, this is R is dense P type, but not open P type. So, yeah, it's very easy to understand. Okay, so, okay, first, yeah, so, it is shown by, so, maybe, this, so, R is defined by character zero. So, R, if I don't leave, R is rational singularity. And, so, this is a very recent result by Mar and Swede, and going, Tucker will talk about, so, it's necessary for, for some P, R P is a rational. Okay, so, so, one way is by Smith, yes, and so, if, okay, so, we talked about, so, should rational ring, and so, yeah. And then, you can choose, so, so, as said, should rational for rings of character zero is equivalent to, say, rational singularity. And so, so, if we have the kernel, and so, if we support this kernel, this is not rational. So, that means, support is, has, yeah, lower dimension, and then, so, yeah, R is open to rational type. And, and the converse is, converse is by Harlan and Mehta, Surivas, algebraic geometry of higher grade. So, yeah, it is too much for, to treat here, and perhaps I must confess that I haven't, yeah, understood. The proof, myself. So, but it, so, very, in some sense, strange that, so, if R is rational singularity, then it is necessarily open FPR type. And then, okay, of course, single one is less, yeah. And, of course, needless to say that single, so, the fact that single P is sufficient, then we can, in some sense, computable. Computable is a very important thing, I think. If, if we need infinitely many P, that, that is not computable, but single P, yeah, certainly computable, yeah, by computer, okay. And then, so, we come to section five. So, first, we must, yeah, define the notion of discrepancy for, yeah, take, so, this is normal. Yeah, it is sufficient to assume normal, but, yeah, for simplicity. And, so, these are some irreducibility by the co-dimension one variety. And, so, we call it, E is exceptional. So, we consider dimension A equal D. So, of course, E has dimension D minus one, but if dimension becomes strictly less than D minus one. And, for such E, and so, we assume that A is a cuboidal sign. If, so, I forgot to say that A is normal, canonical model. So, for some times, principle. So, in general, we can take index one cover, so that, yeah, K is R. Anyway, we have, then, yeah, a general point, D of A. So, let X1 to XZ, regular parameter system of O, X, Z. And, also, yeah, T, so, E is defined, T equals zero, or Y, Y equals zero, near W, Z. And, then, so, we can, so, we can find some, actually, we take differential form, but, yeah. And, then, we can put, sorry, I erased the long place. So, sorry, again, general point. So, zero. And, so, X1, XB, regular SOP, sorry. And, then, put, yeah. Okay. This is your unit. Okay. And, then, define XE equals R of S. Ah, sorry. This, we should put R here, because, so, this is generator of R's. And, so, we define, terminal singularity for every A of XE, yes, positive, to log terminal singularity, minus one, okay. So, canonical singularity, non-negative, and log canonical singularity, at most, minus one. And, so, it is easy to do by blowing up that if less than minus one for some key, then, we can find the infimum of AX. E for X, and we move X and E equal minus infinity. So, so, the minimal is called log discrepancy. This takes, yeah, either minus infinity or some things are, yeah, this is called, okay. Actually, I'm not myself very familiar to these guys, sorry for. It is, so, for example, in this situation, dimension two is very special. If A is terminal, then A is regular. Log terminal, ah, sorry, canonical, then, rational and golden standard. So, so, so-called famous. Log terminal is equivalent to, so, quotient singularity. The field is complex numbers. Quotient singularity means, so, so, of course, G is in SL, then, we come to this case. From dimension three, these things are totally different, and, so, yeah, for terminal singularities in dimension three was, yeah, classified by Shigatou Mori, and that was a very important result in our model theory. Okay. So, so, let me talk about, so, example of, so, this is a project variety and smooth project variety and ample impartial. So, canonical shape is, we have a, so, X, X is obtained by growing up, maximal ideal, and then, so, A is extension asset, and then, we have A, X, E, A plus one. So, for example, a, a famous, a third, and this is square Y, and there is, so, four generators, and I, I planned to calculate, in this case, yeah, this is, yeah, blow up resolution is, then, we can compute that. Growing up is, let's say, X and minus one over three, and, so, all, it's a very famous that all normal logic bring, so, same grouping, same grouping, normal same grouping, regular log terminal. Of course, this should be larger than minus one, but it is, actually, I contained a computation in my regimen, but unfortunately, the computation is mistaken. I, I made some confusion, and so, sometimes I'll make some revised version. Okay, so, now, so, my main point of today is the next one, so, so, A is in character zero, and if A is the regular type, sorry, of, yeah, this is dense is enough, but, actually, it turns out to be open, so, the same thing to say open or dense, and in this case, definitely dense. So, the essential part, the following then, is, so, this is very easy for finite, then, obtained by, this is one thing, and another thing is, in part, this is here. So, this is a very trivial fact. Okay, so, the theorem, for simplicity, yeah, we assume, so, X spec, and, assume that A is F here, of course, this is normal, and, so, then, there is some, this, A, and there should be something. So, this is splitting, and then, so, splitting means, and then, this defines, so, so, put L to be, so, the quotient single field. So, this defines, five defines, and then, so, since we are in algebraic situation, so, X, we can consider, this is, we can define, and so, since we have I to the inclusion, and, we should have, so, we want to say that, yeah, A is, if otherwise, should exist some E of AX, this is minus one, since, now, we have, by definition, so, this is by, formula, one over Q, and since this is less than minus one, so, this number will be bigger than Q, so, consider the generic point, so, so, at the generic point, of course, you know, the localing is a discrete variation ring, and, so, and then, so, if, if something has, is in, in M, Q, so, this value means, this thing is contained in M, so, multiplied by, and then, since this is principle, so, and, so, image, of base of N, so, this contained as, yeah, contradicts to the, this is a splitting, this is identity, so, so, the same thing will be, yeah, part one is in the same manner, so, because, there is some extra, and, okay, so, I, I thought of this proof, when I was in, over, in 1996, and, so, I, I was very glad that the next morning, Yanos Koral said to me, so, you did a very good job, thank you very much. So, yeah, I forgot to say that this is, yeah, contained in my paper with Hara, so, H-A-W-A, so, sorry.


This week. We're really pushing it. I'm gonna do 15 yoga push-ups.  Okay, so yoga push-ups start today Usually our feet are closer today. They're nice and wide like this imagine. There's a bar right here. I'm going underneath Hit underneath coming up and back Okay, so you're almost in this downward dog position Underneath push-up and back 15. Number of breathe. This one's really gonna burn your shoulders Chest, whole body Forearms. Good I'm gonna go straight to the squat jumps. Okay, so get set up in your normal squat position I'm getting low Deep breath going down Squat push-up.  Remember power push Push and let's five more Three Two good Again, things are getting tough breathe breathe through it. I want tuck plank So I'm getting in that tuck position or that plank position I should say From here jump up tuck and back Okay Bit of cardio bit of core going 20 reps Here for half a second Here for half a second when you get here keep it tight Straight and tight feel this in the legs, too especially after those squat jumps Five more burns Deep breath Go for that run. All right, so right after the strength I'm gonna take it few deep breaths Now I got a field if you're by a field You're going for one net one end Back to the other if you don't just imagine you do. I want you to go about 80% of your top speed Don't stop until you get back. Okay. Get that top speed or sorry 80% of your top speed Try to keep it all the way there quick turn all the way back. Let's go breathing breathing Good when you get here quick turn Get lower the ground push off I lost my Microphone, but it's still going all the way all the way again rest when you get there


One of the absolute privileges of our work is that it usually entails meeting all sorts of people and having close contact with a number of people, victims, civil society organization, authorities. After all, human rights work is all about the people and the bonds we create. So the foremost impact is definitely the inability to personally meet all these people. So being further away than usual limits our capacity to receive information and our capacity to advocate and pursue positive change. We have seen a continuance of attacks against journalists and human rights defenders. We focus on supporting the operation of the National Protection Mechanism and we keep participating in the meetings of the governing board where protection plans are decided for journalists and human rights defenders. I've been working here for less than three years but I'm now even more aware that the offices work in Mexico for over a decade building relationships and establishing its reputation as a reliable actor is paramount to ensure that despite confinement and isolation we're still able to carry out monitoring and advocacy activities and being part of a team that earns such prominences and our attention towards those who are most vulnerable and remind us that no one must be left behind.


To enhance your presentation, add images. Images can be photos that you took or you can find images on the internet. Flickr is a good place to find images. This video will show you how to find and save an image to use later in your presentation. Type in the address for Flickr. www.flickr.com Click explore to look for a photo. Type the kind of photo you are looking for in the search box. You need to type in more than one word to describe your photo. I will look for a photo of cleaning up garbage, for example. I type cleaning up garbage in the search box and then I push enter on the keyboard. Now I have a lot of options. But wait, your photo needs to be a creative commons photo in order to use it in your presentation. Under license, select creative commons only. Now I can choose from these photos. Click on the photo you want to use. Right click on the photo to save it. You need to save your photo in order to use it in your presentation. Click save as. Save your photo in your pictures folder and give it a name that you will remember. Click save. Now you have a photo saved to use in your presentation.


The Cavalcade of America, presented by Dupont. Another chapter in the Cavalcade of America in music presented by Dupont, featuring Don Voorhees and his distinguished orchestra. Music of the movies is the title of this evening's story, and what progress the movies have made. Better writing and music, superior direction and acting, and greater technical skills, all of which combine to give us the excellent entertainment provided by Motion Pictures today. Incidentally, one of the contributions of research chemistry to modern Motion Pictures is the improved film on which both picture and soundtrack are recorded. Dupont is an important source of the film used today. Thus, in entertainment as in other fields, Dupont chemists are providing better things for better living through chemistry. Motion Pictures from the days of the Nickelodeon up to the present day. Before we turn the calendar back, Don Voorhees and his orchestra are going to play for you a delightful song from the picture that is playing this very minute. Its title is Swing Time. Fred Astaire and Ginger Rogers are the stars, and the song by Jerome Kern is The Way You Look Tonight, with the noise of a silent film projector and the busy piano player who had to portray single-handed all of the thrills and heartthrobs in the perils of Pauline. It's being tied to the railroad tracks of the Sidley Villa one moment please while we change reel in the days of the Nickelodeon before picture theaters began to use small orchestras to accompany the action instead of a lone piano. The year 1915 brought forth a production that made Motion Picture history and remains today a landmark in the progress of the silver screen. The Birth of a Nation. A musical score was composed specially for the picture and furnished to the orchestras in each theater. And while the theme was brought out 14 years later, become the theme song for Amos and Andy. Do you recognize it? As pictures became more ambitious, the musical accompaniment became more important, portraying every mood of the picture. Let us turn the calendar quickly ahead to 1923 and the Motion Picture of the Clifford Wagon. Special music was composed to help you pick the California Gold Rush in 1849 around Stephen Foster's great song of that period, Oh, Susanna. Many of these theme songs, without the showing to the pictures, became big hits. Diane from 7th Heaven. Ramona from the picture Ramona. Laugh clown laugh from the picture of the same title. Angelomia from Street Angel. Jeanine from Lallertime. The first big theme song hit was Charmaine from the big parade. Remember it? The Singing Fool, which was the first all-music, all-talking picture. It was here that Al Jolson introduced the first song hit to be sung for screen. Silver, Brown, and Henderson's Sunny Boy. Metro Golden Mare produced the rogue song. This was important for it established the success of the operetta type of sound picture. Lawrence Tippert was the star and the audiences throughout the country thrilled to his singing of the rogue song with his delightful singing and acting in Love Me Tonight. Rogers and Hart wrote some grand song for this picture. Don Boris and his orchestra are going to play one of them. Isn't it romantic? Remember the cry from the old Nickelodeon music that's tuned as cheek to cheek from Irving Berlin's 1935 hit Top Half. The motion picture productions of Grand Opera, Operetta, musical comedies and reviews are now making use of all the great composers of our time as well as those of the past. The latest smash hit picture is Swing Time with music by Jerome Kern, from which Don Boris and the orchestra will play one of the outstanding hits, in the jungle of New Guinea, that remote island in the South Seas. The operators needed a heavy electric cable more than one-eighth of a mile long, but this cable had to have a different kind of per jacket. For the covering of natural rubber, ordinarily used, broke down quickly and completely under the direct rays of the tropical sun. So the American manufacturer who received this order turned to Duprene, Couponc's chloroprene rubber for the protective covering on this special cable. Thus we see man-made rubber, a creation of chemical research, invading a country where natural rubber is produced to do a job that nature's product could not perform. Here's another example of the usefulness of Duprene. Certain types of airplanes are built so that their fuel supply is carried in sections of the metal wing, and the seams must be caulked or sealed to prevent leakage of the all-important fuel. A resilient tape made of Duprene is employed to seal these tank joints because unlike ordinary rubber, it is stubbornly resistant to the ravages of gasoline or oil. You may never have occasion to see a New Guinea gold dredge or look inside the wing of an airliner, but you can be sure that Duprene is serving you somewhere in some way. Perhaps it is in the ignition wires of your automobile, in your oil burner or washing machine. In dozens of places, Duprene is rendering superior service because of its distinctive properties. Amazing to believe, this useful product of our modern chemical age is made by Dupont from limestone, coal, and salt. Even though the present cost of making it is quite high, Duprene offers assurance that if necessary, America can be independent rubber supplies from foreign lands. More important now, Duprene gives industry a new material from which to fashion articles of greater usefulness. In creating this chloroprene rubber, Dupont chemists have again made good their pledge, better things for better living through chemistry. This evening completes the summer series of the Cavalcade of America in Music. Next week at this same time, Dupont will resume its popular fall and winter series, re-enacting authentic episodes from the brim pageant of American history. The first of this dramatic series will tell the interesting story of that master showman, P.T. Barnum. Don Voris and his orchestra will supply the musical background.


Malaysian tourism has experienced significant growth over the past few years, as evidenced by the increasing number of domestic visitors and higher tourism expenditures. To better understand the trends in tourism, this study compares two forecasting methods, Fuzzy Time Series and Holt Winter, to predict the number of tourists arriving in Langkawi Island. The results show that Holt Winter has the lowest mean square error, MSE, indicating that it is the most accurate forecasting method. Using this method, the forecasted values for 2020 and 2021 are also provided. This article was offered by Nurfadiha Fuzzy, Nurul Shaheer Ahmadi, Nurhayati Shafi'i and others.


So come on, I'm WaN, terima tumu, istoe nalepi maesekema. Ginkilya wajcha, supotinga maa malala e hangarils. So thank you so much for coming through today. Every person here, we you kwa nidetuma, especially invite. Either through Nae-Mii, Amatru Ali, I'm an Iron boy. Ya ni, I really appreciate it. Kuchu kwa chai mienu, kuku jamu kuni support, kuku anamini. No, it's the humongers. I'm very grateful. Ya, but thank you so much for supporting our big day. I'm very, very happy. I'm very, very happy then. You guys don't get sad. Our Christian grand ambassadors. It's our first endorsement. I'd say as a couple. So for me it's up to celebrating. Next week I have another thing I will announce. My second partnership. So thank you so much.


I welcome you all for the series of lecture. We will start module 9 on Interferometry. In this module, we will be learning about basics of interferometry, that is, what is the interference phenomena, how it happens, what are the theories of flight and how do we obtain the fringes, what are the different shapes of fringes depending upon the type of workpiece and then we will be studying about different types of interferometers and then what are the building blocks of interferometers, what are the different elements by which interferometers are made and what are the various applications of interferometers and what are the various light sources used in the interferometers and then we will move on to the discussion on different types of interferometers like NPL, flatness interferometer, pitter NPL engage interferometer, laser interferometer and then we will discuss about some of the commercial interferometers. Now let us start module number 9, lecture 1. In this lecture 1, our discussion will be focused on the basics of interferometry. We will study about theories of flight and then how the interference phenomena occurs, what are the conditions suitable for the occurrence of interference, what type of fringes we obtain depending upon whether the workpiece is flat or cylindrical or spherical and then we will list some of the available interferometers and then we will study on construction of interferometers, what are the various elements used to build the interferometers and what are the applications also we will see and what are the sources, light sources used in interferometer. These things we will discuss in this first lecture. Now let us start the introduction on the interferometry. Now we will study what is the interference phenomena and how it occurs when two light waves interact with each other that means we have to consider two light waves and when they interact with each other the wave effect leads to a phenomena called interference of light. Instruments designed to measure the interference are known as interferometers. Now what are the applications of the interferometry? Application of interference is of utmost interest in metrology. Interference makes it possible to accurately compare surface geometry with the master. So they are used to compare the surface geometry with the reference surface which is an optical flat is normally used as a reference surface. Microscopic magnification enables micron level resolution and for carrying out inspection or calibration of masters and gauges. Using the interference phenomena it can calibrate the masters and gauges. Lasers are also increasingly being used in interferometers for precision measurement. That means nowadays laser based interferometers are available commercially which are used for metallurgical purposes. Now what are the theories of light? Now light is a form of energy. It is transferred from the source of light to the eye either by motion of material particles or by means of wave disturbance traveling through a medium. Now we have two theories of light. One is emission theory of light proposed by Newton in the year 1975. According to this emission theory a light source continuously emits tiny light weight particles called corpuscles in all directions. You can see here this diagram we have a light source here which is emitting a tiny light weight particles in all directions. These tiny particles move with the light velocity when these particles fall on the retina they produce the sensation of vision. You can see here we have the eye on which these tiny particles fall and then we get the sensation of light. Now there is another theory called the Huygens wave theory proposed by Huygens in 1679. According to this theory each point in a source of light sends out waves. You can see here the light moves in the form of sinusoidal wave like this sends out waves in all directions in hypothetical medium called ether. So for the moment of these waves some medium is called and ether is assumed to be the medium. Ether was assumed to be continuous medium which pervades all space. The existence of ether was assumed since the propagation of a wave motion requires some sort of medium. Now you can see a light wave here which is moving in this direction. So the distance between this any two consecutive points on this wave is termed as wavelength and it is denoted by lambda and this height is the amplitude of light. Now very commonly we used a term called wave front in the interference phenomena. So let us understand what is the wave front. So there are different kinds of wave fronts spherical wave front cylindrical wave front plane wave front depending upon the shape. Now when the light spreads from a point source o we can see here we have a point source o here and light is spread in the wave form in all direction. The light waves travel with the same velocity in all directions. All the waves they move at the same speed in all directions and they arrive simultaneously at all points laying on the sphere having o as sector. If we assume a sphere here with center o with diameter ca then all these waves arrive at this circumference at the same point. So then this front is known as spherical wave front. Now let us assume that the source of light and a form of linear shape o o dash. If this is the case then in all isotropic medium wave front takes the cylindrical shape. So we get the waves from this source we get the wave from this source and they will be moving in all directions. Then this front of waves is known as cylindrical wave front. Similarly we can assume a point source at an infinite distance. If a point or a linear source is based at infinity then the portion of the spherical or cylindrical wave front in a limited region is simply a plane and it is termed as plane wave front. Now let us try to understand Eugene's principle of propagation of wave front. Now each point on the wave front you can see here we have a point source 4 and we are getting the waves they move like this in all directions at the same velocity. This is one wave front, this is another wave front and the radius of this wave front is getting changed with time and here we have a wave front a b. Now each point on a wave front acts as a center of new disturbance. Now if we keep a screen x y in this path of light with a small hole yes is the pin hole. Okay so now the light will pass through this pin hole so this becomes this pin hole becomes another source of light and again we get light from this point yes and again we get the spherical wave front. Each point on the wave front now we consider yes on this wave front a b. Each point on the wave front acts as a center of new disturbance and emits its own set of spherical waves. You can see here yes is the new source of light and we get spherical wave front with this as the center emits its own set of spherical waves called secondary wavelets. So this a b is the primary wavelet and this c d is the secondary wavelet. These secondary wavelets travel in all directions with the velocity of light so long as they move in the same medium. As long as the medium here and here remains same these light waves move in the same velocity. The radius of these wavelets increases with time which we can observe here. The radius is changing with time. Now let us try to understand the interference phenomena of light. Now when two light waves superimpose now we can see here in this diagram we have a source of light. Yes is the source of light and we are getting the primary spherical waves and at this point we have a screen say this is a a b. a b is the screen which is having two pinholes s 1 and s 2. Yes is the pinhole and this is the source of light. s 1, s 2 two sufficiently closed closed pinholes or narrow slits. This s 1, s 2 they can be pinholes or they can be narrow slits. So here from s 1 we get secondary wavelets and from s 2 also we get secondary wavelets and they are getting superimposed in this region. When two light waves superimpose then the resultant amplitude or intensity in the region of superposition. This is so this is the region of superposition. The intensity of the resultant wave is different than the amplitude or intensity of individual waves. This modification the distribution of intensity in the region of superimposition superposition is called interference. Now the primary wavelet from s 1 will have some intensity and the secondary secondary wavelet from s 1 and secondary wavelet from s 2 they will have certain intensity. When they get superposition the intensity level changes. So this phenomena is known as interference. When the resultant amplitude is the sum of amplitudes due to the two waves the interference is known as constructive interference and when the resultant amplitude is equal to the difference of two amplitudes then the interference is known as destructive interference. Now in this diagram we can observe here the light intensity is increasing you can see here we have placed the screen xx here in the path of light. So this is screen xx and we can observe intensity distribution of the resultant wave. So when the two waves intensity of two waves added we get constructive interference and we get a bright region here. When the intensity of two waves they get subtracted we get a destructive interference and we get a dark band. So like this we get a fringe pattern we get bright area followed by dark area followed by pretty like this. So we get a pattern of fringes. Now the constructive interference and the destructive interference are detailed here. So we have a source of light oh this is the origin oh from where we are getting the two waves A is one light wave B is another light wave since the source is same they have same wavelength but amplitudes are different. See YA is having an amplitude of oh yeah wave A is having an amplitude of YA and wave B is having an amplitude of YB. When they combine to increase the if they are face that means you can see here the starting point is same and they are in face they are moving on the same side of this reference line we say these two waves YA and B are in the face then the resultant wave R will have an amplitude of YR. So this is a constructive interference. Now we will see the another case varying the two waves A and B are out of face by 180 degree even though they start at the same point they are out of face by 180 degree which is nothing but of a wavelength. So we have the wave A with amplitude YA and we have wave B which is out of face with respect to wave A by 180 degree and B is having the intensity YB. Now when they are out of face then the intensity of the resultant wave R reduces and the intensity of the resultant wave is YR. If YA and YB values are same then amplitude becomes 0 that means complete interference occurs and hence we get darkness. So that is what we observed here. So this is the constructive interference wherein we get bright area and this is the destructive interference where we get dark area because of complete interference. Now let us learn some more things about the Fringe formation. Now you can see here we have a screen YAB and this is the light source and from here we are getting the light waves. So these are primary wavelengths from light source L. So here we have a pinhole A and pinhole B. So from A we are getting secondary wavelengths from B also we are getting the secondary wavelengths. At some distance there is a screen which is placed parallel to the screen YAB. Now rays YA and B from the same source they have the same wavelength. At 0.01 on the screen the two rays are converged. It means the ray from A and ray from B they travel in this media and they are getting combined at point O1. Since A O1 is equal to B O1 the distance A O1 is equal to B O1 the two rays will arrive at O1 in phase and at this point we receive the highest illumination and hence we get a brighter spot here. Now let us consider what happens at point O2. At O2 distance A O2 is less than distance B O2 and if B O2 minus A O2 which is nothing but the optical path difference if this optical path difference is equal to odd number of wavelength that means 2n plus 1 times lambda by 2 then the waves will be 180 degree out of phase and the complete interference occurs and hence we get a dark spot at O2. So here complete interference occurs and we get dark spot. Now let us study what happens at O3. If B O3 this distance B O3 minus A O3 is equal to even number of off wavelength then the rays are again in phase and point O3 receives a maximum illumination hence we get a bright spot at point O3. So like this we are getting alternate bright dark bright spot dark spot like this. So this happens on both sides of O1. The process repeats both above and below O1 and alternate dark and bright light areas that is fringes are formed on this screen. Now again the fringe formation is shown schematically here. We have a monochromatic light source and then we have a screen with two slits. Say this is a slit S1 and this is a slit S2 and at some distance we have a screen is placed here and we are getting light waves secondary wavelengths from S1 as well as from S2. Now we can observe here in this region the two secondary wavelengths from S2 and S1 they are constructive in nature hence they are getting a bright spot here whereas in this area the two wavelengths from S1 and S2 they are out of phase one by 180 degree. So destructive interference happens and we get a dark band here. So like this alternate dark and dark band and light band we are getting and hence we get a fringe pattern as shown here. This is the light source and the screen with the two slits S1 and S2 and on this screen we are getting the fringe pattern. Now let us study about the conditions which are suitable for the interference of light. So when do we get the fringe pattern what are the conditions are suitable for getting the fringes so that we will study now. Now the separation between the two sources should be small. Now from this diagram we can understand that we have a light source S from which we are getting the primary wavelengths and in the screen AB we have two pinholes S1 and S2 separated by a distance 2D. This 2D should be very small. When 2D is small the fringe width that is D lambda by 2D where D is the distance between the screen AB and screen XY and 2D is the distance between S1 and S2. So when the 2D is small the fringe width D lambda by 2D is large and the fringes are separately visible. On the other hand if 2D is large fringe width will be small and due to the limited resolving power of human height the fringes will not be separately visible. They will be very close and will not be able to identify the or will not be able to count the fringes. Second condition is the distance D between the two sources and the screen should be large that means the distance between the screen AB which contains two slips and the screen XY on which we get the fringe pattern should be large. When D is large fringe width is large and hence they are separately and clearly visible and the third conditions for the observation of interference is the background should be dark so that we can clearly observe the fringes. The typical values for small D and capital D are shown here for 2D should be 0.5 millimeter or 1 millimeter or 2 millimeter so like this the 2D should be very small whereas the capital D should be of the order of 1 meter or 1.5 meter so that we can clearly observe the fringes. Now we can see here the fringe pattern now we get the light from the monochromatic source so part of the light will be reflected from the surface and part of the light is passed through the medium and then again it is reflected from this surface. Now we can have an optical flag for this reference surface and this can be the surface devotees whose surface is to be inspected. Now we have to combine these two waves and then when we combine we get the interference pattern subjected to this condition that means the reflected light says reflected light L1 and this is reflected light L2. The path difference there is a path difference between the path length of L1 and L2. The L1 distance is this much whereas the reflected light L2 will have an additional length of this so this is the path difference. If the optical path difference is equal to even number of wave length that is 2 lambda by 2 or 4 lambda by 2 or 6 lambda by 2 like this then we get a bright band. So this is the constructive interference between the two wave frames or two waves is constructive in nature and the amplitude will be maximum and we get a very bright spot as shown here. Now at this place so again say this is the so we consider at this point this is the light from the source and then we get a reflected light. The light is reflected from this surface and then this is the transmitted transmitted light and then the light is reflected so we get two lights here one L1 and L2. If the optical path difference between these two waves is equal to odd number of of wave length that is destructive interference happens and then the the amplitudes of the two waves they get subtracted and we get a very minimum light intensity at this point and then we get a dark band here. So in between again we have a monochromatic light source and reflected lights. If the path difference between the two wave frames is equal to even number of of wave length then we get a bright spot maximum intensity and bright spot. If the optical path difference is equal to odd number of of wave length then we get a minimum intensity and dark band appears. So like this we get a fringe pattern. So this type of fringe pattern we obtain if the surface to be inspected is flat. Now how we can use this phenomena for meteorological application. So the interference of light using the interference of light we can always test the surfaces for flatness. Now you can see here this is the nominally flat surface and this surface we need to inspect whether it is flat or not. For this so this is the arrangement we use an optical flat. So the bottom surface of this optical flat is the reference. So this these optical flats they are discs of stress free glass or quartz and sometimes both the surface or sometimes only one surface which is indicated by the arrow. So these surface are ground, lap and polished and they are available in different sizes from 25 millimeter diameter up to 300 millimeter diameter. Now using this optical flat and monopromatic light source they can get the fringe pattern and the fringe pattern will tell us about the flatness of the surface which is under test. Now what we have to do is we have to keep the surface which is to be inspected and on the surface plate and then we have to keep an optical flat of suitable diameter over the surface to be inspected. So normally there will be a small gap between the optical flat and the workpiece under test. At one point there will be a contact, at another point there will not be a contact because of maybe the flatness variation there will be a small angle between these two surface which is indicated by theta. Now you can see here we have a light source monopromatic light source and we get the light wave so which is shown here this s is the light source. So now it will be in the wave form. Now this light will fall on the bottom surface of the optical flat and part of the light is reflected back as shown here and part of the light will be transmitted so this is the transmitted light. So this light transmitted will fall on the surface to be inspected and then again it is reflected back and it takes this path. Now we have two light waves these two are combined at the eye. Now if the optical path difference you can see here this light incident light ray it is falling at point here and from here we have a reflected light and we have another light which will which part of the light is transmitted and it takes this path A, B, C and then it will combine at the eye. Now between these two light waves there is an optical difference of A, B, C. Now if this optical path difference is equal to odd number of off wavelength that is 1 lambda by 2 or 3 lambda by 2 or 5 lambda by 2 like that then the complete interference occurs and then a dark band will appear which we can observe here a dark band appears here. Now we will consider as another point where in again we have a light from a monochromatic source and the light is passed through the optical flap it is reflected from the point D and the reflected light will move in this direction and part of the light is transmitted and it will take this path D, E and from E it is reflected and if you take the this path and these two waves are combined here and again if the optical path difference is equal to odd number of off wavelength again a dark band will appear here. Now in between again there will be light source and light is reflected from this point and light is reflected from this point. Now let us assume that the optical path difference here is equal to even number of off wavelength that is lambda 2 lambda by 2 or 4 lambda by 2 or 6 lambda by 2 like that then the intensity of the resultant wave will be maximum and then we get a bright light band here so like this they get a fringe pattern so this shows an optical flat and these optical flats are commercially available in different sizes. Now this shows the formation of dark fringe and bright fringe so this is the workpiece to be inspected and this is the surface which is to be inspected for flatness and this is the optical flat and there is a wedge shaped air cushion between the optical flat and the work to be inspected. Now you can see here this is the monochromatic light source and we are getting the incident light ray and then it is getting reflected from this point and it is transmitted and again reflected. So, here we can observe the optical path difference between these two, between these two waves if it is equal to odd number of wavelength then the dark complete interference happens and dark fringe appears and you can observe that in this region the two waves they are out of phase and hence a destructive interference happens and at this point again we have the light source reflected light a second reflected light and in this region you can see the two waves they are constructive the interference is constructive in nature and we get a bright fringe the reason is the optical path difference here is equal to even number of wavelength. Now we must understand that if angle theta increases that means we have the workpiece to be inspected and then we have the optical flat placed on the surface to be inspected and this is the angle theta. If the angle theta increases fringes are brought closer together say we have for this particular value of theta say we get the fringes like this so this is the fringe pattern if this theta increases say so this is theta 1 and now it is theta 2 which is greater than theta 1 then the fringes will move closer together like this. So here you can see here the distance between two fringes is more for this theta 1 when theta is increased we are when theta 2 is greater than theta 1 the distance between the fringe is reduced and if theta is reduced surfaces become nearly parallel and fringe spacing increases. Now let us consider another case wherein we have theta 3 theta 3 which is smaller than theta 1 that means they are almost parallel then the fringe distance will be large like this fringes move away. Now what happens if the surface to be inspected and the optical flat surface if they are parallel and there is no and the surface to be inspected is almost flat then we do not get any fringes so this is the ideal case. Now these optical flats are manufactured with great care so that we get a very fine and flat surface. These optical flats should be handled with care the work surface and optical flat surfaces should be cleaned with a soft cloth before they are used and never we should never ring two optical flats together or if we ring them together then the separation becomes very very difficult and the flatness of optical flats will be a fraction of a millimeter micrometer. Now let us see how we can check the slip gauge surface for flatness you can see the slip gauge is placed on the flat surface and an optical flat is placed over the surface of the slip gauge which is to be inspected and the complete set that is the slip gauge an optical flat is placed in a chamber where we get monochromatic light source. Now we can see the set of slip gauge and the optical flat from the top surface from the top you can see how when they rotate the optical flat how the fringes pattern changes by rotating the optical flat we can always make the fringes to be parallel to one edge and when they angle between the surface for surface and optical flat surface changes you can see how the number of fringes change or the pitch of fringes changes. Now we can see how the flatness of an anvil surface of a micrometer can be tested by using interferometry the micrometer is held vertically and on the anvil the optical flat is placed and when the complete set is placed under a source of monochromatic light we can observe the fringe pattern now we can see the fringes are almost parallel and they are straight which indicates that the surface of the anvil is flat. Now depending upon the workpiece surface we get different shapes of fringes so that is assumed that when the work surface is flat for example the flat surface of a slip gauge then we get the fringe pattern like this and if the dark band and white band they are parallel to each other straight fringes we get when the surface is spherical when the surface is spherical like this then we get the fringe pattern like this concentrated circles we get and when the surface of the workpiece is cylindrical in nature then we get the fringe pattern like this. Now let us start the discussion on various types of interferometers so commercially the following types of interferometers are available michelson interferometer, diamond green interferometer, fabricated interferometer and the interferometers developed by national physical laboratory that is npl flatness interferometer which is used to check the flatness of gauge blocks and then pitter npl gauge interferometers are available and recently laser based interferometers are also available some of these types we will discuss in the next lecture. Now let us study what are the various elements used in interferometers now we can see here a beam splitter which splits the beam into two parts I can see here we have a beam splitter here incident beam and then the incident beam is made to fall on the splitter part of the incident light is reflected and part of the incident incident light is passed through the beam splitter now here we can see a clear glass and the light will easily pass through that and this clear glass plate will not act as a beam splitter and here when the glass plate is coated with silver part of the light is or full full light is reflected now when the glass plate is partially silver part of the incident ray is reflected and part of it will pass through the glass plate now you can see some of the commercially available beam splitters a glass plate which is partially silver and glass cubes are also available so incident light is partly allowed to pass and partly reflected now different light sources are used see what happens if we use daylight which consists of the different colors you can see here each color will have a different or a range of wavelength so we will not be able to get a very clear fringe so monochromatic light source are developed which will have a sharp wavelength mercury 198 cadmium source krypton sodium helium gas lasers are developed and these like monochromatic light sources are used in making of interferometers and different optical lenses are used to manipulate the path of the light collimating lenses are used and these lenses are used to produce parallel rays of light we can see here we have a light source we get light in this fashion and we want to make them parallel we want to get a parallel beam of light in such cases we use collimating lens and sometimes we have to make the parallel beam of light to fall on a particular point in such cases we go for condensing lenses these lenses gather and concentrate the light in a specified direction now moving mirror is sometimes used in the fabrication of interferometers and workpiece compartment is very essential wherein we get the monochromatic light source and there is a arrangement to keep the workpieces and optical flats so that we can observe the fringe patterns and a detector is mounted at the appropriate place of the interferometer so that we can observe the fringe pattern and we can count the number of fringes computers are used to measure the signal and measure the signal is distanced and sent to the computer for processing of the information optical flats are used to have the interference phenomena and fixed mirrors are also used to deviate the light scales and readings are also used for measurement purpose we can see here an optical flat made out of glass or quartz and here you can see an optical bench and we can see the stands for keeping the various elements like mirrors and then light sources beam deflectors detectors etc etc we can see here a lot of arrangement is provided to adjust the orientation of light source or the mirrors or lenses screws are provided by operating these screws so we can tilt the mirrors or light sources and we can also adjust the height of mirrors or adjust adjust the height of light sources etc etc now let us conclude the lecture one in this lecture we discussed about the different theories of light and the phenomenon of interference of light and how do we get the fringe pattern and what is the shape of fringe pattern we get depending upon the type of workpiece and then the different we listed the different interferometers like APL interferometer laser based interferometers etc and we also discussed about the various building blocks like mirror light sources lenses etc which are used to build the interferometers and we also discussed about the application of interferometer for flatness testing and what are different light sources used in interferometers so with this we will conclude this lecture we will continue the discussion on interferometry in the next lecture thank you