Assalamu alaikum dear students today we are in the lecture number 29 of the database management system course the course code is CS403. At the end of previous lecture we were discussing the where clause of this select statement. As I have told you before that select statement is one of the most used statement in the SQL because the objective of the select statement is to access data from the database. On the other hand select is used to perform different types of operations that we studied in the relational algebra for example join and also Cartesian product and some other forms of joins as well. So, that is why select is a very lengthy and very useful statement and to have a better understanding of the statement we are studying it in different parts and right now we are going to discuss the where clause of the select statement. As I told in the previous lecture the objective of the where clause is to select certain rows because we have discussed it that in the relational algebra we have got two operations one is the projection and other is selection. Both of these operations are performed by the select statement in the SQL. So, when you want to select certain columns then you use the select and at the same time to select certain rows again we use the select statement. The difference is when we need to select when we need to project certain columns we use the columns names in the select statement but when we want to select certain rows using the select statement we use the where clause of the select statement. So far we have used the select statement only for the projection purpose that is to select certain columns. Now we are going to study how do we select different rows from the database by using the select statement. Dear students, I have already told you that the purpose of the where clause is to select certain rows and its objective is to provide a condition that is applied on every row of the table and the rows that fulfill that satisfy the condition they are selected and they are displayed as a result of the select statement. So, let us start today's lecture by studying the format or the general structure of the where clause and by knowing the format of the where clause you will know different variants the way it can be used. This is the select statement that we have studied so far and now where and search condition. See again the where itself it has been given in the square brackets that means this is an optional clause but since the where has been written in capital it means if you are using it you have to give it as such and the search condition this thing is written in lower case. It means that you have to produce it and secondly you have to see that it is written in the angle brackets. It means that its explanation and its further details that follows. You see that what does this search condition mean that we have to give it after the where clause. You can see that there are many different possibilities that we can give in the search condition. This is not very difficult if you just concentrate on that and you concentrate on them one by one you will find that they are not difficult to understand one thing and secondly if you practice them it becomes even more easier. So, let us start with the explanation of this search condition. First thing is that you see the curly braces it means k you have to pick one of the options given in this curly braces and then next is not. What is in the square bracket it means this is optional that is predicate and predicate is given in the angle bracket it means k it needs further explanation. So one option is you give not and predicate then the second option is you again give another search condition this is the second option and then you see then again there is an optional thing and or and again you can combine not predicate and the same thing. It means that k you give not predicate then and or which is optional and or then you can give the same thing in not predicate search condition. You see in the beginning it is not predicate or search condition and after that it is an optional part that you can give it and or and then it is not predicate or search condition. Now in this you see that after this after the curly braces which you see in the square brackets comma dot dot n you can repeat this thing as many times as is required it means that your one search condition is after that you give and or and or and or search condition and then after that you give and or and or search condition and like that this way your whole zone will become a search condition so whenever you place this form of search condition after the wear clause so all that search condition will be evaluated as per the rule that we are going to discuss right now and you will get a single value and that will be true or false so if the condition is true that particular row will be selected and if it is false the row will not be selected. So see how predicate is explained predicate is first thing is expression and expression say it could be a single attribute name like we have so far we have studied different tables like for example student the student may say agar we simply give one attribute like s t id or s t name or s t address it can be considered as an expression. Yes, it's a lot of work that you have to do that you have to give an attribute name or you apply some operator on that you know let's say you can say c g p a star five c g p a multiplied by point zero five so this means that what is written here is expression expression could be simply the name of an attribute or it can be an expression expression means that combination of attribute any operation and again another attribute or some value. This example you will see in the future and you will understand it but the expression is something that you read in other languages for example when you read c in c you will see the expression. This is something that I think you are familiar with already expression about different operators comparison operators are equal to or not equal to or not equal to less than greater than these are different comparison operators. So you give one expression then a comparison operator and then the second expression example you can say where age is less than 20 now age is the name of the attribute less than is the operator and 20 is the value or what we do value that is also an example of the expression. Dear students like operator is used basically to compare those strings when you are not sure about the exact values. So if you want the exact comparison exact similarity exact same values exactly the same values in that case you will use the equal operator but if you are not sure what exactly I am looking for I am searching for a different thing that you want that all these things for example you say that Ahmad in the names of the students are Ahmad whether he is Bilal Ahmad or he is Sagheer Ahmad all these names I need their data or I need to find them. So the students may be living in different cities from all over the Pakistan or even from the other parts of the world as well. So you want to search them as only you want to perform certain operation on the people on the students living in Lahore and the thing is if you say that address is equal to Lahore and the thing is that your address is not Lahore so if you apply the equal to operator then you will not get the desired value because Lahore is not the same. Lahore is written in the address for example X, Y, Z and Dimal Lahore. Now in some addresses Lahore can be in the first position or somewhere in the middle or somewhere in the end. So in such situations like operator is used that you said that this is a string and this is somewhere in the beginning, in the middle, in the end. So for this you use like operator and remember about not that not is basically it is a unary operator. This means that after this you have to give an argument or a parameter. So whatever is the value of the expression after the not, not will simply flip it. This is written as not and an expression. Now if expression evaluates to true then not will do false. So overall the result will be false and likewise if you say not an expression or not an expression evaluates to false then not will simply do true. So not operator basically means that the expression after that if you want to upload it. Now once again what I have said many times I will tell you that same thing can be performed by multiple ways. And everyone will give you the right result. Now it is up to you. This means that what things you find easy or what things are good or if you know about its execution cost then you will also see that which things are more efficiently executed or the result will be given. So on this basis you will select that the different options that we have then how to perform any special activity. Once again the user will never tell you that you have to use this operator, where cross has to be used, select has to be used. This is only and only your job as a designer, as a developer. You said that this user as an example as you are making a database of your educational institute. So it tells you that your library says that okay I need author-wise all the books. That one author's name is all the books, then the other author's name is all the books like this. It also says that I need subject-wise books. That one subject, let's say we call it a database, all the books in the library have their list. The other one is data structures. So all the data structures have their list. This is how it came from, let's say C. So the user told you in plain language or explained to you. Now this is your job that you will define what particular form of statement of obviously select or SQL statement will be used here. So you need to practice a lot of this so that you can transform the requirement of the user into your required query. In this course, you will get assignments and exercises. You will have a lot of practice queries. And apart from this, you will get different practice queries from different practice queries. You practice as much as you can and you will understand this better. Let's move on. After that, we have one more option where we can give another search condition is that expression again not is optional between expression and expression. When your between job operator is used in that case when you have got a range of values and you want that if there is a value, then it will be selected. For example, you say that cgpa between 2.5 and 3. You say age between 20 and 25. So all the rows in which age attribute value is between 20 and 25 will be fulfilling that condition. Similarly, we have expression. Don't get confused with expression. For simplicity, you can understand that expression means attribute name. When you practice, you will understand that expression has different forms. For now, you can understand it as simple attribute name. So expression is not null. When you want to check null, then you won't say that select star from student where address is equal to operator. When we have started, we don't have to use equal to operator. Instead, we say null is used. Let's say address is null. We will practice further. Similarly, you have another expression not in sub-curie. Now, the expression. Between is used when you have got a certain range. From here to here, all the values. In that case, you do not have got the range. You have got the list of values. For example, you say where age in 5, 10, 13. See, between and 13. It means from 5 to 13, all the values will be considered. If you say in 5, 10, 13, it means exactly 5, 10, 13. Similarly, we have an expression. After that, all the comparison operators and sub-curies. Sub-curies is an important feature that we will read in the next section. Similarly, exist sub-curies. Again, we will read it. This was a general discussion. There was a general format of where clause. What are the forms of the search condition after where? After this, we will practice different forms. You will see how it is used. For example, here it says display all courses of the MCS program. See, this is our requirement. This is the simple requirement given by an end user. He said that I need all the courses of MCS. Now, you see, there is no select mentioned here. That you said to the user that select it every day. No name of the table mentioned. And it is not mentioned that you have to use the where clause here. Nothing is there. But as a developer, as a designer, now this is your job. How to transform this requirement into an SQL statement? The first thing is that you need to know about courses. You know that we have stored the data about courses in the course table. You know that. After that, you see that if the user had simply said that I need all the courses of the data. It means that you had to realize that there is no condition here. In fact, you have to show all the courses. So, in that case, you would have given the select statement from the course table without any particular condition. But at this time, if you see the requirement of the user, it is that we need the MCS courses. Now, here, the thing to think about is how will we identify the courses of the MCS program from the course table? You know that we have stored the data about the courses. Along with that, we have stored this thing about every course that what is the program name of this course? This means that the course that is related to the MCS program has the program name attribute in the course table and it will be written as MCS. And in the same way, if there is a course called BSS, then it will be written as BSS. If there is another course called BIT, then it will be written as BIT. So, you know that, in fact, you knew that we will know about the program. What is the program related to this course? We will know this from our program name attribute. Now, how will we put the condition on a course? This means that the condition will be on the program name attribute of which course table? The second thing is that what will be the condition? What will be the condition? Because the program name stores the name of the program. So, you have to check that where you have the value of the program name attribute, MCS. So, this means that the condition will be on the program name attribute. The value check will be MCS. Now, what will be the operator? What will be the operator between them? Which will be checked and then will be compared. Obviously, because we have to find MCS. We need the value of MCS. So, obviously, what your operator will use here will be equal to. This means that your search condition will be that the expression that we were reading normally comes on the left side. The expression will come in the name of the program name attribute. The operator will come in equal to. And the expression on the right side which is basically a value will be your MCS. What does this mean? This means that now this condition you have selected. By the way, you gave the list of attributes you gave the name of the table from. Where? Condition. You have put where. After that, you said that the program name is equal to let's say MCS. Now, what will happen? Now, every row, your course table is equal to 50 rows in the first row. Every row will be accessible. And from that row, the value of your program name attribute will be accessible. And that value will be placed in this condition. Now, when you wrote your SQL query in the program name, the value will be placed in that place. For example, in the first row, it was called BCS. Now, what will happen? You will have BCS on the left. Equal MCS. Now, it is obvious that these two strings are not equal to each other. So, when this condition is it will be false. When this condition is false, it will happen that this row which has been created against this condition this row will not be selected as a result of this select statement. It is not being qualified. Now, let's say that this particular course will not have this display. Now, this sequence will be applied in the second row. From the second row, the value of your program name attribute will be picked. Let's say that there is MCS. Now, your condition will be MCS equal MCS. Now, both the strings are equal to each other. Both the values are equal to each other. So, your equal operation the result of this will be true. Now, when true comes then this particular row it will qualify as an output of this select statement. Let's say that this particular row in which you have the search conditions are true it will be selected. And this process carries on. Till the end of the table. It will be applied on all rows. So, every row will be individually checked and on every row this condition will be true that row will be selected. This means that at the end of this statement or the execution of this statement as a result that row will be displayed and selected that this condition was true. Let's see how this condition is written. It is the same that select course code, course name and program name from course. This is what we have already studied. And what is happening in this or in this situation or in this situation there is no condition at all. It means that this particular form of the select statement it is displaying all the rows of this table. But, if we have said where and look where I have written where I have written where I have written where I have written where the case does not matter then we have said where program name equals MCS. Now this is your condition. Then I have said that as this is the condition that will be applied on all the rows and the rows that are fulfilling this condition will be selected or displayed. The example data we have according to this this is the output of this statement. You see that in this because you have given the name of three attributes in your select statement then in the output you are saying just three attributes course code, course name and program name. And then in this as many courses are coming from course code and course name the program name will be written in all of them MCS. Now as an output that does not qualify the given search condition all of them they satisfy they qualify the given search condition so this is the role of where clause through which you select some specific rows and display them or use them in some other operation. We will see more examples of this. Now I am giving you as a practice that you practice like this. Now I have shown you all the courses of our MCS of the output. But now you do that you display it like this that the output is written like this that the first row is written that analysis of algorithm whose code is CS504 is offered in MCS. In this way in a descriptive style all of you the first course which we have shown you try to transform that course in this kind of output. After a little effort it will work. As I explained to you before that not simply reverses the output of an expression you give an expression like you have given a program name is equal to MCS. Now as I have explained to you that on every row this condition will be applied and against every row its value will be true or false. Not simply reverses it. For which row for this particular record the value of this expression will be true and in which it will be false Now what is its utility what can be used for example if you look at this last query what was in it we said give us the courses of the MCS program if we have this requirement that we give other than MCS courses of all programs then what we will do is program name is equal to MCS and before this condition you will simply apply not. Now what will happen that the rows for which the search condition evaluates to false Now you have given the condition that program name is equal to MCS course of BCS so your condition is false if not true then the overall result of the search condition is true when it is true then this record will be displayed Similarly there is another record there was a course of it your condition will be BIT is equal to MCS obviously it is not equal if it is false then what will you do then this course will be displayed Similarly you see we have a course of MCS Now the condition becomes MCS equals MCS True but what will you do then it will be false then this row will not be displayed So overall we got the impact that all the courses which were other than MCS were selected or displayed so you could do this in any other way but this is also one approach of doing this let's see inverses of predicates value for example this is list of course name offered to programs other than MCS and this is the same query and here it is where not MCS and this is the same and if you look at the output in our example data that we have got in our example database you can see that 6 courses are displayed and your program name there is no record in which the program name attribute is written as MCS so that is the utility of the not operator and there are many ways to use it which we will see in the future and the same thing as we just did we said not and gave an expression we can do the same thing as we can say where the condition of your program name is not equal to we specifically used not equal to operator where operator was equal and that condition was given as an example and here it is that straight away you are using the not equal to operator that means here we have a course of let's say BCS so what will be the place of program name and the place of it will be BCS so what will be the condition BCS not equal to MCS obviously this is true because BCS is not equal to MCS when it is true then this will be displayed but when arrow comes with then it will come MCS not equal to MCS obviously MCS is not equal to MCS so this expression it will evaluate to false when it will evaluate false then this display will not be there so I hope you have understood that as we used not but using the equal operator how will this work and here we are using the straight away not equal to operator so you will get the same result which you can use the between operator the objective of this is when you have got a certain range of values you don't have to do precise values as we were saying here that it is MCS so if it is not normally we use this in the numeric or date for its numeric values and you say that if it falls in this range then it will be selected and if we give individual values we can use the between operator check the values in the range for example list the IDs of the students with course scores having marks between 70 and 80 i.e. students ID and course score display whose marks are between 70 and 80 how will we write this query if you see what we want what are the things we want we have course score and student ID you have seen this straight away now you will see and apart from this the condition when we said that the marks are between 70 and 80 so you will see what condition this is on your total marks which is on the national and final marks so first you will think what is this information that we have all this data so till now in the queries our examples are like this all the data is in one table but this is not necessary this is not required all the data is in one table we are going towards the flex so now we have examples in which all the data is in one table later we will do such queries where the data can come from 2 to 3 tables now you have seen we want course score student ID and total marks so you will immediately think what is this data in enrol this is the enrol table where you have the performance of each course so you will get all these things so what you have student ID and course score you have to display so it will come in your select list you will say select course score and student ID till now you have to display the student which marks are between 70 and 80 and your condition that is on your total marks okay now if your total marks are 70 71, 72, 80 then any of the students will have the same value but any of the students who have the same value you want to select so that means you will select between 70 and 80 so the procedure which is working exactly the same that is you have set the condition on your total marks so in every row your total marks will pick the value and it will be checked the end 80 if yes condition evaluates to true and that row is selected next row next row and like that so all the rows in which the total marks value is between 70 and 80 will be selected and in this case display let us see how we have written this query simple is total marks see that the total marks are not required the query is simply to give us course code but we add something so that we can ensure the query that we have executed it is working properly it is giving the correct result so from and roll and here total marks are between 70 and 80 that is the total marks will check the value in every row and in this row the value is between 70 and 80 whether it is 70 1, 2, 3, 4, 71, 72 or 79, 80 all the rows will be selected and it will be displayed if we look at the output this is the output of this statement we have three rows because we have three of the students whose marks are between 70 and 80 first student IDS1015 and course code CS616 total marks are 80 similarly the second is 75 and the third is 76 all the records we have all the values are between 70 and 80 how the between operator works the in operator now see what was between you have got a range continuously your values are running like 70, 71, 72, 73 and up to 80 but if you do not have a continuous list but rather you have got the certain values in which this is this is so you will say I have to select those rows so for that you use the in operator check in a list of values now give the course name of MCS and BCS programs now see that you have told certain values now the same thing that you want course names in the course table and you want MCS and BCS so in every course table the name of the program which has that particular course that means we will display the course name from the course table and we will apply the condition on the program name so how we will apply the condition again I had told you that the user will not tell you you have to define this in the course table so you have to select course name program name to make it sure that we are receiving the correct result if you omit it then it will be correct from course where program name in MCS, BCS now see one thing you should note that you have written the in operator in your parenthesis you have written MCS and BCS in the course why? because this is a string type or text type value so you know that if such a value is there then that we give in the course if this is the number or date we will give the special number without the course date treatment it varies now see that now we have in it we have got the 8 rows and if you see in the program name attribute in the program name attribute you can see either BCS or MCS apart from this you cannot see any other value or you can say that we have accessed all the courses of BCS and MCS and then the same query you can write this query select course name, program name, from course where program name equals MCS or program name is equal to BCS now in this you have washed the condition see one thing you should note that after where your condition will come that condition will finally be evaluated in true or false in the end you have to give one value now in it as you can see here you have search conditions program name is equal to MCS program name is equal to BCS now if you see both of these conditions then each one of them will have an answer obviously program name is equal to MCS it will be true or false if you see program name is equal to BCS it will be true or false and two true or false we cannot have from search condition ultimately we need to have just one true or false in that what we have more operator here you must have used it in your programming courses that is the role you are playing that you gave one condition program name is equal to MCS and now both of them are getting results your first condition is true or false you have one value for or operator second program name is equal to BCS true or false is evaluated for or now as far as or is concerned now it will be on one side it will be true or false now you know the behavior of or if one of them is true then it will be true now if you see it with the reference of this statement program name is MCS the first part will be true and the second part will be false if true or false is true then it will be true then MCS will be selected similarly the day when your program name is BCS then for every row your program name is equal to MCS will be false but program name is equal to BCS so if it is false or true then it will be true similarly your condition will give you every day in which program name is either MCS or BCS and first if we have the course of MBA then program name is equal to MCS false program name is equal to BCS false then it will not be true through this condition you can structure all the courses in which your program name is MCS or BCS like operator till now you have used equal to operator when you said program is equal to MCS or program is equal to BCS here you have used the equal to operator the equal to operator which is your exact match that the value here is equal to here exactly match like MCS is equal to MCS if MCS is on one side MCS is on the other side then you know that it will not be equal to each other because it is equal to the exact match now the situations where you want those rows which are approximately or they match with each other then you use the like operator for example the people of Lahore now the people of Lahore will be written in the address in the beginning, in the middle, in the end then it will be there in the name of those Ahmad is coming so in this Ahmad will be Hussain Ahmad will be Hassan Ahmad will be Mujtaba Ahmad will be like Ahmad Ahmad will be somewhere then you use the like operator in such situations that when you have got a general condition that such a thing is in a string then give me that daily with like another concept is the wild cards you must be familiar with wild cards you keep using them so from wild cards there is a character who generalizes it here you will have two wild cards one is underscore and one is percent symbol two underscore and percent symbol underscore you use it that as much as you gave underscore exactly the same characters if you say one underscore underscore is Ahmad that means in the second position in the first position but in the second position Ahmad is this is the purpose of underscore and as you say percent but the objective is as much as you want as much as you want if you say percent Ahmad that means whether Ahmad is in the beginning whether it is after one character after 2, after 3, after 4, after 10 after 20 wherever you want then you say percent Ahmad but you say underscore you say underscore, underscore, underscore and then Ahmad that means three characters and then Ahmad that means in the fourth position you say that percent Lahore that means in the place where anything is Lahore in the end and mind it if Lahore is in between you say percent Lahore that means in the beginning but in the end Lahore percent Lahore percent that means Lahore is anywhere in the beginning in between like example for example display the names and credit of the CS programs now in this it is simple to know if we already know that there are two programs of CS BCS and MCS then we can do this as we said before if program name in program name is equal to MCS or program name is equal to BCS but the thing is if we are not sure how many programs can be so for that we can condition that set course name course credits program name from course where program name like percent CS that means that we have percent CS before no matter what one, two, three, four, five at any position if program name is CS then that will be selected and see what is the difference between percent CS and underscore CS that is the difference if we say underscore CS that means if CS starts exactly from the second position like in MCS now M is the first character and CS is from the second position first if we say underscore CS and in any program name we have MACS now the thing is in MACS that starts from the fourth position so underscore the single underscore will not cover that but the percent will cover that that means when you want to be precise exact about the location of your given string then you use underscore and if you want to have the flexibility in that case you use percent and see how like will be placed attribute name like percent CS that means whether it is MCS, BCS, MACS CSA, CSB, CS3 any such program name in which there is CS in the end mind it if we have MCSA now MCSA MB is CS but since CS is not in the end then it will not cover that so see from this where you know tentatively the string is given then you use like and the role of percent and underscore you must have understood see the scope which we have already seen that all the programs that you have and why are you selected here because CS is there and the final part of the program name that is why all these days you have selected and you have also seen that the query that we need all the programs of CS you have read it in three ways through these and by combining and the third condition you have got this like whatever suits you whatever fits you in this given situation order by clause is used to sort the output in a certain order or on the value on the basis of the value of an attribute rather on the basis of values of one or more attributes whatever you want you especially need to use order by clause and it will be that your DBMS will put your output on the value of that attribute then your data table will have no difference this is only for the display for the output let us see sort the rows in a particular order fine this is the syntax rather format select you have given from table name where search condition order by rational but if you give order space by you will give them as such order expression and now I am telling you for the sake of simplicity you should understand from that expression that it is your attribute name after that there are two options ascending or descending you will give one of them and after that you are seeing if you want to repeat this and all this thing is optional Dear students for example of order by we will discuss that in the next lecture the objective of order by you understood that you have an output but that output is for ease of user for understanding you want to see it in ascending or descending order and one more thing is that it will be in your mind that as many times as you want to consider order by the data in your table there will be no difference it will remain the same which will be in your output the examples of order by and the rest of the SQL which we will discuss in the next lecture I would like to say goodbye Thank you


Welcome to the penultimate time slot of the conference. How's everybody doing? I'm pleased to say I'm finally over my altitude sickness, so I should be able to cope with this today. And I'm not sick in bed with 104 fever, which is great because the last time I was supposed to speak at Ruby Conf in Denver, that's what happened. So, praise be. Okay, so, welcome everybody, we're getting going. This is thinking in objects, so who am I? I am Josh Susser, I'm from San Francisco. I do the startup thing there. I'm CTO of a very small startup called Cognitive Health Innovations. We do mental health applications. I'm also on this podcast called Ruby Rogues. Who listens to Ruby Rogues here? Okay, who listens regularly? Who listens religiously? Who listens atheistically? Yeah, okay, great. I'm also an organizer of Golden Gate Ruby Conference. Yeah, okay, who's been to Golden Gate? Who's been to Goga Rooko? Oh, cool, great. So, I've been doing Ruby about seven years, and I've been doing object-oriented programming probably longer than some of you have been alive. I was doing small talk and OOP before it was cool, so that makes me an object hipster. Hold your heckling until the end of the talk, please. So, let's see. So, thinking in objects. This is me thinking about programming in objects, and you can tell I'm thinking because I'm wearing the thinking cap. You may not have a fabulous thinking cap, so I'm here to tell you how you can think in objects better, and write better Ruby programs, and be happier. So, I was very lucky to get to work with Alan Kay for a while, and Alan Kay, he's the guy who invented the term object-oriented programming. He was the creator of the small talk programming language, and he invented the whole concept of the personal computer. He called it the Dynabook. So, smart guy, and he's often quoted as saying something he actually said, which is that perspective is worth 80 IQ points. Now, 80 points of IQ is a big difference.  I mean, really. So, let's look at an example. So, don't shout it out, but raise your hand if you recognize what's on the next slide, and you're only gonna have a couple seconds. Okay, now who recognizes it? So, same thing, just a difference in perspective. There we go. This talk is about the conceptual foundation of object-oriented programming. I wanna give you a vocabulary for thinking about OOP so that you can write better code and be happier while you're doing it. This isn't a how-to talk. There's not gonna be a lot of how to do stuff, but there will be plenty of definitions. So, let's get on with the definitions. So, there's a lot of different kinds of object-oriented programming languages. If there were only one kind, there would only be one language, right? But they have, even though they have a lot of differences, they have a lot of things in common, otherwise they wouldn't be OOP languages. So, I'm gonna go over what those things are, and then we'll get into them. So, objects are things that have identity, behavior, and state. And then once you have objects, you get an object-oriented programming language by objects with encapsulation, polymorphism, and inheritance. So, that's basically my talk. Are there any questions? Okay, okay. I have a little more time to fill, so we'll continue on. Right, so, the punchline is, it's all about messages. When Alan Kay and the Smalltalk team at the Learning Research Group were putting together Smalltalk, they wanted to put together something that let them build computers that worked the way cells in your body worked. And cells in your body, cells are actually very powerful computational entities. What goes on in a cell is very difficult for us to simulate computationally. And the way cells interact, a cell, one cell in my finger, can't tell what's going on inside another cell, and it can't make it do something. The way they interact is, one cell will send a chemical message to the other cell, and then that cell receives the message and does something in response that it knows how to do. And when all goes well, you have a happy life. The object-oriented programming is built so that we can have computer software that operates the same way, which is really the only way that they could think of how to build really large, sophisticated, complicated programs that worked. But we're not talking about Smalltalk, we're talking about Ruby, which is something people have to remind me now and then. So, what about Ruby? Okay, who's seen a slide like this before? I have a picture like this before. This is like something that's the night of drunken hell from a grad student. Who puts these things together? This is the history of the lineage of programming languages and how they all interact. It's really hard to see because it's so small to fit everything on there, but in the pink circle over there is Ruby. And if you look really close, you can see that Ruby comes from Python. And Sather and Eiffel, and there's like Smalltalk way back in there. But, so, you know, I think the only thing these charts all have in common is they're all wrong. Okay, so I flew out here on Wednesday. I usually hate flying. I used to love it when I was a kid, but then the seats got smaller as I got bigger. And so now I kind of dread flying, but I got a really nice treat on the flight out here because I got the best person in the world sitting next to me on the flight. So I got the opportunity to show a couple of my slides to Matt's and got his comment on it, and he said, well, yeah, I got the def keyword from Python. But there's a couple of things for Eiffel and things like that. But really, he agreed with the slide when you next, which is the real lineage of Ruby. And so Ruby is like Smalltalk and Lisp got together and have a love child, and then went off to work and hired Pearl to be the nanny to raise it. So you have object-oriented programming, functional programming, and system scripting, and they all fit together beautifully in this really rich, flexible language. It gives you a lot of expressiveness, a lot of power, flexibility, but it can also lead to a lot of confusion and complications as programming styles collide. So OOP is a different kind of programming style from procedural, functional, etc. I've seen a lot of code that was nominally Ruby, but was actually Java or C or PHP. I mean, okay, who's seen that? Who's written that? Okay, thank you. Okay, so that's enough intro. Onward for definitions. The... So this is what I showed before. Objects are identity, behavior, and states. So let's start drilling into those things. Now, identity is the quality of existential uniqueness. Just because two things have the same value doesn't mean they're the same thing. I have a very good friend who's a twin and he is not his brother. Okay, so I'm gonna try a little analogy here. Who here knows what the Internet is? Okay, who here is a web programmer? Okay, good number of people. When you open your browser and you take a look at a web page like GitHub, there's some communication that happens between your computer and the GitHub server. The way that those things find each other to talk to each other is they use the domain name system to look up their addresses. So we have... We can look up the address for github.com and that comes back as a 32-bit number. We have a dotted quad notation here and that's a unique address for that server. There's no other server anywhere on the planet that has that same address so that means it has a unique identity. Now, we're gonna ignore things like firewalls and load balancers and reverse caching proxies because they don't really matter for the point I'm trying to make here. There's a lot of stuff in object-oriented programming as well, so skipping right along. So we have this unique address. We have the same kind of thing in Ruby. In Ruby, we have an object. Say we have a string foo and we can ask that object for its object ID, which gives us back a unique number that uniquely identifies that object. We can have another string foo and we ask it for its object ID and it's a different number. In Ruby, those numbers have to be different because those numbers represent the addresses in memory. So it's like a pointer in C. So they're always gonna be different. So we have two strings. They have the same value. They're spelled the same. Everything about them is the same. S, double equals T. But if you use equal question mark, that compares the identity of the objects and they're different. So we can say we have S and T and we shovel D onto the end of S to get food. T is still foo. So that's about it for identity. What about behavior? We have a string, we can ask it for its methods. Methods are the behaviors that the object knows how to do. So we can ask foo for its methods and we can see it can compare itself and tell how long it is and reverse itself in a number of other things. And you can create your own behaviors by defining new methods. So here we have a numeric type and we're telling it how to figure out if it's even or if it's odd. Okay, so, you know, we've now covered identity and behavior. And, you know, so that's where we're at. But we haven't built an object yet, but what we have is something that's still pretty interesting. We have things that I'm gonna call uniques here. These are different from singletons. These are things that do not have any internal state. There's nothing inside them, they're atoms. You can't cut, you can't chop them down in any smaller pieces, but they still have, you know, they're still interesting things and they have some distinct properties. So, and we deal with those things every day when we program in Ruby. So we have nil and true and false and use those all the time. We also have symbols that have spellings which make them different and they're certainly not singletons because there's many different instances of class symbol. But once you have a symbol, you never look inside it, the way that it's spelled never matters. It's just its identity that's the interesting thing about it. You can also create your own uniques. Okay, so who got to see Ben Ornstein's talk the other day about refactoring? That was a great talk. If you haven't seen it, you should go watch the video. So Ben showed the null object pattern to say, okay, here's how you create an object that represents somebody using the system who isn't logged in. And he created a whole class around that. So you only need one of these things and you're never going to have, you aren't going to have like 800 of these unique objects representing the same person, right? Or different people because they all behave the same. They're not logged in. They can only do one thing. So here's how you create one of those things without having to define a new class. You just say, I'm going to make an instance of object. There's no instance variables in it. It can't really do much of anything. So I'll start putting some methods on it and you can use the, you know, define a method on guest and you just give it a name. So it doesn't have to look in an instance variable to find out what its name is because it's always going to be the same. So we have an object that just because it has a unique identity is interesting and can do a lot for you. Okay. But it would be really hard to write programs that did useful things if all we had was uniques. So we need some state. State is the data that's contained within an object that lets it fulfill what it has to do to do the behaviors. So data is always in service of behavior. So here we have, we have foo, we shovel a D under the end and then we can ask how long it is so the object has to maintain some data for whatever its content so when you ask it how long it is, it knows.  You can't look inside it. So even though state exists from the outside, it looks just the same as behavior. Another way to look at that is we call that uniform access. Okay. So we have a person class and we want to ask it for its name. We have a, we can create that as an accessor and just say, okay, we have a first name. That's equivalent to defining a method with an instance variable in it that returns that first name value. But what if we aren't storing the names as a first name and a last name? Some people have middle names, suffixes, prefixes, whatever. What if we store it just as a full name? And the way we get back the first name is by splitting off the first word. If we have different classes of users, they can store their names in different ways and as long as they return them when you ask for a first name, you're good. The one way of, so uniform access means that asking for data looks just like invoking an operation on the object. Something that doesn't do this is JavaScript. JavaScript does not have uniform access. If you ask for the value of a property, it looks different from calling an accessor function. The first line is a property. The second line is a function. That means that if you want to change how names are stored, you have to go and hit every call site that accesses that property and turn it into a call in a function.  Back to our analogy about talking over the internet, talking to a web server. If we make a get request on index, we don't know if Nginx is just returning an HTML file that's stored on disk or if there's a Rails application where an action and a controller runs and renders an ERB template and returns the HTML and all we know is that we send a get request and we get back a response with some HTML in the body. The name for this property is encapsulation and I've already said pretty much everything I need to say about it, so here's a picture of a Q-daughter. Our next property is polymorphism. In Ruby, we call that duck typing because if it looks like a duck and quacks like a duck, it'll probably cook up nicely in an orange sauce. The other way that I like to think about this is protocols. Everybody works with protocols if you do web programming. You have TCP that lets you send packets over the internet. You have HTTP that lets browsers talk to web servers. A protocol is just a set of rules for entities to communicate with each other. Another way to think of that is it's an API and objects that have compatible APIs can be substituted for each other. That's the essence of polymorphism. I think that's a pretty easy concept to grasp so I'm not going to dwell on it too much. Let's review for a moment where we're at. We have objects and we've covered encapsulation and polymorphism. That gives us what is called an object-based language. I didn't make up the term, but I like it and I think it expresses something pretty interesting. The secret sauce to an object-based language is dynamic binding or late binding and what that means is that the code that executes when you invoke an operation on an object is determined at runtime, not at compile time. This is what lets us write extensible frameworks like Rails because the code in Rails that calls into your application was written long before you ever wrote the code in your application. There's no way that it could know about your code at runtime or at compile time. Moving up from an object-based language to get to object-oriented, we get inheritance. One more thing about object-based languages. There are a fair number of object-based languages out there. We tend not to program in them for general purpose programming because they're not very fun to program in for general case stuff. I think they tend to get used for things like simulation and other special purpose applications. They're also great for what we do because they're not as efficient because you don't have inheritance. Inheritance is great because it makes you much more efficient in writing code. You get to share code and that's what inheritance is basically about. We all know about dry, right? Don't repeat yourself, DRY. As opposed to wet for right every time. I haven't talked about classes much here, but classes are a way that different instances can share their implementation.  They all share the code through a construct called a class. Classes can share their implementations by saying, okay, I have a super class and then I want something that's a little more specialized than that, so it'll be just like that, but I'll add some differences. I think everyone's probably familiar with that concept, but there are two types of inheritance to think about. One is inheritance of implementation and the other is inheritance of type. Given that Ruby is dynamically typed, inheritance of type doesn't really help us all that much. If you're working in a language like C++ or Java where you have to worry about a static type system, type inheritance becomes much more important. But the point of type inheritance is largely to make the compiler happy so that your code can be better performant. It can have better performance. There's a lot that compilers can do using the information and type systems to make the code run fast. But Ruby approaches things differently, so type inheritance isn't as useful for us. The thing that I want to stress about inheritance is that deep inheritance hierarchies are dangerous. You should try and avoid them. Sarah May said this morning that good code has few dependencies and she showed a few examples of that, including the law of Demeter and how that works. The thing to keep in mind with inheritance is that if you have a deep inheritance hierarchy your subclass is dependent not only on your immediate superclass, but all of your ancestry because any one of those things that changes, any of your ancestors that changes can directly affect you. So you're coupled to all of your superclasses all the way up the line. So it's best to avoid that. And you want to use... I prefer to use composition. So let me take a slight detour here and I want to talk about delegation. So how many people have heard the term delegation? And how many people are confused about what it means? Okay, so delegation gets used to mean two different things depending on which camp you're in or what language you work in usually. Most people who work in Ruby use delegation to mean something that happens in composition. So rather than inheriting from an array you create a new class that contains an array within it and then it composes itself with the array and can take on some behavior of the array by using the array within it. There's a great library in Ruby called Forwardable that makes that really easy to do. Rails has some built-in stuff for doing that within Active Record Models as well. The other type of delegation is used with people who talk about prototypical object languages. JavaScript is one of these languages so is self and in those cases delegation actually means inheritance. So in 1987 at UPSLA there was a really great paper presented by Lynn Andrea Stein called Delegation is Inheritance and she proved mathematically that prototypes and delegation was exactly the same as classes and inheritance and settled a war that had been brewing in the object oriented world for years. And then the next year a paper came out by a gang of people called Treaty of Orlando that made some really interesting and basically covered the whole landscape of different kinds of object languages and what their properties were and if you're nerding out on object languages that's a great paper to read. But that's not the kind of delegation that we usually talk about in Ruby but I just wanted to clarify that. So we've gone through we have objects encapsulation polymorphism and inheritance. But as I said the punchline is it's all about messages. If you can think about objects interacting by sending messages that's when things start to get interesting. So I'm going to show you a bit about how this works and make some recommendations. So there's a mistake that gets made by pretty much everyone who uses Ruby including people on the Ruby core team. And that's that when you invoke an operation on an object they call it calling a method which is wrong. So the real way to think about that is you're sending a message. Why aren't you calling a method? Well, if you're doing delegation when you invoke an operation on an object the real thing you're doing might not even be in that object. It might be in a different object that it's delegating the behavior to. Or even more nasty there might not be a method there at all you might have a method missing handler that figures out what to do when it gets that call on it. So the best way to think about that is that you're sending a message. So however there is actually a way in Ruby that you can call a method directly. I do not recommend this but I will show you how it's done. We have a yeah laugh Eric. So we have a person class and person has a name and then we have an employee subclass and when you print out the employee directory you want to know not just what their name is but their title so the name gets reported as name comma job title. But when you send that person that employee a letter you probably don't want to put you know, joshcesser comma cto on the front of the envelope so you want to address it to just the bare name which means you want to get the name as answered by the person class. So here you can if you look in that recipient name method the first thing we do is we pull the method out of the person class bind it to the recipient object and then we can call it and that's how you call a method in Ruby but don't ever do it or James Edward Gray the second will come to your house and hurt you. So let's all take the pledge from now on when you invoke an operation on an object we call it sending a message thank you very much. The next thing is use objects. Ruby is an object oriented language so use the objects. There's an amazingly rich set of core objects and fundamental data types in Ruby and that's part of what makes Ruby great to use but how does it go? A difference in quantity eventually becomes a difference in quality is the kind of thing that you can get away with doing it a little bit but when it starts to dominate your program design you're not doing object oriented programming anymore. I've seen programs written in Ruby that worked by passing around hashes and one of the keys of the hash was type and that type was a name for the kind of hash it was and each method that received that hash would do a case statement switching on that type and running a different piece of code so they basically reimplemented polymorphism that ran several orders of magnitude slower than the way the Ruby VM knows how to do it. Did people see Koichi's talk where he talked about optimizing the virtual machine internals and method dispatch? You see how much effort VM implementers put into optimizing these operations. You are not going to do better than that so let the VM do it for you. We have these data types. We have blocks and lamb dubs. We have arrays. We have symbols. There's a lot of really cool stuff that's built into Ruby and you can build programs out of those things exclusively but they will not be object oriented. You want to use objects. My friend Cory Haynes said Reification unites behavior with data and he uses the term behavior attractors. When you draw a circle around all of your data and behavior it's really easy to know where to put more. It goes inside the circle. That example I talked about with the hashes and the type keys the behavior in the program for how to interact with those data types was scattered all over the program so if you change something about that data structure you had to go find all of the code throughout the entire system and look and see if that would be affected. That's one of the primary advantages of object-oriented programming is everything is contained within the cell membrane. You know where it is. You know what you have to touch. Here's a few different variations of how you can do something in Ruby. In ActiveRecord we have validations. You have a person and they might have a phone number and you want to make sure that they do have a phone number so you can say validates presence of phone. I made the name short so they would fit on the slide here. Ordinarily I'm more wordy in my code. But maybe the user could have more than one phone. You want to make sure that they have either a home phone or a cell phone so you can use a block to do that. That's pretty easy. But maybe there's like a whole bunch of different ways that they could have a phone. They could have an emergency contact person who has a phone number and you probably don't want to shove all of that logic into a block right there in one line so you create a method and then you use the validate has a phone method to do that for you. But then you say, okay well I don't just have users I have companies that have phone numbers so I want to extract that code from that class and put it in its own class so I now have a has a phone object that does that validation for me. There's this whole progression of how you can scale from using very simple procedural type stuff into using more object-oriented patterns that will help you and make your life easier. So you don't have to go crazy and use a hammer everywhere but you want to make sure you don't get stuck using a screwdriver all the time instead as well. Well, I'll talk about a strained metaphor. Okay, moving right along. Polymorphism. Polymorphism is probably the thing I like most about object-oriented languages. It's it just, it provides a lot of flexibility and it's really how we think about stuff in the world. It's just, we don't interact with everything as if it's the same thing, right? Different things are different. So, as I said before, method lookup is fast. You know, Koichi's talk showed that they put a lot of effort into optimizing these things. You should rely on polymorphism because not only is it going to make things faster, it's also going to help you organize your code a lot better. That's the thing that lets you put everything within the, in that circle so it all is in the same class. So let me show you an example. Last week, Ryan Big, you know, hand, big round of applause for Ryan Big. He's done a lot of stuff for like documenting Ruby and writing some good code. He's a smart guy and he knows Ruby but he got himself in trouble.  I'm like, and he tweeted a gist to something and I looked and I said, man, that person needs to be taught a lesson. He's like, yeah, it was me. So I took a look at it and I saw what I could do. So let's take a look at what happened here. So this is the code that he wrote. And the only thing wrong with this code is that it's not object oriented. This is procedural code. And so I did a little highlighting here so we can see a little more what's going on here. Can people actually see the code? Okay. So let me talk through it. But there's three levels of conditional nesting here. And that's like, you know, it smells really bad. The first thing that you check is you know, is there a product? And then, okay, do we have an admin and are we looking for deleted records? But there's definitely some cleanup that you can do here. I took a look at this for like two minutes and it became pretty clear what was going on. If you look in the first half of the method, you know, between if product and else, the blue else, all of the messages there are being sent to the product. And if there is no product, you're talking to the variant class. And then within each of those cases, there's very similar structures where you're saying, okay, if I'm an admin and looking for deleted, then I get to see all of the things, otherwise I just see the stuff that isn't deleted. So I looked at it, I said, okay, I can do a couple refactorings there. And this is what I ended up with. So I wanted to make it polymorphic, so it was the same message being sent to different classes of things. So I just inserted the aliases for those messages into the variant class. And then within the scope method, I collapsed that first conditional. If you go back here and look, we got if product, else something else. I said, okay, great, we're either using product or variant because in the second clause all the messages went to variant. And then there's only two cases left. You either want to show the deleted things or you don't. So this is the kind of thing that you can do when you learn how to think correctly in objects. And you're using polymorphism, you're using classes. We have objects. Identity, behavior, and state. We have object-oriented programming. Objects, encapsulation, polymorphism, inheritance. And the real secret sauce is messages. So one more thing. My Ruby Rogues podcast is a finalist for the podcast awards. And voting is going on now. It's going to be going on through November 15th. You can vote once a day, each of you. So vote early, vote often. Thank you very much.


Hi, everyone. Thank you very much for having me here today. So, I'll be discussing my use of transcribus in my PhD research, which is looking at legacies of race and slavery in the Encyclopedia Britannica, the first eight editions using a text-binding approach. So, first published in 1768, the Encyclopedia Britannica was published in Edinburgh, Scotland, and its approach to science, reason and the organisation of knowledge was a real exemplification of the Scottish Enlightenment and the wider Enlightenment movement. It was a reference publication with its first page claiming here as a dictionary of arts and science compiled upon a new plan. As well as entries in alphabetical order, as you would see in a modern-day dictionary, this new plan included longer essays to unify concepts that were in disparate parts across the Encyclopedia Britannica in the hopes that this would improve knowledge on certain topics, such as shipbuilding, anatomy, and other various topics that required quite a lot of information. It was published at a time where anti-slavery sentiment was increasing in Britain whilst the Atlantic slave trade was continuing, and that's what I'll be focusing on in my research. So, my research is looking at the first eight editions, which go from 1768 to 1860, and this presents the opportunity to research lots of different stages in the movement towards abolition, so the continuation of slave trade up to abolition and how attitudes changed after that. There's a huge amount of text to work with, and I'll give you a breakdown of exactly what's involved later on, but each edition of the Encyclopedia Britannica ranges from three editions at three volumes in the first edition up to 20 volumes in the final ones that I'm looking at in this period, so there is a huge amount of data to look at. The data that I'm using is from the National Library of Scotland's Data Foundry, which houses select digitised collections as data sets from the library collections as open access data collections that can be reused. The Encyclopedia Britannica data set is very comprehensive, and it's a really valuable resource, and I'm using some of the images that are included here. It has XML files, Met's metadata files at item level, and 155,000 images included. It also has OCR that has been generated from the in-house digitisation process, which, as you can imagine, contains a lot of areas. It was produced using industry standard tools, but that has brought me on to my research with Transcribus. Having good quality machine processable data is crucial in text mining, and the OCR just didn't quite cut it. Just to give you an idea of what I was looking at originally with the OCR and then how we can move from this with Transcribus. There were numerous errors, including misrecognition of the long F, so as someone looking at race and slavery, this was quite a big issue. OCR scripts can be written in code to fix these kind of errors, but it was including this and a lot of other errors. It was going to prove too much for my research. Here are some examples of the different issues. In green boxes we have correct recognitions in the original OCR, which can be found with the QR code here. There was also misrecognition of Flaves, which is not particularly helpful when you're trying to find certain words and text. Due to this, I decided to move on and use Transcribus to hopefully create a text with a lower character error rate. Here are a few other errors that appeared in the text. As you can see, it's quite difficult, so I was hoping that moving into Transcribus, I would be able to create a more usable dataset. I trained my own Transcribus model, and I hand transcribed 27 pages of text from the first volume of the first edition, so 1768, which totalled 17,898 words and 2,368 lines of text. As you can see here, this model turned a character error rate of 0.95% on the train set and 2.41% on the validation set, which I thought was quite a good result personally for a first attempt, so I was quite happy with the initial results. I will also note that I used the preset model for the layout analysis tool because it was my first foray to transcribe us, so I thought that was the safest way to go into it. After training, I decided to try out my model and see what the results were and if I could actually find some of the words that I would potentially be wanting in my research. I used the full text search function to see the return rate on words relevant to my research, so in this case, I looked up slave with an asterisk after it to look up instances of slaves, slaves and slavery, which are highlighted here. I know the writing is quite small, but you should be able to see by the yellow indications how many results I got back, so this was quite promising. I returned 13 instances of slaves, 10 instances of slavery and a couple of instances that weren't relevant such as slavering and slaver. I knew the text result wasn't going to be perfect, but due to the volume of text, I needed a process. My aim was to achieve a good enough low character error rate to use as a usable dataset. Although the text quality that I got back was significantly better than the original OCR that I'd looked at, the layout of the text posed a little bit of a challenge for transcribe us and we've obviously had a lot of discussion about layout difficulties earlier today, so hopefully we might have some answers in the future. Most pages in the Encyclopedia Britannica are laid out in two columns. This is a small section of one of the pages. As you read down the page, you read the left column first and then read down the right column. In some cases, the entire page was recognised as a text region as you can see here with the green box around it, which meant that although the text was split into separate sentences for each column, unfortunately this bled together all of the text, so looking at individual entries is quite difficult because they end up merged across the page. Although this is something that could be fixed manually on a small scale, for the scale of data that I'm working with, it's quite difficult, so I was looking into other options. In some cases, there were instances of lines not being split across columns, but you can see in the very top line of the entry has been recognised as one full sentence across the page, whereas it's actually two separate sentences, and the whole page was recognised as a text region. The image below shows a similar issue, although the columns have been split into two, so this is something that was being picked up in the layout analysis. To try and work around these issues, I tried training a baseline model and ran layout analysis on a sample of my text. This also returned mixed results, so I had high hopes for this, but I think due to the complete mismatch of some pages with quite tricky layouts, I just wasn't getting consistent results with anything that I tried, so if anyone has been working on text with columns and has any advice, I would really appreciate it. I've been in touch with the transcribers team and I've had some advice which I've tried to implement, but if anyone else has had a similar issue, please grab me after the talk, and I would love to have a chat. So what does this mean for my research? My transcribers model has helped to generate a corpse of text with a reasonable low character error rate, therefore providing a good quantity of data that I will actually be able to use in my research. As I have such a large quantity, my ability to process additions at scale is crucial, and transcribers offers the means to do this and to create a usable data set. Initially, to narrow my scope, I'll be looking at the first and seventh editions, which indicated here, so you can see the page count. These are the number of pages that include text, 2,659 pages, and then 17,047 pages, so as you can see, it is a very large quantity of text to work with. A massive advantage of digitally searching for keywords in the encyclopedia Britannica is that you can get results that you don't expect, and this is something that I'm really interested in in my research. There are entries for slavery, sugar, Jamaica. I could go on with a huge list of them where you might expect to see references of slavery, but it's where you get references that you don't expect that is really a point of interest for me. So this is just some very initial results looking at some keywords from the first edition, volume one. So I looked up instances of the use of slaves, and you can see here a list of entries that they appeared in. I also made a note of the context that these appeared in, so what the entry was talking about and found that of these, the majority of the entries were talking about slavery in the context of ancient Rome rather than discussing contemporary slavery. I've also done some close reading analysis of the slavery entries across editions and found that this is a recurring theme in the early editions, and it's only as the move towards abolition becomes stronger and legislation gets passed through in Britain that you see a more firm discussion of contemporary slavery in the sources. I'm still in very early stages of running my searches, so my apologies that I don't have a lot of nice visualisations like the last presentation, they were fantastic, but it's something that I'm hoping to improve and expand on as I move through my research. So by identifying where my selected keywords appear in the editions and how frequently I'm hoping to look at the networks created between the editions and across all of the entries. I'm also hoping to identify when new information is coming in into the editions as time progresses. With each new edition there is usually an expansion of the general word count of the encyclopedia Britannica, so I'm expecting to see an increased mention of certain keywords and hopefully identify that in relation to slavery. I'm also interested whether certain keywords are repeated across editions or if you start to see a decline in the use of certain words. So I'm kind of throwing keywords out in the wind and seeing what comes back and then trying to explore further from there using distant reading and more of a close analysis as well. I'm also hoping to use topic modelling to more closely examine entries relating to slavery and get an idea of the concepts that pop up in these entries quite frequently. I'm also using digital tools more widely, so I'm using our coding language. So I'll be exporting my text files and running them through scripts of code to do my text mining and create good visualisations that has a lot of flexibility for this. The use of digital research methods and resources opens up a lot of new avenues of research and hopefully this will help reveal the extent of legacies of race and slavery in the encyclopedia Britannica. I think there is so much information that is buried within these masses and masses of pages of text that it's quite difficult. It would be more than a lifetime's work to go over and identify this all by hand, so very fortunate that I have my text created by Transcribus to go on and do more detailed analysis that just wouldn't be feasible manually. I'm hoping to dig a lot further into this and will be sharing my findings with the Transcribus community as well. Thank you very much. Thank you very much for this presentation as well. I think it was very insightful. Any questions? We have time for two, I guess. Yeah, go ahead. The good thing is, if somebody is using the microphone, also the people joining virtually can hear everything. It's a red button. It's on. I was just wondering, Ash, because you had showed us the dates for each edition and there's quite a big gap isn't there from late 18th century to mid 19th century. Does the layout become more standardised for the later editions and could that reduce the headache that you've got in terms of recognising both columns? The short answer is no. Basically you have very complex layouts where most pages are just representing two columns of text. Many other pages have the longer essays included, so you have columns, headings inserted into the middle of pages and then further columns and it gets quite difficult trying to figure out exactly where you have to read which is intuitive to humans but more difficult to instill into machine learning. Shouldn't be impossible, so I'd like to hear any further solutions but it doesn't become easier with the late editions unfortunately, which is a shame. Right, we have a second question. Have you ever tried out the block segmentation? This is a special method for printed text and it should be able to work with the two columns. I don't think I have, so that's fantastic. It's something that I've been going around in circles with and I've tried asking a few different people. I feel it's a simple solution for your problem. Lovely, that's what I like to hear. It's a simple solution, that's great. It's in the layout analysis on top where you see sit a bit once, you open it and then you can select printed block segmentation and that's made for such problems. Lovely, I'll try that, thank you very much. Then we are happy that we could help. Thank you very much. Ash from the University of Edinburgh.


The Whistler. I'm the Whistler. And I know many things, for I walked by nights. I know many strange things. Hidden in the hearts of men and women were skipped into the shadows. Yes, I know the nameless terror of which they dare not speak. Tonight, transcribed, it's the Whistler's strange story, Lady in Waiting. Her own feelings didn't match the day at all. It was spring in Pentecostal, and the city had a way of catching the spirit of such occasions. Even Eddie Street and the ravaged Gizmo office building she was entering seemed to brighten up somehow. She paused in her flight of stairs, her eyes examining the directorate. A few minutes later, a secretary ushered her into attorney John Bradley's office. Good on, Miss Cowan. Thank you. You seem nervous. Is there something... It's just that I'm not sure you can help him, Mr. Bradley, and no one else knows what I'm about to tell you. No one else will, whether I handle the case or not. Now, does that make it easier? Yes, much. Well, it's quite simple in a way. In fact, I mean, you see, I was keeping company with someone a number of months ago, a very fine man. At least, I thought he was. Go on. I, uh, wasn't in love with him. But he was kind, thoughtful, and, uh, very much in love with me. I only tell you that so you'll understand. I do understand. Anyway, you wanted me to marry him. He said we'd go away, travel around the world for a few years. He kept talking about that. I, uh, I'm afraid that waiting for me to give him an answer is what kept him here. Well, the case isn't too unusual yet, Miss Collins.  It was Frank Hardison. Hardison? Not the one who... Yes. Believe me, I didn't know any more about that part of his life than anyone else. You never told me. And to meet him, Mr. Bradley, you'd never guess. He was a perfect gentleman. I'd never met anyone. You got 20 years, didn't you? Yes. I remember reading about it. Quite a list of charges. You dealt his guilt. Is that it? Want me to...? No, Mr. Bradley, that isn't it. Then? Well, every month in the mailbox at my apartment, I find an envelope filled with, uh, a currency. Large bills. That's a problem. You don't understand. I know that somehow Frank's sending that money. I want it stopped. I haven't touched the scent of it. I don't want to. But, um, I thought if a lawyer talked to him, explained to Frank that he's really doing more harm than good. Easy. I have a very responsible position, Mr. Bradley. It's the people I work with ever since. Of course.  I'll handle it. I'll drive up and see. Harder than tomorrow. Thank you. Uh, when will I know? I'll be terribly anxious. Could you meet me here about, say, 4 o'clock? No, I'm sorry. See, I'm with an advertising agency, Mr. Bradley. And, uh, we have an important meeting with a client this afternoon. And why not meet me after hours at the old throttle bar? 6 o'clock. Oh, I... And I'll have all the answers for you. I hope so. It's easier now, isn't it? Now that your problem is being shared, and somehow you feel a sense of confidence in attorney John Bradley. Riding home in a cab, you'll hope that it will go as simply as Bradley seems to anticipate. In front of the wreck for Don, you're suddenly aware that the cab has stopped, that the doorman is smiling at you, waiting for you to step out. And you were with us? Miss Jean? What? Oh, I'm sorry, baby. Dreaming again. Here, I'll drive them. Uh... Finds free day like this, man. You should leave that there. The doors get outside. Go for a walk through Golden Gate Park. Nothing like being outdoors. Dan, you're wonderful. How are you feeling? I'm all right. You think you should work like this, Dan? It's outdoors, doctor says. That's where I gotta be. Oh, just worried about you.  And, uh... The way you walked and acted the past few days, I guess you got things in your own mind. Dan, there should be more like you to notice. It's just a business worry, Dan. And I think it will all be cleared up. Tomorrow night. To Miss Collins. No more worries and an uninhibited future. Oh, Mr. Bradley, I'm so glad it went that way. Well, you're not drinking. Oh, I think you don't feel like it. I'm just happy that Frank didn't give you any trouble. But he understood. Oh, he did. Perfectly. And you're right. Partisan's a nice guy. But you still have a problem. Oh. Yourself. The way you've been brooding about all of it. Can I help you forget it some way? How? No. A good start might be a fake dinner as Julius is castled. You can look down at the city and tell it that Gene Collins isn't going to worry anymore about anything. I don't think you should. You can even pay for it, you like. I'll be your first charity. What do you mean? All that money that's accumulated. Henderson doesn't want it back. He hasn't made use for it. He should just forgive you to dust my fee and give the rest to your favorite charity. Oh. And my fee includes a fake dinner. Fair enough. Then I'll drive you straight home. Oh. Fair enough, Mr. Bradley. Right to the door. Mr. Bradley, you've been so wonderful. John. Oh, John. You've been very kind and I do appreciate it. Your favorite lawyer. Absolutely. Will I see you soon? Oh, well, I don't know. Gene. Yes. Get in trouble again right away, will you? I'll handle it. Oh, dear. Good night. I'll call you. Good night. Evening, Miss Gene. Hello, Dan. Well, the spring days kept alive into the night. You're happy, Miss Gene. Yes. Good night, Dan. Good night. Dan. There he is. Dan, you're not really very well, are you? Oh, no. Please, I want to know. If you had some money, try a bit of money. Is there any doctor that could help you? Miss Gene, there should be more. Ma'am, I'll not forget this. Tell me, Dan. I'll tell you this. A long time ago, I had all the care a man could have. It's no good. I'm just standing by the door now. Thank you, my turn. I'm sorry. Don't be. Forget it, Miss Gene. Really, I won't ever forget you. Good night, Dan. And right, Miss Gene. The spring day has kept alive into the night. You see a great deal of John Bradley in the days of follow-up, evenings of the theater, dining and dancing at the smart club. And you find yourself becoming extremely fond of him. And you look forward to each evening with increasing care. Then one evening you say good night to John in the lobby of your apartment building, and go upstairs. And if you're about to open your apartment door... Just a minute, Gene. I'm going in with you. Thank you. In time? No, partner, Gene. I don't like ways run in our business when I'm changing things a little. First, keep your checkbook. Write out the full amount of Frankie's donations. You can't ask me. No, I'm not asking. I'm telling you. I'm telling you something else. You'd better forget your boyfriend. Mr. Black, look.  You mustn't see him anymore. He might get hurt. Well, he's done nothing. Not yet. But I don't like you hanging around lawyers.  But... You can write that check out now to your favorite cherries. Joey Stark. In just a minute, the whistler will continue to write stories. All of us are proud of our hometowns and likely so. In this brief moment before we continue with our program, we'd like to offer a salute to one of our hometowns in America, Des Moines, Iowa. Its history dates back over 100 years to the time when Fort Des Moines was established at the junction of the Raccoon and Des Moines rivers to protect the rights of various Indian tribes in the prairie region. The city today, largest in the state and the state capital, is surrounded by rich coal fields which have contributed greatly to its industrial development. If you carry insurance on your homes, the chances are that your premiums wind up in Des Moines since over 150 insurance companies have offices in the city. And manufacturing is important here, too. Over a third of a billion dollars' worth of its products are sold throughout the world every year, such things as cosmetics, chemicals, medicines, wearing apparel, tools, machinery, food products, airplane parts and many others. It's a city of churches, over 200 of various denominations. A city of parks, over 1600 acres of natural beauty spots. A city of culture with many fine art institutions. The people of Des Moines are proud of their hometown and proud of the way Des Moines has taken its place in the building of America. And now, that too, the Whistler. It vanished in an instant, didn't it, the hope for a bright, clean future. Joey Stark, Frank Carterton's partner, won't believe that you don't know what was going on. Have no knowledge of the sort of deal that sent Frank to prison for 20 years. More than that, he insists that you stop seeing John Bratton. And for the next three days, you're afraid to even talk to him, even on the telephone. The terrible part is the realization that you love John, isn't it? That you don't want to give him up. An hour later, you'll find a note downstairs in the mailbox. Orders from Mr. Joey Stark to meet him tonight. You'll hurry downtown to your office. Hello, Jean. It's John. You shouldn't have come here. I'm saying right here until you talk to me, until you tell me what's wrong. Close the door. But it's nothing, John. Why have you refused to see me? Why aren't you in when I talk? I have the dormant at your apartment making excuses when I know he's lying. Jean, if it's something I've done... No, no, you haven't done anything. Someone called on me. A man. Thanks, pardon me. Oh? He won't believe that I don't know about Frank. It made Joey Stark. I think I need him tonight. No, you're not. I'll meet him. Talk to him like I did with Frank. No. I'll be careful. Tell me where you were supposed to meet him. There's a note. I have it in my purse. It's all there. A hotel on LSD. When you get through here, go home. Wait for me to call. Don't open the door or talk to anyone. You will be caught. I'll handle it. Just wait until I call. Yes? Jean? John? Are you all right? Nothing to worry about. You sorry? Jean, this call goes to a switchboard, doesn't it? Yes. They better not talk anymore. I'll combine the morning before you go to work. Can I mean time? Yes. No matter what you might hear. Remember, there's nothing to worry about. Nothing at all. Not now. Jean, I've been so anxious. Who are you? Lieutenant Williams, I'm a friend. This is Sergeant Knowles. May we come in, Miss Cullen? I'm a friend. I don't understand. There's some things we don't understand, Miss Cullen. Well, until we talk it over. Come in. Sit down. That won't be necessary. I'll come right to the point, Miss Cullen. Tell me, is there... Is this your check? Oh, yes. Yes, it is. Where did you get it? We found it on the body of a man, tentatively identified as Joey Stark. The body? That's right. He was found late last night, brought into the morgue. Took us a little while, looked you up, but naturally, we're interested in learning the connection between you and this man. Now, this check, it's for quite a sum. Cullen, I don't know anything about it. Of course, I wrote the check, but... But what? Did you go to this man's hotel at any time last night? No. Didn't leave your apartment here, I suppose.  I was supposed to, but... Something happened to my mind. Mm-hmm. I know. Get that dorm and bring him up here. We'll see if he's got a memory.  Blackmail setup? No, no. I'll tell you, I just... Well... Expecting something? No. We'll read it, I'll send her away. No, I wouldn't do that, Miss Cullen. We'll both see who it is. Quite consistent. Then I go ahead, open the door. You can drive me to the office another time. Well, Johnny Bradley. What can I wear you? What are you doing here? You lost little time in calling a lawyer, Miss Cullen. And a good one, too. We'll cover this boy. Oh, come in, come in. Seeing how long he's been here... Oh, I work fast, Bradley. You know that. You told me all I want to know. Namely, that this is her check. Check? What about it? John, Miss... They found it on... And Joey starts body. Say, you folks thought we were familiar about Joey. You know him, too, Bradley? Maybe. Why shouldn't he have Miss Cullen's checks? He gave it to him as a... ...favor for someone. Really? Who? What difference does it make? None. If she didn't leave this apartment last night. And we'll know that in a minute. No, uh... Bring him right in, though. Well, what is called a... ...a machine, isn't it? I'll ask the questions, if you don't mind, then. You see, we're trying to find out if Miss Cullen's went to a certain address last night. Now, it's, uh... It's to her advantage for you to tell the truth. Isn't it? Did you notice her leave the building around, uh... Nine o'clock, like she says? I'm asking him, Miss Cullen. Well, Dan? Did she? Miss Cullen... ...never left the building at all last night. I was very late. I had seen her. Mm-hmm. Well, Miss Cullen, if I don't doubt you and your, uh... ...witness, I think a jury might. What do you mean? Dan Rogers, here. Yeah, I have a very good memory. Now, let me see, uh... ...convicted of perjury in 1929, weren't you that? What? Mm-hmm. Perl for good behavior. Brought back on the same charge in 1932, served five years. Yes, a nice, reliable witness. All right. So, what are you going to do? Same thing you do in my position, Bradley. I'm arresting your client, Miss Cullen, for suspicion of murder. You've got your murderer, Williams. But it isn't Miss Cullen. I did it. What? You heard me. I did it. I killed him. Oh, if you don't believe me, pick up the cab driver who drove me over there last night. You mean, out of all the cabs in San Francisco, you would like to... This number is 4211. I made a note of it. Get him. You remember me. We had trouble finding the address. You know what this means, Bradley? Yeah. I know what it means. Don't be half-right. Use Yusofi. For example, do you know how much of our land has been permanently ruined by erosion because of improper farming? Would you say 25 million acres? No, that's only half-right. Brush up on your conservation. Tell your I&E officer you want to study with the United States Armed Forces Institute. Yusofi. It's easy. It's simple. If you don't want to be half-right, use Yusofi. And now, back to the whistlers. It happened, hasn't it, Dee? All that you feared from that very first day when you decided that someone might be able to help you in your problem with Frank Harden. It clear, isn't it? Don Bradley murdered stuff for you. Confessed to Lieutenant Williams to save you. And now a cab driver is about to prove John's story. Yep. I drove him to the hotel about 9 p.m. That's right. Oh, God. Why did you let him stay? Did you wait while he went up to Stark's room, Bradley? No. He went up, though. Saw him walk through the lobby and start up the stairs. Pick up another fare and drove off. Nice. Is that all, Lieutenant? Yeah, you can go. How about you, too, Dan? Hey. Wait a minute. What's the matter? My other fare. Who's this man? Dan. Sure, the old guy here. He come walking out of the hotel as Mr. Bradley went in. Pass them. Oh, that's it, Bradley. Stark was dead when you got there, wasn't he? You lied because you thought that Miss Collins had killed him. No. You killed him. Wow. Wait a minute, Gene.  Dan, let me in. Don't worry about it, son. If they put me on trial, the jury better not stay out too long. Miss Gene, it wasn't just for you. It was for Frank, too. Frank? Frank Hardison put me on the door downstairs the day you moved in here. I left the envelopes and stopped leaving them. Whatever Frank said. You once said that someone else paid the medical attention for you. Won't you, Frank? Yes, Miss Gene. This Joey Stark was no friend of Frank. He was just a cheap hoodlum, blackmailing Miss Gene, but Dan, the killer man. Just to pay back her. It wasn't that, Mr. Bradley. The lieutenant here will be able to show you when he checks on all those aliases of mine. You'll find one of them was Hardison. Hardison? It's me. Frank Hardison is my boy. Miss Gene, he asked me to protect you. You were the one decent thing that ever happened to him. Now, a question. Do you know the story behind the phrase, I have not yet begun the fight? In 1778, John Paul Jones set sail for America in command of the old sea-battered ship Bonham-Richard. Off the English coast, he encountered the English 50-gun frigate Seraphus. In the furious battle which followed, half the crew of the Bonham-Richard were killed and most of the others threatened the mutiny. But despite the setbacks, when the English captain demanded his surrender, Jones answered, I have not yet begun the fight. A short time later, the English ship struck a flag and was boarded by the American crew as their own ship went down. This is by one of many interesting facts which can be found in the history of your United States Navy. Featured in tonight's transcribed story were Bill Foreman as the Whistler, John Stevenson, Vareen Puddle, Sir Butterfield, Jim Nutter, Bill Boucher and Jack Moyle. The Whistler, directed by Gordon T. Hughes with music by Wilbur Hatch, is written in for you by Joel Malone and transmitted overseas by the Armed Forces Radio Service. The Whistler was entirely fictional and all characters portrayed on the Whistler are also fictional. Any similarities of names or resemblances to persons living or dead is purely coincidental. This is George Walt speaking and reminding you to listen again next week for another strange tale by the Whistler.