a tiny bit on my end in case you can hear some baby noise. Excellent. Good afternoon, everyone. Thanks for joining us this afternoon. I'm Dermot McDonnell, I'm a former associate at the UK Data Service and here today to take a look at some simple social network analysis examples, mainly focusing on how we can use Python to achieve that task. So for today, it's not particularly a completely introduction to Python. So it's not for people who have never, ever used Python before, but you don't need to be good at it to learn something today. It's, I don't cover the complete basics of social network analysis either, but I do cover some of the essential concepts. So if you're thinking about learning Python from scratch, then we've got some different materials and there's some excellent open source materials also. And if you're interested in social network analysis, the essential concepts and kind of theories and how those are linked, that we've just recently concluded a webinar series with myself and a colleague, Julia. And so you can consult that as well.  And I'm gonna show you how you can combine the two together. So this is a coding demonstration. So not only can you follow what I'm gonna be doing, but you can also execute the code yourself as we go through the exercise today. So a colleague, Julia Kazmier, will have posted the link to the Jupyter notebooks that we're using today. A Jupyter notebook is simply an electronic notebook similar to your paper versions, and in which you can write programming code. So today we're using Python. If you were an or user, for example, you could use Jupyter notebooks for that. C sharp, Julia, lots of different programming languages can be written using a Jupyter notebook. So if you want, you can just purely follow what I'm doing for about the next 30 minutes. And then we'll have some questions at the end. Or if you can possibly, you can execute the code in real time with myself also. So today we're gonna look at the first webinar and some of the example code we wrote for that. So we've got a Jupyter notebook here, the fundamentals of social network analysis. And the way we do it is instead of trying to run things on our laptops and how to install Python and install packages, and that's kind of quite finicky, technical aspect, instead we can all do this online. So you can click the link that Julia posted in the YouTube chat. It's the same link as I'm gonna select just now. So basically this launches our Jupyter notebook to the cloud basically, and that allows us to run all the code without having to install anything on our own machines. And not only is this quite a useful teaching tool for us here at the data service, but it's also quite useful for you as an analyst as well. It allows you to share your code with others and allow others to make changes to your code without overriding what you've done yourself. So it's quite a good way of doing things. So basically once Binder has launched, so that's kind of the cloud service we're using, you can see a folder like this. So now basically those Jupyter notebooks I showed you before are basically now interactive. So if I click on the fundamentals Jupyter notebook and then that launch basically the programming code we're going to use today, excellent. So this contains a lot more material than we're gonna go through today. So I used some of this in the webinar series from about two months ago. So we're just gonna focus on one aspect today. So if you, basically it's like any kind of webpage just Jupyter notebooks you can scroll down, you can work through it that way. Basically Jupyter notebooks consist of cells. So this logo here is in its own separate cell. Think of a cell like a paragraph or a section, just encloses text or it encloses images or it encloses code. So what I'm gonna do is I'm gonna go to section six of this Jupyter notebook, which contains a very simple and real example of some social network analysis that I conducted myself in my own research field. So I'm just gonna head down to this section of the notebook here. So basically what I was interested in is UK charities and I'm interested in to what degree they're connected. So there's a long running organizational sociology concept known as board interlock. So that's the degree to which boards of companies or charities or public sector bodies are connected. So in this example here, I just defined very, excuse me, kind of rudimentary pilot study of board interlock in the UK charity sector. So I'm interested in what degree, to what extent do we see board interlock among I say UK charities, but we're talking about England and Wales and actually specifically in this example, Manchester. So basically two charities are connected if they have the same trustee. So if I'm on the board of one charity and I'm on the board of another, then I can say those two charities are connected through me. So that's a network kind of phenomenon we're interested in, how are charities as organizations connected through individuals. So to use some of the social networking terms, the things that we're interested in studying, you know, the entities that are connected are known as nodes in social network analysis. There's lots of different terms, depending if you're coming from a mathematical perspective or a sociological perspective, we're gonna use the term node. So a node in our example is a charity. And then we're interested in what we're gonna call connections, but the technical term we're gonna use today are ties. So how our two charities connected and if they are, we can say that they share a tie between them. So ties are simply a connection between nodes. So what I've done is I've taken a publicly available data set, it's an open data set of charities who are at least headquartered in the city of Manchester in England. That doesn't necessarily mean that they only operate in Manchester. We can think of international charities like Oxfam, which will have a headquarters in London, but will operate across 80, 90, possibly more, 100 and something countries around the world. So that's just to give us our kind of framework for what we're gonna do today. Hopefully it's a semi-interesting topic for us to look at. I know I'm certainly interested in it. So the way we use Python to conduct some social network analysis is kind of form of a couple of steps. And the first step is known as the kind of preliminary steps. So that's really before we start importing data, cleaning data, doing analysis, et cetera. We essentially have to tell Python, okay, hey, these are the methods and the functions and the processes we need today to conduct our analysis. So I'll make this tiny bit bigger, excellent. So what we do here is we have a code cell. So basically this is a type of cell in a Jupyter notebook. In contrast to the one above just here, we can see that this is a text cell. We just write plain texts as if it was a Word document or a notebook. We can write here, for example, I can save that, execute it so it actually appears. The code cell is different. Whatever programming language we're using in Jupyter notebooks, this is what we will write in this little section here. So we're using Python for this. So basically we need a couple of different packages in Python. So think of a package as a kind of a collection of programming code that allows us to do things in Python. So for example, the pandas or pandas, I'm not entirely sure what the correct pronunciation is, is basically a data cleaning or a data wrangling package that allows you to import data, clean it up and spit it out in lots of different formats. It's incredibly useful. We're gonna make good use of it today. NumPy's for just conducting mathematical operations in Python from more advanced than just adding or summing or multiplying, dividing, et cetera. We can do kind of more complex mathematical operations. What we're really interested in today is networks. So that's the Python social network analysis package that we need today. There is another one in Python. I think it is graph QL, graph something. I find networks possibly easier to deal with. I think it's maybe a bit more English language based. So if you want networks to do something as a piece of programming code, it's closely related to English. But it's just a flag up that certainly for some of you who are familiar with maybe the R programming language that there's multiple ways of doing the same thing basically. And for us today with social network analysis, we're gonna use the networks package for our purposes. We've got some data visualization package that's quite useful when we draw some graphs in a moment. And then we've got kind of a standard piece of Python programming which kind of just allows us to loop through lists, for example. It's nothing complicated at all. We just have to call it in to this Python session. So that's what we do with the preliminaries. We tell Python, go to where you save all the code that we need and pull in these bits of code to this session today. So if I execute this code cell, I can do it two ways. I can click on the cell here. You can see it'll either be highlighted in blue or green. Green means you're in editing mode. You know, I could type things just here. Either way, it doesn't matter. I'm not editing at the moment. I'm happy with that code. And I can go up here to the run button and just click on it. And that will tell the Jupyter notebook to execute that code. Now, we're not expecting any output from this. So all we're saying is, you know, import these packages in so that in a moment we can start actually, you know, performing some program and some analysis. And there's a keyboard shortcut as well. You can hold shift and then press the enter key. I think maybe hold in control and the enter key as well. But basically, if you wanna execute code today or using a Jupyter notebook in general, go to the cell in particular, you know, click on it so that it's highlighted and press and run. So we can rerun this as many times as we want because we're just saying, you know, load in these packages. And how do you know which packages you need to use? Well, basically, when you learn how to do social network analysis or web scraping using APIs in Python, machine learning, et cetera. Basically, the examples you find will say, in order to execute this, you know, piece of analysis, you will need the networks package, the pandas package, the requests package, et cetera.  It'll be made very clear to you that you can only conduct social network analysis if you use these packages in Python. So that's getting ourselves set up. Excellent. I know I've taken a bit of time, but again, assuming that you're reasonably new to Python and maybe particularly new to using notebooks, I thought I'd spend a little bit of time. So we move on, so we get kind of stuck into things now. So what I want to do is get some data in to my Jupyter notebook. As I said, I've got some data about charities in Manchester and who sits on the board of these organizations. This is, again, public information, so don't be surprised when you see some real names. And it would be quite a coincidence if one of you was one of the examples I'm just about to show, probably quite unlikely. So what I do here is, first, I want to read in some data. In order to do that, I use the pandas module that we imported earlier. You can see here when I did import it, I gave it a shorthand reference, PD. I didn't have to do that. That just saves me time later on. Instead of typing out pandas in full, I can just say PD and Python knows that I'm referring to the pandas package. So I want to call on my data cleaning package in Python. And then I want to use the read underscore CSV method. So that's a function or a method that allows me to pull in data from somewhere on my machine or on the cloud, which we're using today, and load it into my Jupyter notebook. So I do that, so I take this file here in the data folder, I load it in, and then I want, you know, I tell Python, give me a quick look at the data, how many rows does it have, how many columns does it have. And this function here just basically says show me the first 12 rows. So using the head function, just give me the first 12. I could easily say the first five, for example. But we'd execute the code first and see what kind of results we get. So that's really good. So it's clearly working. There's about 2700 rows, four columns. And we can see an example of the first 12 rows and all the four columns just here. So how to interpret the data is basically we have individual one here. This is their unique ID. This is their real name. So this is Rabbi Abraham Hassan. This individual has three trustee ships. So this person sits on the board of three Manchester charities. And then these are the unique IDs of each of those charities. So using our example, and we'll develop this further in a moment, we can initially say that these three charities here are definitely connected to each other through a common individual here, this Rabbi here. We can see that there's a second person who sits on the board of mine organizations, which I'm not sure where they find the time. And again, here are the unique IDs. So again, we can say that these nine charities are all connected to each other through this individual. It gets more complicated later because two organizations might be connected multiple times to different trustees. And how do we take account of that? That's easily done in social network analysis, but it's not a mode of analysis we're gonna undertake today. We're just gonna simply look at are two charities connected or not and how many times does that happen? So excellent. So one of the first things we want to do is basically just take a look at how many charities there are in the data set. You'll notice the way it's currently structured is we have basically lists of trustees. So there's 2,700 records, but of course, so that's not necessarily 2,700 people. It's X number of people on the board of Y number of charities. So we just want to do a simple count of how many organizations are in the data set. So basically I say count the reg no, so that's registration number column by dropping duplicates. So I only wanted to count unique instances of charity number and we get 1,123. So basically there's 1,123 charities in Manchester that are connected through their trustees. So the next thing we want to do is when we're conducting social network analysis is we need to get the data in the correct format. Now this is something we went through and quite a lot of detail in the webinar so I won't go too much into it now. But essentially we want to move from the data set we just had, which is like lists of trustees on a column showing which charities they're connected to. And basically we want every role to be a charity and we want every column to be a charity. And in the cells, in between the rows and columns we want a one if those two charities are connected and we want a zero or a blank space if those two charities are not. So that's a bit probably abstract to conceive and to visualize just as I've explained it. So let's just get straight into it and take a look at how it should look. Excellent, so to kind of give life to what I've just said, basically now we have all of those 1123 charities all listed individually as rows. So here's Charity 208879, so that's a Manchester charity. And then along the columns we have the same charities just listed again that way. And then in each cell a zero tells us if those two charities are not connected and a one would tell us if those two charities are connected. Because there's again 1,100 rows and 1,100 columns Python's not gonna, the notebook rather is not gonna show all of it. So you can see here at the ellipses kind of saying that there's lots more columns in between and that we haven't taken a look at. There's no real reason to actually visually inspect it. I mean an easier thing here would be to just to use the pandas module and to just export the data set and then you could just look at it manually. You could open it in Excel or some open office software. But it's good to do a little visual inspection but we don't need to comb every column in every row to check that there are ones. As you probably guessed, it's much, much simpler just to go across the rows or go across the columns and just say count the number of ones and that'll tell us how many connections there are. So that data set or that matrix that I've just created it's called charity underscore math. That could be anything that's just a variable name I've picked. Then I want to use the sum command. So basically just sum either the rows or the columns. So if I specify the first axis, so that's a way of Python in Python of referring to the columns or the rows. So if I refer to axis one, I'm referring to the rows. Basically, I just want to count, yeah. Basically I want some summary statistics actually it's not just counting, it's give me some summary statistics of all the ones and zeros that we find in the data set. As you can see here, we get the average. So basically each charity is on average connected to about three others. So there's quite a lot of board interlock in the Manchester charity sector. Probably not too surprising, there's, well there are I think half a million people in Manchester. You tend to get kind of the same people volunteering. So it's not like all 500,000 people have a chance of volunteering as a trustee. There tends to be this idea of a civic core when it comes to the charity sector that roughly the same kind of numbers and groups of people perform multiple roles in the sector. So you can see that the minimum number of connection. So every charity basically is connected to at least one other in this data set. And there's one charity, we can see the max statistic here that's connected to 23 others. So that's quite a well networked or quite a possibly quite important charity that kind of sits in the middle of the Manchester charity network. We can start looking that up in a moment. And just to show that you can also just say, well take the information in each column and give me the summary statistics and you get the exact same results. For the simple reason that we have 1,100 rows and 1,100 columns, it's the same organization. So it doesn't matter if you want to count all the ones across or all the ones down the way and we get the same summary and statistics. This is maybe, I'm showing you maybe not complicated Python code to do this. There's a simpler way of using the networks package in a moment. But I do encourage you to take maybe a deeper look at what I've just shown you particularly how we construct the network data set. It's not something you necessarily have to do all of the time. But why I think it's a good thing to do is it really really forces you to focus on the kind of key concepts and just what it is you're trying to do. So by setting up the data set that way of all the rows and all the columns, having charities, I made it very explicitly clear that I'm interested in how charities are connected. Because you probably have noticed I could have done it the other way and I could have put all the trustees on the columns and all the trustees in the rows as well and said, our two trustees connected to each other by sitting on the same board. That's another way of looking at the same data set slightly differently. But basically after all that work, just if any of that wasn't interesting or particularly relevant, it's all in service of getting the data in the right format so then I can start using the networks package which allows us to do the actual core social network analysis. So basically I go again to the networks package here. I refer to it as it's short hand, it's NX. It has a method called from pandas adjacency. So that just means from a matrix data format which I've just created, create a networks object. And we get our result here which basically just says we've created a network graph. A graph for most of us is a more generic term for data visualization, some kind of pictorial representation of data or an infographic is not necessarily a data-based product. In social network analysis, a graph is basically the data, it's network data. So again, it comes from the kind of mathematical origins of social network analysis. So from now on, when you hear the term graph, we're not actually talking about data visualizations. We're talking about that network that I've just created. So those columns with all the charities and those roles, and that's now known as a graph because it's a list of nodes and it's a list of ties. So basically a graph is a collection of all the things that are connected and all the connections between them. But we'll see again, we will see a visualization of that graph in just a moment. But it's just so you understand with social network analysis, a graph refers to network data and not necessarily an actual visualization. So now what I've done with this piece of code is I've said, right, here's my network data and put it into the networks package so that Python now knows that we can start doing some social network analysis methods and techniques. So what we want to first kind of focus on is some network level summaries. So basically we took a slight look at them. We looked at the average number of connections between charities, the maximum and the minimum number of connections. We want to go kind of one step further. We want to look at the size of our network. So how many charities are in the network and how many connections between them exist. So basically a very simple networks method. It's called the dot info method. So give me information about this network data here, which I called char graph. So it produces again some summary information about the kind of the aggregate properties of the network. The number of nodes is 1,123. We knew that from previously. We know that we have 1,100 unique charities. What we didn't know previously was, you know, how many connections existed between them and we get about 1,500 connections. So out of 1,000 charities, there's about 1,500 connections between them, which is reasonably dense. So we can get an exact measurement of density in a moment, but it's a reasonably high number of connections between charities. And again, as we learned before, each charity is connected to, on average, about three other organizations in Manchester. So that brings us to a concept in social network analysis called density. So the density of a network is essentially of all the possible connections that could exist, how many actually exist in reality. So how many possible connections between charities have been realized? Again, thankfully, we can use the kind of English language-based methods that the network's package provides. So instead of asking for information, now we're asking for information on density. So we use the density method. And what I do is I just take that result and I put it in a new variable called density. That's just so I can call it later on. So I want basically Python to print a message back to my screen saying, you know, this text here, i.e. network density colon and then whatever is captured by this variable density here. So actually those 1,500 connections in reality from a substantive perspective, that's a reasonable number of connections between charities. But mathematically, it's actually a very, you know, very small number of actual connections that could exist. So densities measured as a proportion between zero and one or just think of it as a percentage between zero and 100. So for example, basically, you know, there's fewer than 1% of all connections have been realized in the Manchester charity sector.  But we're talking about real organizations, real life here. You don't tend to find very dense, real networks. You might find things like internet networks or electrical wiring networks to be very dense. So of all the wires that could be connected in and plugged in, the vast majority are, that's a good thing from an electrical engineering perspective, real life social networks don't tend to be very dense. So then we can just, we can move through a couple of other social network analysis concepts, you know, so clustering is one very interesting, well-developed concept. And we're interested in one aspect of clustering which is known as transitivity. So this is a concept where basically, if you know, three charities exist in the network and connections exist between two of them, basically what's the probability that the third connection will become realized? So it's better to kind of look at this visually. So basically a connection between three organizations would be known as a triad. So a trio of connections. Sorry, let's make this a bit more legible. Excellent. So this is an example of a possible triad. So you can see we have a group of three charities, internet work, two of them are connected, let's say charity A and charity B and charity A and charity C. So basically the clustering concept and the clustering measure we're gonna calculate is what's the probability of these two organizations forming a connection? So this is the friend of a friend kind of idea in a social network and what's the probability that a friend of a friend becomes a friend, basically. And this just gives us an idea of how likely is a network to become a bit denser? How likely are connections to form when we see this kind of example in the network? So that's a possible triad. So this is what we'd expect to find. We're basically saying, if we see a possible triad, what's the probability of this example coming to life in the network? Let's make this a bit bigger again, perfect. So it's a concept called transitivity. So again, we want the transitivity method from networks. Hopefully you're seeing this is why I think networks is quite good. It's quite a logical and English language-based package of programming code. Again, I'm just calling this variable triadic closure to use the technical term. I could just call it triads, for example. I'm just creating this variable name, whatever I want. Again, ask Python to execute that code. And yeah, I get a measure of about 61%. So that's quite high. What that basically means is if we see examples where charities have an organization in common, those two charities are likely to form a bond between them. So just to scroll up quickly, basically if we see this scenario here, if we see that absence of a connection between two charities, then about 60% of the time it forms into this kind of connection. So a triad or a triangle. So that just gives an idea of in my network that if a charity knows another charity through a common link, then there's a pretty good chance, there's a better chance than not that those two charities will actually form a connection themselves. Okay, so we'll just cover a couple of other social network analysis concepts and methods. So we've kind of taken a look at the network in an aggregate sense. So network level measures of analysis. We're gonna now take a look at node level. So now we're interested in individual charities and what's the maximum number of connections per charity? What's the average number of connections? Can we get some sense of how important certain charities are in the network? For example, so we'll take a quick look at some of those measures just now. So one concept is called centrality. So centrality, there's a couple of different types of centrality that the one we're interested in just here kind of captures importance. So how important is a certain organization in the network? So basically the most important charity in my network is the one with the most connections. So that's the connection between the concept we're looking at importance and the kind of empirical measure we can look at which is the number of connections. So the most important charity is the one with the most connections is what we're saying just here. And this is a measure called the degree centrality. So basically there's a tiny bit more code needed to pull out the degree measure. So basically I have my network data here which I've called chair graph. Hopefully it's got a degree method. So it's like, tell me, you know, it's count all the connections of all the charities in my network. Yep, so I feed that method all of the nodes. So I basically said to Python, look at all the nodes in my network and count all the connections between them. And basically I store the results in a variable called this just here. And what I do is I want to take a quick look at, I've plugged this charity semi-randomly. So charity number two, two, five, one, one, six, tell me the number of connections or the number of ties in that charity has. So this charity has 12 connections to other organizations in the network. Yeah, and then what I do here is basically I just kind of loop through all of the results. So basically I have a list of all the charities and for each charity I count how many connections it has. And now I'm just saying, you know, show me the top 20 best connected organizations. And yeah, so the most important charity in the network, i.e. the one with the most connections is this one here 530-002. Let's actually make this real, that might actually help. We can actually just go and check who this actually is, register, there we go. Yep, search the register, government websites are not particularly intuitive. Okay, so basically it's a religious charity focused on education. And this is the best connected charity in Manchester. You can see where I get the information from. So I have all the list of all the trustees who sit on the board of this organization. And look, it's no surprise that the first person alone has three other trusteeships. So it sits on the board of three other charities. The second person sits on a different charity. Yeah, so you can see just even scrolling through the results just visually it's quite a well connected charity. And I've asked for the top 20. This kind of structure here is a loop. So we're saying for every element in a list, and my list is called this thing here, sorted degree. Again, it's a variable name, it could be called anything. This allows me to specify how many records to loop through. So if I was interested in the top 10, I would say I can make this explicit. So basically come from the first record to the 10th, or I could say 100 if I wanted.  Excellent. So we're going to recreate basically that for a slightly, oh no, actually we are going to do some visualizations. So a very common way of analyzing the number of connections in a network is to produce a histogram. It's called a degree distribution. And it'll be very familiar to any of you, even if you're not primarily a quantitative researcher. It's a very common data visualization type. It basically just shows us account of the number of connections in the network. So what we're doing here is we're just creating a variable that stores account of all the connections in the network. I'm using then the plot function in Python. I'm saying write give me a histogram using this dataset and then just some aesthetic information. So give me a graph title, label the x-axis, and show me the resultant graph. Being a bit funny with the display, let's zoom out a bit. Yeah, there we go, here we go, that's a bit better. So basically we've got a histogram of the number of connections in the network. So basically we can see that the most common connection is there to be one. So most charities only have between one and two connections in the network. But there's about 150 organizations, 130 organizations have about four, between four and five connections. Then there's about 50 organizations have between six and seven connections, and so on you get this long kind of thin tail. So there's a very, very small number of charities that we saw earlier. In fact, there's only two that have more than 20 connections in the network. Hey, and finally, so I mentioned that a graph is a more technical term than social network analysis. It basically just, it's another word for network data. But of course we can create what we understand to be graphs of network data, so data visualizations. Visualizing a network is not actually necessary to do good social network analysis. In fact, it's often not quite a waste of your time, but as soon as the network becomes more than trivially large, so unless your network is just kind of incredibly small, it becomes very difficult to find meaning in the visualization of a network. And we can kind of see this here. So this is the Manchester Charity Network visualized. Just trying to get a better representation of it. So basically we can see that there's kind of a large cluster in this network. Basically there's lots of individual charities that are connected to one or two or three other organizations. And then kind of on the kind of outskirts of the network, if you want to put it that way, we have one organization here that's connected to this one. So it only has one connection. This charity has two connections because it's connected to this organization here and it's connected to this one here. Then this one is connected to three, so this one here, this one here, and this one here. And then this one is connected to one, two, three, four, five, six. So the visualization is trivially revealing of the kind of the network structure, the network properties. But it's highly reliant on the method of drawing the network.  So this is one particular way that I've chosen. It's not quite a random layout. It's got basically an algorithm that decides where to put all the dots. And there's lots of ways of doing that. So here's an alternative way of doing it. This is the exact same network data. I haven't done anything. I'm not trying to trick you with a different data set. But this time I've said, right, draw the network using a circular algorithm basically. And as you can see here, basically, all the charities are on the outside in this kind of rugby ball. And then all the connections between them kind of crisscross. So you can somewhat see that this charity here, if you kind of follow one of the lines is connected to this charity here. Again, exact same network data, but this time effectively I have absolutely no way of, absolutely no way of, you know, defining how many connections are in this network, which charities, which maybe is this charity here because the line is denser. You know, is this the charity with the most connections? You know, I really don't know. So you can visualize the network. It's sometimes useful it may be something that's good for, you know, a poster at a conference. But hopefully as I've shown by executing the code to look at some of the summary statistics, visualizing the network is not central to your understanding of what's going on at all. So if there's one takeaway from today, do not waste your time trying to draw a very beautiful network diagram. It's often not worth it. So that's as far as I wanted to take you today. So it was just to give you a sense of how we work through a social network analysis example in Python. Again, I haven't explained the code on a very, very fundamental level. This isn't an introduction to Python. In general, we have other materials and there's lots of other great initiatives out there, not just from us. And again, we're not teaching you social network terms today either. This is hopefully for those of you who have some sense of both and just wanted a kind of a talk through of here are the steps and here's what's going on at the key moments. So that's the end of the demonstration. To me, hopefully a good few of you have followed along online, which would be really, really good. I'm now going to take a look at the chat. I'm currently minus a lot of IT equipment, so I do not have a second screen at the moment, which is a shame. But I will take a quick look at some of your comments. So that's the end of the demonstration. To me, hopefully a good few of you, excellent. Okay, so Farsi, thank you very much. Hopefully we've sent you the link and you've received emails from us as well. So there's lots of code you can see. I've just basically focused on one simple social network analysis example.  They're full of examples. There's plenty of practice. It's how you really develop your programming. I'm not a computer scientist by training. I'm a social scientist who's trying to shift towards more computational methods in my research and teaching. Hopefully today has demonstrated as well that it is possible to go very far with some decent Python and programming skills in general. The way I find that I explain this is that if you remember those kind of American movies, those military movies and they show the boot camp when there's this kind of 12 foot high wall and they're trying to help each other over it and you're like, how do you get over this bloody thing? It's impossible. But once you get over it, the obstacle course is finished and you can just run nice and freely to the end of the obstacle course. I find learning programming or Python like that where there's a very initial high barrier.  How to install packages? How to import them into Python? What am I doing here? What's the function? What's the variable? But hopefully once you get past that and as I've shown you today, you can go very, very, very far in a reasonably short time with some decent standard basic Python skills. That's enough for me. Definitely enough chat. I'm gonna stay on for another five minutes or so if you wanna post questions. You'll have our contact details. You probably won't have mine because I've just left the data service. But if you can find my details, then absolutely contact me. But I would encourage you to get in contact with my colleague Julia Kazmier or the data service in general. We're doing lots more stuff like this. We're doing drop-in sessions where you can bring your own questions and ask an expert at the data service as well. So you'll probably see my face for a moment while the feed is still live. So if there are any questions, I'm happy to answer. If not, then thank you very much for joining. Really good luck in terms of what you're doing with your computational social science. Thank you. Oh yes, there was a question about charity data. Yes, so I just used one particular subset of data to do with Manchester Charities. Yeah, I would just go to the Charity Commission for England and Wales website and as they've got a data download and they've got kind of like a public search tool. So you could just use the data download and you'd be able to filter by postcode or by city to what you want. There's lots of open data about charities now which is really, really good. Yes, so there's a question from Ian Hamilton. Thank you very much. Are there Python packages for statistical clustering methods? So yeah, PCA or latent block modeling? Yes, so I think in terms of network clustering, if that's what you're specifically, yeah, so block modeling, yes, can be done in Python. It can be done in the, I don't think it can be done in the networks package. I'd need to have another look. I think there's a separate package for that. I certainly know if you use, what's the other software? It's a Pyek. Yeah, there's a couple of other social network analysis packages that you could use that are specifically for doing kind of statistical modeling. So things like exponential random graph modeling. The networks package is quite good for, it does some advanced stuff, but it's mainly for kind of describing networks, doing that kind of, it was kind of fundamental summary statistics. And then there's, there typically tends to be kind of very specific packages for doing more advanced modeling or block modeling. So apologies, I can't give you the exact package name, but there certainly are packages in Python that can do that. Excellent. So if there aren't any other questions, if you do have questions, again, fill through them through the data service, Julia will pass them to me. If not, hopefully see some of you in two weeks' time from today for the next session. And thank you very much. Bye everyone.


Broadcasting Company and its affiliated stations present the Pacific story. In the mounting fury of world conflict, events in the Pacific are taking on ever greater importance. Here is the story of the Pacific and the millions of people who live around this greatest sea. The drama of the people's whose destiny is at stake in the Pacific War. Here, as another public service, is the tale of the war in the Pacific and its meaning to us and to the generations to come. BOSA. THE INDIAN QUESLING Working hand in glove with the Japanese behind the guns of Singapore, is Subhas Chandra Bosa, head of the Japanese-sponsored free Indian government. The key to his character is his hatred of the British, which goes back to his childhood and youth. You are Subhas Chandra Bosa? Yes, sir. You have an extraordinary record of brilliance as a student here at Calcutta University, Mr. Bosa. Thank you, sir. I regret that you have become involved in the unpleasant events of the past few days. I have a complete report on the student strike here before me. Do you have a report of the incident that precipitated the student strike? Mr. Bosa, as master, I must be the judge of this matter. Are the merits of the incident to be considered? I have called you in, Mr. Bosa, because of your part in the student strike. Does your report say that the student strike was caused by the manhandling of some of the students? The pointed question at this moment, Mr. Bosa, is your part in the student strike? Have you evidence that I was involved? Mr. Bosa, I will not tolerate you cross-examining me. I have summoned you here to discuss your part in the student strike. Yes, sir. Were you one of the leaders? One of the organisers? If you wish to discuss the merits of the incident, I shall... Mr. Bosa, I had hoped that I might avoid this. But now I must tell you that you are expelled from Calcutta University. Expelled? I regret that you can no longer be considered a student in this institution. Mr. Bosa was 19 when this happened. As a high-caste Hindu, the son of a well-to-do lawyer, he'd been well brought up. But for years he'd been bewildered by the attitude of the Europeans toward him, an Indian. He chafed under these relationships. Now he was in open rebellion. He went to talk with the distinguished attorney, C. R. Das. You know about the O incident as well as I do, Mr. Das. Yes, of course. We have been treated this way for years. It was the same way at the Protestant European School in Cuttack. How can a young Indian have anything but a feeling of inferiority at the way he's treated? Favor is shown to the Europeans, whatever their colour. But we Indians are discriminated against. Do birth and breeding and intelligence count for nothing? They count for a great deal, Subhas. Then why must we go on being treated this way? It was no better in Ravenshaw Collegiate School. I knew English as well as the English themselves. But the headmaster and the European students ridiculed the way I spoke Bengali. You ultimately won honors in Bengali. Did you not, Subhas? Yes. But why did they ridicule me while I was learning? They have no regard for Indians in school or outside of school. Anywhere. They are rude. They lack manners. And when we Indians have remonstrated as we did in the O incident, we are not dealt with with understanding and patience. We are dealt with as as summarily as if we were criminals. Subhas, I know so well what you have gone through. Many of us have gone through the same experiences. Can nothing be done about it? Yes, something can be done about it. We must do it. We Indians. We must do it for ourselves. Then let us do something. The time is not yet right for you. You mean I should finish my education? Yes, you must. Well, this seems hardly the thing for me to do.  You must finish. My confidence in you, Mr. Das, primes over. It does not seem right to me yet. If you say it must be, then I know it must be. Go to England and there you will gain a deeper understanding. From this time on, C.R. Das had an inspiring influence on Bosa's career. Bosa went to Scottish Church College in England. From there he went to Cambridge. He became known for his brilliance. Isn't Subhas Bosa there handsome? Look at him there. Not only handsome, he is one of the most intelligent students here. Yes, and shrewd. We're going to hear from that man. Mark me. He was to be heard from sooner than most realized. When he finished his work at Cambridge in 1920, he took the examinations for the Indian Civil Service. He ranked with the very top winners. I don't know whether I should take the position. How can you hesitate, Subhas? You have worked to prepare yourself all these years. You finished with those at the very top of the list in the examinations. It isn't that. It's what's happening back in India now. Gandhi's nationalist movement? That's where my heart is. Subhas, just the very few are chosen to be civil service of Britain and India. And you have won that honor. In the civil service, your position will be safe, and a whole career lies ahead of you. I... I want to talk with Gandhi. My sympathy is with Gandhi too. But it is possible that you can do more for India in the civil service than by joining Gandhi's movement now. I wonder... Bosa joined the civil service, but he watched Gandhi's movement day by day. The dynamic events taking place in India gripped him. Within the tall, handsome Indian, his friends saw a fierce struggle. Gandhi has stirred him deeply. You can see that he is seething inside. He is losing interest in the civil service. Gandhi has touched his imagination. One of these days he will leave and join Gandhi. And if he does, his parents will oppose him. Subhas will not be stopped by that. He has the courage to face his parents' disfavor. The conflict inside him is beginning to tear. The agony of indecision consumed him. It was plain it could not continue long. Whatever decision he made would be momentous. It meant that he had chosen to work with the government or against it. Suddenly, dramatically, he made the decision. I have resigned the civil service. I am going back to India to join Gandhi. On the day he arrived in India, he went to talk with Gandhi. Henceforth, these two personalities were to be touchstones, each to the other. Sometimes there was to be the cohesion of mutual understanding. More often, they were to strike sparks, one upon the other. And sometimes there was to be fire. The dynamic 24-year-old questioned and cross-examined the seasoned experienced Gandhi. When the interview was over, Bosa went to his inspiring friend, Siyar Das. There is no question about Gandhi's personal honesty. He is a man of integrity. But specific answers I could not get from him. That is not because Gandhi is evasive. It is because he sees the non-corporation movement with more experienced eyes than yours. I know that is true. I have supported Gandhi's movement for nearly three years now. Your belief in him is sublime. No, not that. But yet the non-corporation movement is our hope. And that is why you, who are strong and young and intelligent and courageous, must join it. If you believe in it, then certainly I must believe in it. Bosa joined Gandhi's movement. He became an active nationalist, took part in the civil disobedience campaign, and within a matter of months was in jail. Here he gained his first experience behind the bars. He neither feared his captors nor bewailed his fate. After several months he was released. Now he was wiser but more determined. He and Siyar Das moved closer together. Together they founded in Calcutta a daily newspaper and named it Forward. I have never seen such a remarkable attachment as that of Bosa for Das. Bosa is almost a disciple of Das. He is his right hand man. That means that Bosa is close to the top of the India Congress. For although Gandhi is president of the Congress, Das is actually the real power. Yes, and Das recognizes the extraordinary abilities of Bosa. In effect, although Bosa is only about 27, he is already a tremendous influence in the Congress. Das became mayor of Calcutta in 1924 and with him into the inner circles of the administration of this great city, Das took Bosa. He was elected chief executive officer. Bosa was becoming experienced in politics far beyond his years. Yet soon again he was in jail. Inside of jail and outside he continued his activities. Again he was released. Day by day now his political influence was growing. In 1926 he was elected to the Bengal Legislative Assembly. Soon he was in jail again. This time removed to a prison in Mandalay in Burma. Look at this. This newspaper. This is scurrilous. Inside as well as outside of jail he was articulate. You see this? You see what this newspaper says about me? Where? What does it say? There. It says that I am the brains of the revolutionary conspiracy. That is a high compliment. It's not true. What newspaper is that? Oh, Calcutta newspaper. I shall sue the paper for libel. You've forgotten you're in jail in Mandalay too far from Calcutta. Being in jail does not abrogate my rights. I'll sue that paper just the same as if I were there in Calcutta. And that's what he did. The suit for damages came with a shock to the newspaper. You know, Tirlig, that we're being sued by Subha's buzzer for this story of yours. About him being the brains of the revolutionary conspiracy? Yes. Well, he is, sir. You know that. But can we prove it? It shouldn't be much of a task to prove that, sir. Very well. Our only defense is to prove it. That I will, sir. I'll get documentary evidence from the government and other places too. Get it. Let's have Don with this thing. Well, maybe it is true, but we've got nothing here to prove it. Everybody knows it's true, but to get evidence that will stand up and court. Well, that's another thing. There, we have nothing here to that effect. Why don't you try the India office? You'll never be able to prove anything against Subha's buzzer. He's a sharp one, he is, and he's able to take care of his self, he is. Is the judgment of this court that the printed statement that Subha's Tandra buzzer, this quote, the brains of the revolutionary conspiracy on quote, is not substantiated by the evidence. The defendant is therefore ordered to pay damages to the plaintiff, Subha's Tandra Bosa, in the sum of... Bosa was rising as a power in Bengal. When he was 30 years old, he was elected president of the Bengal National Congress. The younger element of Bengal were inspired by him. They talked of him, quoted him, followed him. He organized the Bengal Students' Conference. Rallied the younger element and got himself elected city councilor of Calcutta. His influence expanded. He forged ahead. Each time he was released from jail, he came out more radical. When Bosa was 33, he was elected mayor of Calcutta. He administered the affairs of Calcutta from his prison cell. Here is another letter for you from Vienna. Have some friends there, have you? Yes. You ought to do something about... about the... the sickness of yours. What can I do for it here in prison? From Vienna came word from Valabai Patel, another worker in the Indian nationalist movement. Patel, like Bosa, suffered from tuberculosis. And Patel urged Bosa to come to Vienna for treatment. The Viennese doctors are excellent, Bosa. We shall see what they can do for me. Bosa put himself under the Viennese doctors. But he continued to carry on his activities in the nationalist movement.  Yes, yes, I know Patel. My advice is that you forget about Moscow. I am not thinking of Moscow. Have you changed your mind about setting up headquarters there? Our welfare does not lie in that direction. You are wise, Subbas. I too have received invitations from Moscow and from Rome and Berlin too. Did you decline because of your illness? I declined for the same reason as you did. Invitations continued to come. Mussolini invited Bosa to Rome. Still desperately sick, he accepted. His conversations with Mussolini were long and intense. News of these conversations reached India. Bosa cannot be swayed by Mussolini. He is too purposeful. But in his clever cunning way, he may think he can use Mussolini. He had better take care that Mussolini does not use him. Bosa is violently anti-fascist. Bosa was anti-fascist. He railed against the fascists and the Nazis. Observers watched him closely. He is drifting toward Rome and Berlin. He is beginning to regard the point of view of the fascists and the Nazis as the realistic view. That year, dynamic events took place in Europe. Arnold Hitler has become the new chancellor of Germany. President von Hindenburg of Germany is dead. Arnold Hitler has seized power in Germany. And his nationalist, socialist party is... You must not be too much impressed by Hitler's rise to power, Subas. Hitler has succeeded in taking over Germany. But can he succeed in the end? Mussolini is supporting him. But in the last analysis, how much will Mussolini's support mean? We must accept the fact, Patela. But a world revolution is underway. We have seen what has happened here in Europe and see what is happening in the forest. Japan's seizure of Manchuria? Not only Manchuria. Japan has also taken re-hold. Japan is the rising power in the forest, as Germany and Italy are in Europe. Subas, I have been the nationalist movement a long time. You must believe me when I say that we cannot help ourselves by accepting the pattern of events in Europe and Asia as our pattern of progress. Both Hitler and Mussolini have taken direct action. Do you agree with their philosophy? I believe the time has come for direct action in India, don't you? The people of India must work out their own salvation. They must not follow other nations. Let us not talk of other nations now. Let us take our position against Gandhi and his non-cooperation and start a more militant fight for freedom in India. Both Hill won on the verge of death. Bosa and Patel issued a joint statement. Gandhi had outlived his usefulness, they said. The nationalist movement, they said, must abandon Gandhi's policy of civil disobedience and undertake an active battle for freedom. They decided to buy with blood what they had not won with non-cooperation. Now Patel was dead. And now Patel's mantle of power in India fell upon Bosa. Influential Indians who came to visit him went away with mixed thoughts. He has gained in prestige. Yes, because in effect he is Patel's successor. Ah, but is he actually? Well, Hitler and the other Nazi leaders seem to accept him as such. Never once did he refer to his talks with Hitler and Gering and the others. No. Could he be moving toward them? Well, you remember he said that we must hold ourselves apart from the European thinking? Yes, he said that. But he also said that now he understood the Nazi point of view better than ever before. That seems inconsistent. May mean that he is in the process of making a transition from our way of thinking to the Hitler way of thinking. It is plain to me that he has less in common with us now than he used to have. That confirms what the others have told us. Now he is more interested in his European friends than in us. Yes, he gave me the feeling that he was parrying our questions that his answers were not what he really thought. His whole attitude seems changed. Oh, he has changed. He has more power, more money, more money. Money? Yes. Well, you could never spend money as he apparently is spending it on what we know he should have. Where could he get the money? That would be difficult to say. Yes, but this we can be sure of. Subas Bosa is enormously ambitious. More and more Indians came to talk to him. Indians who had been solidly behind Gandhi's non-cooperation movement traveled to Vienna to find out for themselves how Bosa was working for India's independence and to inquire about his plans for them. You must have equality. We cannot have one people above another. This is said to be untouchables. A modern state cannot endure with discriminations against any segment of its people. Your rights must be respected. This is said to the Muslims. The backbones of progress is industry and finance. You must be given the opportunity to develop. This is said to the Indians interested in industry and finance. Indians of all kinds came to see him. Self-seekers, patriots, businessmen, politicians. Beneath the surface, the direction of his thinking was becoming apparent. Observers watched him. It is not so much that he likes either Nazism or Fascism as it is hating the British. He has been impressed with the show of power in Germany and Italy. He can conceive of this power being used against Britain and he considers any opposition to Britain to be in favour of India. Nevertheless, he knows that many Indians, although they are against Britain, are not in sympathy with Nazi Germany and Fascist Italy. Therefore, in his relationships with the Nazis and the Fascists, he moves lightly although now it has become an open secret that he has relations with them. Adrightly, he was gaining power enough to return to India and challenge Gandhi. His opportunity came in 1937 when he was 40 years old. He returned to India and became Alderman of the Bengal National Congress. The time for temporizing is past. The time for direct action is here. Both are gathering so much strength that he may succeed in getting control of the Congress. That is precisely what he is working for. He has built up a strong organization and he is using it to maneuver himself into power. Bosa organized and maneuvered well for the next year, 1938. He got himself elected president of the Congress. It is unbelievable. He is the youngest man ever to be president of the Indian National Congress. What is more amazing is that he won against Gandhi. Yes, and to think that only a few years ago Bosa went to talk with Gandhi and was so impressed that he became one of his strongest followers. Times have changed and so has Subhas Bosa. Bosa, at the age of 41, became a focal point of the world. He was the man who had taken a position against Gandhi, the man who came out against non-cooperation and in favor of militant action for freedom. Our first objective must be complete freedom. This was his presidential address. Once we are liberated, we shall dedicate ourselves to the eradication of poverty, to the raising of our national literacy, to the fighting of disease. These are immediate problems, but there are also long-range problems no less important. We must lay a sound foundation on the realities we Indians know so well. The world weighed his speech for its underlying import. About freedom he was clear. Ours is the struggle not only against British imperialism, but against world imperialism as well. We are therefore fighting not for the cause of India alone, but of humanity as well. India freed means humanity saved. Bosa was in the driver's seat. This was 1938. This was the year of Munich, the year of appeasement. The war drums were rumbling. Now Bosa was in power in India and Hitler was the most powerful man in Europe. Before his election, Bosa had made a hurried visit to London and urged all young Indian students there to get themselves into jail to help the struggle for India's freedom. He was a thorn not only in Britain's side, but in Gandhi's side. And where he would move next, no one could tell. The next year, Gandhi opposed the re-election of Bosa. Bosa has won again! Bosa has won again! Yes, he has won again. Bosa has been re-elected. What will Gandhi do now? Gandhi fought his re-election. Gandhi still has broad influence. But what can he do? Bosa has been re-elected. Soon it was to be plain what Gandhi would do. Gandhi was still a political force. His name still had great influence. He used these against Bosa. The working committee has resigned in a body. Working committee? Why Bosa considers the working committee as the cabinet of the future? It is said that Gandhi influenced the working committee to resign. And sub-hust Bosa is helpless. Gandhi had scored. Bosa resigned. He had reached his highest point, and now his star was descending. And soon it was plain what he would do. Soon a new organization was formed, and the observers made a note of his action. Bosa calls his new organization the Forward Blanc. It is actually a rival organization to the Congress, isn't it? It's made up of disgruntled left-wingers of the Congress. Well, that should give us some idea of the index of what they stand for, and what they'll do. Their principal objective is immediate independence. Immediate independence, eh? If Bosa had been able to go along with his normal plans, he probably would not have come out as strongly as this for independence just now. But now that he's frustrated, well, he wants independence now. Not sometime in the future. The results of his talk with Mussolini and Hitler are now becoming apparent. Oh, it's perfectly evident that he has fascist leanings. I wondered how long it would be until he showed the influence of Rome in Berlin. Oh, it's pretty plain right now. He's come out for a strong central government with dictatorial powers. Uh-huh. He's seen Hitler's government in action. And he's come out for a strong party bound together by military discipline. A strong party bound together by military discipline. Yes, Bosa must have kept his eyes and ears open in Berlin. By this time, Bosa had come to the conclusion that India should accept help from wherever she could get it. With this as a premise, he extended the forward block throughout India and beyond the borders of India. In secret circles, his name was linked with those of foreign agents, Nazis and Japanese. We have been discriminated against. We must join Bosa. His influence reached to Singapore. Who else but Bosa will help us? We must join him. The former. The forward block is our hope. We must join Bosa. To Thailand. We must join Bosa. We must join Bosa. We must join Bosa. Indochina, Malaya, Iran, Afghanistan. Swiftly, dramatically now, World War II was approaching. In a matter of months now, Hitler marched into Poland. He overran Denmark and Norway. Smashed through the low countries, crushed France. And each of these dynamic events had its echo in India. By what right does the British viceroy declare India co-belligerant? March 1940, the Indian Congress has forsaken provisional self-government and demanded complete independence. We must not only have complete independence, we must have it now. Because Chandra Bosa has been arrested under the Defence of India Act on charges of plotting to raise the... Once again, he was in jail. And while he was locked up, he was elected a member of the Indian Legislative Assembly. Vosin, here's a message for you. Thank you. Looks like it might be important where it goes. Who's it from? The British Raj. I asked for a release long enough so that I could be installed in the Legislative Assembly. He has turned me down. Oh, that's too blinkin' bad, Mr. Bosa. You can tell your superiors that I will not eat a bite of food until I am released. Yes, Governor. Another hungry striker on a range, lamy. Off we go. With no food, soon he was ill. So well that his condition alarmed the authorities. He was released. One day he disappeared, and soon back from Berlin came rumours. Bosa has made a deal with Hitler to conquer India. Yes, and have you heard that he is speaking on the Nazi radio? From the Nazi capital directed at India came the words of Subhas Chandra Bosa over the radio. We shall fight against British imperialism until India is master of our own history. Soon records of Bosa's voice were being beamed at India from Japanese-controlled radio stations. And soon Bosa himself was in Tokyo. There, with Japanese warlords, he laid plans for the conquest of India. Then, dramatically, he went by Japanese submarine to Singapore. From Singapore, intelligence filtered out which indicated the nature of the plans worked out by Bosa and the Japanese. Bosa is at the head of what he and the Japanese called a free Indian government. An army of liberation is being formed to march on India. This is the army which attacked in Manipur. When the drive first smashed into India, the Japanese high command issued a statement. All areas of India which are liberated by the India-Japanese army will be praised under the provisional government of a free India. In Japan's interest in the conquest of India, Bosa saw no danger either to India or himself. Japan has no political or economic or military ambitions in India. Japan's interest is the same as the interest of patriotic Indians. To destroy the forces in India, which are enemies not only of India but of Asia as well. Bosa is in the Japanese camp. Yet observers see him as still an important factor in India. Today, Bosa still exerts great influence in the Indian nationalist movement. He is using this influence to his advantage. Many still support him. And in this proportion, he must be considered a dangerous factor to be dealt with in the complex Indian problem. You have been listening to The Pacific Story, presented by the national broadcasting company and its affiliated independent stations as a public service to clarify events in the Pacific and to make understandable the cross currents of life in the Pacific basin. For a reprint of this Pacific Story program, send 10 cents in stamps or corn to University of California Press, Berkeley, California. The Pacific Story is written and directed by Arnold Marquess. The original musical score was composed and conducted by Thomas Paluso, your narrator, Gane Whitman. This program came to you from Hollywood. This is the national broadcasting company.