Hello, welcome to SSC Unitec. My name is Suseen Singh and this is continuation of SSIS tutorial. So today we are going to discuss about how we can load data from FlatFile to our database. So let's start. So as we can see over here, first of all we have to use a Dataflow task inside this control flow in our first image. And in our second image we can see first of all we have to use a FlatFile source from where we want to get the data and then we have to use a ADB destination where we want to load the data. So this is our process which we have to follow. So let's move to SSGT. In our last video we have understand how we can start with SSGT. This is our package. First of all I am going to rename this package with FlatFile to database. So this is our first package FlatFile to database. So as we can see over here we have to use a Dataflow task. So this is our control flow and here we need to use Dataflow task. We can drag and drop this Dataflow task. And once we directly click on this Dataflow task it will directly move to Dataflow panel. In Dataflow panel we can see in our SSIS tool box we have option for other sources. When we expand this other sources then we can see we have FlatFile source. Then we can drag and drop this FlatFile source. So before going forward I would like to explain about connection managers. So what is connection managers? So connection managers are used to make the connection to connect with our source or destination. For example here we want to get the data from FlatFile and then we want to load that data in our SQL Server tables. So here we need to make two connections. First is for FlatFile source and second is for our OLEDB destinations. So before making connection let's have a look of FlatFile by which we want to get the data. So we have this text file. So here we can see it has comma separated values and it contains three columns. First is for ID, second name and then Dataflow. So here we need to load this data in our SQL Server table. So go to SQL Server Management Studio. So in SQL Server Management Studio we can see we have a table with FlatFile to database table. It contains three columns. First is ID, then name and then data output. Initially we don't have any data and we want to load data from our FlatFile which is available over here. We can see it has comma separated values and total eight rows is available over here. So we want to load this data in our SQL Server table. So I need to create a connection for FlatFile source. So double click on this FlatFile source and here we can see FlatFile connection manager. Initially we don't have any connection over here. So we have to make a new connection. So click on new. Once we click on new, then it will ask few things. First is file name. So we need to browse. As we can see FlatFile comma delimited source. We need to open. Then we need to go to on format. In format we can see we have three types of formats available in FlatFile. First is delimited, second is fixed width and last is ragged write. So FlatFile source contains three formats. First is delimited, second fixed width and then ragged write. In delimited we can see values will be delimited by any delimited like comma, pipeline or anything else. And in fixed width column size will be fixed and in ragged write column size also will be fixed. But what is the difference between fixed width and ragged write? The only difference between fixed width and ragged write last column value is variable in case of ragged write. Rest of the column value should be same but the last column value will be variable. So here I am going to load data from delimited file and delimited is comma. So go to columns column delimited is comma. So we can select comma from this dropdown. It has another option as well. So here we can see we are getting the data which is our label in our text file. But we do not want to load double quote which is available in our input. So how we can remove this double quote in our source? So go to on general and here we can see text qualifier. In text qualifier we need to specify that character. That character is double quote. So double quote specified over here. Now we need to go to on column and here we can see double quote is removed. If we want to remove any another character then we can specify. So now go to on advance. In advance we can specify the data types. So first is ID. ID should contain the value of integer. So we can convert this from our integer and now go to on name. Name contains the string and output column width is 50. It's fine to me. If we want to change this we can change by 100 or anything else. Then we can see in date of birth. Date of birth is also contains the string value. Now we don't want string value. We want to convert this string value to dead time. So now we can select dead time, time is time. That is fine to us and go to preview. It looks fine to us. Then we can click on OK. So here we have another option retain null values from the source as null values in the data flow. So what it mean? So when we check this checkbox then if we are getting any blank or zero length value from our source then it will be converted to null. As we can see preview over here then we can see our where ID is 8 then name value is blank. Once we checked this checkbox then we can preview then this blank will be replaced by null and once we load this data in our SQL Server table then we can see null will be loaded instead of blank. Now we can click on OK. We have done our source part. Now we need to make the connection for destination where we want to load the data. So we need to select OLEDB destination. So this is our OLEDB destination. Now we can see we have two pipelines over here. First is for this task will be completed successfully either for failure. So we want to pass this data from this flat file source to our destination. Now we need to make one more connection for this destination. Now once we double click on this then we can see it will ask for OLEDB connection manager. Now we need to create one more connection. I have already created the connection now I am going to delete these connections and create new one. So once we click on new it will open this window. It will ask for server name. Now we can select our server. So here we can see this is our server name and here we need to select the database name by which we want to connect. Then it's SSI status where we want to connect. Now we can check the connection. Connection succeed then we can click on OK. And then we need to click on. So connection has been made. Now we can enable to access the tables. So as we can see we can scroll down and we have flat file to database table. Go to on mapping. So mapping succeed because the input column names and output column names both are same. So it will be mapped automatically. Now click on OK and execute our package. So we can see 8 rows and our package completed way successful. We can stop this package and go to our SQL server. And we can execute our now we can see now we are getting the data and for ID 8 name value was blank in our flat file. So it has been removed by null. So I hope you have understand how we can load the data from flat file towards SQL server tables. If you like this video please subscribe our channel to get many more videos. Thank you so much for watching.


I talk a lot about numbers and statistics, but as the Director of Yale's Prevention Research Center put it in a recent editorial, to reach doctors or fellow colleagues may we need to put a human face on it all. We have known for at least a decade that the leading causes of both premature death and persistent misery in our society are chronic diseases that are in turn attributable to the use of our feet, exercise, forks, diet, and fingers smoking. Feet, forks, and fingers are the master levels of medical destiny for not just thousands of people on a one occasion, like a tsunami or earthquake, but the medical destiny of millions upon millions year after year. We as doctors, a medical profession, have known. Ornish published 23 years ago, but we have not managed to care, at least not care deeply enough to turn what we know into what we routinely do. Were we to do so, we might be able to eliminate most heart disease, strokes, diabetes, and cancer. But saving millions of lives is just a number. He asked doctors to forget the bland statistics of public health and ask ourselves if we love someone who has suffered a heart attack, stroke, cancer, or diabetes. Now imagine their faces, whisper their names. Recall what it felt like to get the news. And while we're at it, we can imagine the faces of others, imagining beloved faces. Now imagine if 8 out of 10 of us wistfully reflecting on intimate love and loss, on personal anguish, never got that dreadful news, because it never happened. Mom did not get cancer, dad did not have a heart attack, grandpa didn't have a stroke, sister, brother, aunt, and uncle didn't lose a limb or kidney or eyes to diabetes. We are all intimately linked in a network of personal tragedy that need never have occurred, which leads to what he's asking doctors to do about it. Put a face on public health, every chance you get. We're talking about heart disease and its prevention of cancer, diabetes. Ask your audience to see in their mind's eye the face of a loved one affected by that condition. Then imagine that loved one among the 80% who need never have succumbed if what we knew as doctors were what we do. Invoke the mind's eye, he advises, and then bring a tear to it.


Thank you. Hello everyone and welcome. My name is Abraham Almarek. I work on hue and I'm a contributor on scoop We're here to take a look at what you can do and how we can use it as a starting point for exploration and real-time interaction with the dupe so So what we're going to start off with is Describing what hue is and then I'll pass the torch to my colleague remain here and he'll give a demo of How to use it? Okay, so what is hue well hue is the Hadoop user experience and that's a it's actually a picture of the hbase browser within hue How many folks are familiar with Hadoop? Show of hands. Oh just let everyone yeah, so Something that we focused on a little bit over the past few months is Making hbase a little bit more usable and looking being able to like really interrogate your data look at your dad and even Mutate some of the cells in hbase, and this is what it looks like so hue sits on top of The what I would consider the Hadoop ecosystem So that means beeswax hive server to Paula HDFS is a job track that map reduce scoop to hbase pig Also, something we'll throw in there is search clutter at least A search service which we also have a user interface for so hue is essentially a collection of apps and we have an app for each one of these services so if you could imagine that's Hive you have you have a query that you need to execute Sometimes we'll use the command line interface or generally you would use the command line interface, right? Well hue exposes as an interface that makes it really easy with syntax highlighting etc Which which romain will show in a little bit what that looks like which makes it really easy really nice And you can even save your queries Okay, so why a UI for Hadoop well So I would imagine that most people are not engineers a and the power of Hadoop can be used by just about Anyone ranging from data scientists to students to people who are completely new to Hadoop So if you're an engineer who knows nothing about Hadoop or big data and you're very interested in learning more about it Well, I imagine that hue would be the perfect system for you to get ramped up on Also, if you're a data scientist at I don't know Facebook or maybe a startup and you have a lot of useful data hue has Hue has many apps that would assist you in that also all of these apps are in some sense Interconnected and it makes it really easy to use all of the components or excuse me many of the components in the Hadoop ecosystem So as you can see It makes it easier to use the power of big data So what is an app in hue? Let's take a step back and think about what applications are in general So if you look at your phone, I don't know if you have an Android or an iPhone It doesn't really matter. There's a concept of an app or an application It's an entity within your within your your phone that allows you to do things performs a certain function That's exactly what an app is in hue There's like I said earlier. There's an app for several of the components in the Hadoop ecosystem HBasebueze scoop high pig HDFS also it provides kind of Cohesiveness and makes it a lot easier That's kind of the goal by the way of hue to make to make it much easier to use the components in the Hadoop ecosystem So how is it how is it built? What what does it look like in terms of the internals? So It's your typical web application in some sense. We strive for single-page layouts So that means we have a very heavy front-end we use knockout.js jQuery rowdy.js Also, we have Mako templates, which is specific to Django So on the back end we use Python and Django. It's a Python and drink Django driven website So the selection of Django is actually quite interesting The apps that were mentioned previously are actually Django apps or extensions to Django apps So it's it's very easy to build your own apps all you have to do is Run this command in the command line build and then hue create desktop app followed by the name and It will create a pre-defined template that you can just fill in the blanks and you have your own app Has anyone here built a Django application before just a quick show of hands So it's it's rather easy This the ramp of costs are very minimal in my opinion So should be I would imagine no problem So going back to the back end real quick For the web servicing layer we use spawning or cherry PY. It's one or the other These also handle all of your static resources such as images CSS files and JavaScript files also, we have client interfaces for many of the Hadoop components such as HDFS web HDFS and the job tracker So what other batteries are included? Well, there's an LDAP back end what that means is that you can authenticate using an LDAP an LDAP server or We have an OAuth back end We also have spin a go and it's very it's very extensible. So if necessary you can write your own very easily So authentication and authorization lives in hue What that means is users and groups and how you access apps within hue All live in hue hue has its own database and within that database we have a Set of users which you can import from LDAP or some other system and it'll be managed within hue As said earlier resources are serviced through hue we support multiple browsers and multiple databases That means a sequel light my sequel postgres and Oracle Now we're also internationalized in over eight languages and one thing I really want to stress is the Kind of cohesiveness of the apps. Excuse me not cohesiveness the word. I'm looking for I believe is how will the apps integrate with each other Let me give you a quick example. So if you could imagine you have a MapReduce job that has executed and it creates a bunch of files on HDFS Then it's really easy in hue to just click on the the job output and go directly to Those that output in HDFS It becomes super easy right that that would be kind of kind of difficult in and of itself You would have to go through the the normal job tracker UI, etc But within hue it's just one click So we really strive for the user experience to be improved and that means the data scientists the newb I mean even developers developers like myself. I like using it because it makes my life much easier see So hue has over 2,000 commits. It's an open source project You can download it from get hue.com We do a lot of video tutorials on there as well So if you need kind of an introduction on how to do things in hue how to work with all of the different components That's all on get hue.com It's a pretty mature project. It's about four years old and There's a lot of there's a lot of supporters There's there's there's a lot of there's a lot of people in the community. Our mailing lists are just So many people comment on the mailing lists. It's it's absolutely wonderful and so without further ado To see all of this in action. I'm gonna pass the torch to my colleague Romain. Thank you So the best way to show the product I mean one of the best way is to show it in action So I'm going to show like a demo starting from some help data of review reviews of restaurants And so you can use ad-oop with you on values query tools like pig-eye to process get some insights Into the data So I'm going to switch To a browser. So when you log in to you, that's the web UI. We support different backends You can use held up or or or even build your custom authentication mechanism Here I'm just in the file browser. So people are a bit familiar with ad-oop But that's the file system where you can see all the files And so you can bruise like a regular browser like going the files Use like quick navigation. We support the HDFS trash so you can even restore like files like in a real operating system You can directly preview images of files or even PDF without having to download them then look at the Look at them on your OS So I'm going to upload a Yelp data file So this is a Yelp data sets challenge. I'm just taking one of these file So in one click I can upload it on On the HDFS so directly for my browser And here is a file so for the demo is only 200 megabytes As you can see it is adjacent files if we if you go on the website You will see the schema of the website basically a series of votes for one restaurant with the final writing on the bunch of text for the review when you upload I want to process data. Most of the time you need to do some ETL processing and clean it a little bit So here I'm going to clean a bit escape some Parts of the text and transform it in a like TSV format to make it easier to be processed by ice pig in Pala So for doing this I'm going to use the pig editor So pig is one of the query tool for Adoop. It's a bit lower level than I've so in pig if you are familiar with pig, so We provide a nice editor with syntax alighting on auto-completion on the HDFS of pass then You can also auto-complete pig keywords if you do a mistake you want highlights, so you avoid typos auto-complot auto-complete, sorry, alias is so I can reuse my ASB And to save some time I'm going to load a script that is going to take in input the file the JSON file that I was I just uploaded load it with a JSON Loader which is native in pig and that I'm going to explode like the map of ratings put each vote into a column And do some string escaping on the text of the review a little bit of cleaning then just dump it So then I can just click it will run the script So it's submitted for for you submitted for you and will print the values logs The page refreshes automatically show you the progress You could look if you're familiar with my previous to the values my produces jobs created by the pig Same it's a lot. It is a live update with a one click to get access to the logs Which is pretty neat. You don't need to go in the common line then go on the Job tracker URL get your job get the logs. It's all into one click The pick script is finishing same we put links for example if I would click on the job was on the Job this script credit warm up reduced job. I could click and just go back to the previous page I was about you also put links on the pass HDFS pass So I was loading this file and dumping it as a CSV on the HDFS So I can just directly click on it See we're just run right now Again, I can just quickly look at it in the browser directly in this case. You see everything is CSV now So now I have a CSV file, but I want to do actual queries on it a good way to do is to use one of the Like main query tool, which is called hive So it's like SQL on top of a dup Before I Need to create a table that represents this data. So you comes with a wizard helping you To create tables for example, I just give like the name of Sorry like the name of the new table I want to create I just put one of the file that I want to import So you is a bit smart and he will give you like a preview or what's the new table would you look like?  You have like some nice shortcuts. You can Provide a list of column names and they will be already pre-filled Sometimes you need to double check the types of your data And I have two sample of rows that let you evaluate if your schema looks correct or not So then I create a table And I can still use like the table browser app in you to look at my tables Look at manually credit table even look at a sample of the data. I can quick sort the sample of the data Go or go directly to another column. So we try to improve the horizontal scrolling When I have my data I'm switching to the hive editor Which a little bit like pig provides syntax alighting we get The tables in your current is selected database or you can switch between databases. So Here are the fields in the review table So for example, I could get the text I'll just get five record Then in one shot I can semi-speak. It will submit the query report the progress On the left. I have a link to the map you see jobs And here is the data we support values Like all the work on table with all the hive properties, so if I want I could upload and use for example custom udf So I can add a bunch of jar in my script Then define a function My upper so we have a blog tutorial about this one But it basically will create a new to upper that transform to upper string text So I'm going to apply My udf on this one On the executed on same well the life progress you can share query you can save queries and Like you see you can upload Use udf quickly a nice example if you use go back to my More interesting example if I want to get the top 10 coolest reviews of restaurants That would be a script I could also use Impala so Impala is like a super fast hive which is developed by claudera I'm going to show you like the same query ought to get the top 10 coolest restaurants I'll show you the difference of speed So here I'm executing the hive Here I'm executing the Impala But I think I forgot to create it Okay, so that's a good story. I'm going to switch on this one and Yeah, so because Impala is not on this machine So yeah, so that's a quick and easy way you always use the lot for hive Now we have support for the new high server 2 on a new product called sentry That has full Provide full security. So that's for hive. So a nice query tool then because it's no SQL I'm going to demo a little bit the new HBase table So many people are using HBase But we added a new UI for example We can list tables in various clusters And provide a nice Search which is just searching on top of HBase So we provide This is a new table where for each restaurant. So the raw key is Idea of the restaurant or the name For each day. I'm going to have the average rating of these restaurants. So day by day. I will have contours that shows The average So you can scroll you can also horizontal scroll so HBase is like a Sparse sparse database. So we have a nicer layout where we can aggregate You can inline edit Data directly without having to deal with HBase API or in the shell We have without work cutting table or adding new rows There's a cooler sparse part of this app is really the search bar so we provide an Autocompete of the rookies So for example, I could select Apple Cafe He will directly pick up the records and do a scan I want the five next rookies after I put this rookie I will just add plus five then I will get five more data. I Can use column filtering? meaning if I want to have only for the month of March only data I Could also do I want from March to end of the year So put a hierarchy and to show I could also do until July So let's search into HBase directly You cannot brought sorry you cannot put data We can visualize Different type of data. So if I go in the cell editor, I will see the history of my cell I could upload pictures or any binary data showing up on the window and Yeah, so for people who want to have a quick look at their data we support also Avro You can preview PDF like in Firebrowser But so pretty nice app and everything is standard and just sits on top of HBase You just need the street service in HBase to be running then you just point To this URL and you will the table will just show up in your browser and then I'm going to finish by the last app which is a search app. So I have all these reviews on ratings I could use solar on top of HDFS to ingest all of my data Define a schema of the fields think like Google search And then search for my ratings. So by default I created a simple schema with metadata For example a bit like the I've table with like a bunch of ratings a bunch of text But I want to search for it and I want to customize a bit the look and feel So that's all the fields in my index But I'm going to show that we can create Something a bit more good-looking for example, I want to add The text of the review we show a preview of what the actual search will look like We could add for example also the rating Which is called stars on the left On go like use a we see weak editor throughout some building we support HTML at the end is just a HTML snippet of code for each result If you want to do advanced styling you can insert a custom style or you can custom javascript You can add solar facets. So facets as what you see sometimes on the left if I want to have Or like in Amazon like range categories of job of products or ratings You could add for example like a rating Facet Then I go back to my search and you see now I have the customized view if you plug it more with it Here is an example of template you could create With like the I click links to bubble maps. I Want to just rest around with five stars. So we'll put a filter on this Like I will look for sushi for example so I Like the search term so you can build your custom search engine and it depending on your use case It's a pretty powerful What to do many people knows SQL even more people knows how to search So that was it Excuse me. What are the three different colors? So here in the yep for each review people can Write if the review was helpful They can then they can say is the restaurant was cool Is the atmosphere or restaurant was funny? It's like we put some colors to show them up a little bit more But you could put your style your colors what you want So get you that calm we have a list of videos that shows in more details what I just presented and I'm also blog post On Yeah, if you have any question feel free to ask them now or follow up or on the user list. It's pretty active. Thank you


More than 2,000 crews and ballistic Russian missiles downed over Ukraine. Ukraine's armed forces shot down more than 2,000 crews and ballistic Russian missiles since the beginning of the full-scale invasion the Defense Ministry reported. Another two ballistic missiles launched from occupied Crimea were shot down over Kiev on March 25 following multiple explosions according to the Air Force. Debrey fell in the Solomiansky and Pechersk districts, damaging a three-storey building in the latter. At least two people have been reportedly injured and eight others suffered from shock. The rescue operation is ongoing. Moscow has launched over 8,000 missiles and more than 4,600 drones against Ukraine as of February 22 since its all-out war said Yuri Inat, a former spokesperson for Ukraine's Air Force. That's the result of the titanic work of Ukrainian air defenders. Thousands of lives were saved by modern air defense systems provided by our partners. The Ministry wrote on X reporting on the destroyed targets. Over the past week, Moscow launched several large-scale attacks on Ukraine's cities, including energy infrastructure. Around 190 missiles, 140 Shahead-type drones and 700 aerial bombs were used, President Volodymyr Zelensky said. Local authorities reported multiple injured and killed civilians. Ukrainian officials urged Western allies to provide Kiev with more weapons, including air defense systems and missiles, to protect Ukrainian cities from regular strikes. Russia has launched its third wave of missile strikes against Ukraine's capital Kiev in five days as part of its escalating aerial bombardment of the city. The injuries and damage appear to have been the result of falling missile debris, as the Ukrainian Air Force said it had shocked down two missiles over the city. Ukraine's President Volodymyr Zelensky renewed his appeal to Western partners to provide more weaponry to protect against the unrelenting aerial attacks. We never tire of repeating that Ukraine needs more air defense, he said. This is security for our cities and saves human lives. Russian opposition in exile leans towards supporting armed resistance against Putin. More and more Russian dissidents in exile are beginning to advocate for armed struggle against Putin's rule, though not enough have embraced this tactic as of yet. Three Russian resistance group members said. There's definitive support from the emigrant centres, says Alexei Baranovsky, a volunteer fighter from the freedom of Russia Legion. Only in the third year of the war did they start to consolidate and provide assistance, though largely delayed, it's there. It would be helpful if recruitment for anti-Putin resistance groups could be expanded across Europe, according to Baranovsky. For now, only the Civic Council, Russian anti-government NGO based in Poland, is actively working with the Siberian battalion, he said. Where's the rest of the opposition in this regard? The direction is right, but we hope for more significant results. We're pleased with the trend we're seeing, said Kold, a member of the Siberian battalion, another Russian resistance group. More and more people realise that, unfortunately, armed struggle is the only way to change power in Russia. Putin's special military operation has made this clear to everyone. In the current climate in Russia, advocating for fundamental human rights, like personal choice, freedom of conscience, religious practice and inherent personal freedoms, demands a more assertive approach. This necessity arises as peaceful methods have unfortunately not achieved the desired outcomes. Kold explained that peaceful protest doesn't work in Russia, and as a result, his group contains people from a variety of walks of life, including professional soldiers, business people, mechanics and so on. At some point, they all understood that simply lighting flashlights doesn't work. Kold, referencing the numerous peaceful demonstrations brutally suppressed by Putin's regime, the stance of prominent figures in the Russian opposition, thankfully, began to change in the war's third year, said Denis Nikitin, commander of the Russian Volunteer Corps. They've realised that roundabouts, forums, petitions, paper cups and flashlights can't budge a despotic regime like Putin's. The opposition's efforts should be directed at supporting the fighting Russian units. I'm talking about the Russian Volunteer Corps, Siberian Battalion and Free Russian Legion, because we are at the forefront of the attack, the spearhead that must pierce the heart of the Putin regime and perhaps Putin's heart in reality. Nikitin claimed that Russian freedom fighters had gained the support of prominent Russian dissidents, including world-renowned chess master Gary Kasparov. There are people who understand that we are carrying out the only possible effective work, waging a war against the regime, Nikitin said. If they're not ready for the same form of struggle, they should either assist us in our fight or not call themselves the Russian opposition. Kremlin instructed to mobilise 300,000 troops for new offensive. In the near future, the Russian authorities plan to mobilise at least 300,000 troops to begin an operation to encircle the second largest city in Ukraine, Kharkiv, for informed sources told Verstka. The next in the plan is Kharkiv and holding the city. This is possibly only in the case of the encirclement. There is a shortage of 300,000 troops. Therefore, everything is ready for mobilisation. 2.0 said a source of the publication in the internal political block of the presidential administration. According to him, no one wants to turn Kharkiv into a second Mariupol, so there is an idea to make it a showcase of how Russians know how to fight in a civilised way. The need to mobilise 300,000 troops was confirmed by one of Verstka's sources from one of the Western military district offices. The source noted that recruitment to the front could begin as early as March the 25th. They'll try to call reservists to fight in Ukraine, men who are in reserve but have signed a contract with the Ministry of Defence on inclusion in the mobilisation human reserve. They can work at any job, but twice a year they are obliged to go to military training camps, which until 2022 were formal. There are about 2 million of them in Russia. Something is planned. Reservists are being recruited now before mobilisation. Whether it will happen or not, I do not know, but last time the procedure was the same. Says an officer of one of the military bases in the Zabei Kalskiy cry of Russia. Although military enlistment offices will begin work with those who have not signed contracts with the Ministry of Defence but are in the reserve of the armed forces. Including those dismissed from service, those who studied at military universities, those who did not serve for any reason, those who completed alternative civilian service and women with military specialties. Military enlistment offices have pasted mobilisation orders into military tickets of many of them previously. They will also try to send conscripts to the war, whose terms are about to end. The task is to sign contracts with those who are ending their service by April and those who ended service last year. Says a source close to the Presidential Administration. They will be persuaded by all means the interlocutor from the Presidential Administration confirmed. A new wave of mobilisation is also possible.


So first of all, thanks, Julia, for joining us to talk about successful women on International Women's Day. I know that you've been very successful at the UK data service. So to start with, I wonder if you could tell us about how you've been at the service and also all about your job? Sure. And I started in October 2019. And so that puts me at not quite a year and a half with the service. And what I do here is I lead the computational social science training team, which is a mix of some research, but mostly sort of ad hoc webinars and workshop presentation on topics of computational and empirical social science. Essentially, we're trying to get people trained in social science to put down their clipboards for a moment and think beyond traditional social science methods into things like maybe web scraping and machine learning or sort of combining the surveys that they're used to using with things like real time wearable data collection and things like that. Just sort of somewhat bonkers things, maybe fill in some of the gaps that are not necessarily very attainable with traditional methods, but that are more attainable now with high powered computing and sort of fully interconnected world of digital nonsense. Is this how new is this kind of approach to using data this way? Is it quite a kind of a new and up and coming way of doing things? It's definitely up and coming, but I wouldn't say it's entirely new. I'd say it's been a bit slow to get off the ground, but it's now gathering speed. And I think that's because it's quite diverse. There's a lot of different ways that people can use computational methods or sort of empirical data collection in social science. And there hasn't been anyone yet who's put it all together as a new category. And so that's that's a development that's helping it gain speed. And is the idea to bring different kind of data sets together so you can buy and look at them across the board so you can have the most impact on changing policy or people's lives? It's certainly one way is that if you gather multiple different kinds of data, you get new perspectives on established problems. But there's also just new problems that we have never been able to address before that we can only address if we have, you know, maybe computers to deal with millions of comments on a, you know, newspaper article or something like that. That wouldn't have been possible before we had sort of computational assistance, you know, and then different kinds of methods and approaches allow you to either address old questions in new ways or with maybe new support, new insights or entirely new questions and some combination of the two. So sometimes in addressing an old question, we discover a new question or a new avenue or maybe a new perspective. Well, that works out as very interesting. And as we said before, you've been very successful in your career so far.  That's a little bit tricky. I think all of the challenges that I faced are kind of the slippery cultural challenges that are hard to document and hard to discuss. So things like discussing over lunch, maybe with colleagues one day, someone had talked about quotas, gender quotas for new hiring projects. And he commented that he couldn't understand how women who got hired wouldn't second guess their ability to do the job on the basis they would think. Well, I was only hired because of this quota. And I said, but if there's no quotas, how does a man who gets the job not think, have I only got this job because I'm a man? And he'd never even considered it. It just never crossed his mind. And it's just that kind of little cultural issue that people. It was nice that we could have that discussion and feel free to talk about potentially tricky things. But it shows that in a lot of cultures, people have never really critically applied some of the experiences that women talk about. Do you think there still is a kind of unconscious bias? Or in that example, there probably was unconscious bias. And is that have things moved on since that point? I guess so I think certainly some some offices and it's it's down to good leadership that individual offices can encourage people to reflect, to be curious about the lived experiences of others and to look into actually, is anyone feeling marginalised here? How can I address that? And others are really focused on, you know, productivity and achieving goals and doing things. And therefore they they don't. Some leaders don't make an effort to ensure that people reflect on potential challenges in the workplace. And I want to come across that comment there, which is out of blue because I know that International Women's Day is all about choose to challenge. So do you think it's important and those situations to choose to challenge and not just accept it? Yeah, it's difficult. And you have to really pick your battles. In that case, I did feel that I could challenge him and I could say, well, you know, don't men feel the same kind of self like imposter syndrome kind of self doubt, but not everyone would feel comfortable challenging people and certainly you might feel comfortable challenging your close friends, but maybe not your boss or you know, there's all kinds of times that are appropriate for a challenge and times that are not. And I think it's important for people like me that are perhaps a bit loud and shouty to challenge more often because the people who are a bit quiet and retiring won't challenge. And I don't know. So far, challenging has not come to any harm for me that I know of. So it's good as well to demonstrate that sometimes challenging at least sometimes doesn't doesn't have any negative consequences and potentially has positives. So is it important that younger women coming into male dominated environments or in any of our really do feel confident that they can speak their mind? Is anything you'd say to younger people to encourage them to find a way of establishing equality in their workplace? That's quite difficult. In an ideal world, I think, yes, everyone should feel confident to speak their mind. But that's the kind of thing that you get through experience. And so far, a lot of people are still experiencing situations where speaking their mind has negative outcomes for them. So I think rather than encouraging women to gain confidence and speak their mind, I think it would be helpful to enforce policies where everyone is allowed to speak their mind. You know, I think I think there has to be a bit of a cultural change that involves both bottom up and top down kind of demonstration of change. So is it this sort of work to be done on that kind of approach and cultural change in organisations? In societies as a whole, yes, I think. I've had the good good luck to work in a few different spaces where there was much less problems than I could see other people having. For example, where I did my PhD was at a university that I think was some departments were 90 percent male and 10 percent female. And one of my friends who worked in that department, she really struggled with people being dismissive of her and critical of things that she did in ways that they weren't critical of male colleagues. And she didn't feel like she had a lot of support in that context, whereas I was in a department that was about 60, 40, still favouring men. It was an engineering university. And but I felt like I could say what I wanted to say. And I felt like there weren't negative consequences for having a differing opinion. So it fits and starts. Some areas are moving faster than others. Actually, you won't have all the answer, but do you have any ideas on how they could how things can change? What policies could be put in place? What makes this more we call it? I think as a data scientist, I'd like to see some more data, some more research, empirical research specifically, and possibly computational research done on the effectiveness of different interventions, because I think a lot of people are keen to point out the issues and they're keen to write up reports about where things are now and they're less keen to go out on a limb and try new things, even though it's only by trying new things and seeing what happens that we can really learn effectively what works and what doesn't. But there's a there's a real unwillingness to try things that might go wrong and might have negative consequences, rightly so. No one wants negative consequences, but you have to you learn by failing rather than achieving perfection the first time. That's really interesting. So anyway, we this sort of all works. We don't on looking at the data or what what works, what doesn't work and then come up with new conclusions. Yeah, well, I think quite rightly people are unhappy to think that they're being made into a human experiment. Nobody wants to be the guinea pigs in a big social experiment. But if we never do any experiments, then we don't really gain good data of a variety of potential interventions in different cases. So, you know, I don't know. I don't have a good solution on how to convince people to be OK with being experimented on, but it must be done.  And I mean, the emphasis should be on the institution to still that safety. Yeah, but exactly how to instill that safety is not clear. And so, yeah, we've got to just try a few things and see how people feel. And, you know, it's it's difficult and it's uncomfortable, certainly for institutions to admit that they haven't been doing the best that they could have been and that things that they had made perhaps dismissed as possibilities are, in fact, quite effective solutions. But, yeah, try and see, throw it at the wall, see if it sticks. Yeah, so it's good that we're aware of these things and we're thinking about it and it's been improvements. But just how this conversation makes you realise that there's no definite, there's no definite solution out there yet, even though we all talk about it. Yeah, yeah, I mean, there's there's we definitely don't know how to fix these problems. And that's not surprising because they're complex problems full of, you know, years of cultural evolution and people who are at best erratic and sometimes downright, you're like, on an un unintelligible, you know, we don't know what people are saying, even much less why they do the things they do. And it can be quite difficult to try and imagine how solutions will go. You know, every economic interventions, for example, often have unintended consequences and we have to expect that some of that will happen with gender policy change, things like that as well. But yeah, if you never do anything, you'll never get any consequences. We'd like to thank Julia today for her time in talking about her experiences as a woman in the world of data, especially for International Women's Day. And we'll be back again soon with some more podcasts from experts.