Here's your host, Jeff Brick. Hi, Jeff Brick here with theCUBE. We are on the ground at San Ramon, California at the GE Research Center. They've got 1,200 people here working on all their software projects. It's a pretty interesting use case where it's an integrated center. It crosses all the lines of business, but they came out here where the talent is and we're really excited for our next guest, Ganesh Bell, the Chief Digital Officer, GE Power and Water, welcome. Thanks, Jeff. Thanks for having me on the show and thanks for visiting us. Absolutely. So we love coming out. We got to go visit the lab the last segment, so a lot of cool toys down there, but Power and Water, that's pretty basic infrastructure stuff. So what's happening in the real world? We hear about it all the time, industrial, internet, internet of things, the real world of power and water. Sure, so in Power and Water, our mission's very simple. It's about providing every citizen on the planet with affordable, accessible, sustainable power, reliable power and water, right? So GE solved some of the biggest problems in the world. It's about infrastructure, big, big infrastructure in all of our industries. Healthcare, energy, aviation, transportation, we solved some of the biggest problems in the world. And in the increasing day of where everything's getting software defined, to solve the biggest problems in the world, you have to solve the whole problem. And the whole problem now means hardware, software, services, the full customer solution. And that's why we're doing this thing that we call industrial internet. Yes, it's enabled by technologies like internet of things, big data, cloud and everything, but it's really about solving customer outcomes. And for us, in Power and Water, that means how do you increase efficiency, right? And how do you actually produce more power, use less fuel, and optimize your operations? So there's a lot of examples. We have it across our power industry, where whether it's running gas turbines, for example, if you just save 1% of fuel, just 1% of fuel across the entire gas fleet of GE alone, not just our competition. We can save about $65 billion of customer value over the next 10 years. So when you think about ideas with 1%, right? So those ideas are happening. We've been connecting these machines back to us with our monitoring and diagnostic centers for the last 10 years. And we've got about 10,000 people who do software in GE, but now we're connecting them to the people and the business processes at our customers. And that's really where the value is coming from. Talk about how different that is, because I think a lot of people are confused. They're like, of course these devices have been connected, and of course, there's sensors on all this industrial equipment, and of course people are paying attention and there's diagnostics on maintenance, et cetera. So how's it really different now? What is the promise that everybody's talking about that's so different than where we've been? Yeah, so let's look at a simple example, right? Let's look at a wind farm, for example. We've had sensors on these machines, on wind turbines for a long time. We've collected everything from vibration data, material, temperature, external air, weather data. But we've been collecting this data over a long time just for improving the machine's performance from a design perspective, right? Because we've been collecting, say, 10 minute interval data. Now we can collect 10 second or a second interval data. And the number of sensors we can put on these machines have gone up, because we can do that. In the past, we couldn't do that. In the past, we were collecting this data. You're right, we're always collecting there, but we're using them for how do you make better machines? Now, in the last several years with GE, we've been collecting this data in real time and thinking about how do we optimize the performance of the machine? So we have this idea that we call the digital twin. For every physical asset in the world, we can create a virtual representation of that asset in the cloud. We call that the digital twin. So not digital twin is continuously being enhanced by the real time operational data, so we can actually improve the performance of those machines. And in our wind farms, for example, about we did this two years ago, we had a product called PowerUp. It was just through software, we could improve the performance of a wind farm or wind turbine, actually, a specific wind turbine by 5%. That's 5% more electricity from the same wind all through software and data, and it's been a big success for us. Our customers are using it across our products, and this year, we actually went even bigger. We launched this thing called the digital wind farm, where it's not just about the individual turbine, but across the entire wind farm. And we can now map every single location in a wind farm and build new modular wind turbines, and we connect that to the power of critics and the industrial internet. We can drive about 20% more electricity from the same wind farm. When you look at the life of a wind farm, that's about $50 million over the life of a wind farm, and that turns into huge, our customers market opportunity and value for them. Right, so you can put more on, you can measure more often, but then you can really take the systems approach rather than the individual maintenance, and that's really where the big, big efficiency comes from. Absolutely, so our customers have been asking us, you know, we didn't wake up and say we're going to do software because that's a new market. We've been trying to solve these whole problems. In the past, there was this machine level problems. Now, customers have been asking us to solve enterprise business level problems, reduce my unplanned downtime, right, optimize my operations, and also, how do I take advantage of market data? So for a lot of utilities and energy companies, there is market demand for energy and they have to take advantage of that market pricing. And to do that, they need to know how much can they push their machines, throttle push their machines to be more precise than a gas turbine, which means you're burning through parts, but we can do the economic analysis and tell them, yes, go ahead and throttle push, capture that price in the market, and you can take your planned downtime at a later date or an earlier date, and we can optimize that entire cycle for them. Really about optimization, and optimization that makes sense not necessarily what you think makes sense at that particular moment in time. Absolutely, and you asked the question of what is, why is it today and what is possible? In the past, we've had small set of data, but even the data get went unanalyzed, right? Most of it is what we talk big data and dark data, most of it is dark data, and now you can analyze it. So with all the digital twin and the power of critics, we can do a lot of these what-of scenarios, right? Because we have the ability to continuously keep doing the what-of scenarios and find the right optimization point, whether it's at the asset level or the business operations level or ultimately at the market performance or the business performance level. And on that digital twin, it's not a digital twin of a prototypical turbine. It's that particular turbine with its characteristics, its maintenance history, its nuances. So you're really A-B testing that particular unit, not some hypothetical something that came out of a lab or the fresh pristine thing that came off the factory floor. Yeah, it is when I say digital twin, it's actually the right fingerprint. It's the right fingerprint of that digital twin. And it is something that continues to live on. And it's something that we design day one. When you design a wind farm, you design a digital version of what that wind farm's gonna look like. So it's given birth at that time, but it continues to live on as the wind farm continues to operate. So let's shift gears a little bit. You're a widely veteran of software and enterprise software. Talk about how within software eating the world and we know that trend's coming, that's really changing inside a big old industrial company like GE who we think of as making big things, locomotives and power stations and dams and turbines and airplanes, but really software is a growing, growing, growing important asset within the base. And then to set up this center, 1200 people, everyone together across business units really focusing on software, software, software, heavy duty research and then getting it into the product groups. Sure, my position is actually just that, which is I'm the first ever chief digital officer in GE. And if you think about that, I've been in enterprise software and startups in big companies and when cloud and big data and SaaS and all the stuff is happening, I used to talk about why the next big software company isn't a software company. And people like, what does that mean until Mark Anderson wrote software eats the world? We didn't have great words to say it, right? But it's really about how companies like GE can use software. It's not just software, but software-driven business models, right? So companies like GE can do that. And which is why I came to GE, because if I believe that the industries can be transformed elsewhere, right? That's where you would be. But the industries have to be transformed by somebody who knows the industrials. And that means it's companies like GE who can actually transform the industry. When we talk about digitization, you know, in entertainment, yes, it's happened. In travel, it's happened. In consumer, it's happened. But to transform the industries that GE is in, whether it's transportation or energy, it really takes the domain expertise. It takes the knowledge of the machine. It takes the physics. It takes the operations. And then you combine it with software. We already had software people. And GE, we've had about 10,000 people doing software all the way from the machine in the controls layer. But when we saw the internet of things and cloud and big data, we knew that this requires a purpose-built approach, right, a purpose-built platform to go capture this opportunity and connect it to the connective tissue back to our core machine and industrial expertise. Which is why we set up the center here and we've had great experience, you know, recruiting about 1,200 people in the valley and people want to work on these problems, right? These are more important, bigger problems in the world to go solve. And to some extent, they're more sexy than consumer problems. And you come to GE and you can solve those problems. So let's shift gears a little bit. We cover the MIT Chief Data Officer and Information Quality Symposium. We've been there for like three years. It's a great show. And it's a very interesting concept of what is a chief. One of the things that always comes up, what is a chief data officer? What do they do? Who do you report to? What do you think is that role? Is it you're managing it? Are you an advocate for data? What exactly is your role and how are you executing here at GE? So there's two things, right? We always talk about CDOs and there's chief data officers and chief digital officers, right? So I'm the first chief digital officer. So I have the data officer reporting to me for all the power and water. To me, the role of data officer is very simple. It's somebody who looks at data as a strategic asset and looks at all of the possible use cases from the data. We have people in our businesses who look at all of the operational data from our machines, from our customers' operations and look at what are the things we can glean from that? Whether it's when do you plan outage time? How do you illuminate unplanned downtime? How do you increase efficiency and so on? So you look at a lot of those outcomes. When you think about digital officers, you're really thinking about building a digital business or transforming a business into a digital business. And the way I think about it is very simple. Every business, we all go to work to create products and services that we market, sell and engage with our customers. And many industries are more digitally advanced than others, right? And some of the industries like energy are not as digitally advanced as, say, entertainment or travel or so on. So when you look at those industries, you got to look at how do you transform how you work and the products that we build and how we engage with our customers. And in most companies, the digital officer just focuses on marketing and customer engagement and that's a very shallow level definition of it. To me, as a chief digital officer, you are the CEO of the software business. So you're looking at, we were being in the hardware business, we also have a services business, but now we're building software businesses focused around the verticals that we are in in all of the industries. So if you're an industrial, we have a platform for the entire industrial world with critics. If you're an energy in power and water or aviation, we have software to help you run your assets and your operations better. So my role, I think of my role as, my peers are all CEOs of the hardware and the services business. My role, I think of as running the software business for the energy. So it's about digitally transforming one of the largest industrial business in GE, which is power and water. So one of the examples that gets stated all the time, GE always, right, they're always talking about how much data gets thrown off by a turbine engine on a cross country flight. Often cited, data pointed at big data shows. One of the things that comes up is when is the day that GE will not sell, and you can pick the water and power example, not sell the engine, but just sell the transportation and they manage the engine. So really transforming that. So now you're selling transportation as a service, propulsion as a service, thrust as a service, rather than actually selling a piece of metal. I think it's a great question. Every market evolves that way. And we've already had examples like that, right? Which I talked about power up in our wind business. We could have just sold software licenses to power up, but we saw an opportunity where power up was about increasing the output. So we went to customers and sold it as a subscription. Effectively, we sold megawatts as a subscription, if you will, right? So we're seeing that opportunity and that's the big advantage of doing this in the context of industrials is we have the, I would say, the X factor, if you will, to innovate on the business models. So it really is on how customers push us. We are delivering, I would say, outcome as a service, in most cases. Customers care about enterprise level outcomes. So all of those are possible ideas when you really do the digital transformation of a particular industry. I like an outcome as a service. I hadn't heard that one. So Ganesh, thanks for stopping by, interrupting your busy day. Thank you. Thanks for coming on. Absolutely. I'm Jeff Frick. You're watching theCUBE. I'm Sandra Mone, California at the GE Research Center. Catch you next time.


Hello everyone, welcome to the NPTEL course on groundwater hydrology and management. This is week four lecture four. In this week we are continuing to discuss the important components of groundwater hydrology and to continue our discussion we will look at hydraulic conductivity which is one of the most important parameters for groundwater hydrology. In the previous lecture we discussed about the macroscopic view and the microscopic view of groundwater flow and how the water interacts and moves according to the material the soil or the rock material. So in the macroscopic or which is called the Darcian view you have a average linear velocity of the water particles because in a discharge queue which goes around a cross-section area A the macroscopic view does not capture what is happening inside. So how does water move inside is not captured and not needed to be honest because in a macroscopic view it is averaged. Okay so you have an average linear velocity so the velocity is assumed the same on the top middle and bottom portions of the soil. But in the real world which is a microscopic view you see on the right hand side you understand that the water doesn't go in a straight line as we see here in the macroscopic then water would go in variable velocities along the soil medium and this is because of the heterogeneous composition and arrangement of the soil particles. So you need to understand that there are microscopic differences and variable velocities. However when you want to describe them in a particular equation you need a macroscopic view. So that's what Darci took over and he modeled this groundwater flow through a pipe. So he didn't want to do it on the land and ground because of the microscopic variations he did it in a lab setting. So Darci found experimentally that the discharge queue which goes into the soil medium is proportional to the difference of the height of the water h which is the hydraulic head and the difference we also call as gradient between the ends and inversely proportional to the flow length which is the length of the soil column through which the water flows. So you have q is directly proportional to h a minus h b we will see what h a and h b is in the experimental setup and q is inversely proportional to the length of the soil column. Before that just to introduce Darci Darci was an engineer in France where he worked on fountains water fountains and he had to supply water through underground to make sure that the water fountains were working properly. For that he didn't have any equation. This was very very old times and so what he did is he took the soil column to his lab and then did this equation and till date it is one of the most accurate very simple but most accurate description of the ground water flow in a saturated system. The assumption is also that the soil inside is fully saturated. We need to understand Darci's approach because Darci was doing this to send water into the fountain and the fountain needed a continuous supply of water for which the soil and everything was saturated and through a pipe column. So we can look at the differences in his experimental setup and how he got it this slide. So in the Darci's experiment what we notice is that the column of sand stopper at each hand so either sand soil rocks materials the material the matrix is kept in a tube and it is stopper a rubber stopper is put on the top and bottom okay. Water saturates the pores water is fully saturated inside the pore space. Now apply yourself to the previous lectures. If we say porous space is fully filled with water that means it is a saturated system and also it means there's no air inside the soil. So this column of soil he took in a tube has full water inside along with the soil particles. How do you establish it? You just continue to pour water until for a long time until the water comes out in the same rate okay. So q is the input rate and if the q water comes out that means full it is saturated. Constant volumetric rate of inflow and outflow of water q so he also maintained q to be constant coming in and coming out and that is what was needed to operate the fountain systems. So he would send a known volume of water which is q which is already measured and meter through a soil column of length del l okay or just l and then he had a stopper and q coming out. The cross section of the tube was measured as a and he had two points of monitoring inside the tube and those can be visualized as wells okay. So he wanted to know what is the flow inside the medium okay. So to understand the flow he put two monitoring points and between the points he's going to calculate the well hydraulics and the hydrology groundwater hydrology movement. So what he did is the first well is called well a let's say a and then the well b is in a lower elevation and why does water move from top to bottom because water flows from high potential to low potential. There's no other fancy instruments here just q he's sending in he doesn't have to push it because gravity is pulling the water from high potential to low potential. So when you put in the well automatically the water level in the well will equilibrate based on the atmospheric pressure the pressure inside should balance the pressure outside and it falls into a particular level after it equilibrates and that is h a we saw in the previous equation h b is the same water level measured in the well b okay. So del h or the hydraulic gradient it would be your difference in the h and divided by the length del l okay. So what we have here is a distance between the wells and also a difference in the head. Also what we need is the elevation of the well inlet from the datum which is zero. So here the table as zero and from there how much is the elevation of hydraulic head one which is h a and hydraulic head b which is at h b. So he measured the elevation of the well z one and z two then he measured the total hydraulic head which is h two and h one. So h two and h one are going from the datum which is the zero elevation. So from the zero elevation you go up to the water level this includes the elevation of the well okay the elevation of the well is the point at which the well opening is there from the ground from the ground which is zero and the hydraulic head is on addition to that the water column height is added. So now we have the total potential okay. So h one is higher than h two and that is why groundwater will flow from h one to h two because high potential low potential okay. So it flows from high potential low potential. So q is given as the volumetric flow rate a is the cross-sectional area h is the hydraulic head which is including the elevation okay elevation of the well from the ground and l is position coordinate in flow direction okay. So how much in the flow direction at length okay. So Darcy's law is given as the first law what it says it is proportional to the hydraulic head difference which is del h del h is the difference between h two h one and h two okay and also proportional to the cross-sectional area directly proportional. It is inversely proportional to the length between the wells between the estimation points. So now any proportionality can be converted to an equation by introducing a proportionality constant right. So this proportionality symbol would go to an equal when we introduce a proportionality constant and that constant is called hydraulic head conductivity okay. So the hydraulic conductivity is given as k and a is the area del h by del l. So if you take the area out I'm not interested in the total volume but I want the velocity or the discharge and that discharge q is rated by the area. So if you divide by area on both sides you get small q which is equal to minus k d h by del okay. So this is a very simple equation but captures the flow of water between two wells and also the discharge rate. Now let's take a step backwards why do we have a minus where did the minus sign come right. So when we discussed it is like okay it is proportional and it is inversely proportional to the length but where did the minus sign come. The minus sign is introduced by Darcy to document that the flow is in the reducing direction of potential. So the negative is a directional value not a value that you put on discharge you cannot minus the discharge okay. So what do you mean by minus discharge there's nothing as minus discharge but the negative sign indicates that the flow is in the direction of reducing head okay. So it flows from high head to low head and that capture that needs to be captured otherwise how do you know which side is the water flowing right. So to document that Darcy had introduced the minus sign and it flows from high potential to low potential or towards the decreasing head the head is decreasing. So all of this is captured in a very very simple equation and to be honest working with groundwater systems for the last 10 to 15 years I've noticed that this equation has been most powerful even compared to the newer equations that come now in that. So there's not much additions done to this equation and all these groundwater models are based on this equation for saturated flow. These equations and descriptions can be taken from Fries and Cherry book 1979 as I said it is one of the very important books that groundwater hydrologist would have and all these government schemes are also referring to these books. So this is the direction of decreasing head so the negative sign please understand it is a direction value not a value that you put on the quantity it is not a decreasing discharge minus discharge plus discharge it is the direction flows in the direction of decreasing head. Also understand the difference between a large q and small q large q is the flow which is a volume and q meter cube or l times 3 on the power but your small q is a rate velocity kind of thing so it is a discharge and l by t so the time would come in this equation whereas in the discharge it is volume per unit time. Moving on we have a horizontal pipe filled with sand to demonstrate Darcy's experiment in a different book I've taken a different approach to just show the same explanation but in a more different way than the Darcy's book. So horizontal pipe filled with sand so there it was slanting but here it is a horizontal pipe to demonstrate Darcy's experiment so as the book says it is originally vertically or slanted position but we can also explain it in a horizontal way. Why do we need to explain it in a horizontal way is to capture the ground system most of our ground is not going to be always like this in the field. So for example in a rural area you want to monitor the ground water flow between two blocks and the blocks are going to be straight it's not going to be like tilted etc but inside the ground it may be tilted and that is because of the layering you remember the aquifers we talked about so the layers might be tilted or different and that causes the head difference. So what this diagram says is that there are two wells and two wells have different head hydraulic head so HA is the hydraulic head on well A and HB is the hydraulic head on level B well B. We cannot ask why is one higher than the other it could be different aquifers it could be different pumping regimes so even though it's horizontal the levels are different now the L is taken as the difference between the wells right and so the basic equation is Q is equals to minus K hydraulic conductivity A which is the cross sectional area times DH by DL DH is your change in the head or difference between the head from HA and HB and your DL is the distance between the wells. Now this DH by DL is called the hydraulic gradient and what you also notice is that the K the property K is the one which captures the property of the soil because area A is basically the area of the cross section H is the hydraulic head which is a water property and DL is the difference between the wells. So where does the property of the soil or the solid come it comes in K hydraulic conductivity. So the conductivity as I mentioned in the previous class is a term given to the soil and how easy it conducts or lets the water to pass through and so we call it hydraulic conductivity which is a property or a function of the soil. Moving on please understand there are lots and lots of units for this values it can be expressed as gallons per day per meter square or meters per day it's all in a L by T dimension which is length by time so if you divide the gallons which is a volumetric term by area you get one length and the day is your time so you can have multiple units so please be careful with the units I recently was in an examination where the student was trying to say no it doesn't agree the data doesn't agree but then the simple thing was that the units were different so make sure the units are captured correctly I am going to stress this again and again please make sure what the unit is and see if it is agreeing when you compare okay so let's look at some of the hydraulic conductivities in the previous lectures we looked at permeability porosity differences and now hydraulic conductivity if you look at it hydraulic conductivity is a function or can be expressed as a permeability and along with gravity constant and your viscosity and your density of the fluid so it is basically a related variable so for clay it is a very very slow centimeters per second so think about 10 minus 9 centimeters per second is the flow rate kind of velocity of water in clay or how clay allows water to flow so it's very very slow so when you go to a field the first question you can ask is what peko soil it is and if they say clay then you would understand that a groundwater recharge would take long long time we'll have some examples in the next class so moving on silt and sandy sills the mixture of clay sand and and particles have a slightly reduced hydraulic negativity or a higher hydraulic conductivity because sand actually can improve the hydraulic conductivity so then when you go to silt sand and fine sand you have more hydraulic conductivity so we're going in increasing hydraulic conductivity okay and then when you go to well sorted sands glacial outwash which is the deposit by snowmelt and well sorted is that the particles are well sorted good space in between then your hydraulic conductivity increases so the highest would be at least on this data set it is the well sorted gravel gravels are bigger in grain size and they have well good built structure because it is well sorted and it has space in between so when you sort something it will have a lot of space in between and that space can make the medium conductive and that conductive will increase the hydraulic conductivity of the material so you would see higher water flowing through recharge and groundwater discharge through well sorted gravel and the least in clay clay also swells it also takes the water and holds on to it so all this is kind of captured by your hydraulic conductivity. We can also use a Fries and Cherries book that we used in the previous lectures for permeability let's take an example of again the silt sand because that equates to your fracture igneous metamorphic rocks if you go back to the class notes you understand that is the dominant aquifer system in India the hard rock aquifer covers more than 60 percent area and so if you take that we're going to take that as an example to understand hydraulic we'll use a centimeter per second or meter per second to discuss the results so you see that it is anywhere between 10 minus 4 to 10 minus 3 centimeters per second it's very slow whereas in your aquifers alluvial aquifers which is around 30 percent in the country the nganges ingus bramaputra the kaveri delta so those regions will have a higher hydraulic conductivity so those would be more on your permeable basalt or your clean sand the sand alluvial sand and then you could see jumps up the hydraulic conductivity can jump up from 10 power minus 4 to 10 power minus 1 if i draw a line here it is 10 power minus 1 centimeters per second so it is moving at 1 millimeter per second which is pretty fast okay so you're thinking about per second okay per day so then you convert it to per hour per day per year always go in time in sequence we'll do some calculations in the next class to show what does these value mean okay but before that i would like to also introduce the hydraulic head concept in this lecture so always note the ranges the range is big and why is the range big for example in silty sand the range is from 10 power minus 5 to 10 power minus 1 so minus 10 power minus 4 orders of difference and that is because of the mixture the combination of your silty sand could be different and also the management of the land could be different but either way it falls within this bank also note that when i just take one value let's say 10 power minus 4 it could be a clean sand on the border of clean sand it could be a silty sand it could be a silt lowest or a glacial tin all these four five different types can be within that's one value so it is up to you to understand first what that material is by by physical and lab estimations and then you pick your hydraulic conductivity water levels this is a very very important concept refer to understand the hydraulic conductivity and groundwater hydrology because that is what we're measuring okay at the end of the day the government and the system would measure water levels and i'm taking a small example just an introduction we'll jump into one lecture on this data to understand this so all this data would convert to a groundwater hydrology equation by the examples we showed in Darcy so this is a picture of the groundwater data that is calculated and captured by the government of Tamil Nadu and on your right is the central groundwater boat so there are two agencies at least in Tamil Nadu collecting groundwater data so they collect the data and then they collect it in different years and months and then they equate the difference to understand what is happening so before that let's get into the hydraulic head concept i've already explained this in the Darcy equation but let's take one well to show how do you calculate the elevation because in the real world the zero is not the table because the real zero is your center of your location where the level is zero so normally the zero is taken as the sea level on the planet the elevation zero is taken as mean sea level okay so where the sea is starting that is called zero and from there the elevation can go up or down okay so let's take z as zero which is your sea level we are somewhere in let's say Chennai and i'm on the beach that level the water level is zero and i'm moving inward into Chennai to measure the groundwater level i need to measure the water level the hydraulic head and from the hydraulic head i will go to q which is your Darcy's equation okay so i go to a well and the first thing you would notice is that the elevation of the well is not there you cannot have the elevation of the well readily the hydraulic head readily calculated so you need to calculate the first step you do is to ask what is the depth of the well and where the well is actually open for measurement so here is the point of measurement and as usual as i showed in the previous slide you would measure the water level or depth to the water so you end up calculating this so you have one value which is the depth of the well and also you have your water level which is side okay and what is needed for your q is h how do you calculate h you ask for the elevation of your ground surface at that point which you can take from topographic maps or digital elevation maps so you have the elevation of this your ground which is one okay now two you have this z okay which is your depth basically your depth of your well which is two and you know your side how much is your side which is the water level which is three but how do you get at two how do you know how much is my elevation from the ground you know from the depth of the well which is this you know okay this is your two and but you won't know this one because you have to calculate this indirectly okay so you know one you know two and three how do you get at h by subtracting two from one you get this area this length okay so you get this length when you subtract two from one so one minus two is going to give you your z okay and you know side which is your three so z plus your three will give your your z plus your three will give you your h and that's simple please understand that this is what we want but in order to get this you have to subtract your elevation using your depth to the well okay so now you get h you do this to another well so I walk to another groundwater well and I do another h I know the distance between the wells and then I would do the equation as q is equal to your minus hydraulic conductivity so since I am in the field I would ask what type of soil it is and then I know the hydraulic conductivity from the values I showed from Fries and Sherry book I know the let's not do the area of your well so let's do your q small q okay so we don't need the area of cross section we just need the discharge velocity so let's say q is equal to minus k times your dh which is your distance the difference between your hydraulic heads by the length the distance between your wells so all this we have done just by calculating two values which is your elevation of the well and then the depth to the well to get at your z which is the elevation of the well from the measuring point then we measured the groundwater level to get at three one minus two got me z and z plus three gives you h okay so with this we would calculate the groundwater hydraulic head for one well we do it for two wells and then we establish the equation Darcy's equation but remember we need the hydraulic conductivity which can be obtained from the previous values which are shown here so always have this you can have this slide I use this slide always to measure all the materials are there so to any system in the world you want and take a government report to understand what is that particular soil and then go to this slide and get the value and you can put it here by measuring the ground surface the elevation of the z but which you get by subtracting the groundwater well elevation or the depth of the well and you get q so please note that we will also look at some q values in the next lecture thank you


It's not like the New York Times has not covered the idea of a lab leak at all or other large prestigious organizations. It's just that there's a sort of a symmetry because I think it goes back to this network that you're talking about. It's kind of understandable. You have a respectable scientist saying one thing. They might have motives for saying that thing, but then that is what gets treated as more credible right out of the gate. And like one striking example to me was the way that this is a New York Times article on the pre-print on the kind of definitive paper that's making the argument that this spillover from animals to humans happened at the Hunan market. New research points to Wuhan market as pandemic origin and kind of the way this was reported. You can see a quote from Michael Warby, one of the lead authors on it when you look at the evidence all together, it's an extraordinarily clear picture that the pandemic started in the Hunan market. Some of those conclusions were softened a little bit once this went through peer review notes there that at the time of publication in the New York Times it had not yet been published in a scientific journal. I do wanna like, you know, talk a little bit about like the way that was covered but also about the study itself. Maybe Alex could weigh in on this. They're arguing that the argument of the paper is they did samples in the market and you can see in this diagram there's a little red hotspot where they're saying most of the virus was transmitted or the hotspot of viral samples was in this little corner like one or two stalls. And then if you zoom in on that, you'll see they've got these two stalls kind of highlighted in red and they've got pictures of raccoon dogs and unknown birds, these basically mammals that plausibly could have transmitted the virus from two human beings in the market and kicked this pandemic off. What's, this and one other paper who was by many of the same authors are kind of like the touchstones for like this is the definitive argument for zoonotic spillover of the virus which is a very common way for pandemics to start. What is your analysis in short of this paper, Alex? If you could keep up that figure, that's actually a great, great touch point. So this analysis made a statistical error and the error fails to account for sampling biases. If you look at that little gradation of number of environmental positives at the middle kind of near the bottom there, that's the statistic that they were analyzing. And so yeah, it looks like there's a high number of environmental positives in this lower left quadrant. However, we pointed out that the Chinese researchers who conducted this work said in their methods, they are prioritizing samples near animal stalls. So they took more samples. That's why there's a higher number of samples. If you look at the percent of samples that test positive, they were actually highest near vegetable traders and in the toilet, in the sewage. So it tells a very different epidemiological story of human to human transmission. And another point on the broader topic of war being out, they looked at where are cases found within the city of Wuhan? And they said, ah, well these cases is found within the city of Wuhan with asterisks cases provided by the Chinese government were centralized, localized around the Wuhan that went on seafood market. However, when you do this statistical analysis of those data, you find that the unlinked cases are actually closer to the seafood market which suggests that these cases did have this bias in case ascertainment. Furthermore, there were cases left out of this analysis, cases that preceded the wet market outbreak. For instance, a December 4th report of a SARS coronavirus case in Wuhan that coincided with an uptick of the use of the word SARS in the Chinese social media app Weibo. So those two data points corroborate this case preceding the wet market outbreak. And furthermore, there's another analysis of these care seeking terms that are less likely to have been filtered which showed that the earlier hotspot of care seeking termed usage in Wuhan was on the other side of the river near a hospital that's one of the closest hospitals to the Wuhan Institute of Biology. So when we try to look at these independent lines to corroborate this and when we account for the statistical biases, their findings don't hold. Their claim of dispositive evidence hasn't lasted the test of time. So that's the reward we had on paper and most of us who were in the field knew that right away. David Roman called this hopelessly impoverished early case data and it was quite evident that that was the case. So for us scientists in the field, it's unusual that these two papers, the other one had similar statistical flaws and a bug in its code that completely undermined it. These two papers were both published side by side in science and then both presented in the New York times of the Atlantic and the Guardian as you know, dispositive evidence that's been solved. And for many of us scientists, especially me, familiar with the standards of evidence in the field before COVID, for instance, looking at the percent of PCR tests that are positive, which we all know was a big debate during COVID in terms of how to measure prevalence in the population to control for increasing rates of testing. For us who are familiar with the standards of evidence and the common modes of analysis in the field, these papers were highly unusual. So was the proximal origin paper by these same authors and then you trace it back to the wall and you find that Anthony Fauci who flooded the diffused PIs in 2019, he prompted this call. He put the people in the room and who did he put in the room with Christian Anderson and Eddie Holmes? He put in the leading proponents of gain of function research, including the OG Ron Fousier who passage that alien influenza through ferrets. So this was, you know, it's like the EPA calling in all the oil and gas companies when they're presented with a study, finding evidence of climate change. When us scientists who are familiar with the other scientists and the debates in the field and the methods and modes of analysis, when we see this it's just that to me is raising alarm bells in addition to the genomic evidence of the fear and cleavage site, not just any fear and cleavage site, you know but a fear and cleavage site in the exact position where Emily's recently FOIA documents proposed to insert it in Wuhan where this was proposed to be conducted and far from the hotspot of that SARS coronaviruses and a litany of other evidence that all combines. And I think for me that's how I think of the evidence is we have to treat these zoonotic origin papers methodically and look at them carefully and see how much evidence they do provide. Once we throw those off the table based on their statistical flaws and other problems with them, all that's left is this mountain of evidence, you know it's like the straw that broke the camel's back. We're looking for that straw but we're forgetting about the many bales of hay that are already there. And that's, you know, our paper was yet another straw but Emily is right to point out and thinking about this evidence you have to look at everything. You have to look at the Wuhan origin, the fear and cleavage site at the S1S2 junction with these weird codons, bioengineers use with all this weird behavior, both of the Chinese government and the researchers connected with the Wuhan labs and coincidentally we found evidence consistent with the exact method for assembly that they proposed and that every time we FOIA new documents we only get closer to the truth that corroborates the theory we've had all along. So what does this all say about the state of science journalism, Emily? Emily, do you wanna take that? I've got opinions but I feel like you have some good ones too. Yeah, I feel like I could rant about this all day but I think just taking the Warby paper as an example I mean, some of this stuff we're discussing is very high level and maybe the journalists just didn't understand you might say, but remember that at the point that that pre-print came out and the New York Times push alerted it. Senior authors of that paper had been caught lying already. We already had the FOIA emails showing that Anderson said that the SARS-CoV-2 genome had features consistent with engineering and then four days later told the national academies that that was a crank theory. And we already knew that Robert Gary another author on that paper had said I can't figure out how this spike protein gets accomplished in nature but it would be easy to do in a lab. And then a few days later started drafting a paper that would dismiss the idea of engineering as a conspiracy theory. So these people were already not credible and Zach you were talking about how people's default is often while this person has a lot of expertise they're affiliated with this prestigious institution they seem the most credible. And in 99% of cases I would agree but I think we're in this unique situation where we're trusting virologists to give us straight answers about whether a pandemic that killed millions of people stemmed from controversial virology research. And these virologists in particular already at this point had a track record of dishonesty. And I think when you're a reporter I think you have to have some healthy skepticism and you know, prestige is important expertise is important but so is integrity. And I remember when this paper got a lot of attention in the press I mean the New York Times alerted it CNN alerted it a bunch of other news organizations. I remember thinking, does honesty not matter? Does integrity not matter? I think it clearly matters. And another aspect here that should have alerted journalists to something strange going on was the fact that I mean literally hours before the war be preprint published the China CDC had come out with its own analysis in a preprint of this data. And they had concluded that the wet market was a super spreader event that it had just amplified an already existing epidemic. That conclusion happens to be consistent with their country's propaganda around the pandemic starting somewhere else but they did collect the data and they have more knowledge about how they collected their own data than Western scientists working on laptops many miles away. And what they said was exactly what Alex just said which is that these samples that appear to cluster around the animal stalls we actually focused our collection efforts there. And Jesse Bloom has analyzed the metagenomic data and found that animal DNA specifically animals that could have served as intermediate hosts like raccoon dogs is not correlated with SARS-CoV-2 virus. And so those samples are not very meaningful which China CDC knew and said right before the New York Times put on its front page that it was definitely the wet market and the lab leak theory, it's a conspiracy theory. And I think that the more information we've gotten the less and less these papers have stood up recently to experts in spatial statistics put out a paper basically saying that the War B paper is extremely flawed but there hasn't been a lot of follow up that I've seen in the Western media at least to correct record and to also look at some of the evidence that's accumulated on the lab side. So there's a lack of like follow up also. I don't know that I am hoping for any specific policy outcome or change in the press. I just honestly want there to be clarity and a national reckoning for everyone who lost loved ones to the pandemic or who lost businesses or major milestones in their life. I honestly think this will probably be my last investigation kind of been approaching it that way. I don't have a lot of hope that, you know, journalism will tomorrow be like, actually this is very valuable reporting and we should reward it. I think I just want there to be clarity and some closure for people who are grieving. That's really my one hope. Well, that's kind of surprising and disappointing. Why is this, why are you thinking this is your last investigation?  I don't know that I will be employable after this. At least in the media as it exists now. So yeah, so I don't, I guess think a lot about like, this is how I want the press to change. You know, I just want there to be clarity around this kind of most monumental thing that has happened in modern times. So that's really my hope. Hey, thanks for watching that clip from our new show, Just Asking Questions. You can watch another clip here or the full episode here. New episodes drop every week. So subscribe to Reason TV's YouTube channel to get notified when that happens or to the Just Asking Questions podcast on Apple, Spotify or any other podcatcher. See you next week.


All right. Good morning everybody and welcome to the March edition of the Wikimedia Research Showcase We're coming We're here in San Francisco on a rainy and sad day But I'm gonna warm your hearts with some amazing research and we have a crowded room here. There are some special guests we have a Robert West from EPFL and we have a Mikaela Catasta from Stanford and and of course the rest of the crew here and I'm delighted to have to do today two presentations that are really part of a major program that the Wikimedia Foundation research team has been conducting in collaboration with a number of institutions including EPFL and Stanford and We have a Titsanopi Cardi with EPFL is going to give a first presentation today about Metallurgical challenges in the use of the category network in Wikipedia to extract some meaningful relations and this is going to be followed by presentation by our own Diego Who is going to present the second part of this work about section alignment? Across languages in Wikipedia. All of this is part of a major program that we're investing into with with his partner Institutions now to try and understand how to design Recommender systems for article expansion. So I'm really excited to have you here as usual some house rules That presentation is going to be Broken down into two parts 25 minutes followed by a short Q&A Please stick around at the end of a showcase. We're going to have as usual our final open discussion At the end of the two presentations Miriam ready was here on the on the hangout is gonna be our host on IRC So if you're following a discussion on the IRC channel with media research Please ping her with your question and she will relay them to the room. So with that I think we're ready to get started and it's another stage is yours Okay, I'm sure the screen Okay. Hi everyone and thank you for the opportunity of presenting today this research This is as I said is part of Large project about sections recommendations I work on this project with Mickey Katasta from Central University Bob West my supervisor in EPFL and the Laila of Wikimedia Foundation today, we are going to talk about taxonomies and in particular how to clean the category network to Extract a taxonomic structure from the categories. Let's start with the definition of taxonomy Taxonomy is an hierarchical organization of concepts. Typically They are these entities are linked with is our relation like in this case we have a As small example of taxonomy where we have animals, mammals, felines and You can see there is an is a relation between felines and mammals and animals This is a very intuitive and natural way to organize concept and to reason About model the world in the taxonomy the terminology The typically people use the term hyper name and Iponym to define a hyper name apparent of our concept and Iponyms the sector of children so animals is an Iponym of mammals in for example in this case and Taxonomies are useful in many different domains. For example In a image classification now people use taxonomy to For example in satellite images to Bind a different level of doom to different Level into a hierarchy of the taxonomy so forest could be labeled as a green area in a different level of doom another typical contest is the medical domain where People can model Diseases and symptoms in In a hierarchical structure and can run on top of these Automated reasoning tool to find the similarity between concept and to generate a new knowledge another application could be Question and answering both So a program can infer new knowledge from the model and answer a question about An entity that is not Explicit in in the model. Let's see an example to make this clearer Again, we have a small taxonomy about animals and In the example labeled as number one we can by following this transitive is a relation infer that Feline is an animal. Even if this is not explicit in the model You can just follow the path of is a relation and the infer this new knowledge not Second example related to the question and answering both is Number two the sample label as number two we have a statement that All animals must eat to live and these are is Property in the top level entity animals as you can see on the left and If we have a question for the boat Like does Garfield have to eat the boat can check That Garfield is an instance of a domestic cat and by following these Path of is a relation can infer the answer to the question even if again, this is not explicit in in the model but we are interested in Wikipedia and One good and nice application of taxonomies is a test actions recommendation Layla presented this problem in the showcase of December, but to give you an intuition Basically, we are developing a recommender System to provide the recommendation to the editors about possible missing section in the article. This is a The interface of in vision for a next generation of the Wikipedia editor and In this case basically we want to generate a recommendation Based on on the concept. So if it's a biography, we want to extract a Set of section relevant for a biography In let's see in practice How these can work in these example we have people scientists and actors scientists and actors are children of people and scientists and actors have a set of Frequent sections that in particular our early life personal life and death and For the specialization Scientists that we have a word for actors. We have a filmology But if we go up and we search for common structure we can generate Template composed by early life personal life and death That are representative for the biography of people in general the problem with this is It's a nice application, but generating a complete taxonomy manually is not feasible because Wikipedia we have many many languages and Potentially thousand and thousand of concepts So we should find a way to automate this task But the good news is we have a category network So the I'm sure you are familiar with the concept of category in Wikipedia, but let's see an example In this case that we have the page of Claude Picasso's a son of Pablo and At the bottom of the page there is a list of categories Assigned to this article in this case we have a French artist the French photographer from French journalist if we click on a French artist, we can see the details of this category and we have subcategories and parent categories is Potentially we can see this as a set of Iponyms of the entity French artist and Iponyms for the parent categories. This is a It's very tempting to use this a category network to Generate a taxonomy Unfortunately is not so easy And people that work with the category network No, it is because the problem with a category network is it's community generated and it contains a lot of noise in in terms of links and entities in Internet work Let's see a couple of example of noise that we can find in the category network one Is the network as loops for example if we Check the category government and we go up into hierarchy by following the path public administration public economy Economic policy we reach again the same original category government in our Implementation we used The algorithm described in this paper a fast and effective heuristic for the feedback access problem and basically the feedback are set problem. It's a way to Organize the graph topologically based on the in and out to degree of the edges And once we have these graph sorted we can remove the back edges To convert the network in a direct taxicic graph A second problem with the network is not all the parent represent a taxonomic Relation and to going back to the previous example French artist as a parent category called French art and This is for sure not Is a relation because we cannot say an artist is an art even if the two concepts are related You cannot include this in a taxonomical structure This is a Active research topic There are many researchers are trying to extract it infer taxonomies using the category network The most recent works are multi we be and add taxonomy In both cases they start from the English version of Wikipedia. They extract Taxonomy for English in the case of multi we be using two taxonomies one for the pages and one for tech from for the categories and In the case of head taxonomy only based on the category network But in both cases that they use a linguistic feature to clean the network like in the case of head taxonomy they use the heuristic and heuristic based on plural to infer that the category is likely to be taxonomic Entity These two approaches work quite well But they have a different goal compared to what we want to do because they Have to go to have an high precision on taxonomy with clean ontological structure in our case. We are more interested in having a higher cola in on the category network because in The case of sections recommendations we want to generate the sex recommendations based on the categories of the article. This means we want to Maximize the number of Recommendation we can generate from one article Yeah For this reason for our specific use case is more important to keep The original topology of the network and only remove what can add noise to our recommendation You know, so let's see how our method works in practice Let's assume we have some generic high-level Types for for each article in Wikipedia if you think about a taxonomic structure articles in our taxonomic Should Categorists are taxonomic I should share the same types for example if We have the category scientist What we expect is to have only articles about people in in our case, we use the dbp. Yeah to assign the Label the type to the articles but we can use the Wikipedia or other semantic knowledge basis And in the case of dbp. Yeah, we selected the 55 top-level types are the top-level of the dbp. Yeah hierarchy and these Types include the people organization events Works so very very very generic types So now we can include introduced the concept of purity We can define a category as a pure if The distribution of the types is homogeneous like in this case on the left We have a category and the five articles of the same type We can visualize these with this Instagram where We see we have a five elements of type square in As part of this category This is the case of scientists where we have only people in the sub In the assigned article The other possibility is a non pure category in on the right and then in the visualization we can see we have a distribution that is more heterogeneous like the category football is not taxonomical and In our case we define it not pure because in football we have People Because of player we have events because of matches and organization because it contains also teams So the idea at this point is to run a bottom-up approach to clean the network by using This concept of purity and remove the category where Different concept converges and create an impure category This example we have again football with football people football venues and Football teams say the bottom part is composed by pure categories and Football is not pure. This is a real example taken from the real category network so let's see in how in These algorithm work sir in in detail Imaging we have a teaser This is a small small graph. Let's start with the Leaf of the graph like here is a category on the left bottom of the slide and We can visualize the teaser to this the types distribution On this Instagram and we can see this is pure because we have only three articles of the same type So we mark this as pure and we go to the next category again. We have the same structure same Kind of distribution and we can mark this as a pure We can go up in the hierarchy and We can check the distribution of the top category the parent category by considering all the articles in this in this dream because If the is our relation is respected they must be of the same type So again, we have here a pure category. These could be the case for example of football people where we Found a pure category now, let's move to the other leaf and Again, we have a pure category even if the distribution of Types is based on a different different type. This can be the case of Football venues where we have a pure category different from with a type different from football people So now we can move up and we reach the category that in the previous example was football where we have a very heterogeneous distribution and And here we can mark this category as not pure In this case, we can decide to remove the category. We can remove the edges It's it depends on the application this method It's more like a framework to clean the category network You can use it for different purpose and depends on the use case Consider that if you remove the category You are not disconnecting the Graph because typically in the category network a category as multiple parents for example on the left side that we had Football people is possible to have an additional parent that is a sport people That is pure and that we keep these the subs reconnected To the taxonomy the problem is the interreal world There is no It's not possible to have a perfect pure category because of the noise and again, it's a community generated and some people can assign categories to articles even if they are Not completely related so for this reason we are more interested to Mark as a pure the category with a high and balanced distribution like in this case we have isolation of types distribution where we have a high prevalence of people and We can assume this category is the pure To measure the unbalanced of the distribution. We use the the genie coefficient genie coefficient is typically metric using the economic to measure the Inequality of wealth in a country and in our case we want to measure the genie coefficient of the distribution and Set a threshold to decide when a category is pure and when is not Like in this case that we have two examples at the top histogram it's Could represent depending on the threshold pure category while at the bottom there is another distribution that is more heterogeneous and if the Genie coefficient is below the threshold that could be marked as a not pure category in Now you Are wondering how we can select the threshold again, this depends on your specific use case in our case for sections recommendations, we use the manually annotated data set of seven under Samples where we annotated a set of transitive is our relation like British scientists is a People is a person. Yes a positive example and Gaelic game ground is a football. No, it's a negative example. We use a transitive expansion of the is our relation to Basically Mark as a negative a path that should not exist in the taxonomy And then we prune the network with different level different threshold of the genie coefficient and We computed using these annotated data set precision and recall and based on these we decided the best The best value to have a balanced Set between precision and recall in conclusion We presented a language independent framework to clean the category network. This is more a framework because it's a sort of parametric Method and you can customize it for your specific use case for example, you can choose different Types from different data sources. You can use wiki data. You can use the dpdia or additional semantic and all these bases You can use a dynamic threshold. We didn't do it for our case, but you can for example choose a genie coefficient a threshold for the genie coefficient based on the size of the category in terms of article or the depth in the category network because for example categories on the top of the hierarchy are Are more likely to receive a noise from the bottom categories an additional customization can be the way to select the threshold of the coefficient the genie coefficient we use a manually annotated data set, but potentially You can use a different a different Method based on a different objective function like you cannot optimize the The threshold for a specific use case using a validation set and finally You can decide to delete or remove edges again based on your use case If you want to keep a super high recall or if you want you prefer precision Again, I repeat again. This is a part of a bigger project. So about section recommendations So if you are interested in more details that is there is the link to stay updated and and Thank you very much if you have questions Thank you very much this piano for the presentation. I think we have a few minutes for questions So when I ask first Miriam, there's anything that you can relate from IRC Yeah, I think you're the first in the queue that you Be sure And I was already familiar with his words. So really nice to see it presented in such clear way I have a question about something I've been thinking of for a while that is about the robustness of Categories as a function of who asked them. So I don't have any good data to decide here But I imagine that in some cases that might be batch editions of categories At large scale to an entire set of articles There may be a single bought or single users performed just because they decided with a good idea to add that category to thousands of articles and Conversely, I imagine there are Articles who are the addition of a category as being done more organically by a large set of users And I was wondering if that could be used a signal that some categorization more robust in some other types of organization Or basically just like someone going bananas Have you because of any thought about something like this? Yes, I mean in our specific use case We didn't find this problem For the section of recommendation it was enough what I presented there and in of course You have to apply additional Check on top of the gene-efficient if you want to be more robust to this specific cases Again, I present these are more like a framework to clean the network and you can plug a different logic to Manage a specific use cases or specific This case to clean This kind of noise I jump in here briefly. I think this is a great point. It points also at at a at a whole different research direction namely basically building a taxonomy of Of category usage like what are even the use cases for which Categories are used on Wikipedia because it's not only this Kind of pseudo taxonomy, which is what we would ideally want But people use taxonomies to organize their work like there are tax there are categories that are called stubs in Stubs around universities or something like that. So that shows you that it's really a meta There's this whole meta level and and characterizing one of the use cases and Are they successful or not? I think would be super useful in order to understand whether the category system is Already good or whether it should be redone or revised in any way Very, very. Yeah, I agree. There's a bit of like an edit persistence Direction that could be applied to categories as well. Yeah, that makes no sense Cool, I want to open it up. If there are other questions now You have anything from IRC Not for the moment Okay, so how about we're right at the half hours? So we can move on to the second presentation. Thanks to this channel Stick around until the end The second position by Diego and he's going to present It's gonna be a presentation about the second big part of this project around the second section alignment Yeah, go to the stages yours Yes, okay, and so hello everybody before attending this talk and So today I will talk about the lightning will give you a sections across multiple language Beyond the automatic translation. Hopefully. I'm sorry. I have to interrupt. Can you press present? It's pressed We are not seeing it yet on the screen. Okay, wait, okay What about now just give us a second. No, not yet Okay, let me stop sharing and share again share screen Okay Now yes, now it's working. Thank you. Okay right So I'm saying I would present this work About the line sections across multiple language on this young world with a Baja Bob and Layla and So the main idea of this of this Part of the of the project is create a dataset with section names align across languages so with alignment we mean all translation I will explain the difference between translation and alignment and Why this is important on this has on many different applications in the context of this project For sure the most important is do cross-lingual section recommendations So if we know something about an article that is in one language and Someone is writing an article in one language. This article is already in many other languages We can use the sections there to do the comment sections in the new language and In general for languages that we don't have too much information as everyone knows when you do recommender systems Having not too much data is a problem So if we can connect all the languages For every single language you will have the knowledge from all Wikipedia. So that would be very useful for sure is also useful for improve the content translation tool and always and also In general, it's a good idea for general and abstract ontology for section titles So we can think later in our section system items more than they just a title and this has many applications For understanding, but we keep it up those languages. So what do we mean about? Alignment so here, for example, you have sick languages and you have two section names Or protection titles in each language and what we want to learn is the mapping between on this this section. So for example Here you will see that on the ones that they can pronounce Premiers in Spanish is the same that awards in English And I was learning how to say in Russian, but now I forgot but you see they're in Russian and in Japanese and Farsi And on the other side, you have the alignment between the biography biographia Biographia in Russian also in Japanese and Farsi So You can say, okay, that's that's translation. No, so why not just to use automatic translation services? Can we trust in them and Anyone using sometimes, I don't know any of the public translation tools that are available knows that they have some limitations They're far from perfect. They're usually good for understanding, but they're don't produce excellent translations like human level translation and also Doing some tests from from our specific use case We found the accuracy depends a lot on the language So for example, we test in English to Spanish and in English to Farsi and the the quality of the Translations were very different while English to Spanish was acceptable in in many of the cases in English to Farsi the Result was very different But moreover This is a bit different from translation because there's some constraints that we have or some challenge that we have Where we want to for example keep the style and conversion of each Wikipedia edition So There ways that people write sections that are not necessarily translations of others in different languages And we want to keep that we don't want one language dominating the style of the other Of the other languages For example in in English you will find that notes and reference are usually two sections while in French This is usually one section with these two things inside of that section The other important thing is that we want to give equal importance to all languages So we don't want to use people's languages Meaning that we don't want for example to translate to every language to English and then from English to the other languages But we want to do translate use all the information possible So if we if we have information in in 200 languages We want to use that information to learn the mapping for the section for the language 201 And the other big challenge for this basically considering these two things is building the ground through So basically given that we don't want to use a people language. We need to build a ground through from having people that speaks All pairs of languages. So for example, we will need to find someone if we want to use labian and Bengali We'll need someone speaking this to language with Encyclopedia writing quality But at the same time we have this constraints and challenge, but we have also Assets and opportunities. So we have a this very active and committed community that is interesting These kind of things in Wikipedia. We have people self-reporting their languages skills here with the table template I was playing in a minute. We have entity links across languages. So Thanks to wiki data. We can know that a page talking about global warming in Spanish is the same page in French in Bengali in Arab, etc. And The other good part is that Given that now many people use in NLP use Wikipedia data We already have pre-trained models deep as a big set of different pre-trained models using wiki data across different languages Particularly today, I will explain a bit how we use the Babylon project models So let's start off. What are we doing to build in the training data set? So I forgot to say this is a work in progress. So this is what we are doing exactly now so the first thing that we did is try to build this training data set and We put some some constraints some Things that we want to to have in in our data set So basically select a set of languages with different scripts. So not just Latin but Cyrillic and Arabic and Other other scripts and different coming from different families not just in the Europeans But other other families of languages. So in the in the right hand of you can see The languages if you don't speak all these languages, maybe this is easier So we will considering mother Arabic, Russian, French, Spanish, English, and Japanese So we will use this for learning how to do these mappings and then we want to apply for all the others For all the other languages. So even that we don't have someone speaking the labia not Bengali We will learn from here and we will try to translate their To transpose their with with disperse So what is we'll be available we'll be available is a template where people can self-report their language skills So basically they they annotate from zero to five with five being the most proficient and Zero being that you don't speak the language So people can say which language they speak and in which level so then we can go to the public information in in the public service query and we can query there and Get all the user names and people that speaks this that self-report that they speak these languages and Surprisingly or not in this six language that we select we find some overlapping all of them So the most basic so most of them will speak English, but we also find people speaking Arab and Japanese or friends to Russian or friends to Japanese and Japanese to Russian etc. So These people is the one that we're contacting now through their user pages to To ask to them for helping us to build this ground through this is what we are doing exactly now later on I will give some pointers if you want to help us on that task So exactly what is the problem definition? So basically having two sections is one and is to where is one as two are from different languages We want to know whether as one is the translation or the mapping of as to across and do this and repeat this across multiple We can consider this as a link prediction task. So if you are familiarized with the graphs This will be a multi-part graph or K party graph which Where each key group it corresponds to a language and Notes are sections titles and the links represent the the translation or the mapping Just a quick reminder if you don't remember exactly what is a key party graph. So a Key part graphs are graph Was a vertices are or can be partitioned in two different independent sets? So the most usual maybe you have here is a bipartite graphs So typical example of this is you have a graph where you have notes that are actors and movies and So you will have linked between actors and movies But not between movies or between actors in our case the key will be equal to the number of languages in the in our data set for for this training data will be equal to six and As you can see in the right that would mean that we'll have mappings from one language to other and not inside within the language Whether the features that we will consider for this we will use automatic translation for sure It's not perfect, but it's a strong signal that we can use We will use the Levenstein distance or also known as edit distance that basically measures the difference between the different characters between two strings and this is especially Useful for languages that are close like our Portuguese in Spanish either Italian and Portuguese and Or Dutch to German, etc. Also We're planning to use the upcoming links within the sections. So basically where they are linked to But the two things that I will explain today. They are the most innovative and interesting I think it's so we will use word embeddings across different languages and they what we call the co-currence count So an embedding what is an embedding? Maybe you heard about this is has been kind of trending in the last years in the NLP world So basically when you embed an object It's taking a set from one Domain and projecting another domain preserving some structures that you are interested in more specifically a word embedding it's when you take words and You project in a victorial space And in this vectorial space the the structure that you preserve is the semantic structure So two words that are close in the vectorial space Should be close semantically Here I'm projecting use Disney I'm projecting this This section titles and Here you see in two dimensions. So this is what this algorithm Disney does, but in fact you are using vectors of three hundred dimensions And you can see that for example, you have some clusters that are pretty clear that they're related like career and career or in the right top you see Bibliography, filmography, discography, publications, works, gallery or on the bottom you see awards and honors So this is very useful usually it's done in one language And it's what you use to measure so you can then measure the distance between these two vectors for example using cosine similarity and That will mean that if two words are close in the vectorial space. They are semantically close, semantically related And the point it is that if you do this in multiple languages The usually the embeddings what we're using in this case is fast text So maybe you have heard about word to beck or glove also in this case. We're using fast text If you learn if you learn in different languages, you won't get these characteristics because you won't have the words similar in the vectorial space so here you see that just taking the cluster about Bibliography and filmography in the left you have the The section titles in Spanish And and you see that both generate clusters, but they are not close between them So where here is where it comes into the play the Babylon project These people that basically is doing alignment between different vectorial space in different languages What they do is a dictionary based alignment of those languages. So basically they give information They know that same points are they are the same for example using a dictionary or using nouns and with this they align they perform a linear transformation and this allows to put in the same space both a pair of languages So after applying this linear transformation, you can see that all the all the words are clustered The the cluster is not the alignment is not perfect as well as the as the Embedding is always not perfect But now we have a more clear sense of the sense of distance here gives gives more information about Semantics so this is one of the main things that we're using to understand that the things are translation this plus the other Plus the automatic translation and also what we call the calculus and counts. So basically Week data allows us to align articles. We can know that an article is talking about a wiki data item That can be a biography or can be any item of wiki data can be a place or whatever. So basically We can align the table of contents of Every single article in every single language and if they exist in these two languages We can count the co-currencies. So basically, we know that in the same article that usually appears Personal life appears Primero Sáñez in Spanish or Publications and works or publications and publicaciones. So this this doing this Co-currency counting was to get a good information about this alignment So as I said, this is ongoing Research or we are contacting people that know we know that speaks are one of these two of these are six languages So if you are one of these persons, please help us here You have the pointers if you cannot remember now, please send us an email and we will put In contact with you. You also have the fabricator task in the bottom So this is about the mapping But for doing a good mapping we also need to understand our synonyms So within the same language, we know that there are two sections that means Exactly the same or partially the same And this is very important to know when you are doing the linements Because this will help to do better linements and also to to get a better understanding of the of this Of this hierarchy of sections of these graph of sections So here is an interlanguage Interlanguage problem. So basically having two languages from the same language. We want to know if it's one on us as to our synonyms and one interesting thing that that we understand here is that Many times you have our synonyms like exactly the same Meaning but also many times what you have are partial synonyms. So two section or titles that are usually use it In similar ways, but they cover they have an overlap, but they are not perfect translations And for doing this we are using again the embeddings. So the embeddings in one In one language that you already saw Again the edit distance This is also very useful because many times and it's related with the third feature that is a subset So basically many times you have One section that is overlapped with another one even in the in the title. So education on background It's overlapped with education or with background. So this is very useful also to understand when you have synonyms but the The feature that I want to explain you today is about what we call gift IDF similarity So basically Our intuition here is that two sections are likely to be synonyms if both tends to co-cure with similar section But they never co-cure among themselves almost never co-cure among themselves. So basically if you If you have for example background and education They can't co-cure, but it would be rare that the co-cure, but they will cure cure with similar sections And the problem with applying this is that the most frequent sections tend to co-cure with almost all the sections So for example reference see also and these kind of sections are present everywhere so we need a way to to remove this and one way is to do with Kind of a stop word removal. So basically have a blacklist But the problem with this is first is there's some sections that are very frequent, but this is a They're they're not Equally distributed across all languages and also that is difficult to we don't have a stop This is top words for every language that we're working with so The solution that we developed for this was basically represent sections as TFIDF vectors So now you need to think as a section If you know what TFIDF is and you need to think on a section represented as a bag of sections where the Term frequency will be the number of Co-currencies of these sections with other section and then we can apply the standard metric of cosine similarity between these two pairs of sections and Then we can divide or filter somehow With the number of co-currency of these two sections Yeah basically basically that We had got super good results with that. So here you can see some examples This is just considering the TFIDF similarity for sections that never co-cures. So the co-currency here is zero between them So you can see that there are things that are very easy to get But there are things like for example the last one line up and bad members that It would be difficult even where they were the embeddings, but the TFIDF similarity works pretty good with that Yeah, so As I told we are working on this We are now in the part of building the ground through for both for section alignments and synonyms So if you don't speak Both of the language two of the languages two or more of the languages that we're working with But you speak one of them you can help us with the synonyms You can again refer to them link that they post earlier and if you speak about more than More than one of these language. It would be very helpful if you can help us label it them We're testing different frameworks for the link prediction or link detection and Another thing that maybe it won't be part of This particular research, but it's something that for sure will be working if you sure is to improve this word embedding so now This word embeddings alignments are done based on a dictionary list synonym list Between these languages But for sure with information that we already have in Wikipedia with especially with wiki data We can come up with a better way more more effective way to align embeddings across languages and I think Yeah, you have some information there, and I think that's also please I'm happy to hear your questions Thanks very much for the presentation Diego And with that we're gonna open the room to questions as well as Anything coming up on Irish Miriam, is there anything any question from the channel? So we have a question from the YouTube channel from Dumisani asking what language translation localization engine is used for this project So as I mentioned We are not Basing everything in a search in a translation engine the test we have we have done till now would be with a Google translate Also with Bing But for the rest of the project We are not tied with any of them. So we might use another And another Engines I forgot now the name of the one that is used by the translation tool that it's a different open source one that We might use for production as this Is it a yandex Diego or no? I Don't think so. I think we have something that different, but I don't remember It strikes me that This work aside from the the use case that we're trying to solve here internally for the as from the recommendation sexual recommendation problem Is a much broader Problem in a solution that you guys are designing And the data that we're collecting is going to be potentially useful for a much broader set of machine translation and P use cases, you know, we're doing here, right? Yeah, absolutely one one comment like that. I think Diego mentioned this in his presentation I think the key and focus here is here is not to get just a translation in a way that is Understandable by the audience but to get the level of translation, which is encyclopedic. So this is kind of Highly specialized translation for the case of an encyclopedia, which is really useful I mean, that's kind of an addition to everything else that machines are considering But the machine translation on the web is not done within context, right? So now we're saying Refine and constrain yourself to Wikipedia. Yeah, what would be the translation? Yeah? Yeah, and it relates I was thinking as you were presenting about a question that we hear in product A lot is machine translation sufficient for creating stubs or adding to articles And what is the reaction of the people who might read that? From one language to another. So I think this could be very some of it. I'm not sure exactly what parts but it could be very useful to the product department. So we might want to ask you about that. I also see sorry just to jump in on another potential use case here is We key data descriptions like labels and descriptions. I know this is complex problem of how to map concepts concepts with the articles to a variety of entities in we data, but it strikes me that For some sections of we P articles that map to entities. This work could be really helpful to identify good candidates as Descriptions or labels for this item. It's just me thinking aloud. But I see we can get a good also benefit significantly from this work. Yeah, for sure. I think section translation. So section titles usually had a lot of semantic power, let's say, no, if we are able to do this, maybe Labels from different types will be easier. And then yeah, anything that we can do after this works. The concept of alignment is very interesting for anything any grossly well work that we do in the future. I have a question about I think you mentioned that in some cases are just not a one one to one correspondence between languages. There's some section titles which might mean what Before we have two different things in the other language and I'm not sure. I think later maybe you just finished out this project. But a year ago, we had An intern who worked on a just counting the most frequent section that essentially we had some observations like this, like, I think one example was A wall or sea on the French Wikipedia. It's see also, but it's also external links. So they work with internal links. And I think it's also with literature, bibliography, North references. There's a lot of different approaches. And post that map into in this framework or Can you like recommend two different section tiles as translation of one section? Yes, so this will be one of the goals. So knowing that because if you have two sections mapping to one section. So let's say the example with French, you have sections and in French is one section. If we also know that these two sections are not synonyms in English. So we know we know that nodes and reference are not the same thing And they map to the one both map to one section in French that would means that Basically this section is represented in two different things In in the other language. So it's something that we are considering. So basically the synonym detection part is for helping also in that If we know that two things are synonyms, we can collapse them in one node and then we can select one of them By different things popularity might be one, but maybe there are others But if we know that they are not likely to be funny synonyms, uh, this will help for this A node one to one translation recommendation One to end Um, we have another question from youtube, um about Are there any plans to represent languages not supported in google translate? mostly african languages Yeah, for sure as I said, uh, the The translation We're not just using google and the automatic translation is one of the features that we are using but not Not not the not necessarily the main one and We already have What they did with the Babylon project. I think they have around on 90 different languages listed there But given that this is based on fast text. So basically and uh, and uh, it's a it's a open source project We can align to any any embedding where we have, uh Information so basically if we have a wikipedia in one language We can create embeddings for that language and then we can easily, uh, I want to mean easily is really easily Align with the other languages for sure for languages that we don't have too much content too much, uh In this case pages, uh, the quality will be lower than uh, in uh, in very populated wikipedia But for sure it would be better than uh, than nothing and for sure it could be Useful for for do the the recommendation of the most popular sections I think that's all from the channel Thank you Diego Cool. Um, can I make a one remark? um Yeah, so um, you know, basically I want to start with the first part of the presentation, uh, I think um Tiziano started working on the project of what we call cleaning up the category network with all the caveats that he talked about in september 2016 And we have been meeting basically every week and discussing this project since then And there were weeks that we just left after the meeting and I was not sure if tiziano is going to come back the next week I just want to say that he has done an incredible amount of work and has worked on one of the hairiest problem That Bob Michele I have seen in our research career And it's amazing to see this coming to this level and I'm really looking forward to Doing the daily releases that we have been discussing associated with basically like empowering others to use What you have brought this project to tiziano You have this been discussing, you know, blog posts and like how to use this as a community as a feedback to the work that the community has been doing on Potentially cleaning up parts of the category network with the caveats that Bob mentioned So thank you for all of that work to the three of you, but tiziano you specifically used a lot of heavy lifting over the past year and a half Um, and of course, uh to Diego Bob and Bahar, right? I mean, this is the project that we started in in In the past, uh, I think six months ago and it's been it's been moving fast and it's really exciting area. It's I attended a weekend daba remotely over the weekend and as do me and others have thought of in youtube There's a lot of discussions about what is our role in the smaller languages, right? And what do we do if the internet or web is not supporting these small languages? And the work that we're doing in this space is critical to basically empower Users editors readers in these languages to have access to the content So it's just push as much as we can in this space and get the results data everything out. Thanks for everything you do Yeah here I also want to ask, uh, uh, Michele involved with any additional questions or comments that we have an opportunity to have you here Sorry for calling you off like this Oh, yeah, can you help me soothe my daughter? Yeah, I think we'll pass Um Yeah, no one thing that I want to mention about, um the language that are not what is represented in wikipedia is that, um So this is um The quality of word embeddings improves with the amount of text So I hope that instead of if we if we can produce a medium quality, let's say Recommendations in these small languages. I hope that this will help people to write more and in While they write more the word embeddings will improve and with this improve all the alignment will be better And we will produce better recommendations So, uh, I think it's very important. Um to work in this uh in this kind of spaces because, um, I think can be uh, it can break the inertia of So this happens with everything. It's kind of uh, uh, networks or effect, but um But even with uh with some uh, thousands of articles we gotta start working with embeddings So I think we can we can work on this with uh with most almost all the all the projects in wikipedia So I think it's a good starting point Yeah, and I also want to say I just uh retweeted the call for participation in the labeling campaigns Obviously if you want to be able to cover Uh, all languages including like uh underrepresented languages really critical that we get um Multilingual speakers to help us out. So please if you're watching this and if you want to help Check the links that you'll find uh In the presentation or on twitter or on the research list Sweet anything else Miriam from the channel or from the room? Not really No, I think we're good. All right, so I think we're gonna wrap it up Thanks everybody for joining. Thanks to our speakers Diego and tiziano And everybody else who joined us here and I RC and on youtube, of course And we'll see you all uh next month in april the next showcase


White noise is defined as a random signal having equal intensity at different frequencies. It's used for audio engineering, generating random numbers, and of course, sound synthesis. It's easy to generate white noise using a simple circuit that sort of misuses a transistor. By reverse biasing this NPN transistor, we're essentially pushing current through the opposite way it was intended to flow. To do this, the reverse voltage needs to go above a threshold of 6 volts. Then the noise is generated and amplified by that second transistor, and it sounds like this.