Hello, welcome everybody. My name is Mandy Chassel, and this is my colleague, Dan Wolffson. We work for a company called Pragmatic Data Research Limited, and we're both closely associated with a project called Aduria. The topic today is actually about three open source projects. Aduria is obviously one of them. Another is a project called Open Lineage, and the third is Marcairs. All three of these projects are part of the LF for AI and Data Foundation. We're sort of closely related, and this is really an example of a collaboration that we put together to show a broader value than each one of the projects could produce. We call it truly Open Lineage because it's all about creating traceability between the execution of work across different workloads. That sounds quite simple, but actually to really be able to use and analyse it, it's a complex problem. So the plan today is to take you a little bit through what lineage is, some of the challenges, and then what we've managed to achieve together with the three projects. All right, so yes, very simple. What it is, what we did, and what we got from it. So let's start with the definition of lineage. So let's think about what could be happening. So here we have three regions of a company all doing very well. All the graphs are going up. But when the CEO looks at the combination, something strange has happened. The graph is going down, so something is wrong. The question is, where is it wrong, and what is wrong? Are the values incorrect? Where are they incorrect? Something's inconsistent. What's happened is the way that the data is being combined, causing some sort of, you know, are all the units the same, anything like that? Or is there something missing? Has one of the regions not actually reported any numbers? And these are the types of questions you get when data doesn't look right and where lineage can help. There are other questions as well. Are these coming from the right sources? Are the schemas matching? Are we actually tying different, the wrong types of data together? And we also need to understand whether it's right in the question of time. So it's not just did it flow from the right place, but did it flow in a timely enough manner that the aggregation worked correctly? And often, once you know these things, there are follow-on questions that you can answer. Sort of like, okay, this data is wrong. Who do I talk to? So who owns that data? And also if I own data, I need to know who's using it. So if I need to change things, I can actually let people know that things are changing. So lots of questions that you can have when you start to share data among different teams. You want to switch over? So a lot of this is about building a healthy data ecosystem. And this is really, you know, one of the main points is that this is a team sport. We do need to have the different teams communicating with each other in order to understand how the data is flowing and what you can do about it and how to understand between the different teams. There's implicit contracts often that are taking place between these different teams. So if we move on. And so the key around this is the data lineage. And the data lineage really talks about how the data is produced, how it's consumed, how it flows through the system. And we can do a lot of things by understanding that, all the different attributes around that. So both understanding the sources, understanding the targets, understanding how it's flowing, how long it takes to flow, all those characteristics, the inputs and outputs of each process that's taking place. And so we can talk about lineage and lineage shows how the data is flowing from its origins to the different destinations. And there's not just one lineage graph. There's lots and lots of different paths through the system that the data takes. And so even when we're starting from a single source, that data may be promulgated through many different chains of lineage to a lot of different targets for different purposes at different times. And so we generally, we want to understand that, understand the traceability of that information so that when we're looking at from a consumer's point of view this data arrived, we want to be able to say where did it come from and how did it get here. And then we want to be able to look at things, once we understand that, we can look at things such as impact analysis. We can say if somebody changes something upstream from me, what how's that going to impact me? If I change this table, what are all the systems that are going to break behind it? And so we can get to different kinds of impact analysis and then we also get to data observability in terms of understanding how the data is flowing. Is there a breakage in the chain? Is it taking too long? Is it in some cases taking too short? And so we can start to get to a governance by expectation by observing what's actually taking place in the system as we collect the facts about how the data is moving. Let's go on. There you go. Yes, that's fine. So I think Mike has slipped, let's just slip aside that one. OK, so let's start diving into what we would expect in lineage. And the traditional approach to lineage, it came really from the ETL engines. So these are engines that move data from one place to another or copy it really, copy and transform. So we had this idea of a graph, like a flowchart basically showing how the data was moving between the systems. So here's an example of this type of job. It's reading from a file, it's looking stuff up in a Hive table and writing out to a Kafka topic. Now also today we have a lot of microservices and API calls. So everything is connected via request responses. So our lineage graph has to think about not just a flow of data. We have to think about control flows and we also have to think about core responses. So let's start sticking things together because there are a lot of technologies that support lineage. And this is a good thing. But on their own, they only tell us about one box. And one of the things that we've been doing as a collection of projects is saying, well, things do come from a long way away and they do pass through a lot of technology. How do we bring that picture together so that we can really see how this data got to here? And that's such a key challenge. It's the heterogeneous nature of the way that we put systems together today. Now, I'll just go back to this. You see here I've got different technologies here. One of our projects does, Nigeria, is it has a set of types that describe all types of technology. And so if I take that same graph and move it through, what you see here are the metadata elements that we would use to describe the shape of this inside Nigeria. So we can capture and Dan will go into this in a little bit more detail. This is what we call design lineage, which is the structure of how things have been implemented inside Nigeria. But that's all great. We know what's deployed, but what actually ran. And that actually is not what we do in Nigeria. We're focused very much on exactly the structure and how things flow. So Dan, you want to go into that? Sure. As Randy mentioned, we can simplify things by thinking about design lineage and operational lineage. Design lineage really represents your intent. This is how I intend things to operate. This is how I'm going to transform the data from one thing to another. How I'm going to deploy that out. But when you do that design, that design may give you multiple paths to take through the system. There's not just a single way through. It may be that if the data is, let's say, critical data, it goes down this path. If it's less critical data, it goes down that path. If it's potential fraud, it goes down maybe a third path. And so what you want to also do is understand what really happened. And that's really what you can see through the operational lineage, is you see what in fact happened. How long did it take? Where did it really flow? What systems did it touch? Where did the data really come from? How many rows got moved on this process run? All those kinds of details are part of the operational lineage that we can see through the system. And that we can, by looking at that, we can understand both how the system is operating well and how it could operate better and what to do when the system fails. And did it actually fail? Because you can see that. Suddenly I'm not seeing rows move through my operational lineage in this particular path. There's something that broke. Maybe. And so that's again why we need to look at these things. Thank you. So in terms of Nigeria, as I said, from our perspective on our project, we were very good at design lineage. We can capture all sorts of information from different technologies and show how things should work. But this operational aspect became, it was a problem to us, we didn't really have capability for it. So as we started to hear about these other two projects, Open Lineage and Marcairs, we started to realise that between us we could pull together a good story. And particularly since, as you start to examine this problem and the way systems work together, it's not a simple division between the technology that produces design lineage and the technology that produces operational lineage. So if we start to think about something very simple, here's a process that takes data from one database and writes it out to another. So from a design lineage point of view, this is the sort of metadata that we would capture in something like Nigeria. And then when the processes ran over time, we would have lots of process instances being captured and this is what we would call operational lineage. So this is saying that the process ran three times and in each case we transferred a certain number of records between the two servers. So we know how often it's running and how much data is being transferred. So that's very simple separation between what we knew at deployment time and what we are getting from observing the run time. But now let's think of files. Now files are much more problematic because they appear when things run. They're not necessarily deployed out of a CI CD pipeline. So what we may know at deployment time is that we're going to create source files, we're going to appear, say, in a landing area and we're going to put them somewhere depending on the logic and the process and that we're deploying a process that's going to be monitoring the landing area and creating things as they go past. So as the process runs, we are picking up information about source files and each one is creating a new instance of the process that is creating the destination file. So that capture of lineage is operational in exactly the same way as above but it's also creating the design lineage for this at the same time. So you can imagine that the operational lineage is being harvested to create the equivalent design lineage that we would have had at deployment time for the database case. So the distinction between operational and design lineage is actually quite fuzzy and you say, well, maybe you just need this then you don't need the thing above. But actually each engine uses its own way of identifying things. And so the more traditional catalogue that understands the relationship between things is then able to stitch together the knowledge that we get from the different engines. So the two work very well together because we have a knowledge base and then we can see what's running and we can correlate the two perspectives using a combination of the knowledge from both areas. Right, Dan. Okay, and that really sets up the motivation for the open lineage project. It's really about how do we get to some of these common interfaces that allow us to publish these events out from all the different technologies so that they can be consumed and that we can link them together. So open lineage is an open standard for the collection of the lineage information from the pipelines as they are running from the different technologies. And if we go on. And where we can see that it fits in is that there's both producers of this information and consumers of this information. And so some of the producers might be libraries such as Pandas or runtimes such as Spark, DVT and Airflow that can produce the open lineage events and that can be consumed by technology such as Amundsen which is another data catalogue or Egeria or Marquez or anything else that wants to listen to these events as if they're taking place. And so before we had open lineage if you had a metadata catalogue such as Amundsen or Egeria for that matter then you would have to write special adapters to each of the different engines to go and capture their kinds of events. And so you would end up basically with a rats nest of different kinds of connectors and every time one little thing changed everybody had to go rush around and change all their adapters to be able to consume that information at that point. Whereas with open lineage, we can get a nice smooth transition. We make life easier for everybody both the producers and the consumers and in fact we add more value because we make it easier to tie these things together. And that's really the main goal behind the open lineage. So the open lineage contributors let's move forward one include projects such as Pandiz, Marquez, DPT, Amundsen, great expectations on the quality side, Microsoft, Iceberg, Parquet and others are looking to join as well and looking at how to participate and we strongly encourage that community to continue to develop and grow. It makes everybody's life easier. All right, so I'm going to now talk about the work that we did together as our projects. So this is actually what the open lineage standard is and my mic is slipping again. This is all it does and this is its power actually. The fact it does, it defines a structure of an event that is an open lineage event and it's called the run event and it defines an HTTP endpoint and that is HTTP not HTTPS and that will become significant a little bit later. So the idea is that an engine like Spark when it's running it just is formatting its own logs into this run event and publishing it to an endpoint that it's given the address of when it runs and that's the standard that we have with it. The event is quite straightforward. It has an event type which says I'll be starting, stopping, doing something else and you can describe the job that's run, the job parent and the inputs and the outputs. That's roughly those sections and then we have facets and these facets are what the different technologies can do and add and in fact we could pull an open lineage event say into a gerio, a gerio called augmented and add new facets and then push it out to somebody else. So it's not just the first person can put facets in. Downstream people who are consuming it can also add the facets. The facets are, there are some facets defined in the standard and you can add your own. So it's extensible, you can experiment with new ideas and then when you're sure of a particular facet then you can push it back into the standard. So that's how it is organised and here are some examples of the flows and so if you think about this process that's running there's an outer process and then three sub-processes. So you see the starting and the stopping of the main process then the first sub-process that stops and the whole thing is tied together because the outer process is the parent of the sub-processes. So you can re-establish the hierarchies within processes that are running. The reference implementation is Marquez. So Marquez is a metadata repository that's designed to monitor the running of processes. That's its focus area and so it supports the HTTP endpoint and whenever we're testing if it runs in Marquez then the events are good. So it is considered the reference implementation. One of the things that I worked on is a thing called the proxy backend. So because the endpoint is HTTP and that makes it nice and quick and easy to set up and things you can't deploy the catcher in an enterprise environment. It needs to be kept very local to the actual runtime engine. So the proxy backend is deployed into that runtime engine into the secure enclave that the process is running in and then it publishes out the event. So it acts, as I say, as a proxy. In the first instance we use Kafka and so now the events are coming out very fast from the Spark engine. We only need to keep the proxy backend up and running at the same time as the engine so we haven't got to keep it up 24 by 7. And then we have the ability for enterprise capability to pick up the events from the Kafka topic. And so here in the yellow boxes is Adgeria listening on the Kafka topic, pulling the events in and processing them, extracting the design lineages. We talked before correlating things and augmenting the events. So we're literally going from open lineage to open lineage. We also put in support for the HCTPM point for small, for the occasions when the organisation wants to go straight to Adgeria. I suspect the proxy backend will be used more than this direct but we want to just to keep it to have a complete implementation and have another endpoint and the proxy backend. So it makes no difference whether it comes in through Kafka or through the HCTPM point. And then, so we're capturing we also need to produce. So Adgeria itself is a governance engine, it runs processes. And so it produces open lineage, it's a processing engine. And so here we have sort of capture of our normal metadata, this databases and files and things that are going into the metadata environment. We're capturing the lineage and we're correlating the two together. We're also publishing either the open lineage that came from an external engine so it can be stored and processed later or our own open lineage showing how I'll get the governance processes running because we think it's as important that the governance is governed as well as the data that it's managing. The publishing could, we have a log store so we publish the events in their original format to the log store so any other process can process it offline and we also can publish to Marcairs and you can use Marcairs' API to do additional analysis on that work as well. And as Dan said, one of the things that becomes very important, particularly when you've got a lot of things running, you can't rely on people looking at the logs and trying to spot errors. You want systems to do that. That means you need to know what should happen and those processes that know what should happen are then monitoring the logs to making sure that what should happen is happening because generally it's easy to spot when something fails you get exceptions but when something doesn't happen that's much, much harder because there's nothing happened. But these types of monitoring, if it knows that the process is supposed to run every 10 minutes and nothing's happened for a half an hour, it can start raising alerts to say something has fallen over when we're missing something there. So this is bringing this together. We have the open lineage coming in. We have Marcairs acting as one of the destinations for the lineage for monitoring and Nigeria is receiving, augmenting, publishing the open lineage plus also it can run the processes that do the validation of the operation environment. That's a bit tiny. But here's just an example of viewing the log store. So it's useful for certain types of processes you may need to provide proof that the process ran successfully and so this log store can provide a place to you might not keep it there forever but it's a place to gather that information so that it can be preserved for later use. And this is Marcairs. So this particular picture is Marcairs showing the various runs of a governance process running in Nigeria. So we're now seeing not just Nigeria capturing and publishing but also we're now using Marcairs to monitor how the processes are operating within Nigeria which is really nice. So that's how we're doing on time, that's fine. What was the value? It's not odd function, we've shown you what lineage is and why it's so important but the thing that has been amazing for the three projects is each of us has our individual value that we're all proud of and our ways of working but as we came together not only did we build this extra value between us but each project became more mature so for example Nigeria taught the open lineage team about SPDX tagging. Nigeria learned about Drop Wizard which gave us a really great way of producing lots of sample applications to drive Nigeria and very many different ways. The Nigeria team gave an open lineage to the POPSI back end so between us we're not just collecting things together but we're all enriching each other's communities through this collaboration and so I think this has been to me the beauty of this not only has it solved a very difficult problem for a lot of people who are working with data but our projects have got better as a result of the collaboration. So I think we're on to questions so has anybody got any questions on what we covered or how the projects work together? In any of these projects so these projects are all LFAI and data projects if you go to the LFAI and data site you can see the projects listed and for each project you click on the tile and it will show you how to contact and get involved in the different projects. So for Nigeria for example there's weekly calls community calls and we cover different topics every week and there's also office hours and other things that we have to help people get involved. Each project still operates independently Open lineage is they meet once a month and the aim there is they've got two aims really one is to capture is to get the standardisation of new facets from different aspects of processing and also to bring in new integrations with the engines they're much more focused on getting engines to produce open lineage than thinking about the consumers but I think that's more of a state of time. So that's really a lot of the calls we have with Open lineage is about new consumers and possible extensions to the standard. Nigeria is a very big project with lots of tracks so there's tracks focused on supporting security governance as tracks are focused on supporting data governance we're doing a lot of work with sustainability at the moment looking at how we help organisations think about sustainability monitor and understand their operations in a way that means that they can start to improve the way they operate. So for each of those we have a large scale roadmap that shows how each of these tracks are operating. Generally we have a community call every Wednesday and that's where different stages of those pieces are talked about but if you want to see more of how that roadmap goes because it's open source generally we release every month and that's so that we can keep the levels of software up to the latest levels of all our dependent software and whatever's ready goes. So we can't say this will definitely be ready in October but it's like this is the next thing that will come out and when it's ready it will be released. Does that help? I mean there's an ongoing flow of new integrations new capabilities, new use cases. It's a pretty active project. In going in many directions. So we've got other people here from the Nigeria team as well who are contributing on different tracks really. Yep, go ahead. Yeah, please. You talked about data governance. Myself I'm involved quite heavily with standards. Yeah, lots actually. So our sort of byline is open metadata and governance and we started it about three or four years ago and it was, well, let's follow the standards we want to be open. And what we did was we pulled together lots of standards around catalogs and data descriptions, i.e. metadata about particular types of data, into a patchwork layout and that's our open type system comes from those standards. So what you see is although because they overlap and we needed to reconcile those standards you say well it doesn't exactly spell how that particular standard works. But if you start looking at those standards just map very easily because they were the source of our type system. When it comes to governance the standards like ISO 27.01 for security maps straight on to the standard. The dam of staff marks maps very, very closely on the way that we do things. But what's fascinating is that the way data governance is described is very close to the way security governance is described and then you bring the idle stuff in and you can say oh wait a minute this is all the same. And so Adgeria defines things called governance domains and they're focused on each of these things. But actually they can collaborate and things that are being done in security can help data governance. So we allow that meshing of actions within the organisation for the different domains. Does that help? There's another dimension as well which is around management of standard values on what we often call either valid values or reference data. And so again you'll have different standards to your de facto and de facto standards and of course local standards within companies around particular kinds of data. Whether it's country names or area codes or whatever it's going to be there's hundreds or thousands of these and being able to capture those and use that as part of your quality program to validate that the data that you're seeing conforms to the standard sets of the reference values is incredibly important when you're trying to integrate the data together in order to gather meaning. And so Adgeria supports both the management of the reference data and the mapping of the reference data and then tying that into semantics through glossary terms and other things. Anything more questions? OK, well thank you so much for listening and coming to this I hope this was interesting and please connect with any of the three projects that we've talked about today I think I might have put them on. They are as I say and we continue to meet and collaborate to bring these projects together. So thank you so much.


This study found that robot-assisted gait training using the GTRA device was effective in improving ambulatory function and performance of activities of daily living in patients with stroke. The results suggest that the GTRA device is a safe and feasible option for patients with gait impairment after stroke. However, further research involving different diseases and larger sample sizes is needed to confirm its utility. This article was authored by Kim Tae Kim, Yong Min Choi, Jang Hyuk Cho, and others.


The prevalence of non-alcoholic fatty liver disease, NAFLD, has been on the rise due to the global increase in obesity and insulin resistance. While liver fat plays a major role in metabolism, its exact impact on the development of type 2 diabetes and cardiovascular disease remains unclear. Dietary interventions may reduce liver fat levels, but the magnitude of these reductions and the optimal timing and dosage remain unknown. Additionally, there is no consensus regarding the best method for measuring liver fat content or the most accurate way to assess the effects of dietary intervention on liver fat. To address these issues, a workshop was held to bring together experts in the field to discuss the current state of knowledge and identify areas where further research is needed. This article was authored by Harry P. F. Peters, Patrick Shorwin, Petra Verhoef, and others.


Everybody at Forest and Shawn here. We got a five box bowman chrome or sorry bowman's best university football delight five box break for Christopher C Here appreciate you sir. Let's hit you a super. Such a panic super Dunze panics dual auto super back refractor to 99 Javon Bullard It's a 50 bow nicks nice gold D.B. Martinez blue refractor to 150 a Jared verse the 100 first auto is will shepherd refractor auto Or Vandy or also known as JJ McCarthy on one of gunslingers and orange geometric on a 25 or shot the binion and a nice card here Jersey autograph Travis Hunter I have us hunter Jersey auto for the Colorado a two-way player there It's a good one drew a large studio stars and gold geometric to 50 Braylon Allen, Wisconsin Okay, first box the hunter was nice moving on Raymond higher now Buckner refractor To 99 camera rising shaping to 150 on the blue refractor 75 Julian Fleming on the yellow refractor. There we got Reese Reese on the pink lava to 100 Gold geometric auto to 50s jacquory Brooks Brooks was here Refractor auto Troy Franklin. Hey, it's Franklin. Oh Nice, we got Notre Dame duel Sam Harman and Audric estimated 25 awesome hits Harman and estimated 25 do a lot of graph and Southpaw slinger There you go. Huskies Michael panics. I know you'll enjoy that one And a 15 is a drew pine 15 a 15 drew maybe related to Chris. I don't know there we go better box there I think I would say that it's better box Estimated you're happy with that one. Oh, geez. Oh Geez, whoa, that's a thick card. What is that gonna be? Two relic autos in this box. All right. All right I'm on board with that Refractor Iran's day Iran day gods in the second Graham Mertz to 150 with a red behind it and it is Troy Franklin three of ten orange to 25 Bronson Baron Blue refractor Dallas Turner to 150. All right. Goldgear metric auto Julian Fleming a Blake Corum Jersey autograph late Corum Oh, no Vaughn, you know what that means Means Anthony's on the sauce and check this one out Michael panics patch auto to 49 as Exactly what we're looking for for Chris Michael panics three color patch auto That is a thing of beauty right there clean sig three color. Yep.  All right refractor gold refractor. We didn't skip the refractor Oh, there's refractor. I was gonna say we just get we skip it the refractor Like us the 50. There's a refractor Kool-Aid. I think that might be an extra numbered card then it's in front of the refractor 250 M at Moorhead, BC 200 Mike Michael Williams Jury and Dickie out of 99 Orange Auto Cade Club Nick color match at 25 a Refractor auto of Jason Marshall forgetting Jason Marshall will Rogers Jersey autograph I don't think there was a bonus they was out of order Rogers masterpieces bonics and Gold geometric Paris Jones out of 50. Yeah, there's a good chance. You don't hit on a break like one-on-one Or like a flawless serial number style break Unless it says in the description you pretty you you'll get like a regular hit list pack Sometimes you have like higher-end ones for higher-end breaks rather than like a regular hit list pack But yeah, she's a hit list back Unless it says in the description refractor Breyland Trice Yeah, high-risk high reward on those it's a 99 trade Benson Trash run up, buddy 75 Casey Thompson. Well trash 200 Javante Barnes. I got Ravens I'm going lions, dude. I'm saying lions are taking it Graham Mertz out of 99 go Ravens Lions vends and Ions Riley Leonard orange elements of the game on out of 25 the new Notre Dame QB That's a nice hit Phil Moppa number to 15 on the auto here another key to club Nick Gold auto to 50 masterpieces shoulder Sanders and Number to five JJ McCarthy on the back three of five white geometric That'll do right there to his stuff is scorching since he won the Natty all the Michigan stuff is out of this JJ McCarthy white geometric to five who will get that mag for your Chris Pretty good stuff man. Congrats on that panics again, dude. We'll get these out to you


What's going on everybody got Kyle and Donnie here with latent sports cards and we are ripping 22 and 23 top star wars chrome galaxy It's a six box multi-year mixer number five. This one is random letter style We did a random eyes for the breakout paul b with h down to steven c with o and i good luck guys Let's see what we got on this one Start with the 22 galaxy. We'll save the 23 for second Good luck.  Good luck here guys 22 galaxy to start First refractor that one is going to be a crosshair see there crosshair And a refractor this one is storm troopers assess the gear going to s use the force No, definitely. I'll definitely be using the force here All righty guys tend to go tend to go donner salete breaking next There's a refractor that one's leia that's going to p princess leia going to p Uh, it's not full yet snowball. We got to fill it up first Um, we have donner salete next and then plates and patches Got to fill them up first. Um, this one is going to be um original art cart ot five and refractor this one is lando L there on the refractor lando calriesian refractor this one is patrolling the cantina for p Right. Oh, right. Good luck. Good luck everybody Is that redemption redemption? Oh, yeah It is p1 vintage vader on the base card atomic that one's pretty sweet vintage vader that'll go to v Vintage vader on the redemption nice one there atomic to 150. That's a that's a pretty sweet card Very nice one there for the v spot on the redemption Well vintage e-wap card when all these have been like starting to hit ebay like the redeem version of it and it looks good Nice. I don't think I've seen that one yet.  Was that you dustin nice, man? Congrats dustin. There's a refractor of x wings in formation for x Again guys elite donner's football coming next got just tend to go on that one um refractor this one is anakin And we've got omega for oh omega refractor the star wars Atomic this one is yoda. Um atomic is yoda on um dagoba. It's a 150 Donnie's Donnie's looking at it. Donnie's staring at he he's getting all the atomics He's trying to anyway Refractor we've got a this one is obi one and boga Oh there obi one don't have the yoda yet I mean you have no Refractor c3 p. Oh didn't mean to intrude d right in close to 10 nice I think it a vader one not too long ago like that There's a vintage art of the ewoks Um, this one is refractor darth vader for d Thanks refractor vader. There's a sick portrait there Tend to go guys tend to go on our next one donner's elite football There's a refractor. This one is luke skywalker on indoor l Dark forces that's going to be d um on the star wars game card there d for dark forces a refractor ben kanobi for b There's refractor Or not refractor han Look like a fractor there These like ones like the obi one in the tarkin that have the black back around the heart. Yeah, some of them are Yeah, ooh green got an auto green First up we got bush for b and auto is jamey king as aura sing that'll go to a Character name there aura sing to 99 on the green auto Go to a jamey king as aura sing and there is the first box Do six of these guys? Alrighty guys still tend to go on donner's elite. Alrighty box number two star wars galaxy Alrighty good luck. Good luck guys Good luck 22 galaxy I can do it my right hand but I can't I can't do it my left hand. I don't know We got another auto this one is han solo. This is solo investigates s And our auto is weakway prior to her reqway papaiter that's going to go to w on the auto there. That's w earl brown The actor there darth vader There's a refractor this one is luke skywalker in action It's a little ben kanobi Hey, thank you simon grabbing a spot in plates and patches random teams number seven It's a refractor chewy chubaca there on the portrait for c There's a rebel assault star wars with the r-spot And x-wing for x and refractor this one is going to be bounty hunters b So I thought that was a map of the us vs Like that's random No idea why I don't either refractor of um the emperor's power tea Righty guys still tend to go donner's elite football break it next Jedi knight j And this one is a refractor galaxy's greatest scoundrels for g galaxy's greatest scoundrels So donny is what galaxy's greatest scoundrels There's a lord vader ready to ship out. I don't know those are very nice I'm seeing I'm sorry There's a mace windu that one is mace's barrage m Dude, this is so hard to find The atomic of that. Yeah, that's a sick card, man. No one wants to sell it. Yeah, I like that card a lot There's a refractor that one is um, is this guy a boss boss kb. That's what I thought Um refractor of darth vader vs obi 1d Got a wave this one is um laya organa. That's a 99 nice one. They're l that's that's nice Laya organa on the wave to 99 parallels and this are awesome, man. Always really nice There's a um jango fit And that one is uh e walks under attack e orange It is star wars x wing on the space combat simulator game that'll go to x First letter after the star wars on the game cards here that one is 15 of 25 Nice nice one there x spot And a sky walkers these sky walkers t refractor laya And this one is life on tatooine l the walker got This one is chubaca's anguish c refractor. Um, this one is The sky walker legacy for t Yeah, there you go dusty Vader's a good one. Yeah, but um, some of them are darth vaders. So that could miss you out on a couple but The v spot for vader definitely works. Um, there's boss again b That one's a refractor of um hunter h box three So Right, yeah, i'm surprised he did that It's not very jj play Oh, yeah, you got the orange x wing too. Geez. Yeah, I forgot about that So you got the vintage vader and the orange x wing so far. All right, dustin go pretty good Not bad my friend not bad Appreciate you joining sir. Hey ryan taking bowman chrome a random team for tomorrow night forced Got a refractor that one is anakin and a refractor of omega for o It turns out that's a pretty good spot dust that we hit it right for sure I think those x wing game cards might be the only ones there's one x wing um base card too There's a obi 1 o that was a nice one to get sir There's a mojo This one is number two fifties for b boba fed is ambushed nice one there be Those are sick man the mojos. It's a 50 and r2 d2 Oh my god Let's go Legally still in here. Where's legal? Go birds There's a refractor of didn't mean to intrude d Yeah, the boba fed is Maybe a little boba with a parallel This one is going to be luke skywalker on indoor l Kirk is having a game. Oh, yeah We'll just knock that out.  There's a refractor vader darth vader d and auto is going to be az 3 to 99 on the green for the a spot there az 3 That's who was a beast at florida state too, man. Kirk's like I don't like that. Yeah, don't hit me like that A refractor this one's the refractor. That's anakin anakin on the a spot right, this is a Turnover kind of game Marking down short Is that swift again there short? Oh, they're gonna do the They're gonna do the play Do the play give me those points got a this one is we quay w Do the play one yard rush works every time just do it A few percent a refractor that one is fin um for f It's too easy. All right. He's like six yards into the end zone there. Come on, man There's a star wars battlefront two nice one there be great game Star Wars battlefront two and this one is wicket w You think if you're running back to Philadelphia, would you be mad? Yeah, none of them ever get like consistent touchdowns or touches There's a um Isla Secura a Old boston scott obviously isn't mad. He's been there for what six years now. Yeah You've been chillin it like the third string the whole time Refractor that one is lord Vader and his troopers Empire strikes back there's um, that's the game poster for uh t the empire strikes back Got a refractor of ben kanobi b Refractor this one is a card game of chance a Vader on the base All right, you guys still tend to go Move sketch. Oh nice sketch Good luck guys. It'll be random to all at the end. Everyone's got a chance at this one on the sketch Got a bosc or um bush for b and our sketch I'm not sure who that is, but that is a uh sketch by frank san sony Any star wars saga? I am not sure who that is. You got any help tony? No, it's a nice-looking sketch though. I'm thinking um It's not luke's and Yeah, anybody know who this is? Nice sketch either way. We'll get donny to um Get that one put away. I am not sure. We'll random that's all at the end though guys Really not sure who that is We will random it There's a um, that one's po and refractor of yoda and luke. Why? Looks like chubaca. Good call. Good call. Ah gary Figured it out redemption Wow another redemption It is docking bay 94 on the base wave refractor that'll be to 99 That will be the millenium pelican. It'll be for d spot there docking bay 94 Nice one there. Oh, is it schmi? It'll be schmi, I guess I don't look like the phantom in it schmi so much though Docking bay 94 there for the d spot two redemptions out of three packs and a sketch for three boxes Yeah There's a yoda and luke um that one is luke's lightsaber l All righty guys, I'll do it for the 22 star wars. Um, got 23 coming next. Miss a few packs here. Let me pick them up Yeah, I'm not sure guys. We'll random at the end either way guys to all spots Random that one at the end either way guys still got 10 to go on our next break. It's going to be donress elite football It could be schmi, but she never wore a herald that also she looks Reminds me like the outfit that they're wearing it like um, it looks like she should be like a senator or something Yeah, exactly Yeah, I have no idea what character that is Kind of looks like his mom and the phantom in this. Yeah, maybe man. Maybe I don't know All right, you guys still got three more to go. This one is 23 galaxy now. Good luck Let's see if we can give another sketch Yeah, guys, we'll do a random to all at the end on that one Still got 10 to go on donress elite as well guys. Let's get that one filled up can break that next empty Got 23 now A wave to start us off It is the 75 that one is going to be aza ventrish Assaged ventrish for a a spot there to 75 starting off at the weight pretty solid assaged ventrish There's a refractor this one's sick. It's a uh face the dark side f. Oh, yeah, that art is awesome Um refractor of Darth Vader and his bounty hunters as well It's d That's cool. We pulled the uh orange or that For that one. Oh, yeah. Yeah I would say yeah, that looks that card looks really good with some color There's a Yoda on the refractor. Why is Yoda for w? Why is Yoda got a refractor? This one is um gemorian charge g and a vader imagined reimagined vader Those are sick Um, we still got to fill it first Garrett. We still got to fill it first Yeah, I uh probably should have we got 10 to go and donress I got 14 to go in the first plate 16 in the second and six still on the divisions So we still got to fill it up first Garrett. Um, we'll probably break uh donress elite next Then we'll get going on some plates and patches Still got to fill it up though Refractor this one is luke. Um waiting through the swamp w Refractor this one is a weight of destiny w a lot of w's so far A lot of w's so far on the um on the refractors at least That weight would be um From the admiral f. Yeah, Yoda not honest Anakin only reimagined True And the viking stink There's a refractor. This one is marching atsts m Are you got w2 dustin? There we go? There we go There's a refractor. This one is night must fall in 11 no regular season 0 and 1 post season that graphic didn't need to be made There's a uh dawn of the rebellion for d and maz for m on the women of star wars Um refractor set there Pulled that uh super fracture the first day we had this stuff true There's a um do or do not d there is no try. Uh-oh donnie. Oh It is a 150. It's going to be atomic of the torture droid t The torture droid atomic to 150 and we've got a saga continues Um star wars poster one refractor the sacred tree for t and this one's pad make Um queen amadala. That's q queen amadala there on the uh women of star wars set That one looks really good with a parallel. Yeah, that's sweet a red one of them Oh wait, I think she might have another I think she has a base card. That's really really cool There's a refractor vader redeemed for v looks like one of those uh famous looks like a model but Her I might have seen it. There's aqua. This one is going to be Showdown on dagoba Um aqua to 199 that one's gonna be s And a refractor um poster There's a refractor of becoming sith b. It's kind of cool too like floating the thing over to him his helmet Oh, yeah Thank you joel taking one in the mega dual case Thank you joel down to nine single digits on that one And michael taking one in our random teams placing patches number six Thank you. Thank you guys Let's keep on filling up some uh Some football here still got um tend to go on donner's elite break next Oh That's that show get in there davante. Give me points davante There's a refractor of imperial eye Um that one is going to be for lom Hurts got level two Refractor is vader carbonite chamber c All righty guys box two of star wars um galaxy 23 It's gonna be our fifth break of the day our fifth box of the break. He burned him Good lord Right off the rip good lord.  There we go birds. There's a um, he's all yours h See this is what I thought the first that should be like 28 seven now. Yeah, it is uh officially taken off Thank you There's a refractor face the dark side and we've got a uh zukis z refractor of yoda Now one is why is yoda w poor gamorian guard The gamorian guard had a rough rough go of it Refractor gamorian guard gamorian charge g Is there a card that I missed? It was that card it was base Oh, I just said poor gamoreans That guy could see uh You know a very good run mojo We've got luke to 15 nice one there on luke stands ready l nice mojota 15 nice one there l spot Luke stands ready refractor of wading through the swamp w There is a ton of w cards here a ton in 2023 Get christian taking a spot in the bowman choice tomorrow with um random divisions tomorrow with forest Appreciate you guys still tend to go guys still tend to go on our next one Um donner's elite looking for some pin pal autos turn of the century autos in the new 2023 class Refractor is from the eyes of the admiral f we got a reimagined set a wave It is um to 75 that wave is going to be a squad of bounty hunters will be a Nice one there close up a boba It's a sick one there on the uh wave refractor is marching atst's m There is a refractor of night must fall in and assage a Refractor is dawn of the rebellion for d There is do or do not d. There is no try return to the jedi poster Queen pad may for q or no way. This is a queen portrait a Queen portrait and that one is leah princess leah p on the women of star wars set The women of star wars that was pretty good Yeah atomic refractor this one is the sacred tree And atomic is vader and his storm troopers. That's d Darth vader and his storm troopers on the atomic to 150. Nice one there d Refractor is vader redeemed for v and a return to the jedi poster refractor is becoming sith b They cannot run the ball Patriots signing in book now car. What is going on with the patriots? Uh qb room What is going on? There's a imperial agents i and vader questions to darth vader. Yeah Why are you bringing in 15 different quarterbacks the last two years? But you guys like me. I thought you liked me There's a refractor that one is the carbonite chamber dual seat righty. Thank you. William taking one an elite And cori taking one in the second um place in patches random teams So looks like we're single digits now an elite guys nine to go. We'll break that one next Guys last one of the case here Six box multi-year um galaxy mixer. Good luck. Good luck guys. Let's see another sketch Do a random at the end of the sketch we did get earlier Refractor that one is ambush from above a we've got a sabine ren s Refractor that one is the empire's forces for t For pic no Say oh my god Looks like it. There's a daybreak on hop d And return of the jedi poster This one is uh shiny return s Got a refractor patrolling the indoor moon p and forlump Addison all the way got x factor. This one is the 10. It's gonna be nine of 10 embattled han and shui That'll be for the e-spot nine out of 10 on the x factor nice hit e We'll mag that for you on the um x factor embattled han and shui That's a sick-looking one there nice hit for the e-spot First x factor i've seen out of here Yeah, not over yet newb, you're right Is lobot and this one is the 199 it's going to be reimagined art of the dual Yeah, that's a sick one dustin for sure Refractor is tuskin T got darth vader for d I ain't getting a spin around like that That's a ricky There's aqua to 199 this one is going to be ravenous rancor Rookie on rookie and our rookie didn't do well. Yeah One of the rookies won a fracture of projection and this one is art of the dual All we have to do is run the ball 30 more times Or the clock out Refractor a bucket of bolts be Refractor is leia princess leia p Refractor is um a shiny or a squad of uh bounty hunters a and reimagined this one is um reimagined Art that's uh r2 R2 and cg i haven't seen a lot of that hair. I was like a c3 with hair Makes sense There's a refractor this one is diabolic duo beat and kikori taking one in donress as well Nice down to eight Who breaks next guys look for turn of the century autos looking for some uh pen pal autos Refractor salacious a s salacious b chrome Refractor is yoda's hut y And this one is padme for p women of star wars to left Refractor is um the bespin administrator t The name of the title is That's something Refractor this one is uh vengeful for pursuit v And a return of the jedi poster All righty guys, that'll do it for the break. We'll do a random to all guys here at the end This is going to be for this sketch It was by uh frank sanzone. Not sure who it is. I think it might be schmi not a hundred percent sure Me set up a random here guys. Give me just one second ready roll two dice any times we'll go It is four times another double four two and two All right, you guys good luck all of our letters here a down to w we'll go four times. Good luck. Good luck on the sketch one two three and four s s is taking it home On the sketch appreciate you guys that will do it for star wars chrome six box multi-year mixer Got donress elite coming next. Thank you guys


I didn't know that picture was going to be up there. Ah, hello, everybody. My name is Brooke, and I'm here to talk about a project called ViveCamp, and share a little bit about what we've done to nurture physical manifestations of digital connections. So at present, ViveCamp is a gathering. It's somewhere between a festival, an unconference, and a retreat. But the reason I'm up on the stage right now is because ViveCamp and the scene that we built it for have already been doing a lot of the things that I hear people and organizations working in the network society sphere talk about wanting to do, but often struggle to achieve. However, 10 minutes is not a very long time, so I'm mostly just going to give a high-level overview on what ViveCamp is, and if some of this speaks to you, I'm happy to go into more detail later about how we've achieved some of the things we have. So I joined Twitter in 2020, and the algorithm kind of lumped me together with the kinds of people I felt like I had been searching for my whole life but had struggled to find. You're gonna see the term teapot in a couple of these slides in short of this part of Twitter. This bubble is sometimes called that, although that name and how to define the scene is very hotly contested, so I'm also not gonna talk about that too much, except to say that the aspects that speak to me most are the kinds of people who are maybe very smart, very kind, and kind of weirdos, people who struggled to fit in with mainstream institutions. So Twitter algorithm lumps together like many people. Then ViveCamp unleashed an avalanche of gatherings around the globe of increasing size and duration. ViveCamp is an event built for and by people who met each other on Twitter. So about a year after I joined, someone tweeted that they wanted to see a multi-day gathering. I quoted it, and I pulled together the early team from people who responded to that tweet. And so many lives have changed as a result of that moment. We started meeting weekly, and we kept building in public. Even the name ViveCamp itself was chosen by a Twitter poll. And just a note about this event, despite being comprised largely of people who spend all day on Twitter, at ViveCamp, people unplug, and they connect face to face. So in this presentation, you're gonna see a lot of tweets and not as many photos. But what is it? So I just wanted to hang out with my Twitter friends, but it turned out that I had a lot of Twitter friends, and they had a lot of Twitter friends, and we just kind of hit a cultural chord. The first ViveCamp was over 400 people. The second one took place this year, and it was around 700. It's multi-day, there are cabins, we provide food, and the content is all co-created. We don't hire speakers, we don't bring in hired acts, anything like that. And a lot of it's even just built on Twitter. People will tweet, maybe I should do this at ViveCamp, or what do you guys think about that? And this is just a nice tweet. If you're wondering what ViveCamp was, it was an eager war coming online, the local organizational principle of which is not just love, but boundless curiosity and play.  There were a lot of people who went into ViveCamp also feeling like I had, like they didn't, they hadn't been able to find their tribe, and a lot of them found it there. I'm just gonna do a couple of quick slides to give you an example of the kinds of things that happened at ViveCamp. This one was somebody set up a typewriter and wrote poems, they found prompts. This prompt was the first sentient AI's love letter to humanity. This one was a collaborative tarot deck. Somebody got a bunch of attendees together, they all drew their own cards, and then that was available for purchase after. And at this last one, there was a theater production called The Dating Show to Save the World, which was actually super impressive, it was really, really well done. One Twitter user called it part of the long tradition of alchemical, mythopoetic storytelling, which has its roots in the ancients and its branches in our future. But it wasn't without controversy. Some people found some of the themes upsetting or dark, which leads me to one of the things that I think is the most unique about ViveCamp, which is that ViveCamp doesn't end when people leave the site. ViveCamp ends a week or two later after people are done posting long, introspective threads on Twitter about their experiences there, which is one of the really unique aspects of this hybrid on and offline thing. This person says, it's funny to have a gathering of Twitter addicts because it's like we all have the same big diary. So everybody's reflecting on how things went, but also seeing each other's reflections and reflecting on those. It's like a virtual decomp, but it also kind of extends the experience into the wired, fascinating and completely novel, he thinks. So what he's referring to there are like the Burning Man decompression events, which take place in person, they're ephemeral, they're fleeting, there's no real record of them. But with the sense-making that happens on Twitter for ViveCamp, there's a record. You can look back and see like, oh, this is something people struggled with at this event, have we learned from it? And also changes the nature of the experience of the event for everybody, because you get access to perspectives you never would have had insight into. And so it's kind of like this whole journey that everybody goes on together. It's really interesting. We also do the people from the crypto scenes will be familiar with this. There's like kind of the side event things. So ViveCamp itself is three to four days, but the whole thing is more like two weeks. People just sort of, we roughly use our Discord server for infrastructure, but everything's very decentralized. People just get together and do things. One of our attendees looked in that and thought, well, what would happen if you take out the ViveCamp? So he just did a thing in SF in August. He managed to manifest around 80 people. It was side events with no main event. They just got Airbnb's together. They had gatherings. They had meetups. But that's just kind of scratching the surface of the kinds of things that have been inspired by ViveCamp. Manifest took place this year. And that was one of the co-founders of Manifold Markets, which is an online prediction market platform, went to ViveCamp and thought, what happens if you combine ViveCamp with something like EAG, which is a conference for EAs? And it turns out it was phenomenally successful. People loved it. It was featured in the New York Times. And that was particularly meaningful for me because one of the things that really motivated me to create ViveCamp was the belief that when you get smart, driven people together in a room, they're gonna work on hard problems. They're gonna have interesting conversations because that's just what they love to do. But most of the containers built for those kind of people, things like conferences, tend to be very draining. So the idea is like, be able to have these conversations then go kayaking or go dancing, join a wrestling match. I think it's rejuvenating. It can be even more generative. But it doesn't stop there. The combined impact of ViveCamp and something called JestCamp that took place around the same time in Europe has led to this wave of camps happening throughout the world. And it's really cool because if somebody can't make it to the ViveCamp in the U.S. instead of being bitter or jealous, they say, well, how do I make that happen where I am? And then they go and do it. There's so many of these. They tend to be named after the person hosting them. So this one was formed by a person named Kai. But there are so many of them that this person asked, what would a U.C.amp look like? And the MeCamp industry seems promising. So it's become like a whole meme. And I think part of that has to do with what this person called an agency explosion. And I don't really have clear answers on how this happened. But I think it's related to the fact that we were building in public. We had basically little to no events experience when we started out. And whenever people come to us and ask, is this happening at ViveCamp? Or what kind of vibe will be there? We ask them, like, how can we help you create the kind of event you want? And we're a tribe. Everyone genuinely cares so much that the world rearranged to bring us together. Brooke just channeled lightning to make this happen in six months because it was clear to her how important it was we meet ASAP. And the thing is we can do it. We are capable of great change. We own the memes of production. We have loads of crypto money floating around. This was in March of 2022. And we have some of the best minds of our generation in all this without explicit coordination. So that's kind of what we're asking next, is, like, what can we do with explicit coordination? So again, the algorithm brought together like-minded people. We unleashed an avalanche of gatherings around the globe of increasing size and duration. And now it's time to build something lasting. I really like this map that Richard Bartlett tweeted. He talks about how we figured out how to go from temporary to permanent online relationships. And with this wave of camps, it shows that we can do temporary local community. So what's next is to try and figure out how to do permanent local community. So that's sort of the angle that we're investigating next. And yeah, so ViveCamp, it's run mostly by volunteers. We've gotten one Raketa's grant, but other than that we survive off of ticket sales. We do this because we believe in the vast potential of humanity and we believe in working together towards a future where we're all flourishing. So if you'd like to support us in some way, if you'd like to collaborate with us on events, or if you'd like to consult with us on how to bring your digital community into the real world, please reach out. Thank you.


Hi, I am Atatiyari and I am going to present our work on cross-lingual multi-sentence factor text generation for generating factually grounded Wikipedia articles using Viki data. First, let's try to understand the motivation behind this work. Our work builds on X-language, which proposes the task of cross-lingual factor text generation, in which English resources are used to generate content for low-resource Indian languages. It also puts forward the dataset for this task. Now, X-line focuses on only single-sentence generation, wherein, given a set of facts, you generate one sentence. We explore the problem of paragraph level generation, which is a naturally more challenging task. The more data you have, the greater will be your model's propensity for hallucinations. For this, we put forward several explicit measures for reducing hallucination in this problem. All in all, our work represents a step towards automating the pipeline of generating natural text Wikipedia articles for a given entity. The contributions of our work are summarized here. First, we construct a dataset for paragraph level cross-lingual factor text generation with a clearly defined train test split. This split is based on the two important attributes of coverage and covariance. Next, we investigate several methods for clustering of facts, namely statistical and end-to-end. Next, we investigate the efficacy of several explicit hallucination reduction measures for this task. These are coverage-based prompting, a rebalance beam search, and rewards based on coverage and covariance. Finally, we propose the experiment metric, which is an extension of the parent evaluation metric, which is capable of handling cross-lingual data. The merits of this metric are that it offers more interpretability in terms of precision recall, and that it is capable of handling divergent references. This figure describes the overall pipeline used in our work. First, starting with the baseline dataset, we construct a multi-sentence cross-lingual factor generation dataset by concatenate consensus. Coverage and covariance metrics are used to create training and testing split. Next, we come to fact organization, which involves fact clustering, that is, grouping different facts together, and fact ordering, which is deciding the order of these fact clusters. For all in fact organization, coverage prompts are added to our input to inform the model of the quality of the reference available. This is fed to a free-to-enemptify model, and RL rewards are used to guide the training. Next, grounded decoding strategy is used, which pays attention to the source, following which we obtain a natural language output, which is evaluated using experiment. Next, we describe the dataset use. As previously said, we use the existing SLI dataset to construct our dataset. We concatenate single-sentence examples for every entity in the order in which they appear in the article to construct a paragraph-level dataset. The dataset contains over 100,000 data points across 12 different languages, with several examples with more than three sentences. To create a high-quality test dataset, the instances were partitioned based on two metrics, coherence and coverage, which will be described in the further slides. First, coherence is described as a quality of being logical and consistent. This is necessary since the multi-sentence examples constructed are not always coherent. Since no dataset for coherence classification exists in Indian languages, a synthetic dataset was constructed in which pairs of sentences from Wikipedia articles were used. Positive examples were continuous pairs of sentences, whereas negative examples were randomly permuted sentences. A classifier was trained on this dataset, and coherence scores were described to each paragraph as the average coherence of every continuous pair of sentences. Coverage is a measure of how well facts are represented in the sentences. This is an important measure since X-Line is a partial-aligned dataset, as not all the facts in the reference are present in the source, as can be seen in the example here. For this, a dataset was constructed by removing facts from the web energy dataset and then manually annotating these samples for coverage scores. Next, a coverage classifier was trained on this dataset, and each example in our dataset was coded using this classifier. Next, we move on to fact organization. The intuition behind this method is that facts often occur together. For instance, date of birth and date of death are two facts which occur together in many Wikipedia articles. Explicating this organization can help the model generate higher quality examples. For a baseline approach, we used a bird piece classifier with the number of sentences trained with ground truth data. Then clusters of facts are obtained using spectral clustering with the number of clusters equal to the number of sentences predicted by a classifier. For an end-to-end approach, this problem was treated as a text-to-text problem, and an MTFI model was used to obtain the cluster from a given set of facts, as can be seen in the example here. This step not only gives the grouping of the facts, but also the order of these groups. Multiple generation methods were explored as part of our book. For a baseline approach, MTFI models trained on end-to-end generation were investigated. Two such models were used, one trained on single-sentence generation and one trained on multiple-sentence generation. In our fact-clustering approach, sentences are generated for each fact-cluster separately and then stitched together. Next, coverage prompts were investigated. In this, a coverage prompt of low, medium, or high was provided at the time of training. This value was obtained using a coverage classifier, and this informs the model of the quality of reference available. During inference, a prompt of high was always provided to generate texts that align highly with the reference. Next, a grounded decoding strategy that tends to the source was used. In this, the coverage score of each candidate token is added to its probability score of generation. To increase vocabulary overlap, selective standardification is used here. In this, Indo-Aryan languages are represented using the Dev Nagri script and the Ravidian languages are represented using the Malayalam script. Finally, to further improve the quality of generation, RL-based rewards were used. A reward based on coverage with source was defined using the entailment probability of correction with source, which measures how probable a token is based on the source if it isn't available in the reference. This handles divergence references. And next, a reward based on coverage with reference was used, which is defined using a blue-based reward. All these rewards are added for sentinel generation. Next, we describe the experiment metric that we propose. So blue and rouge are metrics that rely only on the reference text. This is problematic when the reference in the source do not align entirely. Parent addresses this problem by aligning the engrams from the generated end reference text to the semi-structured input. However, parent is defined for a monolingual setting in which all the three texts, the source, the reference, and the generated texts are in the same languages. For our metric, we use cosine-based similarity instead of exact string matching. This metric shows a greater correlation with human evaluation and offers more explainability in terms of the precision entry. The performance of the various methods investigated based on blue and experienced source is summarized here. As can be seen, each method offers an iterative improvement on the previous one with the most sophisticated method which uses factless string, coverage prompting, grounded recording, and green-source RL-based rewards presenting the strongest performance on both blue and experienced. That brings us to the end of our presentation. In conclusion, we explore several methods for improving the quality of generation in cross-lingual factor-text generation on a packed-up level. We also propose the experiment metric which is more nuanced than traditional energy metrics. Our proposed method shows improvement on multiple metrics across many different languages.


Imagine what this looks like if you only had $1,000 when you moved to the US, but everybody told you that you're on H1B, you don't have product management experience, you feel like it's actually impossible. But what if you're able to double your salary, actually smoothly transition into product management, how is this going to change your future career trajectory and your future income? Hey guys, this is DoctrineNCB, a direct product and featured in Forbes. I've helped 100 people lend their dream PM job offer in fan companies and unicorn startup and continue to get promoted as a product leader. If you're interested in product management course, you should go to PMExcelsure.io and learn more. And make sure to check out our Instagram stories, we're able to find so many people just a very similar background like you and how did they make the transformation have those breakthroughs. I believe one of the days we're going to see your successful stories as well. If you like any of the free tips we provide you today, please make sure you hit the like button and subscribe to the channel so that you'll be notified every time published in your video every Tuesday. Today, I had a pleasure to invite our guest speaker on our product leader podcast series, that's Sherry. Welcome Sherry. I'm very, very excited to hear about how you conquer the struggle and actually double your salary. And thank you very much for joining us today. Thank you, Dr. Nancy Lee. I'm really very excited to be in your channel today. Welcome, welcome. So Sherry, why not do a quick introduction, all the audience has been waiting to hear your stories. Sure. So my name is Shelly and I have like four years of experience building B2C products. I currently work as a product specialist at Deloitte and I love building great products which delight customers. So this particular journey of like getting into product management was exciting, challenging and honestly it was something like a dream come true. How exciting. Sherry, why don't you do this? You mentioned that dream come true. Why don't you tell us your top challenges when you get started, especially you mentioned you're lacking lots of resources when you try to transition into product management. Sure. The most primary challenge that I was facing is my confidence because you know that time like I did, I was not confident at all. I didn't have that confidence to go and conquer things but now I have it for sure. That time I just kept thinking about what I had. So I was like I didn't have much resources. I shifted from India to United States with just thousand dollars and then you know there were so many things happening. The COVID just came in and you know like for H1Bs like for F1 students it was so hard to get a job during COVID because they would not accept people with H1B. So I was between going for a dream job and also like looking into my immigration problems. So that was very challenging at that time and I literally didn't know many things about how the US market works. I used to work in a consulting firm and then I did not have enough resources and time to manage everything. To overcome this, what I did is I joined the PM Accelerator course. When I first talked to Dr. Nancy Lee I had a thinking that this is my last shot and I'm gonna like give my 100%. I'm gonna give my time even after so many things that was happening in my life. Then I decided to join Dr. Nancy's class so that I will have a class to attend and I will give that time. Then made a roadmap of my week that how I'm gonna like give my time to the class, do my homework and other stuff. Actually your story is so touching regarding building confidence and a lot of the audience are thinking about the same thing, saying oh I'm not good enough, I don't have this, I don't have time, actually yourself was a consultant like working 60-80 hours per week and you didn't have money either. We get started with your career in the US but you made it happen. This is amazing stories that everyone loved to learn from you. Thank you so much for sharing with us. Can you tell us more regarding your challenges transitioning into product management with a consulting background? Sure, so being a consultant every time I really had time crunch. I used to work like 80 hours a week. It was like my calendar was always filled with meetings scheduled. I didn't even get time to like have lunch in a proper time. I was always here and there. But then I changed my mindset over there. Like I have to do this, it doesn't matter. The first thing I can't leave my job because I need to look after many things so I can't leave my job. I am on visa, I have to do it together anyhow. So that's how what I did is I learned actually. So one week I was into this thought that how I'm gonna manage my time and what all I need to collect it for. So I made a roadmap. So every Sunday I would like do my class in the PM accelerator course and I would make a roadmap of my week. How does my week look like? I have a plan. It's in front of me. My cell phone or my iPad is giving me a reminder. It reminds me and that helps me manage my tasks, right? So I learned that I did that. So the very important thing in transitioning to PM was my time management. My time was really hectic. I was also going through some weight loss because I really gained a lot of weight with stress and bad eating habits. So I had to go for exercise also at least five times a week. But I made a plan. So my takeaway over here is if you plan for anything if you just make a roadmap as a PM skill for each and everything you'll be able to do it. Because you know you'll have notification. You'll have alarms and you'll have so many things. So you have to do it. You have to get it from your bed actually. So Ibrahim, thank you for being vulnerable with us. Not only like stress from work and also gaining weight, you have to work out. It's like no time working 80 hours per week and doing planning. And actually I'm a big proponent like for planning and this is my plan. This is my plan and it's like crazy. This is my like scheduling as well. That works. That works and actually we should teach this inside of PMXR. Like for people to better organize. Yeah, exactly. This is amazing. This is very important. It's just a simple word like time management but this is very important. My second challenge was storytelling. I was not aware of like what storytelling is actually. I didn't know if you ask me a question. I would straight away tell you. I would not like follow any framework or think about it. Storytelling means I was not able like sometimes I think I'm not like I'm not able to craft things very clearly or you know I'm not able to tell my things very clearly or you know my tone changes. So I started working on those. So what I used to do. I purposely used to go to mock interview session which we had in the PM accelerator course and I used to speak my thoughts. I used to speak what I prepared then I used to get feedback there and that was very embarrassing all the time but still I did not stop doing that. Like you know like in front of so many people people are telling you know this is wrong you need to work on this. So this is really embarrassing and it's okay but it's okay. You should keep in mind you don't have to think other people like about other people. You have to think about yourself. I knew I'm gonna improve by this. So I did this. Then one day I really went to the, you must have remembered Dr. Nancy like I came up with such a good answer about. Yeah and you praised me out there right? Like oh that's such a beautiful answer. So you know just give it some time. Don't be shy. Practice. Let other people give you feedback and I'm sure like one day you can crack Google as well. So it's like it's nothing like it's there's nothing impossible. That's for sure I can assure and if you feel if anything is impossible for you like you can talk to me you can call me actually. Thank you so much Jerry. And thank you for empowering everybody and and you're right. It's like I saw your improvement. It was like if you had beginning and then it was like it was so good. Yes and I bet lots of people just felt like I'm not good enough trying to improve but the marginal improvement are very important for people to continue the progress and also seeing how other people oh this is possible. She improved see how she came in through like that now saying I can do this as well. So that's a power having a community. This is amazing. So now tell us your third challenge and how did you conquer those. One of the major issue there was networking. I really didn't want it to network. You know on LinkedIn it's very vague. You send 500 requests 300 people accept your request. Nobody like reply on your messages or in mails or whatever you do. It is so difficult but if you craft a message especially for a person that person will definitely reply. That's what I learned in PM accelerator class like craft a message individually for a person. Suppose you see an ex job so you want to email a person from that ex organization so you really need to craft that message for him that resonates with the other person that resonates with you and that's how you connect. So connecting and networking is very, I know it will take time but this like trust me this time is worthy. After I started doing that I got more than 60 interviews. I never failed a recruiter round and you know like out of 50 interviews I got like five offers in which one of I had is like doubling my salary and like so many other good things. So start connecting. So start connecting means show your interest. Like if you start showing your interest if a person knows because there's a lot of opportunity in this tech world. It's 2022 and we know how tech is growing. I got a lot of leverage of time. I never used to have time so scheduling interview was like a challenge for me. I really used to tell them to wait for a week but they waited and I interviewed there. So that was ready. So I understood that okay I need to connect. If I connect things work. This is amazing. I love your gut that is okay because you connect with a hiring manager and recruiters is like I'm very busy I'm freaking concerned. I'm working 80 hours per week. Can you wait for a week? They actually waited for you. This is amazing. Actually it happened. Awesome and actually the same strategies apply applicable to our students. Sonia and she did the same thing. She was like six months pregnant. She was like I can't just say I'm busy and she had a full-time job. It's very uncomfortable. She said your efficiency drop significantly when become a student. That's why I like this. Five times less efficient once you're pregnant. And then she said the same thing. She said they should start like I'm the center of the interview. Hey you guys slow down or she reject people for interview. She was the one managing her own time learning time management telling others they do not do things or she need to make them wait. It's so powerful. So powerful once you know how to connect with others. Sherry, so let me answer this very most important interesting question. You had double your salary and given you only had $1,000 when you moved to the U.S. or like as a student on HMB. So what is the most important shift that pushed you to the next level? I had to change my mindset and my attitude towards learning. I used to think that okay this is not possible that is not possible but no. If you want something everything is possible for you. Nothing is impossible in front of you. So always before going to school or before like graduating I had a vision that I have to be a leader in this industry. I have to own my career and you know like act as a leader. Do some things that really affects or impact the community. So I totally changed my mindset. Thinking that okay this is my goal. I'll be working towards it. If I reach here to some point it's fine but one day I know working like this one day I'll reach to my goal. So that was the you know the most important shift that believing in yourself. So I started believing in myself. If I used to see someone speaking very well I used to like learn. Like I used to imitate them like speaking that way so that I learn. If I want to speak that way I have to learn right. I have the full planning. Okay planning changes but there's no harm in planning. You can also like work on that planning. So I believe and I work like that honestly. This is amazing. That's why you're contributing back to the communities. Can you tell us more regarding how you leverage communities how you help others and how other people help you as well? Yes. So one very beautiful story about this. My husband was a software engineer and I helped him transition to PM. Now he's with work day working. So he was interviewing for product manager role. But he was offered senior product manager role. So that is one way I'm giving back to society. Okay next was resources. So I would get a lot of resources on YouTube, YouTube, Googling but you know there was no validation. But when I went to PM accelerator classes I used to go to class. I used to have my question ask question discussed with the community and honestly guys community is a blessing. Like you will never regret getting into it because you learn from different people and you learn from different minds like different perspective of people like what they are saying about and what they are talking about product. So that way joining that community gave me confidence. Like okay I get to read so many answers. I get to read so many things. That gives me a quite good knowledge. This is amazing. Thank you so much. That's why we invite you back to be our guest speaker on influential product theater podcast series because you actually the definition of strong leader is helping others to grow together. This is amazing. Awesome. So can you tell us what the day to day look like as a panel manager right now in your current company? So right now I am working on okay so there is some gap analysis that I need to do in at Deloitte which is a consulting company but it has a lot many of its own product. So there is a gap analysis happening between industry like tech products and the consulting product. So I'm working on that. I'm also working on a pricing analytic website where you know where you can shop for consultant. When you come on the website you can shop for consultants. Like if you want to build something you need support or you need to change your legacy system into cloud. So basically I'm working in cloud and it's like my day starts with daily stand-up meeting then a little bit of market research then I go into customer research what other companies are doing and then I have like a lot many stakeholders meetings. So mostly my days are filled with meetings and rest of my day I give it into research to know like why we are building this what people want how we can analyze data to make this better. So mostly goes into me stakeholders feedback and then you know reprioritizing it. So a little complicated that side but I enjoy doing it. This is amazing. Thank you for sharing with us for lots of people who want to learn more about what the data will look like as a plan manager and what's the skill set. Feel free to check out more videos on top of the screen right now and you can start learning for free starting from today. Awesome. So Shirley let me ask you the last question. What specific advice you have for others who want to jump-start your plan management career? I would say follow your instinct like understand at the very early in your career don't wait for too long and understand like what you want to do.  I would say in early career just try to toggle between things and see if you're interested where does your interest goes and then choose it like it's never too late you can always switch into different professions like that's completely fine but you know I would like what I did is I thought I understood I made an effort to understand what I like to do so I did this so because I believe like if you're interested in something you're more likely to be successful. Yes and I love what you do this is amazing and that's combining your hobby your passion with work as in the best thing ever that happened to everybody we spend so much time working every day it's better to work something you really love now specifically if people already decide they want to become a product manager so what advice do you have for them if they decide we want to become PM what should I do next? So if you have no idea and you want to start on your PM journey I would say enroll in Dr. Nancy's BM Accelerator Program and also do your own research own study set up a intro call to understand more about the program see a few videos on YouTube there are many free videos on YouTube and then you decide that how you want to go about it. Awesome thank you very much Sherry and for sharing with all of us and for people who is interested in PM Accelerator and feel free to go to this webinar with the step number one you to attend this webinar learn more about our program and also schedule an interview with our head of mission to see the qualifier for the program we're going to link the webinar in the description of this video and feel free to check out all the free training on YouTube as well thank you very much Sherry and thank you for joining us I'm gonna see you next time and really have you to join us today thank you alright see you guys if you like the free tips we provide today please make sure to like the video and subscribe so that we notify every time we post a new video every Tuesday and this is Dr. Nancy Lee from PM Accelerator I'm gonna see you next time please comment in the description let me know what kind of new videos you want me to film and we're looking forward to hear from all of you guys subscribe and like and see you soon bye