Good good morning. Good afternoon. My name is Alejandro Saocedo. I'm the chief scientist of the Institute for ethical AI and machine learning Today, I'm going to be talking to you about the state of machine learning operations in 2019 I'm going to be covering a high-level overview of the ecosystem So I'm gonna try to go through several topics within 10 minutes You'll have several reading material that you can access to check more so to get started the motivations Is to give an overview of data science projects and more specifically the small ones When you have the traditional sort of data science workflows where you get data, you clean it, select some features, iterate and build your model When you're happy, you can put it in production, perhaps wrap it on a flask service You know, perhaps just, you know, do a very simple web app around it It works relatively well. There's not that many problems However, as the data science team grows and the requirements grow, there's an increasing complexity of the flow of the data Each data scientist want to use their own tools. They have their favorite languages Serving models becomes increasingly harder And also when stuff goes wrong, it's really harder to trace it back As your technical function grows, also shoot your infrastructure And this is why it's challenging because we're dealing with the intersection of data science, software engineering and DevOps Creating this, you know, intersection of machine learning engineering, bringing best practices from each field And that's why we created a list of libraries to deploy, version monitor and scale your machine learning We're not going to have time to cover the entire list, which focuses on our eight principles for responsible machine learning But what we're going to cover today is three principles, responsibility, orchestration and explainability So the first one focuses very similar to what was discussed in the previous speaker on the data and model provenance and versioning of them And this often is explained as the abstraction of the pipeline on its computational steps And this is primarily on separating them on the data that is coming in, the actual code and configuration and the data that is coming out By abstracting this atomic step, it allows us to go even one level higher and have our entire pipeline or entire ETL pipeline abstracted in this sort of compliance perspective And there are several libraries that are trying to tackle this challenge of reproducibility, running something in one environment, being able to get the same results in the other side And some of the libraries to watch, they're not just the ones that I'm going to mention, but one includes data version control It's a git like client that allows you to do version management of your data, code and output so that you can actually track and version every single step Model DB, it takes in a different step, a different approach, it actually allows you to track all of the inputs and outputs that your code generates And then gives you a dashboard to be able to visualize the performance of all the models and all of the experiments that you've run Finally, Packyderm is a framework that allows you to actually build, as they call it, compliance machine learning pipelines And this means that you can be able to store and version every single run of your inference pipeline So if you have a machine learning model that is trained in production, every time that it runs, it would actually store everything that goes in and out The second one is model orchestration, training and serving at scale This involves computational resource allocation, which is hard because it's like building an operating system which is completely distributed and your resources are all your notes However, there's been a lot of really interesting areas which I'm going to delve One of them, Algorithmia, this startup in the US has a very interesting comparison of the usage of traditional servers versus their actual usage And they actually show all of the unused space that they have by just having those servers at all times By using elastic servers, you still have some inefficiencies and by using serverless, it actually becomes more efficient There was actually a very interesting paper that was published a few weeks ago that was taken with a lot of criticism in the serverless space Because that actually highlighted some of the, you know, downsides that right now serverless doesn't yet have And I would recommend you to check it out One of the things that I didn't mention is that you can access the slides on that corner on bit.ly slash mlops19 A few libraries to watch include Selden, which does orchestration primarily on TensorFlow servers The second one is ML Leap, which dives into the serialization of models So it actually converts the trained models into JSON equivalents and then loads them back again and runs them across, I think, these three libraries And the final one is DeepDetect, which covers a broader set of machine learning frameworks And it builds a unified API to be able to interact with all of them for training and inference Which I recommend to check out The last one is Unexplainability, this covers the challenge of explaining and understanding black box model situations This requires going beyond the algorithms through not just the machine learning best practices, but through tools, process and domain expertise The way that we've often tackled it in a talk that we also link is through three main steps Data analysis, model analysis, and finally production monitoring The data assessment includes like things like class imbalances, assessment of protected features, understanding correlations The model assessment is understanding feature importance, you know, using model specific methods And the final one is the model production monitoring, which is assuming that as soon as you put a model in production, it has the danger of starting to diverge Whether it's because of the data changes or another reason So it's important to set metrics up front to be able to monitor that Some libraries to watch include LE5, Explain Like I'm Five, which has several tools to be able to open up models both from the NLP side image and tabular data The second one is SHAP, which is a unified set of best practices that have been brought together and represented as shapely values They have model specific as well as model agnostic approaches and allows you to visualize in a neural network, for example, what are the areas of the network that influence most that result And the last one is one that we're maintaining, it's called XAI, which allows you to analyze data sets as well as models and set metrics to monitor in production All of them, all the links are set below, you can access them when looking at the presentation And unfortunately, 10 minutes is definitely far from enough to be able to cover all of the challenges that you face, especially now with this intersection in machine learning And that's why, you know, I recommend you to go check out some of the open source projects that are there You know, the list itself is open source, so if there is libraries that you know that are not currently there, please feel free to add them And also jump in and contribute, the majority of the projects are open source, except the two ones in the bottom with the dollar signs, which are commercial But well, there's no open core as well in the other ones, but I do recommend you to check them out And yeah, so with that said, you know, I managed to do it on time, which is quite happy And we have, I guess, time for one question, and yeah, thank you very much


Remember Gino? He's back in me form. Again, originally introduced as a costume for the me-fighter in Super Smash Bros. 4 on the Wii U and 3DS, Gino had been missing from Super Smash Bros. Ultimate. With the release of Sephiroth as part of the fighter's past volume 2, Gino's costume is making a return. While it's nice to see Gino get some love in Smash, many Super Mario RPG fans are disappointed that the character isn't getting the full fighter treatment. This frustration is understandable, and from the sounds of it, Masahiro Sakurai feels your pain. He's been pushing for Gino to get a more significant role in Smash for years. When the Gino costume was originally added to Smash 4, Sakurai was asked why such an old, mostly forgotten character had been referenced in the game. He said, To tell the truth, it's because I wanted Gino to be a playable character. He has a gun for a hand, and I think he fits in really well with Smash. During Brawl, I thought it would be great if I could add him, but in the end it didn't become a reality. He's very popular. When talking about older characters, Gino always gets a lot of requests. He didn't appear as a fighter, but I was able to achieve something kind of close to that as a Mii costume. Alas, as Sakurai has recently pointed out, he doesn't always get his way when selecting new Smash characters. He said, When thinking about new fighters for this game, we don't just pick my favourites or draw from a hat. More often than not, these discussions start by Nintendo approaching me with an idea. Of course, if I can't do that idea justice, I'll tell them no. So, no matter how popular Gino may be, and no matter how much Sakurai himself might want to include him as a full fighter in Smash, this isn't likely to happen unless the higher-ups at Nintendo really want to see it happen. That is particularly unlikely, considering that the company is actively moving away from traditional role-playing games set inside the Mario universe, and doesn't seem eager to revisit their collaboration with Square Enix. According to Kensuke Tanabe, director of Super Mario the Origami King, since Paper Mario's sticker star, it's no longer possible to modify Mario characters or to create original characters that touch on the Mario universe. This change in approach for Mario RPGs comes straight from Shigeru Miyamoto, who has actively discouraged game designers from putting stories in their games. According to Tanabe, there were two main things that Miyamoto-san said from the start of Sticker Star. It's fine without a story, so do we really need one? And, as much as possible, complete it with only characters from the Super Mario world. It's likely that Miyamoto is opposed to characters like Gino, who expand the Mario world because they stray from his specific vision for the franchise. Hence why the Paper Mario series is deliberately shown to be separate from the traditional Mario canon. Not to pile on Miyamoto, but he has also had similar conversations with Yoshiaki Koizumi. The two of them have often butted heads when it comes to storytelling in games, and Miyamoto is generally unimpressed when Koizumi tries to expand the lore of the Mario universe. A character like Gino stands in star composition to the simple, clearly defined direction that the Mario series has taken in recent years. Meanwhile, Yoshihiko Mayakawa, director of Super Mario RPG, has said that he's in no hurry to try and revisit this universe. He said, Actually, I don't feel like I want to return to that world and use those characters in another game. I feel like I was able to accomplish what I wanted to with that world and those characters and the gameplay ideas that came out of those, and now I mostly feel like I want to make something different. So with Nintendo remaining apprehensive about bringing Gino back into the spotlight, or adding new characters to Mario RPGs in general, it's unsurprising that the company has rejected Sakurai's pleas to get Gino into Smash. Thus, if you're disappointed that Gino is only available as a Me Fighter costume, bear in mind that Sakurai is trying his best. This costume might not be what you've been hoping for, but it's something. That's the moral of this story. You won't always be able to have everything you want in life, but that's okay. Things do feel a little bit better in that moment where we're appreciating the things that we do have.


Digital pathology is becoming increasingly important in modern clinical practice due to the advancements in information technology. Artificial intelligence, AI, and machine learning, ML, have enabled the automation of image analysis and the extraction of valuable data from digital images. This data can be used to improve diagnostics and predict outcomes, as well as to develop new diagnostic algorithms and prediction models. This article was authored by Mihaila Moskulu, Roxana Moskulu, Christina Gina Daskulu, and others.


Hello my darling extraterrestrials, I am Kim, this is Dusmoats and Velikor, and Arusha in the end of time is delightful. Just utterly delightful. This sucker is a middle grade reader by Roshni Chakshi, who as you know I kind of adore. Our snarky heroine Arusha lives above the museum of ancient Indian arts and culture with her mom, a distracted and often absent museum curator. So she's got a lot of spare time on her hands. Which she uses to concoct fanciful stories about the fabulous vacations she totally goes on to compete with her wealthy, somewhat snobby classmates. But when three of them try to catch her in a lie under the evil red eye of recording video, she offers proof that she wasn't lying by lighting a lamp that her mother told her is cursed and therefore should never be lit. And she maybe sort of releases a demon called the Sleeper upon the world. What follows is a fast paced adventure filled with Indian mythology and quick thinking heroines and quite a lot of screaming. Their journey takes them through death and reincarnation, the other world, a Costco with a mystical armory department and a sentient palace that has abandonment issues. Aaru and her soul sister Mini, who's her partner on this adventure, use their wits, their skills, their compassion and their imagination as they quest to return the world to the way it was and on the journey they grow into themselves a little bit more. Just enough for the next adventure. Now don't get me wrong, I really enjoyed this book and I really recommend it. But a thing that I noticed was that it doesn't sound like it was written by Rashi Chakshi. Where is the delicious language of Star-Touched Queen? Where are the poignant moments of introspection from Crown of Wishes? Though, to be fair, all three books have a plethora of semi-nonhuman quirky side characters. Yes, yes, I said semi-nonhuman. I guess it feels like some editor somewhere took all of Rick Riordan's books, boiled them down to the salt and said, this is the formula now. Use this, it'll sell. Which makes sense because this was published under Rick Riordan's new imprint with Disney Hyperion. I love Rick Riordan, but I feel like Rashi Chakshi had something new to offer to the genre and she wasn't given that opportunity. Her work has been food for my soul and it was strange to see this one so skeletal on that count. But there are three more books coming in this series. Maybe this story is like the girls at a town hungry for a better understanding of the world and really just getting started. But if you are looking for a fast-paced adventure with quick-witted girls with a lead, definitely check this one out. I would put the reading age at like 8 to 12. It's funny, it's fast, it has girls sprinting headlong into situations that they're totally ready for. They just don't know it yet. If that sounds like your cup of tea, definitely check out Arusha and the End of Time by Rashi Chakshi. Come talk to me. I love talking about books and I'm honestly a little confused about this one because I can't tell. Like am I wrong or did she do it intentionally? I don't know. I feel like Rashi Chakshi could write the next Alana song of the lioness series. Something where your characters grow into adulthood along with their readers. Something where nuance and adventure can run hand-in-hand into the sunset. I don't know. But I feel like she has that potential and I just didn't get it from this one. So come tell me I'm wrong in the comments. A viento!


Girdling the earth between Cancer and Capricorn lies a great belt of tropical forest. From the basins of the Amazon and Orinoco, it runs across the middle of Africa between the dry savannas and on to Southeast Asia and the islands of the Pacific. At its heart are the equatorial rainforests, the greatest celebration of life to evolve on earth. Above ground is a riot of vegetation and varied life. But the soil from which it stems is paradoxically poor. The secret of the forest's abundant growth lies in the unique way it recycles its nutrients. Decaying vegetation is rapidly broken down by the profusion of fungi, bacteria and insects that inhabit the forest floor. And just as swiftly, the organic matter is reabsorbed so efficiently that very little remains in the soil. No rain falls here than anywhere else on earth, yet the forest absorbs it like a giant sponge, turning sudden and violent onslaughts into a steady release of water. Its tangled roots holding back the soil, its streams and rivers running clean and clear. In their richness and diversity, the tropical forests are the most complex of all ecosystems, and still the least understood. But from them have come an astonishing number of our staple foods, essential medicines and industrial raw materials. How many more remain to be discovered, we can only guess, and we may never know. For now, after millions of years of evolution, the great forests are threatened with extinction. No one can say for sure how much has already been lost, but there's evidence that a fifth has been destroyed in this century, most of it in the last 20 years. If the rate of clearance continues to accelerate, another two-fifths could be destroyed by the year 2000. In 40 years, the tropical forests could virtually disappear. If they did, the result would be more than an ecological disaster. It would be catastrophic for millions of people who depend on them for their survival. Why then is it happening? In Southeast Asia, hardwood timber means foreign exchange, revenue desperately needed for development. Not every tree has value, but felling just one in 20 can damage up to half the surrounding forest. And is already 30 times what it was in 1950. If extraction rates continue to rise, most of their prime timber will be exhausted before the end of the century. Even so, the international timber trade accounts for only a fifth of the total forest clearance. The rest is being destroyed in the struggle for human survival. East Africa, where the tropical forest fringes out toward the dry savannas. The daily cutting of wood for fuel. In the foothills of the Himalayas, the same relentless attrition of woodland at the forest's margin. Nepal and Kenya, two countries caught in a predicament that affects close to half the world. Two billion people are dependent on wood for survival. Condemned to destroy it faster than nature can replace it. And as the forests recede, the back-breaking trek for fuel grows longer every year. For each family, just a few pounds of firewood a day. But worldwide, it means the loss every year of over 60,000 square miles of tree cover. Much of it from fragile arid lands and vulnerable upland watersheds. For these people, simply providing enough food is in itself a struggle. To be compelled to spend their days in the endless search for fuel is an added burden almost impossible to endure. And all for a few sticks of wood with a pitiful yield of energy. For the urban poor who have to pay for it, the growing scarcity is pushing it beyond their means. Whether they use charcoal or wood, providing for basic needs can cost a family up to 40% of its income. For the poor throughout the third world, the energy crisis is not about oil, but about wood. In the rainforest areas, it's not fuel that scares. It's land. Down the trails left by the logging companies have come the squatters, the landless poor. No room for them on the good farmland. No jobs for them in the towns. Desperate for somewhere to settle and grow food, but unaware of the infertility of rainforest soils. As a result, the practice of slash and burn agriculture has become the greatest single threat to the rainforest's survival. All the more tragic, because the destruction achieves so little. Burning the vegetation to get at the land gives it a deceptive injection of fertility, and for the first year or two, crops flourish. But without any further replenishment, the soil soon becomes exhausted. Yeels fall away year by year until crops fail altogether. The promised land, an illusion with no future. In Nepal, the same relentless pressure of population, but different consequences. For centuries, its people have farmed the fertile valley floors and terraced the lower slopes of the Himalayan foothills. But with the desperate need to bring more land under cultivation, there's been only one way to go. Now the terraces climb even the steepest slopes. The trees cleared for crops as well as fuel, while cattle and goats strip what's left of the vegetation. Roughly half the world's people live in areas where watersheds are inadequately protected. Where the pressure of growing numbers is destroying the tree cover that holds the soil. In Nepal, when the monsoon breaks, the consequences can be devastating. It can take just five years to strip fertile land down to bare rock. But with every monsoon, the damage spreads far beyond the hills, and rushing thick with silt on their way to swell the floods on the plains of India and Bangladesh. When the forces of nature run uncontrolled, national frontiers cease to have any meaning. The only reality is the appalling extent of human suffering and loss. The threat to the tropical forests. Every minute 10,000 trees fall victim to the overwhelming pressure of people's need for timber, fuel and farmland. As a result, natural habitats are being destroyed, vast areas of land degraded, and control over soil and water lost where it is most needed. At the root of the problem, too many people, and a chronic lack of alternative resources. The predicament of millions in the third world who are destroying tree cover not from choice, but of necessity. What then can be done to avoid the impending disaster? In Nepal, the beginning of restoration. Replanting denuded slopes with a quick-growing species of pine. A pilot project aimed at preventing further erosion in a district close to Kathmandu. Trees to stabilize the soil. And to control the runoff of water, check dams. A series of simply constructed walls to plug washed out gullies and ravines. It's a start, but the task is formidable. This is one of the poorest and most inaccessible regions on Earth. To multiply these measures along the entire Himalayan front would take years of effort, and international aid on a massive scale. And even then, it would be futile unless at the same time, the destructive ways of the past can be changed. Take it out. Remaining woodland, a new plantations need protecting, but government can't provide the manpower. It's the farmers themselves who are being persuaded to do it, through the local village councils. At this meeting, they're discussing what finds the guards should impose. What's the name of the plantations? I'll give you the address. I'll give you 10 rupees for 5 rupees. Is everything alright? If you're not a farmer, I'll give you 10 rupees. If you want to keep the plantations, you can keep one. It's asking a lot of poor farmers to give up their traditional rights. But they will if they can gain in other ways. Here in Nepal, they're beginning to get help from government. Fertilizers, better seed, cheap credit, access to markets. The chance at last to get more out of their land. And what they must grow is not just food, but fuel. In Kenya, this has been government policy for a number of years. They're gradually overcoming the farmer's reluctance to grow trees by providing them with varieties of eucalyptus that take only 7 years to mature. Kenya's aim is to turn chronic shortage into self-sufficiency by the year 2000. Not just in the rural areas, but in the fast-growing towns. Here on the outskirts of Nairobi, the guarantee of regular supplies. For the market price of a single bundle, they can get a permit to gather a load of plantation trimmings every day for a month. The plantation is one of many that are being established throughout the country. The logs are for sale in the city. The trimmings a bonus for the people who live nearby. It's still the same backbreaking load, but it's only a short walk home. But for the government, the solving of the fuel-wood crisis is a daunting commitment. 50 million trees to be planted every year, and that's just Kenya. Worldwide, it would require investment on a scale the developing countries couldn't hope to make, unaided. Growing trees instead of destroying forests is a simple enough idea. The problems are always finance and people's resistance to change. But when these are overcome, it can work a remarkable transformation. A pulp mill on the coast of Mindanao. What makes it unique is its reliance on wood supplied by small independent tree farmers. It's a scheme developed by the paper industries corporation of the Philippines. On signing an agreement to grow trees for the mill, they're given access to technical assistance and a bank loan, repayable only when their plantations start to produce an income. Many of the tree farmers came here as squatters and found only despair. But now they have a stake in the future. Instead of a life of poverty and bare subsistence, they can afford to invest in their land, restore its fertility and produce a surplus for market. And after only seven years, their first trees are ready for harvesting. A commercial enterprise in the Philippines has shown what can be achieved with foresight and imagination. For the company, a valuable addition to its timber supplies. For the settlers, a secure and steady income. And for the surrounding forest, a reprieve. Here at least in one small corner of Mindanao, the threat from slash and burn agriculture has been removed, making forest land productive. In Malaysia, the approach is even more radical. Faced with a rapid decline in her reserves of hardwood timber, the urgent need was to develop an alternative source of revenue. The plan they came up with was to retain over half the remaining rainforest, but clear the rest completely, replacing it with tree crops native to the forest, mainly palm oil and rubber. At the same time, they're solving a chronic social problem, providing work and housing for thousands of poor, landless families. From the more backward rural areas. Within two or three years of arrival, they're working the plantations for themselves and earning an enviable income. Twenty-five years of experience have proved that the plan is environmentally sound. The plantations function like natural forest in holding the soil and regulating the release of rainwater. By the end of 1980, they were over 740,000 acres of oil palm in production and nearly 500,000 of rubber. Forest and people transformed together. Thousands of families, given a new economic status and access for the first time to the amenities of a thriving modern community. Time for recreation. Money to spend. And a revolution in the quality of life that comes with electricity and clean piped water, health clinics, libraries and schools. Malaysia's land development scheme is a unique achievement and offers an important lesson. Selected areas of forest can be sacrificed and successfully converted to more intensive use, but it takes rare expertise and ample resources to do it. The Trans-Amazonia Highway cut through 2,000 miles of Brazil's virgin forest. To some, it was the symbol of a nation laying claim at last to its vast uncharted interior. To others, it signaled only destruction, the first assault on the last great reserve of tropical wilderness. The scale of the clearances that followed seemed to confirm their worst fears. But the fate of Amazonia is still an open question. Its area is so vast that most of it remains undisturbed. Undisturbed. But for how long? This is the world's greatest untapped reserve of tropical timber. And as the hardwood resources of Asia and Africa are depleted, the pressures on Amazonia are bound to grow. Must its exploitation lead inevitably to its destruction? Or can the rival claims of conservation and development be reconciled? The Brazilian government is engaged in a major program of research to find out. Their studies are concentrated on one small area, but from it will come evidence valid for the whole Amazon basin and the answer to a fundamental question. How much timber can be taken out at any one time without destroying the forest's power to recover and go on yielding indefinitely? 65. On each of a series of trial plots, they select a different number of trees for felling. By varying the intensity of logging, they will be able to compare the effects over a number of years. In an area logged two years previously, an investigation of the forest's natural powers of recovery to discover what species are regenerating. This is a good species, isn't it? This is good. It may be that nature needs to be assisted by additional planting, so that too is being investigated, using seeds gathered from some of the finest specimens in the forest and reared in a nursery. I'm going to mix these together. Where were they collected? Results from the trials and experiments now underway cannot be expected quickly, but they're the key to a unique opportunity, a chance to plan the future exploitation of Amazonia's hardwood resources by working with nature, rather than against it. Meanwhile, elsewhere in Brazil, the growing demand for other forest products is being met by a different strategy. Forests where none grew before. Already, a total of 25,000 square miles of unused marginal land have been planted, and plans are in hand to more than double the area. Brazil's commercial plantation program is the largest in the tropics, already providing two-thirds of her needs for pulp, lumber, and fuelwood.  Yields are up to 20 times those of an equivalent area of natural forest. If only financial resources on this scale could be made available throughout the tropics, the pressures on the natural forest would be eased dramatically, reducing the pressures. Everywhere, this is the crux of the issue. For the one thing we need, this is the crux of the issue. For the one thing we cannot do is to recreate the forests themselves, rebuild what nature evolved over millions of years, a forest reserve in which the only intruders are scientists. So much remains to be discovered. To lose this rich storehouse of plants and animals, this vast laboratory of living processes might be to deprive ourselves not just of knowledge and understanding, but of resources vital to our future well-being. In every distinctive environment, zones like this need to be established and totally protected against any disturbance. Area is large enough for all forms of life to continue to flourish. Enlightened governments have already begun to build this ultimate line of defense, this refuge of last resort. In less than 50 years, it could be all that's left. The tide of destruction continues to advance. But even now, it's not too late to turn it back. What's needed is a bold and urgent attack on the root causes of the problem, a concerted response from the whole world community to help raise the level of existence among the poor of the third world. Only when land is used far more productively and when energy can be provided cheaply from alternative sources will the threat to the tropical forests be finally overcome.


Hello team. This is JJ. Give it a few more minutes. One more minute for late commerce to join and then we'll get this started. And in the meantime, people who have joined if you want to take in the dock with your name, please do. Also any volunteers for scribe, scribing today's meeting. Highly appreciated. Who's running our meeting today? Are we waiting for the moderator?  Yeah. Do you want only people that have something to say to put their name in there? Or should we say no update in there? Yeah, no updates. Probably fine. Yeah, okay. I think you don't have to walk around the room. Yeah, I'm also doing the in the chat. Yeah. Thank you. I'm also doing the mimic thing because my wife at home is not good. So I'm using laptop for no for doc and phone for the call. All right. So as people have their name, let's get started on chicken. Emily. Emily Fox. Hello. It's Emily. So lots of great things that happened over the past week. We got all the CFPs reviewed. We got an agenda put together. It's been put out all the attendees have accepted. And we have great sponsors, which means that we've got lunch and happy hour. So lots of great things for security day. Yep. Thank you. Yay. Yeah. It's a, it's a happy half-heart essay here. So. Oh, we don't have two scribes. I'll, um, I can be a scribe as long as somebody else can do it while, uh, because I'm also going to talk about the open assessment. I do need somebody else is willing to type. Yes. Can someone volunteer for state? Please. I'll try to cover in for scribe. Um, so I don't, uh, next to me, so I'll just give my update. I don't have much of an update. We, I'm working with the Harvard to see if we could actually pull in all the policy documents that are lying around and then into the six security repo so that it'll be easier to track and discover. Six security. Emily Fox gave an update on that and, uh, we'll be Sarah Dan and I will be meeting up with, uh, with John Liz on 30th and we'll circle back with the team with any updates from there. Daniel. Next. Yes. So, uh, unfortunately for the last couple of weeks, I couldn't attend the meetings. Now I'm fully back and I saw that. Falco assessment kicked in. So I need, I will need to catch up with this, since I was one of the volunteers that wanted to do it. Yeah. Yeah. I think there's a, um, issue open now that, um, Robert opened. So if I'll try to put it in the notes. Um, but yeah, that would be great for you to, you know, time in on the issue and make sure that you're on the list. Perfect. Thank you. Because I found it. Yeah, if you can link that. Yeah, I'll stick it in the notes. Thanks. Do you want to go next? As I come off mute. Yeah, we've got a lot of work for the security day at cube con. And one other note, we will need someone to be able to do an update at our next meeting on the first. So taking volunteers for that. Oh, these are the updates for the to see about like, what are the six doing like all the things. So, um, so yeah, so JJ did that last time we wanted to iterate on the format a bit to make sure that like, have a little more lead time so thanks for mentioning it, Amy. I mentioned it, I was able to go to the policy team meeting in the afternoon last week and mentioned it to Howard and and team and Erica are the leads of that team so where it would be great if we could try to get that PR done. I'm mostly talking to JJ because Howard's not available this time zone. If we can get that PR closed I get much feedback. Then that would be sweet to link to it's really formalizing what's been going on for a really long time. But it's nice to surface that I think and that group's been doing some great stuff. So, um, I think that I'm available. Oh no I'm not it's IIW next week. So I can help with the slide JJ in terms of iterating on the content but I can't present. I'll take it I'll take. I'll do the presentation I'll work with Sarah and Howard on getting the content on the slide. So I'll just take the air on that. Great. Thanks. Thanks for bringing it up Amy. Am I up JJ should I just go for it you're already on it. Um, so I've, um, I mentioned that the syncing up with the policy group. I also had, um, I've been catching in advance of IIW which is October 1st to 3rd if anybody's local. Um, I highly recommend that it's an unconference focused on identity that's been happening for 30 years. Like the OAuth standard came out of work at that group to, you know, get everybody to stop sharing names and passwords. And they're doing a lot of really interesting thing. The last couple have really a lot of people are focusing on self sovereign identity, which is pretty interesting to track. So catching up on reading I dove into verified credentials which are a new W relatively new W3C standard that is emerging and Howard actually chimed in on Twitter and asked if that would be relevant for the group. So, since I'm learning about it, I thought I would ask other people here we can talk about it later but just kind of want to put it out there that I'd be up for seeing if I could get somebody from that effort to present to the group of people thought it was relevant and interesting. And, and then my other update is also on the agenda which is I've been helping contribute to the open assessment and JJ's asked me to talk about it a bit today. Thanks. If you can post the IAW link on the It's in the agenda chat. Oh, perfect. Okay. So I just stuck it in an ounce. FYI. So, who's next Christian Kemper. No updates. Ray has no updates. TK has no updates. Roger has no updates. But if anyone wants to chime in or chat on anything that that's been mentioned, please feel free to. Otherwise, people that have not added their names that have anything else that they want to talk about these two. I do see. I do have a question. And I don't remember seeing it and any of the guides or docs that we have as the security work group is working on assessments and evaluations of projects and those documents are in draft but available to the group. What is the standard practice for using the information within those draft documents. It's a good question. Colin user one. It's Emily, by the way. I know. I have to say that. So what exactly are you is the question. So for when when security is doing a draft is doing an assessment of a particular project. And we have the draft and all of our recommendations and commentary and updates that we're posting through the get flow is publicly available. Anybody can really go in and see the PRs they can see all the comments they can see the dialogue that's going back and forth. But I don't know that we've officially documented. At least not that I can see what the expected use of those draft documents are or if there's a caveat on them or disclaimer that all the information contained within this PR is draft until officially posted and made available on X site. I didn't know if that was something we should be discussing concerned with take advantage of like how what is the expected use for security members or people outside of security when viewing the contents of draft. Unofficially published documentation. I think that's a really good question. Well, I think I actually think it's a it's a I think it would be important to have a caveat like, I've just been kind of like well it goes without saying that all this stuff is unverified until we all approve it. But if somebody dropped in from, you know, wherever and wasn't aware of it, it could be amplified in a way that would be undesirable or reading, right. And so we get access to interesting information about some of these projects that's not necessarily publicly available until we start interacting with them and actually writing it down and creating a ticket and submitting a PR on it. And like, how do we provide assurances to those organizations with our due diligence, ourselves, as well as outside of that somebody coming across it. Well, I think it might be worth looking at actually. I think we have. Let me look at if we look at the, let me see if I can get on zoom and share my screen. Okay, are people saying this this is that the security assessments for the repo. So, um, so we have this like this language of caveatting the whole thing, right, that it's at least the intent of this description was this is both to give you a path into thinking about the security of the project, not replace your own process for determining whether it's a fit for you. And so there's, you know, there's framing the assessment in general right that we had a lot of discussion early on that we didn't want these assessments to be approved. If we don't believe that they're binary. And that, you know, we've been careful to say like just because the project has work to do doesn't mean that's a negative thing it's in fact a positive outcome of the process and so forth and so on. So I think it might be good to, I don't know, reflect on this bit right to make sure that we're, you know, looking at this six months after we wrote it I'm not feeling like it really conveys that aspect of the caveat aspect. And then the other thing is, I don't know where we would put. I think it would be good to have something in here that at least says if people are reading all the words while things can draft. They're just an individual's opinion and should not be taken as truths or something like that, which is basically I think that the spirit of the team like if somebody asks a question. If the project reviewer doesn't have an answer, that doesn't mean that that is unanswered right. It doesn't mean that the person has raised an issue that's a real issue. It could fall out that all that thing was a misunderstanding. And that needs to be communicated somehow. I might like from a disclaimer caveat for people that are coming to the repo and coming across this information. I was thinking maybe a line or two in the read me and potentially expanding upon the code of conduct because we are a security focus special interest group that above and beyond the normal humane code of contact. So if you have caveatting in there, if you're a member of this group, the information that you're going to come across is always in draft and not be considered actionable or taken and running up the flagpole, for instance, but something to that effect. If that's what we want to do. I just don't know and I hadn't, I was thinking about it earlier today and went looking and didn't see anything beyond specifically what you had cited. I think you'd be up for doing at least an issue, if not a PR that sort of proposes something I think that'd be great to add to and I love the idea of having because we've, we've, you know, we we've gotten presentations before where people have said, okay, don't tweet about this yet. It's not published. And people are generally very respectful of that and I would love to see that reflected in our code of conduct right that there's like, you know, there's two sided thing like of course, if you know people don't engage with us positively we may publicly document our findings, but until it's officially reported it's not confirmed or whatever. There's like, this is a reporting rigor. I think we have a we all practice and newcomers should know that. I'm working on that. So that's what I just described it. So, if there isn't anyone else who's willing to check in, then we can dive into the opera. I think let me ask if there is anybody from anybody from any other working groups, you got cubes, you got the policy working group that anybody has attended that wants to give an update. I have a question regarding our partnership with other six. How does it look because there are some issues in for example signal that security issues that are hanging there for proposals that are hanging there for like years and is do we have any way to influence it somehow for example. Wait, so where I didn't miss her. For example, I found one interesting thing in a signal. There is a proposal for for a runtime change and it's there for like two years. I was wondering how what is our relationship with Kubernetes 6 if if we can influence something or change. I don't know. If you can point to the specific issue and talk about it, it will be good but the overall stance. So that is essentially cigar top rates on its own and influencing cigar. It's not the goal of this group. It's rather to help cigar then surfacing what the issue is to the rest of the rest of the Oregon community. That's that's the objective. But if you do want to bring it up into this group and talk about the specifics of it and why you think it should be prioritized and we can surface that to wider audience. I think when I always come back to our charter and mission right so our mission is to reduce the risk the cloud native applications expose and user data or allow other unauthorized access. So if there is an issue right in the world. Particularly in CNCF projects because we're part of the CNCF so if we if one of our projects is has a issue that is we think is risky to cloud native applications and the ecosystem. Then I think highlighting our concern like we have a forum here right and we have the ability to we could invite SIG off or a project to discuss issue that we consider to be risky and we can talk about why we consider it to be risky and what they know about. And I think that forum creates opportunity for action. Daniel if you know that issue and yeah I would create an issue in our report to bring that issue up and talk about that in this group. Perfect. Then I will prepare it. Thank you. Yeah and I think wherever possible we should like you know we can plan ahead and like you know invite people from the relevant projects or other SIGs to have a discussion. Yep. Yep. Yeah. Okay so if anyone else has anything to talk about if not then I'd like Sarah to give an update about the OPRA review learnings from that. And for the rest of us to chime in to see. I think it has an attempt. Yep. So I participated in this security assessment. For those of you who are might not be following the details here. We're on our second so all of the assessments are tagged under this assessment tag. I remove the is open. We have three assessments in total is completed. Open policy agent we are on the verge of completing and Falco we are on the verge of starting. So our goal is to have five assessments and then reflect on our process. Of course if anything is in our way we can update our process but we are you know maybe steps here we're doing our second assessment. And then we want to talk through our learnings but not you know deep dive too much in maybe we should do x, y or z we just capture those and so we also have another label for the assessment process. So don't check this. So you can see there's a lot of open issues, right, that are like, if you're participating in the assessments or observing them or you know hearing about the meetings and you're like, Wow, they should really do x, y, z. You can look at everything labeled assessment process and this is the time for us to be capturing what we're learning or ideas about how to improve the process and then we'll review all of these issues. So we'll go back to these first five assessments and then do some improvements. So that's kind of like the big picture of where we are. If we go back to the issue right so we're here going through this checklist of things and now we have the PR out for the assessment summary and we will Amy need to schedule a to see presentation shortly. Whenever there's an opening on the calendar and if we can wait until after cube con I would be delighted. So touch base with Liz I think Liz would like it to be not waiting that long. So let's not go into the details of scheduling here. Okay, happy. Let's just chat offline but just to let you know that we want to make sure that at least Justin and ash and one of the co chairs is at that presentation whenever we decide to queue it up. That is totally fine. So, so this is the assessment we opa. Some of you may recall we had a presentation by opa some time ago. And they presented their, you know how it works. It's, we have a background where basically policy is a big part of security right we have a breakout group that focuses on policy. In order to say that you have a secure system, you need to make sure that you actually have some policies and they're being followed opa is a project that helps with this by having a by making it so that you can write your policies in this rego language and then validating that like doing the policy enforcement and implementing those controls in ways that are like can be machine you can reason about with machines code. So that's kind of I kind of went through the summary but I'll go through this now a little bit in order. We have this maturity section which is kind of a, if we don't quite know how to define it. So with each assessment we make it up. But we have this idea that as context for how we think about the improvements that we'd like to see, we want to have some indication of how widely use this project is. It might not be the arbiters of success, but rather, echo that information because it, it affects how what recommendations we have if this is very early new technology that's experimental we might have different recommendations then this is used by almost every, service on the internet so here what we did was we collected a set of companies that are used by opa which you know sort of indicates that it's under like quite a bit of use and linked to their list of adopters and then also they're getting community participation from a wide range of adopters, although you know like the noting that they're primarily from styra, because there's been a bunch of conversations in the to see about wanting to support open source projects that are primarily one company to have participation from the community that that's robust, if that company decides to do other things right. Getting to the sustainability there's a little outside security but it's a port force affected, it affects security because we've seen a large, a significant attacks of late that are based on something becoming not maintained anymore and nobody paying attention right. And so, so that at least seems important to me. And I think I saw a question in the chat. No, thanks. So, so I kind of went a little bit over the design I think the key takeaway from our perspective is that if you have heterogeneous infrastructure or a high rate of change where lack of policy enforcement would create a big business risk. That's when the added overhead of implementing OPA would be valuable. So, this is a common situation right that people have on premise cloud or multiple clouds or different way or different services that all need to have similar or the same policies. And whenever you what we're seeing is in that heterogeneous infrastructure that presents risks because people can't reason about their policies or know that they've been implemented. So, and that's sort of common in this cloud way. And so, the, the, you know, the added benefit of OPA also presents risks right so it's great we have this policy as code expressions that you can, you know, you can implement the same across heterogeneous systems and separate your security code from your application application code. But then these are really like they it requires the same care as code and some, you know, and there's concern that the imp that there will be a false sense of security just because you're using OPA. So, we, you know, a lot of our discussions were really around how do we how do you think about this policy language and make sure that it's saying what you want it to say, and that people are understanding what they're expressing when they express policy in this language. Sorry, do we have a feel of who the target persona for OPA is in our security persona that we have. This looks like it would fall into the platform implement. Well, it's so I think we have this in here somewhere. This is their self assessment. We have the goal somewhere I thought we had the target user. Well, we might not. If we can write that in the notes. I want to just double check because I thought we had that somewhere, but it's a good question from what what I remember. I think the target user is the operator or the developer and that you could use OPA. I mean, like Netflix uses OPA and they're not a platform per se. I mean, I don't know, maybe they have APIs, but it's primarily to secure it. No, no, but the platform there could be people at Netflix that implement the Netflix platform for use of other Netflix engineers, right? Right. Offers the platform to outside users. That's true. That's sort of an interesting. Also, Christian, if you want to go chime in on the PR with comments about this, that would be super valuable to I mean, I'll do that if you don't. Yeah, yeah, sure. Oh, that'd be fabulous. Thanks. Yeah, yeah. For pointing that out. So, yeah, I wanted to have everybody in the use cases doc. We have these personas that are different users are operators, administrators, developers and users and platform implementers and the security assessments are supposed to focus who uses this stuff. And so we should make sure that we covered that. But I think that's interesting. There might be opportunity for looking at who's using OPA to find some of those platform implementers you've been looking for Christian. Yeah, because I know in the gatekeeper project, they have separated these personas. Gatekeeper is one of the OPA sub projects. Right. And then they have separated these personas and I was wondering what the official. Okay, thanks. Cool. So, um, so do you do, are people feel like they're familiar enough with OPA? Do you want me to talk through some of the self assessment to talk about what it does or should I go straight into the recommendations? How do you Anyone has any Or maybe we should start with any, any further questions on what OPA does. Yeah, anyone wants any basic info about OPA. That'll help them understand what the review is about. I would say no, because I think we've talked about OPA or why don't you take like, yeah, it'll be worth the 30 second intro to OPA before we Okay, so so generally it's for controlling access to a service. And with the copy up that I am not an OPA person I've never actually used this technology hands on so people feel free to correct me. The this separates the data coming in from the policy. And so general the data and the policy are combined into a intermediate document that's evaluated for a decision and one of the And so what you're writing this policy in a regular language in this regular language and then your data is expressed in something like JSON, and then they're evaluated and the decision can be yes, no, or I don't know, so that you can compose these policies together. So, and then the this OPA piece is generally deployed as a sidecar, but they have some libraries and different deployment models so you can, you know, you could bind it into your service or, you know, run it as a sidecar. Whatever mode you want to run it in. Okay, any questions or observations. Yeah, I do have a comment to a question possibly. With the OPA, because from our security, since we're the security working group, we are expanding the attack surface, meaning, you know the OPA itself will be opening up for some vulnerability in being attacked and the policies could be manipulated. And I was wondering if, have you seen anything specific as to what maybe the preventative measures that OPA is taking or recommending. So I think that with the addition of any part of your system right your, if you add anything you're expanding the attack surface, but then you have to think about like, is the issues you're mitigating. Okay, bigger right then what you're attacking what you're, what you're adding and so that's part of our record our analysis where you shouldn't be you probably shouldn't be using OPA. If you have very, very simple policy and a homogenous system right just because it would add more complexity than is merited was sort of like our analysis and to answer your question though, we have we went through this process of kind of articulating what things are risky, right, and that if OPA is successfully attacked right this is your point of policy access and that is, you know, pretty risky. And so we went through there's actually like a lot of sharp edges around, have you set up OPA correctly, and are you managing your policies effectively because OPA isn't a policy management system. So you have to outside of OPA, figure out how you're going to distribute your policy that's this gatekeeper project that Christian mentioned. So it's very, it is a piece of the puzzle, it is not the solution by itself. And I think that's the key thing that we want to surface so that people understand what they're getting when they adopt OPA.  Yeah, that's a good point. I kind of assumed that that would be the case. They probably decoupled. I wonder if OPA is also giving some recommendation as to for the implementation. So for example, if you have a centralized policy engine for a complex environment such as what you were leading to earlier, as far as where OPA could be implemented, multiple clouds, multiple, you know, bi or different type of environments that all trying to consolidate the policies so that we have a uniform and consistent policy rather. And when you do that, obviously, you're bringing that in a kind of a brain or some sort of a think area or somewhat centralized in some sense. And, and that's becoming even more sensitive to the operation of the whole enterprise and the users. And I wonder, does OPA going to the implementation part of it as far as how to any recommendation. So they actually have pretty comprehensive deployment docs that go through a couple of different models. And I mean, I think that this gatekeeper project is really about them saying, Okay, well this is a common, like, expanding beyond the, the agent right that evaluates policy to the other parts of the ecosystem, because there is a need there. But I think there's a really good questions. And it really like, I think that's kind of a good lead into our recommendations. Which, like I mentioned, focus mostly around like the potential for confusion. And the, there's sort of an assumption, I think, in this whole project, which, which maybe understates your point to K, which is that you like, you need to. You need to manage your policies really well and make sure that that doesn't become, you're not just moving your, your, you know, like where you're being attacked to someplace that is less secure. Right. Yeah. But what we're like, just on a personal note, there seems to be this sort of common pattern in a lot of the exploits I read about, which are that things are not configured. Things are people, things are not configured the way that people think they're going to, they are configured right that the systems become so complex people have so many VMs and services running that it's easy for something to not be secured at all. And that things end up being wide open on the internet unintentionally. And I think that one of the things that makes me interested in following opa is, you know, that's the thing that you're mitigating is the sort of, oh, oops, forgot to secure that right forgot to update this, you know, I have to update my policy in 15 places and it's different formats. And now I'm just, you know, it's too easy to make a human error that misses those so. So it might be good to Kate or somebody like, you know, like if in reading the overview, like do we address those points? Yeah, I think it is also called out on the review in terms of what scope of opa is and what it solves and what it doesn't. And there is some call out there but it'll be useful to chime in on the PR and add to that as well to say to be clear on an increase in attack surface versus scoping it down to thing and tooling to mitigate some of these. Once we chime in, for example, on this and are we going to consolidate all our issues at some point and feed it to the people. So all of the opa, this is this process has led to these writing up or highlighting opa issues. So many of these opa issues that are in the project recommendations came out of the review. And so it's the reviews really owned by there's a self self assessment where Ash is, he's a contributor to opa and he owns like getting that over the line. And then either here we report issues into opa so that this once this PR is in there are open issues tracking everything we raised. So if we chime in we're doing sort of two things one is producing this document which which is kind of like anybody's guide into understanding the security profile the risk profile, the benefit of this particular project, but also allowing us to track these open issues. And it's sort of chatting about like writing these issues such that if there's been talk that we want to re review these assess like assessments periodically like maybe annually that maybe if a particular project hasn't added any features in a year right that we could do a cursory review and just look at the issues and do a quick update right whereas a project has added a bunch of features related to security then maybe we would do a full assessment. And so we're trying to like sort of queue this up so it's easy to update. If that turns out to be about something that we know it's reflected in reality. Does that make sense. Yeah, yeah. No, I think that's that's good. So, at this point, the only code that has been contributed is very style. I know it's the contributors are mostly styro. So there's basically there's a chef. I just looked at these 77 contributors with ash. You know, kind of like look through the top contributors. And I don't remember which of these people it was but there was somebody in these top four that was from chef, which seemed to me to be a good sign. And then the there's somebody from Google who's pretty far down because it's mostly spec stuff here we go. This. Tristan has worked on mostly the rego to spec, which is also kind of like a sign that it's not just styro but it does have the vast majority of the contributions are styro so overall for the ecosystem, provided this, you know, continues to be adopted. I'd like to see more contributions from other companies but that's a, I think it's a process, but they're, they seem to be making good progress on getting wider contributions. Did we have any confrontations and during this assessment in a way that some findings were conflicting with the idea of hope or whatever. Well, I think confrontation would be too strong word. We had some good discussions around what as as as now, this being our second one has become the norm, which is like where are the edges of opa's responsibility, right, particularly around usability, and around defaults right so it's very challenging to make things secure by default, because the most secure thing by default is to just turn off access completely and that's not useful. So how do you make it less likely that somebody is going to do something incorrect, because they don't know what they're doing. And how, and so we talked about, you know, really I ended up having a brainstorm because the, like, ash came in with a stance which I think is, you know, sort of reasonable from their perspective, which is that. Well, you know, we're giving you a sharp knife, people need to do these different things. So we can't, and we don't know what your policies are so what, you know, there's not much to do there other than and, you know, initially all of the project recommendations were turned into documentation improvements. Right. So very quick time check. We have 10 more minutes. And a couple of things on this. If this discussion that you already had, if it can be captured and put it on to a GitHub or if you have already done it part of the assessment. So that is, I'm talking about to a here. Okay. So, so I'll just wrap this in the next few minutes. All I wanted to know bring up is like we only have 10 minutes if you had more things to cover then I would offline this and then cover otherwise if this is what you wanted to do then you're more than I'm more than happy to. Yes, so I just, this is the whole thing, which is that we shifted some of the thinking around, was it possible to make code changes that would make this easier to use. And so we've linked some of the ideas here if people have ideas in this realm. We can dive into the specific issues. These are really starting points that then the OPA team and anybody who wants to get involved can, you know, sort of add in ideas. And then I'll just round out by saying talking about the CNCF recommendations, similar to our last project in total. There are certain things that the project is not well positioned to do. And so if the CNCF wants to support this project more having a study of user practices around this. You know, whether that people are catching common patterns, and then also learning from the end user companies where there may be specific integrations that should be higher priority based on what people are doing with that would be more impactful than maybe other things that they might do that we that we don't have visibility into what the CNCF end user companies may. So that's That's us. Yeah, so was it Brandon who asked that question. I forgot I didn't catch the name. So this is first time brought up about the discussions that happened, or the controversy. Yes, Daniel. Daniel. Okay. This first time. But that might be also an interesting thing too. I don't know part of the reflection about, you know, what do we really learn. What was, you know, what were the things that were maybe unexpected by either the product team or us. I like that way of thinking about review of these security assessments. Yeah, those are good inputs. So we have seven more minutes. Any thoughts questions on the process itself the project. And again, I think the reason reason for bringing this up in this forum is the PR is open. And I'll be I'll be doing a review of that but anybody who wants to chime in and add any comments on that that needs to be considered and it is very helpful. If not, I'd like to give a couple of minutes to Michael do see who joined joined later in the meeting to do a check in. Sorry, JJ. That's yours, Michael trying to give an update. Yes, on the security day. Sorry, I missed the first part of what you said. This is, I just want to give you a couple of minutes to give a check in and an update. Emily Fox gave update on security day at least touched upon it. So if you want to give a little bit more detail on that. Yeah, sure. Sorry. So as Emily probably already said, the schedule is not there and published. We've had extremely good response from sponsorships, which has been extremely positive and that means that we're able to provide things like lunch for attendees. I, Amy, have we gotten an update on registrations. We have not. Okay, last I know it was around 81. And so now that we have a schedule out, we feel like we'll be able to push even more on registrations. I think we were thinking about 150 capping it somewhere around then. That's kind of the next thing we need to start figuring out is how much room we have and then what we can effectively do with the space that we have. So that's probably the next priority that we're going to be working on on our weekly calls. Outside of security. The FACO project we're coming up for our yearly review. And we're getting ready for that, which is going to be on October 15. I believe that's correct. Amy, is that right. Sorry coming up from you. Yes. So we're excited for that. We've made lots of good progress. We're shipping some very cool interesting features, one of which is we're actually probably merging some of this code in today. So we've done GRPC based outputs. So this is kind of been one of our sticking points is that a lot of our outputs and alerts have been kind of done in a more synchronous fashion. And with GRPC it allows us to kind of offload the alerting engine from the main FACO engine. And then we can have subscribers that are written in whatever language. And then those subscribers can then forward the events and alerts into whatever system like Elasticsearch or Kafka or whatever it might be. But having this kind of GRPC based streaming service is going to be really beneficial to the project. I was running some numbers just around like how we've been performing sandbox in the sandbox versus pre sandbox. And one of the interesting metrics I saw was the four sandbox, we had about 34 daily users in our Slack channel. After sandbox, we have about 104 daily active users in the Slack channel. And then a weekly active user perspective, it went from like 60 to 200. So the community is really thriving. We've got a lot of activity. We got a lot of stuff going on. So we're really excited about how we can kind of see the CNCF engine really helping us out and benefiting the product. Any other questions I can answer? So on that note, I think Krishna was scheduled to have a demo of Falco next week. So it would be worthwhile for you to coordinate with her to add any of these stats to the demo presentation. And we wanted to push it off till next week. We wanted to have the GRPC code kind of a little bit more finalized so we could actually show that off to the members of security. I think we are right about time. Anybody else has anything that they want to bring up? In the next two minutes. If not, it's a wrap. Thanks everyone. Bye.


So just to start a bit of context we're going to define what we are going to use for this talk and Just so you know the talk is available at my GitHub repository and everything is there demo examples and this very talk and its sources so In Python the package is usually a trio files Containing a special file the name of the root folder is the name of the package and you can import it under the right Conditions so you can temper with the import paths or something very well a lot of super interesting talk about different fixtures which Modified the way import machinery works in Python and well in some system in some context we actually do use those for legitimate reason for example in nix We use pth files and sites to make it easy to import any package in an isolated way Like virtual of neighbors you to do so So everything we install from any pipe pipe I like server Is a package and every package manager you've used is also something that's provides for package and use these indexes so Now thinking about people like us users We install packages manually we type in our terminal or by using graphical interfaces To install some packages that could be numpy that could be sci-fi that could be whatever you need and We do install those on a computer We're running a certain computer operating system and on this computer there are things those things are some source code as sash keys personal fire professional fire secrets and To us that they might be something that we don't care about that may be something that we forgot that we have But to over people that might become they might become valuable targets And that's really important to think about this because the computer that we carry on contains very important data and some people Try to enforce the fact that you have to encrypt your computer to prevent for data loss and catastrophic catastrophic failures and We can also Extend these kind of ideas to servers and deployments because they're kind of the same except if you replace the human interaction by automation So some valuable targets reachable at install time when you install a package our SSH kids SSR servers Bros or profiles and whatever you name it Because all of those are available in your user. So sometimes you can try to do things to isolate Where you're installing your packages, so you use docker or you use something like that But not everyone does that not every package manager provide for package is a package isolation. So Yeah Just a quick reminder how to publish a package to pipe PA We read down instructions to use our package.  So maybe we don't really care about The fact that we can run arbitrary code at install time But unfortunately we have some example of annoying things and really bad things done with the with with these pictures so for example date you see is very known a very known package to deal with dates and Some people some of you might remember the Python 2 to Python 3 transition Which was quite painful and some packages we which were not compatible with Python 3 started to create new packages compatible under the name Python 3 dash package and Of course someone published the Python 3 dash that you till package But that was not innocent package so on 2019 Someone published a malicious package behind this which was which were depending on the jellyfish package and playing her with a homo glyph attack playing on the I or L confusion and this payload Try to attack a very specific person Because if you unpack everything gets the code extra, which is on the issue by the way You can see that it was trying to test if someone had a very specific file Which had not a very random name or one was not on every computer So we can infer that it was a quite sophisticated attack towards one developer So it's quite frightening and makes you wonder who Frequent this attack are in fact and Some over talks talks about that There are a lot of malicious packages which uses typosquadding or even like I think Honest typosquadding Some package were definitely downloaded it too much for example DPP client And recently PI PI real rollout mandatory two-factor authentication for critical packages and you might have noticed this last week there were some drama about it because By PI did a great job. I think giving out Free two-factor authentication to critical package maintainers But some maintainer didn't want to enable two factor of the patients survey I agree with that.  And I think that's a physical question. So let's put it aside The question we ask is why Arbitrary code execution is needed most of the package we install are mostly declared declaring their their dependencies so we say that NumPy depends on some numerical library and so on and so forth and people or any package manager use this to perform some topological sort and version resolution But our our Our formats our the way how we describe our package does does not take it account the problem that Comes with system level dependencies So when you install something like beautiful soup which comes with HTML parser You need some XML parser and you want a fast XML parser So lxml is one good candidate and it requires to install say libraries So you have to coordinate with system package manager You have also the problem of when you're using numerical libraries such as side power 9 pi you can change the inner Acceleration to choose an acceleration which works better on your hardware So for Intel mkl you will get a better acceleration or inter Intel base processors Open blast is the generic implementation which works quite fine for every processors Which was read by the research and you have the same for AMD and so on and so forth same for TensorFlow and CUDA because you have some specific specifics about GPUs and You have a new purely Python packages also Which are read written in another language so says rest must be compiled and exposed to your Python package and When you people install such a package and you don't have any will for it you need to build it and Python is not going to install a rest compiler for you. So the author is going to install a rest compiler for you And you have many over example of why this is needed So we can argue that maybe that's the wrong model and should do something else extra but that the reality so you have to work with with this and This is possible because there is no real art in our teacher problem but I was some interesting talk about the update framework and and many over Attempts to try to solve this problem, but not all can account for everything. I just mentioned and This is still very a work in progress so one of the problem I think is is hard here is that not only you have to support like Linux distribution we which all have their way to do things Some are source-based you take our clinics gain to and so on for some are binary based they beyond open to extra and They don't necessarily agree and who is responsible to bring you to the final mile of dependency And also what something I don't say is about windows macOS and so on so forth And finally software just have to make assumptions There are no easy way to discover some standard standard sometimes so for example when you do a say library Or native library you can use something called PKG dash config to discover library paths and pass parameters to your compiler and that's requires you to provide a PC file and Sometimes upstream developers forget about them and Downstream develop that downstream maintenance have to add them manually, but they don't necessarily get them upstream So it creates a lot of fragmentation and difficulties to handle everything So downstream motors have to do work around spyton developers spyton package Packager have to do something in their setup dot-bytes so that the experience is streamlined and very user friendly and As a result you have a great success of using something like Docker and France because they provide a streamline experience in a clean state Which would be here a container? So we could decide that yeah, that's the state of thing and we might need to restart Packaging someday and and do it fine do it good and get some 10 years milestone Plan to get us there But this talk is about next So let's do a next crash course What is next so the idea behind next is a functional next language and It's a general package manager, which is able to work with over package managers The provoking idea of next is we break the file system your artist on there So you don't get any slash us or extra you only get your state file So slash home and and so on so forth everything is a sim link of something in the next store So that makes it really easy to reason about what you have in your system And everything is recorded and you have a cryptographic ash. We'll come back to the cryptographic ash later Packages are not really packages in next. We don't have any concept of packages we have concept of derivation, which is a more general concept and We express something in the next language and produce outputs So that can be a package that can be a system You need file that can be a bash script that can be a Python script everything goes So here's an example of five system your artist and are You can find those as being srv us or var that you often find on your system But on next those either don't exist or are sim link of a special next or path so just to get some Example of what is next here is the next language now So this is an expression to build a Python package So you have some import of the package the list of packages we have so that this is a function signature We have a function call we call to build Python package function.  We give version We give the source notice that we give the ash. That's something that's prevent In next network access is disabled except for Derivation which have ash so that we can already know the results in advance So if by PI PI try to change the contents the result would be rejected because the ash would change So this protects us a bit but not a lot And we have some metadata about the package So the the next path so we say that we have a next path here is the cryptographic ash A nice property of next is that this cryptography ash uniquely identify your your package Either things are input address that means that your package is Function of the inputs if you change them by and you depend on them by your package is going to change Or either content addressed that means that either if not by change Changes, but your result won't change because it's a minor API Bomb or something the ash won't change so that's really interesting because you never built software twice except for very good reasons and That makes it trivial to cash Derivations and provide them to everyone And then you get an ace not a nice name and Get now now you get something that you all used to which is the bat at the bin folder the leap folder extra extra So here's some glossary basic glossary of the next terms. I'm going to go fast on them The next door is the file system where everything is recorded including the derivations the store path is Either outputs produced by a derivation file or a different derivation file itself So you can send derivation to someone else and you can build it for you and you can get the outputs Derivation is the recipe. So when you say I want to build tools like we we've seen earlier You say that this is a bit a python package and all python package have a way to be built which is running the setup dot pi and and then we just give some Parameters that we need so the source the name the version as some matter like some metadata And what is interesting about it is that even a full nix-west Linux system is just one big derivation Which is your kiln or your in-intererity? Yours boot script extra extra and you can have a tree which I will showcase later of your world system And then the mathematical concept of closure, which is that's when you have a binary relation such as software X dependent software Y Often in the packaging ecosystem you want to get the closure of this so we you look at the graph and you say Okay, I'm going to get all the packages I need which depends all on themselves so that I can have the full software package and Working on my computer. So in nix we often talk about closure of a software And that means the transitive closure behind the binary relation of dependencies So now maybe I lost some people of the audience and maybe a question I would I would ask is how many people are using nix here? Yeah Well, that's cool for all the newcomers here So maybe it's a bit obscure so far and it's very normal because that's how nix is perceived by a lot of people and One try one thing it tries to do is to use theoretical framework to get the complexity of software tamed So as a goal we nix as hallway as hallways have had build reproducibility So what I explain about input address or content address so once you build a software once you can retry to rebuild it And you will get the same very package Something later on that would be funny And build operation or sandbox by default so net no network access no arbitrary access to the user-prize system or even installed Dependencies I cannot try to see if you have like I don't know numpy installed if I don't need them by so That's pretty powerful because we help us to see what the software is actually doing in the sandbox and You still have escape hatches if you take some software like steam Well steam is a very bad behaving software because it brings its own libraries because it's trying to solve the problem of libraries So some games are not statically linked They come with a lot of libraries and nix doesn't take the library of overs So we have to make it pretend live in a file system hierarchy standard using namespaces and Yeah, everyone is able to use team on next s even if it looks like quite hard Composition with local package manager is interesting and important nix is not about reinventing the wheel It's not about like taking over peep or poetry is trying to work with peep and poetry So that it can handles the thing that the distribution the Linux distribution distribution has to handle while peep and poetry Focuses on what it has to solve And trivial caching as we have reproducibility Nix provides a very big binary cache Which has everything you contain so even though nix-west is a source-based distribution you don't recompile that much I Think the only time I recompile is because I'm changing something in the c-compiler and I'm recompiling the whole world So I think that's this this chart for repology.org shows something very interesting that Nix has a lot of package projects and non-unique package projects and Of course, it's because we are automating a lot of packaging and we are like eating packages from by PI and so on and so forth, but we're doing that with a lot less Maintainers than something like the art Linux user repository And I think that's very interesting to to to see that because We have a lot of data on what is packaging what is a package and how people to use this package and what does this package does We have a minimal image. We have came very far on reproducibility on that and We have Only two path which are not reproducible. So this is the the website tracking nix-west reproducibility So it's trying every path That is a derivation in the minimal image and trying to see if it can rebuild it twice on a different kernel Different hardware different machine different time And see if there are any differences. So For pie so as you can see Python and rest something are not reproducible Sometimes we're at 100% and sometimes we're not so it's it's continuously tests and as you can see we can look very deep into the problem and see that some Pi C file is changing when you were rebuilding twice. So maybe non-deterministic optimization or timestamp related issue and That's pretty cool to be able to do that So the question now is how we can leverage nix as a tool So has nix is very strict and the word outside is quite forgiving We have to not reimplement the wheel. We have to reuse existing tool like peep on poetry but we have to left nix handle the downloading the native dependency sourcing and capture something in an expression and We put peep and poetry and every tooling that we use like this in an offline mode That means that we instruct peep to say that we're going to do an offline install and should rely on not using network access So for now, I'm going to do use a very simple demo of that So here's a nix common Where I'm going to to create a shell so nix shell is very interesting because it's you're all used to virtual on Well nix shell is virtual of generalized to everything. So you can use any package any binary temporarily in some sort of virtual and once you exit the shell Those package are not in your environment anymore. So what I'm going to do is to have a python 3.9 environment with some known to be annoying package to install sometimes on some systems and Show that it's quite easy and what happened when you do that Of course so you can see that I made a rookie mistake, which is Assuming that bash was available in my path. It is not I have to use on which is the post extender for that and You can see that next is going to compute the closure of the packages I asked so I have some nice information about the download size the impact size and Get everything that I need So it's doing the work and downloading and now I'm creating the environment. So I'm in the nix shell right now So I do have numpy. I do have scipy. I do have tensorflow and pytorch Once I exit this This shell you can see that I don't have those package in my environment So it's indeed working and I can perform some advincet stuff on this thing because I can change The inner the inherent numerical library. So I can do Intel and Cal based numpy or scipy And it's quite easy this package This script can be cached so that I can share it with others and they can have the instant shell and not download Not build a lot of things So it's quite cool and continuous integration becomes a lot easier to test with this kind of fixtures and the second demo is about a self-contained script sometimes we build some scripts one line script one file script, sorry and We don't use External dependency because it's annoying to package them Or to package a one file Script with back with dependencies. So here's an example we request So I just want to do request get European.IU I don't have request in my environment. I don't want to install request I would like to share this file with friends that might use this package Might use this script rather and you can see that my Code tool is not finding request but I can still run it and get an answer because Nickshake has been used as an interpreter for shebungs, which is a feature of the shells and This way you can do self-contained Python script trivially. So We've seen how nicks can be leveraged as a tool for doing Python development We didn't showcase how to do Python deployment, but the idea is the same you build some closer You send it to the remote server and the remote server just use it You can do a lot of advanced stuff meanwhile You can check that the closer is coherent with something you can see in the closer you can generate a so for a bit of materials and a lot of stuff, but the problem with nicks is that it's learning curve is quite insane so not a lot of people are that much gray bird but some tries and Nick's is like very frustrating for new users used to download binaries from the internet and trying to run them We have solutions for that, but it's still not easy and Nick's cannot solve everything so Nick's cannot solve things that were not build for Nick's Meaning that we have a system to determine which packages is hit by CVU, but Have you ever seen a CVU informing you about the very function who is Affected in the security of the library So you cannot like Say that this is the this particular function which is affected This is the ash of this function. This is the ash of the code. So we cannot automate easily Whether we are affecting or not by a CVE That's something which is a work in progress Severe are very granular sometimes they say to you that in your dependency you have a package Who is renewable? But in fact, you're never using the package. So you're not affected and Nick's cannot solve that Tooling that takes log files and produce next expression are not enough. You have a lot of each case is including in the Python Registry sometimes some URLs are not stable And you have to try to predict things and try to find out about hashes and so on and so forth So it's a bit difficult today varies a bug on poetry which Breaks a lot of things and I just got it by it and I wanted to do a third demo, but it doesn't work Limits of the standards are also Complicated to work around You should take a crate like cryptography Which is a Python package for doing crypto We have to do this kind of work around to support it because it brings a rest compiler in the in the play and unfortunately The files doesn't bring any ashes about the the ash of the dependencies of the the rest Project so we have downstream to support that and try to work around behind the fact that if you're in there A certain version we should use the wheel if not we can compile it And here are the dependencies hashed and we can use it extra extra and it's very very complicated So we have been working with cryptography guys on that, but the format doesn't support it. So it's hard And advanced attackers we just moved to bug doors and now it gets very complicated. It's not it's not a technical problem So we need in the future more foundational work to be compatible with Python We disable bytecode optimization because of errors are not determinism We did a we're not using a lot wheels because It's super hard to get right the many Linux ABI problems But at the same time we have things like trust X which solve distributed trust issue when you have multiple binary cache How do you trust this binary cache? What is your policy? Do you want to do a majority vote? You ask every cache. What is the cryptography hash and you say that? If the majority vote in favor of ash and something is not doing it Then you reject this cache. So it's quite interesting. We have the salsa framework Which is a framework pushed by some big carp tech Companies to get high level of assurances in your supply chain in integration with the ecosystem is are being published at the moment so the conclusion is Installing packages in bite is dangerous by construction fixing this is complicated Attacks are running in the wild and we can only measure the public surface raising the cost for attackers quite easy by constraining the attack surface with nix at least and The real win are in fact that we have a lot of data produced by nix We have the integration with the external ecosystem doing multi language project becomes easier with nix and That's pretty much it. I will left you with some references and do some QA if I have time for it


Hi, everyone. My name is Thomas Stantec from NetherMind. I started this project one year ago. Piltrack is with me here. It's our developer, the giant, a few weeks back. So with NetherMind, we build a .NET implementation, a full Ethereum protocol, which means the FP2P. It means Ethereum virtual machine, all the blog verification, synchronization, private chains, main net drops, and so on. Yeah, so we already say we have the first version to download from our downloads page, and we encourage you to go there and see our packages for Windows, Linux, MyCoS, let's say, cross-platform release. We continue to add all the missing functionalities. We keep building on top of what we already have. We add functionalities that will be needed for the open-source development on top of it. So obviously, you have Gats, you have Parity, and this is as much as we go, so we have the full implementation. And if for any reason valid or not, you or your clients want to go with .NET Core Solutions, and this is technology that you feel more confident with than definitely would like to go and try to experiment with NetherMind. So when we were applying for a grant that was, I think, late June, we were still working with Robson Testnet. Now we have multiple rounds when we did the full sync, full archive note on the mainnet. And this goes very well. We have good performance stability. And obviously, this is all open source for you. We also build the tools that will be targeting in the financial industry mainly, but also the cybersecurity companies or the fraud detection for extracting the data from the note in real time. So we analyze the pending transactions. We analyze the transactions before they even fully executed on the block. So we stream the data while the contracts are changing. We stream the data about whatever our customers are interested in, whether it is our exception, storage changes, whether these are some specifics of the network parameters. So the GitHub page that we encourage you to visit, we slowly build a community around the project. We see more and more contributors. People add more test networks, support. So literally last few days, we had contributions for the Covan connectivity. Sorry, we could be, Covan is coming next, I think, and then we should have the upcoming. So I think the girly is coming in the future. Ethereum 2.0, so we're following the research and we believe strongly that we'll have the fire party deliver also the beacon chains. We'll be there with all the other implementation in March. We have the same confidence and we're confident that we'll deliver the main nets and so now we can go forward and go forward at the edge of the progress of what's happening in Ethereum ecosystem. So that's what I was mentioning, the real-time data streaming, but also since we have this client full implementation and the ability to meddle with the internals of it, of the protocols, we plan to build all the integrated tools on top of it for trading, for pricing, for better modeling. And we hope that will lead to improvements of the market's maturity, of the ability of people to find the intrinsic value of Ethereum based on gas, based on the network characteristics and so on and so on. I want to deliver that to our customers. Thank you so much. I'll show you in a moment the running note. However, as you will see, this is as much as we can go with the UI in our world. And even here you can complain that the color of the logs is maybe not the perfect. So this is the live demo in a way that on the right side, you'll see the public mainnet synchronizing. It started a synchronization just a few hours ago, I think. And this is probably not the fastest possible because of the network parameters here. On the left side, not a perfect screen, but what we have here is the, our private network that we just spawned for some gaming solution on the private network. So this is something that we show that can run with Truffle when you just run Truffle test. Obviously it's something that we would normally run against Gnash or something else, but obviously will support all the JSON RPC calls in the latest releases. This is not what you can download yet, but as soon in our next release, you should have all the JSON RPC support, more test nets, better performance over time and more and more functionality. And this is mainly what I wanted to say, even as I have some time remaining, five minutes was more than enough to show you what we have now. And thank you so much. And we invite you to talk to us more about the solution, tested experiment with download, come and join us on GitHub and so on. Thank you so much.