Today, I'm with Padma Gordon and she's got a book coming out That she's been working on for a while and I wanted her to share about it I've worked with Padma in her business for a while and she is a counselor and a mindful embodiment teacher. So she works with individuals and couples Well has been in person but has done a lot online as well and she also leads an online women's group Among and also teaches them some online courses. So Padma, welcome Thank you so much for having me George. It's so nice to be with you. Yeah So I'm excited about your book coming out You actually did a crowdfunding campaign to the book and you already reached the goal for the for the crowdfunding even during these times Which is remarkable which shows you that you have a community that's supporting you and that's looking forward to the book So maybe you could tell us a bit about the book first Sure, and I just want to say that I just I feel incredibly grateful and humbled by the the outpouring of generosity not only for my book crowdsourcing campaign, but for you know, what's happening in the world It's just it's very it touches my heart. It really does So so the book. Yeah, the book I feel like it's it's a very timely moment for this book because it's a book about relationship and it's called being together it's going to come out this summer in July and It's really, you know, the thing about relationships is relationship begins with You tending to yourself the first relationship the primary relationship is to yourself and then how you are with yourself How I am with myself is also how I am with you and how you are with me So the book starts there and then it's it's really though a book about how to have a thriving monogamous long-term relationship Yeah, and that's been difficult for decades But especially now when yeah when people are needing to spend a lot more time together it's even more so Yeah, tell us a bit about so this is really helpful that the framing is and you know We start with ourselves and then that extends out to those we're with So what does that mean relationship with ourself? How do we cultivate that? It's a good question well what I do is I Spend time in nature and I practice meditation You might practice yoga or whichever kind of internal discipline Qigong You know, whatever is whatever feeds you So just really taking some time to be with yourself and tend to yourself and it's really about loving yourself So it starts with self-love Because we can love others to the degree that we love ourselves So doing practices and and also just tracking into your inner dialogue Right, so if you're doing something cheering yourself on if you're being a certain way If you're being generous if you're helping out say in this moment the elders in your neighborhood Then really appreciate yourself for that So gratitude is a great thing and gratitude is one of the chapters in the book Gratitude appreciation and that really starts with ourselves and extends outwardly It's beautiful so The book tell us about it's kind of the the Genesis the book like what what inspired you to I mean You've actually been writing for years. So you have more than a material for probably several books But yeah, tell us more about kind of what inspired this Well what inspired this was is the fact that I've been in a relationship with my partner now It'll be two years in June and It's been one of these relationships that has really It's been Wonderful and beautiful and challenged me and made me, you know turn and look in the mirror and look at myself and And we've been really doing the work together the work of relationship the play of relationship the dance of relationship And I thought well if this is so challenging for me and I'm someone who has worked on myself and is you know Really committed to evolution internal evolution personal evolution Then wow this this is God of this is an edge And we know it is as you brought up at the beginning George, you know It's such an edge and we have so few examples of thriving by thriving I mean where people are happy it doesn't mean they're in bliss all the time or they're just totally In love and la la land. No, it's not that it's wow we are a team we love each other and We are we are evolving each other together and giving ourselves space to evolve so Relationship as evolution is really one of the primary themes of the book so so that's really what started me on the journey of Writing about this and this is all brand new writing and all the writing I have done and all the reflection I have done has certainly gone into this because the book then Speaks to my own experience my own narrative is sort of one piece of the book and then I have drawn from the work of longtime experts who have studied relationship for 40 years John Gottman and other people who are really, you know Have just been in it for a long time So I'm drawing from research and then I did personal interviews did primary interviews and interviewed people who from my perspective Seem happy and people who've really Stayed the course because that's one of the pieces is you will thrive if you stay the course Which means you're staying in integrity with yourself as you move forward because there are plenty of us who stay in Relationships for a long time that actually aren't working. So to me, that's not thriving Yeah, well so say more about that what does that mean by stay the course and being an integrity with ourselves so You know in the course of every relationship, there are challenges And there are incompatibilities that come up You know, of course in the beginning of relation if there's there's honeymoon period and Everything seems to be, you know, oh, it's gonna last forever and And then of course, you know, the little compatibility start to emerge and we start as we get to know the person better I've got to know ourselves better and and what we really want and what they really want and So how do we then yes, this it's a wonderful idea this disbalance between staying the course and being an integrity or or both that are Being sort of working together. So tell us more about that Thank you. And George, I can really Know that you're speaking from your own experience. Yes, I know how long you've been in your gosh, it's been 12 years Yeah, yeah, and we've certainly gone had some big moments, you know, but we are we're here. We're happy. So Wonderful. Congratulations. It's It is it's going through those those big moments to me are the eye of the needle moments, you know Will we get through this or will we just part ways? Yes, we say do much can't do it And it's like the too much can't do it is can I can I really listen to myself? Can I do my own inner work? Because that's what it's about. We're in we jump into relationship You meant you you sketched it out really well in the beginning. We're just in love, right? And part of that is is You know the chemicals neurochemicals that are flooding our brain, right? We're we're just high on this love cocktail and it's great and it's bliss and Yeah, so I speak to that a little bit too the neurobiology of love Plays in a little bit. I just touch upon it, which of course is its own book so Yeah, so we we we get into relationship and then things come up and we have to keep coming home to ourselves Which is why these practices and the connection that we have cultivated and are continually Cultivating with ourselves is so important because What does it mean only only you know and only I know inside myself? What does it mean to be in personal integrity? Because in the realm of relationship We can it's easy to slip into self-betrayal because it plays into our our earliest wounds, right? Our early wounding and this speaks to our what's called attachment material in psychology Which I also you can't write a book about relationship without mentioning or delving into attachment to a certain degree So it's really it's a chance relationship is a chance to evolve as an individual and Together and the truth is that we are stronger together It's it's actually easier in a certain way to be alone, right because we we see you know all these We see or we hear these stories of great meditators great monks who have been in the cave and have been on the mountaintop and If you watch my promotional video it speaks to this And then you come home you come down off the mountaintop and you get into relationship and then Here's your stuff in your face right away. So it's a chance it's an amazing container to both evolve and To heal to heal those wounds that really in my experience can only be healed in Relationship either in an intimate partnership or in a in a counseling Relationship a safe container because that's part of it like we can thrive because we are held in a safe container Safe container of what sometimes called a two-person system. We're in it together, right? I've got your back You've got my back and then we can take more risks We can step further out into the world and offer ourselves because we know that someone's got her back so it's It's it's a big thing and also just one more thing I'll say about that is that You know people tend what I've learned and what in my experience and from my studies my research over these last Years is that people like being around happy couples, right? Because it's stitches back to if you had a Childhood and hopefully some people did where there were two parents and it was it was solid It's like here. We are we're held the king and the queen are in their place and oh, I could relax because all as well The king and queen the mother and the father or the two mothers and where the two fathers because this just goes for Two people holding the place Holding the place and it creates safety. So you might just explore that. Oh, how do I feel when I'm around a happy couple? So well, that's there's a lot there and a lot to reflect on and kind of I'm seeing you know these archetypes right and this sort of very deep Sort of patterns of relationship that go back in our psyche thousands of years And and have evolved us as human beings and this is why we thrive in certain ways And of course with the modern times like you said It can seem easier to be single Like you said, you know the the monk or the the spiritual person out there doing the thing But also these days, you know the digital nomad right like the on Instagram You see people who seem like they're you know living a lifestyle of freedom and they're very like every every two weeks They're somewhere else and you know taking pictures and like wow, you know if I were single then I could do those things right now now, of course with the lockdown can't do that Which is ironic, right? It's just part of the lesson here. It's like wait but but but you know, it's like a lot of the That kind of that kind of glorification of single hood Sometimes you know doesn't doesn't acknowledge the fact that we still are human beings that evolved to be with other human beings And and to and to benefit from well the human touch with somebody that that we can be you know in lockdown together but But so so actually I'd love for you to because you work with people who are both couples and you work with single people The book is probably better for for couples. I imagine of the people who are in a relationship But you know, maybe maybe not so tell us about that Yeah, that's a great point. Um, you know, I really I wrote this book for people who Actually probably are either young say you're in your 20s or your 30s and you're just the beginning of your relationship journey and you want some guidance because Guidance is a little sparse up there in this arena, even though there are many many wonderful books on relationship but I guess examples a little sparse real-life real-time examples and Then I know at least in the community that I'm a part of here. There are many single people in the 40s and their 50s and So I wrote this book for you too because it's possible. So part of what I Posit in the book is that relationship Having a healthy long-term thriving relationship is possible for everyone as long as you're willing to show up And the show up part is a pretty big deal The showing up means that you're willing to look at your stuff You're willing to stop yourself in your tracks When you're getting triggered and I'm speaking from my own experience and I feel like especially in this moment in time I my experience is that we are being asked to clean up our acts and to really interrupt our habit patterns and Just Really take a close look at who we are what we want and where we've been Because everything is everything's on hold External things are on hold So I feel like in relationship. These are the places and I'll say for myself that I have really I feel like I've become a much I'm much more loving person and a much more Gentle person and it's really interesting because my partner is extremely gentle and I'm gentle I'm also pretty intense and I have had to or have chosen to and I'm choosing to Really study him as part of being in relationship. I'm studying you. You're studying me How can I relate to you? It's actually a very it's an act of generosity. It's an act of compassion it's an act of kindness to actually relate to somebody in the way that is Will serve them and So that actually your communication reaches them because that can create connection it creates harmony and If I'm just standing my ground and digging in my heels and wanting to be right then that's not happening and then we're that's creating separation and What I want and then what my sense is that we as human beings want is we want to experience love We want to live with open hearts We want to love and be loved and and it's scary. I get that it's really scary and one of the chapters is on vulnerability and Because it's vulnerable to be in relationship to really dive into love and to To show yourself to another person and yet it's it's profound It's intimate and again, it's healing and it evolves us So I really wrote this book for For everyone and and as much probably as much or more for single people Than for couples, but perhaps if you're a couple that's really struggling This will definitely serve you and if you're a couple that's really happy. You can just say great We're doing all right Or or a couple that's content and you know, this book may help them to deepen And thrive more because what you're bringing forward is a lot of the reflection that In relationship, we sometimes don't have time to do, you know with busy lives with just You know, sometimes relationships become like well like like teams like you said I get we're a team But then that sometimes the practicalities can take over a lot of day-to-day, you know It's like oh, yeah, who's who's cooking who's who's cleaning who's you know Gonna do this Aaron who's gonna take care of this or whatever and it's like, okay this is a chance to deepen and to Do the work of personal growth together and so I I think that this book is would be beneficial I'm looking forward to it myself so Did you have something like a snippet of the book you wanted to share with us? I know it's You know, I don't know how much you're able to share at this point I can I can read a little bit from the introduction Yeah, just give us a give us a flavor of it Feel a little snippet and I just wanted to say that I'm I'm super excited about this book and I'm excited to dive into revisions and Gotten really great feedback from the various editors that work for this publisher. So that's thrilling Let's see. I'm just gonna go over to this So, let's see Begins like this Long-term relationships are an epic challenge for many of us They call upon us to be real and to reveal our Magnificence as well as our shadows to another person rendering us vulnerable Relationships invite us to be present and discerning Ultimately, they can serve to evolve us into the best most loving version of ourselves Over time the possibility is for a committed relationship to become a spiritual practice One in which you awaken to your true nature If you stay the course and don't jump ship when you are tested and Things get uncomfortable Being in a long-term relationship provides a container for you to evolve into the person you were meant to be This is also true for Jen and Jay where our story begins Jen and Jay have been married for 48 years and have three adult sons throughout their Relationship Jen and Jay have had many ups and downs and even a brief period of separation What has kept them together? There are lots of things and among the most important are acceptance of one another Being able to laugh at themselves and each other's behavior Wanting what the other wants for themselves and the practice of gratitude Gratitude supports the flow of love which sustains their relationship Just read a tiny bit more Gratitude plays a key role for Jen and Jay in their small day-to-day interactions in their relationship Jen loves to cook so Jay does the dishes Jen told me about how she has consistently asked Jay to use hot soapy water and wears glasses while he washes his dinner dishes So that you can see what you're doing Echoes Jen's lilting voice across the kitchen Jay does not comply with her wishes, but Jen doesn't become angry with him She simply smiles and without saying a word she rewashes them to her satisfaction Jen's thought is I'm so grateful that he took the time to wash the dishes in the first place so It's just a little bit of The introduction Thank you, and it's great that you incorporate You know real stories and so as research as well as your just your beautiful language in there So looking forward to this. Well, of course, we'll have the links in the description of the video to where people can Follow you and find out and sign up for your newsletter so they can find out when the book is out Anything else you want to share before we complete this conversation? You know, I just I just really first of all, I thank you so much George. It's wonderful to be with you. I really appreciate who you are and Really acknowledge you for doing the work that you do on yourself and in your relationship so I'm just name that again and I want to say that This is a great time everyone to Do your inner work. I feel like that is what is being asked of us is to do the deep inner work The world that the decks have been cleared Essentially to do the work that is the most essential work and not even work the most essential The deepest need because it doesn't have to be work. It can also be it's the play of relationship it's the it's the dance of relationship and This is a tricky time. I know for people who are alone Just know that that you're held you're held in love and that we are we're all in this together and And I encourage you to keep your heart open and share what you have to share Because because it's needed I Look forward to hearing from you. Please reach out to me Love to be connected with you and I just I will say I'm offering free meditations on Thursdays at five o'clock Pacific time where we breathe and move and meditate together and Yeah, I've been reading from my book on Facebook live. So if you go to my Facebook page, I'm a Gordon You'll see recordings of things I've read from the book Great, and we'll be sure to put the links in the notes of the video. So check that out Thank you so much Padma for your work and congratulations on the book launch and yeah, we're all looking forward to it Wonderful. So good to be with you George. Thanks. Okay


All right, so I think it's about get to the time and let's get started. Okay, so I'll conduct this talk in English since we have some foreign guests here. My name is Qian and today I'm going to talk to you about how we use large language models to help us to simplify our cluster management. A little introduction about me. I started my career at Google as an SRE and in 2019 I moved to Ant Group to become a tech lead and manager for infrastructure SRE team. Basically I focused on say Kubernetes and other cloud native like solutions in Ant Group. As an SRE we wear different hats during our day-to-day life such as say we write automation programs to say to optimize our like toils and operations, right? And sometimes we do like fire fighting to handle production issues. And in the past few years we've seen a huge growth in our like cluster size as well as the pod size. But still we remain very fixed SRE headcount. So that means we need to seek any every help we have to help us to facilitate our day-to-day life. And this year I believe like language model is a hot topic. So we are trying to tackle this issue with language model. Okay, so before we get into the details I want to quickly clarify that this talk is not about like a GPT-101 course. I'm not gonna touch any pre-training steps or any other like algorithms for supervised fine-tuning like PFTs. And I know land-train is kind of popular these days. So, but this is not a land-train tutorial and this project has nothing to do with Kubernetes GPT project on GitHub. You know, in SRE what land? We handle a lot about failures and incident stories. So please think about this as a post-modern review or a progress report on our exploration to language models instead of any like best practice. There's no best practice at this point of time I think. Okay. I will conduct this talk in the following four sections. I will quickly go through our motivations about why we are gonna use language models and I will then do a quick definition on our goals and requirements and comes to our fun part about how we conduct different experiments or attempts to solve the issue. And hopefully you guys can all take some, like get some idea and get some takeaways. All right, so remember the topic today is about cluster management and here's our motivation.  And your big was it was watching behind you and asking you for the reason for the mitigation steps and et cetera. So you are desperate to find out, okay, so what are the correlations between these notes, right? Are they all from a specific CPU model or if they are on a specific conversion or a particular type of application was running on top of them. And you have a million assumptions here and you would like to quickly verify them. But I highly doubt that on the high, such high pressure and under such like emergency scenarios you can type those 2.0 command correctly in a single pass. And so that's why that's the first motivation. And it took us about an hour to actually figure in this particular case, it was from a demon set change that actually has some performance issue on a certain typical CPU model, okay. So in the post-modern review, one action item we agreed was to how can we shorten the time to pinpoint or to locate a problem, right. At that point of time, actually multiple tech leaders joined the war room and they contribute very thoughtful and very valuable information and their assumptions. But they are not from the Kubernetes team so they cannot actually get into the touch with the cluster management but their ideas were actually valid. So we need a solution to help them to let them can do some quick self-service like mitigation or their verification by themselves. And also like during the mitigation stage on this incident, we have to ask users to do some self-service operations like to scale up their deployment or to do some node reboot and some traffic drains. So these actions were actually not that frequently used in their day-to-day life, right. So they cannot remember all those commands correctly or like can quickly pull out all those knowledge they need. So we categorize this type of issue as the operational efficiency in the entire picture of the cluster management world. As you can see, it may just be a tip of the iceberg and we are facing many different challenges. I think each topic listed out here actually was a separate discussion but let's be more concentrated today. So let's go to the definition part. So remember, I think for us we conclude that what we need is actually a solution to help us can quickly transfer like human intention or human language to like actually machine API calls or whatever can interact with the machine directly, right. In the operational world, everything is just like a command to a specific system or like sending an API call or just tuning some like kernel like parameters, right. All those data are actually structured data. But human intentions, we have a million or a thousand way to like express them. So what can we feel in this question mark here? So before the language model like becomes a hot topic I think like we write all those automation programs we write all the like past web front end, all the solutions.  So in this scenario, I think we are actually writing a user interface and that user interface actually does the translation between the unstructured data like between our intention to what the machine or the cluster management actually need the APIs, right. And since language models naturally takes the human language as an input. So we think, okay, maybe let's just try that. Let's use a language model for to be the new user interface. But using that in the SRE land is kind of like dangerous. As you can see, like we categorize the issue into, to use that we have to be very careful about the input and the output from the language generated by the language model, right. I think we listed out the four different criteria we need to verify if a model is valid and can be used in production. And first is about accuracy. The accuracy here is actually, we have a very high standard. I think it's about above 99% accuracy. Remember like you are actually operating with the cluster, right. If you want to reboot a single note A, but the language model generated answer to reboot a note B, it will be totally unacceptable, right. And also like the latency, I think it's we actually all want this thing to happen really fast, right. Otherwise you may miss the window to actually mitigate the issue. I know like if we experienced some like online, like chat DVD or whatever, all those chat models, sometimes the latency can be a little high, which is not that acceptable in our scenario. And also like for the security concern, we are not going to share our internal data to like any public models, right. You are not going to handle in your like internal security. They will be very vulnerable, right. So we have to like use our own model and at least we need to host it by ourselves. And for all those internal APIs and for all those like our internal operations, they are like involving really fast and language model has to like keep up with the pace of the API involvement. So that's the fourth criteria. Okay, so enough for the abstraction on this type of problem and now comes the fun part. So next I will share some experiments we did to play with the language model. So the first, the first attempt is about to do like API invocation. By the time we had this idea, I think at that point on land chain and auto GPT was attracting everyone's attention. And so everyone was talking about how can we make an agent to learn to use some tools to help us to solve a particular like domain, issues in a domain, right. So let's make an agent that can solve the issue for Kubernetes. So we quickly do the following diagram to express the workflow of, so when SRE speaks some language to express their intention, it can go through a planner model which is a language model that can help you do to plan the actions. And the actions can be executed by different executors and each executor is corresponding to a deterministic API call or a function that can interact with the machines directly, okay. So, so far so good, okay.  We, at that point of time, we already deployed a chat bot. We call it the ops bot in our like, in our production to help our on caller to try to solve their on call issues. So what we had at the point of time was like, if you say there was an alert fired and you got notified and you would like to do a simple restart and you just type in the chat bot, say okay, so restart dash dash component API server dash dash cluster full so that the support will do the restart for you. But at that point of time we had already deployed this bot in production for at least a year, but it was not frequently used and people were, people don't like it because they cannot remember all those commands correctly and they cannot type those commands like in a single pass. So here comes the language model, right. You would like to naturally think, okay, so if I can say just say help me do a simple restart and the model can understand and extract the named entity, let's say the component entity called the API server and the cluster entity and then filling the API call and just make the call. Okay, so I think the workflow is kind of very straightforward but we quickly hit our like bottleneck which is we cannot use the open models because open models does not have the internal knowledge from our perspective, right. So you don't know how we, in end group, how we do like a model with machine restart or how we form our APIs, right. We have to teach them. But for security concerns we cannot actually use the OpenAI or the ChatchBT. And luckily at that point of time we already started our like internal language model development, so we already had some like internal models to do like supervised fine tuning on top of them to see the result. So the question next is to generate all those training data you need, right. As you can see, the training data was about like a bunch of question and answer pairs where I can write some templates in a prompt and say, okay, so restart a node or I have different ways to express this single action, right, and you have that very fixed action format to fill in the model. And you then just replace the node with different entities in your domain. So here's our first experiment. We picked about 10 APIs from our like op spot such as like scale up your deployment or reboot a single node or like say help me retrieve the support in a specific namespace. And then we generate about like 40,000 question answer pairs as the training data for our fine tuning. And we then use like five different model candidates and we throw the data in and hopefully to see magic happen. And it did happen. And we picked three models. At that point, I don't think code llama or llama two was available. So we use GBD Neo and some GBDJ as a benchmark. And we have our internal models to do fine tuning as well. So as you can see, it did help us to like extract the named entity like their IPs like the pause and like the namespace as well. And also it can classify the intention like in a very good accuracy. So we quickly like made a prototype and replace our chatbot with this language model. Yeah. However, when we asked the user feedback, it was not that good. Actually the model failed in multiple aspect. And one thing that it failed to comprehend different constraint set by the users when curing for Kubernetes resources. I'll give you a few examples. And let's say, okay. So, you know, users may ask, help me retrieve the parts in appending status. Plus the part has been there for four hours or they want to get all nodes in a specific kernel version. Or they have all these kinds of strange questions that you cannot enumerate all of them in your training data, right? So I don't know if any one of you can write a single kube control command to get all these information. At least I can't. So, but let's see. So someone in my team said, okay, maybe we just try to train a model, right? To translate the human intention to the kube control commands. Plus some like shell script like to do some grab to do some AWK to get the result. But these ideas got quickly beat out because for the several reasons. First, I think shell scripts are less structured and it is too like flexible and too powerful. It can actually generate malicious data and can be dangerous if you blindly trust the output. For example, what if it generated command say kube control delete parts dash dash on namespace, right, say it will be doomed. And it's hard to validate, right? How can you evaluate if this shell script you have a million ways to write, to get the same result in shell script, right? So at that point of time, we had another like idea from our current team. Our DBA team, they had this called DBGPT, this idea. So the idea is about to translate human intention like in text to SQL language. So we think, okay, what if we can curate Kubernetes resources like what we do using a SQL format? Because SQL format can be more controllable and auditable, right? And we have so many like SQL data that we can use to do like fine tuning or do model training. So we had our prototype and we say, okay, what if we can do this like, instead of letting the language model to curate a cluster directly, let's add a cache layer. So the cache layer was like a controller and followed by the list watch concept that you can actually cache all the parts, all the nodes, all the resources you want. And we do some tricks to convert them to like, to a format that SQL engine can understand and can retrieve the data. And that's when we can get some like table schema from SQL, right, from the database schemas. And together with the user's question and their intention, together with the table schema, we throw them to the language model and hopefully it can generate the SQL that can be used to retrieve the data. Okay, so because of the time limit, I won't get into how we do the SQL conversion here today, but we can do it like offline or in another time. So we had this idea and we had this prototype, but still we are, the users were still not buying it because there are still too many knowledge, internal knowledge that are represented by, let's say the labels and annotations on those parts. You know, like everyone, especially for all those custom controllers or the operators, they love to patch like labels and annotations on those parts and nodes to make them special and give them like different meanings. So it seems like we get stuck here. So what can we do? How can we like improve this? So let's go back to some basics. So at that point of time, we say, okay, let's take a step back and let's see how test the capability of the language model. So we run two simple tests. One simple test is to say, given a YAML snippet, can a language model actually say extract the targeted value based on the label's name? I think this is a very decent and very simple task, right? It's just a key value lookup. And so we quickly like say, generated some synthetic data, let's use YAML snippet and we generate a bunch of questions either asking like one label value, two label values or mix a bunch of them and to see the result. And we use our internal model to do like fine tuning to test it. Okay, so these table and diagrams here, actually different people can have different interpretation from that and our insight from our insight. One thing is that the model's accuracy actually is determined by the coverage of the training samples. What does it mean? So about if the model can learn like 60% of the label like in the training data for the rest of the 40%, if the label itself was not shown in the training dataset, it can still like recognize them and successfully extract the value with a very high accuracy. So that means like the models are not just memorizing everything and it does have the ability to learn a pattern. So that gives us confidence. But in reality, I don't think anyone's gonna going to ask this question to the model, right? You can just do it by yourself or just do a simple graph to get the answer. But what if we ask if the language model can extract the target value based on the label's meaning instead of by the label's name, right? So here I give another example, let's say there is another like custom key called XYZ as a label and there's a background knowledge here. Let's say if that label means the pod is able to survive during a short-term system blip. Okay, so when the user asks if the pod is going to survive during a crouplet hot update, which is like a short-term system blip, right? The model should be able to answer with yes. Okay, then we do another experiment in this scenario. But due to the time limit, I won't get into much details but in this time, I think line chain frameworks was already popular and so we tried to use that and so to combine the background knowledge.  And also we know like everyone says there's a new job called prompt engineer, right? So we try different prompt techniques, for example, like the train of thought. They do actually help to get the results better. For example, as you can see, no matter which on the model size, whether it's 1.3 billion model size or six billion model size, with the COT and the memory and the background knowledge, it can actually answer the question better, especially for those labels and keys, the model doesn't see in the training data. Okay, so putting things together, right? So here's our final solution to say, how can you use language model to carry a single Kubernetes resources with different user constraints that set up? So what do we have here? Like we have a prompt engine that can do different prompting strategy like train of thoughts or self-consistency or anything you can mention. And the engine can actually interact with our internal knowledge database which can fetch corresponding background knowledge with the user's input, right? Together with the POTS SQL table schema, we can send them to the language model to get back the SQL itself. And then on the caching and the Kubernetes part, we also add a federation layer so that we can only, not only can we just query for one single cluster, but also we can query things across multiple clusters. So that's our final design here. All right. So a quick summary. So what have we achieved so far? So remember, like in our experiment, no matter it's like API invocation or the kubectl, kubectl like get command, I think language model has the potential to be a good SRE pilot. And it has already demonstrated its capability to do named entity recognition as well as like intention classification. And for these very specific tasks, I don't think you need like a 100 billion size model or a smaller model may already be enough for this type of case. But still like remember our task is to operate the cluster, right? But as this point of time, I don't think language model itself can operate a cluster by itself. Still it needs careful education and humans needs to do like reevaluation. So it can be a good co-pilot, help you to generate the command quickly or help you to like say write a bunch of SQL or write a bunch of configuration code. But let's not be that creative, right? Because in our scenario, we want operational efficiency, we want productivity and that's a very deterministic problem. Okay, so there's no silver bullet like to all the questions that are listed out here during our journey. I think hallucination is the top enemy, especially in our scenario. And long context is actually one of the challenges we are going to tackle next because we did a bunch of like, we estimated how many like tokens we use for a single query actually we send to the language model. Currently it's about like 4,000 tokens each time. So that's about the, I think all the common models or the currently available online, they all just take about 40, like 4,000 tokens one time, right? So if we want to add more like background knowledge or we want to do more like prompting techniques, it requires a longer context and that's the issue we are going to tackle next. And everyone's talk about like multi-modality, like using text pictures and et cetera. But in the SRE Waterland, I don't see any models today that can handle time series data like logs, metrics or trace as well. They all like have the similar issue with the long context problem. So I think everyone can think about this because I think there's still a long way to go. So finally, what are the lessons can we learn from today? I think if you want to like use language model in your own scenario or in your own application, you can use open AI as a very good starting point to verify your idea. At least it can be, it is still a very good like baseline model you can use to test in your own scenario. But once you determine that, okay, this might be the task or this might be the things I want to solve and you want to involve your own model, please focus on collecting your user data. And you know like data is the key to this problem. As we all know like we all say that the quality of the data actually determines the quality or the upper bound of your model's performance. And algorithm itself is the way to help you to achieve that goal. But data is actually more important, all right. And in our journey, in our story, I skipped a lot about our engineering effort but actually in the past six months, I think we devote about 80% of time during for engineering work because training the model or do some supervising is kind of like straightforward as long as you can prepare the data, right. And please don't think of AI as a magic and sometimes I think in the SRE words, sometimes some if else can already do the job and if it's that case, you don't have to use the AI, right. So with that, I will conclude my talk today. We have our plan to like open source our model and also the coop query engine but it's still in progress and so you can follow this official account to like to keep up with our updates. The official account is a account that I created last year. My own intention was to like write some stories about SRE work life or like do some SRE best practices but yeah, I'm going to do a, there's not much in the official account yet but we'll do it later. Yeah, so thank you all for coming today. I know it's about time and it's about holiday time so happy holidays. Yeah, I can take some questions either Mandarin or English. So the question is about like how much time we spend on during the training and to get a model, right. So it's about like getting the data. It's an engineering effort. We write the code and we use the templates to help us to boost up all the data. It's about like two weeks work. Two weeks, yeah. And the actual model training because the model size is kind of small. It only takes about I think less than a week to get your like first the model so that you can do some quick verification. The other question is actually you mentioned this is the command, right? This is the command of the CPCR and then you just, and then there's the level. At least the previous command this is a very common question. So it's like, it's a child GBT. I, they're not always trained. We only have a lot of people to ask him. So it's like compare with child GBT. Other people already have this model. You have some use. The question is about like comparing to like child GBT like child GBT itself is already, is also involving, right? People are asking about like you can draw questions to child GBTs and they can get trained. I think in our scenario, our model is more focused and dedicated to solve a specific like domain. I think that's why like these days everyone is talking about all those vertical models like in each different domains or areas. I don't think there's a, you have to compare them, whatever fits you. Yeah. I think that there are some things that are very common, child GBT has already trained a better model for you. And for some of the things that we personally specified, we really can, right? For each company, for each project, they have some training again. I don't know how to integrate them. Because if it's something common, it's obvious that child GBT can't be very, very much. And it's already trained very well, right? Let's do it again. Is it better? I think so. Yeah. So we should like leverage the power of the open source models like CodeLama or other ones for all those very common and very base languages, right? But for all the specific domain languages or domain knowledge, like what N-group does for their API calls must be different to like what your company does, right? So that part you have to like either use like in-context learning that provides this information to the model or you do the fine tuning based on those like CodeLama or the other models already available. Yeah. You don't have to do it from scratch. Yeah. Yeah. Actually, I have a question. Actually, if it's something like Commander, it's actually quite simple. Actually, we also wrote it quite accurately. I was wondering if I could write it. I did write it. But for some troubleshooting, you just said at the beginning, the biggest problem is troubleshooting. Yeah. Together tracing, magicers, and loggers. And then for this kind of troubleshooting, do you think it can do more things here? Can it be done through this? After I ask a question, can it be better than us to do troubleshooting? Because sometimes we do troubleshooting through a push. But ChatchBD, it's actually not a, I understand, it's not a cherry process, right? So you say, do you think there will be more contributions on the side? Okay, so the final question is about, like for troubleshooting, how can ChatchBD or the language model help us? Yes, I haven't seen a lot of work in this area, but my gut feeling is it can help, but we have to teach the model first. By teaching them, I mean, we have to prepare lots of data. Like think about it, like if you are a seven year old child, you have to learn something new, right? Your teacher will ask you to recite or to memorize something, like repeat it again and again, right? So that's what we do to language model, right? During the training phase, right? So yeah, we are definitely trying that, but I think it will still require time, yeah. Thank you so much. So my understanding is this system and tool is actually used for, I guess, figuring out ops issues, especially during instance, right? Have you folks considered anything or done anything about, I guess, reliability side? About like how to make sure the model doesn't like, you know, like the thing that serves the model doesn't have issues or like, there any kind of like failover plan for this? Thanks. Okay, so the question is about how we can use language model for like, reliability reviews or to figure out if there is, like say during architecture review, say there is, if anything, we can spot, right? So this model is used during, I guess, when you are doing operations, right? When if there is any kind of issue, you're trying to fix the issue or run commands. So it's already part of the tools, the operators or SREs are using. I guess the tool has to have some, some kind of availability guarantees. Is there any kind of like consideration about the whole, I guess the model serving system or like the tool chain or is there any kind of failover plan in case the tool or the model doesn't work? Yeah, it's just like not available. Thanks. Oh, so the question is about how to guarantee the model itself, the model's availability by itself, right? I think that's the topic about ML Ops. We can discuss it later. Quick question. How do you ensure the circle generated is correct? Did you send it back to the model to modify it? We haven't tried that, but yeah, we have some like, we have two cases. One is to like use like human evaluation. Like say you have a hundred questions you can ask, right? You have your humans, yeah, you can write your own SQL, right? And you can compare it with your like, the generative SQL. And I think the accuracy that it can be, it's not that important. Why? Because I didn't show my demo today because it was not there like ready yet. The demo itself, let me picture it. So when the user asks the question, right? And the SQL actually generated. And the generated SQL will actually show up together with the SQL queries result, right? If the human itself needs to verify like if the SQL, if the data is not what he wants, right? You can modify SQL and do the query again. I think the tool itself, the main purpose is not to get this thing a hundred percent correctly, but to help you to quickly like get the SQL there. And so that you can do a small tweaks. Yeah, thank you. Thank you for sharing with us your journey applying LAM in this scenario. And that's a very interesting case. What my question would be when you open source to this model, are you gonna really release the entire model? So because we probably have the same issue as we are private domain and we do not want to use check GPD the major problem is they will have the data. So will you be sharing the model? And so we can try ourselves. And this is certainly the very early days. And I really applaud what you did for, I think there's a lot of application including networking, there'll be a lot of operation can be resolved by this. And as well as the LLM, I think there is a place to reduce the human resources. The question is about like what's our open source plan? Yeah, so we are thinking about it. Like first like we are like running experiment on like open source models like Lama two or like code Lama. And I think like open source model itself is one way and the other way like what the keynote talk today is this morning like the hugging face team, they are like collecting all the data, right? Or if we can contribute the data, then maybe like different people can use them different ways. Yes, that's another option. But yeah, everything is still on plan and you are welcome to join us to talk later. Hi, yeah. So let's say if I'm having a reader role on a Kubernetes cluster with, if I use the natural language problem, we inherit the same role-based access permission. So your question is about the permission to operating a system, right? Yeah, correct. Yeah, so currently as you can see, like as I mentioned, like touching the system itself is kind of dangerous. So we have several guards. First, as you can see, we only tackle the curing part. So this is a read-only model. And secondly, you don't want to like list out all your resources, right? So you will like consume lots of memories and bandwidth to the API server. So we add the cache layer, right? So that's the two safeguards we have. Thank you. All right, so that's it. Thank you all.


Welcome to this briefing on ARC identifiers for the Coalition for Networked Information. Since our last CNI briefing in 2020, the world of ARCs has grown considerably. In particular, the number of ARC organizations increased from 600 to over 1,000, and the former ARCs in the Open Community Initiative became the ARC Alliance. We're also happy to see significant new software services and publications. As a reminder, ARC stands for Archival Resource Key, a special kind of URL that can serve as a persistent identifier. Collectively, libraries, museums, archives, data centers, and publishers have created over 3.2 billion ARCs for content such as scanned books, medieval manuscripts, botanical specimens, fine art, scientific data sets, and public health documents. The CNI briefing two years ago covered ARCs for national history, genealogy, and publishing. Today we will cover ARCs for physical samples, biomedical artificial intelligence, and vocabulary terms. My name is John Kunze, and when I finish this preamble, I'll pass things over to Dave Igles from the University of Kansas. Dave has deep experience with earth science and technology and is lead developer of the Global ARC Resolution System. When Dave is finished, he will turn it over to Tim Clark from the University of Virginia. Tim is an associate professor in the School of Medicine, and he brings tremendous expertise in strategic thinking around a wide variety of biomedical identifiers. When Tim is finished, he will pass it back to me, John Kunze, from the ARC Alliance. I'll close the briefing with a look at permalinks for vocabulary terms, which combines my long interest in systems and standards for metadata and identifiers. Why should we care about ARCs? The average lifetime of a URL was once said to be 100 days. At the end of its life, a URL link breaks, usually giving you the dreaded 404 not found error that most of us have seen. Irritating at best, it's a minor disaster for memory organizations. ARCs are similar to DOIs or digital object identifiers. They're both persistent identifiers for accessing content and metadata. And ARCs are found in many of the places where you'll find DOIs. In contrast, ARCs come with no fees, no limits on how many you can create, and no metadata requirements. From the beginning, ARCs were designed to be decentralized and to identify any kind of thing, digital, physical, or abstract. Here's what an ARC looks like. At first glance, it's a URL that carries this internal label. To the right, the five digit name assigning authority number identifies the organization that created the ARC. Further right, the part after that names the thing that the ARC is assigned to. To the left, the host name makes the ARC actionable or something you can click on. It's also known as the resolver. ARCs are unusual among persistent identifiers in allowing organizations to run their own resolvers. The part starting with the ARC colon is the core globally unique identity. It doesn't depend on the host being available or on the future existence of the worldwide web. ARC organizations include a wide variety of memory organizations, nonprofits, for-profits, and government agencies. A few are listed on this slide. We've seen accelerating adoption in South America, Africa, and India. To see an ARC in action, an example from the Louvre Museum appears in the lower right corner. That's it for the background. Now I'm going to turn things over to Dave Igles to present our first use case. Hi, my name is Dave Igles and I work at the Biodiversity Institute of the University of Kansas. I'm also fortunate to be a principal investigator of the Institute of Samples, or I Samples Project, which is sponsored by the National Science Foundation. My Samples is a standard space collaboration to uniquely and consistently and conveniently identify material samples, record metadata about them, and link them to other samples, data, and research products. This effort would not be possible without globally unique, resolvable identifiers such as ARC identifies. Physical samples taken from the Earth environment provide a critical role in the sciences and our understanding of the natural world. These samples can vary considerably. For example, a geologist may collect mineral samples or ice cores, an archaeologist may gather various kinds of artifacts, and a biologist may collect samples for natural history catalogs and genetic analyses. Once gathered, these samples may be utilized in many different ways. Some are simply archived as reference material for later use. Some are analyzed immediately and consumed in that analytical process with only the digital artifacts remaining. The derived products or subsequent analyses often fall on the basis for publication, hence the original samples effectively provide a factual basis for Earth science knowledge. There is often value in revisiting analyses and other derived products. In these cases, it is often instructive or even essential to access original samples, tracing the provenance graph backwards from publication to analyses to the sample material, and even the location on Earth where the sample was originally taken. For this to happen, it is essential for samples and the derived products to have globally unique, resolvable identifiers. There are several types of identifiers in active use and ARC identifies bigger prominently in the physical sample communities for uniquely identifying not only the physical objects themselves, but also derived products including metadata, photographs, videos, analyses and publications. iSamples provides a hierarchical infrastructure of searchable physical sample catalogues with a penultimate collection called iSample Central, which gathers content from the various other collections and so provides a catalogable physical samples. The catalogues provide a simple way for researchers to find samples of interest, searching by time, location, collector or various other properties. Since each sample has a globally unique, resolvable identifier, it is trivial for the researcher to visit the original catalog and to link additional information to a sample or group of samples. For example, here we use iSample Central to visit Morea, a part of French Polynesia in the South Pacific. Each of the colored points represents a physical sample. The towers of points indicate the many samples were collected from the same physical location. Each sample point shown here has an arch identifier which is displayed when mousing over the sample points. Clicking on a sample, on a point, brings up the summary metadata for held by iSamples and of course, the identifier forms a link back to the original collection. Following that link we visit the Smithsonian Institute to see more details about the specimen and from there we may traverse cross references to analyses, publications and other material. This simple linking between samples is facilitated by arch identifiers. There are literally millions of these records, already in iSamples and the number of participating collections is anticipated to expand significantly as the project continues. In conclusion, globally unique, resolvable identifiers are being widely adopted by the earth science community as a common mechanism for referencing physical samples and all derived products. Arch identifiers are a good match for the technical requirements and fill an increasingly prominent role for the earth sciences community. That's it for my part. Now I'm going to turn things over to Tim Clark to present our next news case. Hi, I'm Tim Clark from the University of Virginia School of Medicine and School of Data Science and I'm going to talk a little bit about using ARCs in AI for biomedical research. Many people in my field will tell you that AI is now transforming the field of biomedicine and biomedical research and bioclinical research. And it's doing this by making very complex predictions that are kind of amazing really, but they can be very opaque as to how those predictions were made and they require explanation and robust evidence to be presented so that they can be trusted. So for example, an epilepsy detection model here in this diagram reads brain MRI data and processes it through a complex machine learning model and reports out patient is diagnosed with epilepsy, 85% confidence, right? The clinician wants to know why did you make that prediction? It's not a simple decision tree. It's something much, much, much more complex. Can I trust the AI models that made these predictions? You can't trust them. It's like working with a colleague. If a colleague told you I diagnosed that patient with epilepsy, you would like to know why he made that diagnosis. And you'd also like to know how he was trained. Is he board certified? Where did he go to medical school? And what's his experience in diagnosing that disorder? So I'm going to talk about two biomedical use cases for very deep AI machine learning. One is a predictive model of cellular response to drugs. I can use that, we hope, in drug development, pharmaceutical industry. And a predictive model to detect up to a week in advance, kids in the neonatal ICU that are going to have some life-threatening trouble. And the idea here is that the impending problem may not be evident on the surface to physicians and nurses and that they'd like to have some warning, long enough in advance to see if they can avert harm. So all of these things require results, validation, and explainability for the model. And there's ways to do some work statistically at the other end to tie the predictions back to the feature sets. But these feature sets, especially if they're processed a lot, even to the point of, how did I select the feature set? You want to have some robust way of saying how the feature set was derived. The pre-model explainability and the post-model explainability go together. We produce things called evidence graphs for pre-model explainability. They describe how results are obtained, and they provide supporting evidence, and they also allow reusability of the components. So we give every graph node in our network diagram essentially an ARC that is resolvable first to metadata about the node, which could be data, software, computational parameters, and then resolvable to the actual data software parameters, models, and so forth for explainability. ARCs have a huge advantage here, where in cases where you're doing thousands of chain complex computations and they provide the ability to specify the metadata flexibly in an open way. Beyond sort of the core bibliographic style metadata about the data set, you would like to know a lot of other things. So the first use case I'll go to is the predictive response modeling of normal and disease human cells. This project is part of the NIH Bridge to AI program, which we're doing in association with colleagues at Stanford, UCSD, and a number of other, Yale, and another of other institutions. So the idea is let's construct an accurate cellular component architecture. What's going on, and what are the structures and protein aggregations and interactions inside the cell based on high dimensional data, and use it to construct a deep learning model that can predict the response to biochemical perturbation. And of course, as I said, we have to interpret and explain the model results robustly. These are some, as a montage of some figures generated out of our first paper on this approach, where we have protein interaction data generated using affinity purification aspect. We have protein localization using immunofluorescent subcellular microscopy. And we have a whole complex process for taking these, calibrating distance, interprotein distances, placing them in the cell, describing the interactions, and building up communities of components. So the second version of this approach is going to add to the microscopy and mass spec lab products single cell RNA-seq processing, which will add a genetics genomics component here. And it'd be very, very powerful, we hope. And the tools pipeline that you see here, the second column from the right, is it does integration of all these components to produce the cell architecture. And then the last column on the right, fear integration, that fear stands for findable, accessible, interoperable, and reusable. It's an important acronym in the AI, NIH, and health care world is to generate the provenance graphs as things are being produced, the evidence graphs, and track the provenance of data sets. This is a preliminary map of cell architecture from the 2021 Nature paper, where the first results were published. And this was generated by a complex process. This is a provenance map of the process that is very formal, actually. And the blue rectangles are data sets. The blue ovals are algorithms. The purple ovals are AI models. The green rectangles are hyperparameters for model training. You can see that when we are going down the path to this thing in the red box on the right, the derived cell architecture, that's going to be the basis for modeling predictions. It's very complex. And it's quite possible for deviations or variations in different parameters and data sets and so forth out on the left to have significant effects on the right. And we'd like to know what those are. And we'd like to be able to rely on a well-defined and actually machine interpretable representation of the process. Each one of the nodes in this network is an object that has a set of metadata assigned to it with an arc assigned to it. And it resolves to the foundational data software hyperparameters and model and computations and computational conditions that derive this architecture. So arcs were something that we used in that. And we're being used in that. This is something that we did really this last year, fairscape. This is tracking what the vital signs information from 6,000 babies over 10 years using a large number of algorithms, many, many candidate features, 100 terabytes of data, algorithmic result clustering, and 17,000 or more actually computations per result. And we were able to do predictive analytics on these babies and predict adverse medical events. Actually, we were able to predict them seven days in advance. Every patient, every result or every patient that generated a result had a graph left like this generated that showed the provenance of the results. And these are stacked up for all the babies in the experiment. And the complete graph is 17,000 nodes. We know that it's important to be able to generate all these with persistent identifiers and arcs are free to generate. And they allowed us to use very flexible ways of representing the metadata, including a formal ontology called the evidence graph ontology to formalize this representation. So these are all serialized in JSON-LD using terms from schema.org and the evidence graph ontology. You can resolve an arc in here to return the metadata that gives you formal ontology terms and formal vocabulary terms. You get directed a cyclic graph of provenance slash evidence. We know that from this serialized representation that the results were derived by a certain specific computation. You can find information about the computation by resolving the arc and resolving a further URI in the metadata. And we know that they came from a certain data set. So this is really, really important. And it was great to be able to do this so that we know that we can rely on this result. So to summarize, why do we use arcs in our work? We use them as persistent IDs to robustly represent the data in our evidence graphs and all our results and all our computations and all our primary data. Their freedom and their metadata is flexible and there's a large ecosystem of users and developers supporting them. We use arcs for complex evidence graphs on AI machine learning predictions in biomedical research in the lab and in the clinic. And every node, every data set, computation, software, model, hyperparameter set is resolvable through the arc to its contents. Each node is persistently identified with an arc. So in conclusion, I'll just say that for us, arcs are very useful, highly flexible, and scalable persistent identifier model. They're especially useful for traceable complex computations. And again, I'll just reinforce. If you're doing work in biomedical research, the predictions you make in AI have to be traceable. They have to be explainable. Otherwise, you can't use them. Not in any sense where lives may depend on them. We're happy to chat with perspective arc users. And thank you very much for your attention. That's it for my part. And now I'm going to turn things over to John Kunze to present our final use case. This last use case describes work supported by the metadata research center at Drexel University. Why do we want arc identifiers for vocabulary terms? Because linked data and the semantic web rely on persistent URLs for concepts. Thousands of work groups across the world are inventing vocabulary or modifying existing terms. But we don't have easy ways to share and get feedback on changes. Some of this vocabulary labor adds an enormous value of great interest to researchers and scholars globally. So we built a crowdsourced metadata dictionary called YAMS. It lets anyone look up terms to reference. You can log in to comment on other people's terms. You can also upvote or downvote terms. And you can watch them so that you get an email if they change. This particular term is labeled vernacular class, which means it can be changed by the owner. But terms labeled canonical class are done changing. There are also deprecated terms, but they too will always be available. You can add your own term and immediately walk away with an arc persistent identifier to reference it. And there's no problem creating multiple terms with the same name string. This one has 28 alternates. But each alternate will have a unique arc concept identifier. You can also add tags to terms and import terms in bulk via a CSV file. Standardized metadata is less common than some people believe. The official story about the metadata standards that one's institution uses often hides an unofficial story. When catalogers, scientists, and archivists really care about accurate description and have internal workflows to support, what tends to happen is that existing standards are modified to suit local needs. Projects all over the world are giving rise to their own dialects of standards and interoperability suffers. This problem was described in 2004 with the first cross-domain metadata standard. That was the Dublin core, which would not allow more than 15 elements because the standardization process was so difficult. Domain-specific communities liked Dublin core and wanted to add a few elements, but they had to create their own vocabularies. As a result, here's the metadata universe that we inherited. Lots of metadata elements, right? But when we zoom in on a given area, we find with some dismay that those aren't elements, but entire vocabularies are ontologies with lots of conflicting overlapping elements inside them. Each point is an island of non-interoperation. It's not just cross-domain vocabularies that spawn dialects, but also narrow domain-specific vocabularies. In cryospheric science, which is all about frozen water, there's widespread disagreement on generalized terms such as glacier and puddle, and on specialized terms such as phrasal ice. It happens in any domain where object experts care about accurate description in their area of specialization. So with so many alternate terms, are all of them necessary? Who decides? Traditionally, busy experts in your fast-moving field sit on standards committees, which can take years to reach consensus, both for the initial draft and for revisions. For logistical reasons, field testing is often very limited before standards are voted on and published. We think YAMS is an efficient and interesting alternative. YAMS is not a standard nor an ontology. YAMS is a living dictionary of metadata terms containing all parts of metadata speech, all domains. Unlike an ontology, a dictionary tolerates unrelated terms sitting next to each other. Each term is like a proposed nano-standard. Some are upvoted, others not. Reputation-based voting, like Stack Overflow uses, resists gaming and helps standards committees choose among the best terms. YAMS is a microservice that ontology, software, and linked data can reliably use by referencing arcs for metadata terms. That's all for now. Thank you for listening. If you have any follow-up feedback, please reach out to us. We'd love to hear from you.