to the series on Valero that CloudCasa is doing. I'm Raghuram Naurakonda. I'm Chief Architect for CloudCasa. As you already know, probably CloudCasa is a Kubernetes backup service. So it works as backup as a service. But in this series, we really want to get into the details about Valero and there are quite a few topics that we have to cover. This is first in that series. And in this particular talk, I'm going to cover at a very high level what Valero is and how it works and things like that. But I do want to spend a few minutes on the CSI and in particular how to configure the CSI, what are some of the issues that we see from time to time, right? And if time permits, I also want to talk about the file system backups and restores supported by Valero, right? And before I just start the actual talk, I want to point out that everything that I show today in my slides and all the commands that I run, everything can be seen in this page. We will email you this link and this contains all the slides. As you can see, all the commands, pretty much everything. So you will be able to, if you have any questions, you can always get back to this page and follow along, right? All right, so with that said, let me start with this presentation. Again, there is quite a bit of material to cover, so I hope I don't overshoot the time. So I'm going to be switching between the slides and the demo because I want to really show the commands and how they work. And so we'll see the installation backups and restores. So many of you are probably aware of this. Valero is a very popular open-source project. It is a Kubernetes backup tool. You can tell that from many different things. You can see the activity in the Slack or how many pools are happening on the Docker Hub images, or for example, the activity in the GitHub, right? Valero GitHub issues and from all those criteria, it's kind of clear that there are a lot of people out there using Valero, right? And apart from your basic backups and restores, Valero has a couple of interesting use cases that you can think of on this cluster migration that you can do backups on cluster A, for example, and you want to migrate in a particular application to another cluster, maybe from on-prem to the cloud and things like that, you can use Valero for that. And also, for disaster recovery is another use case that Valero can be used. Let's talk about different types of backup because Valero does support different types. The most basic one is the resource data. So this is basically nothing but all your resources and their JSON specs. So Valero just picks up the JSON spec of every resource in the backup, puts them in a tar file, and copies that to the object store, right? That's the most basic backup that you can do. But typically you're going to do much more than that, right? You'll have persistent volumes in the cluster. You have application data residing in those PVs. So you want to back that up. And there you have a few options. One is you can snapshot the PVs. And if you're using CSI, that would be the CSI snapshots. But if you're not using CSI, which is getting rare now because CSI is very popular and that is definitely the future of storage in Kubernetes. So most people must be already using CSI storage in their clusters. But if you are not, for whatever reason, for example, if you're using the native EBS volumes on Amazon CKS clusters or Google persistent disks or Azure disks, those cases also, Valero can take the snapshot, right? Even if you are not using the CSI. And then the third type of the backup that I would like to talk about is what we call the backups, right? These are not snapshots. There is data transfer involved. So data gets read from the PVs and get moved to the object store. And in this category, Valero can use a couple of different tools. One is RESTIC, the other is Copier. These are completely different tools not related to Valero at all. Valero just makes use of them, right? And I'll discuss a little bit more detail in the next few minutes. The next thing that you want to know is the object stores, right? Backups obviously have to go into some storage. They need to be stored somewhere. And Valero supports a wide variety of object stores. And most of them you can consider as S3 compatible storage. Obviously AWS S3 is one of them, but not just the one, right? There are quite a few S3 compatible storage out there. Even Azure Blob storage, Google storage. So there are many different kinds of object stores that Valero can back to. So let's talk about installation. I think once I talk a little bit about installation, I'll go to terminal and I'll show you how to do the installation. Now, there are again multiple ways of doing that. Valero comes with its own binary called Valero. So that's the CLI method. You can use not only to install, but pretty much to manage your entire Valero installation in the cluster. So that's what I'll be doing today. I'll be using the CLI, but there are other ways. For example, Helm is one way to install, but there are other ways out there. And to use the CLI method, the first thing you would do is go to the Valero GitHub page, go to the releases section and you can download the binary here, right? So I have already done that, so I'll be using that. Before you start the installation, there are a few questions that you have to ask yourself, right? Because depending on the answers to these questions, the installation command or the installation mechanism is going to be different, right? The first and the most important thing is where do you want to store the backups? Like I just explained different types of S3 providers that Valero supports, but at this point you have to make a decision which particular one you want to use, right? Then do you want to use CSS snapshots? I would say just enable it all the time because almost definitely you will be using CSI storage or are you going to use non-CSA as well? So these are all the questions that you need to prepare for you need to answer before you go ahead with the installation. And finally, do you want to enable the file system backups? In most cases, I think the answer to snapshots and resting backups or file system backups is going to be yes. In our particular case, I'm going to use AWS S3 bucket for the backup. I will use CSI snapshots. I'm not going to bother about non-CSI in this talk. We will definitely do one more webinar on that because it requires more details to be covered. And finally, I want to enable resting backups. So with those answers in mind, let's go back to the terminal and go ahead with the installation. For all my demonstration today, I'm going to use a mini cube cluster. It is a local cluster which makes things much simpler. MK is my alias for mini cube. So that's what you will see me using. So if you do list the clusters, you can see that there is one cluster running. Valero, and again, I have an alias for cube control. So if you see the namespaces, it's a bare minimum cluster, right? So this is the install command that I'm going to use. I'll briefly explain what this is doing because we decided that I want to use the AWS bucket, right? So you have to select the plugins based on that. So I selected the AWS plugin. I want to enable CSI. So I enabled the CSI plugin as well. One important point to note here is what version would you pick, right? So I'm actually using Valero 1.10. So depending on the Valero version, you need to pick the right version of the plugin. And how do you know which version to pick up? So each plugin has its own GitHub repo. For example, this is the GitHub repo for Valero plugin for AWS. And you can see on the GitHub page there's a compatibility table. And for 1.10, Valero version, you need to use 1.6, right? That's the plugin version. And in the releases here, you can see there is 1.6.1. That is the latest plugin available. And you will see that's the tag, Docker Hub tag that I'm using. Similarly for CSI plugin, this is AWS, right? So this is the CSI plugin. The table says we need to use 0.4.x and the release seems to be 0.4.2. So that will be the Docker Hub, Docker tag image tag that we'll be using. The next option is use node agent. This is basically telling Valero that we want to enable file system backups. So this would install node agent on each node in the cluster. We want to enable CSI feature. And the rest of the flags are all saying, I don't want to configure any backup storage. Unfortunately, Valero command requires you to configure some backup storage at this point itself. But personally, I don't like combining the installation with configuration. So we are going to say, we don't want to configure any backup storage at this point. We're just happy installing Valero itself, right? So let's go ahead and do the install. I have another alias here. This is basically cube control, the check things in the namespace Valero. So let's first see what namespaces we see. You can clearly notice there is a new namespace created called Valero. Let's see what is in that namespace, right? So you can see a Valero pod up and running already and you see another node agent. Let's go ahead and see more what other things are there in that namespace. So you can see a deployment called Valero, which is running the Valero pod. That is actually the Valero controller that monitors all the Valero resources and then acts on them, right? And then you have a demon set called node agent. And this is a one node cluster, but if you have multi node cluster, there is going to be one node agent per node and that is required for the file system backup, right? So this is the installation, it's up and running. So let's move on to the next step, the process, right? Once you install the software, the second thing you want to do is to configure the backup storage.  While creating that, you give the configuration details. So let me go back and create that storage location. You can do that using cube control if you want or you can use Valero CLI. Again, I'll be using Valero CLI throughout this talk because it makes things a little bit simpler. Now to configure the AWS S3 bucket, there are a couple of steps to do. The first thing is you need to pick what identification you want to use for the S3 bucket. There are different ways of doing that and I again pick the simplest method which is access key and secret key based, right? There are other ways of configuring AWS S3. For example, there is a method called IRSH which is IAM rules for service accounts. Definitely, this is a recommended approach. This is more secure than using access key and secret key but this is quite involved. And in fact, incidentally we published blogs just few days back on how to do this. So Valero does support this mechanism and I highly recommend that you go and check this out. For today itself, I'll use the access key and secret key and for that method, the first thing you need to do is to create a secret, right? So I have a bucket. I just want to quickly show you bucket that I have going to use the AWS command line. So this is the name of the bucket that I'm going to use. Right now the bucket is empty. There are no objects there, right? So let me go ahead and create the bucket. But before you do that, the first step is you need to create a secret which contains the credentials for your bucket, right?  The file is going to look something like this, right? So default and access key and secret key. So I already created that file with my access key and secret key. So we'll go ahead and create the secret. The details are in this file, AWS credits, go ahead. So the secret is created. And the next step is creating the backup location, right? So let's do that. So this command is saying, I want to create a backup location. This is the bucket, right? This is the bucket name is intro demo, right? Let's do that. And then the credential is the secret that we just created. The provider is AWS. It's in US East 1. And the name of the backup storage location, I want to call it AWS demo, right? So we created that successfully. Let's see there is a command backup location get that is going to show you the details of the storage location you just created. It's telling you that it's available and the access mode is read, right? So by default, we need to be able to write and read. So that is the access mode we used. And this phase is very important. If for whatever reason, there's a problem in accessing the bucket, maybe you don't have sufficient permissions. This phase is going to be not available or pending or something like that. But in this case, you can see the bucket, well it will find a bucket accessible. So it says the phase is available, right? So we have the bucket now, backup storage is all done. So let's go back to the next step. The next thing I want to do is do a very basic backup, very simple backup that is consisting of the resources. There are different ways of including what you want to include in the backup. Typically I recommend doing a full cluster backup because you should include as much as possible in the backup, but at the restore time, you're always more selective. You want to restore only what you need. You have the option, you can say I want to backup for namespaces or you can say I want to backup only if it can be given label or even given resource types like PVCs or parts and things like that. For our demo, we will just do backup for entire cluster. And this is just a basic resource backup because we don't have any persistent volumes to begin with, right? So let me go ahead and create this backup. I'm calling it basic backup. And do note that I'm explicitly providing the storage location here. That's something that we created just now. There is no need for you to do that if you configure the storage location as default. There is a way to do that. But again, explicitly providing that it is always better, especially when you're doing demos, right? You don't, you want to have as less magic as possible. So here, let me go ahead and create the backup. It created and you can always do backup get. That's going to tell you how it is doing, right? So here it's telling us that backup is in progress. It's completed. I mean, there are not that many resources on this cluster. It's a very small mini cube cluster. So it's no wonder it finished in pretty quickly. Now, there are different ways of finding more details of this. So you can do, for example, describe on this and that will tell you how many resources are backed up, timestamps and things like that. And here it's telling you CSI volume snapshots. Obviously there is nothing included because we don't have any PVCs, right? Even there is a, even more interesting way to see more details about the backup that is you pass the details option. And here you're going to see the full list of resources that are included. So this is all the resources that are backed up and in this particular back, right? And finally, I want to show you how things show up on the bucket itself, right? So this is the bucket that we used.  And the most interesting of this is here the basic backup.tar. This is essentially an archive of all the JSON specs of the resources and all the lists that you have just seen. The JSON spec of all those resources are part of this tariff. The other you can notice is the logs. So you can, there is a way to see the log files for this backup. The others are practically nothing, right? There's very few bytes because we don't have any volume snapshots included in this backup, so you don't see anything else. I want to show the logs as well. So let's go and run that command. So if you do this command, it's going to show you the log file. So essentially it's basically pulling the log down from that S3 bucket and ensuring the log. So in this case, everything is completed. So you probably don't worry about the log, but in case of any errors, any warnings, sometimes the backup completes partially. In those cases, you can just do the logs and see if there are any errors, right? That's about the basic backup. Let's go to the next step, which is the CSI. So I do want to spend a few minutes here because the primary focus of this talk is how it works with persistent volumes, right? You're not going to have a cluster without persistent volumes, right? In most cases, there is a persistent storage. There is probably stateful applications like databases. So it's very important that you backup the data. And CSI is a standard to expose all the storage systems to the Kubernetes cluster, right? So with this Kubernetes implemented CSI, I think sometimes starting in 1.13, so it's quite a few releases back. And with that support, you just request the storage for the containers and your parts by creating these resources called PVCS, Red Persistent Volume Plates. And the recent CSI versions, and when I say recent, probably three or not even more, maybe five or six versions, you can, the CSI spec interface supports snapshotting the persistent volumes. And that is very, very important for the backup applications, right? But remember, one important point to note here is not all CSI drivers implement these snapshot functionality. The example here is AWS type EFS, the elastic file system. It does implement the other parts of the CSI. You can create the persistent volume claims, you can position the storage, but you will not be able to take the snapshot because snapshot is an optional functionality of the CSI, right? Configuring the CSI is not a joke. It's not part of the core Kubernetes as I mentioned here. So when you do the default installation of any Kubernetes distribution, whether it is cloud provider or on-prem, you are not going to get the CSI artifact. So you have to do something extra. And that extra includes installing the CSI customer resource definitions. There is something called external snapshotter that provides some common components to all the CSI drivers. So you need to install that as well. And finally, the CSI driver itself that interfaces with your storage, right? So as you can see, there is quite a bit of configuration that you need to do to get the CSI to work properly. In cases of cloud provider clusters like EKS, Azure clusters and GKE, the process is a bit simpler, definitely. But when you're talking about on-prem clusters, you need to check the documentation for each distribution for each provider and see how you need to do that because I can almost guarantee that the configuration is going to be tricky. And if you don't do that configuration correctly, what happens is that the snapshots will not work. So you can create a snapshot resource, you can request a snapshot, but nothing will happen, right? And in case of Valero, what happens is that Valero would wait for 10 minutes. It creates a snapshot, it issues a snapshot request, waits for 10 minutes to see if something is happening and if snapshot is not ready in that time, it will simply time out. So Valero will log an error message and we'll just move on with the other resources. And in fact, we see many users complaining about that. We see many cloud CASA users complaining about CSI snapshots timing out and we see Valero users complaining about that as well in the GitHub form. And we try to answer as much as possible what to do in these scenarios. And in almost all cases, this turns out to be a problem with your CSI configuration. So in order to make this process a bit simpler, right? We answered many questions, these are the steps that you need to do to verify the configuration. So based on that experience, we went ahead and developed a shell script which automates the process and makes things much easier for you. So this is the link that describes the details on how to do that. For example, I'll quickly show you that. Okay, let me hear. So this is the page that talks about how to use this shell script and run it locally. I'm going to quickly show that here on my Minikube cluster. So this is how you run it. Right, directly from online, you can run it. If you want, you can download it locally and run it. You can take a look at the script, what it is doing. The script will tell you what it does, but I'm just going to say yes. And the script failed right away. It's telling you that the CRD is volume snapshot and these things are not there. So there's nothing for the script to do. So I'm using Minikube and on Minikube, the way you do that is enable the add-on called volume snapshots. So I'll do that. But again, if in your distribution, it's going to be different, right? You have to follow the instructions for your distribution. So let's go ahead and run the script again. And this time you can see that it found the CRDs, right? So it went past the step and now it's saying I don't find any CSI drivers. So let's go ahead and do that as well. And again, on the Minikube, the way you do that is enable the CSI hotspot driver. So let's do that. And while this is happening, it'll take probably a few seconds or maybe one minute. So let's go back to this page. Not this page, yeah, here, right? So here, the page will tell you what exactly it is doing. It is creating a small PVC and then it'll issue a CSS snapshot for the PVC and it'll check if the snapshot is working or not, right? Okay, we can see that this is finished. So let's run the curl command again. And this time it found the CSI driver, found, so it's going ahead and it's creating a PVC, a pod. Now in that particular test path, it just means that it's able to create a pod and PVC and now is testing the volume snapshot creation for that PVC, right? And here it's waiting. It created the PVC on one volume snapshot and is waiting for the driver to do that. And actually you can see that the test has passed, right? And which means that our CSI configuration is in good shape. So now we can go ahead and further do the CSI backups with Melero. So let's go back to our presentation. The script is just cleaning things up so it will be done soon. So Melero's support with the CSI, so Melero supports taking the snapshot of CSI volumes but there is a little bit of more configuration that you need to do. So to take a snapshot, we're using CSI, you need to have a volume snapshot class. This is very similar to the storage class that you have with persistent volumes. You need to have a corresponding storage class, right? To provision dynamic storage. Similarly to create a CSI snapshot, you need to have volume snapshot class. And then you need to set this particular label on that and so that Melero knows which one to use because in theory, you can have more than one volume snapshot class. I would also like to point out that in the new versions of Melero, we are people are discussing how to improve this, right? For example, if you have only one volume snapshot class, maybe you don't need to set the label, right? Because you can automatically use that. So in the newer versions of Melero, this process would be a little bit simpler but for now with 1.10, we need to do that. And also I just want to quickly point out that when you restore from the CSS snapshot, Kubernetes allows you to create the PVC directly from the CSS snapshot. So there is no real restore happening and there's no data movement in the restore. And that is what Melero does. It creates a new PVC and it'll mark snapshot as the source of the PVC. So let's go back to the cluster again. I'll show you the one that I have the storage class. I mean the snapshot class. So you can see one snapshot class, the deletion policy is delete. It's highly recommended that you keep this deletion policy as retain. Otherwise, in some cases, you may lose the snapshot. So I'll go ahead and edit this snapshot class and change this from delete to retain. Sorry, let's do get again. And you can see that change took effect. And I'm also going to set the label here on this. So this is the CSS volume snapshot link, right? So now with that done, we can go ahead and create the backup, right? So let me do that. Oh, actually before you go ahead and create the CSS backup. We don't have any CSS part, right? We need CSS PVC. So I have a small test PVC and part using that. So I'm going to do that. So I'll quickly show you what I have here. So I have a small PV claim. It's coming from the storage class. You can see this is the CSI storage class. There's a part where I'm mounting this PVC and slash data. The part really doesn't do anything. Just sleeps for some time, right? So we're not interested in what the part does. We just want to see how the PVC is back up. So let me apply this file. And we will notice, we'll see this part coming up. You see the new namespace test at CSI. And let's see what is there in that namespace. So you see the part is running. Let's see the PVC, right? The part, the PVC is bound. So let's actually get into this part, eject into this part. Here I'm going into that and slash data. That's where the PVC is mounted. You see there's no data now. So just for the purpose of verifying restores, what we will do is create a very small piece of data, right? So you see a file here created and all you see is just one line with the dates down. So just make a note of this because when we restore this PVC, we should see exact same data, right? So we have the CSI now, so let's create a CSI backup. Yeah, so this is exactly the same as the previous backup command. So when Vero sees a PVC, a CSI PVC, and it finds a snapshot class, it would automatically create the CSS snapshot because we already included the CSI plugin, right? So let's go ahead and create this. So backup is created. Let's see what's going on with the backup. It's in progress. Let's see what's going on. Should be finished pretty quickly because there's not much data. So as you can see the backup has completed. Again, I'll quickly see the describe command with details. Sorry. So you can see that it is clearly showing you that one CSI volume snapshot will be taken, right? So let's go ahead and do a restore now from this. And here I want to briefly talk about the options. Here we're saying I want to restore from this backup. The backup is just created. And like I said, your restore is usually more narrow compared to in scope, right? Compared to the backup. We did the whole cluster backup, but I want to restore this particular namespace test and CSI. And not only that, I want to restore to an alternate namespace. So this also shows a cool feature of Wellero where you can back up, you can restore one namespace to with a different name. Here I'm saying I want to restore test and CSI to a new namespace called restore CSI, right? Let's go ahead. Restore is created. And let's see what's going on with this. It's already completed. And let's see what namespaces we see here. So you see the restore CSI namespace here. And if you see the pod, part is up and running. Let's see the PVC, right? And there is a PVC created. Let's again, get into this pod, exit into this, go into the data. And you see that the test file is there, is restored with the same data. I quickly want to show the PVC here. And I want to point out this particular one. So when Wellero created the new PVC in the new namespace, the data source is pointing to the snapshot. So this is what I meant when I said you restore from the CSS snapshots by creating PVCs from the snapshot. There is no data transfer involved in this, right? Okay, that completes what I want to talk about this CSI. Let's move on to the file system backup, which is third kind of the backup, right? So this is a different type and there is actually a data transfer involved. So in this type of the backup, Wellero reads the data from the PV and the data is transferred to the object storage. And for this purpose, it uses one of the two tools that you can use either RESTIC or Copia. Both are kind of similar. They both support compression, deduplication, encryption. There are some limitations. You can read them on the Wellero docs. And in fact, we will do one future webinar on in more detail to cover both the tools. We'll talk about the differences and we'll talk about how you can choose or what fits your scenario, right? In this particular demo, I'm going to use RESTIC. That is the default with Wellero. And one other point to note here is, in some cases, when snapshots are not supported, for example, if you have an FS or EFS type of persistent volumes, there are no snapshots possible with those PVs. And in those cases, file system backups are pretty much the only option that you have. And when final point is you cannot combine snapshot and file system backup. So you can either do snapshot backup or file system, right? Not both. So let's go back and see how they work in practice and then we'll come back to the slides. So we will use exactly the same PVC, the CSIPVC and the save pod to backup using the file system backup, right? So let me create, yeah, I call this FS backup. And you notice there is an additional option here called default volumes to FS to backup to true. So basically we're telling Wellero that in this particular backup, any persistent volumes that are found in the backup should be backup using a file system, right? That's what this option does. So let's go ahead, create that. And let's do get on FS backup. Progress. So this will take a few seconds. Again, it's the same data, right? So not much data, so it should really finish pretty quickly. And once this is done, I want to show the contents of the bucket again and talk about what object you will see on the bucket. Yeah, so while this is going through, again, if anybody does have any questions, feel free to throw those up in the chat. But I know there was one question there from Antonio. Yes, so we'll share this and probably also post these generally on the site and on YouTube, Antonio. So if you do have any other questions or want to see anything further in depth, by all means feel free to ask and we'll certainly share that with you. Yeah, thank you, Evan. Yeah, and I want to also show this again. This is the link that we will send to everyone. So all the presentation, all the commands that I'm running right now, everything is documented here. So in great detail. So please feel free to take a look at this. Okay, so the backup is completed. And again, let's do the describe command. So keep making this mistake. And you can see here, it's showing you that RESTIC backup is done. And I want to show the bucket. And suddenly you're going to see a lot more objects here. And you can see something created called RESTIC. And underneath that, you will see one, they call it repo. So RESTIC has a repository and all the backup goes into that. So Willow creates one repository per namespace. So you can see one for restore CSI, FS backup. I mean, these are all the namespaces, right? So you're seeing all these objects created underneath those repositories. Okay, so that's the backup. Now we will go ahead and do the restore exactly the similar scenario we did for the CSI. So let me go ahead and create that. Yeah, this is the restore command. We are saying I want to restore from FS backup. Again, restore only one namespace and I want to restore to a new namespace. This time I want to call it restore FS. Go ahead, created that. And then let's do get on the restore object. So it's restored in progress. It's completed not much data really. So again, let's see how many namespaces are showing up. You do see the namespace called restore FS. Let's see what is there in that.  And I want to show this PVC and distinguish this from what we saw before. You know, contrast this with the PVC we saw from the CSI restore. So with the CSI restore PVC, you saw the data source, right? Because the PVC came from the snapshot. But in this case, you don't see any such thing. So what happens here is a PV is dynamically provision. That is a blank, right? MTPV. And then the data is restored on top of the PV. Actual data is moved red from the object store and copied to the PVC. So again, let's eject into this. This is the restore CSI, right? This is, yeah, this is the restore FS. And again, we'll go to that mounted directory and you see the test file. And you can see the date, right? So the data is correctly restored. All right, that's about the file system restore. Let's get back to the presentation. A couple of key things that you want to notice. You want to make note of with the file system backups. So the data is being read directly from the PV, right? There are no snapshots in the picture here. So if the data is changing, actively changing, it would change probably if you're using applications like databases or any other applications which will be frequently or periodically changing the data on the disk in the PV. So your file system backup may not be consistent, right? Because the PV backup is happening directly from the data, live PV. So for this reason, you would like to use what are known as hooks. Valero supports this hooks to freeze and unfreeze or freeze and power the application. So before you do the backup, you want to freeze the application at which stays all the data is flushed into the storage and probably the application is put in some backup mode. Then you do the backup. And once the backup is done, you take the application out of the backup mode, right? And this is very, very important. So you should take a look at Valero hooks to make use of that, to make the backups consistent. That's about file system backups. Let's briefly talk about some limitations about Valero. Again, we talked about this already. But one important thing I want to point out is Valero is a single cluster solution, right? So it always talks about a single cluster. You install one cluster, you create all the custom resources on a single cluster. Backup storage location is also tied to the cluster. So everything that we discussed so far, everything is happening in the context of a single cluster. But it's very rare that people will have only one cluster, right? In fact, the recommendation is to have, people always talk about multiple clusters, multi-cluster solutions and things like that. And in fact, we see from Valero forum questions and things like that, people are using tens and possibly hundreds of clusters. And so what happens is that you will end up writing some tooling on top of Valero. You will never directly use Valero to manage individual clusters. It quickly gets out of hand. So what we noticed is that companies and users are writing their own scripts and even sophisticated tools on top of Valero. So most probably you would end up doing the same thing. And the other limitation I want to talk about is backups are not done from snapshots. So the ideal way to do the backup is snapshot the PV and then you read the data from snapshot, right? That makes the backups consistent. But unfortunately Valero doesn't support that at this moment, right? So I'm happy to say that we are thinking of working with Valero maintainers and in helping address these limitations. So stay tuned for updates in this direction. So we definitely want to help Valero project address this especially the second part, both parts, but the second part is something that we already solved with the CloudCast solution. And CloudCast already supports multiple cluster management, right? And we also solved the problem of backing up from the snapshot. So we want to share that knowledge and contribute that back to the Valero project so that everybody can get benefit out of that. So stay tuned for that. These are just the links that we already, some of the projects that we talked about. Right, in conclusion, I want to say that Valero is definitely a very popular open source option for you if you're thinking of backing up your Kubernetes clusters. There are a lot of people out there using that. So you should definitely give it some thought. But all I would say is read the Valero docs. There are a lot of things that you need to really think through before you go ahead and start implementing the solution. And like I said, you should give thought to implementing something on top of Valero to make this process a little bit more convenient for yourself, right? All right, that concludes the presentation. Hopefully I didn't overshoot too much. The time given to me. I finally want to mention about the KubeCon Europe that is coming up next month. Aaron, you want to talk about this? Yeah, so just real quick, I'm sure a lot of you will be attending or will be attending virtually the upcoming KubeCon in Europe. So that is April 18th through the 21st. We will be doing some virtual passes in the coming webinars and stuff, but stay tuned for all of that. And again, in the upcoming series for the Valero, titles and everything that we're going to be doing over the next coming months. So if you do have any questions on anything as far as the events, KubeCon, Valero, the webinars, Cloud Casa, feel free to always throw those up in the chat and we'll certainly get to them. But with that being said, I think that's everything for today. So I appreciate Ragu going through the presentation and demo and everybody that is on with us today and will continue to do these over the next coming months and weeks as well. So again, thanks everybody, thanks Ragu and we'll catch everybody next time. Okay, thank you, thanks everyone. Thanks guys, appreciate it.


kicked off let's let's do it so I don't know what it is for everybody but I know from me it's it's an evening I think so good evening good morning for everybody afternoon so we've got a few things in the action I review again we'll sort of remind people about the December hackfest and the preparations for that I think we're still looking for a venue so Tom will cover that brief reminder of the hyperledger wiki migration that's underway and communication tools and then Richard is going to go over our three quarter platform and introduce that for us and materials are hosted in the wiki there and maybe somebody could post that link into the to the chat so people can dive in and then vipin is vipin on oh yep there he is so vipin will go over his proposal for a numbering scheme for the hyperledger improvement proposals and and then Arno would like to discuss briefly some changes to the incident procedure in code of conduct because I think there's a number of other projects that are interested in maybe leveraging our code of conduct and then finally if you have time left we'll have workgroup updates so Todd do you want to kick it off with the hackfest reminder yeah sure thing we'll move pretty quickly through this for the hackfest we're still targeting December 5th and 6th in New York we think we have a location identified for that we're just finalizing a couple remaining questions and we'll update the group as soon as that's confirmed onward from there the wiki migration has begun as we updated last week we've locked down the old wiki on github so if you are one of the workgroup leads or have a substantial amount of content over there if you can go to the old wiki and continue to migrate that over I know a lot has already moved but there are still a few straggling things and we'll continue to work as well to to get that done and then the last thing as it relates to communication tools and Brian I don't know if you have anything to add here but if you have not signed up for an account for discourse which is at discuss.hyperledge.org please do so continue to test out that platform let us know your feedback good bad whatnot we want to see if that's a viable solution to use moving forward and Brian I don't know if you have Brian or because I don't know if you have anything to add on to that no it's not as urgent as the wiki migration and it's also not intended to be necessarily a lack of placement but potentially some things we don't use none of us for now that we do use back for we could we could there so that's great and if there's no sorry go ahead I was I was just going to add that you know one of the concerns I think regarding slack was that some people couldn't access it from China the feedback from here seems to be that it isn't a problem it's a problem but it's not generally a problem so yes I don't know how you know whether there's any urgency and I don't think that we're intending to replace slack completely but yeah give it a give it a kick there's a few people that have piled on the one but there's still only like one thread so be useful for people to get a taste of it okay anything else that's all in that front okay then I'll kick it off to Richard everybody can you hear me and can you see my screen I can't hear you and now I can see your screen it was up for a second and then it disappeared you may just need to re-share it there's a play button okay Richard Richard you should be the only one with an open line now great okay so can you see the screen now no okay how about now is it visible let's keep trying so it doesn't look like you're sharing anything there should be at the top here okay I can see now don't don't remove that I'm not gonna go for the screen I'll just go like this yeah good okay so hi everybody Richard Brown here so I'll try to be quick because to introduce a an entirely new platform on a call like this in a constrained time is is always going to be hard so so what I what I what I wanted to do was just highlight some the thought processes that's gone into to quarter and some of the some of the big the key I guess the key design some key aspects of the design that the difference to two other platforms that the reason I offered to present is is two-fold we released our introductory white paper back in the summer and I offered to present then but scheduling around my vacation and a few other things made it difficult so so I've owed this updates to the hyperledger TSE for some time but today was particularly good timing because we announced and this morning it's on rotors if anybody wants to search for it all three announced the quarter will be open-sourced it'll be open-sourced under the Apache to license on November the 30th and it's our it's our desire to to contribute it to the hyperledger project for consideration for incubation there are there's some details we have to work out there and obviously we if we can make that work we obviously have to go through the incubation process and incubation approval process but it's our strong desire to bring this in as another one of the incubated platforms under the the hyperledger project umbrella so I thought now was a good time now would be a good time to to brief the community on what this thing is and what we are what we're trying to do so if you're following along on the PDF I've moved to slide two and titled quarter a unique approach to distributing ledgers so so the first thing to point out is the quarter is is a genuinely new platform it we built it from the ground up it's not a fork of of another platform it's not it's not if you like a clone of another platform it's something we designed and built from scratch targeting the job virtual machine makes heavy use of a very large number of other open-sourced libraries but the the core that the core quarter is new code but it is a distributed ledger it's not a blockchain as well come on but it is a distributed ledger and with a particular focus designed to allow parties identified parties so as we'll see that there's the independence here for a permissions network to our permission parties to record manage and synchronize agreements that that exist between them so you can think of it as a platform for recording the existence status and evolution of of agreement legal contracts legally binding accords between identifiable parties as we'll come on to in the design it's it's heavily inspired by I think at least I hope it captures many if not all of the benefits of what we regard as blockchain platforms but with some very different design choices based on an analysis we did with our membership some time ago about why we think certain aspects of some blockchain designs are not appropriate for some scenario I should point out I don't I don't claim or I don't claim to report that pork recorder is the answer to to every problem it is different as people will see and it's intended to be a contribution to the the diversity of code bases that we'll see what we'll see for the foreseeable future as I said it's our intention to open source it on November the 30th and to to to contribute it to the the Hyperlegio project if we if we if the TSE agrees to to bring it in for incubation to get to get our heads around what the what the underlying philosophy of the underlying design of of of of quarter isn't and why it works the way it does it's useful to go back to a key question that we in our members thought deeply about the at the end of last year and the question the question we asked us of and if anybody was on the the white paper called yesterday you'll have heard me make a similar point yesterday the question we asked ourselves was there is so much hype and excitement around blockchain and distributed ledgers but none of the original platforms whether that is Bitcoin or Ethereum or even Ripple to some extent but certainly Bitcoin and Ethereum none of them were designed with the express aim of solving problems that the real businesses have and regulated financial institutions in particular but they but they but they do solve real problems you could argue that the the design goal of Bitcoin was to create a system of censorship resistant digital cash and its architecture satisfies that goal it's a very specific very unique very elegant architecture and it's in its solution to that business problem how can I create some citizenship resistant digital cash if I look at Ethereum Ethereum's architecture has some similarities with Bitcoin but it's also very very different and and my arguments and the conclusion we reached in our analysis was that's because public publicly it solves a different business problem the problem that Ethereum is trying to solve one could argue is to create a you know an unstoppable world computer from which many other many of the consequences flow so there's an interesting observation here that two two different platforms two different sets of business requirements have some commonalities but they also look very different so so as we now look at the the application of this technology to finance which was the the brief that was given to us by our members we had to ask ourselves what what what are the what are the characteristics that that that unite these blockchain platforms and and what of them is a particular relevance to finance and and the the two key conclusions we reached which led to the design of quarter was that and it is both of these when you hear them you probably some of you may even roll your eyes because they're so obvious but they have to be said is that the thing that you know of you unite to these these different blockchain platforms is that they bring or they allow us to bring groups of of entrusting parties parties who don't fully trust each other bring them into consensus about the existence of the nature and the evolution of a set of shared facts absolute Bitcoin nobody knows who anybody is and they don't necessarily trust each other but there's a set of shared facts that everybody comes to consensus on how many bitcoins are who owns them which addresses on them rather who can spend them on Ethereum similar threat model similar model about to trust each other and who doesn't but the set of facts isn't very different is the state of a global shared virtual computer but that model kind of fits but the the entry point to quarter is then to ask ourselves well if we've got this if this genuinely is a new space this decentralized database space if you mis-distributed ledger space if this new space is a is is one where we can bring parties who don't fully trust each other into consensus about the evolution and existence of a set of shared facts what are those shared facts that are pertinent in finance and again you can say the obvious is the answer is obvious but it has to be said the set of shared facts of relevance in finance are pretty much an empty piece of data the finance processes it it's the contracts is the web of legal contracts that exist between financial entities and their clients if I if I have a balance in a bank that is a contract between me and the bank this is the bank goes with the money if I have a if I've entered into an interest rate swap perhaps between sort of a corporate customer bank then there's also a contract that needs to be managed so so the so the challenge we set ourselves was or the conclusion we reached was yes distributed ledgers in blockchain technology that's look to have huge value when applied to business because if we can come to consensus about these shared facts then we don't need to record them multiple times in different systems and and deal with all the discrepancies and all the all the risk that that follows and we know what the underlying building not needs to be it's this idea of shared agreement these these these shared facts the reason why we then began work on prototyping the ultimate led to quarter was the the observation that it does not follow therefore that those architectures that were designed to solve obviously the answer for this problem that that I've just articulated in finance and some of the reasons for that are ones that of course have been well rehearsed in different ways by the project in the tiers things like whereas the Bitcoin and blockchain Ethereum architecture requires all data to be shared with all parties at least in their early incarnations that's not appropriate to where you have obligations to your clients for privacy in reality business contracts for example often require a multi-step process of negotiation and backwards and forwards before we can actually sign to say it's final so often needs to be the ability to move things around backwards and forwards rather than just broadcasting transactions and a few other things that we'll get on to as I get into the deck but but those thought that thought process is all what led to us to it's type and ultimately start building quarter what I propose to do now in hopefully in short order in which really is charged quite quickly it's just to give you a taste then of what the fundamental building blocks of quarter are and then and then and then and then describe a few of the features of the end that make it different and hopefully interesting to the community and perhaps something that will be a worthy contribution to the debate so before I moved to slide three I realized that introduction was was quite lengthy so I'll just pause make no questions and maybe told if you can confirm that actually can be heard and I haven't been talking to myself and everyone's lines were muted so you'll just have to individually unmute if you have questions for great thank you okay so I'm so sorry three and this is a very very simplified idea of you know what we're trying to achieve with quarter it's the idea that a deployed quarter network you can think of as just containing and managing a huge number millions billions of what we call state object a state object you can think of is a very reductive it's a very atomic thing state object is the fundamental unit of data in quarter and it represents it represents a thing it's intended to represent an agreement so so here you could have an agreement that represents was intended to represent the existence of a balance at a bank so we see the first version of this and a bank London bank has issued a hundred pound US dollars to a customer who is who is exports are limited and the idea is the thing we are trying to achieve with quarter is to ensure that those who need to be in consensus about this fact in this case just the bank and just its customer at this point have that object and consensus about it and there is no there's no ambiguity as to whether that is the most current version and the the task we're selling ourselves is as that as that agreement evolves perhaps the money is paid from the exporter to another bank maybe it's paid to a shipper maybe the cash is redeemed for for physical cash or the cash object is redeemed physical cash or or back to a normal bank account we need to make sure that everybody who needs to know the status of this object have the same one and they know that they know they have the same one and they can be no ambiguity so so at a very at a very high level you could argue this is a so far so much like quite a few other platforms and we've got the underlying data model which is a state object that represents a legal agreement it's visible to and shared only with those who have a legitimate need to see it I'll define what I mean by legitimate need slightly later and our and our objective is to make sure these objects evolve correctly approved by the right people share to the right people at the right time so very very very simple underlying objective so a little bit more on state of objects because these really are the the fundamental building block in the in the architecture so I'm on slide four now so this is a another view of and I use cash throughout this because it's a necessarily the dominant or most interesting use case but I think it's something that everyone can everyone can engage with so so if I wanted to model the fact here we have a hundred dollars this time issued by a different bank I've never done here but this bank what does the developer need to do on the platform to make this work so if you to implement a cash contract recorder you define the state object you know what are the fields we need to care about who issued it what's the amount what's the currency who's the owner but we also have two two other specific things so that the developer is expected to include one is the obvious one which is the is the reference to the governing contract code as we call it think that is the smart contract code and this is code that governs its evolution now importantly and I think in in contrast to a theorem like systems I guess in contrast to fabric as well is that that our contracts make a distinction between verification logic and transaction generation logic and by that I mean the absolutely minimal thing the developer has to do is documenting code verification logic state transitions so if I if I if I possess some some some cash issued by a bank that's here the the consensus learn is concerned with whether with whether any proposed updates of that object in actual fact and replacements of that object is objects are immutable in the platform where they're given updates is valid or not the the the question of how a valid transaction is constructed is it is literally kept as a as a separate concern so that the amount of data on which we have to be or the amount of logic with which we have to be in consensus with our peers is deliberately minimized and is focused exclusively on verification for those familiar with the Bitcoin model it's it's quite similar to to the verification model of Bitcoin and the and the second thing is we always link to governing legal pros as I said the although this platform is far more general purpose than just finance it was inspired by requirements from finance and the one of the key things which point to achieve is that state objects recorded on the platform correctly signed correctly notarized as I'll come on to are legally binding they are they are authoritative for the facts that they purport to represent and therefore we need to explicitly link to the overarching legal pros from which that legal enforceability derives so so what is the governing contract what do we do in the event of disputes what happens if something exceptional happens that hasn't been allowed for in the code so so building blocks state object what is the what are the fact what is a single fact which we want to be in consensus with somebody else what is the code that governs its evolution and in the event of dispute or in the event of of questions what's the overarching legal pros that governs it so again very simple quite deliberately reductive and minimalist data model from which the rest of the system builds I'll pause there for questions before I move on and so Richard this is Chris so what what form does that take because you've got you know issuer blank and owner blank agree that yada yada yada is this some form of you know like an XML legal you know legal XML document or is this just a PDF of the contract help me help me understand how this yeah oh yeah so really really good point and we and yeah there's no magic in this so so if you're thinking PDF you think the right thing so although there are and we know there's active research by by many firms and academics on being able to derive the executable code from from the legal from the legal domain and things like that the model here is is intended to be far simpler so the legal pros you can think of but the right mental model for this is to think about the old days in which in which sort of household insurance was sold you go to a broker and they had that there's a stack of paper which was the insurance policy with the details left blank and then while you sat there the broker would fill in the details and then stamp it at the bottom so you can think of the legal pros as being template the PDF template and the the state object filling in the gaps so that you have the template plus the so the template is something that lots of people will reference that's slow changing there's not many of them and then you have instances of state objects which is part of some transactions and the template plus the state object plus the appropriate signatures those things combined and what give you the the the legal contract okay interesting okay okay just before I move on there's a couple of questions one from VIP in on the the chat about can you and should be traced yes we'll come on to that in a minute Jeremy's asking about I think I was asking quite a few questions versus FPML yes we've done some work with integration with FPML one of the projects just very quickly on this one of the projects we we did but it feels like a long time ago now but it was back in April it was showing and this was public it was made by one of our members Barclays showing how a two parties to a deal to this in a case an interest rate interest rate swap could could collaborate that collaborate with it in such a way that they could produce the appropriate documentation in that case is to the standard the standards body for derivatives and have that information flow straight onto the ledger we didn't use FPML for that that will that that's something we'll get to later privacy I know that's a key question I will get to that in a moment I'm also conscious that I don't want to consume all the time on this call so Chris unless you tell me otherwise I'll try and be done by 40 but if you want me done before that shout to the minute I think that should be good because I think I understood from vipin and Arnaud that they did not have cool okay so I'm sorry I want to I won't dwell on the next few charts except to say that state objects you can think of as the as the as the system at rest they are the facts that are recorded by the system state objects are immutable this is this is this is it's it's very different to the Bitcoin UTXO model but if you're familiar with that the idea of states that are produced by transactions consumed by other transactions and that's the right mental model to have so the way to think of ways to think of quarter is at rest there's a whole bunch of state objects that represent the current state of your note and you'd have just the objects that pertain to you or which you've needed in order to verify transactions and I get to that in a minute and transactions are how the system how the states evolve so a transaction either produces new states such as an issuance transaction as we have here so a state that previously didn't exist comes into life and when it's appropriately signed and distributed to the right parties there's now a new state object representing a new contract on the system and the states can also states have the impression of evolving because an existing state perhaps money that's currently owned by me I can pay it to somebody else and I do that through a transaction it consumes zero or more input states so they are now dead if you like they're no longer on ballot and it creates zero or more new states so the model based on essentially a set of active immutable objects and transactions consume some of those some of those some of those immutable objects and produce new ones yeah so very much you know in inspired and then and based in some ways on the UTXO model I'll get on to consensus and privacy on the next few charts the back to the transaction verification so we now think we have this machinery we have a trans have a transaction which can assume input and produce outputs this is now where the the contract code comes in because if I if I want I produced from these transactions and provides you know helper and help methods for doing this I then send that to the people who need to verify it and their job is not to and this is important it is not to read rerun the transaction creation and rerun transactions and logic it is to verify that the proposed transaction is valid under the rules of the system and this is quite important for quite a few use cases we see where the the the the act of creating a transaction is either computationally intensive perhaps in some necessary scenarios or where the algorithm that you use may need to be proprietary but where the verification of whether that transaction is is potentially valid or not is much much is much cheaper computationally or doesn't require proprietary code so that that that ability to split the the the the constriction of a new state for the world from verifying it's it's it's it's it's it's correctness it's quite important in our model but it should be said in cases where you don't mind everybody running the same code to verify then then they say you know as a trivial implication that that is possible so so how can we deal with with double spends and so we think about the model we have here we have transactions that consume inputs and produce outputs so the obvious the obvious and I argue the the the solve problem we have to to solve from a from a transaction processing and and consensus management perspective is I could produce I could create two transactions and the try to spend the same inputs to two different people so the the double spending problem that Bitcoin has and of course that's that's what blockchains are for right you send the transaction to the miners or the validators or whichever whichever architect you use and they they they use a group that that that mine that that that that blockchain cluster of you like is trusted by all all participants in the system to to select precisely one transaction out of the pool of conflicting transactions and when when that when that selection is being made often by often by grouping several of them into a block and allowing it to allowing it to be committed in some way you can know that your transaction was processed and the other one wasn't we generalize that quite heavily so we say that the the service being provided by miners in public blockchains by the by the transaction processes generically in in in private blockchains it's a it's a it's a transaction uniqueness or notarization service that it's providing whether it's a single centralized node whether it's a blockchain whether it's a PBFT cluster the service being provided by the miners is one of a testing the given transaction does not conflict with another transaction that has previously been committed but whereas i think other architectures tend to have a single transaction selection mechanism for a for a whole system and there's a single blockchain we allow multiplicity of what we call notaries and that's for several reasons so first of all these notaries can can be centralized although that's obviously not not not not optimal and or they could be in in in some scenarios half performance wrapped up access clusters and in in more exacting scenarios they'll be present time for tolerant clusters as an aside you could implement a notary on top of an existing blockchain or on top of them you can even delegate down to bitcoin but that's not that's not the usage we envisage for what we're doing but but importantly you can have lots of these notary clusters and that's for several reasons one for a global network we may want there may well be a lot of transactions that take place only between parties in a single geographical area and because we don't have a model based on sending data to everybody that data can be localized let's say to London can be can be localized to the parties in London who are trading with each other and they can be a high performance notary that they all trust in London but they can also interact with parties for other purposes you know elsewhere in the world where we're perhaps a globally distributed pbft notary would be more appropriate so there are so we support multiple notaries for those kinds of reasons we also support them for what we think for what we call onboarding reasons it's likely in the early days of these these networks our belief is that certainly for some use cases many institutions will be very reluctant to rely on on consensus services provided by by diffuse groups or by groups that are outside of their control or which for which they can't necessarily be held accountable so it may well be the case that participants in the system need to self-notarize some transactions so for all those reasons we support what we call up what we call pluggable consensus that can be large numbers of notary clusters each running different algorithms and providing different qualities of service but all part of one network and that will explain more in the forthcoming technical white paper just pause there's a few other questions whoa there's a lot of questions right i'm not going to attempt to answer all them until i've just got a bit further and then i'll whisk through them all so clearly yeah okay lots of questions okay so that was that was consensus very quick quick point on one other piece because we are focusing on the the the the the the creation and agreement of of legal contracts and we know that this could be quite a complicated process we also have something we call the flow framework which allows developers to specify the the backwards and forwards interaction of transactions and partly signed transactions between participants in the system so you can think of this almost as a decentralized workflow or a decentralized choreography feature so that if three or three four five however many parties need to collaborate to to construct a transaction and get it correctly signed they can do so on the platform using the same reliable point-to-point messaging network that the platform uses but but all that backwards and forwards all that ephemeral data that is just there for the purposes of constructing a valid transaction none of that then gets committed to the global state and so you know the parallel is it doesn't void the blockchain in the way they're trying to do workflow on say a theory and often does there's a lot more to say on that but I'll skip past it so so privacy so the the mental model that the people should be beginning to build apart build is that the transactions are only shared with those who have a legitimate need to know so I need to define legitimate in cases where the the the deal is what we call these linear linear states deal a deal where the the parties to the transaction actually are known upfront and and and change slowly perhaps the evolution of an interest rates swap the evolution of the deal then the data simply is shared between those parties so an example here we see bank a's vault have a name for the wallet and bank b's there's an interest rates swap held between a and b we see the the the very top two green boxes in banks a and b's vaults those those those objects and their evolutions will never be seen by anybody other than a and b but what you also see is you see some cash states and and and in the cash case well you know there's no magic going on here if i i'm a bank and i've issued some cash to you and then you then pay to bob and bob pays it to to charlie if charlie's going to accept that they need to know they really do have a have a claim on the bank of alice me who's just paid it to them so it's so it's so it's so in that in that scenario in order for for charlie the ultimate recipient of the of the payment to be sure they have indeed ended up with a claim on the with a claim on the on the initiating bank bob who's paid paid charlie has to provide the transaction chain necessary for for charlie to satisfy themselves of the provenance of that data so included in the included in the architecture is the ability for the recipient of a transaction to to request its relevant dependencies from the sender now an obvious question is well hang on richard you know you're trying to pull the wool over our eyes doesn't this just doesn't this just degenerate to to a full public blockchain and the answer is um the answer is no and the reason for that is this only this this transaction walking only has to go back as far as the issuance of the issuance of the asset cash in this case so it's a partial transaction tree that needs to be provided number one number two we use address randomizations you don't know who the who the parties work with the previous transactions and thirdly because of the use cases we're addressing and looking at where the the the target is regulating financial institutions who have regulatory obligations to monitor transactions and ensure that that inappropriate transactions don't happen in reality we expect these chains to be very short because the issues of assets will will have an active part in their processing as well but the but there's um but but but i'm not pretending there's anything magic here the the the desire here is to build a system that shares the absolute minimum of data whilst still being still being um still having a high integrity so so let's slide and then i'll look through some of the questions so so in summary um we've um talked about states so data is shared on a need-to-know basis so nodes as I just said will provide the dependency graph of a transaction if necessary for that the receiving nodes to be able to to verify it and but there's no global broadcast um more technically um states and we persist we have an embedded relational database we'll um we'll support external relational databases so that information that's processed by the nodes can be can be joined with private data held by by by banks there is a scheduler built in so that states can ask to be woken up or for transactions to be run in the future because of the UTXO model and the no global broadcast transactions can happen in parallel on different nodes without either node even knowing the other one is doing anything um and we talked about how it's um it we're targeting the the the job of virtual machine um a couple of other noteworthy points because we don't broadcast data and we have very precise needs around where data flows nodes are actually arranged in an authenticated peer-to-peer network communications direct and it's over a reliable store and forward messaging so there's an underlying MQ and Q network upon which quarter runs and then finally as we discussed you know that there are no blocks transactions are notarized individually and we support um applicable consensus and multiple consensus mechanisms running at once it will be open sourced on November the 30th um there's still a long way to go the code is we're very very pleased with the progress the team has made but there's still a long way to go so um so we're really looking forward to to working with the community to to to see to what other uses it can be put but also to to encourage a large large range of contributions um I'll pause there I said I'd be done by 40 L bar was just ticked over to 40 but I know there's a whole bunch of questions so um so I'll pause but I'm happy to answer a few questions Chris yours talk about the job of virtual machine and we're actually writing in a language called Kotlin which you can think of it's from JetBrains as a um as a sort of like a more modern Java but without without the complexity of of Scala and we found it to be very very productive and our developers the exception of one none of them knew that language when they joined and within a week they were all productive very concise and very productive so I'll pause there for questions or for us to move on thanks Richard um any remaining questions for Richard or maybe we can just sort of take Richard or maybe we can so I didn't hear that too yeah I didn't I didn't catch the question looks like there could be many questions so there was like there could be many questions so there was a proposal to have it spent on the mailing list yeah absolutely um I'm very happy for that we um although the code won't we're still um getting it into the state we want the code won't be up till November the 30th but we're very happy to discuss on the mailing list um yeah very very happy to yeah I think I see Hart suggested that as well I'm delighted to yeah um um actually I just a couple of other questions and MicaMintel's asked you know the language and assumptions are very financial will this work for the utilities that that's a very fair point um we we wanted to we wanted to have a very firm and clear set of requirements so we could engineer against something real if you like and finance is clearly where this this comes from but when you look at the underlying architecture state objects representing agreements between parties transactions that evolve them um the actual the actual machinery is is is very general purpose we chose a specific domain understandably we chose a specific domain um for the um in order to to drive a coherent architecture but um but I'd expect to see a lot of use outside finance as well yeah for sure okay well thanks Richard that was um I I know personally I'm I'm really excited about this and I'm glad that R3 is doing this so I'm looking forward to see how this evolves um so let's let's do that let's take any remaining questions to um I guess the hyper ledger dash technical discuss list would probably be appropriate right and I'll ask I'll ask some of the team to jump onto that as well right super super thanks thanks okay thanks Richard and uh so now let's transition to vipin vipini you on off mute yes there we go so this is to revive uh something that was actually in the initial proposal for the project template proposal it is actually there in the proposals which says that every project that is incubated or somehow considered to be part of the hyper ledger would have a number assigned to it much like the numbers assigned to bitcoin improvement proposal or ethereum improvement proposal or any of the other systems where there are large numbers of proposals or projects incubated so that that would you know basically the point is that somebody's got to assign these numbers there there has to be an editor or someone uh either from the linux foundation or somewhere else who would say okay now that this project is accepted for proposal this is the number assigned to it that allows us to gather them together in a single matrix or or a place where each proposal numbered proposal and its description and everything you just click on it and you go to that project so this is more of an organizational you know or a governance kind of thing and i and i think in the beginning you know when there were only two it was all right but now we are we have at least five or six and now more with quarter so that is essentially the trust of uh you know what i wanted to say and basically we have to assign numbers and to organize ourselves in a way that the wiki is easier to navigate and everything is easy to find at least on the top level that's it thanks for coming so i mean what what i have been doing and of course i i need to go and do some things to the wiki but was to have all of the proposals added to the wiki and and link directly from there and then to update to indicate the status whether it was approved and then linking to the minutes of the meeting which is better than my intent i don't know that we necessarily need to have a numbering scheme but i mean we could but then i was just linking to the wiki the title so that people could figure out what they're well the number is in the uh original uh template that the tsc approved so although we didn't set up a mechanism for doing so it's already there it's not a new thing yeah no i i i understand i understand but then it's some matter of maintaining some sort of you know like you said editorial oversight over the number assignment and so forth but yeah we could we could do that i don't see a problem what do people think don't all speak at once yeah uh sounding silence is the silence agreement disagreement or so this is on all speaking can you hear me so i you know i cannot feel like criss i think you know i don't mean to push back because i understand it was in the proposal we accepted initially and so maybe we should just implement what we had agreed to but i understand the numbering scheme would be a good way to force some kind of organization but fundamentally it's just another piece of information that you have to keep track of at the same time so i'm all for making sure that we have a clear list of all the different projects that are going on and get better organized on the weekend this regard i don't know that you know we necessarily need to have a numbering scheme for that but again you know i wouldn't push it back if vpn thinks it's important that's fine with me not that i think it's important i i'm just following practices good practices that have been followed in other projects and it seems to work because you say bit you know bit 32 you know what what it means actually you know the guys who are in the in in the bitcoin world for example would know that so it becomes more of a reflexive thing and a path towards a process it is not you know anything magical it is just a way to organize yourself and since you follow this process at least you know this is a very very sort of elementary thing i mean it's not even that complicated it's been there ever since internet like you know rfc is numbered and if you search for a number rfc number then you know what it is these kind of schemes have been there for a reason because it's there in an open community and everybody refers to each project with different names so when there is the commonality at least with this numbering scheme you get that if you if you really feel like it it should not be there i'm not going to stand against it but you know i feel it's a good idea because you're getting more and more projects into incubation and to organize it better it would be better to do it that's all i have to say on the matter okay um well i already have to go through and do some spelunking because i after sawtooth i never went back and and linked to the uh to the tsc minutes when they uh the proposal was approved so hi hi this is jared so i'm actually supportive of it i actually think we're all kind of implicitly saying the same thing right there's going to be a list of projects somewhere like that's just kind of a standard organizational tool so just kind of the bookkeeping of a you know whatever whenever something comes across your table give us a next number in the series i think makes sense because as time goes on it's highly probable they're going to have many projects with very similar sounding names um and you know since we already have to have a tabular list of this or we should have a tabular list of somewhere just for an organizational standpoint i don't see necessarily being problematic to just in the next column just say okay you know this is project one this is at one this is at two that you know as they come in and whoever's updating that list so someone's updating this you know maintaining the list somewhere just give it the next number in the sequence this way there's no ambiguity uh you know about different names or or similar sounding names as you kind of go forward so i i do think that there's probably value of the proposition and i don't it doesn't seem that what anyone's saying is is kind of an odd spot and if you were saying oh well we should have a tool to be track of it and list of all the you know list of all the projects okay so if you have that then there's pretty much zero overhead just to put a number next to it as well so i and i do see that there's a potential for some ambiguity and similarity in naming conventions of projects that could potentially cause some confusion so i think this would make it kind of unique and unambiguous but that's why i think okay so i i don't think we actually need to make a decision per se unless somebody wants to object to implementing what we have agreed to it's more matter of implementation and it means that yeah we got to go in and do the work so exactly i have uh sort of on my to-do list to go through and clean up the proposals page um with links to the minutes where they were discussed and approved and so forth and i can add numbers okay thanks whipping unless anybody disagrees we can move to or no yes so cut of conduct is a very small matter and i don't think it's going to be very controversial either but uh so in the cut of conduct that we have adopted there's a section an incident procedure that basically directs people to report problems of incidents to a mailing list it's conduct at least dot hyperledger.org there is the the Linux foundation is in the process of trying to advocate the adoption of this cut of conduct by other projects so that they would have a standard cut of conduct across the board and uh in in the process of that one of the the project said that they were reviewing our cut of conduct and somebody raised an issue against using a mailing list the reason being that it might actually discourage people from reporting abuse because they don't know what it's being sent to and um what if you know the person you might want to report to is actually on that list for that matter and so i actually check with the DER3C because as you know we basically took the cut of conduct from the DER3C and then adapted it they actually have a much more elaborate incident procedure and i checked they do not use a mailing list for exactly that reason and so the basically what i think you know it makes sense to me and this is clearly something we didn't think about when we adapted the DER3C cut of conduct for us and so what it would require basically is to identify at least two people so that we can name the people give a their email address as opposed to use some kind of opaque mailing list for the incident procedure of course the question is who these people are right so um and Todd helped me out with how this was implemented or not as the case may be but the the mailing list that we have is really just a mailing alias right and it's not like a public transparent list that anybody can subscribe to and people can see what's going on but it's a link to like Brian and Todd and some others that are part of the members of the staff if i if i recall correctly of how we agreed to go off and and do this so it's not really a mailing list it's really a proxy you know an email alias if you will for those that are responsible for following it following through on incidents that potentially violate the code of conduct that people want to report so i mean i think i would agree if we were posting these things to a public mailing list that anybody could subscribe to and then see your name go by saying well that Chris guy is being a real jerk today you know then but again the problem that it's not just that you know other me people may subscribe to the list i understand that the problem has to do with you don't know what is going to specifically so again you just said maybe it's Brian and Todd or you on it i don't know that what if i want to report against you you know i don't know i'm not going to be comfortable sending it to that alias not knowing who it is going to so that's the point that is being brought up so can you hear me yes hi well from linux foundation we for most of our other code of conducts we do identify specific individuals who would be the appropriate contacts we'd like to give out more than one in case you know one of the people listed is actually one of the one of the people that's being you know complained about in this case you know for instance we could put you know a couple people from the linux foundation and one person from the community or two people in the community if it would be helpful but you know the key thing to do is to ensure that we also have a back end sort of procedure that if one of us is contacted you know that there's a routing mechanism to make sure that we handle it in a structured standard way so if you put you know for instance two people from the hyperledric community and two people from the linux foundation if any one of us is contacted you know there should be sort of like a a path you know for what happens once we're contacted that's that's all that my biggest concern just so that one person isn't going off and you know sort of handling this on the run without proper oversight and guidance some of these issues that come up at various points may require legal counsel to get involved there may be you know some challenging conversations to be had and throwing that on to you know random one of four people is not proper but does that make sense so so so Mike you're saying basically that in the others that you've done there are individuals that are identified but it still goes to some sort of an alias is that what I heard no there's no alias that it used at all for instance it'll say contact Angela Brown at linuxfoundation.org or Mike Dolan at mdolan at linuxfoundation.org then doesn't that have the the characteristics that you just described where then somebody's running off and potentially dealing with this without sort of having that you know sort of full process so Angela and I are involved in pretty much any incident that comes up yeah for instance if it came to one of the two of us we know exactly how to handle it if you add you know somebody from the community to that list so it's Angela Mike or you know Dave I don't think there's a Dave on here but I'll just use that random name and it goes to Dave what we wouldn't want is for Dave to go off and try to handle it on his own or set forward it to a mailing list you know we need to make sure that Dave is colluding in terms of you know how to properly handle an issue that's raised that's my only point there okay um I mean I well okay I suspect that's okay except I mean again I'm just sort of if you just want to keep it to the linux foundation staff um Angela I I think that's probably the most appropriate thing and maybe you know if we just list to the the representative is Angela and Mike then you know let's let's do that although again I think administratively if whoever it is the gym science responsibility to is just added to an email alias that actually makes administration that a little bit easier than having to go through and update all these pages I understand that the email alias is an easy way to handle it the challenge is just we get this feedback a lot on projects that have done this is that the person who is potentially submitting something doesn't know who that's going to and is therefore reluctant to submit through that process and then uses some other process and it goes to somebody outside of the chain of normal response well I you know my preference would be then that we just maybe then identify who it is and uh again I don't have a problem I just thought administratively it would make more sense to do an alias but I get I get the point um then I you know I suggest we update this and uh you know if we're going to do this across the Linux Foundation which I think is a great idea um I guess then this it's not really well I guess you know you're gonna have like the the Cloud Foundry which isn't really technically the same as Hyperledger or some of the other collaborative projects because it's its own legal entity and we can clearly have all quite magically updated to to be one. No but I don't think we need to go that far I mean we just need to to have two or three people for our project and when people adopt it they are going to make their own copy and change the names and that's fine. Yeah okay and by the way for reference uh you know the W3C they only list W3C people on that list uh they have a whole bunch of people because they chose different uh like a person for each of their side physical sides but we don't have to do that obviously but uh what I'm saying is I don't think it's a case where we need to have somebody from the community so I think I mean two or three people the Linux Foundation is fine. All right I agree with that. So if everybody agrees I think this is mostly tutorial I can work with Mike and we can add a couple of names there and be done. I think that works and we're over time anyway so I think that's going to have to be there is a resolution here. Okay well good call people um again we are out of time so I just request that the workgroup chairs each send a brief update to the mailing list so that we can keep track of what's going on. It's been a couple of weeks so if you haven't posted one recently it would be appreciated if you could do so and with that I think we'll end the call. Thanks for us. Thanks everyone. Thanks. Thank you. Thanks Richard. Thank you.