Okay, it's recording. So welcome everyone, we'll have a restart of our research and user group finally after quite a long stop. So just the topic today will be HPC and Slam integration and Guard will talk to us a lot about it. So I'll pass it to him quickly but just a few items as I see quite a lot of people joining. If you can add your name to the attendees here in the Google doc that I put in the chat, we'll go through introductions for those that are joining for the first time the group just to know each other. One thing I will just mention before is that we're looking for a new co-lead for the group. Jamie moved to other roles so if you would like to help out it's mostly around keeping a healthy backlog of the topics to be presented and then reaching out to speakers. That's pretty much it. So if you would like to help I'm glad to onboard and eventually I would be glad also to delegate to others to take over. So we can start with introductions. I see Dennis, Priyanka you are joining for the first time as well. That's awesome. Now I'm scared because I actually have a slide with Priyanka face. Oh don't worry at all. Nice to be here folks. Thanks for letting me hang out. Awesome. Then we have Dennis first time. Yes, hello. It's the sound working probably, hopefully. Yeah, you're from where? From Finland, from Alt University or my employer. Doing my master's thesis studies working with HBC Sloan integration. Amazing. Thank you. We have Nikos, you presented but yeah as an attendee do you want to say something? You have your sound. I see his microphone is green but we don't hear him. Like we can come back to him and then Raj. Let's test. We can hear you. Perfect. I had to configure my microphone. Yeah, so this is my second time joining. Well, technically it's the first time I joined was about two years ago when I presented Containers Assets, my project in the CMCF to the user group. And yeah, I will concern alongside Ulfric Argo. Awesome. And Raj will check his microphone. I don't see him in the list. Can come back to it often. Anyone else first time? Davide, you want to introduce yourself? Hi, Ruin. Davide Pastorino from Italy here. We are here with two of my colleagues because we were curious about the Kubernetes and Slurm integration talk. As we are a company that specializes in HPC based in Italy and I am a solution architect. So I'm very interested in integration with Kubernetes as well. Awesome. Pierce, I see a few people I think first time as well. Take the chance. He has no microphone but he's okay. But Pierce is from the SKA office and he was also in our ad user booth at the last CookCon. He was stuffing the booth as well. And I think, did I miss someone? Vitsa? Yeah, Vitsa Albus also the first time here working for SIRV. That's the Dutch NRIN in the multicloud group. So we are developing, yeah, Kubernetes clusters and we are looking how to integrate it with HPC facilities. So that's the reason to be here. Awesome. You're at the right place. All right, Diego. We have two deals with Diego Bakken. Hello, I'm Diego Bakken. I'm working with David and with System and I'm a solution architect in HPC. And I'm here to listen to talk about the Kubernetes and Slurm integration. Awesome. And Matteo? Hello, everybody. I'm also here to listen to the talk. I'm a colleague with David and Diego and with System. It's a company based in Italy. Amazing. Thank you. Awesome. Thank you, everyone, for joining today. We'll have a few of these sessions coming. So this will be pretty interesting. I will stop sharing and I guess Eduardo, you can start again for giving us this introduction. Anytime. So thank you, Ricardo, for inviting me. This is kind of like a post-CubeCon presentation. I had the opportunity to share this idea with Ricardo. Someone that is here, I see on that a participant, but it's not on the Google Doc, is Andrzej Beltre. So I have been working with Andrzej. He works at Sandia National Labs in North America and the US. And we have been brainstorming this idea on how can we better integrate that old-school HPC systems and the new cloud-native. So I call this presentation Cloud-native Slurm. And it's our point of view, the way that we see Slurm integrating into cloud-native ecosystems, different from current implementations that are out there. This is kind of like my take on it. Who am I? I have quite a bit of HPC experience. I started doing containers for HPC at Scylas for those who know singularity containers. So I was kind of like the initial contributors there. Then I moved it to OpenShift, trying to get OpenShift to work for HPC environments. And now I work at NVIDIA doing similar what I did at OpenShift, but now for focused on accelerators. So I'm now focused on helping people to get their GPUs working in Kubernetes and not just working, but at the scheduling level, right? Like how to optimize resource consumption for me. So that's kind of like a bit of background of who am I and why I'm very passionate about this topic. So I will carry an agenda on what efforts do we see in the Kubernetes ecosystem? What is Slurm? What is KCP? And then the final idea. And I will open the room for some Q&A. So, Kubernetes HPC efforts, right? Sometimes it's feel like this, right? Like, Kubernetes is awesome, but we are somehow trying to use it for things that it was not built to, right? So there's a lot happening to make Kubernetes work for AIML, for these big distributed workloads. So then we get into this ecosystem. And this is like the perfect way to talk about this, right? Like, we have right now for those who are signing to the mailing list, we have two new working groups that are visit, that is the working group serving and the working group accelerator management. And to me, this is all part of the same, right? Like, there is a growing need to make Client Native better for HPC in all sorts of ways. AIML, that is like the busing word right now, but like we have here Ricardo that he's from CERN. So like we want to use Kubernetes for scientific real purposes, right? Like curing cancer or looking at the stars or many other things we are getting to do with Kubernetes, but we are realizing that your net was not designed for that. And that we need to to your net is to get it to do what we want. But to me, it's surprising, like why putting so much effort into twisting your net is to do something. If there are other tools that already do it. So it's because your net is provides that developer friendliness, right? So that's where we are here. And we want to move everything into a cloud native way of doing things. There are some good things that have come out of these efforts like this working like this research usage group, or like the working group batch, new tools like Q, like the job set patterns like the cellular plugin that allows you to kind of like inter interject into cure net is way of thinking projects like the flux framework operator. But all of them fall short on what a real HPC system is today, right? So if we compare Q to what a slurm is or to a like Torque or ACC Condor, those tools are being tested and proven for like these high scale systems on where cure net is still growing together. Another thing that I want to point out is that Q the job set, they are pre scheduling tools, right? Like Q and the job set help you to optimize on a pre scheduling decision. Once Q releases the job, right? For those like I'm not going to get into much of how Q works. But basically Q poses a job. And then it on poses the job on some things have been checked out. And then it handles the job to the cure net is scattered, right? And what is once the job is on cure net is scattered hands, it's like we cannot do much on it. There is a scheduler plugin. But the thing that I don't like about the scheduler plugin for cure net is is that do you end up in a situation on where you have to synchronize to a scheduler's and to a states of the word, right? You have to be moving information from pure net is to whatever you have in your escalator plugin. So your scheduler or decision management is aware of how the system is going, like if which nodes are super utilized, which nodes are not that much utilized. So the scheduling decision is optimized, right? So like, that's why I don't like the scheduler plugin, because you land into these synchronizing to a scheduler's and to a state managers at the same time, and that can get into a lot of errors, because I've been there. And that also can get you in a not optimal situation when you want to do high performance scheduling, right? As a community, we also have that was not good. We also have the MPI operator that is a great tool that it's was developed for Qflow. And now it's kind of like being proposed as a standalone tool, because it is very important for AI ML training, mostly when we are doing this distributed training, or for any scientific tools I have seen, even people trying to do like bioinformatics in on top of Kubernetes using the MPI operator. So a quick introduction to Slurm and now, Hey, Priyanka. So sadly, the YouTube recording of Priyanka saying that she loved Slurm was cut. So I kind of like hack it here. But it's really amazing when when Priyanka goes and mentions that Slurm plus Kubernetes is like the drive of doing things, not like one on top of each other or like those weird mixes that we have seen some projects out there. But trying to get them to live side by side is a good approach, right? And this just happened two weeks ago during a cube con keynote, right? Like something that the HPC ecosystem was not even aware of, right? Like, I was chatting with someone from skmd, a Slurm company. I was like, Hey, guys, your logo was at the keynote, and they were not even aware of that, right? Like, this is fresh. So we as a cloud native community, we are realizing that we need this and we need a community solution on how can we use a Slurm when we need it. And for the things that Slurm has been designed to do for over 20 years, and then fall back to Kubernetes for everything else. But it's not just like falling back and killing a Slurm and turning it on and off every time we need it. But a way that we can use a Slurm as another tool, right? Like for those that have seen the cloud native landscape is a growing sheet that at some point, I think they are going to start selling the actual bed sheets for everyone, because it's so huge. But what is a Slurm for those that know the cure net is architecture, if you really look this picture like from really, really, really far away, it's kind of the same, right? Like is the control plane is the nodes and is the API server kind of, right? So we have the Slurm D in this picture, the yellow part will be to the cure net is people will be kind of like the API server, right? Like is the most important thing where all all the other components kind of like reached back to. And we have the Slurm DVD that is the database that to cure net is ecosystem people will be easy, right? Like this would be kind of like the easy where all the state is managed and is stored there. And we have the the demons and we have the nodes and we have like all sorts of CLI tools for interacting with a Slurm, depending on the things that you are doing, right? Like a Slurm has this very optimized way of like, if you're running just a batch job, or you're just doing like a s run, or if you're doing a MPI job, or you're just queuing or you're checking how the queue is going, they have different CLI tool for it on where in the cure net is ecosystem, we have Qtl for kind of like everything, right? We have this one tool to interact with everything. And it's one of the things that we like from cure net is like this single API to manage them all. But why is Slurm is so awesome as Slurm supports PMIS as Slurm supports MPI on this largest HPC cluster like these things that we read on the news that they win awards and that it requires like a full city electricity for a day, just to keep these systems running versus on cure net is we have never tried kind of like a system that big, right? Slurm supports containers pretty well, right? So you can run from singularity and other HPC based containers. It also supports regular Docker and Podman, what we end the cloud native ecosystem are used to, right? What got me into this idea is learn recently when I said recently like in the last couple of years, not like so fresh, they introduce it as learn rest API. So as learn rest API is a open API compatible 3.0 compatible is a stateless. So it's just like basically, you do a request and it throws you like what happened to the request, but it's not like storing a cache or saving anything that is happening during the API calls. It has two modes in it. Basically, you can talk to the run as learn rest binary and get it to do things. Or you can do it as a unique socket or a TCP that is constantly listening from from calls on a specific port that this is the part where I want to focus on. That's why it's highlighted here on the security side. It supports JSON web tokens for notifications. And you can have like multiple users which two communities will be kind of like map it to what we have as service accounts, right? So this is where we are already starting to see that we can have some mapping between what is learn can provide the rest API. That is what we like at the cloud native ecosystem by like doing everything via APIs. And how can we from a cloud native ecosystem that we like API, we can start calling as learn using the API, right? And we have this really nice tool that we as a community have been building that is called KCP. A project that was born pretty much during the pandemic because it was born like during the 2020-2021. And it's a cure net is like contraplane without notes or pods, right? So it's a very interesting picture. This was one of the talks that they gave in the last KubeCon on where KCP is basically a way to have a uniform API to communicate with any platform, right? Like you have a platform that you want to expose via an API and an API that we all know that is the cure net is API using kind and using Jamel because we all love to have our laptop full with Jamel. So KCP help us do that. Like it help us put a API from like from facing to any platform you want, right? But how does that look like, right? It's basically a very stripped down version of cure net is is a contraplane with API server, control manager, because the control manager is pretty much the secret source or the magic behind cure net is what we all love to build operators and controllers and this reconcile loop is what allow us to do that. It has it see and we can then build custom controllers just utilizing this like this is a high level view of what KCP is cure net is without pods without containers without notes.   But if you do cube CTL get notes, it will fail. Like it will actually throw an error. If you do QCTL get pods, it will throw you an error. But it actually knows about secrets or back like anything like role, role binding, all of that is still there. It has namespaces and it has the most magical thing, custom resource definitions on which we can build controllers and we can then get crazy doing whatever we want. Now, how do we get to cloud net tips learn? So after I show it Homer in the first slides is like after breaking everything trying to make cure net is be an HPC platform. Why not using these tools that we as a community in the cloud native have built to make a slurm interact with cure net is right. So now we are basically putting a translator next to a slurm so a slurm can sit down at the same table with cure net is and they can share information, but they sit on different sides of the table. Like they are different clusters. They don't have to manage or synchronize a state of the cluster because they are their own clusters. And then you can run a slurm optimized for what it is optimized. And then you can have cure net excited using tools like that's why I put here infrastructure. There are multiple tools out there from in the HPC side of the house that we have werewolf or we have what used to be bright computing that's now inside of NVIDIA or many other tools that they can help you pre provision and out very quickly to a cloud provisioner like you can just move AWS Google or Azure machine to be one type of node to another type of node, right? So you can have two clusters and just like change nodes.  And the good thing is that you can send the entire Jamel as we right now know for the MPI operator that we have already defined MPI job when using the MPI operator, you could just throw the entire Jamel to the slurm KCP cluster and we will have a controller that will translate that into a regular slurm job using the slurm API, right? Like there is no need for extra complicated things or like cure net is managing pods of slurm and trying to synchronize the state of the word, but it's a cure, it's a standalone slurm cluster that understands cure net is Jamel, right? So it's a uniform API because we can have a single control plane. It will enhance developer interaction because we will just do Jamel as we like to do Jamel. It is multi-Q compatible. Right now the Q project is exploring this multi-Q option on where you have a small cluster as we see here on the slides and Q is basically deploying jobs to all the clusters and then on the other side the slurm KCP will get the job and if the job is compatible it will run it and it will communicate back to Q that the job was accepted, right? So that's how multi-Q works. It will query each cluster that it has on this list asking like can you run this job? Can you run this job? And we will tell multi-Q like yeah, I can run the job because a slurm will now understand Jamel, right? And it's cloud and on-prem compatible. We can still have nodes and jobs concept, right? Because a slurm on its API it provides a way to query a slurm about the state of the nodes. So we can communicate that back to a central control as or another talk that was really nice not to like self-promote myself but we as a community are promoting a cluster inventory API that will live in this high-level control plane and then we could communicate to this cluster inventory API how my slurm cluster is going, how many nodes do I have, how many nodes are healthy, the CPU utilization, all of that in a way that other Kubernetes components understand. So we can just translate from the cluster out and we don't need to be complicating things synchronizing the state of the word between a slurm scheduler and Kubernetes scheduler. How it would look like internally once we communicate with KCP that is going to be the the front-facing control plane KCP will have a controller running that will translate all the Jamel into a slurm API. Once everything is translated into a slurm API it becomes a regular slurm workflow on where a slurm is going to communicate with a slurm internal command controllers and it will deploy the jobs to the compute nodes. So it will basically look kind of as we know Kubernetes architecture but is now a slurm and we can communicate with a slurm via Jam. The roadmap for this project the proof of concept I think is the same with public repo I think we will be open sourcing it in a couple of months since my friend Angel that I think is still here is from a national laboratory we have to request some permissions and things like that because I think San Diego is like a weapons lab or something like that politics that I don't want to get everyone into it but we are requesting for all the required permissions and things like that to open source it and we will be open sourcing the proof of concept really soon so everyone can run it can run a demo and we plan to have a paper submission for the super computing conference in KubeCon North America in November so that's kind of like now my idea on how this is going and now I open two questions. Awesome thanks Eduardo. I think there will be quite a few questions feel free to drop them in the chat or raise your hands and I will take notes in the minutes as well. Anyone want to start? I will start by apologizing with Priyanka I had you in the slides. Oh don't apologize I was honored I was not expecting that. Any question to start? Otherwise I can ask one and then we hi Giuseppe. Yes hello everybody I'm Giuseppe from SURF. SURF is the Dutch National kind of IT research center and I'm the program manager for for cloud and edge technology now after this introduction we are trying to solve a problem basically our problem is how do we fractionalize our supercomputer how Kubernetes can help doing that so one of the one of the the angle that I would try to understand in this project is what problems are we trying to to solve because I might see the problems for our research community and I would like to to understand when you approach this project what was the problem that you wanted to solve maybe it's a bit of a generic question but I want to see if it's aligned with our strategy and agenda. Yeah I can start with that again I think I'm going to waste his name so Angel I see him here in the list he's in a laboratory where like a 100,000 nodes is just like the beginning of a research so Kubernetes is still not made for that and they want to run these huge shampy jobs or these huge deployments and they want to also use these new technologies like PMIX that is still not supported by the MPI operator and that kind of like force them to still use Slearn but all their developers and the new researchers they want to use Kubernetes and they are also running Kubernetes internally for some like CICD testing and like minor things right but the new researchers and new developers they want to use Kubernetes and they know how to use Kubernetes but then they face that their cluster where they are going to run their actual research is this big all thing not saying that Slammer is all is that that to the new researchers the new people it looks like all right so what if we can have the best of both worlds rather what if we can give this new wave of researchers new wave of developers a front-facing interface that they know that they are used to managing the Jamel and then we the upside of the house we manage things right like we have these interfaces where we translate Jamel into a Slearn native objects or a Slearn API right so the the need that got us to this idea is that we have these clusters that have been running a Slearn for years and they are not going away and they cannot be reprobation into Kubernetes clusters in the next 10 years or so right like this is these are like long running projects so it's like how can we interact with that Slearn cluster because we know we cannot reprobition it but how can we give it like a fresh look with a cloud native tool yeah thanks thanks for the answer I just want to add one last thing and then I'll shut up it's it's great because I think we are we are aligned also from our perspective as a search but I have another another follow-up question one of the I am a Kubernetes guy by you know by you know the uh background but we always have lots of cultural problems when we discuss Slearn and HPC in the community of HPC is very let's say conservative and from our point of view it looks like we are trying to explain why things could be better with the Kubernetes architecture but I don't know for you guys but for us it's kind of going to help heal cultural bubble and if you have any any any tip or any point of view on that we will we can gather our forces together you know to be more in change yes yeah I think I can start but Ricardo or Priyanka can't follow here I used to be a hard on like Kubernetes everything person for quite a few years but and even that I come from like a old school HPC right but now I understand that we have tools for everything and that's why I said that Homer is light at the beginning where Homer is trying to use a drill for hammering and now we like Slearn is there Slearn is perfect it has been going for 20 years it runs on like top supercomputers why we want to twist your net is to do what Slearn already does we like at the cloud native because it's same we are already used to having this huge landscape of like there are thousands of projects which you just choose what like you have a need and you go and pick from the landscape and you use a tool to solve your need right like we don't twist the existing solution to solve a problem we just go and choose a tool why don't we just add Slearn to this big cloud native landscape Slearn logo should be there right so instead of trying to twist many tools and cure net is to do what Slearn is doing is just like get Slearn into the cloud native landscape big sheet and then away for cure net is and all the other tools to interact with Slearn then Slearn becomes just another tool on on the ecosystem right like it's just a tool so that that's my point of view with this project right like I want to get Slearn to be in the cloud native landscape and to be seen as just another tool right like Slearn means a tool because inside Slearn you also use other tools like MPI MPMIX they are not part of Slearn they are separate projects that you then use so it's like that's my point of view now yeah Priyanka do you want to add something yes I'll just I'll say that cultural challenges are very real the good news though is that we have all been battling them in different areas for so many years now when Kubernetes was open sourced in 2014 the whole question was oh developers are separate from operations right and the whole shift to the dev ops mindset or the SRE mindset where engineers were also in operations took a while and a lot of folks on the on both sides whether it was the sysadmin or the business logic developers were resistant so I would guess that the kind of resistance you're facing to set is with your researchers is similar to that and at least based on past experience what I can say is that as more and more new researchers developers start going into this field and as Eduardo was saying they are more Kubernetes native and they want to use it the slurm folks will slowly see that oh this expands my influence this expands my impact and there will always be a small contingent that will never want to be at all involved and that's fine that's their choice to make and you cannot convert everyone but it will be a slow and steady trickle I would say so my advice is just keep at it keep presenting stories of success of how new use cases have been developed by using these technologies and everybody likes to have success and they will follow along and be patient because it'll take a while thank you thank you it's nice to not be alone in this adventure thank you yeah I don't think you're definitely not you're definitely not alone we have a similar experience internally we have new workloads that got used to having Argo workflows and using all the tooling on top of Kubernetes that makes things much easier GitOps kind of deployments and now we face the need to access a lot of the resources that everyone wants like GPUs that are backed by supercomputers and slurm clusters and we have to find a way to bridge between the two internally we have the same issue Dennis you have your hands go ahead yes this is also quite relevant to my work so at Alt University I'm working closely with the Finnish Center for Scientific Computing so CSC specifically on the Ruby supercomputer on how to get kind of the Kubernetes integration going there because we're we're facing a really interesting challenge is that most of the the code that is being executed on Lumi right now is very GPU heavy so all of the GPU time is basically utilized and we are mostly idling the very powerful CPUs that that machine also has and people are really interested in finding solutions on how to utilize those CPU side resources while the GPUs are doing are busy doing simulation work or LLM so or whatever you have and for the CPU side people really want to do dynamic bursty workloads and maybe something really complex like running a Kafka and Spark or something like that in parallel and for this purpose we have this similar conflict that that the researchers are really interested in in I brought up Kubernetes as sort of a way to deploy these systems very easily however the actually getting Kubernetes and this ecosystem running on that supercomputer I've gone back and forth with CSE quite a lot and there's this resistance in the mindset and they are stuck with their with their software visions and and what they can support on those nodes so having something like KCP or interlink which will be the topic of our future talk in this in this use group as well will be sort of a key enabler of seeing that Kubernetes success so my my goal is essentially to provide an intermediate a kind of bridge to bridge this ASPC world and the Kubernetes space before we can eventually then maybe just run playing Kubernetes on the on the HPC nodes all together and skip all of the intermediate parts but something kind of we need an intermediate layer to to like Priyanka mentioned to show this success and be able to evaluate it in the current context and this is kind of a really important focus of my work at the moment excellent this is a very good discussion any other question I see a couple on the on the chat but feel free to raise your hand if you want to ask them directly if not I can ask are you by Brian do you want to go ahead you have your hand hand raised or Sanjay yeah okay can you hear me yeah I can yeah okay sorry I just unmute so yeah very good talk so I hear some keyword here it's that we try to to introduce Kubernetes on HPC or supercomputer or HPC data center or on-premise but actually I come from a the Kubernetes world and we try to benefit from all this we try to find the HPC capability within the Kubernetes actually we are using manage the Kubernetes like AKS, GKE so for me there are two major issues today one of these we try to utilize the unlimited cloud resource so which means that we rely on the cluster auto scheduling feature so which means we need the computing and we can quickly provisioning the environment and deploy the image and run our computing workload there and when we finish we just decommission all the environment so I'm not sure for this by introduce slurm integration will that help on this direction or because this is really our bottom neck today like how fast we can having a 500 node HPC cluster being be ready on this on this cloud I'm not sure if this is for this on-demanding or auto-skidding direction where we talk about really for a traditional supercomputer or the the data center environment Eduardo do you want to answer? Yeah so yeah if if you're a cloud native person because that's is like this slurm conversation is mostly for as others have jumped to say like they are from a university or a research environment where they have this on-prem not a cloud on-prem cluster as like thousands of nodes and it has like a predetermined software installation and it has to be running for the next 10 years because it's under contract and things like that so it's like it's really hard to move from a slurm to another thing right so that's that's why people like Denny's or I forgot the first question name we want to have a way to interact with a slurm from cloud native tools right like from Kubernetes or other cloud native tools but if you are just on on the clouds and you have you want to have a 500 nodes slurm system I think you can get those from Google and Azure I seen some products from Google and Azure where you can get like as slurm clusters on demand and I mean if at some point you want to have a Kubernetes cluster and as a slurm cluster all in the cloud running and you want to move information to one another you are going to face the situation we are facing like in today's presentation that is how how I make these two ecosystems interact right so that's how I see you being able to utilize as slurm right like going to this cloud hosted slurm yeah yes thanks for the explanation I think the so when we I think in the last five year when the CPU time we we maybe we can do this on demand cloud but when the GPU come into this AI journey the GPU is really a limited resource probably this will do more help for the even via on cloud we cannot really really get GPU sometime really just on demand we need to reserve the GPU for like one year or three year contract to benefit I think for that situation the cloud native more turn into a similar to this supercomputer or on-premise situation so I see the value for this project can extend it to the cloud environment yeah thank you awesome so I see a couple of hands raised Sanjay is the next yeah Sanjay yeah hey can you hear me yes oh all right nice presentation Edward so couple of questions one is uh when you talk about moving nodes between clusters how do you reason about which nodes to move and like if you have to do like things like you have to pre-empt some workloads or things like that how do you reason about those things and the other thing is that typically slurm clusters they have a way of operating where you you have multiple users they get allocations and let's say node hours or things like that and the scheduler then tries to do fair share scheduling and to manage you know the resources allocated to these users and so these are typically set up for certain windows how how would that have like get managed if you suddenly take away some resources from the slurm cluster to the Kubernetes cluster so it seems like there needs to be a little bit of you know either some resource management layer to this well as slurm itself is a resource manager I I see team on the room team could you help me with the second one I think you are the best person to answer this one Tim is saying that his microphone is not working but maybe the the messenger ah it does go ahead go ahead go go go sorry I was trying to fix the mic problem so what was the question again oh I'm sorry I was referring to Tim Winkberg but oh okay yeah yeah the other team in the room but yeah actually so I see a couple of a slurm developers here in the in in the room but for the first question is ah how do you reason about oh no microphones ah we need to invest in microphones around here ah how do we reason about the resource management between the two clusters so that's why ah on the last slide I had a picture on like a central ah control plan rally multi-cluster ecosystems usually what you have is like a cluster that you call like a management cluster and it's a cluster with very few nodes compute nodes is a cluster just made for running controllers where you reason about resources right so ah we are discussing and it was discussed during this cube con a couple of APIs to be able to have a cluster inventory of all the clusters that you have in your pool and then how to make decisions out of these inventories right so that's how we will be done right like this is an ongoing effort and ah we are working on that as as a community right like there is a couple of caps open about how to do cluster inventory and we are working towards that the cluster inventory kept just got merged and we are working for ah the alpha to get into the next urinary release so that's that's how we are trying to solve that problem on how do you reason on resource management management between multiple clusters then how do you take a note from a running a slurm cluster and add it to a cure net is cluster how is lorm is going to ah react to that ah that I think that's a follow-up question more suited for like like a it's lorm ah admins that to myself yeah maybe maybe just a to clarify the integration that Eduardo is describing talks to the slurm API so there there shouldn't be a need to reallocate resources right if you have your slurm cluster it stays there and you just submit things via Kubernetes but you don't have to reallocate notes was that correct Eduardo yeah no ah as I showed in in one of the lines show that you can use tools like a werewolf or ah ah bright computing where you can read quickly reprobition a note like change the OS and things like that right so like in an scenario if you have a bare metal you don't have any access to the cloud right like sometimes like universities ah you have a fixed number of notes sometimes you want to reprobition these notes from being Kubernetes to being a slurm very quickly like in less than a minute you can do there are tools to do that so ah ideally you can like have as elastic Kubernetes as lorm situation where you have fixed one hundred notes but you can at some point have a bigger Kubernetes partition or then have a bigger is learn partition in this fixed one hundred note situation okay thank you so we can move to the next one which I believe is Nikos yeah hello can you hear me again yep go ahead yeah okay so this may be a bit on the other side a bit more provocative but so from my perspective I am on the complete opposite end of most people here we are mostly working on HPC's on the computer center side I used to work for CMS which is the virtual organization that actually submits jobs to the to the HPC's to supercomputers so for CMS we're using it's the condor and then from it's the condor we had gap to do a translation to slurm and now this adds another layer to from slurm to go to Kubernetes so my question is now we're three layers deep has there been any proposal or hasn't has people looked into submitting jobs directly to Kubernetes or at least having an entry point like native to Kubernetes that is can coexist with a slurm cluster without going through this many translation layers yeah I can I can reply quickly to that one so there are a couple of projects there's a batch working group I think Victoria is here so if she can jump in she's the best one to talk about this there are two working groups here by the way right yeah no but I mean I mean for the native Kubernetes which was the question there's this batch working group that has been working on this project Q and extensions to jobs and job sets that aim at exactly that making Kubernetes native as a scheduler to to this kind of workloads and then all the other tools can rely on these primitives is Victoria still here Victoria is here yeah I am here but I don't know anything more about it okay okay all right Alex do you want to add something yeah maybe just a PSA on those two working groups they were mentioned at the beginning of Eduardo's talks but there seem to be a lot of people here who are very interested in this topic and I lead one of those conversations the batch working group for the CNCF where this kind of conversation is exactly the conversation we have every time so if you're interested in this kind of conversation Eduardo it might be great to have you come and talk in that group as well but yeah just to let people know there are more conversations than just this one going on about this very thing um please come out I'll hit a a link down in the in the chat if anyone's interested um the one that I run is separate from the one that Victoria is involved with more which is as people have been saying just directly related to Q and all that kind of stuff I had a question about slunk and if anyone has seen it or heard from it recently and how that compares and my understanding is that has a bit more of the addressing some of the questions in the chat about balancing slurm resources and kubernetes resources within the same cluster whereas I think here you've described more just sort of static clusters one is slurm one is kubernetes stuff I think slunk is a bit more integrated do you have any feelings on that yeah that's why I made a lot of point on that slide on where you run into synchronization problems right like two three years ago I was part of a project where we were trying to use the a scheduler plugin to change the regular scheduling for kubernetes to flux that is a a lever more base scheduler that is kind of like similar to slurm and the thing is it works when you are talking like a hundred nodes or down when you start having like a thousand nodes or more than a thousand nodes then you end up in this problem where you have this baby scheduler living inside kubernetes and kubernetes has to be telling this baby scheduler all the time how all the nodes are going right so that's why I was I'm making a point where the synchronization starts to get very complex right like you need to be secure you need to have a controller that is basically getting the information from all the nodes in kubernetes and the kubernetes as it is right now is not the best hpc node based demon for decision making like it's learn right like it is it won't feed us learn with the right information to make optimize scheduling decisions so projects like as long I don't agree with them on where like yeah they will work on as small clusters but again when you start talking about these big clusters like thousands of nodes you will run into a state synchronization issues because at some point is learn that is running on pods it's not going to know the state of the word as kubernetes knows it right so then you are going to deploy a job as learn will think that the left side of the cluster is busy and the right side is is is available but it I just asked just too complicated of a question I suppose I I'm sorry it's all my fault yeah I had mic issues earlier and it was the platform so maybe something's going on in the background talking about optimized scheduling but like high performance computing you will start running into situations right so that's why I feel like as long should be that's the way I said at the beginning this is my take on this situation as long should live standalone install it as it is recommended and we just interact with it right like we just interact with this learn and we can then interact with this learn and make it be elastic with these tools as I've been mentioning but not having as learned to like having any scheduler leaving inside another scheduler is a situation you don't want to leave in okay so out of interest of time because we only have like one minute and a half left apologies I think that will thank Eduardo a lot for the presentation and also everyone for attending this was a very very well attended and active discussion so clearly we have to continue we have another meeting on the same topic scheduled for two weeks from now Diego I see you're here so you're all good to present in two weeks yep yep so Diego also gave a presentation at Cook on two weeks ago and we can hear about what he's been working on with interlink what we could also do if you want because everyone is here if you want to present on topic on any subject reach out and we can schedule we probably would be useful also to get an update on Q and what they've been working on around jobs and job sets at some point so I'll bring that one up and put it in the schedule at some point all sounds good it looks like the Slurm folks are working on something similar to sunk if I'm reading the chat correctly so just wanted to suggest maybe if they ever want to present Nathan you're here is Nathan's mic not working you hear me yeah yeah we can we might just I don't have we don't have anything planned right now so I'll get back to you awesome yeah happy to follow follow it up as well and other than that I think thank you so much everyone for attending thanks Priyanka for for attending as well and yeah talking two weeks and let's follow it up on on Slack as well thank you thank you thanks everyone thank you Ricardo for hosting thank you bye everyone bye bye


Hello, and welcome to this presentation of the STM32L4 Embedded Operational Amplifier. It covers the features of this IP, which is widely used for conditioning analog signals. Each operational amplifier inside STM32L4 products functions as a general-purpose analog amplifier, which may reduce the need for an external standalone op amp. As these op amps can be configured in standalone mode with all terminals available for the user, it is possible to use them as a voltage follower, non-inverting and inverting amplifiers, as well as analog filters, such as low or high-pass filters. They can also act as a pre-amplifier for the ADC input. The operational amplifier inside STM32L4 products offers general-purpose rail-to-rail inputs and outputs. The input offset voltage can be calibrated in the application to achieve minimal offset. All the terminals are accessible by the user so that any operational amplifier function can be configured with external passive components. For low-power applications, which do not need high bandwidth, a low-power mode is available, which only consumes 45 microamps from VDDA. For current sense applications, a dedicated low leakage input pad is available on the BGA-132 package. The op amps also provide on-chip functions, including voltage follower mode, while their on-dive feedback resistance can be used for the programmable gain amplifier function. This slide shows the block diagram of the operational amplifiers. The STM32L4 integrates two operational amplifiers. Several switches are used to configure different functions. Each op amp can be configured differently. The operational amplifier can be configured as an internal voltage follower, where the internal switch connects the output to the inverting input. In internal voltage follower mode, the GPIO pad assigned to the op amp's inverting input is free, so that it may be used with a different function as subsequently assigned in the GPIO selection process. This STM32L4's op amps support non-inverting amplifier mode with four fixed gains available. The feedback resistors can be selected to have a gain of 2, 4, 8, or 16 to support the programmable gain amplifier function. It may also be beneficial to use the op amp as an ADC input so that the full dynamic range of the analog to digital converter can be applied to signals having a small amplitude. The op amps also support external connections on outputs and the non-inverting input to create the PGA mode. It enables the connection of an external capacitor to add low pass filter characteristics to the PGA configuration. The op amps have a very low leakage inverting input pin to support the current sensing application. The dedicated pads are only available on the BGA132 and BGA169 packages and have one nanoamp leakage which allows for a current to voltage conversion with high precision. The operational amplifiers are active in the following low power modes. Run, sleep, low power run, low power sleep, stop 0, and stop 1 modes. In stop 2 mode, the op amp is not available, but its registers content is kept. In standby or shutdown modes, the op amp is powered down and must be reinitialized for use if returning to one of the higher powered modes. The following table shows performance parameters for the STM32L4's op amp. The op amp can work from 1.8 to 3.6 volts from the VDDA supply with rail to rail input and output. The offset voltage can be calibrated down to 1.5 millivolts. It has normal and low power operating modes. Normal mode can have a gain bandwidth of 1.6 MHz while low power mode only has 0.42 MHz of gain bandwidth but comes with the added benefit of reducing power consumption. Thanks to its fast wake up time of 10 to 30 microseconds, it is not needed to keep it always enabled but only when the functionality is necessary. The STM32L4's op amps are suitable for the pre-amplification of the ADC input because the integrated PGA can enhance the dynamic range of the analog to digital converter. It is handy to use as a non-chip voltage follower instead of using an off-chip standalone op amp. Its low bias current allows it to properly perform as a current to voltage converter for the current sensing function. This is a list of peripherals related to the operational amplifiers. Please refer to these peripheral trainings for more information if needed.


CHAPTER VIII. The composition and food value of cocoa and chocolate. The early writers on chocolate generally became lyrical when they wrote of its value as a food. Thus, in the natural history of chocolate by R. Brooks, 1730, we read that an ounce of chocolate contains as much nourishment as a pound of beef, that a woman and a child, or even a counselor, lived on chocolate alone for a long period, and further, before chocolate was known in Europe, good old wine was called the milk of old men. But this title is now applied with greater reason to chocolate, since its use has become so common that it has been perceived that chocolate is, with respect to them, what milk is to infants. A more temperate zone is shown in the following from a curious treatise of the nature and quality of chocolate by Antonio Colmenero de Ledezma, a Spaniard, physician and chirurgian of the city of Equija in Andalusia, printed at the Green Dragon, 1685. Quote, So great is the number of those persons who at present do drink of chocolate that not only in the West Indies, once this drink has its original beginning, but also in Spain, Italy, Flanders, etc. It is very much used, and especially in the court of the king of Spain, where the great ladies drink it in a morning before they rise out of their beds, and lately much used in England as diet and physics with the gentry. Yet there are several persons that stand in doubt, both of the hurt and of the benefit which proceeds from the use thereof. Some saying that it obstructs and causes opalations. Others, and those the most part, that it fattens. Several assures us that it fortifies the stomach. Some, again, that it heats and inflames the body. But very many steadfastly affirm that though they should drink it at all hours, that even in the dog days, they find themselves very well after it. End quote. So much for the old valuations. Let us now attempt, by modern methods, to estimate the food value of cacao and its preparations. Food value of cacao beans. In estimating the worth of a food, it is usual to compare the fuel values. This peculiar method is adopted because the most important requirement in nutrition is that of giving energy for the work of the body, and a food may be thought of as being burnt up, oxidized, in the human machine in the production of heat and energy. The various food constituents serve in varying degrees as fuel to produce energy, and hence to judge of the food value, it is necessary to know the chemical composition. Below we give the average composition of cacao beans and the fuel value calculated from these figures. Average composition and fuel value of freshly roasted cacao beans, nibs. Cacao butter. Composition 54.0 equals energy giving power 2,282 calories per pound. Protein, total nitrogen 2.3%. Composition 11.9 equals energy giving power 221 calories per pound. Cacao starch. Composition 6.7, energy giving power 472 calories per pound. Other digestible carbohydrates, et cetera. Composition 18.7 equals energy giving power 472 calories per pound. Stimulants. Theobromine. Composition 1.0. Caffeine. Composition 0.4. Mineral matter. Composition 3.2. Crude fiber. Composition 2.6. Moisture. Composition 2.6. Moisture composition 1.5. Total composition 100. Total energy giving power 2,975 calories per pound. It will be seen from the above analysis that the cacao bean is rich in fats, carbohydrates and protein, and that it contains small quantities of the two stimulants, theobromine and caffeine. In the whole range of animal and vegetable foodstuffs, there are only one or two which exceed it in energy giving power. If expressed in quite another way, namely as food units, the value of the cacao bean stands equally high, as is shown by the following figures taking from Smitham's result published in the Journal of the Royal Agricultural Society 1914. Food units. Turnips. 8. Carrots. 12. Potatoes. 26. Rice. 102. Cornflour. 104. Wheat. 106. Peas. 113. Oatmeal. 117. Coconut. 159. Cacao bean. 183. These figures indicate the high food value of the raw material. We will now proceed to consider the various products which are obtained from it. Food value of cocoa. Average composition and fuel value of untreated cocoa. Cacao butter. Composition 28.0 equals energy giving power 1183 calories per pound. Protein. Composition 18.3 equals energy giving power 340 calories per pound. Cacao starch. Composition 10.2. Energy giving power 718 calories per pound. Other digestible carbohydrates, etc. Composition 28.4 equals energy giving power 718 calories per pound. Stimulants. Theobromine. Composition 1.5 equals energy caffeine. Composition 0.6. Mineral matter. Composition 5.0. Crude fiber. Composition 4.0. Moisture. Composition 4.0. Total. Composition 100.0. Total energy giving power 2241 calories per pound. Soyable cocoa, i.e. cocoa which has been treated with alkaline salts, is almost identical in composition, save that the mineral matter is about 7.5%. As cocoa consists of the cacao bean with some of the butter extracted, a process which increases the percentage of the nitrogenous and carbohydrate constituents, it will be evident that the food value of cocoa powder is high and that it is a concentrated foodstuff. In this respect, it differs from tea and coffee which have practically no food value. Each of them, however, have special qualities of their own. Some of the claims made for these beverages are a little remarkable. The Embassy of the United Provinces and their address to the Emperor of China, Leiden 1655, in mentioning the good properties of tea, wrote, More especially, it disintoxicates those that are fuddled, giving them new forces and enabling them to go to it again. The Embassy do not state whether they speak from personal experience, but their admiration for tea is undoubted. Tea, coffee and cocoa are amongst our blessings. Each has its devotees, each has its peculiar delight. Tea makes for cheerfulness, coffee makes for wit and wakefulness, and cocoa relieves the fatigue and giving a comfortable feeling of satisfaction and stability. Of these three drinks, cocoa alone can be considered as a food and, just as there are people whose digestion is deranged by tea and some who sleep not a wink after drinking coffee, so there are some who find cocoa too feeding, especially in the summertime. These sufferers from baleousness will think it curious that cocoa is habitually drunk in many hot climates. Thus, in Spanish-speaking countries, it is the custom for the priest, after saying mass, to take a cup of chocolate. The pure cocoa powder is, as we saw above, a very rich foodstuff, but it must always be remembered that in a pint of cocoa, only a small quantity, about half an ounce, is usually taken. In this connection, the following comparison between tea, coffee and cocoa is not without interest. It is taken from the Farmer's Bulletin 249, an official publication of the United States Department of Agriculture.  These figures place cocoa as a food head and shoulders above tea and coffee. The figures are for the beverages made without the addition of milk and sugar, both of which are almost invariably present. A pint of cocoa made with 1 third milk, half an ounce of cocoa and 1 ounce of sugar would have a fuel value of 320 calories and is therefore equivalent in energy-giving power to a quarter of a pound of beef or four eggs. Cocoa is stimulating, but its action is not so marked as that of tea or coffee, and hence it is more suitable for young children. Dr. Hutchison, an authority on dietetics, writes, Tea and coffee are also harmful to the susceptible nervous system of the child, but cocoa, made with plenty of milk, may be allowed, though it should be regarded, like milk, as a food rather than a beverage properly so-called. How to make a cup of cocoa Tea, coffee, and cocoa are all so easy to make that it is remarkable anyone should fail to prepare them perfectly. Whilst in France, everyone can prepare coffee to perfection and many fail in making a cup of tea. In England, all are adepts in the art of tea making and many do not distinguish themselves in the preparation of coffee. Cocoa in either country is not always the delightful beverage it should be. The directions below, if carefully followed, will be found to give the character of cocoa its full expression. The principal conditions to observe are to avoid iron sauce pans, to use boiling water or milk, to froth the cocoa before serving, and to serve steaming hot and thick cups. The amount of cocoa required for two large breakfast cups, that is one pint, is as much as will go when piled up in a dessert spoon. Take then a heaped dessert spoon full of pure cocoa and mixed dry with one and a half times its bulk of fine sugar. Set this on one side whilst the boiling liquid is prepared. Mix one breakfast cup of water with one breakfast cup of milk and raise to the boil in an enameled saucepan. Whilst this is proceeding, warm the jug which is to hold the cocoa and transfer the dry sugar cocoa mixture to it. Now pour in the boiling milk and water. Transfer back to saucepan and boil for one minute. Whisk vigorously for a quarter of a minute. Serve without delay. By just ability of cocoa. We have noted above the high percentage of nutrients which cocoa contains, and the research conducted by Jay Forrester shows that these nutrients are easily assimilated. Forrester found that the fatty and mineral constituents of cocoa are both completely digested, and the nitrogenous constituents are digested in the same proportion as in finest bread, and more completely than in bread of average quality. One very striking fact was revealed by his researches, namely that the consumption of cocoa increases the digestive power for other foods which are taken at the same time, and that this increase is particularly evident with milk. Dr. R. O. Newman, who fed himself with cocoa preparations for over 12 weeks, whilst not agreeing with this conclusion, states that, the consumption of cocoa from the point of view of health leaves nothing to be desired. The taking of large or small quantities of cocoa, either rich or poor in fat, with or without other food, gave rise to no digestive troubles during the 86 days which formed the duration of the experiments. He considers that cocoa containing a high percentage of cocoa butter are preferable to those which contain low percentages, and that a 30% butter content meets all requirements. It is worthy of note that 28 to 30% is the quantity of butter found in ordinary high class cocoa's. As experts are liable to disagree, and it is almost possible to prove anything by a judicious selection from their writings, it may be well to give an extract from some modern textbook as more nearly expressing the standard opinion of the times. In Second Stage Hygiene by Mr. Eichen and Dr. Leicester, a textbook written for the Board of Education syllabus, we read page 96, in the better cocoa's, the greater part of the fat is removed by heat and pressure. In this form, cocoa may be looked upon as almost an ideal food, as it contains proteins, fats and carbohydrates in roughly the right proportions. Prepared with milk and sugar, it forms a highly nutritious and valuable stimulating beverage. Stimulating Property of Cocoa The mild stimulating property which cocoa possesses is due to the presence of the two substances, the obromine and caffeine. The presence of the obromine is peculiar to cocoa, but caffeine is a stimulating principle which also occurs in tea and coffee. Whilst in the quantities in which they are present in cocoa, about 1.5% of the obromine and 0.6% of caffeine, they act only as agreeable stimulants. In the pure condition, as white crystalline powders, they are powerful curative agents. Caffeine is well known as a specific for nervous headaches and as a heart stimulant and diuretic. The obromine is similar in action, but has the advantage for certain cases that it has much less effect on the central nervous system. And for this reason, it is a very valuable medicine for sufferers from heart dropsy and as a tonic for senile heart. That its medicinal properties are appreciated is shown by its price. During 1918, the retail price was about eight shillings an ounce, from which we can calculate that every pound of cocoa contained nearly two shillings worth of the obromine. Soluble cocoa. Whilst Forrester states that treated cocoa is the most digestible, experts are not in agreement as to which is the more valuable food stuff, the pure untouched cocoa, or that which is treated during its manufacture with alkaline salts. The cocoa so treated is generally described as soluble, although its only claim to this name is that the mineral salts in the cocoa are rendered more soluble by the treatment. It is also sometimes incorrectly described as containing alkali, but actually no alkali is present in the cocoa, either in a free state or as carbonate. The potassium exists, quote, in the form of phosphates or combinations of organic acids, that is to say, in the ideal form in which these bodies occur in foods of animal and vegetable origin, end quote, fritched fabrication, do chocolate, page 216. Food Value of Chocolate. I ate a little chocolate from my supply, well-knowing the miraculous sustaining powers of the simple little block, from Mr. Isaacs by F. Marion Crawford. Whilst the food value of cocoa powder is very high, the drink prepared from it can only be regarded as an accessory food, because it is usual to take the powder in small quantities, just as with beef tea, it is usual to take only a small portion of an ox and a teacup. But chocolate is often eaten in considerable quantities at a time and must therefore be regarded as an important foodstuff and not considered, as it frequently is considered, simply a luxury. The eating of cacao, mixed with sugar, dates from very early days, but it is only in recent times that it has become the principal sweet meat. What would a sweet shop be today without chocolate? That summit of the confectioner's art, when the rich brown chocolate is the predominant note in every confectioner's window, what would the lovers in England do without chocolates, which enable them to indulge their delight in giving what is sure to be well received? As a luxury, it is universally appreciated and because of this appreciation, its value as a food is sometimes overlooked. During the war, chocolate was valued as a compact foodstuff, which is easily preserved. Dr. Gastinu Earle, lecturing for the Institute of Hygiene in 1915 on food factor in war, said, chocolate is a most valuable concentrated food, especially when other foods are not available. It is the chief constituent of the emergency ration. Its importance as a concentrated foodstuff was appreciated in the United States, for every comfort kit made up for the American soldiers fighting in the war contained a cake of sweet chocolate. There are a number of records of people whose lives have been preserved by means of chocolate. One of the most recent was the case of Commander Stewart, who was torpedoed in HMS Cornwallis in the Mediterranean in 1917. He happened to have in his cabin one of the boxes of chocolate presented to the Army and Navy in 1915 by the colonies of Trinidad, Grenada, and St. Lucia, who gave the cacao and paid English manufacturers to make it into chocolate. He had been treasuring the box as a souvenir, but being the only article of food available, he filled his pockets with the chocolate, which sustained him through many trying hours. We have already seen the high food value of the cacao bean. What of the sugar which chocolate contains? Sugar is consumed in large quantities in England, the consumption per head, amounting to 80 to 90 pounds per year. It is well known as a giver of heat and energy, and Sir Ernest Shackleton reports that it proved a great life preserver and sustainer in Arctic regions. Our practical acquaintance with sugar commences at birth, milk containing about 5% of milk and sugar, and when one considers the amazing activity of young children, one understands their continuous demand for sugar. Dr. Hutchinson in his well known Food and the Principles of Dietetics says, the craving for sweets which children show is, no doubt, the natural expression of a physiological need, but they should be taken with and not between mills. Chocolate is one of the most wholesome and nutritious forms of such sweets. Both the constituents of chocolate being nourishing, it follows that chocolate itself has a high food value. This is proved by the figures given below. As with cocoa, we have first to know the composition before we can calculate the food value. The relative proportions of nib, butter and sugar vary considerably in ordinary chocolate, so that it is difficult to give an average composition. There are sticks of eating chocolate, which contain as little as 24% of cacao butter, whilst chocolate used for covering contains about 36% of butter. As modern high-class eating chocolate contains about 31% of butter, we will take this for purposes of calculation. Average composition and fuel value of English eating chocolate. Cacao butter, composition, 31.4 equals energy giving power, 1,327 calories per pound. Protein, total nitrogen, 78%. Composition, 4.1 equals energy giving power, 76 calories per pound. Cacao starch, composition, 2.3. Other digestible carbohydrates, etc. Composition, 6.4. Together equal energy giving power, 162 calories per pound. Stimulants, theobromine, composition, 0.3. Caffeine, composition, 0.1. Mineral matter, composition, 1.2. Crude fiber, composition, 0.9. Moisture, composition, 1.0. Sugar, composition, 52.3 equals energy giving power, 973 calories per pound. Total, composition, 100.0. Total energy giving power, 2,538 calories per pound. In Snyder's Human Foods, 1916, the official analysis of 163 common foods are given. They include practically everything that human beings eat, and only three are greater than chocolate in energy giving power. The result, 2,538 calories per pound, which we obtain by calculation, is lower than the figure, 2,768 calories per pound, for chocolate given by Sherman in his book on food and nutrition, 1918. Probably, his figure is for unsweetened chocolate. The table below shows the energy giving value of cocoa and chocolate compared with well-known foodstuffs. The figures, save for eating chocolate, are taken from Sherman's book, and are calculated from the analysis given in Bulletin 28 of the United States Department of Agriculture. Food value of foodstuffs. Cabbage, 121 calories per pound. Codfish, 209 calories per pound. Apples, 214 calories per pound. Potatoes, 302 calories per pound. Milk, 314 calories per pound. Eggs, 594 calories per pound. Beef steak, 960 calories per pound. Bread, average white, 1,180 calories per pound. Oatmeal, 1,811 calories per pound. Sugar, 1,815 calories per pound. Cocoa, 2,258 calories per pound. Eating chocolate, 2,538 calories per pound. Food value of milk chocolate. The value of milk as a food is so generally recognized as to need no commendation here. When milk is evaporated to a dry solid, about 87.5% of water is driven off, so that the milk left has about eight times the food value of the original milk. Milk chocolate of good quality contains from 15 to 25% of milk solids. Milk chocolate varies greatly in composition, but for the purpose of calculating the food value, we may assume that about a quarter of a high class milk chocolate consists of solid milk, and this is combined with about 40% of cane sugar and 35% of cacao butter and cacao mass. Analysis and fuel value of milk chocolate. Milk fat and cacao butter, 35.0, 1,480 calories per pound. Milk and cocoa proteins, 8.0 equals 149 calories per pound. Cacao starch and digestible carbohydrates, 3.0 equals 56 calories per pound. Stimulants, theobromine and caffeine, 0.2. Mineral matter, 2.0. Crude fiber, 0.3. Moisture, 1.5. Milk sugar and cane sugar, 50.0 equals 930 calories per pound. Total, 100.0 equals 2,615 calories per pound. It will be noted that the food value of milk chocolate is even greater than that of plain chocolate. It is highly probable that milk chocolate is the most nutritious of all sweet meats. It is not generally recognized that when we purchase one pound of high class milk chocolate, we obtain three quarters of a pound of chocolate and two pounds of milk. And of chapter eight. Chapter nine of cocoa and chocolate. This is a LibriVox recording. All LibriVox recordings are in the public domain. For more information or to volunteer, please visit LibriVox.org, this reading by Allison Hester of Athens, Georgia. Cocoa and Chocolate, their history from plantation to consumer by Arthur W. Knapp. Chapter nine, adulteration and the need for definitions. Cocoa. Cocoa might conveniently be defined as consisting exclusively of shelled, roasted, finely ground cacao beans, partially defatted, with or without a minute quantity of flavoring material. The gross adulteration of cocoa is now a thing of the past. And most of the cocoa sold conforms with this definition. Statements, however, get copied from book to book. And hence we continue to read that cocoa usually contains arrow root or other starch. In the old days, this was frequently so, but now owing to many legal actions by public health authorities, this abuse has been stamped out. Nowadays, if a public analyst finds flour or arrow root in a sample bought as cocoa, he describes it as adulterated and the seller is prosecuted and fined. Hence, save for the presence of cacao shell, the cocoa of the present day is a pure article consisting simply of roasted, finely ground cacao beans, partially defatted. The principal factors affecting the quality of the finished cocoa are the difference in the kind of cacao bean used, the amount of cacao butter extracted, the care in preparation, and the amount of cacao shell left in. The presence of more than a small percentage of shell in cocoa is a disadvantage, both on the ground of taste and of food value. This has been recognized from the earliest times, see quotations on page 128. In the cocoa powder order of 1918, the amount of shell which a cocoa powder might contain was defined grade A, not to contain more than 2% of shell and grade B, not more than 5% of shell. The manufacturers of high class cocoa welcome these standards, but unfortunately the known analytical methods are not delicate enough to estimate accurately such small quantities so that any external check is difficult and the purchaser has to trust the honesty of the manufacturer. Hence, it is wise to purchase cocoa only for makers of good repute. Chocolate. We have so far no legal definition of chocolate in England, as Mr. NP Booth pointed out at the Seventh International Congress of Applied Chemistry. At the present time, a mixture of cocoa with sugar and starch cannot be sold as pure cocoa, but only as chocolate powder, and with a definite declaration that the article is a mixture of cocoa and other ingredients. Prosecutions are constantly occurring where mixtures of foreign starch and sugar with cocoa have been sold as cocoa, and it seems therefore a proper step to take to require that a similar declaration shall be made in the case of chocolate, which contains other constituents than the products of cocoa nib and sugar. We cannot do better than to quote in full the definitions suggested in Mr. Booth's paper. The author refers to the absence of any legal standard for chocolate in England, although in some of the European countries standards are in force, and points out, as a result of this, that articles of which the sale would be prohibited in some other countries are permitted to come without restriction onto the English market. He suggests that the following definitions for chocolate goods are reasonable and could be conformed to by makers of the genuine article. These standards are not more stringent than those already in forest in some of the colonies in European countries. One, unsweetened chocolate or cacao mass must be prepared exclusively from roasted, shelled, finely ground cacao beans with or without the addition of a small quantity of flavoring matter and should not contain less than 45% of cacao butter. Two, sweetened chocolate or chocolate. A preparation consisting exclusively of the products of roasted, shelled, finely ground cacao beans and not more than 65% of sugar with or without a small quantity of harmless flavoring matter. Three, granulated or ground chocolate for drinking purposes. The same definition as for sweetened chocolate should apply here except that the proportion of sugar may be raised to not more than 75%. Four, chocolate covered goods. Various forms of confectionery covered with chocolate, the composition of the latter agreeing with the definition of sweetened chocolate. Five, milk chocolate. A preparation composed exclusively of roasted, shelled cacao beans, sugar and not less than 15% of the dry solids of full cream milk with or without a small quantity of harmless flavoring matter. Mr. Booth further states that starch other than that naturally present in the cacao bean and cacao shell and powder form should be absolutely excluded from any article which is to be sold under the name of chocolate. End of chapter nine. Chapter ten of cocoa and chocolate. This is a LibriVox recording. All LibriVox recordings are in the public domain. For more information or to volunteer, please visit LibriVox.org. This reading by Allison Hester of Athens, Georgia. Cocoa and Chocolate, Their History from Plantation to Consumer by Arthur W. Knapp. Chapter ten, The Consumption of Cacao. The war has caused such a disturbance that the statistics for the years of the war are difficult to obtain. For many years the German publication, The Gordian, was the most reliable source of cacao statistics, and so far we have nothing in England sufficiently comprehensive to replace it, although useful figures can be obtained from the Board of Trade returns of imports into Great Britain from Mr. Theo. Fasmer's reports, which appear from time to time in the Confectioner's Union and elsewhere, from Mr. Hemel Smith's collated material in Tropical Life, and from the reports of important brokers like Montsur Woodhouse. In 1919, the Bulletin of the Imperial Institute gave a very complete resume of cacao production as far as the British Empire is concerned. Great Britain. Since 1830, the consumption of cacao in the British Isles has shown a great and continuous increase, and there is every reason to believe that the consumption will easily keep pace with the rapidly growing production. One effect of the war has been to increase the consumption of cacao and chocolate. Many thousands of men who took no interest in sweets learned from the use of their emergency ration that chocolate was a very convenient and concentrated foodstuff. Cacao beans cleared for home consumption. In 1830, 450 English tons. 1840, 900 English tons. In 1850, 1,400 English tons. In 1860, 1,450 English tons. 1870, 3,100 English tons. 1880, 4,700 English tons. 1890, 9,000 English tons. 1900, 16,900 English tons. And in 1910, 24,550 English tons. Cacao beans imported into United Kingdom. In 1912, 33,600 total imported tons. 27,450 tons retained in the country. 24,600 home consumption tons. 1913, 35,000 total imported tons. 28,200 tons retained in the country. 23,200 home consumption tons. 1914, 41,750 total imported tons. 29,600 tons retained in the country. 24,900 home consumption tons. 1915, 81,800 total imported tons. 54,400 tons retained in the country. 40,300 home consumption tons. 1916, 88,800 total imported tons. 64,750 tons retained in the country. 29,300 home consumption tons. 1917, 57,900 total imported tons. 53,100 tons retained in the country. And 41,300 home consumption tons. The above figures are compiled from the bulletin of the Imperial Institute. The total imports for 1819 were 42,390 tons. This sudden and marked drop in the amount imported was due to shortage of shipping. There were, however, large quantities of cacao in stock and the amount consumed showed a marked advance on previous years, being 61,252 tons. The Board of Trade Returns for 1919 are as follows. Cacao beans imported into the United Kingdom. From British West Africa, 72,886 tons. From British West Indies, 13,219 tons. From Ecuador, 9,153 tons. From Brazil, 3,665 tons. From Ceylon, 903 tons. From other countries, 13,820 tons. Total, 113,646 tons. Home consumption, 64,613 tons. It will be noted that the import of British Cacao is over 75% of the total. Before the war, about half the Cacao imported into the United Kingdom was grown in British possessions. During the war, more and more British Cacao was imported, and now that a preferential duty of seven shillings per hundred weight has been given to British colonial growth, we shall probably see a still higher percentage of British Cacao consumed in the United Kingdom. Value of Cacao beans imported into the United Kingdom. To the nearest 1,000 pounds. In 1913, the total value of Cacao beans imported, 2,199,000 pounds. From British possessions, value 1,158,000 pounds. Percent from British possessions, 52.7. 1914. The total value of Cacao beans imported, 2,439,000 pounds. From British possessions, value 1,204,000 pounds. 49.4%. 1915. Total value of Cacao beans imported, 5,747,000 pounds. From British possessions, value 3,546,000 pounds. 61.7%. 1916. Total value of Cacao beans imported, 6,498,000 pounds. From British possessions, value 4,417,000 pounds. 68%. 1917. Total value of Cacao beans imported, 3,498,000 pounds. From British possessions, value 3,010,000 pounds. 86%. 1918. Total value of Cacao beans imported, 3,040,000 pounds. From British possessions, value 2,549,000 pounds. 83.8%. 1919. Total value of Cacao beans imported, 9,207,000 pounds. From British possessions, value 6,639,000 pounds. 72.1%. That the consumption of Cacao is expected to grow greater yet in the immediate future is reflected in the prices of raw Cacao, which, as soon as they were no longer fixed by the government, rose rapidly. Thus, Acura Cacao rose from 65 shillings per 100 weight to over 90 shillings per 100 weight in a few weeks, and now, January 1920, stands at 104 shillings. World consumption. The world's consumption of Cacao is steadily rising. Before the war, the United States, Germany, Holland, Great Britain, France, and Switzerland were the principal consumers. Whilst we have increased our consumption so that Great Britain now occupies second place, the United States has outstripped all the other countries, having doubled its consumption in a few years, and is now taking almost as much as all the rest of the world put together. It is thought that since America has gone dry, this remarkably large consumption is likely to be maintained. World's consumption of Cacao beans to the nearest thousands tons, one ton equals 1,000 kilograms. USA, pre-war, 1913, 68,000 tons. War period, average of 1914, 15, 16, and 17, 103,000 tons. 1918, 145,000 tons. Postwar, 1919, 145,000 tons. Germany, pre-war, 1913, 51,000 tons. War period, average of 1914, 15, and 16, and 17, 28,000 tons. Postwar, 1919, 13,000 tons. Holland, pre-war, 1913, 30,000 tons. War period, average of 1914, 15, 16, and 17, 25,000 tons. 1918, 2,000 tons. Postwar, 39,000 tons. Great Britain, pre-war, 1913, 28,000 tons. War period, average of 1914, 15, 16, and 17, 41,000 tons. 1918, 62,000 tons. Postwar, 1919, 66,000 tons. France, pre-war, 1913, 28,000 tons. War period, average of 1914, 15, 16, and 17, 35,000 tons. 1918, 39,000 tons. Postwar, 1919, 46,000 tons. Switzerland, pre-war, 1913, 10,000 tons. War period, average of 1914, 15, 16, and 17, 14,000 tons. 1918, 18,000 tons. Postwar, 1919, 21,000 tons. Austria, pre-war, 1913, 7,000 tons. War period, average of 1914, 15, 16, and 17, 2,000 tons. Postwar, 1919, 2,000 tons. Belgium, pre-war, 1913, 6,000 tons. War period, average of 1914, 15, 16, and 17, 1,000 tons. 1918, 1,000 tons. Postwar, 1919, 8,000 tons. Spain, pre-war, 1913, 6,000 tons. War period, average of 1914, 15, 16, and 17, 7,000 tons. 1918, 6,000 tons. Postwar, 1919, 8,000 tons. Russia, pre-war, 1913, 5,000 tons. War period, average of 1914, 15, 16, and 17, 4,000 tons. Canada, pre-war, 1913, 3,000 tons. War period, average of 1914, 15, 16, and 17, 4,000 tons. 1918, 9,000 tons. Italy, pre-war, 1913, 2,000 tons. War period, average of 1914 through 17, 5,000 tons. 1918, 6,000 tons. Postwar, 1919, 6,000 tons. Denmark, pre-war, 1913, 2,000 tons. War period, average of 1914 through 1917, 2,000 tons. 1918, 2,000 tons. Sweden, pre-war, 1913, 1,000 tons. War period, average of 1914 through 1917, 2,000 tons. 1918, 2,000 tons. Norway, pre-war, 1913, 1,000 tons. War period, average of 1914 through 17, 2,000 tons. 1918, 2,000 tons. Other countries, estimated. Pre-war, 1913, 5,000 tons. War period, average of 1914 through 1917, 8,000 tons. 1918, 11,000 tons. Postwar, 1919, 26,000 tons. Total, pre-war, 1913, 252,000 tons. War period, average of 1914 through 1917, 283,000 tons. 1918, 305,000 tons. Postwar, 1919, 380,000 tons. The above figures are compiled chiefly from Mr. Theo Vazmer's reports. The Gordian estimates that the world's consumption in 1918 was 314,882 tons. In several of our larger colonies, and in at least one European country, there is obviously ample room for increase in the consumption. When one considers the great population of Russia, 4,000 to 5,000 tons per annum is a very small amount to consume. It is pleasant to think of cocoa being drunk in the ice-bound north of Russia. It brings to mind so picturesque a contrast. Cacayo, grown amongst the richly-colored flora of the tropics, consumed in a land that is white with cold. When Russia has reached a more stable condition, we shall doubtless see a rapid expansion in the Cacayo consumption. End of Chapter 10 End of Cocoa and Chocolate, Their History from Plantation to Consumer, this reading by Alison Hester in July 2008.