Now we are going to see how you can incorporate sustainability in computing and specifically we'll talk how you can make energy efficient placement of workloads on edge. So who we are? I am Parul Singh. I work as a senior software engineer at Red Hat and work in the sustainability team where we are incubating various tools and technologies to make green efficient computing. I'm Kai Liu. I'm a University of Toronto student and I'm currently working as an intern at Red Hat as a software engineer in the sustainability team. So we thought that the best way is to take a community based initiative on environmental sustainability. So we already have a CNCF environment sustainable tag. If you are interested feel free to check out our proposal. We've also collaborated in the community. One of our POC that we did was Carbon Aware Scaling with Kera. We did this with Microsoft where we're investigating how to use a carbon intensity to make workload scaling decisions. We've also collaborated with IBM on making a VP a recommender for Kubernetes that take into account container level energy. So let's get into the agenda. It's very simple. We'll start with some background of how things are right now in sustainability and computing. And then we will very briefly talk about a sustainability stack specifically to projects Kepler and the model server. And then we will get to the demo which is the main objective of today's talk. How you do energy efficient placement of the workloads. So we have a quote here that sort of sums up how bad carbon emissions are in the tech field. So in 2021 an ACM technology brief estimated that the information and communication technology sector contributed between 1.8% and 3.9% of global carbon emissions which is actually more than the total carbon emissions emitted by Germany and Italy combined. So that haunts us to ask question how can you do energy how you can measure energy in while you're doing computation. When you don't have access to the infrastructure maybe it's on private on public cloud or it's on premise it's a hybrid cloud structure it's a bare metal. How are you measuring energy and specifically indirectly where you cannot install specific hardware to measure it and how are you attributing energy consumption of workloads in a shared environment where you have shared resources for processes pods and containers. So that got us into thinking let's build a sustainability stack and the first project that we got started was Kepler and the Kepler model server. Before we get into the projects let's talk about mythology how do we calculate or how we measure energy. So our principle is based on our mythology is based on the principle that CPU power consumption can be directly proportional to CPU power energy or like how much CPU energy power you're consuming. So for example let's say you have a container that is consuming 10% of CPU you can say that it attributed to 10% of CPU power or you have a workload that has five containers and it is running each of them are consuming 10% and totally you can say that totally they attributed to 50% of power consumption and we didn't make this up our principle you can check out the white paper that we referenced and this a lot of work is being done in this area so our mythology is based on the many scientific papers. So let's get into our first project which is the Kubernetes based efficient power level exporter and what Kepler does essentially it uses software counters to measure power consumption by hardware resources and it exports them as Prometheus metrics. So Kepler does three things the first it's reports per part energy consumption including from resources like CPU, GPU, RAM and it supports bare metal as well as VM and it uses cloud native stack like Prometheus and Grafana to render dashboard where we can see the energy consumption. Next Kepler is supposed to measure energy consumption so it's very important for us that the energy footprint of Kepler itself is very low so we use EBPF to probe software and kernels to get these readings and at last Kepler uses machine learning models where you don't have access to the existing power monitors on the cluster. Kai you will get more into that but wherever you don't have an existing power meter Kepler uses its own machine learning models to get the readings. So let's take a look at the second part of this Kepler sustainability stack which is the Kepler model server project. So a little bit about the Kepler model server by default Kepler will use a supported power meter or some power measurement tool to measure node level energy metrics like CPU core and DRAM and then it will use that to try and predict pod level energy metrics. So what happens when Kepler does not have access to a supported power meter? That's where the Kepler model server steps in to save the day so the Kepler model server will provide trained models for Kepler to predict these missing energy metrics and they use software counters and performance metrics as the features. So the current tech stack of the Kepler model server includes TensorFlow Keras for the models and Flask and Prometheus for exporting these models for Kepler to use. So let's take a look at some examples of the node level energy metric models that the Kepler model server currently has implemented. So the first one will predict CPU core energy consumption. We're currently using a linear regression model for that and it uses the following software counters and performance metrics as the features and it's also capable of online learning. That way we can refit and retrain the model in the future with more up-to-date data to make sure the performance does not get worse. And then another example is also a linear regression model which also is capable of online learning. However, this time it's used to predict DRAM energy consumption and it uses a different set of performance metrics as the features. So let's take a look at how the model server fits in Kepler as a whole. We can effectively split the model server into two parts. The first part is the training phase. So here we'll have training workloads on a cluster. Kepler will retrieve the node energy metrics and performance metrics from the cluster. And Kepler can do this because a power meter will be of course provided. It'll then aggregate these node energy metrics and performance metrics and a collector and then export them as Prometheus. The Kepler model server will then scrape these node level energy metrics and performance metrics from Kepler's Prometheus metrics. And then I'll set up training, testing and validation data sets to train and evaluate and then save the regression models as long as the error is not that high. And we can use TensorFlow's API to do this. And then finally we'll then allow the models to be exported for Kepler to use for prediction. And that leads us to the second part which is the exporting phase. So how do we export these models for Kepler to use it? So we can export like the archive file of the model or we can like export the model weights for Kepler to use to make a prediction of the energy metrics. And we can do that with Flask routes and Prometheus metrics. And in the future, we also want to try and include, try to export the models using open telemetry metrics. So let's get to the interesting part. How you do carbon intensity aware scheduling. So the use cases are considered a scenario where you want your workload to be fueled by as much as renewable energy, say solar energy or wind energy during the day. And at night you want to use fossil energy because the renewable energies are not available. Or you just want to control the carbon intensity of your workload. So the second is what we are going to demonstrate today, how you can control the carbon intensity. The use case premise is you have a multi-node cluster and the nodes are running in different zone. And this is useful for long running batch or machine learning workloads where you are online training a machine learning model. And the reason is because we are migrating the workloads according to the carbon intensity. So let's say that you have a workload where just the container spires up, it does some computation, gives you a result and just dies. You don't need to, it won't be very useful if you introduce this because it's just running for a very short time. So this is best suited for our workloads that are running for many hours or like over days, best is like the machine learning training models. And the demo setup is we have three nodes, open shift cluster. We are using Prometheus as a monitoring stack and we would be using features of Kubernetes like taints, tolerations and node selector to control the placement of the workloads. And we have a carbon intensity forecaster. Kai, can we tell you about that? Right, so that leads to the question, how do we forecast the carbon intensity for a node? So we have an exporter which will scrape from numerous data sources from numerous public energy APIs such as electricity map, what time, national grid. And it'll retrieve this carbon intensity data as time series data. So then this will be exported as Prometheus metrics. The carbon intensity forecaster will then come in and scrape these Prometheus metrics, retrieve it as a carbon intensity time series data and they can use that to update the models for each of the node. So the models can now be used to predict the forecasted carbon intensity of a node some hours in the future. And it's important to note that the carbon intensity forecaster and the exporter are extendable interfaces. So we want it so that the carbon intensity forecaster can introduce, you can integrate new models. And for the exporter you can integrate new handlers to handle more different types of public energy APIs and different types of data sources. So let's get into the flow of how this work. We periodically assign node labels according to the forecasted carbon intensity of the zone that node is present in. And there's a cron job that makes requests to the carbon intensity forecaster. And the carbon intensity forecaster will return the carbon intensity in future, sometime in future, for example, like two hours. And for example, we have three nodes. So the carbon intensity forecaster returns the forecasted carbon intensity for the nodes. We sort them from high to low. The highest get a label red. The lowest has a label green. And anything in between has a label yellow. And the pod also have to specify its intention. So for our demo, the pod that we'll be running says that I like the node that has the label carbon intensity as green. And we also have to provide the tolerations because the pod does not want to be assigned to a node that has carbon intensity as red. So you can see in the diagram, there are three nodes, node 1, node 2, node 3. The node 1 has label red. And whichever node has label red, you also add a taint that this node has very high carbon intensity. And node 2 and node 3 has no taints but label green and yellow. And in the pod spec, you see that we have a tolerations for the key carbon intensity with the value as red and effect as no execute and the tolerations seconds of five. So it means that if the pod is running and a matching taint is applied to the node, then the pod will stay bound on that node only for five seconds. And it will be evicted after five seconds. So let's see how the transitioning happening. Let's say a node is transitioning from green to red, which is node 1. And the pod was already running. So you will label the node as red. It get tainted. And the taint is carbon intensity red and no execute. So the pod gets evicted from node 1 and gets assigned to node 2, which is transitioning from label red to green. So you have to label the node as green and you have to untaint the node. And be aware that tainting just ensures that pods are evicted by the nodes if pods have no toleration for the taint. So the labels ensures where the pod would be assigned. But if a pod is already running on a node, the tolerations ensures that they get evicted from the node if the carbon intensity is higher than the pod wants. So let's take a look at how it all fits together. So on the left, you can see the public energy APIs and the data sources that we have available that provide us with the carbon intensity data. So we have, for instance, national grid, electricity, map, what time. And in the future, we want to include more data sources. So then the carbon intensity forecaster will scrape from these data sources and retrieve the carbon intensity time series data, which will then be exported as Prometheus metrics. The carbon intensity forecaster will scrape these Prometheus metric, use the carbon intensity time series data to update its models. And then external applications like the cron job can then make an HTTP get request to the carbon intensity forecaster to retrieve the forecasted carbon intensity of a node, some hours in the future, for example. And then so not that the cron job has the forecasted carbon intensity of all the nodes, it'll then start to label the nodes. So nodes that have the highest carbon intensity, forecasted carbon intensity, will be labeled red. Nodes that have the lowest forecasted carbon intensity will be labeled green, and anything in between will be labeled yellow. Okay, so we are going to see the demo now. So you can see that we already have a cron job running on the right-hand side, if you see. And it was last schedule 117 seconds ago. So, and we have another cron job that is just running and you can see it over here. It's just got schedule seven seconds ago. So let's, and it is finished. So you can see that it has applied tints and label. So these are the labels. So you can see that the node <IP_ADDRESS> has a label green and it has, yeah, it's here. And it has no taint available. So now, let's see how my pod spec looks like. So my pod is called CI test. It has a node selector with carbon intensity as green and we have the taints over here that says carbon intensity red, no execute in five. So, and over here in the top left, you see that we don't have the pod running. So I'm just going to run that pod. Okay, so the pod is being created and it will take some time. Yeah, so it got terminated. Now let me explain you why it was terminated. We are not modifying the scheduler. So the scheduler just assign the pod to wherever it wants to. But the label and the taints ensures that the pod get assigned to the right node. So the first time that we ran it terminated because maybe that node had attained but now it is running fine and let's just see which node it got assigned to. Okay, so you can see that it is running on a node that is <IP_ADDRESS> and a cron job is running. So we will see what taints and node it has applied. So you can see that this node has the label as green and the pod has been scheduled on the same node. So that's, we are not dealing with the scheduler. The scheduler is not touched. It's just the node selector and the taints and toleration that is taken care of. So we have some lessons that we learned. So the first lesson that we learned was that finding zone carbon intensity data is actually quite very difficult. There's not too much public energy APIs out there that let us use their carbon intensity data. And also of the ones that do allow us to use their carbon intensity data, sometimes there are time points that are missing. So in the future, we wanna try and remedy this by trying to get more and more access to carbon intensity data sources. That way the chances of a time point being completely null is extremely low. We also realized that our queries are very simple. Like we just make queries, what is the current or what is the average carbon intensity in zone? But we need to have a solution for more complicated queries, like which zone has the lowest carbon intensity and is the carbon intensity low within the past 24 hours. And we also want to support multiple electricity carbon emission provider as we already talked earlier. So we have taken an interface and you can extend our interface whenever new sources are available. And we want to improve and integrate this whole thing and integrated with green software carbon away SDK, which are already doing a lot of work in making carbon aware SDKs. So the roads ahead for us is to take this approach and apply it to a multi cluster scenario. And we are exploring how you can do that with KCP and integrate carbon intensity awareness in Kubernetes six scheduler profile. So we are not touching the scheduler, but we want this to be assigned into a scheduler itself. So we are exploring options with trimaran target load packing, which is already a scheduler that does an excellent job at optimizing a node for workloads. And we just want to take that scheduler and introduce carbon intensity into it. That's all pretty much. So here are some references. If you're interested in our projects or a whole sustainability stack, this QR, will lead you to a GitHub organization where you can see all these projects. And the link to the white paper that's a base for us, you can explore that. And we are ready to take questions. That's nice, no questions? Thank you so much. Thank you so much. Thank you.


In this presentation we will reconcile our year end payroll forms and in essence double check that they've been filled out correctly. So we have here our forms 941 and the W3. These are the two forms that we can use to reconcile each other and make sure that the two forms have been filled out correctly. We can double check in other words our quarterly form by tying it out to the W3 and vice versa. Check the W3 to the quarterly forms. Remember what these quarterly forms are doing is they are calculating the three main taxes which are the FIT, federal income tax for the employees, the social security tax for employees and employer, and the Medicare tax for employees and employers. This is going to be done on a quarterly basis so we've got January, February, March, April, May, June, July, August, September, October, November, December, or quarter four, quarter three, quarter two, quarter one. The W3 of course is just a yearly form that will be reported at the end of the year summarizing all the W2s that we send to both the employees and the IRS. So what we can do is double check these forms because we can see here on the totals here we have the total wages, social security, and Medicare wages as well as the calculations for federal income tax, social security tax, and Medicare taxes which have been withheld. So to do that we're going to try to match these up. Now what we don't show here is the 940, the form 940 which is the year end form and that's because it's reporting FUTTA which is just a tax for is an employee or only tax and therefore not reported on the W3. It is possible for us to kind of compare the FUTTA wages but we still have to do some reconciling to make sure that we're picking up the correct wages on the FUTTA form which is a yearly form, the form 940. So common misconception would be that the quarterly forms, the 941s, the IRS wants us to report everything quarterly and then summarize that same information again on the yearly form 940. Not necessarily the case, the forms 941s report the three main taxes, the big taxes, the FIT, federal income tax, social security, and Medicare which the IRS wants to see on a quarterly basis whereas the form 940 reports a tax that's usually a lot smaller FUTTA federal unemployment tax and therefore the IRS is happy or content enough I guess just to see that reported on a yearly basis rather than on a quarterly basis. So they're actually two different things for reconciling there. So if we look at what we have these we have quarter three and quarter four in this example we only have two quarters rather than four quarters and that will make it a little bit easier for us to show how this reconciling process would work. If we had four quarters of data then of course we would just add up the four quarters of data. So on the 940s over here this box, box two is the wages and tips box and these are really wages and tips for FIT calculation meaning this and this for the two quarters should add up to this box one. So if we were to tie this out to double check everything at the end of the year we're going to say okay the nine one four two five point one plus the one three five nine oh nine point nine should add up to the two twenty seven three three five and you might even I even subtracted sometimes two two seven seven three three five minus that should go to zero so that would tie out and then we're taking a look at the federal income tax withheld box three should tie out to the federal income tax box two on the w3 now again you can't tie this number to this number because it's too complex because it'll change by employee but we can sum them up and double check that way so again it's not a flat tax so we can't do a simple calculation from the wages to the tax we can take the one seven over three point two six plus the two five two one two point four and that should give us the forty two two fifteen sixty six if we subtract that forty two two fifteen point six six back to zero then we'll take the social security wages so we have the social security wages here and here that should tie out to social security wages here on the w3 so if we were to add the social security wages up we're adding the nine six nine seven three point five and the nine seven six three two point five to get the um 194 606 so if we subtract out what we have 194 606 back down to zero and then we're looking for the social security here now this is where it gets a little bit complex this is the actual tax and uh it this could differ from the tax that we're going to see over here so let's see why we're going to add up the 12024.71 and down here the 1206.43 and that's going to differ and it will always differ by the way the 12065 57 why because we multiplied this by twice the amount this in other words is not 0.062 but 0.124 or twice the wages in other words it's the employer and employee portion so if we take this amount and divide by two we'll get just the employee portion which is what's reported on the w2s and the w3 same's going to be here so this is the medicare taxes on these two lines if we add those two lines we're going to say that's the nine six nine seven three point five plus the 144232.5 giving us the 241 206 so if we subtract that out 241 206 back down to zero and then we're going to do the same calculation for medicare wages so we will add those two up okay so that's going to be and hit something funny there I'm not sure so 2812.23 plus the 4182.74 that's the 699497 again that's going to differ from what we have here because we see this rate of 0.029 which is really twice the rate that we typically would probably know 0.0145 because it's both the employer and employee portion therefore if we take this number and divide by two we're going to get the 3497 which is the number we want the 3497 these are the numbers that we can generally reconcile in most cases now there may be some reasons why these numbers would differ and if there were we'd want to make a reconciliation to know exactly what the differences are so when questioned if questioned which we quite possibly could be by the IRS in terms of some type of audit we could give responses to any kind of differences between the W3 and the 941s. Remember once again that we're talking about we only had two 941s here because it was our first year of operations our first year of payroll and we only had two quarters if we had four quarters we would have to add up the four quarters then we're having the form 940 which we can't tie out as neatly with with the other forms the W3 and the W2s why because the W3s and the W2s are employee taxes and this form is an employer tax and they don't tie out also to the quarterly forms the 941s because those report different taxes they report federal income tax social security and medicare so the 941s do report some employee error taxes their portion the employer portion of social security and medicare but doesn't report the employee error tax of FUTA this is the only tax form that reports the employee error tax of FUTA we can however look at the total payments for all employees the total compensation and try to reconcile that in some way to the amounts that are on the W3 and the 940 it may not reconcile exactly to any particular box it be closest to box to the medicare box on W3 but it could differ by a cafeteria plan but we can reconcile that difference and know what the difference would be between the total payments here and the medicare calculation on box three of the W2s and the W3 and box three of the W2s and the W3 would of course also reconcile to adding up the 940s which we just did on box 5c in part one off the payroll you