Hello everyone. Welcome to the Jenkins infrastructure weekly team meeting. Today we are the 9th of May 2022. Sorry, I don't want time to pass. Around the virtual table we have myself, Damien DuPortal, Hervé Le Meur, Mark White, Ison Deller, Stefan Merle, Bruno Verarten, Kevin Martins and our visitor, Sartac, is that correctly said? Yes. Okay, let's get started with announcement. The weekly core release 2.404 is out, at least release packages and Docker image. So I assume as usual, we have the release checklist item to be finished, changelog. As usual, they will be done a bit later, but that's a go for us to deploy that new version fully that will fix an issue we are seeing within FraCI, where Jenkins is having issues with Unicode characters in GitHub pull requests, only with the 2.403. So last week weekly, the new one should fix the issue. Do you have something else about the new weekly? Okay. Do you have other announcements? Okay. Let's have a look at the upcoming calendar. First of all, the next weekly 2.405 should happen next Tuesday, like every Tuesday. So that will be 16 of May, if I'm not mistaken. 2023, I don't remember when is the next LTS. We had one last week, so I assume it will be in a few weeks. So I will put it in A unless someone can find it. That should be at least three weeks before any release for us. Let's check if we have announced Jenkins security advisory. The last one was the 12th of April. So no security advisory announced. The next major event currently Mark White and Alex Brondes are attending the CDCon in Vancouver. I think the last day is today, and they will be headed back. I don't know what are the upcoming events where we will have a Jenkins team member. So we'll have the NAA, unless you know. Nope. So then anything else to add to the calendar? The next LTS will be the 22 of June. 2.346.1 2.346.1 Thanks. We have plenty of time in front of us for this one. Perfect. So let's start with the work that we were able to finish last week. We had a contributor, a new maintainer of a new plugin, Jack plugin, who looked like to have some issues and we were able to help them to release the first version of their plugin. Now it's more discussion about versioning. So we can consider this issue closed. Unable to create accounts, we have a user who tried to create the accounts. I'm not really sure. I think for this one, they had the issue, the logs of the application mentioned cookie. So in that case, when you have the cookie error as per the source code, it means that the user already had a session to account Jenkins IO on their web browser. That one doesn't happen often. Usually it's when you have multiple accounts and you have not unlocked your previous account when you try to create a new one. It's considered spam because that means you already are trying to create multiple accounts in an automated way in the same web browser session or the same current session. That might be another issue of course, but usually that's what the cookie means because the web server detects cookie with an active session. If you see that issue and you are human and not trying, try to clean up your cookies as usual or unlock from the other session properly. Then we had the Debian packages starting with no longer published. So that one is closed as not planned. I will continue with the list on the left if it's okay for you. Renew update center certificates. We were able to renew the certificate used to sign the metadata of the update center and the tools with success. So now we are four persons having access to the CA, the certificate authority in charge of signing this certificate. That means anyone from the team is able to generate a new certificate once a year usually, but only four persons on the world have the ability to sign these requests for generating a new certificate. It's now Kosukei, the creator of Jenkins, Oleg Nenachev, Olivier Verna. And now myself as an infrastructure officer. That CA is valid for five years. That means the four of us for the five upcoming years have to take care of that credential. So that one is okay. We were able to generate and unblock the old jobs and everything is back. No issue and nothing was broken, of course, for the end users. We had the delay of six or seven hours last Tuesday before there is an internal safety system in the update center generation that stopped building the update center and updating it 30 days before expiration of the certificate, which we met the 2 of May. We can change using an environment variable that has been logged on the issue for next year. I still need to update the calendar notification though for next year. Update controller to the latest LTS. Last Wednesday we had an LTS release. So we updated all our controller based on the LTS version with the plugins. Side note, that one embedded as your virtual machine plugin update, which had a breaking change. So we were beaten by this one, but we were able to fix that on all the controller in less than one hour. So we had a tiny queue of 15 builds on CIG and Kinsayo due to that. As soon as it was fixed, then the queue was treated immediately. Update center is missing weekly release. That one was closed. It should have been closed as not planned, but no action expected from us. It looks like that when we have a new weekly LTS release, we need to have the update center to regenerate the new version. And in order for that, you need a new plugin. So there is that time window between the new core release, either LTS or weekly, until there is a release of another plugin. There is no need to regenerate just for the core release. So that's what Mark explained and the reason why there were no action for us. The case here, just to be sure that everyone follows me, because it might have been too quick. Since we weren't able to successfully build the update center, no new plugins were updated. So the new core release wasn't taken in account. As soon as the update center was updated with the new certificates, then everything went back to normal. Any questions so far? Okay. Then we had dance and page duty notification on data dog warning notices. So thanks for taking care of this one. That's way less alert for us, especially when we have a machine using, let's say, 80, 81% or 85% of the hard drive. It's only a warning. It's not blocking. It's not emergency. It only decreases. It's a penalty for the IO performances. And thanks for adding that nice bot that every day tells us and remind us to check the data dog monitors for warning since we don't receive alerts anymore. That's all for the task that we act on. On non-plan, we had the Debian package starting with 2.0. We had a user that was using an accidentally working URL for the Debian packages. They should not and they have been told to use the official URL somewhere else. So they should not have any issue in the future. We had user that looks like thanks for taking care of that folks. Forgot to use an MMPassword. Someone messed up, mixed our L-Desk with their company L-Desk. So they wanted to get access to their own Jenkins, but yeah, nothing we can do for these people. And again, missing update center for 2.0. However, that's what I explained earlier. That one is focused only on the update center generation and the other is a consequence of the certificate. Any question whatsoever? Okay. So let's move to the work in progress. First of all, I'm taking in the order on the left. Increase disk space for system pool. So we have a private Kubernetes cluster hosting private Jenkins controller. And the system pool is a node pool, a collection of virtual machine where the technical services, let's say, you can see that as plugins, the plugins of the cluster in charge of taking care of the data, for instance. It's running on these virtual machines. These machines have initially 30 gigabytes of system disk. We receive alerts that we have warnings and alerts that this disk are quite often used. And we are almost full. The goal is to increase the disk space to let this machine have the expected performances. So we tried just before that meeting with Stefan. And we had an issue with this one is changing the default system node pool of an AKS cluster. It requires destroying and creating a brand new cluster. That looks like to be an Azure constraint. That means we cannot increase that disk. We want to try because the documentation is not really, let's say, rich. We are missing information. We might be able to use either special Terraform attributes for letting the Azure API, they can create and recycle between two system not pools. Or we can try creating a system not pools on our own to see if it works. Still not sure how we could do that, but might need to recreate. Changing this resource, require, destroy, recreate of the cluster. So another solution here is that we don't need that much of that. Most of this data is consumed by the Docker images of the services running here. So I propose that we instead of focusing on this, we try moving, we have another issue. We discovered that some of our bots are running on these not pools and they should not. They should run on the Linux pool. So if we don't have this not pool anymore, we expect the disk usage to be decreased. So proposal is that we first move the workloads that should not run on that system pool. Eventually you might need to add a specific system pool for that. And once we don't have any of these images and Genix ingress, the free bots and maybe a fifth, we only let the Azure default DNS and CSI pods, then we shouldn't have any warning anymore. Does it make sense? Do you agree on that or do you have other ideas? Okay, so let me, let's start moving up workload to decrease this usage. By the way, we took the opportunity to upgrade the Linux pool disk size. That was set to 50 gigabytes and we were using, we were using 80 81. So we are just on the limit of the warning. A nice reminder from Stefan is that in Azure, the way you pay for the odd drive or the SSDs, you pay for the next limits. So we were using 50 gigabytes. That means we can increase until the limit of 64. We will pay the same. You don't pay depending on the size you pay by, I forgot the English words, but it's a 32 and 64. So whether you have a 33 gigabyte disk or 64, you pay the same. So that's why we increased it to 64. That should also remove the last warning for these missions. So upgraded Linux pool looks good for you folks. An issue. Okay. We had a user asking for their building a plugin on CI and can say, they have an issue. So what they are doing is that they need a jar dependency from Maven on their plugin. And the jar dependency is part of the GitHub source code. Their POM XML had a Maven repository name, which has a specific IDs that points to the local files. So if you build locally on a machine that work and what I discovered is that our setup with the mirroring is trying to retrieve the artifact. From ACP from our artifact caching proxy. I would have expected Maven to detect the file dash column dash dash and implies that it's local so it shouldn't but doesn't seem to work that way. So I've opened to pull requests to be reviewed by you folks that adds two IDs to the list of exception. The one they use on the one and local both names means a few you can define a custom one to be excluded. So anyone else will be able to do the same trick. So once merge, we should be able to try this one and validate the issue. So yeah, I need a review and that's all and then I will take care of text testing text testing that and go back to the end users. Any question. My great trusted CI Jane Kinsayo from AWS to Azure Stefan, can you give us a status report of the tasks that's your biggest. Since the end of last week, we now have the three VMs set up in Azure in the same network with oldest there. Subnet, we have the Boons VM, the controller VM and the permanent agent VM. Now I need to still work on the network security configuration and then I will have to start on the on the specific migration of data. Next, work on security groups and then start migrating data. Okay, cool. Start preparing the migration first, but yes. Migration data. May I ask you, you can start also the puppet port. I don't remember if I've written that, but I validated it locally with the vagrant virtual machine that Ubuntu 22 with the Jenkins controller profile on Puppet works perfectly. I mean, it's it's Docker container so they should I need your help because I've never had it a new VM in Puppet. So I know that we need to register it in the Puppet master but may need some. Okay. Both everyone, I know how to do that. I don't know just information. I don't know if he or if you I let's let's see, but which both of us should be able. No problem. It's written that you need help to bootstrap this part makes sense. Puppet works perfectly on Ubuntu 20.04 for the Jenkins controller. Go to add this VM one security groups are okay. And before migration, of course. Yeah. Any question on that one? Nope. Okay. Can't access apply tools account another user for another plugin release. Most of the time we have issue. While we have your Kevin and Bruno, I might have a request that I'm making the request without thinking but we might need help. And we might need to catch these two contributor of two different plugins to check if we if there isn't something on the documentation to be fixed and improved fixed because we discovered that the factory has changed its UI. So part of our documentation explaining how to get your maintainer Jenkins encrypted password to put on Maven when you do a Maven manual release. That one doesn't not seem to work anymore for the UI. We might need to check with G frog, but or at least remove the part of the documentation that guides users step by step with the screenshots, because it's a brand new version of Artifactory. The curl common line still work perfectly. Yeah. So that one is fixed and the rest these users seems to have it a lot of e cups. And started from scratch. So that one might be the best target. But that user did not read carefully step by step. So a lot of mistakes were human. But that might be a way to improve the current documentation for brand new contributor. I'm not sure. Yeah. Do you know if the UI at Artifactory JFrog is stable enough for us to start the documentation or should we wait. I have no idea. I guess it. Right now it might seem it's exceptional in the sense that they they are advertising since December about the brand new web UI that you can switch to. And it looks like that the ports where you you go to generate the Maven settings file that one despite sticking to the classic UI is still now using the new I and I think that's the breaking change that recently and expected breaking change. But I might be wrong. So there is that fix about the UI. Because that one we need to fix the documentation even if we say hey it's command line only. I don't honestly don't mind. And the other one is maybe we have room for improvements, but we need the time to check on this one. I just wanted to share with both of you. Thank you. If you have any questions don't state even if it's now or later on this topics. Both users were able to release their plugins. I haven't closed this one because I wasn't I haven't checked the last feedback from the user though. They should. So that might be close today or tomorrow. No expectation. Okay. Cut that check one last time. Any question. Okay. The next one make environment and description fields mandatory for bug type issues. So as requested by Alex, he wants to add two mandatory field when someone opened a bug on the Jen Kinsey tracker on Jira. That has been it looks like there has been no objection on the mailing list. So I tried to. Yes, sorry. Daniel made an objection about the environment. I think there was an objection for one of these two field. I might have missed. Okay. I will look with Daniel, but in any case I need help from Daniel or Mark because I tried to understand Jira and I failed. And I'm not really willing to learn Jira. Honestly, it's my brain is not wired for that. So I've asked Mark for help on that topic. Ask Mark for help. So when Mark will be back from Vancouver. Thanks. Survey. Looks like. Which is problematic as it would need an excessive list environment. And this list is not static. Okay. Technical. It has been explained in the mailing list. Okay. I think I will try to move that away and let the board or the core developers with admin rights because not really related to infrastructure. And honestly, I don't know how to do that. So I need help from someone to teach me. I can fail to deploy artifacts. That one should be closed as well. I need to check one last time if it's okay for you. I closed it. And you, is this person continue to ask you questions about versioning. And you continue to answer them. But I don't think it's. I don't. I'm not sure. No, it's another. Okay. No, that's still seren carbonates. So I should be able to close it because it's duplicate of. Oh, no, it's manual. Okay. Okay. That person is still having a misconfigured main and settings good, good points. Okay. So that one is open and need work from us or pointers. We have a password resets that should be okay. No answer from the user. I will wait 24 hours before closing without answer. I'm not sure what's the problem of that person, honestly, because they even gave the link where to resets. So. I haven't seen any mail sent for after a password reset. Okay. Thanks for checking. Can you had a comment on the issue to tell that's no email? Where's there the account exists with this email? I haven't checked that. I checked the account existence. So both of our diagnostics points to that the person never, either the person never sent the form for a password resets or the person is not who this, who they pretend to be. So in other case, without any answer, we'll close the tomorrow. So no, no expectation. This one past releases sites are taking long time to load. So that one, I will move that one back to the backlog because it's working and we were able to give a solution for the end user for the problem specifically. Initially, it was answering 503 errors and was slow both the errors are gone since now three weeks. And one of the two links is working really fast and the other links is slow. We need to prove, we need to prove that's where are the performance e-cups to see how we can act. So that's, we need for that to use data doggies. So that's something inside the end of the survey. If I'm not mistaken, that's correct. Just a minute. So, yeah, very is that you in charge of connecting Apache for the mirror. With data doggies to collect metrics. Specific Apache metrics. No, I haven't done anything about that. Okay. Are you willing to do it or shall we take the issue? Yes. Will you have time for the upcoming milestone? Maybe. We'll see. If it's a maybe, then it's a no and we move that to backlog and we'll see later. I don't know. Okay. So if you don't know, it's a back to backlog. As they are no emergency or blockers now, because we cannot take 20 or 25 issue without being sure if we can have them. That's the idea. So I prefer if you don't know, it's a no. And then we don't take it and we do what we can. The next one, unless there is a question on the performance issues. Oh, I haven't shared the solution. That person was trying to programmatically list the weekly releases from and they use the link from the download page on Jenkin's website. The area is I pointed them to the Maven metadata XML file from artifactory, which is the source of truth. That's the first thing when we have a new war released there. So they should only have to curl that XML file, parse it in XML and extract the list. We already have Shell code doing that for the official Docker images. That's what I pointed to that person. Looks like they are using Golang, but I mean, that's a file to get on parse and treats. So they have a way to have the core X source of truth with acceptable performances. That's why it's not a blocker anymore. Good for you. Then analytics before I haven't heard back from Olivier about that we need to migrate some Google analytics. It's named properties. Olivier gave me another person administrator on permission on that property. But we need wall administrator. But it does not seem that Olivier has the wall admin. So, yeah, we'll see. But at the same time, Gavin proposed again to go to use Matomo, which will be a self-hosted analytics platform instead of Google analytics. So that could be interesting because he tried that successfully in parallel for the past months. So I asked him if he can open an issue for Matomo listing the technical requirement to see which cluster we should host it. We need a database. It's self-hosted, et cetera. How much data do we need? So that will mean a new service on the public Kate's cluster. No, if we can import existing data. No idea. You can ask him. I mean, we still need it. It will be still nice to have the access to analytics. Even if we don't use it after that. Yeah. But yeah, in the worst case, we install Matomo and we start from scratch on a new set of data. I mean, we can always request the data until we start using Matomo and then... So, yeah. So that one will go back to the backlog until we have an answer from either Gavin or Olivier. Because there is no action expected from us. It will go back on a milestone if we have to act or something. Worst case, Google Analytics will continue working. It will automatically migrate in June, I think, or July. Even though the scary message here, the details and the administration tells me it will be automatically migrated. But with new, eventually new settings. That's why they recommend you to do it manually and check.  So that's why I'm...  Yeah, we might lose a few features, but I mean, collecting data for analytics is not something we should do. So the advantage of Matomo, it's clearly way more powerful in terms of respecting private life of our users. But we have to host it and to store the data. That's the double-edged sword here. So that went back to backlog until we hear from news. Until Olivier and Gavin answers back. Any question? To be quite honest, I have no idea. Are someone here using the data from Google Analytics for Jenkins platform? Those data or just... The data from the Google Analytics for the Jenkins platform. Never did. So it might be useful, but I have no idea what data is tracked. I assume it's something on the Jenkins. So your website tracks user and their path, but no idea. I'm not sure what would be the goal, honestly. Worth asking the question. Maybe it's a naive question. I don't have knowledge of these tools. So how would you use these tools, folks? That's not for us. That's for marketing purposes or... Yeah, that's not for infrastructure or for... The idea is to point out where in the world we are trending, where there is, I don't know, interest. And there's a lot and a lot and a lot of data in Google Analytics. So someone who knows how it works can extract some sense on it. But as you said, we are not the one that will use it likewise. Yeah. It's like CIG and Kinsayo. We don't use CIG and Kinsayo directly. We are not plug-in maintainers, but we need to know the use case of the person using the infrastructure we maintain. That's why I'm asking the question. Hervé, do you have previous experience with Google Analytics? It's in context, so... Okay. I will ask Givin on the IRC channel. He might have answers for that, and I assume not. Okay. Next issue, currently open. Use a new virtual machine instance type for CIG and Kinsayo. Work in progress. Sorry, I've left the lock on Terraform Azure. I need to start working on it again. Was able to find a downsized instance with a bit less memory. The price difference is low. It's like 40 bucks per month for the VM itself, but that will be a virtual machine of last generation. Size found. Yeah, I have almost everything ready for creating an empty shell the same idea of what Stefan did with Trusted. So we should be able to do that soon. VM ready to be created. Need pair and review. So almost there. I don't have any specific question for this one. No. Okay. Next topic, my gut application from System Pool to Linux Pool on private gates. So the scope is the three bots that we are hosting. I assume we weren't able to work on this today and it was a long weekend in France. So I assume we have to work on this one. Is it still okay for you, Hervé, to take it this week? Yes. Okay. This one. Delayed by core resist. And long weekend. That one should solve the issue of System Pool using too much data as reminder. So that one is important and we should max it out. Can't create account as usual. I will pass on this one that needs to be checked. Add the chamber to agents. Hervé, could you give us a quick status? It's your big central task. And I've got this module installed on a Nano server image, but I've got two errors. Not looking, but two errors. When I run this module from Python directly, it doesn't work when I call launchable alone. I'll try to put the DLL corresponding to the two functions. I see in the reward in the system 32 of windows, but then I have another error related to core CLR, which is power shell core. Okay. Do you think that we need to say okay, accept that we will have to use windows server core instead Nano server? I don't know. Maybe it's acceptable to get two errors listed in the log. If my chamber is working as intended, maybe we don't care about these two errors. I don't have any information about it. Okay. Can you check with Basil if it's working for him in the current state? I propose that we release your latest changes with the DLLs to at least fix some of the errors. When I'm adding the DLL, this error is blocking. I'm sorry I misunderstood. The question is with the currently deployed image version, we should have launchable with the initial error, but that should not be blocking because it still looks like it's working right? It seems so. Okay. Can you check with Basil the next steps for testing and if it doesn't work, we'll be using windows server core and that's all. The impact means a longer startup time for the agent on ACI, but that's something we are going to have in the future because we need to move these images to Packer, which require building on windows server core. We cannot use Nano server for all in one agent. At the moment on time, we will have to migrate windows server core. On the perfect world, we should do that once ACI workload will be moved to Kubernetes workloads because that means once we have the image on a single node, the image will be cached, which is not the case with ACI, which don't know the image on each call. That should make it pretty invisible, but if we have to, we'll have to start with windows server core error at first. Is that okay for you? Or do you want to hand over the task because it has been a lot for you on that part? No, it's okay. Okay. Let's... Zoom is capturing my keyboard. Let's check with Basil with the current state if it works. Otherwise, we'll have to use windows server core as base image. Just one quick question. Part of this means fixing the Docker in-bound agent repository. Do you think you will have time to work on this today or tomorrow? It's not blocker, it's just a question about your timing. Yeah, it's okay. Thanks. Artifact caching proxy is unreliable. So, we add the DNS issues below, haven't seen more issues on Digital Assign, which is a good news, and the Azure port should be solved by the CI Genkin SIO migration. However, based on the latest changes that Team Yakom did on the Azure VM plugin, we should be able to already migrate the CI Genkin SIO agents, at least the virtual machines, to the same network as the ACP. So, that's why I'm keeping that issue open. That should be separate tasks because CI Genkin SIO virtual machine migration might be done quickly, or maybe it will take weeks. I'm still not sure. So, if you don't see any objection, I will work on migrating the fmr agent or anyone interested can take, but as part of the Artifact caching proxy reliability with the new inbound Azure VM. So, that's why I'm keeping that on the incoming one. I will add a command feature. It should be able to migrate CI Genkin SIO fmr agents to the ACP network. No question. The next one, add a cluster public gates. I raise it okay for you to start working on this one. I don't remember. I might have said I wanted to start on it and I did not have any time during the past week. Yeah, we have to make the migration plan and start playing it. Okay, we can do one service per week or that kind of... It's a rule of thumb that should not be a problem for most of the services. Plan migrations, pick a service, move it and iterate. Okay, is it okay if I add you and if you need help for review, don't stay to ask. Looks good for you? Clean up and import and manage Datadoc monitoring in Terraform. Can you remind me the status of this one? I haven't worked on this. Several unmanaged or duplicated Datadoc monitoring and I've created this issue for Terraform. Okay, do you think you can work on it on the upcoming milestone or should we put this one back on the backlog? We can work on it. Yes, okay. So we keep it for the next milestone, temporary name resolution, failure and plugin bomb builds. So we need more investigation on the core DNS. Embedded in the EKS clusters. So we already removed Datadoc agent custom custom props that we're making request outside on CIKs need to do the same for EKS public.  Still, we still need to understand why core DNS component is not able to resolve all the DNS requests outside. Is it because not powerful enough? Was it a transient error on the AWS network? We don't know. We need to go deeper. There isn't anything of use for now. We talked about security groups blocking the outbound DNS request. It does not look like it's the case. So we'll have to try a bit more with debug container on the node pools and see what happens. I think that's all for the current milestone. Stefan, I talked to you about IAM 64. Yeah, probably forgot to put it on the milestone. Okay. So I won't put it on this milestone since it's not finished, but can you report just where you are? We'll add it on new items. I managed to try the new Azure IAM 64 VM that are built by a packer. And now they are used on infra.ci, the Jenkins.io. It's working great. So we will be able to remove the Amazon AWS last element in the infra.ci with those last IAM 64 VM. So it's for infra.ci and packer. Next step for infra.ci remove AWS IAM 64. Okay. Is that okay for you to then continue on CI Jenkins.io and add the new template and remove the former AWS? Yes, it was pleasure. Do the same Azure for IAM 64 and once it's done, let's smile, clean up packer image to remove any AWS code. We don't want to build AWS virtual machine image anymore. Yeah, we have the CLI update too. Looks good for you to add this on the upcoming milestone. On the upcoming task, I'm going to work on this because I already started Ubuntu 22.04 campaign. I've removed this one because I knew I was off, so no time to work on this one. So the next step here will be the node pool on AKS. Since we are playing around with the system pools and everything. The goal is to see and start creating new system pool and new node pools and migrate everything in green-blue to use Ubuntu 22.04 instead of 18 on this one. I won't go further on that topic because we already have a lot of things. Do you have other topics that you see that you will want to work on on the upcoming milestone or that looks like important for you on the backlog? Let's have a look at the incoming issues. We have a new issue registered with the wrong email accounts, password resets. I don't see another new triage issue. Anything else you want to work on? Do you have other topics you want to bring? I'm going to stop sharing my screen. If it's okay for all of you, we'll see each other next week or later today or tomorrow for the people who work with me. Have a nice day, have a nice week. Thank you.


The Ministry of Health's dental unit is embarking on a campaign focused on proper dental hygiene in St. Lucia. The brush well tooth brushing campaign will engage students within the ages of four to six years old. Senior dental surgeon Dr. Sherry E. Frim Lacompt says the goal is to raise awareness of the importance and need for good oral health among children. Kindergarteners and grade one students are not too young to learn that proper dental hygiene practices can help prevent bad breath, tooth decay and gum disease which ultimately increases their chances of maintaining a healthier mouth and teeth as they grow into their adult years. Dr. Lacompt notes that through the campaign infants will develop the required skills and habits needed to ensure a healthy mouth is maintained. The objective of this activity is to support parents in their efforts to keep their children's oral health in good stead as children at this age do not have the manual dexterity to brush properly and some parents do not have the time on a morning to help them brush their teeth. The activities will include coloring of the teeth which helps expose plaque oral health education and teaching modified tooth brushing techniques. The Ministry of Education has given its full endorsement to the campaign. In the past collaborations such as these has led to lasting effects in the lives of our students. Today the Ministry of Education applause the dental service unit for this initiative and it is a very innovative idea which will create and increase awareness of good oral hygiene among our students. Puberant Secretary Jenny Daniel heaped high praises onto the unit for its innovation. Now I want to pause a little to place a special emphasis on the fact that we are proud of and me in particular because as Puberant Secretary of the Ministry I must say that I don't feel leaned upon to any great extent by the dental services unit. It's almost as though they don't exist but very quietly they trudge on and they make landmark very very very inspiring and impactful initiatives and achievements in so far as dental health is concerned and they deserve a round of applause. Health Minister Honorable Moses Shabbatis also echoed words of commendation to the dental unit for their strides over the years. The brushwell to brushing drive will incorporate an island-wide coordinated effort with other units of the Ministry of Health and the full support of the Ministry of Education. All school children in the kindergarten and grade one classes will have the opportunity to be engaged and participate at the same time as the tagline says brushing together for life. The brushwell tooth brushing campaign which will be undertaken at schools around the island is sponsored by the International College of Dentists. Director of Global Operations Chelsea Segrin says the ICD is extremely proud to be part of the initiative. We are confident it will reach its goal of raising awareness of effective tooth brushing techniques to foster better oral hygiene practices and overall health in the children of St. Lucia. Dr. Lacombe was recently bestowed a prestigious fellowship from the ICD for continuous efforts in the field of dentistry. The ICD, through their Global Visionary Fund, continues to provide support for projects such as the brushwell tooth brushing drive. For the Ministry of Health Wellness and LLA Affairs, I'm St. Justin.


What was the crux of the Red Zone in you today from the first drive and then a couple minutes before the quarter? Yeah, you've got to find a way to get in the end zone. We knew coming into it, that was going to be important for us. And you don't have to take a look at it and see exactly what the issues are. But we had our chances. We moved the ball. We put ourselves in great position a handful of time. Special teams opening kickoff, you know, put us in really good position. And when you're playing a good football team, you know, in an important game, you have to find ways to score. And you have to take advantage of those opportunities. And that's something we have to be better at moving forward. But we'll take a look at the film tomorrow and try and make those corrections as we move forward. Still a chance, you know? So you're saying there's a chance, isn't that the old movie line or whatever? Listen, I've seen crazy things happen in this league. And so we've got to hold up our end of the bargain. And that starts next week. And so when we get back to Atlanta, we've got to find a way to have a really good week of practice, get ourselves in position to play well next Sunday and keep that narrow focus and see what happens. What does that say to you? Got to be better, you know? One of seven is not good enough. In short-yarded situations and, you know, where you just need a yard, we've got to find a way to, you know, really convert those at a high level. And good football teams do that. And we weren't good enough at that today. And so it's got to be an area of concern for us and something that, you know, we work on this week. But, you know, I wish I had a better answer for you. We've got to be better. Why is it that the best version of the Falcons hasn't shown up consistently in those games against the best at home? I don't know. I think I keep going back to the, you know, we have to be opportunistic. We have to take advantage of the really good opportunities that we have. And in important games when you're playing good football teams, you know, they take advantage of those. And we, you know, we haven't done that. And so, again, it's got to be, you know, a point of emphasis for us as we get back into practice. You know, when we get turnovers, when we get short field opportunities, it's got to, it's got to be blood in the water. You have to go attack and you got to find a way to get the job done and get in. First of all, the quarterback only answered? I didn't see, I mean, I'll always say yes, but I didn't see the, I didn't see, I mean, I only saw a little bit from outside there. It looked tough to tell. From the outside, you would look in at that first, the team should be able to run the ball full time and get a half a yard. Why is that harder than that? You know, I'm not sure. You know, again, we have to look at, you know, specifically when we watched the film of where we didn't operate as well as we needed to.  But we'll take a look at it and, you know, we'll try and rep it and improve there. And sometimes, you know, you just, you have to harp on the smallest of details in those type of situations. The combination blocks, the physicality, or even in the passing game, just the level of precision has to be higher down there. And, you know, that's something we need to work on. I guess that was the question, why throw it twice? I suppose when you pound it and it doesn't get in, it makes you reconsider what you're doing. And so, you know, as a player, I never overthink those things. Our responsibility is whatever play comes in to make those plays come to life and to make them work. And I think we can, you know, whether it's run or pass, collectively be better. And, you know, there's no rulebook on what you have to do in those situations. We just, whatever is called, we need to find a way to get it done. And the early on there, like you were telling us, doing that motion won't know what you mean, but what's happening there? Yeah, I mean, those, they happen once in a while, you know. It's just part of it. You have a motion on. I was a little bit down on the clock, and that's just, that's part of the game from time to time. Yeah, I think, you know, I think the level of precision goes up. You know, the space is smaller, and so the windows become smaller.  You don't have the backup threat. And so you have the sidelines and the end line that kind of make it cramped. And so there is a level of timing and spacing that has to be there. And I think, you know, at times we've been good at that through the year. At times we need to be better. And so, again, you know, I think we need to rep it. We need to practice it. We need to get to work. It's the only way, you know, I know to improve. And we certainly need improvement. I am, you know, I feel like I've done everything I can to protect myself. And, you know, I have my booster and I feel comfortable. So I feel good about, you know, myself and what I've done to try and protect myself as best I can. Yeah, I think, you know, I don't get involved in those things. I never have, when it comes to, you know, those kind of decision making things, I kind of generally let that to the people who are more involved who understand that stuff better than I. I usually just whatever the rules are, try and follow them as best I can and make sure, you know, in a case like COVID, I, you know, I've tried to do everything I can. And what I feel comfortable with to protect myself.


So we'd have 6000 each or 7000 each depending on the age if under 50 or older 50. So enter the amount shown below that applies to you and then we have these are kind of our thresholds now that are going to be starting to apply the income uh thresholds. So single head of household or married filing separate and you live depart from your spouse for all of 2022. You enter the 78,000 so married filing jointly 129,000 in both columns but if you checked no uh on either line 1a or 1b enter 124,000 to the person who wasn't covered married filing separately and you lived with your spouse at any time during 2022-10,000. So you're going to be severely limited as is often the case if filing married filing separately because the iris is going to be kind of suspicious of those scenarios. Now I'm not going to go through each of these items more uh one by one so you kind of read it over if you would like to but we're going to be then kind of taking these general ideas and then applying them to tax software in a future presentation. Let's just I'll try to just kind of recap the general scenarios that might come up. So let's say you're married your spouse has a 401k and you're modified adjusted gross income. This is the adjusted gross income you know after the above the line deductions but it's kind of modified so it's still a general idea of your adjusted gross income. In 2022 if you are married filing jointly and your spouse has a 401k you can take the full deduction for the IRA contribution as long as your modified adjusted gross income is below 204,000. I believe if your modified adjust gross income is between 204,000 and 214,000 you can take a partial deduction. Meaning it starts to phase out at that point. So let's say that you are married you have a 401k and your modified adjusted gross income is below the threshold. So now you're saying you have the 401k plan. So if you are married filing jointly and you have a 401k you can take the full deduction for your contribution to a traditional IRA as long as your modified adjusted gross income is 109,000 or less in this situation if your modified adjusted gross income is between 109,000 and 129,000 you can take a partial deduction. So then we have you are single head of household or qualified widower. So if you have a 401k or other retirement plan at work you can take the full deduction if your modified adjusted gross income is 68,000 or less if your modified adjusted gross income is between 68,000 and 78,000 you can take a partial deduction. Note that in all of these scenarios the workplace retirement plan doesn't have to be a 401k plan. It could be a 403b or whatever kind of qualified retirement plan. That's just the kind that most people have so that's the general concept. So then we have if either your or your spouse is covered by a retirement plan at work and your combined income exceeds the threshold set by the IRS you may not take the full tax deduction for your IRA contribution for married taxpayers filing jointly. The limit is 129,000 if you have a 401k and 214,000 if your spouse has a 401k for single taxpayers the limit is 78,000 and of course the general IRA contribution limit is going to be the 6,000 per person generally and 7,000 if you are 50 or older as we can see in our worksheets here. Now again that whole scenario of course is quite complex so the general the general rules would be you know the IRA contribution 6,000 if you're under 50 7,000 if you're older 50 if you're have access to a 401k plan either you or your spouse it's going to get more complicated in those scenarios and as your income goes up you may have limited limitations and phase outs to be able to access the IRA so you will be able to do tax planning and the software can kind of help us out with that last minute tax planning would be the general scenario that you might want to kind of have in mind when these questions come up as they do because the IRA is always going to be that last minute type of thing that you could take a look at.


To integrate Zoom with Moodle, we first check out if there was any existing Moodle plug-in for Zoom. And we found one in the Moodle plug-in repository. This plug-in is developed by UCLA, University of California, Los Angeles. So our journey with Zoom started by installing the UCLA plug-in to one of our test sites and we started testing it. On top of that, we started talking to course creators, administrators, teachers and several people about their work processes and requirements on the technology. So basically how the UCLA plug-in works is that the activity creator is automatically set as the host of the meeting. In our case where course creators are not necessarily the teachers of the class, that wouldn't work out. So we basically needed something where the course creator can either assign a host if the teacher of the class is already known, or that could be left blank for later, let's say till the start of term. Another thing that wasn't covered by the UCLA plug-in was any handling of exceptions, say a teacher fell sick or a guest lecturer wants to appear in a class. Zoom itself provides a feature called alternative hosts, but that feature wasn't supported by the UCLA plug-in. So we decided to create our own plug-in, but not from scratch. We simply forked UCLA plug-in to create our own plug-in. Then we developed our require features and we also did some improvement. For example, we handled automatically Zoom authentication. So imagine the teacher click on the link, start meeting, and within this link we include a token that we grabbed from Zoom API that's going to identify the user when it starts the meeting. So no need for the teacher to type any Zoom login or Zoom password is already automatically logged in. So challenges, Zoom was a new system and never used it before, never heard about, so we first had to learn how to use it. Then we map our user stories with Zoom technologies. So Zoom was a new technology for most users involved. So a lot of training sessions and comms had to pave the way for a successful launch in May this year. The launch went very well and we were able to gather feedback for further improvements from the pilot group. Of course, successful launch always attracts like other people or other colleges got interested in using the Zoom plug-in. With new people interested, they always come with new requirements as well. So one of the major things that wasn't covered by version one of the plug-in was any handling of lecture recordings. So what we basically wanted to achieve is that the teacher can simply hit the record meeting button and the recording becomes automatically available in Moodle for the student afterwards. For us, it was very important to have a secure solution to store the recordings and to protect the privacy of the students. So we identified few issues keeping the recording in Zoom Cloud. The first one was all the recordings were under the teacher account. So what happened when the teacher left or decided to delete the recordings? There is no branding, which means no watermark and no... And the cost to keep your recording in Zoom Cloud is just prohibitive. So we decided to move our recording to AWS. Why AWS? Because it's the biggest cloud provider. It's available in each region our businesses are and we already work with AWS. So let me show you our solution, our stack we have in AWS. We can deploy the solution in less than 30 minutes using CloudFormation script. So all our recordings are stored in S3 bucket. We use Elastic Transcoder to add watermark and copyright statements. And to download or stream the video within Moodle, we use AWS platform. With this service, our S3 bucket are not publicly exposed. And we provide a whole bunch of statistics regarding our usage. With this solution in about 30 minutes after the class ends, it's already available for the students in Moodle. So how much does it cost to run that in AWS? So using AWS calculator, let's take an example. You have 1,000 videos, one hour each. Each video if you 100 times. It will cost you around 2,500 dollars. So let's have a look how Zoom activity actually looks like in Moodle. In the top area you can see basic information about the Zoom meeting. And in the bottom area you can see how recordings are displayed. So version 2 with recording went live 2 weeks ago. We're still gathering some feedback regarding the recording management. And we're now going to start the process to submit this new plugin to Moodle HQ. So we can share it with the community. So we know this was quite a short presentation. And we didn't cover every detail, as much detail as we wanted. So if you have any questions, please ask. Thanks. I'm actually from UCLA. And I work with Rex Lorenza, who together with the rest of our team created the Zoom plugin. And we worked extensively modifying it to take care of some of the use cases that you mentioned as not being covered. So I just wanted to make sure we were on the same page. You said that our plugin made it impossible for the students and or participants to enter the meeting if the host wasn't there. Is that correct? So you needed to have the host. The host had to have the meeting going before the participants could enter it. I'm not quite sure what you mean. But now we were talking about that the host is automatically locked into Zoom. He or she doesn't have to log in when you hit the start meeting button. You don't have to log in to Zoom. The system does it for you. So I think I was talking about something slightly different. But I guess with our system, a lot of our instructors use Zoom by taking advantage of the recurring meeting option. They go in, create a meeting, start it. It's open forever. And they also then choose the enable or allow students to enter before host. They can go into that meeting space anytime they want without the instructor knowing. So for some instructors it's quite a useful feature. And my sense was that that wasn't... I think what we had to change was when we create Zoom activity, you need the creator of the Zoom activity has to be the host. So the teacher has to create an activity. So the problem we have is when we create this activity, we basically don't know who's going to be the host. But we need to create an empty shelf, an empty activity to be here. And later on we will set up the host. The problem persists. Thank you. Other questions? We probably have time for one or possibly two questions. No? Just quickly, that fee again, two and a half thousand dollars. Annual, monthly... Per month. It was per month. When you generate first month if you create one thousand videos. Okay. Thank you Nick. Thank you Andrea.


Daj bomo dobrati tega poslednjanja. Zdaj, različno je zvojno, da je plenskej, vsakom plenskej, z tvojim početem poslednjem poslednjem, ale ja ne moram pošli soveni počet. Sve, da je zdaj. Na tvoj zelo milenijom BC je način karakterizovati, prezens in ideologičnih in motivativnih in traktivnih netorov, ko je spravil na Evropa in včasnih. Tudi te netorov skupili materijali z kompleks arkelogičnih asemdečnih, v kateri včas trajev in praktičnih je vzvečil na vzvečnih arkelogičnih asemdečnih. Vzveč je tudi adriatic setina fenomenon, kompleks pattern in long-range intertvine in traktičnih, karakteriz empty terrenija in terrenija, to ima izvanjo kada je. dochodnjo karakteristiko taj sokoli tenovog stavina kulturina, včetko je najbolj v centre delmešča, vzupovovana in splitоде, je zamah bila velika presenj brzovna. Privolj i ramčen nino, in nekaj nimi vizantov je poljondoho, Singolo in multipli inomrbi, tudi izstavil v jazkih, in tudi zvršovana objevačnja tračnja. D interconnected goods are rare, almost nonek existents, in they take shapes of ceramics, broken, and scattered throughout the tumulus. Distinctive t 담erahvistina include, but are not limited to, of course, bowl with thicken and rims, and different types of tankards, old figures, One of the most famous in literature is the so-called Kotoratz type. The mobility of this Dalmatian set in communities is postulated by the presence of these very distinctive ceramic types across the Balkansk special. Most important for us today across the Central Mediterranean This is旁raje map in dedikating this, the spread of these cetina evidences. So I'm going to approach the cetina phenomenon from the point of view of the resources, and their appropriation and the consequent construction of different landscapes, which I call the cetina escapes, enphasizing the cultural and symbolic dimension. As I said, before most of the conclusion that you are going to see here are based on ceramic analysis, but not only of course. So in contrast to natural raw materials resources as are socially productive construction, expressing what people perceive as relevant for their life, no matter if it is about their physical or their social needs. Resources are regarded as a product of social appropriation and construction first, either as material sources, but also as mental perceptions. Such an approach allows to specifically access to societal transformation and also to evaluate resources impact to societal change in general and to understand the construction of these materialized environments. Archeological record, as I pointed out already, suggests that certain communities were fairly mobile human groups who created their landscape by projecting their ideas and views on the world as they find it, putting emphasis on train, tracks, routes, campsites and special places, so arenas. In his lifelong work in the Sibenic state area, as I mentioned the core of this so-called Zetina culture or Zetina phenomenon, Mario, which excavated an impressive number of tumuli concentrated in the upper part of the Zetina valley. His research was undertaken at a very local scale. In 1970s, he was the first to recognize the presence of this special transdreatic tankards of Zetina type, so acknowledging the importance of transdreatic connections, so the connections between Italy and eastern Adriatic. By adopting a cultural history approach, Blago Evovedarica researched the Zetina culture at a wider scale, so encompassing the whole territory of former Yugoslavia and Albania, and putting particular emphasis on overland connections. Whoever is only in the 90s, which characterized by the flourishing of Mediterranean archeology, that the research scale on the Zetina phenomenon became wider and encompassing the entire Central Mediterranean and Greece, and emphasizing especially transdreatic and transunion connections, especially thanks to new excavations in Italy that yielded the great numbers of Zetina evidences. On the other hand, however, this interest for these big narratives and for seborn nature of the Zetina phenomenon resulted in the almost complete neglect of inland connectivity. So the current interpretative model for the Zetina phenomenon emphasizes only seborn connectivity, and of course, this case. The main hypothesis on the nature of the Zetina network can be summarized more or less as follows. So the spread of Zetina features to western Greece, southern Italy, Sicily, and the Maltese Archipelago can be regarded as a fairly homogeneous and single pattern of connectivity that took place at the very end of the turn millennium BC, in which a pivotal role was played by the so-called, what Maran called the Argonauts of the Western Balkans, who spread over the central Mediterranean. According to Maran, which was followed by Bruban, Zetina is the common denominator behind the migration of small groups that move from Dalmeša, that are connected to the circulation of metals. However, according to Rejken Kacela, the phenomenon began pretty earlier about the mid of the third millennium BC, and of course, Balkan and Adriatic sefarer were not the only player in the Zetina phenomenon. So Adriatic sefarer, so this phenomenon of connectivity which has a score in Dalmeša, is only one phase of a more nuanced and complex socio-economic and cultural network. So, as we saw, resources getting strategies and commerce connected to metallurgy are generally indicated as the main driving forces behind this overseas expansion of the Zetina phenomenon. However, the data tower disposal show that the Zetina phenomenon is a far more complex pattern of connectivity and mobility, in which resources getting strategies was deeply intertwined to ideological and social aspects. And this especially in the research undertaken by Fohrenbacher in Palagruža and in Dalmeša, which were recently published. So in the last year, I would say, we have a really huge number of new data to address the really established model. So, as regards overseas expansion, recent research by Fohrenbacher in Palagruža and at Nakovana allows us to reconsider these evidences connected to Zetina expansion. Palagruža is a very tiny islet between Italy and Dalmeša and represents a key site by which to address the significance of Adriatic Union sefaring and seascapes constructions. Lekking odd basic resources such as water and fuel and therefore being unsuitable for a permanent settlement, the islet was used as outpost for the Gargano Church Supply Network. Analysis show that the abundant litics found at Palagruža are of Apulian origin with 97,5% of those recovered made with the church from Gargano. Data from the Spilakeve and the Hillfort of Grad both located on the Nakovana Plateau on Peješac, a peninsula on the Eastern Adriatic Seaboard of Southern Croatia provided crucial information about raw material supplies. From the early Neolithic right up to the Iron Age, virtually all the Nakovana Plateau litics are made of church imported from the Gargano Peninsula. Church trade from Gargano to Dalmeša was a flourishing activity and set in expansion to get advantage of already existing and persistence trans-Adriatic connections ongoing from the Neolithic. However, it can be noticed that Palagruža increased in importance as a seaport during the third millennium BC, becoming a stop on a wildly traveled route. As mentioned earlier the large majority of pottery from funerary context is very richly decorated and it is usually found broken and scattered throughout our tumuli. Preference for the use of decorated pottery in ritual activities is suggested by find on Palagruža where a good number of set in a pot shirt was recovered together with highly symbolic objects connected to bell beaker ideology such as stone wrist guards and faint arrowheads. This finds suggest that the eyes let was not only a simple trade outpost, but that it occupied a significant place in ritual activities connected to the raw material supply and sifying. Set in ceramics found at Palagruža were likely manufactured elsewhere and brought to the eyes let together with animals, water and other goods during expeditions. The first set of petrographic analysis ever available for Dalmeša under taken on set in a pot shirt suggests that there were possibly of course manufactured especially for activities connected to burials and rituals. So in other words they were manufactured to be broken. This is particularly evident when we compare of course ceramic from funerary context and ceramic from settlement. So set in ceramics from funerary context and recovered at Palagruža of course both are of worst quality. The presence so let's keep from the reatic on the other side of the set in a expansion that is the central Balkans the presence of set in a material in the interior of the Balkans Bosnia and western Serbia is well known since the k8s as pottery with set in a features has been recovered in several tumuli and hill forts. Available evidence suggests that the spread of set in a features in the Balkan interior embeds processes of transkrutivation. That is the adoption of cultural practices and their social cultural recognition. In the sense that material like objects and tools and immaterial culture rituals and technologies are generally not just simply imported or copied and blocked but that they instead appear as the result of negotiations which thereby permitted these features to enter diverse social and cultural environments. This process can be well observed in ceramic which represent the most abundant and peculiar archaeological evidence for the set in a phenomenon. For example this is just one among many some vessels from the set in a tumuli in Dalmiša recall double handled beakers which are widely spread put in the Armenohori communities so the so called Armenohori type cantaros which in Dalmiša context contrary to what we can observe in Armenohori are decorated with typical set in a patterns. Phenomena of transkrutivation in pottery production can be observed also in Greece especially in the known site of Olympia. Olympia material indeed reveals extensive interaction within groups with different technological traditions and I have to say that actually all this research project which is focusing more on set in a but also on other cultural group in the Balkans show that transkrutivation in pottery production was extremely extremely important feature and then characterize the relationship of several group from the Panonian plan to Greece. From this overview it is clear that set in is a complex phenomenon in which both seabor and overland connectivity played relevant but different roles in the construction of different set in a skates. Set in a seabor mobility and resource getting strategy as we can observe in Palagruža was associated with objects having high symbolic value connected to ceremonial warfare. Risk guards are also also known from a good number of set in a site in Dalmiša in the wider neighborhood of the eastern Adriatic such risk guards appear in bell beaker context in northern Italy Toscana and Lazio as well as Sicily and Sarginia. In addressing early Cycladic archaeology Brutberg suggest that the existence of a certain association between the use of long boats and the development of a warrior and male-centric ideology in which the use and ostentatious display of weapons had a significant role. Given the importance of overseas connection for set in a communities resource acquiring strategies and their association with bell beaker paraphernalia it is possible to hypothesize that traveling the sushats Palagruža gargan routes to get to the church from Italy was also considered as a means to increase social prestige. During the second half of the third millennium BC Palagruža acquired importance moving so for being a simple stop over and a seaport used in the church trade to becoming a proper ritual place where different ritual activities connected with the journey to Italy took place. Interaction patterns of set in a communities in the Balkans and in Greece on the contrary seem to have been characterized by marked transculturation processes that hint at a long lasting and frequent contacts between neighboring communities which in some cases even share the same areas as we saw for example in Bosnia where set in a tumuli are close to evidence. These relations place less emphasis on the material their beaker ideology and point reader to at least partially share symbolic language. Together with the spread of typical pottery type set in a phenomenon is also characterized by the emergence of different cultural landscape expressed physically by the diffusion of brutal mountains throughout eastern Adriatic and Neonian areas. This can be observed in the northernmost and southernmost peripheries of the set in a phenomenon that is in northern Italy and particularly in Greece where the diffusion of tumuli occurred at the same time when the diffusion of the set in a features that prehistoric social landscape were divided in the territorial blocks occupied by distinctive archaeological culture is recurring in archaeology. However, anthropological research proves that many ancient landscape were more politically and ethnically continuous than discontinuous. In this letter approach the food social landscape with their mobility and migration are termed here in its basic definition of a one-way residential location to a different physical or perceived environment but at least an individual were not a single event but an ongoing process. So both material and material resources played a role in the spread of set in a features by means of both seabird and overland mobility at the end of the tour millen in BC and in the construction of different and intertwined set in escapes across the Mediterranean Thank you for your attention and thanks to these colleagues who I am working with.