Hello, everyone. Welcome to another episode of Open Infra-Live. We love getting the community together on YouTube and LinkedIn and everywhere else that we can reach on the show every week. So today's episode is super exciting because we're going to be talking to some industry experts from Bloomberg, G-Research, Line, and OVH Cloud about their sessions for the upcoming summit. We're so excited to be getting back together in person after being apart for so long and that summit will be in Berlin in June. So we really want to make sure everybody here gets a little sense of some of the awesome talks that will be happening at that summit. So it is in Berlin, June 7th through 9th. We got some information here on the screen and the price increase is the price, the ticket price will be increasing soon. And we're just excited to get everybody back in addition to these speakers today who are going to be a little preview of their presentations for the summit. We also have some amazing speakers from China Mobile, BMW, Volvo, Kakao, tons and tons of great speakers. We've been confirming new speakers even in the past week. So we'll be continuing to let you know who else is coming and when to be there.  So there's a code OILive2022. So if you're watching this on recording, I'm sorry, but you should have joined live. But this is why we love to get everybody on the live stream that we can every week. So we got a little something special for you if you did tune in if you're on live. So this code is something you can use and you've got to use it today. So that's your that's your chance.  The link is that will take you to the event right to registers. This bit.ly slash open info summit that's on the screen here. You can also go to open info.dev slash summit. Just read more about the summit. Look at the full schedule, all that kind of stuff. So that is what we're here about today is getting hyped and ready and prepared for the summit. And to do that, we're going to bring on some amazing guests to talk about what they're going to be sharing more about at the summit. So to start with, I want to bring on Ross Martin, who is going to tell us, dude, where's my VM? Hi, morning, everyone. Yeah, so just a quick bit about me. So my name is Ross Martin. I'm joining you from just outside of London. I work at the cloud engineering team at G research that joined there about three and a half years ago when we created the cloud team. And I personally focus on open stack private cloud and stuff. I've been working with open stack since 2014 where I've built a small public cloud platform in the UK for a company called Memset. It's fairly successful. And sometime later, I've made the research to start work on a Greenfield private cloud project. And we've been having lots of fun. So who's your research? We're a leading quantitative research firm, mostly based in London, but we're actually looking to expand to Texas soon. We create software to analyze and manage large data sets. We identify patterns in this data using the latest machine learning techniques and try and predict future movements in financial markets around the world. As a company, we're a huge proponent of open source software, choosing this route wherever it makes sense to. And we're keen to push back upstream where possible. We've even got a considerable in-house open source team to help us do that. We're responsible for a couple of significant features in open stack recently, mostly in colorizable. Lots of my head is vault integration for managing secrets and TLS encryption for end-to-end API encryption, which is really cool. And we've got about 10,000 lines of code merged now. So why talk about scheduling? So from an open stack compute perspective, like we're running multiple regions, multiple AZs, continuing to expand at a quick rate. Initially, we had quite a simple estate, just some VMs in a single region, but we've been through quite a significant period of growth. And we've been adding lots of capabilities for things like GPU and NVMe. And recently, actually implementing Ironic with lots of different types of hardware configuration or resource classes in our terms. This means we've now got quite a reasonably complex cloud, over 150 flavors. When you couple that many options with some of the internal goals that we have at GR, like the ability to rebuild every server every 30 days and still keep the cloud efficient and running, you know, we've got one hell of a game of Tetris going on. So it has been, it continues to be a challenge. And based on our experiences, we just want to share with the community some of the techniques and features we've used to help deliver this cloud. And I guess, what would we want you to take away? Well, you know, come to this talk, pitched at beginners to intermediate operators, if you'd like to learn about the basics of how the scheduler works, how the traits API works, host aggregates and how they all fit together, as well as some configuration examples and automation examples along the way. So if that sounds interesting, then this talks for you and if not, then maybe it isn't. There's a fair few of us attending the summit this year. So if you see any of us, come and say hello. My colleague Scott and I will be delivering this talk. Dude, where's my VM? Although it should be instance now doing ironic on the Thursday about 10 past 11 on room B09 and will be around for a chat after. Well, that sounds super cool. I was going to ask you a couple of questions. One, did you say you're expanding to Texas? Did I get that? Yeah, it's quite a recent thing, but we are hiring. So yeah, jump to the website if anyone's interested in that area. Well, I'm coming to you live from Austin, Texas right now. So, you know, I'll have to find out more about your expansion plans coming into the great state of Texas. So more of a question for you on your cloud and your talk. You're talking about how you have some different flavors or instances for different workloads, kind of the generic standard sort of virtual machine.  Can you talk about kind of why you need different types like that and how people decide which is the right one to use? Yeah, I mean, for us, at least at GR, we kind of get a generic instance, just like you say, just to run with the MilVM sort of KVM defaults. Our users might use that for development workstations or proof of concept work or kind of simple applications testing. But a good example of like a specialist workload would be a service we run internally called Armada. GR is actually open source this. You can find it on GitHub. It's a tool for scheduling and running batch jobs over multiple Kubernetes clusters and it's capable of scheduling hundreds of jobs per second, so I'm told. So in that example, we would use bare metal machines to get the most performance. And in our case, we'd use Ironic and deploy steps to configure that and to configure the BIOS to give them the best performance. That's really cool. Yeah, I think one of the things that I love to learn about from users, especially, you know, at each summit is all the other open source that you've created and you're sharing that sort of works alongside open stack or other other Kubernetes other popular open source tools to kind of make it all work. Better together. And so hopefully we can we can learn. I can catch up with you at the summit learn more about that. So thank you so much for the preview of your talk also for everybody on the live stream. You should drop a comment. Let us know where you're joining from, you know, we just our guests today we have from joining from all over the world. And I'm sure those of you who are on the live stream are joining from a lot of different countries. And that's always fun to share and get to know how many different countries we're talking to today. So thank you for that. I want to next bring up Styler Tyler Stahecke and find out how he flipped the cloud. I don't know. I want to hear more about this from Bloomberg. Yeah, Mark. Thank you. So while we architected the latest generation design of our private open stack clouds to scale up from the get go, we started our, you know, state like everyone else from a group with a Greenfield cloud from ground zero. No VMs and a moderately sized control plane, almost unsurprisingly, once people got a chance to play with open stack, things really started taking off. What our metrics were telling us is that not only were we growing fast, we were growing really, really fast for anything as a cloud infrastructure team. We had internally witnessed in the past. So we quickly found ourselves needing more controller hosts. There were some parts of the control plane that we had to optimize a bit for our use case and things of that nature. But here we are a couple of years later with tens of thousands of VMs and a single cell and we're ready for more. And the secret driving force behind this is data and metrics. Our founder, Mike Bloomberg, often says, if you can't measure it, you can't manage it. So regardless of whether you're aiming for a modest deployment or aiming to scale out to thousands of VMs and beyond, it's a matter of when and not if you'll start uncovering stress points and be contending with hardware failure, network hits and other kinds of things. So let's talk about one of the stress points we hit when scaling for a bit, because that's particularly interesting and along the lines of what we hope to discuss with members of the community this summer in Berlin. Off to the right is an exciting example of one stress point we found and addressed. So for some backstory, this past summer, we had been developing and rehearsing some open stack point release upgrades to our clouds. And while we were doing so, our metrics started telling us that our databases were starting to build up some pressure. And the last thing we wanted to do was go into an open stack point release upgrade, which required database scheme upgrades with database pressure. So kind of like looking for a quick win. And very quickly, we were able to identify that most of the database pressure stemmed from the Nova metadata service. And so with maybe a hundred lines of Ruby and Python, we were able to codify a deployment that targeted that specific issue. And while it almost dropped our database compute load in half and reduce the request latency for uncached Nova metadata by almost an order of magnitude. And so that's super exciting. What really led us to this issue early and told us where the squeaky wheel was was the data metrics we had collected. And so this dovetails a bit into the second bit of advice that we'd kind of like to discuss. And that's when you have an open stack cloud in production. You're going to need to optimize it, expand it, upgrade it, whatever you may be doing, you can't afford to guess in production. And even when you have a modest cloud, you have to stay agile because open stack has six months release cycles. It's a really agile project that moves quickly. So at the end of the day, you have to cope with in order to cope with this, you need CIC pipelines and you need to be able to spend a development clouds that look and feel like your production systems. When we did that same series of open stack upgrades this past summer, we broke a lot of development environments, finding out what worked for us and what didn't. Fortunately, because we had those and they resemble our production environment so closely, we were able to seamlessly upgrade both staff and open stack on our single cell clouds in a single maintenance window on the weekend with no impact to the VMs, but also zero API downtime or any impact of any sort. And I say that like it's extravagant, but really for us, these uptime requirements are a must and not a nice to have. So we'd kind of like to show the community and work with the community to see what we've done and hopefully convince more people that open stack is a great solution for private cloud providers. Well, thank you. I'm convinced, but I might be biased, but no, this is great. I mean, we have been excited to follow the Bloomberg journey for a while. And of course, being able to share it at the summit is the best place, but it's great to have a little preview here on YouTube and LinkedIn and everywhere. I guess my question really you touched on this already a bit, but you've went from relatively small cloud to just massive, massive scale with open stack. So for people that are at that earlier part in the journey, you kind of mentioned CICD and things like that. But what type of advice do you have for people to kind of prepare themselves for that massive explosive growth that often happens once you give developers access to that kind of infrastructure? Really, it goes back to the, you know, if you can't measure it, you can't manage it. You need metrics on everything. So, you know, the kinds of issues that we see even today still in production environments, it's just not things that you expect. And without the data, you know, you can't alarm, you can't see those issues. You don't know which part of the system to turn to next. So even if you're a small cloud, you really need to kind of have eyes on everything. So, you know, what, you know, what direction you need to go in. Okay, awesome. Well, I think to make sure we get enough time for everybody, I'm going to move to the next speaker, but we're going to bring everybody back on at the end so we can take more questions from the group. But our next preview of their session summit for the Berlin summit, we are excited to hear from Lide and we have Radiff and Yushiro. So please take it away and tell us about your session. Yeah, thank you, Mark. Actually, I think our session has something in common with Bloomberg's like how they blew up the V and how they blew up the cloud. So, last year, basically, we at LineCorp, we started to figure out effective ways to actually understand how our system is basically scaling. And what are the various points, checking points of matrix and what are the various indices which we have in our system. So like based on the SRE Bible, which Google has released earlier and with coordination with our in-house reliable team, we were able to formulate a set of key indicators that can be used to demonstrate how our clusters are performing, how the systems are performing effectively or not. So this result in project was which we called as a cluster monitoring project from our site. We developed it and we were able to deploy it on all the regions which are underline. And we were able to do it for both the test as well as a production cluster. We included various matrix in it, which included not only the API matrix, but also Kubernetes, master queue matrix, hypervisor, control plane and so on. Using these metrics, we were able to integrate the solutions and we were able to readily get the information. And using open source, some certain open source tools and certain in-house solutions, we were able to utilize that data, which we are able to gather and push it to Prometheus. With Prometheus, Elasticsearch and other components, we were able to visualize that data onto Grafana and we were able to then monitor and understand what are the various points where our system is basically working properly or whether there is some problem which we can experience or not. We did have a lot of outages while we were expanding and we are still like expanding and we see there are multiple issues which we can observe. And the objective of this project was to make a sense of all the available data which we have and to understand how these clusters are operating at scale and what we can do to improve the functioning of the clusters. One of the main decision points which we had for our like OpenStack cluster was using RabbitMQ and I think we are all aware of how stable RabbitMQ is at scale. However, we did have multiple outages, not only with RabbitMQ but also with NOVA, IPI and other components of OpenStack and using this cluster monitoring project, we were able to understand our system better and we were able to identify various issues which existed in the clusters. Next, I would like Yushiro-san to explain further. Okay, so my name is Yushiro, working at Line Corporation as an infrastructure software engineer with RayDip-san. So in our presentation, we calculated the ratio for VM operation like create, delete, reveal and so on and API response by using Message2 and visible on Grafana dashboard. So then our team defined some SLO for line cloud platform for the first time and we are now managing our private cloud to achieve our SLO. So here is just an image about the Grafana dashboard. So this is our development environment data for past 30 days so that we can see by visible on the Grafana dashboard, we can understand how many VM is created and how many times a VM is deleted in one day, one month or past six days or something like that. So for our previous presentation, we explained how to calculate them and how to correct from the metrics data. So in this presentation, we will share our operation result for six months and some point and to use these values. And this is a final topic of structure monitoring components for multi-region. So then most of you are using OpenStack and about some private or public cloud service. And the important point is some monitoring cluster. Of course, yes, but the much more important thing I think is the availability of monitoring component itself. So once the monitoring service has that so that no data is consumed and we cannot see that actual cluster behavior correctly. So always getting the monitoring data is the most important. So in our presentation, I will share that actual monitoring structure. So how from the service deployed on the Kubernetes layer or how load balancer is configured or something like that. So for the conclusion, I will say about the expected audience. If you are interested in some monitoring OpenStack cluster or calculate some SRO and some, if you feel some pain points to manage like kind of laboratory or OpenStack as a component and if you want to talk with us about some outage handling or something like that please come to Berlin and join our session. Yeah, that's all. Thank you. Well, thank you. The first question I had actually because I know that line is massive service but certainly in Japan everyone knows about it but it may be that some people on the live stream aren't familiar with line. So can you actually just explain what line is? I think you have 170 million users or something massive application. Like what does line actually do for end users? Okay. It's a messaging app, I believe and many more things. Is that kind of what it's used for? So it's not just a messaging app. It's a full blown ecosystem. So line has multiple services. It includes line messaging but it also has line music, line pay and there are a lot of other services as well which are currently working. There are about 84 million users who are currently using line in Japan. If I remember correctly, Yushiro-san, can you please correct me? Yeah, maybe so. Please go ahead. So we are now providing some new delivery services, kind of demo account like that and a bunch of users are using right now. So the more kind of user is maybe 188 million user and kind of the line message is maybe 4.9 billion messages in daily or kind of like that. Wow. Huge, yeah. So billions of messages every day and music are connecting people and it's all powered by OpenStack. I think that's like super cool for people to know that these everyday services that people rely on can use OpenStack infrastructure on the back end. The users might not know but I think it's really cool. So I do have a question on the actually OpenStack side. So you talk about outages and you had to deal with them and try to prevent them. What are some of the most common sources of outages for people to be aware of? Rabbit and Q. Yeah. That seems to be a common one I hear all the time and I know we've met with your team in Japan and that seems to be the most common. So hopefully we can all come together and talk about that at the summit and how to avoid those kind of issues. It seems like you've found a way to manage through the issues even with Rabbit being a problem. So Rabbit and Q is definitely one of the issues but we are using our project not only to monitor the messaging queue but also to monitor other issues as well. So sometimes for example if we do have an OOM killer on the control plane or for Kubernetes or on a hypervisor or maybe for any particular let's say neutron agent or an over compute site we are able to monitor those things as well and understand and find out what the reason was. So even in the back end if there is a problem like an out-of-memory issue we can at least monitor it on the Graphana dashboard and go back and try to solve that issue. Good. Well thank you. I know that your session I have a feeling will be very popular at the summit. I'll definitely be there. So thank you again. Again we're going to bring everybody back on at the end but we have one more session and a couple of the speakers that are joining us to tell us about it. So this is from OVH Cloud who by the way is one of the sponsors of the summit. So thank you to OVH for sponsoring the summit. And for this session we have a couple of speakers. We have David and Mohan Kumar. So please welcome on the stream and tell us about this session for Belin that you're going to be presenting. Thank you Mark. It's great to be here. I'm really excited for the Berlin summit and I can't wait to be there to share our story about our three years long journey that led us to the point when we introduced our free services to our public cloud offer. And by all three services I mean stuff like virtual routers like floating IPs and external gateway. So about integration of these three components of OpenStack Neutron we'd be talking about on our session. And I think the key point why we'd like to share our story with the community is our unique scale. OVH Cloud have OpenStack spread around 30 regions with 1,000 compute each region. And in total it is more than 400,000 virtual machines running every day in our cloud. And with that scale in mind adding new component or even enabling part of a component since virtual routers were enabled in OpenStack Neutron for a long time. But for us it's not just changing some APIs, it's not just changing some configuration. We had to make a lot of preparation, testing some adjustments to make sure that it works fine. It doesn't break anything currently working. And that the new features are also stable and are in good condition for our customer to use it. So to give a better understanding what we had to go through, how we went from point A to point B when these new services provide for customers, we'd like to take our audience with the journey with us. And we'd like to start with telling how our network stack looked like three years ago. So we'll take everyone back to the year 2019 where our infrastructure was based on OpenStack Neutron. And at that time we have our own networking agent that was sitting on every compute host in our cloud with huge technical depth because we made a lot of our changes to it to suit our need. We also introduced two new types of network, one for public network, one for private network. And it just didn't quite suit L3. So we decided that we can't go with such kind of architecture, we need to change it. We spotted some challenges with changing it. And I let Mohan to talk about these challenges and these changes that we made. Thanks, David. Myself Mohan Kumar, a working as OpenStack developer along with David for OVH Cloud. First, I would like to talk about some major challenges we encountered. First, we need to skip our multiple releases to get working L3 services. Then we need to split agents as well. So to make it adaptive for our own custom, private and external networks. At last, we have to adjust the existing code around L3 agents and drivers to make it fit for our cloud. So these are three major challenges what we encountered. And we like to talk in brief like how we were able to solve the challenges and the issues what we seen in the protection. Then we started our beta. So in the beta state, we see again some stability issues around router state, the agent communication. For example, the agent restart taking such a long time. So it make it other services to get sync with the L3 services and to make it work. Again, we are able to get some other issues basically around the scalability of our cloud and the messaging queue around agents and drivers. So in this beta state, we are able to mitigate these issues with the help of our expertise scene. I will encourage and invite audience to get our experience and the scalability issues and how we are able to solve those issues. So again, to highlight few of our future plans, so we soon moving from beta. And we also try to add a few services using L3. For example, we try to add HRJ proxy as a service using our L3 services. And again, the next future plan is something like update to new OpenStack version. So currently we are running a stain in our protection. We soon try to move to our recent OpenStack release to make use of the recent code bug fixes went through around L3 services, scalability part. So to make better use of the code change, we try to soon move to new OpenStack version in the production. Great, great. Well, thank you for giving us a preview. I wanted to go back and double check the number I heard you say. You said, I know this is one of the biggest public clouds in the world, OpenStack powered, of course. How many machines did you say you're managing with OpenStack? You mean virtual machines? Virtual machines, yeah. More than 400,000. Wow, that's incredible. 400,000. Well, that's insane scale. And it sounds like you've learned a lot along the way. You were talking about the L3 services. And why didn't you start with that at the beginning? What caused you to approach it in the sequence that you did? So basically, there's like historical reasons. But OVH, when we started with kind of public cloud providing, first we had our something that we can call bare metal as a service. It has nothing to do with OpenStack or with Ironic. It was our own software. And in bare metal world, like when the client orders a bare metal host, he gets this host and he gets the IP address associated with it. And then when he gets to that host, he can list the IP address, he sees it. And we have a bunch of clients using that model, very happy with it. So when there was a decision, OK, we want to provide a virtual machine-based cloud. And which is OpenStack, we decided that we would like to have similar experience in terms of networking for our clients. So then we decided, OK, so in OpenStack, there is like this. You have a private IP and you are hidden behind either external gateway or hidden behind floating IP. And it's not the same as in bare metal. So we decided, OK, we need to have our own network type, our own agent that would suit this. So it would be similar experience. So that was from the beginning. But then we had clients coming and coming from, for example, different OpenStack-based clouds that they have some automation. They say, hey, my automation is not working. I need to change it. So we decided that we should have two models working side by side. OK, yeah, that makes a lot of sense. And probably a lot of other users out there have similar kind of customer requirements or they started with other environments before they embraced OpenStack. So that's probably a pretty common scenario. I think I was super lucky to visit the OVH headquarters in France a few years ago. And one of the things that I learned about OVH is that there's a lot of innovation, not just in the software and the services, but actually in the hardware, in the data center. So I don't know if you all can talk about that a little bit. But I think it's really interesting the way that OVH actually manufactures its own server designs, data center designs to drive more efficiency. Do you all want to talk about that at all? Yeah, I'm not an expert, of course, in that part of industry, part of OVH cloud. But, yeah, we are manufacturing our own servers and our own cooling to drop the cost. And, yeah, it's kind of like water cooling. So that helps drop down the efficiency, the power consumption and drop the user of our electricity bill, basically. Yeah. Green cloud initiatives. Yeah, I think that there's a lot of people that are very interested in how we can be more efficient with our data centers given all of the challenges we have around the world with power consumption and climate change and all that. So the Green Data Center, I think we actually have had episodes on that and may have some future ones on this for the show because it's a topic in and of itself. But I know I kind of put you on the spot on hardware, your software team, it's just a cool part of the OVH story and for those who aren't familiar, it's kind of fun to talk about. So I think that with that, we are going to bring everybody back on the stream and see if we can go into Q&A here and see if there's any questions that come in from the chat and so if the rest of the guests can come back on screen, this is awesome. We have people from all over the world who are going to be coming together in Berlin in June and I guess my first question would be how many of you have been to Berlin before? None. Okay. Oh, we got one. Sorry. Yeah, no, I came to Berlin. Was it 2019? I think it was the last time. Yeah, great city. Had a really good time. Enjoyed the summit. Recommended. Good. Well, for everybody else, I think you're going to love the city. I really love going to Berlin. They have just incredible kind of music and art and culture and all kinds of other things going on as well as being a tech hub with a lot of super smart developers doing interesting cutting edge stuff. So be great for everybody who hasn't been to have that experience. So I know that many people will be coming to listen to your stories in depth. And I think that, you know, as somebody said in chat, that this feels like a super fast summit on fast forward that we're having today. It's kind of like, you know, every session in three minutes. So we'll have a little more time to slow down when we get to Berlin, hopefully and learn more about what you're doing. But I'm curious what people or topics you're interested in learning about, not just what you're sharing, but who you hope to meet or what topics you hope to dive into. Does anybody want to take that? Sure. I guess just listening to some of the other presenters here today, I mean, OVH Club with hundreds of thousands of VMs is, you know, it's always exciting to talk to other industry experts, you know, in areas within, we're focused on things like private cloud, public cloud, you know, all those topics. But also just, you know, what other people are doing with OpenStack, how they're using it, what issues they're running into. And, you know, what's great about OpenStack is just the sense of community that, you know, provides. There's all kinds of people, as you've mentioned, all over the world. It'll be fun getting to know everyone. You sure? Oh, it looked like you were going to jump in. Do you have a topic you were looking to learn more about at the summit? Yes. For my side, so that maybe the last time I joined the OpenStack summit was that maybe I ran PTC or something. So the very long time. So currently I'm excited to join OpenStack itself. And hopefully that, yeah, OVH clouds about infrastructure and cell infrastructure, I'd like to hear about some outage handling and large-scale rabbit MQ or something, some hyperbiter scaling way. I'd like to know about that. Yeah. Very excited. Good. Good. Anybody else? I feel like I'm sort of repeating the others. But yeah, I think for our whole team with the last year or so, just working on scaling our cloud up, it's the same really. We just want to find out what other people are doing, pain points, you know, the kind of operator wall story sessions that you have. Super interesting to us. So yeah, just coming to learn and find, learn from your mistakes really. Yeah. Good. Good. Well, thank you all to our guests today. I think that we are all heading, I know that we're all heading to Berlin and we want to make sure that everybody else who's interested gets a chance to come.  If you register today with this code OILive 2022 on the screen. And if you go to the bit.ly slash open in for summit, I'll take you straight to Eventbrite or you can go to openinfo.dev slash summit, see the whole schedule. Find out, you know, what sessions you're most interested in. And I think that that is a wrap for our conversation today, but I do want to tell you a little bit about some of the upcoming episodes.  We've had a number of episodes that have been very popular on open in for live about large scale from the large scale. And the next episode is April 14th. And this is 1400 UTC 9 a.m. central time. So you can figure out your time zone, but it's the same time we have it every, every session. And it's going to be a deep dive on Yahoo. So Yahoo is another user that has insane, insane scale and has war stories and scaling tips and all kinds of stuff. So be prepared to come in to that and ask them hard questions because they enjoy that. And I also want to thank all of our member companies. These are the members that are backing the open in for foundation and make everything possible. If you want to become a member, there's a guy named Jimmy MacArthur who'd love to hear from you. You can book a book a time with him at any meeting with him at any time. And he's our friendly guy standing by for anybody who wants to talk about membership. And just remember that we also want to have many more future episodes of open in for live. And you can submit your ideas for episodes at ideas.openinfra.live. And you just go to openinfra.live. You can learn about upcoming episodes or submit your ideas. So thank you everyone. And we will see you next week on Open In For Live. And I will see all of you in Berlin. Thank you so much.


Okay. Good morning everyone. My name is Patricia Kiran. On behalf of my colleagues here who are with us today and the remainder of our team who are in lecture theatres and labs around the country, I'm delighted to have this opportunity to update you on our progress towards advancing global manufacturing competitiveness through Irish engineering graduates focusing specifically on design simulation for the process industries. Now, just to review our project very briefly, you know that it's a discipline specific project. We're focusing on chemical engineering and related disciplines which are obviously of particular importance to the Irish economy with reference to the very strong and the very buoyant pharmaceutical industry in particular. We're focusing on higher level skills, simulation skills as they relate to these two processes and what the objective of the project is to deliver a suite of reusable online resources housed on a shared open platform and they're focused in four main areas chromatography, fermentation, process control and distillation. Linking the four of these areas is the use of industry relevant design and simulation tools, computer based simulation tools. Our project team consists of representatives from all of the higher education institutions in the country offering chemical engineering programs, chemical engineering or process engineering programs, as well as representatives from cognate disciplines including biological sciences, biotechnology and process control. And we're fortunate to be joined by representatives from across the full spectrum of external stakeholders, employers of chemical engineering graduates, developers of simulation tools and users of those tools to give us a 360 degree view of the project. Now, what have we done since we last met with you in June of last year? To summarize what we've done I think I'd like to just look at the structure of the project. Our project is packaged into seven work work packages, book ended by inception and integration, but the bulk of the work is contained in four thematic work packages on chromatography, fermentation, process control and distillation. And the multimedia development for all of these is undertaken in work package two, which is really bringing the whole package together. What we'd like to do is to review our progress on a work package basis starting with multimedia development. And this work package is led at CLT by Dara Coakley and by his colleague Gerrard Osulawain. And Dara is going to start now by introducing the whole process of multimedia development. He's going to illustrate it specifically with reference to the chromatography resources and that package is led by Carmel Hensie from UCD. Okay, so today in terms of the multimedia work package, 34 digital learning objects have been developed. So these range from a variety of kind of formats from video to interactive assessment to animation to simulations and e-learning presentations. And the intention with the project because the project began in March, the intention is to continue develop digital learning objects until the end of the project. So for the remaining six months of it. As well as this, the multimedia development work package has been focused on the development of an online platform, which Patricia mentioned to house all of the digital learning objects. So maybe just quickly go through the process and using work package three as an indicative example. The initial step for all of the thematic work packages was that all partners were required to develop concept maps, which would outline the main kind of learning area and subsections that they wanted to develop learning objects for. Following on from this specification document or specification development documents were developed based on each of the concept maps. And the idea was these with that they would maybe help identify potential learning objects and also associated requirements that would go along with these in particular learning outcome requirements. Once the specification documents were developed, an online form was provided to partners. So this online form was essentially a way for partners to identify quite specific requirements and to give indicative examples of other learning objects which were out there. So the idea with this was that it would maybe help us communicate with the subject matter experts in terms of, for example, if they wanted to demonstrate the use of real life equipment, well then perhaps video format would suit that quite well if they wanted to try and put across something a little more conceptual than an animation might fit that format better. So based on these templates were provided to the partners. So for video was to be developed storyboards, script development templates were provided. Partners fill those in and on the basis of that then the digital learning objects were developed. So for example, in work package three chromatography, there was a requirement to show the use of real life equipment to a large degree. So video was primarily the, or excuse me, video based learning objects were primarily developed based on that. As well as that, there's a web presence available at kemi.ie, a project branding and visual guidelines have been developed. And as well as that, excuse me, as well as that an online learning platform has been developed at platform.kemi.ie, that's ongoing. And this platform was developed on the basis of work done for a previous national forum project telltools. But during the lifetime of the project as well, we've also the partnership has also gained access to an additional learning platform called Grasp, which is based around developing adaptive learning paths for students. So the intention is to continue development of learning objects and also to pilot both of the available online platforms with students. And based on that to move forward with either one or both of the learning platforms as a way of kind of facilitating student learning. So that summarizes the development process illustrated with reference to chromatography. And just to very briefly overview the key outputs in all of the other work packages since the project started in March of last year. In fermentation, which is led by Brian Freeland and by a technology at DCU. They've completed development of a full database of fermentation trials. 16 very data rich fermentation processes. Fed batch runs have been completed. The richness of the data set is facilitated by access to an offline gas, an online gas analyzer and also to an in situ biomass probe. The sort of process analytical technology that's used very commonly in industry, but to which students normally don't have access. Those path tools have been fully integrated into the system and a DynaChem simulation for the fermentation has been developed using that data bank to seed the simulation. There's a DynaChem tutorial in progress and they're also in the process of developing a Nybert bioreactor tutorial, which is going to use the same facilities as at Nybert as we saw in Carmel's chromatography videos. Process control, while the other three work packages are on very specific unit operations. Process control obviously is an area which links all unit operations or which is applied in all operations. That's led by David Darin at Dublin Institute of Technology. They've developed a MATLAB prototype graphical user interface. There's an illustration shown here on the screen for a very simple but deceptively complex process for heating a tank, which can be explored by the students using PID controller for different conditions of PID control. What's nice about this GUI is that it can be customised to illustrate different controllable processes using Simulink, which is another package produced by MATLAB and also commonly used in engineering curricula, not just chemical engineering curricula. Dave and his colleagues have developed a database of online MCQs relating to a series of basic process control principles, and those are being piloted this semester by the process control class in UCD. Work packages six and seven are dealing with distillation, which is really the classic chemical engineering unit operation, and I'm combining the two of these because a lot of the activities are very closely related. Here this is led by my colleague Damien Mooney in UCD, and this has focused on the development of experimental systems and associated simulation-based exercises for students in chemical engineering. We've developed two entirely new experiments, one a simple laboratory-scale batch distillation experiment, one a laboratory-scale batch distillation experiment with reflux, which represents a big step up in terms of complexity, and this was one that we hadn't anticipated at the outset of the project because we just didn't have access to the equipment at that time. We've also redeveloped an existing pilot-scale batch distillation with fermentation experiment, and all three of them have been designed so that they're using the same chemical system. This means that students will have access to three different systems across two different scales and two different modes of operation. For all three of them, we're using a high-spec refractometer for the analysis that was purchased through the project. We have developed, this is a graphical from a standalone simulation, an online simulation, for a McCabe-Thiel method for analyzing continuous distillation processes, and as I speak, there are two third-year students supervised by the post-graduate student recruited to this project who are analyzing data that they collected yesterday evening from the pilot-scale distillation experiment. They're analyzing the data this morning, this afternoon, they will be working on a batch simulation, an Aspen batch simulation, which is one of five simulation exercises that we've developed supported by three tutorials for these exercises. In terms of impact and evaluation, as Dara had mentioned, we only started in March of last year, and due to difficulties in recruiting students, which we had anticipated but which were far worse than we could have imagined given the buoyancy of the job market for graduates in the area at the moment, the National Forum has approved a no-cost extension to the end of this year. We've already begun to make significant impact, and I think where we can see this, first of all, is in terms of dialogue and discourse and in terms of impact on learning and learners. The project has been characterized by very heavy involvement of undergraduate and postgraduate students. We've had more than 20 interns involved in the project so date, interns or project students, three MNJC students, including one working with Carmel who submitted a thesis last semester, and more than 60 undergraduate students have already been involved in testing or in using the resources. Shown here are three of our interns who are working on the distillation and integration package, shown at the facilities of one of our external collaborators, APC. In an unanticipated development to the project, Federico Orifiche, second from the right, the MNJC student, he's going to start a four-week internship with APC next week. APC had come on board as an external partner who employ a chemical engineering graduates who use distillation, chromatography and fermentation experimentally and who also simulate these processes. Federico is going to work with them with the view to developing more industry-relevant simulation exercises for use in the project. In terms of impact on the learners, again, we're at the early stage. We've already begun to collect quite qualitative feedback from the students who have been using the resources, and this is some feedback from students in our core third year chemical and bioprocess engineering module last semester on unit operations who used the ASPIN exercises and associated videos. It's very early, very qualitative, but the feedback has been extremely positive. These are the same students who as part of the spiral learning approach that we're adopting are currently working on the distillation experiments this semester. In terms of impact on teaching and learning practices and a culture of enhancement, there's, again, two entirely unanticipated developments that I'd like to highlight. In this 10 credit core laboratory module, which our third year students take in chemical and bioprocess engineering, this year we've unprecedented three new experiments and two of them I would really attribute to this project. One on protein purification via cation exchange liquid chromatography, which really came about when we were reviewing our treatment of chromatography as part of Carmel's work package. And this is a project, an experiment that's been developed by my colleague, Iskani Jimenez-Delval, and which will require students to use Carmel's resources. A second experiment on pilot scale mixing, a real workhorse of chemical engineering practice, has been new one allowing students to work with a 50-litre vessel fully instrumented in the lab. In three hours they can export lower, only a very limited range of conditions, but they're going to use dinakem utilities, again in a way we hadn't anticipated, to allow them to understand what would happen if they changed other process variables. Impact and sustainability, we're using these resources already. We know that they are of relevance to chemical engineering students and colleagues in all of the programs in Ireland. We've gotten some additional impact through our external partners. We've been invited by dinakem to give a webinar to their international users group towards the end of the year. But really what drives us is the accreditation imperative. All of our programs are professionally accredited. And what we are doing is we are incorporating the learning outcomes associated with these resources and the resources themselves into our accreditation applications for sustainability through and beyond the life of the project. It's been developed by the chemical engineering community in Ireland. We're not working in competition with one another. What helps any of us helps all of us. It's focusing on key unit operations, key learning approaches which are relevant to a very wide range of students. And most importantly for student engagement, they're really focused on the development of professionally relevant skills. The students can see an application for these. The final topic that we really need to look at is the strategy for the platform sustainability beyond the life of the project. But we're using it. We want to use it. We're committed to using it. I'm confident that we'll find a workable solution to doing that. Thank you very much.


Welcome back to my channel and welcome back to another what I eat in a day to lose weight on the WW freestyle program I'm taking you guys along with me today on all of my meals snacks Dessert we're gonna hit up another open house You're gonna see the dogs and I have some halls. I have some fall halls coming your way So if you want to see what I eat in a day to lose weight just stay tuned is my haul from Hobby Lobby I went to Hobby Lobby TJ mags and big lot, so I'm gonna show you guys everything I picked up So first I got this really cute steak for my front yard It says welcome to our patch everything at Hobby Lobby that was fall was 40% off So super cute to put into my yard. I got this adorable. Hello fall wood block I want to use this in my videos and then also just for decoration around my home. I thought it was really cute I also got this gobble till you wobble. I think this is adorable I thought this would be cute as a centerpiece for Thanksgiving Loved this thankful and bless. It's really cute. It's kind of like a picture frame Back and it's just a decoration. I love it and again everything was 40% off couldn't beat it I also picked up some pumpkin cupcake liners. These are actually really really cute and then these really cute fall Autumn leaves sprinkles for cupcakes and of course you guys will be seeing that in my fall baking So that's everything I got at Hobby Lobby. I want to say I spent right about $35 So here's my breakfast for this morning. It is about almost eight o'clock and I'm hungry I've been up for a couple hours So what I'm having for breakfast is first time and finishing up our watermelon that we purchased last week It is delicious. I got it from Trader Joe. So of course that is zero smart points I have one slice of my Sam's Choice turkey bacon. This is all I had left over after meal prep So I decided to go ahead and cook it up in the microwave. That is one smart point I have one whole egg and one egg white that I scrambled up Which is a little bit of salt and pepper for zero and then I'm most excited about my bagel So what I am having is I am having the western bagel. This is the alternative Bagels and this is in the blueberry you guys these western bagels are so good They are full-size bagels Depending on what kind you get the points do range. So let me tell you a little bit about how the points work So western bagel does carry an alternative bagelette. So as you can see these are small bagels So they're just a smaller version of their full-size bagels. This pumpkin one is amazing These are only one smart point So great to have is a little side with your breakfast or lunch if you do have two bagels It does bump it up to three points, but one is only one smart point They also carry a perfect 10 plus multi grain That is two smart points and that is a full-size bagel similar to the blueberry Now the blueberry bagel this one and the cinnamon are my favorite These full-size bagels are only three smart points. That's it you guys So this entire bagel is three points. You can't beat it. I mean nowhere else Can you find a full-size bagel bagel for only three smart points? So I'm going to be having one of my favorite blueberry bagels their pre slice I just toasted it and added just a couple sprays of the I can't believe it's not butter So if you are interested in checking out western bagel I highly recommend it if you are a bagel lover these bagels are amazing You would never know that they aren't just regular bakery bagels So hop over to western bagel.com. There is a link down in the description box below Use my code here on the screen and save 10% right now They do have a build your own box where you can pick your bagel So you could do some bagel acts you could do some full-size And on that you get free shipping plus you can use my discount code So it's kind of a double discount and free shipping. So again, my code is here on the screen Just to let you know my bagel is 120 Calorie so not bad. It has some whole grains. No artificial sweeteners low fat An excellent source of fiber and calcium. So again three points for a full-size bagel So my breakfast you guys Four smart points three for the bagel one for the bacon can't beat it. So I'm going to have some breakfast. Hello Yeah, hi buddy. Yeah, mommy's having breakfast. So of course you're right here And she's kind of back here in the corner lola What do but she's of course over here too because breakfast is being served. Aren't you babe? Yeah, you guys will have some green beans. Yeah, you guys will have green beans when mommy's done with her breakfast My babies it is morning snack time. It is just before 10 a.m I have been busy busy this morning. So I'm going to have myself one of my Chocolate orange cream built bars. I really like this one It reminds me of those chocolate covered orange sticks that you can buy around the holidays It is that good. The orange flavor is on point Delicious. So the situation with built bars are they are three smart points? They are 110 calories with 15 grams of protein six fiber four sugar and four fat I store mine in the freezer. I feel like it just helps keep them a little bit fresher But I usually will take them out of the freezer for a few minutes before I eat them I actually prefer them cold in the freezer in the pantry. I like them always But if you're not a fan of the texture you can throw it in the fridge or freezer And that definitely changes the texture for you. So I always have this as a morning snack with the 15 grams of protein It keeps me Super full which is awesome and it keeps me full until lunch and it's only three points And again full of great ingredients delicious tastes like a candy bar Absolutely amazing amazing amazing. I would not go a day on my journey without my built bars. That is for sure So if you're interested in built bar go ahead and click the link down in the description box My code here is on the screen. It will give you 10 off and free Shipping so definitely get some built bars if you haven't tried them yet. They again are staple on w w They're great on keto. They're just an excellent product and they taste Delicious. So this is going to be my morning snack And my tj max hall mainly revolved around diesel and lola But I did pick up a few fall halloween things. So I found this really cute pumpkin silicone mold This is gonna be for a halloween baking video. I already have something in mind So cute. Love it. I also I had to get this pumpkin. I think it's beautiful It's white with a gold Stem and it says hello fall. I love love love it I got this plate for my videos just so that I can put my baked Goods on this plate. I thought it was really pretty and it was like 499 great deal And lots of dog treats. So I found these natural belly bones dental care yogurt Bones again, if you don't follow my channel or know this about diesel and lola They cannot have grain and they cannot have chicken. So finding treats is really hard And they're generally very expensive. So when I find some that they can have I buy them So naturals belly bones. I found these farmer market natural duck and vegetable treats These are beef and lamb recipe. These are like little beef sticks. They're gonna love those Also, I don't like to give them raw hides unless they're the digestible kind And so I found these pork rolls They're really just pork hide And so I thought this would be nice for them to have to chew on And then this is something that I was excited about this is the happy teeth And these are himalayan cheese flavor dog chews 399 two of them in there. I thought I'd give them To them to kind of last them a little while. They're pretty darn good size So lots of dog treats at tj max and some fun fall stuff So here's my lunch for today. It is just about one o'clock I have to head out here in a few minutes to my open house. So I am having my meal prep my last one from last week So I have three of my basil sun dried tomato Parmesan meatballs so Good so good three smart points for all three I have three smart points worth of my roasted potatoes Zero for my brussel sprouts. It's just sprayed with I can't believe it's not butter spray And my second fruit of the day some grapes. So this entire lunch six smart points Lastly what I picked up at big lots. So I went here looking for stuff for videos fall decor Didn't find a lot of fall decor, but I found some fun stuff So I found these really cute placemats. They're gray on one side and this really cute pattern on the other I picked up two one for my husband and myself our table likes to discolor with heat So I wanted to make sure that I had nice cute placemats for us for dinner My table is black. So it'll kind of get that white Tinge to it. So I grabbed those. These are for us just to have when we eat dinner And then I found this placemat. I thought it was so pretty and really I wanted it for my videos I thought that it would be cute with you know plates on it for photos and things So I picked that up. I want to say it was like three bucks can't beat it Found a couple other treats there. I went there first before tj max. So my dogs are stocked up I found these cheese Bones dental bones instead of rawhide. So I was excited to find these they're rawhide and grain free $7 not bad And then I also found these rachel ray savory roasters. These are 100 beef No fillers no chicken no grain. So I grabbed those for $2.50 Also found a couple really cute plates. So I found this blue Pattern one and this aqua one these were buy one get one half off. So I couldn't pass it up Wax melts. I found these these were three dollars Which is actually a great deal because this is a big pack and this one is pumpkin pancakes. It smells amazing And then I also grabbed the apples and cinnamon which also smells amazing And then at checkout. I saw this cute little apples and cinnamon lotion. You guys this was a dollar Like literally a dollar. I don't know why my camera doesn't want to focus on it But anyways, there we go a dollar at big lots apple cinnamon hand and body cream I'm just going to throw that in my purse and then I found these for Thanksgiving We always host Thanksgiving and these are so cute. They say gobble gobble gobble And these were three dollars at big lots. So these are just to have on hand for Thanksgiving And lastly great deal on paper plates. My husband warms his breakfast up usually on a paper plate 118 of them for five bucks not the cheapy ones. These are good quality paper plates So that is everything that I found at big lots. So here's what I'm gonna have for an afternoon snack I'm actually on my way out to my open house. So on the way out there. I'm going to have one of my Iconic protein shakes. I love these. This is the cafe latte This is a grass fed protein shake with 20 grams of protein three grams of sugar 130 calories and it is only two smart points just like the premier protein It is so good in my opinion Way better than premier the ingredients are better Everything about this is just a better all-around protein shake And the taste is in my opinion a lot better the cafe latte does have a little bit of caffeine So it's nice for that morning or afternoon pick me up So this is what I'm going to sip on on my way out to my open house. It's about a 50 Minute drive from me. So give me time to drink my protein shake if you're interested in iconic There is a link down in the description box. My code is here on the screen. It will save you 15 Percent off of your order, which makes these very similar in price of the premier And again in my opinion a way better product So I'm going to have that and then after my open house I actually have to head to meet another client So I'm going to bring an additional snack just because I don't know when I'll be home for dinner And these are from the protein wise website. They're the protein wise brand. These are the caramel sea salt bars They have protein. They have fiber. They're gluten free and this is what they look like So this is a good size bar. It is that Size and it is only for smart points. So not bad. I'm going to be bringing one of these I really wanted one of my granola bars. I made in meal prep, but this is just a little bit more portable So I'm definitely gonna have that tomorrow because they're really good But these you can buy off the protein wise website. Again, there's a link in the description box It'll give you $10 off your first order. So basically you can pick out $10 worth of free product So I'm going to take this with me and then I'm going to sip on this and that's going to be my afternoon snack Hey guys, happy sunday. I just left my open house It was a flop. It had so many things going against it Okay, first of all, I couldn't find it because my nav took me to a completely different area than the actual house I think the development is fairly newer. So the nav hadn't caught up yet. Secondly, it's a gated community So for the first little bit of the open house, I couldn't get a hold of the agent to get the gate open So clearly nobody can come to the open house if the gate's not open And it's a $500,000 house on a holiday weekend. So it's fine though. I was able to get some work done while I was there So it ended up being okay, but I also wanted to share with you guys what happened yesterday So yesterday afternoon, I was out the entire day looking at houses with my clients So one of the houses that we were at I parked and when it opened up the house Well, they ate their lunch in the car and the husband came and got me and someone had backed into my car Yeah Someone backed into my car.  I'll go out and take some footage Apparently I had blocked his driveway a little bit I didn't even realize it was a driveway It was so narrow and small and the front of my car was in his driveway a little bit But I was parked and he hit my car and he was mad and I was trying to be as professional as possible because my clients are right there But I told him I said I apologize for partially blocking your driveway I didn't realize this was your driveway But you should be paying attention when you're backing out So long story short my husband called the insurance and it's 100% his fault because my car was parked regardless If I would have been 100% blocking his driveway or not It still would have been his responsibility to see where he was backing out So now I have a dent in the side of my car and that's just another thing that I need to deal with so Craziness, but I wanted to let you guys know what had happened with my car and I'm going to take you outside right now and show you the dent. So here's where he hit my car I don't know if you guys can see that It's so sunny. I can't even tell if you can actually see it, but there's a large dent right here Tons of scrapes and things down the side of my car. It was an f-150 truck And it looks like he may have even potentially buckled a little bit here Of my hood. So Happy day to me. So here is my dinner. So I have one serving of the gnocchi This is a total of four smart points I have two of the turkey meatballs that is three smart points for two meatballs And then I have half of a serving of the spaghetti squash mix An entire serving is three quarters of a cup and that is four points So I decided to only have half of a serving for two So I have three for the meatballs two for the spaghetti squash and four for the gnocchi So this is my nine smart point dinner Bring me on another what I eat in a day on the w w freestyle program to lose weight I hope you enjoyed spending the day with me I love taking you guys along with me letting you know what's kind of been going on in my life playing a little bit of catch up Love it. So I hope you enjoyed coming along and if you're new to my channel, hello Welcome. Thank you for coming by make sure that you hit that little subscribe button and that bell That way you'll be notified every time I upload a new video. You don't want to miss a single one Please give this one a thumbs up comment down below Let me know what you thought of this what I eat in a day And if you have anything that you'd like to see in these vlogs, let me know as well And I'll see you guys all in the next video. Bye