T elections 6-5 arkani milila. No kutuna nka hivatsihida kukuna kutuna nk Lagini, kutuna nutunga, kutuna ninawa waka. kutuna ninawa waka, kutuna ninawa waka W لحi aussi wakinii. Nakao wasi kfani wa mountain bihaka IUWI toteo dew으면. Aka kawalaibiyavu na udi mweniwe. Na udi wajayi, Afri skaw finallyevu wakinii na unidushia wajayi. Qiltia wajaya pwaniqikia. Afri skaw amama'u waj drugs za fabanya kwa nama kwa udi wajaya hiraya wajaya pwaniqikia. Sipilasha wajaya, masama'u mweniwe na udi mweniwe. Tisha wajaya, wajayi waja mwa mweniwa kwa mwaniya ngami udi wajaya hila mwena mwa wajaya. Most of the time, drought occurs and for example last year we had a drought and almost you almost lost 200 kettles, you almost remain only with 65. With Africa, I can walk 30 kilometers. The app has helped us to know where to find pasture. Understanding that the density of cell phones in particular Kenya is growing quite big and that people are starting to transition from feature phones to smartphones. It was important to now create an application. The thing about Africa that is unique, it has been co-developed with the pastoralist. So some of their traditional practices have been incorporated, their migration areas as they know them have been incorporated and then also some of their cultural practices are in there. Since I'm born a person from this area, I know the place like the back of my hand and sometimes if I get some pasture for my animals and there is no water, I alert my people and if there are predators I just alert them using my app. The app has also helped me in reducing the distance I'm walking and also it has reduced the cost because I just open the app and I can quickly know where I can move to. When we were developing the technology, there are a couple of challenges we are going through. In some of the areas that we are working on currently, the density of data connectivity is quite low. Then also illiteracy has been a bit of a challenge. We've tried a lot in the app development to use iconography so that people can be able to just see the icons and work. This really improved our lives as pastoralists and the mobile technology has really made a big change in the pastoralist world.


This training is significantly important because 2CR is basically, you would say, like the guardians of Europe. You know, the soldiers are conducting PCCs and PCIs. Their leadership is making sure that they're adequately prepared, preparing their vehicles, PMCSs, like that once they hit the box is hot. This squadron is focused on readiness at this time. So there, that's part of derotation and part of their evaluation to make sure that they're prepared for whatever's going to be coming up next. Our presence in Europe is significantly important. I think these exercises, because we're not only doing them alone, we also have multi-national forces that are here with us. And just the presence of us here sharing with NATO partners and allies, it creates a strong sense of unity. Because if we were to go to war or something was to happen, we're not going to do it alone.


Well, I'm excited to share with you this fresh revelation from the word that the Lord has given to me and it's concerning those seasons in our lives Where we sense a distance between us and the Lord now There are times in your life where you will feel like God has distanced himself from you But rest assured that is only a feeling the Holy Spirit abides with us. He does not leave us still Even though the Lord does not really leave us or distance himself from us He still has a hiding nature and I want to talk to you about the three reasons why God will hide We're gonna be talking about that on this edition of Spirit Church, but first Steven Mock to zoom us here He's going to lead you in some anointed worship worship with him and then we're gonna get right back into this lesson here It's Steven Mock to zoom a holy spirit. I call him The hidden dove and we're gonna show you in the song of Solomon Where we find the Holy Spirit's hiding nature revealed now first I want to read a scripture to you so that you understand that all prayer all seeking all worship Everything that connects you and the Lord is birthed by the Holy Spirit. It's inspired It's invoked by the Holy Spirit the scripture says in Psalms Chapter number 27 verse number eight My heart has heard you say come and talk with me and my heart responds Lord, I am coming So first the Lord will call unto you and then you are summoned into prayer first comes the royal invitation Every time you've had a desire to pray It was an invitation from the Lord for all spiritual desires come from the Holy Spirit So when you get a desire in your heart to seek the Lord when you sense that gentle nudge of the Holy Spirit Pressing you into the places of prayer know that he is drawing you closer The Holy Spirit Has to invite you into the depths of God's presence before you can go and you may say well There are times when I desire to be in God's presence and I don't think the Holy Spirit has spoken to me Well, the truth is that that desire itself is the Holy Spirit drawing you for without him We have no desires that are spiritual whatsoever now looking to the song of Solomon I found the person of the Holy Spirit in the song of Solomon and he's represented symbolically now in the song of Solomon There is a conversation between two lovers a man and a woman And the young man represents Christ and the young woman represents the church It is a prophetic parallel for christ and the church So the Holy Spirit actually appears a few times in the song of Solomon First of all in song of Solomon chapter 8 verses 6 and 7 the scripture says Place me like a seal over your heart Like a seal on your arm For love is as strong as death. It's jealousy as enduring as the grave Love flashes like fire the brightest kind of flame Many waters cannot quench love Nor can rivers drown it The Holy Spirit here is the unquenchable fire of God's love fire is a symbol of the Holy Spirit The Holy Spirit is also found in song of Solomon chapter 4 verse number 16 where the scripture says Awake north wind rise up south wind blow on my garden And spread its fragrance all around Come into your garden my love taste its finest fruits The Holy Spirit is also represented in scripture by wind And here the Holy Spirit is the alluring wind That spreads around the fragrance that brings together two who love each other and to my point here on this lesson The Holy Spirit is also found in the song of Solomon chapter 2 verse number 14 remember The young man represents christ and the young woman represents the church The young man tells the young woman in song of Solomon chapter 2 verse 14 My dove is hiding behind the rocks Behind an outcrop on the cliff Let me see your face. Let me hear your voice for your voice is pleasant and your face is lovely now He is saying that the dove is hiding and he uses this language to try to draw The young woman to come and seek after him notice here That the young man who represents christ has a longing to be with the young woman which represents the church or you He says this let's read it again. Let me see your face Let me hear your voice for your voice is pleasant and your face is lovely now We often use this kind of language in worship songs to express our adoration of the lord But did you know that the lord desires to be in your presence just as you desire to be in his and Again the communication here the language being used here is the dove in the rocks christ is saying That he is giving the holy spirit to the church It means that the holy spirit also it also means that the holy spirit has a hiding nature in fact Other scriptures point to this hiding nature of the dove, which is a symbol of the holy spirit Jeremiah chapter 48 verse 28 Says you people of moab flee from your towns and live in the caves Hide like doves That nest in the clefts of the rocks. So here we see again that doves have a hiding nature In fact in the book of psalms the israelites were said to have found treasures in plunder Among the treasures they found doves so psalms chapter 68 verse 13 says Even those who lived among the sheepfolds found treasures Doves with wings of silver and feathers of gold So treasure is something that is found here again We see that the dove has a hidden nature a hiding nature Now the holy spirit never leaves you let me make that very clear But he does hide and what I mean by that is though he abides near you Sometimes he makes that Unapparent sometimes he makes it seem as though he is not near and he does this For three reasons But before I give you those three reasons Let me give you one more verse to demonstrate that god does have a hiding nature Isaiah chapter 45 verse 15 says verily Thou art a god that hideous thyself. Oh god of israel The savior So why does he hide himself number one?  He is leading you The holy spirit is calling you into deeper waters When the holy spirit hides your desperation and your hunger begin to intensify And something deep within you says I have to search for the lord When you sense a distance between you and his presence and again, it is only a sense You Are driven To seek him with passion. You are driven to seek him with fervor even as I am talking now Within you there is an intensification A desire To seek the lord and that's what happens when you talk about his hiding nature. So number one He hides so he can draw you number two the holy spirit hides to test you second chronicles chapter 32 verse number 31 says However, when ambassadors arrive from Babylon to ask about the remarkable events that had taken place in the land God withdrew from Hezekiah In order to test him And to see what was really in his heart Psalm chapter 14 verse number two says The lord looks down from heaven on the entire human race He looks to see if anyone is truly wise if anyone seeks god In those seasons Where you feel the distance between you and the lord Where you feel like There is a gap between you and the heavenly Where you sense as though you cannot hear god where you don't Feel that abiding warmth of his presence In those seasons where you cannot see god at work The lord is watching you He wants to see how you'll respond now some believers what they do in these seasons Is they grow angry with god they they stop seeking god they stop praying they stop reading the word they stop worshiping They stop committing their lives because they base their lives on feelings rather than on faith And here is the merciful aspect of god's nature and action If you start to distance yourself when the lord hides his nature When the lord hides his presence The lord will come out of hiding and draw you back into where you need to be But he's trying to teach you something He's trying to test you. He's trying to get you to function on faith and not on feelings He's trying to get you to operate on your trust in his word and not on what you see in the natural And so this is why he'll do it. He will do it to test you.  Are you faithful to do the work of the ministry or do you because a lack of passion and emotions? Seize to do the work of the ministry.  He doesn't want to have to Do those things for you. He wants you to learn to seek him yourself And so in those circumstances where you're asking where's god at work in this or how do I see the lord in this situation? In that place that is where god is testing you and watching you pass that test Seek him despite what you feel And he will draw you even closer and trust you with greater riches of the spiritual So number one he hides to draw you number two He hides to test you and number three the holy spirit hides to purify you The scripture says in josea chapter 10 verse number 12 So to yourselves in righteousness reap and mercy Break up your follow ground For it is time to seek the lord Till he come and reign righteousness upon you When you seek god righteousness is rained down upon you The holy spirit withdraws because it is in our seeking that we are purified When we seek the lord we are being Purified we are being made holy.  You are being purified So the process of seeking makes you one who can receive all that god has for you So number one God hides to draw you number two The holy spirit hides to test you and number three the holy spirit hides to purify you I want to pray with you now And ask the lord That wherever you are in this season that you would seek after him I'm going to pray that the fire of the holy spirit that unquenchable fire of god's love Would stir within you a passionate pursuit of his presence I want to pray that your heart your spirit would be ignited that there would be an inferno Of love for god burning inside of you So let's pray now that he would give you a hunger a desire and the discernment to seek him Father in jesus name I pray for that one receiving this prayer right now And I ask you lord That by your spirit you will draw them you will lead them You will show them father The way into your glory The way into the depths of your presence I thank you holy spirit that you have a hiding nature I thank you holy spirit that you allow us to seek you I pray now That hunger would be stirred A fire would be ignited And that seeking would begin I hear the lord saying he wants me to tell you it's time to ignite again The fire of your prayer life It's time again to seek the face of god Lord help them do it Remind us holy spirit when we get too busy and distracted In your faithfulness remind us to seek you In the precious name of jesus we pray And I want you to say it because you agree say Amen Well, that is it for the lesson. I want to welcome now the new members of spirit church. There you are up on the screen We love you. We are praying for you I always say that because I always mean it and in fact we are now at almost 3,000 members 3,000 global members of spirit church If you would like information on how you can join the spirit family use the information at the bottom of the screen sign up today It's 100 free. You'll receive a fresh weekly teaching from me and your email inbox on sunday mornings And then you'll also be able to reply to that email for prayer support from our ministry staff So join today become a member again It's free and also you'll be joining a community of believers from all around the world I want to read your comments now and the comments are from last week's video The power of the holy spirit by the way, you leave your comments here. I will read them on next week's video Terry writes I've been asking god for the gift of healing for quite a while now As he began to pray my hand began heating up I'm so glad I watched this teaching and that he confirmed my gift in this way Another commenter writes hi, I have just subscribed to your channel I have been struggling in my life and finding it hard to find my way I was raised in a non-religious family that didn't pray I wanted to find god and change my ways and I found this video It helped just by watching, but I know I have a long way to go. God bless you Well, god bless you and thank you for watching spirit church here on encounter tv I want to ask the spirit family to pray That they would find the lord in ways they never thought they could find him That the holy spirit would draw them closer to jesus than they ever imagined possible Another commenter writes my ear pain was cured. Thank you lord. Thank you pastor. David exactly the left ear like you said it May god bless you. Well, she's referring I believe to a word of knowledge I gave on last week's video and that's the wonderful thing about our channel Is we like to just let the holy spirit move this is his channel The holy spirit is free to do whatever he wants to do So often words of knowledge will flow miracles the healing power of god and the presence of the holy spirit Those are the things that make this channel unique Menna writes definitely steven's best awesome worship and if you haven't seen that video, I agree It's definitely I'll say definitely one of his best You have to go check out steven mock the zoomers worship playlist My absolute favorite worship leader, which is one of the reasons why he's featured on this channel But go take a look at last week's video listen to that worship and let it just I want you just to get into worship And let the presence of the holy spirit touch you Karen grace writes thank you brother david for the encouragement in this teaching I feel the power of the holy spirit as you preach this lesson Indeed apart from god. We are nothing more power to your ministry god bless Well, thank you for writing into us Karen. I'm glad to know that the ministry is blessing you And really this is my heart. I love to see people's lives touched and transformed And affected by the power of the holy spirit Now I have some really exciting news for those of you who have been following our campaign To reach a thousand new thirty dollar a month partners We are now less than 100 partners away From reaching our goal. Can you imagine that? Take a look at where we are as of this week This is where we stand in our campaign Now when we started a few months ago, we needed a thousand partners now when I say partner I'm not talking about a one-time donor. We appreciate one-time gifts and they definitely help But the partnerships really help us to structure the ministry And plan for the future thus making our planning and our resources more effective in what we do So here's what we need We needed a thousand new 30 dollar a month supporters and now we need less than 100 Here's what we're going to do with that support. I always tell you the reason we are doing ministry Is to win souls. That's our why okay, and I think you know that by now that my heart is just souls But I'll keep saying it because I want you to never forget it. It's about souls. We need to win souls Now here's what we're going to do with that support Our ministry likes to keep things really simple Because I believe in order to become dynamic you must first become specific and simplicity Allows you to do a few things with excellence rather than many things with mediocrity So we are going to with that support Do more events and that's key. We want to do more events in more places more often more miracle services Like you've seen here on the presence and power playlist on our channel And then We also are going to be moving into our new ministry facility and the monthly partnership It really helps to cover the monthly cost of that new facility You know, there's rent There is utilities. There's security. There's maintenance. There's cleaning. There are many things insurance There are many things that come with moving into a new property So the monthly support helps us to take care of those things, but here's what we're going to do with that facility We're going to broadcast more often and in higher quality We're going to build a section where you can come in as a studio audience and sit in on the tapings We're going to hold weekly services. We'll call them. I don't know my school of ministry or bible training or something But it's about impartation. You'll get to come. Steven will be doing worship. I'll be teaching the word, but we'll see you in person We're also going to be able to live broadcast from the studio We're also going to house the new tv network that we're building that's internet based There are so many things we're going to be able to do with this new facility, but bottom line We want to win souls through events and media And your monthly support helps us do that on larger scales than ever before look at You can see it. I can see it. My ministry team can see it. Even the critics can see it We are on the verge of a massive massive expansion And so I want you to be a part of this mark my words I'll say it again and again and again You're going to see a day where you and I together are filling stadiums And what does that mean? It means thousands of people coming to know Jesus And you're going to see that with your own eyes and you'll be able to say I was there at the beginning So help us today Sign up to become a 30 dollar month supporter. Do that now if everybody who watches this video signs up Whether you're watching from the philippines or south africa or nigeria Or you're watching from the uk or canada or singapore wherever you're watching this from you can sign up to become a supporter And if you become a supporter today I will send you a copy of either carriers of the glory Or 25 truths about demons and spiritual warfare for becoming a 30 dollar month supporter or more I'll send you one of those. I'll sign it. It'll be my gift to you an initiation gift to say. Thank you So do that today help us take the gospel all around the world like never before If you're watching this on youtube wait until the very end of this video A red button is going to appear that you're going to be able to click If you're watching this in the app, wait for the video to finish when the video goes away You'll see a button that says partner with david If you're watching this anywhere else use the information at the bottom of the screen sign up today become a monthly supporter Help us take this gospel all around the world Sign up today Well, that is it for this edition of spirit church until next time. Remember nothing is impossible with god Thank you for watching encounter tv. Don't forget to subscribe Also, help me win souls by spreading the gospel through events and media Make a one-time donation or become a monthly supporter by clicking on the donate link now


So we've done several videos on iodine in the past one more recently showing you guys a nasal oral flush That can prevent you from getting sick and the two older videos One was iodine in general and the second was about halogen toxicity Expanding on everything you really need to know about iodine today We're going to brush up on those two older videos starting with number one Main reason you need iodine is because the government pollutes our water supply with two of them and allows Another one to be used in food production and by one I mean this family of elements called halogens fluorine bromine chlorine iodine Astatine and tenacine and those latter two aren't important today, but the government puts fluoride and chlorine in our water and bromine is very common in our food These are essentially toxins. I mean, yeah, we might get them in trace amounts in nature But in the amount we're consuming them now They bind to basically every cell in our body inhibit organ functions, especially the thyroid and Since the government doesn't make sure we get enough iodine and kind of demonizes salt Our body is full of these toxic halogens and the only way to remove them is to one remove the negatives from our food and water supply and to Increase iodine consumption, although, you know, just taking iodine on its own is certainly better than nothing Moving on to number two is the function of the thyroid gland where most of the iodine is actually stored in our body thyroid hormones T3 and T4 Help us produce energy regulate our body's temperature in our adult lives But as children they even play a role in our normal growth and development so many people have had their thyroid is removed and are on permanent replacement medication, but Those people are a good reference for the extreme symptoms You can suffer without a properly functioning thyroid illustrating how important the organ is and When these toxic halogens accumulate to a certain point that is when your thyroid fail So it's probably very specific to Certain areas where the water is especially bad, but it is reversible if you haven't had the gland cut out yet So remove the fluoride bromine chlorine from the lifestyle and take iodine I mean, I think I did a whole video on organs that doctors commonly remove and how to remedy it We do have a thyroid glandular on organ supplements calm Which is actually the same thing you will get prescribed from a doctor except what we have is, you know 100% grass-fed high quality and natural and I know a lot of you guys are Taking this as your thyroid medication. So please let me know how that is going number three is prevention of disease The Japanese which are known for being some of the healthiest and longest lived people Coincidentally have the highest iodine consumption and that certainly has a lot to do with preventing all of these toxic halogens from wreaking havoc on your body for dozens and dozens and dozens of years and I done is especially known for Preventing breast cancer on its own. So that's a pretty quick overview of why we need iodine But how do we know how much to get and is iodine on its own enough? We know the RDAs for basically any nutrient are completely incorrect The government is creating those recommendations just to poison us or make us malnourished They're either in excess or not enough The RDA for iodine is 150 micrograms per day Which isn't even a fraction of what we realistically need most people Experts are recommending over six milligrams per day of iodine Especially to detox those halogens. What's also important for detoxing those halogens is consuming enough salt Something the government has also made us afraid of and by poisoning us with mercury through dental fillings ocean pollution Our selenium intake is greatly inhibited Selenium being the most important Synergistic element with iodine if we're talking about making sure we're getting every vitamin mineral element fatty acid our bodies need These two are typically overlooked selenium and iodine So if you haven't been taking these frequently or you've never taken them You need to take some steps to incorporate them into your diet And I don't like using seafood because of how polluted the ocean is I mean caviar is okay because it's super high on omega-3.  I don't know Brazil nuts try to get I dine through food if you like to but see how you feel doing that and If you're not noticing a difference then maybe try a supplement and do like a control group And I have sea salt written here, but what I really mean is you just want a high quality natural salt Preferably a land salt because we did mention that ocean pollution But you absolutely do not want table salt and a salt playing a main role in giving these negative halogens Something to bind to as they leave the body So it's kind of crazy how you know the government says everything is okay And in reality, it's this like elaborate scheme to poison the american population. So Thank you guys for joining me. Hopefully this helps some of you guys out The main takeaway for me is you make sure to consistently take iodine and selenium So if you guys could please drop a like on the video Leave a comment down below Subscribe so that youtube can unsubscribe you next week and be sure to check that notification bell So they don't Notify you of my videos therefore you can go to frank dash to find out calm See all of my businesses including organ supplements where we have a bunch of cool things that you guys might want to try out Thanks again guys, and I'll see you for tomorrow


This is stock number 11079. We are here at Summit Automotive in Fond du Lac, Wisconsin. You're new and used SUV headquarters today. We are checking out this super clean 2018 Chevy Tahoe LT1 with the Z71 off-road suspension package. This vehicle has 5.3 liter V8 motor. It has been fully safe and inspected by our service shop has a fresh oil and filter change. All the fluids have been checked and topped off and this vehicle is 100% ready to go. Black clear coat is the color. We shoot all of our videos in 1080p 60 frames per second. So if you have HD capabilities on your computer tablet or smartphone device turn them on right now because it is definitely your best way to check out the quality and condition of the vehicle before seeing it in person. And if you'd like to check out all the photos on this Tahoe in the upper right hand part of your screen is a link right to our website. Click that and check us out there. This one comes with the 18 inch painted alloy rims. Really nice condition. No scuffs or scrapes on them and it has Michelin Defender 26565R18 tires. These tires are like new. I'd say 90% of the tread left maybe even better on those tires. Front fender no dents or dings on that. Comes with the HID headlamps, the LED running lights and the factory fog lamps. Front bumper parking sensors. No dents, dings or cracks on that front bumper. Really nice condition. Headlight lenses are nice and clear and the chrome trim grille is in nice condition as well. Didn't see any dents or dings on that hood. Passengers side front fender. Very nice as well. No dents or dings and that passenger side rim no scuffs or scrapes. As you go down this side of the vehicle take note of how clean the body is, how reflective and mirror-like that paint is. We take these HD videos so if you are far away or even if you're close by and you just can't make the trip down but you're still interested in purchasing the vehicle, you can see the vehicle, hear the vehicle and have confidence in what you're looking at before you even get here. So then when you do get here there's absolutely no surprises. Has the roof rack up there and that's why we offer the Summit Auto Fast Pass option in the upper right hand part of your screen link right to our website where you can apply for financing, get pre-approved, see a payment quote, even appraise the trader all from the comfort of your own home. So when you get here all you got to do is drive the vehicle, love it, you'll be all set to go. Back rims are in excellent shape as well. Those back tires are just as new as the front tires. Very very new tires on this Tahoe. Four-wheel disc brakes, no scuffs or scrapes on those rims. It does have the factory painted step bars and as we come around to the back of the vehicle, rear bumper, nice shape as well. Comes with the towing package which includes a receiver hitch and seven pin wiring. Back up parking sensors, the rear gate is in nice condition too, no dense ordings on that. And this is a power lift gate. Goes up nicely. It does have the receiver hitch cover that's in pretty nice shape. And these third row seats do power fold down. Just press these buttons here and watch how fast these things go down. Always amazes me how quick they go down and up but really nice and clean back here. And to get this gate down you just press this button and it'll come on down like so. The other cool thing you've got this button here that's for the flipper glass so that'll open up that glass so you can get to the back in case you've got a hitch or a camper or a trailer or something on the back of your hitch and you're not able to open the gate. You can still get back there and get your stuff out. And as you go down the driver's side just as clean as that passenger's side, no dense ordings on that rear quarter. That paint is very nice and glossy, does have the capitalist fuel fill for full disclosure. This back rim, no scuffs or scrapes on that. And down the rest of the side of the vehicle looks really good. No dense, no dings, very very nice. Do get blind spot monitoring, directional signals in those mirrors and they are heated.  There are no rips, there are no tears on the seats. Very nice condition both these front seats are power and heated. Factory all-weather floor mats throughout.  Power windows, power locks, power mirrors. These mirrors do power fold in. I always like showing that both sides are working good so that you know that all the options are working. Memory driver's seat and Bose sound system. As we hop inside the vehicle here you can see that this one has 35,831 miles. Digital speedometer and Z71 instrument cluster trim there. Very nice and clean. Comes with the heated leather wrap steering wheel, no scuffs or scrapes on that. Information center controls, Bluetooth controls, cruise controls on left heated steering wheel and forward collision warning controls. Does have these six speed automatic transmission. Actually I always want to double check that. So you can put it in L and then hit the tap up button and that'll tell you how many gears you have. So this is the six speed automatic transmission. This one has the Chevy MyLink radio and FM and Sirius XM radio capabilities on this radio. You can see it does have the factory navigation system. There's your AM, FM and Sirius. This is also where your backup camera is going to show up. You can also hide stuff behind there. It has a little USB power point back there as well. Then this also has projection so you can project your cell phone to the screen. So if you got Waze or Google Maps or some other navigation system on your cell phone through Android Auto and Apple CarPlay you can project it to the screen and then you can use that navigation program if you'd like. CD player, dual climate control, heated seats. Passengers side seat and floor mat are in excellent condition and you do get a power sunroof. I'm running low on battery here so I'm going to speed this up just a little bit. You got your home link buttons for your garage door, security systems and lane systems, on-star capabilities in the mirror. We'll take a quick look at the back seats and then we'll check out under the hood. Back seats are just as clean as the front seats. You do get the second row bench seating. These seats are heated back here. You get the Blu-ray player up there to get to the third row. You just kind of flip those up. Third row seats are in excellent shape. No rips or tears back there as well. Smells very clean. This vehicle's never been smoked in. You got your rear air controls, two sets of headphones and we'll take a quick look under the hood. I would personally like to thank you for checking out the video today and hopefully from this HD video you've been able to tell just how clean this Tahoe is all the way around. Inside out under the hood we have the 5.3 V8 engine bay. Very clean runs, very smooth and this vehicle has been fully safe to inspect by our service shop. It has a fresh oil and filter change. All the fluid tab and checked and topped off in this vehicle is 100% ready to go. See more pictures of this Tahoe or one of our other 450 new and used cars, trucks, SUVs. You can go to www.summitauto.com. Full pictures and descriptions of every single vehicle from two locations all at summitauto.com. If you'd like to check out more HD videos you can go to youtube.com slash summitauto. Remember to like, subscribe and share on this video and all the videos that you see there. In fact, in a second we'll see a link to subscribe to our YouTube channel on your left. A link to more Tahoe videos like this one on your right. If you haven't been to our website on the bottom link to this vehicle on our website click those, check us out and we really look forward to helping you with this super clean 2018 Chevy Tahoe LT1 Z71 in black clear coat. Thanks again.


Hello everyone, welcome to this session on sectional anatomy of abdomen and pelvis, okay. So in this image, we can get a brief overview of how a sectional anatomy of abdomen looks like, okay. So in this particular session, what all particular topics we will be covering, so the other list of topics that I will be covering in this particular session, I will be covering a cross-section at C12 vertebrae, and cross-section at L1 vertebrae, and mid sagittal section of male pelvis and female pelvis, okay. And as far as exam point of view is concerned, these two sections are very important because in practical exams, there are specific specimens for this mid-sagittal section of male pelvis and female pelvis, okay. So let's begin with the cross-section at C12 vertebrae, this image we can see cross-section at C12 vertebrae, so before beginning with the section, first we will have to orient ourselves towards the image, so here I have given the orientation mark, so this is the anterior aspect, this is the posterior aspect, okay, and this is the left side, and this is the right side, okay. Whenever we study sections, we always study from the inferior aspect, okay, because later on when we learn about the CT scan and MRI, most of the sections are taken or they are studied or viewed from the inferior aspect, okay, so if we learn gross anatomy with an image viewed from the inferior aspect, later on it will be easy for us to identify the various CT scan and MRI images, okay, so this is the cross-section of the abdomen cut and viewed from the inferior aspect, so mortal structures we can appreciate in this image, what is this question mark structure, so this is the large organ that is the liver, okay, so when we cut and view from the inferior aspect on the right side, the large organ which will be visible is liver, okay. Nowadays in postgraduate examinations, image based questions are asked, so at times in cross-section, any structure they can label and they can ask in multiple choice questions, okay, so that is why sectional anatomy is also very important and what all other structures we can see in this particular section, this is the stomach which is seen, then which are these two structures, this is the right kidney, this is the left kidney, this is the abdominal aorta and this is inferior vena keva, what is this structure, first of all what is this, connecting the liver and the stomach, there is a part of the peritoneum which is referred to as the lector momentum, okay, so this is the lector momentum and this is the right free margin of the lector momentum containing otolvane, hepatic artery and bile duct, okay, so this is the right free margin and just behind the right free margin this space which we can see, this is the epiploic foramen, okay, epiploic foramen is the communication between greater sac and the lesser sac, okay, so just behind the stomach this space which is seen, this is the lesser sac and this S stands for the spleen which is present on the left side, okay, then apart from that which all structures are seen in this section, here we can see the body of the vertebrae, 8-well vertebrae, then this here is the final chord, then adjacent to the finest process of the vertebrae we can see erector finer group of muscles, okay, then when we come on the anterior aspect which muscle is this, this is the sector abdominus and this is the muscles of the anterior abdominal ball, here are three muscles in anterior abdominal ball, external oblique internal oblique and transversus abdominus, okay, so these are all the various structures seen in this particular section and for abdominal aorta T12 is a very important landmark because it is at this level that the descending thoracic aorta continues as the abdominal aorta, okay, and one more very important topic with respect to this section is epiploic foramen which is just behind this right free margin, okay, so we will learn a little more details about this epiploic foramen, so here we can see a cross-section of the at the level of epiploic foramen, so schematic diagram to show various relations of epiploic foramen, so this arrow mark is in the epiploic foramen, okay, it's also referred to as foramen of wind flow, okay, so two names for this particular region that is the epiploic foramen also referred to as foramen of wind flow, the communication between greater sag and lesser sag, okay, so this is the lesser sag which is shown here, this is the stomach, okay, so just behind the stomach is the lesser sag, okay, this shows the lesser momentum and this is the right free margin of the lesser momentum containing portal vein, hepatic artery and bile duct, okay, so this is the portal vein, bile duct is towards the right side, hepatic artery is towards the left side and behind both these structures is the portal vein, okay, so how to remember this which structure is to the right, which structure is to the left, we can remember the duct is dexter, okay, dexter in the word means it refers to the right side, okay, so on the right side there is bile duct, on the left side there is hepatic artery and behind both these structures is the portal vein, okay, so which all relations of epiploic foramen we can learn from this image, anterior relation of epiploic foramen is the right free margin of the lesser momentum containing portal vein, hepatic artery and bile duct, posterior relation is this, which structure is this, this is the inferior vena cava, okay, posterior relation of epiploic foramen is inferior vena cava and behind which is C12 vertebrate, okay, so anterior and posterior relations we can learn from this image, so superior and inferior relations how we can remember, where do we find for epiploic foramen in a detected specimen just behind the right free margin, okay, we place the finger just behind the right free margin, so if you imagine just below the finger is the first part of the duodenum, okay, the pylorus continues as the first part of the duodenum, so inferior relation is the first part of the duodenum and the superior relation is the cordate lobe of the liver, okay, so liver is above, first part of duodenum is below, anteriorly is right free margin of lesser momentum containing portal vein, hepatic artery and bile duct, posterior relation is inferior vena cava behind which is C12 vertebrate, okay, so the other we give relations of epiploic foramen which we should be knowing, okay, now let's cover the cross section at l1 vertebrate, l1 is a very important landmark in abdomen, if you have learnt about the various cauldrons of the abdomen, two vertical planes and two horizontal planes which divides the abdomen into nine quadrils, right, the upper horizontal plane which corresponds to the l1 vertebrate, okay, upper horizontal plane is also referred to as trans pyloric plane, okay, so what anatomical structure we can expect in this section can expect the pylorus of the stomach, okay, so this region which we can see this is the region of pylorus of the stomach, okay, and there are many other structures which are similar to the previous section, here we can see the liver, then kidney, right kidney, left kidney, and spleen, okay, so these are all structures we saw in the previous image as well, P which is mentioned, this is the pancreas, the region of the neck of the pancreas, behind the neck of the pancreas is an important anatomical event that is the union of superior misentric vein and slinic veins to form the portal veins, okay, so here is the formation of portal veins and here this is the inferior vena keva and abdominal aorta, okay, then here are the coils of the jejunum, then this is the slinic flexure of the colon which will continue as the descending colon and this is the transverse colon and this green color structure is gallbladder, okay, so here are the various structures seen at the level of l1 vertebrae, there is one labeled image for this, you all can just go through all the labeling, so there are some additional labeling also shown in this image, peritoneal cavity is shown, linear alba and palsy form ligament connects the anterior aspect of the liver to the anterior abdominal ball, then diaphragm is labeled, okay, there is one artery which is labeled here, superior misentric artery, the anterior branch of abdominal aorta, right, so rest of all most of the things we have covered in the previous image, let's compare a gross anatomy specimen with a specimen of CT scan, okay, how does a CT image look like as compared to the gross specimen, here we can see CT scan, whenever we see a CT scan the bones will be looking bright white color, okay, so here we can see the finest cells of the vertebrae and these are the ribs which are cut and they are looking bright white, so here is the right kidney, you can compare it with this gross specimen, bright kidney, right, liver, liver, here is the slin, in this here is the slin, here is the coils of intestines which are seen, okay, so when you learn about the gross specimen it will become easy for you to identify the various structures, later on when you see the CT scan and MRI images, okay, that's why cross section is important and always this orientation is important, okay, always we learn the cross sectional anatomy from the inferior aspect, now let's cover the met societal section of male pelvis and female pelvis, okay, so in this image what we can see is the met societal section of a male pelvis, technical orientation to this image, this is the anterior aspect, posterior aspect, superior and inferior, so in exams met societal sections of male and female pelvis are kept and mostly you are asked to identify various structures from anterior to posterior, okay, so anterior moves as genitals, then here there is the pubic symphysis, just behind the pubic symphysis there is urinary bladder, okay, this is the urinary bladder and there is a small space in between pubic symphysis and urinary bladder, this space is referred to as the retro pubic space, also referred to as retro pubic space of red zeus, okay, R-A-T-Z-I-U-S, red zeus, okay, so the urinary bladder which will continue as the urethra and surrounding here we can see this is the prostate gland, then behind urinary bladder this structure is the rectum and behind rectum this is sacrum, okay and when we see the folds of the peritoneum between rectum and urinary bladder there will be a one fold of peritoneum which is referred to as rectovacycal pouch, okay and you see when you learn about the female metrogytal pelvis there will be two pouch which will be seen because between urinary bladder and rectum there is an organ intervening that is the uterus, okay, between urinary bladder and rectum there is a small sheet of fascia which is referred to as rectovacycal fascia of denonvillus, okay and in front of the sacrum also there is a fascia which is referred to as pre-sacral fascia of walled air, okay, we have certain fancy names which you need to remember which attempts can be asked in multiple choice questions, okay, so I will repeat again between urinary bladder and rectum there is rectovacycal fascia of denonvillus and in front of the sacrum there is pre-sacral fascia of walled air, okay and what is this image this is MRI image of the metrogytal section of male pelvis, okay, whenever we learn about the soft tissue structure MRI is referred, okay and whenever we have to learn about heart structures then that means bones if you have to study then CT scan is referred, okay, soft tissues are more clearly seen in MRI, so here is the genitals and this is pubic sinfices and urinary bladder and rectum is not seen clearly so somewhere here is the rectum, okay, so let's see a gross specimen of metrogytal section of male pelvis, so here we can see a gross specimen, so here this is pubic sinfices then urinary bladder and this is rectum, okay and if you learn about the fold of peritoneum between rectum and urinary bladder there will be rectovacycal pouch, so in the metrogytal section mostly will be asked to identify various structures from anterior to posterior, okay, so if you have seen an image and learnt it you will be able to answer an exam, okay, now let's see metrogytal section of female pelvis, in this image we can see metrogytal section of female pelvis again the orientation remains the same anterior posterior superior inferior anterior most structure which is seen here this is the pubic sinfices then urinary bladder and uterus is the structure intervening between urinary bladder and rectum, okay and behind rectum is sacrum, so just like I told you previously between rectum and uterus there's a fold of peritoneum which is referred to as recto uterine pouch between urinary bladder and uterus there is bacyco uterine pouch, okay, so this pouch you can remember from rectum to uterus is recto uterine pouch between urinary bladder and uterus is bacyco uterine pouch, okay, then behind urinary bladder in the upper part you'll see uterus and in the lower part there'll be vagina, okay and when you trace the rectum that's the anal canal and in male pelvis I told you about rectovacycal fascia of denonvillus that won't be present in female pelvis okay but this pre sacral fascia of aldehyus that will be present, so again this is an MRI image of the metrogytal section of female pelvis, so here we can see this is the pubic sinfices then urinary bladder and uterus and this is the rectum and behind rectum is again sacrum, okay, now again let's see the gross specimen of metrogytal section of female pelvis, here we can see the gross specimen again let's try to identify various structures, this is the pubic sinfices and here is the urinary bladder and this is uterus and this is rectum, okay and behind rectum this is sacrum, okay, so we'll be able to identify various structures from anterior to posterior, okay, at times only specimen will be there and first you will have to identify whether it is metrogytal section of male pelvis or female pelvis, okay, for that you'll have to look for this organ, okay, uterus is present that means it is a metrogytal section of female pelvis otherwise you can see as the male pelvis, okay and also in male pelvis external genitalia testis and scrotum will be seen, okay, so let's summarize what we have covered in this particular session, we learned the cross section at T12 vertebrae, what important anatomical landmark was there at T12, one was epiploid foramen, then T12 is also a landmark for abdominal aorta, the descending thoracic aorta continues as the abdominal aorta, okay and these are the structures we learned and L1 is the important anatomical landmark for trans pyloric plane which divides the abdomen, anterior abdominal wall into line quadrant, okay, so pylorus of the stomach will be seen at this section and other structures also we identified and whenever we learn cross section we should make sure that we are drawing diagrams which are viewed from the inferior aspect okay and orientations are very important and we learned about the metrogytal section of male pelvis, female pelvis, MRI images of male and female pelvis as well as a gloss specimen images, okay, this was all about this particular session, for PDF handout of this particular session you will can what set me at this number and please do watch the other sessions of this youtube channel, okay, thank you.


ॐ ॐ ौshthat the chedda You don't have to do like this to raise a point of order, use your tongue. You will start dancing also.  ॐ his conviction. ॐ his sentence. ॐ Honorable members, I'll price the facts related to the incident to the house on 13th and 14th December 2023.


So, welcome everybody. My name is Pard Obman and I'm an analyst at McGill University. It's part of the Collaborations of the Digital Research Alliance of Canada. I'm going to talk about the way we are using EasyBuild. We've been using EasyBuild for about seven years on a national level and our present challenges and developments with a future outlook on the design decisions we have to make around this point in time for a new iteration of our national press. So, first I'm going to explain a bit about our organization and then get a bit more on topic about EasyBuild, etc. So, what is the Digital Research Alliance of Canada or what is taken over from Compute Canada and you see that name popping up pretty much everywhere still because changing names is hard. It's basically a collaboration of different organizations across Canada where we provide supercomputing and other digital research services to researchers across Canada.  So, there's no payment there and then some researchers have special credit money, they can have contributed systems as well. So, part of these clusters are basically owned by the researchers in sort of a condo style setup. So, we are organized in five regional consortia. So, we have ASTEP in the east, Uncle Quebec, Brian, Compute Ontario and then out west we have West DRI. It's formally split into two organizations, BC, British Columbia and Paris. This is the research infrastructure. And then we have member institutions which are the universities also like research hospitals. We have about 250 technical staff distributed on one of these universities. Almost 20,000 user accounts of all the students and postdocs and TIs across Canada and six bigger like nationally managed clusters for clouds, about 300,000 cores, thousands of TPUs and hundreds of petabytes of storage. So, you see quite a diverse range of research disciplines. Of course, we have all the traditional disciplines, but in recent years it's gone all over the place with a lot of genomic, very informatic users now and digital humanities also growing. So, how it used to be that we had pretty much every university that did some research had its own cluster, but that wasn't very efficient. That was a lot of fragmentation. So, we decided to get a smaller number of national systems like five proper HPC systems which are bigger and then can be managed sometimes even by distributed system administrator teams. But every researcher in Canada has access to these national systems. So, these were mostly put into production in 2017 to 2019 and with the newest one of ours, we had McGill University was put into place in late 2021. So, we now have some sites that have no cluster anymore, but they still need some support like they have like an analyst that is interested in Saskatchewan and he just worked with the researchers there. So, they have like local support. So, what we wanted at the time is to say no matter what cluster you log into, you can get a similar interface. So, you can have your submission script and it works on one cluster and then you can pretty much trust that the same submission script maybe a few modifications in number of course things like that will work on a different cluster. You don't have to recompile your software, it can be easily transferred. So, if one cluster is down or you get your allocation somewhere else, you can without with very minimal effort move with the cross, which didn't used to be the case. It used to be the case of gas to pretty much rewrite the whole script. And for that, we decided to use a distribution mechanism, CPFS. There are many other people who use that now and I'll explain a bit later what that means. And one innovation that we did on top of that is to introduce what is called the compatibility layer, something that sits in between the operating system and the software modules. Basically, we abstract away the OS. So, you can run this independent of whether you have sent to a 7 or 8 or Ubuntu or some other distribution installed. We do the automated installation because we're using Easy Build. Otherwise, I wouldn't be here. And we have Elmot as the module tool like most other people here. So, the background is really like where does this come from? This came up in the poll just yesterday as well. It's like, yeah, most APC clusters use enterprise linux distribution called Reasons. It's mostly vendor support, like the vendor support IB drivers for infinite networking and often only support on a couple of linux distributions. And so, system administrators will not put the latest, greatest fedora, for instance, on the cluster. That's like playing with fire. So, they tend to have the older enterprise distribution. We saw yesterday that's still a large proportion of the server users are using sent to a 7, which is getting quite old at this point. And you can see the versions linux kernel 3.10, TC485, etc. And so, sent to us 8 and 9 are quite a bit newer. But then when some people are used to desktop linux distribution, they get, oh, we have Fedora 38 and we have linux kernel 629 and TC13. And I'm running a program on that. I'm just copying it over to the cluster. And I expect that to work. And I complain to support that. That doesn't work and say, sorry, you can't expect that to work. So, and some people just install a software and what you do when you try to get software to work when you go to the documentation of that software and the software will just say, please install the software, please install the dependencies to install dependency to do sudo and to get install whatever you need. So, they type that in on the cluster and they'll get the sudo lecture. If you trust that you will receive the usual lecture, please type your password, type the password and so on. They can even carry a message you're saying, you have been reported. And these messages are all sent to Santa, which then will not give them presence for Christmas. Anyway, so they try that, but it doesn't work. Yeah, but it doesn't work that way on HTC systems. We have modules. So, traditionally, people write modules. What does that mean a module? Most people don't know what this is, but there's some that might not be so familiar.   Very old now, but this is for illustration because this is how we used to do that back in the day. Now, how were they created? Of course, by hand. You take another module file, you copy it, you adjust what you need and then you have a new module file. If you did a little bit better, you have like a logging system. We said, oh, I did this today. I created this file. I modified it like this, so you have like a record of what you did. So at least you have some history, but it's still very messy. One advantage is you make yourself invaluable.  So this is why easy building exists basically is to automate these things in a more consistent fashion. So what we are having is a design overview and this is the layered view that you will also probably see in Casper's talk. So I'm spoiling that a little bit, but I cannot present my talk without a spitter. So our software setup is layered with a top layer. It is called the easy build layer. It has all the modules. So the compilers of the software is an easy build. Anything that's compiled with the Intel compiler or NVHPC or UCTPGI, anything that uses MPI, put in a module, people can load that software using module load, whatever. Under this, it's a compatibility layer which has more of the boring dependencies. When you use easy build without any modifications, you'll get a whole bunch of dependencies in the beginning which are fairly boring that people usually don't load directly like the other tools. You have a module for it, but scientific users are not particularly interested in that software. They want to load Chrome X or NAMD or some other software that they need for their research. These are more like the boring dependencies that we have put in the compatibility layer. And it also replaces parts of the operating system. So we can have very a minimal compute nodes on clusters. We have a very minimal amount of software which then can live inside a RAM disk, just an image in memory, and all the rest of the software comes from CVMFS by our software. Then under this, it's another gray area where we have something that could be in the compatibility layer, but it's very system E, like IB client libraries, CUDA client libraries are in the system, but IB libraries, we put them in the compatibility layer because it works fine. It can be overridden if it doesn't work well. And then at the very bottom, we have stuff that needs to be on the system, needs to be installed by system administrators, like the Linux kernel of course, the demons, the drivers, CUDA runtime libraries, anything privileged, like we're not touching anything that needs root, that is always local. And sometimes there's software that has licensing restrictions that we cannot put in CVMFS, not even a restricted CVMFS that is internal, like fast past these restrictions. It is, no, we have to abide by the copyright and then that's it. So what is CVMFS? A CVMFS like a distributed file system. It means that you have, the few of the files you have is the same no matter where you log in, but it's distributed using various layers. So basically when you publish some software that we do, we first put it on what is called the stratum zero, then the software is distributed from the stratum zero to multiple stratum ones, and then the individual sites, they will pull that data from the stratum ones, so they can have a main stratum one server and if one of the stratum ones is down, you can use another one, again cache into what's called a squid, and then we have client nodes which take that data from the cache. So the first time you use a file on CVMFS, they may be a bit overweight because it can download over the network. The second time you use it is all cached and it's all super quick. So what we're using for the compatibility layer is a system called Gen2 prefix used to be, we used to use something called NICS, but we found Gen2 to be a better match, a more minimal solution to what we want. And so Gen2 prefix is just like Gen2 Linux, so Linux distribution where everything is compiled from source, but it's put in a prefix. So instead of putting everything at a slash, the top directory level is put at some lower directory. And it's just the management system to use e-build to compile software. And we use that Gen2 to compile anything that's of little scientific interest and where the performance is not crucial. And so we have that layer of lots of convenient tools, including editors, libraries, build tools, just common LS for users, et cetera. And then we have newer versions of what is in, for instance, CentOS 7. And we use that as an abstraction layer between the OS and scientific software stack.  Yeah, the question is whether I'm using Gen2 2020 to compile stuff on easy build? Yes, it is the bottom layer for easy builds. So we're using that as the system layer more or less. So for instance, when you install easy build as a bootstrap or your bootstrap PC core, really, and without any modifications, it will try to bootstrap from the system. And for us, the system from the easy build perspective is Gen2 presets. So for easy build, I put the same scheme for the logo, but it's not quite, we get rid of Nix, we replace by Gen2, but here we get rid of the old logo and replace by the new logo. We use that to just automate installation of scientific oriented software and then it models files them with Lmods and things covered already. So we use that for the hierarchy, sort of a hierarchy model system so that we lose it below the compiler and an MPI and a cooldown module to get a particular view of the module. So conceptually, what we have a builds, we call them recipes, like an easy config is a recipe, an eBuilds recipe. And we store them in Git repositories. We have worked the easy build repositories, but over the years that got a little bit messy and every time there's a new easy build release, I do. I sync it with the upstream easy build and particularly for easy block. And I got into quite horrendous merge conflicts at some point.  So we have a more clean separation between the between our easy conflicts and the upstream one. So that's sort of work in progress to do that. And Gen2 overlay already works like that. And Gen2 overlay is just an overlay over the upstream Gen2 where you just have the eBuilds that Gen2 themselves don't provide. So how do we decide what goes in easy build? What goes in Gen2? Let's say, is it performance critical? Or does it depend on the MPI? Or does it use Fortran? Yes. In that case, we'll use easy build. Are there multiple versions needed via modules? Like for instance, for something like CMake, people often need a newer version then provided by the base layer would mainly be easy to use easy build. And for anything else, we will use Gen2. So this is a little bit different from the decision list for easy which we covered in the next talk. So for easy, they put a lot more in the software layer. It's easy build and we put quite a bit more in the compatibility layer. It's just a different design choice. There's no right or wrong. It's a different way of doing things. Then the way our workflow works is that we have a specific build note. We have a custom user, an ED user that compiles software. We can do a test install on our home directory on the build note. Then we install it as the central user. Then we can test it on the build note. We can deploy it on the CV Invest test repository, the DEF repository. We can test it on the cluster with P-ROOT to change the path. You can use F10 or Singularity to change the path on the cluster to do some testing. And then we can deploy on the production repository. We know it works. Sometimes we shortcut this a little bit when it's a new module but nobody else uses it. Then there's no risk of breaking things when it's just a new module. But when we update a module that's already there, we have to be more careful and test it more carefully. You may have noticed that we used to use NICS and we tried a Tentool. Why did we do that? In NICS, there's a somewhat complicated scheme where they can use multiple versions in parallel using hashed directory names. This mostly worked but we had sometimes that hashed directories would leak into users' environments that would sometimes confuse users. Sometimes we would update software in place and the hashed directories would be garbage collected and just disappear and user stuff would break. And the main problem that we had with that is that NICS is designed to work at the top level and we had it at a cent which is in between two levels. It wasn't really designed to work that way and we ran into that because of basically using something in the way it's not designed to work is asking for problems that way. So with Tentool prefix, we don't have the complicated symbolic links with hashed directory names and it's just like more of a minimal solution of what we need. Another important part of our software stack is the Python wheels. So basically instead of having a very fat Python module plus custom modules for Python extensions, we have a more basic Python module, a bit more like Python there, where we just have enough infrastructure to build Python wheels and to install things with Python and pip, etc. And we supply wheels that users can install in in virtual environment. So we have a wheelhouse, which is all set automatically, people install Python packages. So what we tell them to do is to set up a virtual environment to do virtual and some environment. They enter that environment, they pip install whatever they need, and then they can load their stuff. And so this way we have a very flexible setup for the users without having fixed versions. Users can even compile their own wheels if they want to. And how that works is that we tell users if they want, say, TensorFlow, they can type a fairwheel, TensorFlow, and it will find TensorFlow that we're having the wheelhouse. We have now a lot of wheels in the wheelhouse, more than 10,000. Actually, right now we have more Python wheels than we have modules, amazingly. I was quite astounded by that myself, actually. So we have a witty page about that and people will just type a fairwheel, TensorFlow, and they say, oh, we can install TensorFlow 2.11 via pip, and it will get the TensorFlow that we know it's optimized to our system. And it's available for free Python versions at this point, 3.8, 3.9, 3.10. Those are the ones we Python versions that we supported at this point. We dropped Python 3.7 about a year ago, and now we've just introduced 3.11 to the mix, but most default is now 3.10. Do you see how it goes over the years that the number of modules keeps on growing, also because we keep the old environment? So right now we have just over 10,000 modules and even more Python wheels. So what does it look like? Now you may know that in EasyBuild we have the common tool change, Boston Intel, and there's a new release twice a year, so there's always 2028, 2020D, etc. We have a more slower cadence for the default modules, like we have something every two to three years, but we have something where we compile everything, and we don't do that every six months. So at the moment our production environment, what we call standard M2020, is mostly based on Intel and OpenMPI 403 and GCDyn 30. So now there's a lot of software that sits at the GCD core level. It means that we doesn't need MPI, it doesn't need BLAST or anything, it just needs GCC. And so when people do module fill to see, for instance, the vast majority of bioinformatics software is just GCC core compiled, and they see that and we already have the arch optimization flags to make it work fast enough. And then there's a much smaller subset of software that we compile with Intel or MPI, or both. So we compile still, we compile most software with GCD930, it's getting a little bit older, but it works fine for the vast majority of things. We did a slight modification where we had a new production cluster, Naval, which used AMD by the other ones, used Intel and after some benchmarking, the link is here. I found out that for AMD it was, BLIST was more performant than using MKL, so we used FlexiBLAST to decide which back-end used for linear algebra. On the AMD process we use BLIST and on the Intel process we use MKL, but the users just see all we're using FlexiBLAST. We do have newer tool chains available, for instance, there's, we have 2022A tool chains. Users don't see the tool chains themselves, that's more an easy built internal thing to us, but they just can do Motorola OC11.3. If they do that, they get a newer open MPI, they get a newer software stack, if they have to, but it's not what they see in the default module view, it's just that people need something newer, they can just load the modules. And we made that the default in April 2021, but the older software stacks available for people who want it. Now we're working on a new environment, of course, we need to replace the standard end 2020, and that's like Intel, maybe we use standard end 2023, that would then have mostly software from this year, but it would be in production in 2024, because it takes some time to, we compile everything and then make it a default. So from beginning to end, we give ourselves about a year. So the plan is now to use something more based on force 2023A. We found out that there's not so many benefits to compiling things with Intel, there's some things that work better with TC, some things that work better with Intel, but when we want to distribute to the public, it's always safer to depend on open source software and not have licensing issues like you'd have with Intel. So this is all based on software that hasn't been released yet, so we're probably sticking with a TC 12.3 and open a PI415, I suspect those will go in the 2023A common tool change, they haven't been set in stone yet, but probably that way. So there are some things in the pipeline, we have to weigh pros and cons of using that, that the CC13 is better support for newer AMD GPUs. There's an open MPI5 that's been a long time coming and it may be more performant to use. So if we are very risky, we could include those, but time experience has told us that it's based off to be a little bit more careful and then leave the more risky stuff to users who actually benefit from it. So every time we had a new iteration of the standard environment, we have to say, oh, what goes in the software layer, at the top layer, what goes in the compatibility layer, do we have to move things around? And so some of the things we've learned is that build tools are also a bit of a headache, like what do we do with CMake, Ninja, Mason, etc. They are sort of automatically installed if you install Gen2, but easy build often ones like a specific version, sometimes the version from 2020 is too old, so you want to have a module for that as well. And so we made a say, okay, instead of having a base version in the Gen2 prefix and a module for it, we just kick it out of the Gen2 prefix and have only a module for it. Now there's a lot of overlap in the compatibility layer with what EZ is using, you could almost say that EZ is developing a subset of what we're doing, so I'm trying to learn from EZ much as they're sort of learning from us. And so I'm picking up a lot of ideas about how to do things more properly, more automated in particular. Another thing that caught my eye is the solution for our path from a guy named Harman Stoppels who works in Switzerland, seems to be Dutchman. So what we're doing instead of using LD library path in our stack is that OR software has an R path and links to a particular library using the R path embedded in the binary. The problem with that is that in order for that to work, we have a linker wrapper. So instead of using the regular LD, users execute a linker which then inserts the the libraries in there. Now EZ build also has functionality for our path of minus minus rp, but instead of the linker level, it uses a compiler level and it doesn't directly translate into the users. If the user compiles their own software and loads the module, they still need the LD library path otherwise at runtime the software will not pick it up. So there's an idea from Harman Stoppels saying let's change the SO name like a particular field inside the library and then when you link that library it will just pick up the runtime path automatically without any need for a wrapper or LD library path. So that's a very promising solution because it will work. It will basically work without additional hacks apart from changing the libraries themselves. So we're looking into that. Yeah, that's another thing. Yeah, but what Kenneth says is speed things up. It eliminates some load storms. We're not as affected by load storms as an upstream EZ build. That's because so much of our stuff is in the compatibility layer that they already find a majority of the libraries usually in the compatibility layer and then the R path is fairly small only has like two, three, four entries and so it only looks in those two, three, four entries. If you have the libraries embedded in the binaries, it will not search a list of directories for that particular file. It can directly open that file and doesn't have to spend a lot of time searching for the library. So that's why this is not a good solution. So I know that that Eleanor Kais was was looking at this as well and getting a full request for EZ build to do this. He is more of a spec developer so it's been optional and now in spec for for some time. Another thing that we've we've seen is a bit of a war story. Like we ran into a bug in CVMFS where we updated some some file in place. We did a Tlipsy update because somebody said, hey, this is a cool feature of Tlipsy. Okay, let's let's put it in. And then I I deployed it. I tested it and then boom, we got a lot of complaints. So it was quite embarrassing. And so basically I pushed a new Lipsy library into CVMFS and processes started crashing randomly. And I was just checking what's going on here. I did a shot 256 some on the on the library and it would give different results every time I did it. What is going on here? This was very bad. So we scrambled and referred it to the older version, which you can do because CVMFS keeps a bit of a history. And so this became a bug in CVMFS and eventually the main developer CVMFS concluded this was a particular nasty case of cache poisoning, which they have fixed in a newer update of CVMFS 2.10. So people can be a bit it can stop worrying about in place updates. And because it's fixed at this point, there's still synchronization issues if you do in place updates that have MPI processes. These updates should be atomic because otherwise you can have a process that has a newer version of a library on one compute note than on an other compute note if you see the best updates doesn't arrive at exactly the same time. So it's still something you have to be really threat carefully if you do in place updates on CVMFS. Another thing you see when you use a compatibility layer instead of just using the basic RPMs on a cluster is that there's software that is not used to it and particularly you see that with proprietary software from vendors. They've done their QA of course on particular Linux distributions. There's different workarounds we're using. We have a script called setRpaths.sh which basically patches all the binaries to set the RPath correctly so they link to our libraries. And then we have to work around some users. Some use for whatever reason who user lips 64 as LD library path in the BESH RC, of course that doesn't work, we tell them please don't do that. But there are also commercial packages who do these kind of tricks in wrapper scripts that launch their binaries. So then we have to patch the wrapper script of the commercial vendor. And of course there are also users who just download binaries from random places and it happens a lot with Anaconda to a lesser extent with Julia. So they just do condi install much they do on their local computer and they fill their home directory with a ton of binaries. Sometimes that works sometimes it doesn't like in yesterday afternoon. Some people end up downloading a custom open MPI which they use in ethernet network. It's outside of control and it becomes very hard to support. So we basically strongly discourage people to use Anaconda. Our Python page is very much like please don't use condi. It's just so hard to support. And so there are cases where it's really too hard to make things work with our software layers. Okay well we throw our hands in the air we give a used use container. Okay it works it may not be optimal but at least you get your work done you get your paper published everybody's happy. The other thing we're working we have some compatibility layer OS boundary challenges like sometimes gentle prefix is designed in a way slightly differently than what we're doing. They even are set up in a way that they can run demons in the compatibility layer because it's more aimed at local computers where people have have root and privilege like that. So we have a slightly different use case where the demons are all working from the OS level and for that reason the directory says farge should really not be in the prefixes should be at the base level. So there's some things that we work around with symbolic links just simulinking from the prefix to as far you can also patch it in files so it's still working on trying to smoothen out the compatibility layer a bit like that. But other other hand it works remarkably well we have unlike easy again we even have a whole main desktop compiled in the compatibility layers so people can can run can run VNC so they can run VNC server via tiger or turbo VNC on a compute node and then connect to it via VNC even using Jupyter Hub so you can have VNC in the browser window and use virtual TL accelerated graphics even in a window. So there's a bunch of things working with with TN2 that works remarkably well and we just have one main desktop where we don't have to rely on the OS. We even have some SE Linux support like the file system labels it's like on time system in society LS that you have in the compatibility layer doesn't show some information that comes from SE Linux so we have that as well. Then another peculiarity is that we have a funny system where we basically collapse tc core into a system by using a common lib standard c++ library we're using hooks and easy build basically change the name of pool 1.0 tc core 113 we just pool 1.0 so basically change from tc core to system on the fly and it works but it's a little bit messy and sometimes it conflicts with the dependency of the way easy build results dependencies. So I'm still sort of trying to figure out how to best resolve this and and I noticed in the easy build repository there's something called tc core minus system.ev that may be a better solution for us it's been put there a few years ago it severed a bit from bitrock but it may give us a better way maybe using as gentle compile tc instead of an easy build compile tcc may be best because I mean why would you why do we have two different versions of tc 930 at the moment we have one compiled with gen 2 another one with easy build is really necessary to have the two of them so it needs a bit of sorting out an experiment. So that's basically concluding my talk one thing I should say is that the public part of our stack is available everywhere we have documentation linked here you can if you set up cbms on on on the linux laptop it has to be x86 unlike easy by the way but most people still use x86 you can get the whole stack you can load the modules which is nice a chance to research as well because they can get the modules on their own laptop or workstation and thereby get the same environment as on the clusters you just use that testing at their own leisure and so the the what we can do with for instance sharing a bit of work with with easy on the compatibility layer for the future and that's about it for my talk at the moment let's try this you can use this as a mic for the audience here are are there any questions for bars yeah I was wondering how many systems make this cfmfs stack available in Canada you're asking how many deploys is available how do you mean how many freedom wants or well because I was looking at the website I was curious and I saw something like six or seven systems being listed there as being part of the alliance but I thought you had a deployed on more systems now he says a six systems well that part of the that mounts the cfmfs but the more systems that mount it so if we go to the earlier slides we have we have five large national hbc clusters they all mount this software stack this is the primary use case but what we've seen is that the regional clusters or the fundamental clusters they also mount the cfmfs and we actually have a uh a voluntary spreadsheet where we ask people if you use our stack please please tell us that we use it and we found out that we had about 30 entries and and there's some from like really small use there's some individual laptop but there's some some research institutions some universities most about half across Canada and then a few in France a few in other countries that use it for various reasons great thanks any other questions yep sorry maybe I didn't just didn't understand it enough but what do you do with Python packages that are not in your real world post-story then I'm not in our real okay so what happens when somebody just install some random Python packages like if they do clip install numpy they will get the numpy from a wheelhouse because we have it there or tensorflow if it's some random Python package that we haven't as a wheel yet it will attempt to download this source Python package from PyPy if it's pure Python that is never an issue just download the source code it will create the wheel and clip install it so it will it will not install binary wheels from PyPy but we have a configuration file for PIP that basically blocks the installation of binary wheels from PyPy because they're usually not compatible with our stack but it will download source wheels the one exception we have is MPI for Py because that needs to link to a particular MPI version and for that we basically block it from PIP installing it so PIP install will not find it because it refuses to install both from source and the binary package and for that we say please use module load MPI for Py with ORF like a custom module for MPI for Py and there's also some cases like QT we have the Python the corresponding Python package installed with the module so when people do module load QT they will get Py QT on the site as like an extension so it's not it's not completely purely real I mean the fast majority is wheels but there are some things implemented by modules so when you say that if you don't have wheels you try to download the source and then build it yourself does this usually work or are there any tweaks that you have to manually make and how do you handle dependencies in that case if they are defined you know whether in requirements or stuff like PIP file and so on so is that taken care of automatically how do we deal with dependencies so usually it just works like people will just install the virtual environment they start a virtual environment they do PIP install my random PyPy package so as long as they do that the PIP install will just download the random Python package from PyPy and try to create the wheel now if it's pure Python code it will just work it will create the wheel install it boom as long as it's compatible with the particular Python version fine if it has a list of dependencies then of course the dependencies will be resolved mostly dependencies will usually be in our real house we'll use the wheels from our real house the particular version is not on our real house it will of course then download that version from from PyPy and try to compile it and if that uses particular c code for which the module is not loaded then of course it will fail in that case they would tell users okay please write to us with your issue or help you out how often does it happen that users have it's one of the more common support tickets that people have issues with Python packages so yeah I mean it's it's like I mean we can we can handle the load and and like I said given that we have more than 10,000 wheels it's one of the one of the big workloads of our team is just installing those wheels in the wheelhouse so so so Python is an extremely popular ticket topic it's yeah there are multiple tickets per maybe even per day it's it's that yeah and and but most of them are straightforward so maybe small add-on to Dennis this question on the dependencies it usually works but you're you're not controlling which versions get installed right now PyPy figures it out by itself it's going to take the latest version and if you're lucky enough it'll all work but if you do the same installation tomorrow you'll get different versions of dependencies and that yeah you're losing that control yeah we're losing that control maybe users don't care right if it works yeah and I had one more more challenging tickets where the user just for whatever reason tried to combine a new PyTorch with an older NumPy and he just had the requirement of NumPy 193 in his requirements and I said why do you do that and he never answered that question I say but you can't combine old with new that way and expect things to work but yeah so I don't know it's it it it it it then spit out a message of the ABI being incompatible and he said yeah but my my machine learning still produced outputs so it seems to work even though I cut this ABI yeah but I say yeah but you're playing with fire it's it seems to work but it may crash at random times or give incorrect numbers and you just don't know we have a we have a question by atlack as well so on collaboration between the alliance and easy how about running the same reframe tests on both software stacks would that make sense so the easy test suite we're working on that gospel is probably gonna talk about but that's an excellent idea of looking at your your reframe test yeah I'm not sure if they work out of the box but we should certainly do more every reframe testing there's another question they have a basic suite already of reframe test is the osu micro branch marks and the module loads just to make sure that all the combos work yeah another question in zoom how many employees are involved in the software installations um I'd say about 15 are on a on a regular basis installing software or wheels or easy 15 or 20 out of the top of my head so we have like the four rs and p people Maxime Poisson, Charles Coulomb and me and Doc Roberts and then uh and then there's a bunch of people more local you may have seen them in easy build on github as me atlantic paul is oliver stoker oligy if he said alekirachi at our woodcar so so just out of the top of my head I have like five names there so it's and there's probably about 10 others so


Question. Which program is in Guinness World Records as the world's longest running TV quiz show? It's academic! It's academic is a Saturday morning television program for high school students. The show has been broadcast in the Washington area for 50 years. Teams from local schools face subjects like history, literature, mathematics, science and sports. Susan Altman is the producer of the show. First of all, it's a great age group to work with. They're fun, they're enthusiastic. They say funny things without even meaning to half the time. Ms. Altman's mother, Sophie, was a TV producer in the 1950s. She started its academic after local officials asked her to create a program that showed extraordinary students. And at that time, this was in 1961, the United States of course was going through a lot of upheaval with school integration. And they wanted things to help out, to make the schools look good and to show that integration could work and work well. The show's host, Mac McGarry, says the teams became more diverse over the years. When we first started, the young people from various countries were not in evidence so much. But now each program presents young people from all over the world or at least descendants of people from all over the world. So this is a microcosm of the way America is today. Mac McGarry is 85 years old. He has hosted its academic since the beginning. He has asked more than 200,000 questions. He's gotten more than a few funny answers. We had a question one time. Who defeated Napoleon at Waterloo? And the young fellow answered, Duke Ellington. Another time, we had a question. I thought, who along with Frederick Engels wrote the Communist Manifesto? Groucho Marx. The correct answers are the Duke of Wellington and Karl Marx. Kelly Reader knows that. She led the team from Rockville High School in Maryland last year. Me and all the other people who are on the team, we've put in a lot of effort over the years and a lot of time to be where we are. And I mean, it's a really huge honor to end up on the A team. Her school has won the championship on its academic several times. Secretary of State Hillary Clinton competed on the show when she was a teenager. So did astronaut Timothy Creamer. Maybe one of these students will also rise to fame. I'm Barbara Klein.