Russia's Defense Ministry said on Friday that it carried out an overnight strike on Ukrainian ammunition depots. Tonight, the armed forces of the Russian Federation carried out a group strike with long-range precision-guided air-launched weapons at the ammunition storage sites of the Ukrainian troops, the Defense Ministry spokesman Igor Konoshenkov told in a daily briefing. The target has been reached. All assigned objects are hit, the official said, adding that the armed forces of the Russian Federation continue to conduct a special military operation. The attack comes amid Ukraine's anticipated counter-offensive that has put Russia on the defensive in its 15-month-long invasion of the country.


It's a great pleasure to welcome Oshima Satoru from the University of Tokyo and former Japanese ambassador to Saudi Arabia and also to Korea, to WPC TV. Oshima-san, a perhaps deceptively simple question. What do you think of the American pivot to Asia? Well, simply, we kind of welcome the policy followed by the current administration because of the fact that in view of the rise of China, which does have a positive element in certain areas, it does have some worrisome elements as well. So we feel that our security will be better served by a stronger interest of the Americans and peasants in this area. Is the implication that you cannot really improve relations with China unless you have the backing of America to somehow influence China? No, I think as we have been working recently with the Chinese, which resulted in this meeting between the two leaders through diplomatic channels, diplomatic efforts, we have always been open to dialogue with the Chinese and we managed to first take the first step rather difficult a couple of years in the recent past. So we are looking forward to developing on this positive step. Of course, the situation is not very delicate and very difficult, but I think there are a meeting of minds that we should move forward. Is the question of the shrine a difficult one? Why can't it be resolved? Well, first of all, there are many sensitive, difficult elements involved in this whole Yaskin shrine, which sometimes does not get through to foreign audience. First of all, Yaskin Shrine is a monument according to our admittedly religious feelings to commemorate or respect people who died for the country in various past wars since modern times. So it was not built for those individuals who died in one particular war, particularly the recent, most recent war at all. So it's natural for the many natural Japanese people to feel respect and would like to pay respect at the shrine and political leaders who are of course Japanese as well and you understand the Japanese sentiments and they have their own sentiments as well and they would like to pay respect. There are of course some sensitive political issues involved with our neighbors, particularly the Chinese. We understand the sensitive feelings on the part of the Chinese, but then we hope that they will also understand that there are certain feelings on our side, which is not necessary to, what should I say, entombolize the Chinese sentiments, what we have done in the past during the war with China. Of course we are very much remorseful and we have said so publicly, but the fact that we pay some people in Japan pay respect to the shrine is based on our sentiments to pay respect to the war dead and I think this happens with any nation where they had unfortunately wars in the past and people who have died for the country. Ashim Sun, thank you very much. You're welcome.


Firefighters say the home is a total loss. Atkins is on tour but his children were at the home. Back everyone, today we'll be taking a look at country superstar Trace Atkins' homes.  I'm talking about everything from his new home to his country retreat. Born on January 13, 1962 and raised in Louisiana Trace Atkins discovered his passion for music at an early age. He sang gospel music with his family and learned to play the guitar. But over the next two decades Trace then established himself as one of country music's biggest stars with over 10 albums and a string of chart-topping hits. We're talking bops like this ain't no thinkin' thing, ladies love country boys and of course honky tonk Trace has been inducted into the Grand Ole Opry and his awards include Billboard's top male new artist, the ECM for top new male vocalist and American country awards for a few different things. Aside from his booming music career Trace even appeared in TV shows and movies like King of the Hill, The Lincoln Lawyer and Deepwater Horizon. Most recently he starred in the drama series Monarch. Trace's larger-than-life personality, signature baritone vocals and undeniable charm have made him a beloved icon in country music and beyond. Now that we've gotten to know the man behind that cowboy had a bit better let's mosey on over and explore some of the amazing homes Trace has lived in. From secluded retreats to Hollywood Hills mansions Trace definitely knows how to live in country music luxury. First we'll saddle up and take a look at the bachelor pad Trace used to call home in Brentwood Tennessee. Located just outside Nashville Brentwood is one of the most exclusive birds in Tennessee with a median home price around 500,000 but this posh neighborhood is a hot spot for plenty of celebs and athletes looking for privacy in a peaceful paradise. Trace's former Brentwood home was built in 2014 and spanned over 4,000 square feet with six bedrooms and five bathrooms. It's out on almost half an acre of land so there was plenty of room for Trace to get his cowboy kicks in. This two story brick beauty also had a walk-in closet in each bedroom so he had space for all his stylish cowboy couture. Trace's stunning property featured a patio out back perfect for him to host his famous summer barbecues. Inside there was a home theater in the basement where I bet Trace had many movie marathons with his family. I mean with that silky voice of his Trace was probably born to narrate movies too. One of my favorite features here had to be the beautiful view of the trees from the upstairs deck. This picture perfect property was valued around 1.6 million dollars which makes complete sense for a celebrity home of this caliber. Sadly Trace no longer lives in this Brentwood estate because it was destroyed in the major fire back in 2011. Reports say the fire started from some faulty electrical in the garage and spread quickly throughout the home. Luckily Trace's family got out safely but the house well not so much. What a nightmare it just goes to show even celebrities aren't immune to tragedies like house fires. Well after that devastating blaze Trace and his family decided that it was time for them to rebuild and make new memories. Let's take a look at the fabulous estate Trace now calls home. Even though losing his precious Brentwood home was totally traumatic Trace picked himself up and dusted off his cowboy boots to build an even more amazing estate in the same neighborhood. Trace's new Nashville home is another brick beauty that spans around 4,000 square feet. This pad has all the trimmings Trace needs to live his best cowboy life. We're talking six cozy bedroom, five bathrooms decked out in the finest marble and even a recording studio where Trace can lay down tracks and channel his musical muse. Of course the home theater made a comeback in the basement because Trace needs room to watch his favorite westerns in cinema style surround sound. Trace meets sure to include the little luxuries that make a house feel like home like walk-in closet to spacious patio in a three car garage. And what about this adorable neighborhood just like his previous crib Trace chose to rebuild in Brentwood's exclusive suburbs. Can you blame him? We're talking manicured lawns, white picket fences and home so beautiful they look like they popped out of a magazine spread. Trace's place features lush professional landscaping with flowers and shrubs surrounding the property. Out back Trace can host epic pool parties and barbecues in the spacious yard. Inside sunlight drenches each room thanks to huge windows that highlight the breathtaking views. In the morning Trace can take his coffee out on the upper balcony and look out at all the trees surrounding his estate. At night Trace can cozy up by one of the home's multiple marble fireplaces and relax. However Trace chooses to enjoy in his new Nashville residence is the ultimate celebrity dream home. When Trace wants to get off the grid and enjoy a slower pace of country living he heads to his secluded rural retreat outside Nashville.  Trace and his wife Rhonda transform this worn-down farm into a beautiful countryside oasis that reminds Trace of his childhood. Nestled among rolling green hills his rustic cabin getaway gives him the perfect space to relax, recharge and get back to his roots. Trace's countryside cabin is all about bringing the outdoors in. Florida ceiling windows flood the home with natural light offering panoramic views of the wilderness. You might even spot deer, turkey, or other wildlife roaming outside. The interior features gorgeous reclaimed wood that adds warmth and charm to the home. Trace made sure to incorporate natural textures and materials like stone, wood beams and rustic metal accents that feel erpy yet elegant. Of course music's central to Trace's world so his country retreat features an amazing recording studio where he can lay down new tracks surrounded by nature's melodies. There's also a cozy screened-in porch that acts like an outdoor living room. Trace can jam on his guitar or curl up with a good book in one of the comfy wicker rocking chairs. When Trace is craving some good old-fashioned southern barbecue he fires up the custom-built outdoor kitchen and grill. We're talking a full outdoor island with a cooktop, bar seating, and everything needed to host an epic country cocoa. Trace can whip up his signature ribs and cornbread while guests sip sweet tea and two step-to-acoustic guitar on the fire pit. The surrounding countryside offers endless opportunities to hop on the quad and go off-roading. Or Trace can saddle up his horse and go for a trail ride under the stars. At the end of the day an outdoor shower lets one rinse off before settling into the cozy master suite which has its own stone fireplace and massive walk-in closet. From the natural wood finishing to the comfy covered porches this retreat is Trace's perfect escape from the city's bright lights. The country charm reminds Trace of simpler times and lets his creative side flourish. We all need a peaceful place to recharge and Trace certainly found his. Now that we looked at three of Trace Adkin's homes that wraps up today's video. But before we go answer this question for me. What would be the best part about having a cabin in the woods or country vacation retreat and why? Let me know your thoughts in the comments and don't forget to like and subscribe to this channel. I'm Kara the Vampire Slayer. Follow me on Instagram to chat and I'll see you all next time. Bye! Hey everyone it's Kara or Kara the Vampire Slayer as you might know me and given my love for all things house and home I recently wanted to get more hands on myself and I started tackling DIY projects at my own house. Being a new homeowner there are plenty of unexpected things to fix and it's cool to learn what you're doing and hopefully be able to fix it yourself. There are also plenty of DIY projects suitable for beginners like me to do and ones that improve the look of your space drastically. Follow me and I'll teach you what I'm learning and motivate you to join me.  Follow me on fix it with Kara and we can chat.


Hello everyone, welcome to the Jenkins Infrastructure Weekly Team Meeting. Today we are 27 of June 2023, around the table we got your servitor, Dimon DuPortal, Erwe Le Meur, Mark White, Stefan Merle, Kevin Martins. I don't know if Bruno will be able to join. Let me comment out his name. And that's all. Let's get started with an announcement. So the weekly 2.412 is out, at least core package and docker image, core package and docker image, I assume. And change log is out as well. So as far as I can tell, complete. I haven't run the full checklist, but I've confirmed the container image is there. I'm building with it now. Looks great. Efficiency. Nice. So ready to roll for us on the infrastructure. Didn't see any show whatsoever. I'm saying that as an announcement tomorrow. Don't forget we will have a LTS release tomorrow. So please don't break the infrastructure. Dimon DuPortal, please. I'm talking to myself as usual, because with the person who tends to break the infrastructure the more, I think I'm number one there. Do you have those announcements folks? Nope, I don't either. So I guess I did have a question. We've got a pending poll request to use Maven 3.9.3 on the packer images, but should we wait a day and not merge that until after LTS tomorrow? So can we merge it? If the checks are okay, we can merge it without any risk because it will not be deployed to the production. We will need to trigger a full release of the templates. So as soon as possible is okay. But yes, better to wait for at least under LTS tomorrow for triggering that release and deploying that to the whole system. I had forgotten. It's a multi-step process. It's not just declare the change. It will then propagate through multiple steps. But good point. Good question. Thanks for raising it. That's always better communicating. Let's wait before deploying Maven 3.9.3. A note about this, so I'll explain this discover painfully that kind of problem. As a reminder, the usual download mirrors for Maven projects are only hosting the latest available version. So as soon as a new version is there, the previous one is unavailable. So if you want to have something stable over time, such as a Docker file downloading Maven, then you should use one of the archives. So you go to Maven download site. You go to check older versions. And here you have a set of mirrors that have the whole history or at least enough for sustaining one or two versions back. Maybe mirror it in ACP. And no, we are ready. So thanks for the proposal, LRV. But first, that's a complicated topic, but we are fighting about abuse. And that one will be abused. And above all, that will forbid external user to use it. And adding it in as a transparent mirror from a curl is absolutely quite the problem. Because it's not Maven, it's not Maven downloading Maven that ACP has been built for. It's curl. So that means you need to intercept the curl calls or declare an HTTP proxy compliant with the proxy protocol. That will require a lot of work. Anyway, that's not a problem. We already have solved that problem one year ago and that has been solved by Alex on different components. So next week, we will have a weekly release as usual. It will be 3rd July, is that correct? And next till yes is tomorrow. It's a holiday in the US the 4th of July now. Correct. Yeah, it's a bank holiday here. Independence Day. Thanks to our British friends. I guess the question mark is the Independence Day going to influence the weekly release or our European friends are in up? Yeah, I think I don't see any reason to change anything about that. It will release automatically. It will build automatically and will do the changelog. It's an easy thing to do, no problem. So no next security release, no next major events. So unless you want to add something about an enhancement calendar, you can proceed to the work that was done. Okay, let's get started. We finally found the root problem of the certificate not being renewed on our virtual machines on some of our virtual machines. First, there was a dead link slash user slash bin slash third but was the link to snap. So when you were when that nice third but renew dash queue crown tab was executing, it was trying to call snap, renew dash queue, which of course triggers an error that say, Hey, I don't know the command renew. Removing the same link was not enough because the crown tab have a path of their home, which is different from the user. And of course, on some of our machines, but on all our machines, it's installed on us as a local bin, which is not on the default path of the crown tabs. So yeah, even fixing that the next error was set, but command not found. We fixed that by specifying our own crown tab using Puppet. So we control, we now write logs of the renewal. So next time we'll be able to see the errors immediately the day after we control the absolute pass and all the parameters. So we tested it and it's renewed 24 hours later, all the certificates, including updates, junk in SIO, but archives on the third one. I don't remember which one. It's in the issue. So it work on three different machines. So I hope we should be okay with that. I can say 100%. Let's see with the monitoring in the future. Is there any question about that topic? Sorry, I messed up with that issue. Yep, let me update. Okay, any question about the SSL certificate renewal for updates? Nope. Thank you for doing it. And congratulations to all those who did not interact, intervene interactively to hide the problem again. Next issue we were able to close. We were able to, so we received the email notification from Fastly about Apex domain junk in SIO, not the website, www.junkin-sio. And it was trying to get a certificate for junk in SIO. And it was failing to do so. So we diagnosed the issue and we created the missing DNS record. And that was a rabbit hole that we were able to close. So we ended up managing everything properly. In the same way, everything running on Fastly is now using a DNS record for ACME and the proper CNAME record. All of them managed as code on Terraform in an automated way. So we imported, we changed, and now it's done properly and Fastly was able to successfully renew the certificates. There is a lot of different causes that last from, that happened during the past 18 months that leads to that issue. So no need to go back and pass more time. You have details on the issue. It's fixed and there should be no problem. If anyone has questions or want more details, please ask. Okay. We had, so next issue, we had an outage on DigitalOcean Network that was particularly on requests coming from or coming to the US East 1 area of AWS, including repojunkinci.org and our platform with the DigitalOcean Network in Europe, where we have some of our agents. No builds were armed, but as a matter of safety, we disabled DigitalOcean for one full day from CI Jenkins.io until the outage was documented on DigitalOcean pages. That's what the documented is on the same area as what our monitoring shows. So we were right to disable that. No impact because CI Jenkins.io is also able to schedule workload on AWS. So that was quite transparent to the users on CI Jenkins.io. That was also an opportunity for a Kubernetes cluster grade. I will tell more about this one later. Finally, the fourth task completely finished. We had a user having issues with their plug-in, permission with the plug-in. So we gave them instruction and since three weeks we haven't heard back from them. But we did everything we could. Everything is green and they should be okay to proceed. So we closed the issue. I hope they will open if they see any more problem. The main problem was permission on the GitHub repository of the plug-in. They didn't have the proper permission everywhere and the team helped a lot on pointing the proper resources for the repository permission update. Any questions? Remark on these four tasks? Okay. We had an interesting amount of account issues. Most of the time people that seem to try their own Jenkins and they end up on our help desk. I'm not sure why. I haven't checked the indexing status of account Jenkins.io. Could it be that with the new IPs that we will be indexed on Google? I think that's worth checking. But there might be a reason why the whole end there. I don't know. Why do we have repo carouselabs? I looked. It was already done last week. We can't hear you, Ervish. I saw it that you were talking. Can you repeat, please? Yes. No result in the first page of the result for account Jenkins. Okay. There is a weird reason here. The carouselabs repository we already discussed last week. As far as I can tell. I added the milestone because I removed the exception in the settings for the artifact caching proxy which was left after the resolution of the testing on Maven. Cool. Thanks for thinking about that. We forgot about this. Is there any other task on this one that you had in the pipe or it's complete? No, it's complete. That's why I didn't reopen it. We might now need to think about things other repository like this one. Not really like this one, but it's the exception discussion for later. We don't have a problem with this and we can add more exception if needed. Let's finish the artifactory bandwidth first. Thanks, Ervish. Let's get on the work in progress. First one, Kubernetes 1.25 because I mentioned it earlier. Since we disabled Digital Assign for at least 24 hours or 30 hours, we said no one is using the Digital Assign clusters. One is the ACP used by the second cluster which is used by CI. Disabling from CI makes no one use both clusters. It was a great opportunity for us to start the upgrade to Kubernetes 1.25. Not only if we broke or if something went wrong, that's not a problem. No impact, so good exercise to get started. I can say the same for the other clusters in the coming weeks. That was also the opportunity to use the credits that Digital Assign are granting us for free only for using the highly available Kubernetes control plane. Great opportunity to break the service because switching to HA is shutting down the control plane for a few minutes. So we had a planned outage but no one was using it. By the way, that's a funny one. The credits for using HA is an incentive they gave the users, Stefan and I discovered that because in fact it forced the user to migrate the control plane to the new network architectures which means they have tons of users using the low cost single not HA controller with old network and they want to get rid of that old network. So the incentive is they give credits for people to use the new HA system. That's a nice incentive for migrating legacy networks. Yeah, and to accept the downtime of a few more times. Yes, true. Exactly. That's easier to accept downtime because then you won't have any more due to the HA. So everything went fine on the upgrade. So that means the next step I'm proposing and it's challengeable as usual. That will be the EKS clusters. So that one might have impact. We need to announce it to CI Jenkins IO maintainers because during the upgrade of this cluster we won't be able to run any bomb builds. The other build will be run on digital ocean because that's the opposite of what we did last week, but no bomb build. That's not a problem. Not a problem. We commonly only run full scale bomb builds when we're evaluating a pull request. So not a problem. We just have to announce it. Okay. Need announcement. Is there any question on the Kubernetes 1.25 topic? Right now it's hard to evaluate the downtime and the availability for us. So I don't know. Stefan, you are off starting Fridays, that's correct. So definitely Thursday. Okay. You will be working Thursday? Yes. Okay. But yet anyway, you won't be upgrading Kubernetes 1.25 to a way I guess that's you on high. I can have you on the phone in the car going to Paris, but that will not be quite useful. Hervé, I know you are off Fridays, that's correct. I will be off Monday. When are you back, Hervé? Friday to Monday. I'd like to ask for Tuesday too. Okay. So I guess either I do it alone Friday morning or we can move on next week. That will be the risk. So I'm at Friday morning and I will ask you, Hervé, for a review of my plan Thursday if it's okay for you. Does it sound good, Mark, given we have LTS tomorrow that might generate bomb builds after the LTS? It might, but it's quite harmless for us to say. Well, the bomb builds it will generate will be very low impact. They won't be the full test suite. The ones that would hit hard are the full tests. And there we just tell people we'll be offline while we do this upgrade. We need to do these upgrades. I'll be able to review it on Friday on the road during a post or something. And also this topic is a little bit less odd now that since Digitalison was the first one to switch to 1.25, the other a little bit later. But we have also AKS. And AKS is required in order to finish the upgrade from Ubuntu, Bionic to Jami. We need to have Kubernetes 1.25 because that one unlock worker using the new Ubuntu. And that's why I started working on this. And the thing is that we need to migrate first Digitalison and AWS before Azure because it's less prone to problems and it's a linear complexity. So that's the reason why I privileged. Otherwise, you are correct. If we didn't have the AKS with Ubuntu 22 challenge, I would have delayed that one in a few weeks. Good point. Really good point. Is there any question? Nope. Okay. What did we have? Proposal for application. So cluster, survey, your turn. That is the most important one for now. Cluster migration. Can you share the status for this one before closing? Yes. So I've done a lot of cleanup and follow up after the migration of every services. Almost all of them are done. We have some additional tasks which can be done later. We've deleted almost everything on the ProdPubic.js cluster. Your orbit is still running on it. Damian made an announcement about it last week. And we've planned to destroy this cluster today. At the end of the day. Almost three years for this cluster. Three hours of uptime. Motion three hours of uptime. Deletion to be done today. So based on that, if we delete it either today, even if we are late and do it tomorrow, that task will be closable for the upcoming milestone. Is that correct? I would have wanted to do it today. I'm sorry for that array, but yeah, that was a bit short. But so congrats. That's a huge win. Which give me the perfect transition to that issue. So after the meeting, I will also try to, that one most probably will be merged into an existing one. Now the next major step we have with that new cluster is to ensure that IPv6 is enabled for GateGenKinsey. Two reasons for that. First, we don't have overlap and everything has been set up and it's better to do it as soon as possible. And even with that, we might have missed element, but we can fix it. And that new issue has been issued. So thanks, thank you and thanks, Padek. That was an email from a user. Since we have migrated GateGenKinsey to the new cluster, and since we have IPv6 records, the RFC compliant clients, such as Curl or some Java client, are trying the IPv6, but it doesn't answer. So depending on how the proxies are configured, some users see the service as broken. Oh, we've had a lot of delay for the fallback too. Yes, absolutely. So we need to fix that IPv6 problem. I checked a bit. We don't have a lot of elements to solve that issue at first sight. So that's why I propose that we work on this one as soon as possible once we have destroyed the old cluster. Is that okay for everyone? So the priority is solving the problem with GateGenKinsey. We have other services that could benefit from running using IPv6 publicly, such as AccountGenKinsey or the other public services. But the priority here is to get GenKinsey to solve the issue that has been described by RV. Is that okay or that's absolutely challengeable, but I propose that we scope to this one and maybe keeping that issue open. And then we will follow up on the existing issue about IPv6 everywhere. Yes. As a reminder, IPv6 is the default outbound system in India since September 2022, which means with this one we will help a lot of our friendly Indian colleagues. Any other question about on the IPv6? Okay. Yeah. In that case, that's the person from Dutch Telecom survey. But yeah, makes sense. That's their specialty. Artifact caching proxy is unreliable. So yesterday I was able to finish after some fix ups and fix ups of fix ups and fix ups of fix ups of fix ups. But now the Azure Virtual Machine Agent of CI GenKinsey are running on the same network as the ACP Azure. So they change region and they change the way to communicate and they have way less latencies because there are two subnets of the same virtual net now. There is no cross virtual net changes. So we are ready to continue to retry enabling ACP on the ATH. I'm not sure Mark if we should bother Basil or start again from the late work he did because the pull request where he tried has been merged without the ACP enabled for most of the ATH builds. Only the first, there is the first quick things that generate a jar file which uses ATH today, but all the big builds are not. So that's worth trying again. And if it works without issue, we can close the issue. Otherwise, we can start diagnosing why is it failing again, but that most of the issue are TCP connection error, which is typical from the network overlap. Now we don't have the network overlap now. So I'd say let's explore it ourselves and we can submit the pull request to propose it and watch how it evaluates. The same network as ACP. Let's evaluate ourselves first. Is there any question on that topic? Just a note on this one. I published after a bad copy and paste, I published the datadog key in clear. I realized that and I rotated the key less than two minutes after and I didn't cook any metric sense with that key during the two past minutes. That key was only used by CI Jenkins IO agents. So it appeared that some of us have a Git guardians or service like this one that detects the key, which is a really good thing. So I'm sorry for that because I didn't communicate to the whole team on this one. So I will try to do better next time. Yeah, sorry. But the key has been rotated. So no worries. The risk is, worst case someone was able to send metrics to datadog. And I'm so pleased to see that you can make mistakes too. I'm not a machine. Thank you. That's what a machine would say. Sorry. On that topic, CI Jenkins IO. So now I have everything ready to prepare the migration of CI Jenkins IO to a new virtual machine in a new network with a new generation of CPU, etc, etc. So I'm working on bootstrapping again that virtual machine now. And the next step will be migrating the data file system and plan a date for that, most probably not that week. So I assume next week, Tuesday, eventually Wednesday next week. But I need to work on that topic for that milestone. Is there any question on that new CI Jenkins IO machine? Okay. Mirrored Jenkins CI is missing some necessary metadata file. Important topic here. User complain that it's really hard to use the project behind Fastly in China. There is no Fastly node inside China network, which means for Jenkins.io websites, the plugins website, but also pkg.jenkins.io. And that's the problem here. pkg.jenkins.io is the official way to install packages for installing Jenkins on Red Hat or Debian. That's quite the issue for them. So they realize that we have the mirrors and they try the mirrors, but the mirrors are missing some indexes because the package index is expected to be at pkg.jenkins.io. So we have a problem that we will have to solve here, which is close to another issue that we had a few months ago and we weren't able to solve. The index here proposed to download files from getjenkins.io slash something slash dot deb dot rpm dot whatever. But it redirects to a mirror, but some of the mirrors don't have unlimited storage in time. So most of the time it's not a problem except when someone wants to mirror everything from the beginning. And in that case, they start to run on issues. So we might have to think about pkg.jenkins.io availability for a China user. I don't have an easy solution right now. It's a complex topic. But the good thing is that discussing with that user helped us to discover that it looks like we already have mirrors available in China that we don't know about. The user mentioned them on the issue. So at least the takeaway from short term will be adding these other China mirrors. That's for me, that's really good new. But yeah, the whole problem of the man on the middle possible attack. How do we manage all of this? That might be complicated. So the problem is yes, we have digital in India, but you have the great firewall. It's a matter of network latency more than location. But that's the default. And we have a mirror inside China already. It's hard for our China users to download. Mark, I think it's worth adding that to the next, to bring that outside the Jenkins infrared team at least, especially with the context of us working on removing the China Jenkins website. We might need to start searching for a solution equivalent to Fastly, but that could allow us to project local local files or local CDN in China mainland. Is it worth it? Do we have users? But given that the user here is taking time and it's not the first time, I guess we have to start that that question on the broader, yeah, outside the borders of the Jenkins infrared team. Agreed that it's if we're going to do a different, if we're going to a different form of mirroring that needs a broader discussion than just, than just infra, absolutely. Eric, do you happen to know if the new Cloudflare, if Cloudflare has CDN nodes in China? I don't know, but I'm on the, on the page saying that they got one, the Cloudflare China network. So that could be interesting to have both Fastly and China and Cloudflare then outside the Jenkins infrared team. So if it's okay, we'll comment out saying, is does it make sense to ask the user to help us to, to bring that subject because we need a kind of sponsor that is able to drive it from China. They won't be left alone. We will work with them, but we need someone there to help us. As far as I remember, this question was raised a few years ago. We started with some contributor, but then the contributor stopped being there. So that was hard to have a sponsor locally. Is, is this a place where Cloudflare is also a candidate for at least consideration as, as one alternative? Alternative or complimentary. Complimentary. Okay, got it. Both are okay. It's just a few, a few more lines when we deploy, but that, that can be an idea. Also, we have the same problem with the update center, but again, that's a topic I don't want to spoil everything right now. Okay, thanks. Thanks, Stephen. And also, it's worth contacting Fastly if it's okay for you. I don't mind contacting them asking, okay, we have that problem, want to project in China, do you open to have project ideas, POC? Because that's, that's important. That's, that's a legit use case. And we're not alone with that. Any other question here? Okay. We'll comment on the issue. That issue will go to the backlog because I don't see any direct action for us. We, alas, we cannot help the user as it, unless someone has an idea. And in that case, I'm happy to help. No, okay. Okay. Just to avoid spoiling too much, next topic I propose is data dog integration on CI Jenkins.io to observe the instance and the builds. Harry, can you give us a heads on that topic? So you resolved the issue of the communication between the data dog agent working on the CI controller working inside the Docker container. From that, the plugin has been configured for working correctly. But now we have two set of trace. So when we previously add on, which probably comes from the, the catalogs collector, why is the new one is the one collected by the data dog agent. So one is correctly recognized as a Jenkins integration and has everything every event, every tags, everything. And the other has nothing. I've displayed my custom parser and it's currently shows this log appearing as error. And they don't have any global tags in them. I've tried to add a global tag in the plugin configuration and say that Jenkins.io. And while the patent got back, I saw the tag in the data dog tag. But anyway, since the other source did not add any tag, I think we have to close this collection. I don't know if there are other container collected on this instance. I don't think so. No, we don't have others. So that makes sense to disable it on the instance. That's okay. I have to do that. And I've added some point to the issue like infrastructure integration. And the last point, but I think I won't need it for closing this issue. Okay. I'm not sure to understand what is infrastructure integration. Let me open the issue quickly to what I meant like that. Okay. So, okay. Is it okay during the upcoming days? You will try to describe what you want to do with this one. You are having network issue, right? I'm the only one. No, it's okay. Okay. So I see your way. Can you hear us? Okay. No problem. So then the next step for you will be to a message. Okay. Cool. Thanks for this one. There is a nice CI integration section on Datadog that is now connected to CI Jenkins.io that shows traces. So we can now, through Datadog, see for each build of CI Jenkins.io, how much time did we spend on the checkout, on the SH steps, on allocating agents. There is a lot of pipeline build errors or time, or there is a lot of information already available now. So for everyone, if you want to diagnose bomb build slowness, that's the next step. That works very well. I don't know if we can open it publicly or to just, we might need to create restricted account, but that could be useful for everyone. But now we have a full observability. Thanks to that work. So thanks for that. That's really useful. I know how to expose dashboard publicly, but I don't know if we can restrict access to that. If we have dashboard of only CI Jenkins.io, that's okay. We just have to be careful not having infros CI Jenkins.io publicly. Just about these logs, we have also the logs from CET-CI and Prestid, but they are ingested like the previous one as container logs. And these are container logs, not build logs, which is way different. You don't have credentials in the Jenkins coin, but you might have credentials on the build logs. The issue with them is that they aren't recognized as Jenkins integrated logs. Absolutely. And in order to do that, that's a setup puppet to the data.agent that collects the logs. You have to do this that way. That's the alternative to the plugin as far as I can tell. Is there any question on that topic? So I consider we'll work on this next milestone. Matomo started, I've tried to draft a plan under a summary of what is needed to host our own Matomo instance based on all the information that gave in shared. So the next step will be start the work. The first step will be ensuring that the image, the Docker image to avoid persistent storage and to have some plugins and setup in the image is working and published. Then we will have to set up on Terraform the MySQL instance and the Elm chart and we should be able to have a first installation. So we only have a technical task to do now unless I missed something. So I propose to keep that issue on the upcoming milestone and we will do tiny steps by tiny steps. I don't think we will be able to finish it for this milestone, but at least we can get started on real tasks. Any objection on this one? Okay. Let's go. MySQL instance plus Docker image. There is a, yeah, in fact, Gavin already gave us a lot of information so we don't have a lot to think and more to act. Thanks Gavin for that. Stefan, can you give us a head on the IRM 64 migration on public, the public cluster? What's the status? Did we load? Oh, just a message from Stefan. Okay, he lost the internet. Okay. I'm moving tasks. My turn to share Ubuntu 22 upgrade campaign. So we did upgrade the old VPN that allows us to access third CI and CI Jenkins IO. So VPN is done. The CPU Z machine as well. So VPN, we had issues that has been solved. So summary is be careful. Sometimes default settings change between distributions. So the default forward policy for IP tables was a low on Bionic, drop on Focal and is again a low on Jami. And of course, they have a nice migration assistant that say, hey, if you have a setup, I will keep it. So once it has been set during the first upgrade step, it was set to drop. It was kept as drop. Even though the migration assistance was saying no change. Yeah, it took me quite some time to understand why the VPN was working, but I wasn't able to reach CI Jenkins IO, a third CI Jenkins IO from the VPN network. Fixed and documented. So yeah, two hours of VPN outage. Sorry for this one. And the next steps are CI Jenkins IO. That will be the virtual machine migration and AKS. So we have a CI Jenkins IO VM migration, AKS 1.25 cube. And the last one will be updates Jenkins IO. That's the most complicated. I don't have a plan right now. About that airway, you had a nice proposal that we haven't mentioned there, but that was mentioning it because that could be the solution for the open to upgrade for updates Jenkins IO, at least for the part in charge of the update center, not the PKG. Are you okay to share your idea? Yes, since Cloudflare R2, S3 like object storage blob product has zero egress fee. I think we could try to use it to distribute the update center system file, which is consuming a lot of common with everyone. In detail, we seem to have what we need to have a S3 compliant storage with that. They also propose an open source sponsorship. So it could be nice to ask them to submit an application for that. I took a look at the existing issue and CIP. I have some point to discuss. I think, like for example, if I understood it correctly, we have updates at Jenkins IO serving an HTTPS update, but we have also updated Jenkins CI.org, which is serving these updates without any TLS for Java version 8. So I was wondering if we have to keep that and if it's still the case today. Yeah, and the answer to that kind of question is let's talk to Daniel Beck. He has forgotten more about the update center and its complications than I will ever know. Absolutely, and that's worth having that topic in parallel, because if the answer is we don't have to maintain the no TLS for whatever old version since we are using GDK11, if we are able to say, oh, okay, we communicate one time with a blog post or one community, and then we add a redirection update Jenkins CI.org will be an HTTP permanent redirect to the Jenkins IO with HTTPS like we do for Jenkins IO website. And that one would help, but good point. I forgot about this one. So yeah, nice check. And the other part of the study is you have to check and trusted what exactly is the pipeline running since as I understood they aren't stored as code. And until today, I didn't have a proper installation configuration of my new machine so I couldn't access CI or trusted. Yep. So if you need, because the non-infraars code jobs on trusted CI are in fact just a bit of glue, there is just a few environment variables and it runs a shell script. And the shell script is available in GitHub. So if you need, I can take screenshots. Yep. I saw the generator on publish, but if it's only glue, I'll see when I saw it. Data center updates. But the idea is that that shell script will be added to AWS S3 commands. In the first time, we can absolutely start with the air to object store. And the command would air sync the generated JSON to the current update VM and also copy it with AWS S3 CP to the new one. And then we can start testing on that new system. As I was saying, the no egress fees is really appealing for us. But also that means high availability for the update center. No need to maintain a virtual machine with Apache server. And we have Cloudflare as a CDN network everywhere, including China. So that could be a double win here. So that's really a good idea. And even if we have to pay, because we'll have to pay a bit, the costs for the update center use case, that's really nice catch by our way, because we don't have a lot of storage. It's expensive if you have a lot of storage. For instance, PKG Jenkins IO with a gigabyte of packages. I don't know, but at first sight, it's more expensive than AWS S3. In the case of the update center, that will be worst case, 40 bucks per month compared to the six to seven K per month of bandwidth, not mentioning how much we spend on the virtual machine and the storage. So clearly that will be a win. So good catch. There is an issue about update center migration to another cloud. If it's okay, we will add this one to the milestone and I will add the link here, because the topic is only the update center part of the virtual machine. We will sync on that part. The second part though is PKG. And for that one, we might have to work on, yeah, there is still some work to do for Ubuntu upgrade that will require help, I think, from core maintenance. Yeah. Isn't PKG the one that builds the installer for us? That's the one that has that strong coupling right now to the Jenkins core installer and the change from Ubuntu 18 to 22 is a dramatic change in that area. My understanding is that Ubuntu 22 has been done. We cannot use Ubuntu 20, but never in real life. So we need a way to test it without triggering a release that will be safer for everyone. I think that's the most because maybe that could just work out of the box. If it's okay, I propose we finish the cube IPv6 and with airbay whenever we'll be able to start working on that part, that will be a subject I'm volunteering to take unless someone is interested. Good for you. So Stefan was back and his internet is dead, so I consider, I don't know the status of this one, but given he will be off, we should be able to either hand over this one if he has a direct item that we can do, most of the time, that may be migrating services. Otherwise, it will go back to the backlog. Okay for you? Yes. Backlog or handover from Stefan. Also, that one is on the milestone, I added it so we can mention it. That was related to the need to remove the IP restriction to access trusted CI through the SSH passion, the bounds machine. The work here needs to be underlined because we need to plan what has to be done in order to use the VPN as a way to restrict access to the bounds machine or find another solution. I propose this one back to backlog for now, because it requires a lot of network issues and we have to finish the public network, the cleanups, IPv6 before going on this one. Most probably, that one will come back to us really quickly because might need to be done for third CI. Today, third CI is inside the subnet of the whole overlap network and is available only through the legacy VPN. So we will have to migrate that instance on a new network and we might need or might not that has to be discussed with the security with a threat model session. Do we need to create a dedicated virtual net with third CI that will be the same pattern as trusted, meaning we need the VPN to have a way to reach these separated networks. Can we move third CI on a subnet of the private network like in third CI or release CI or do we need another solution? But most probably the pattern will be same as trusted, eventually in the same network as trusted. But in any case, we will need to connect the VPN and create a network peering that requires some work and planning. So that's why I propose back to backlog for now. We don't have a blocker here. Is that okay for everyone? Yes. Last item for today is assess architecture bound with reduction option. So Mark, can you remind us just to help my brain to focus on that topic to context switch? What's the status for this one? So we've got to I owe a whole request proposal and the beginning of a discussion about the changes to the plugin palm and the Jenkins core palm to switch to generate an incremental that we can use to test drive the switch from public everything to public only the components we build that we deliver and lots of work still to do there. We have to prepare on palm builds. Oh, okay. Now my brain is inside. Yeah, so it's we know we need at least two and we've got we expect lots of conversation about this and a number of places where hey, the info team including me are not experts in Maven Maven configuration and consumption. So we'll look for coaching and help from others in the from core maintainers and others. Thanks Mark. So on our side on the infrastructure as administrator, I would need to create an alternative repository tree with one or multiple private repository and that alternative tree should be used as a support with the parent pumps. The goal is to apply the proposal we made. I've started the documents that Mark and Derwe have started to review. I will want to review it a bit with a short core of people including maintenance and as soon as we start to have a consensus on this one, I will take care of adding it to the issue so we can start having a public things. We need also validation from JFrog. Right. Yeah. To review before adding to the issue. So I'm keeping this one on current milestone. Do you have other topic while I'm looking for the triage issue? I do not and I think other than the the complication of the bandwidth reduction project, that's the that's the big one and it's going to be a lot more work. Okay. I don't remember which one we discussed on Monday when reviewing the cleanup of the public. It was IPv6. You're lucky. A user did it for you. Victory. Is it bad or is it good? I don't know. It's good for you. It's more for us but yeah. A good point about the Windows 22. I saw a question on different locations, public and privates about the Windows 22 support for the images. So everything is ready. We are just waiting for someone to spend time updating the existing pull request on the agents, Docker agents images to start choosing the new Windows 2022 images. We need someone to spend time on that. That's what is missing. But we have both Windows 2019 and 2022 available with Docker, CE and everything ready. They have the same content so they can be used and we provide the labels and they are documented. So that should be just a matter of spending time on shaping the pipeline. So anyone interested is welcome. I haven't had that time and I don't think I will for now but if it helps users they can contribute there. Let me add this one. New issues. I can't remember if we already had one. Was that, but I didn't find it. So we had one and we had an issue because for the Jenkins Infra it's closed because we did our work and the agents are available and documented. So now someone has to use them which is out of the scope of the Jenkins Infra. But that's a good question to raise because since no one worked in this one, ready to start working. But nothing to do on IFRA except support. So those are the pull requests from the user from March. Exactly but the pull request was done before we provided the images as far as I remember unless I missed another one. But yeah if there are infrastructure issues of course we can open an help desk and support them of course. Oh no it was not from March. It was from March last year okay. Thanks for checking and sharing that concern. Oh Stefan just in time. Can you hear us? Can you hear me? Yes. Okay so it's okay can you describe us the status of the IRM 64 migration of some of the services on the public cluster? Yes I did add two lines in the what not moving them in. I have some interrogation about let me check SIRT manager. There is three as you can see SIRT manager cane ejector and webhook. I think there are IRM compatible but I would like to double check with anyone. I think we can start kind of easily with the migration of report because it's just engineering and it's already compatible with IRM 64. But in fact the first step is to lower the level of the Javadoc which is already IRM 64 for it to be able to spawn on non IRM to change the setting of the nut pool because we need to change the taint if I remember correctly the word and then go back on the migration again. Okay thanks for the reminder. True this looks good so is it okay for you to be your mission before the holidays or the two days that are left? If you give me a few minutes of your time to kind of finish not really finish but stop the the parallelism of Packer image for my brain to be ready to move on that task that will be awesome. Yep but that's just for my brain. Yep we'll do a bit today later and a bit tomorrow then. Perfect. Sorry to have missed all the time. There is a recording and notes so no worry. Thanks for going back. I think that's all for me we don't have any more triage issue. I don't see messages from you about new topic do you have other topic to bring or or we can close. I think we can close one two three we can close. So for everyone watching the recording see you next week. Nope.


Good evening, ladies and gentlemen. Welcome back to those who were here for last night's lecture, and a very warm welcome to those of you who are joining us tonight, just for this evening. My name is Nargis Farzad, and I'm the current chair of Center for Iranian Studies, who are hosting these amazing lectures, the Jam lecture series, which we started and inaugurated them in 2012. So it's a rather special year this year. I'll spare you the other bits of last night lectures for those of introduction for those who heard it. But I wanted to say that I'm sure you agree with me. In the spellbinding lecture of last night, Professor Robert Hillenbrand introduced us to the great Mongol Shah Nome and the importance of this extraordinary illustrated manuscript of Ferdowsi's Book of Kings, which for those of you who may not know of it, the one or two of you, who I'm sure will not admit to not knowing about this amazing book, is an epic poem whose composition took almost 30 years before it was presented to the Ghaznavid monarch in 1010. The book narrates the history of Iran from the dawn of time till the Arab invasion of Iran in the 7th century CE. Professor Hillenbrand drew our attention to the large scale and the style of paintings of this Shah Nome last night. Yes, I'm so sorry, I just somehow lost my place there. Absolutely. The manuscript that he dated to around 1330, and he spoke of the emotional intensity of the images, the eclectic style, the individuality of characters depicted, the artistic mastery, and the splendor of its illustrations. Professor Hillenbrand trained our eyes to focus on the broad color palette with the dominance of yellows and reds and opaque watercolors, as well as the gold and silver. The manuscript, which in its original form probably consisted of about 280 folios, I hope I took these notes down correctly last night, with 190 illustrations, although it is thought it was never completed, left Iran in the early 20th century and was broken up and its pages were scattered. And if I'm not mistaken, you mentioned that there now exists some 57 individual pages in a number of collections around the world. Having mesmerized us with the title of the talk last night, which was, what makes the great Mongol Shah Nome great, we need to find out why the evening tonight is entitled, What Problems Does the Great Mongol Shah Nome Pause? So to find out about that, please welcome Professor Hillenbrand to the podium. I want to reflect tonight on one of the best known works of medieval Iranian art, the great Mongol Shah Nome, but paradoxically, any closely focused study of it reveals not only huge gaps in information, but also deep seated problems that hinder our understanding of it. The familiar problems of date, provenance and patronage, have monopolized the attention of most scholars who have tackled this manuscript. Of course, it's plainly unwise to proceed with say, with any analysis, until such basic facts are established. But the corollary has been the neglect of many other curious aspects of this enigmatic masterpiece. For many specialists, the great Mongol Shah Nome has been for many decades, above all, a problem. For it so obstinately refuses to fit into what by general consent constitutes classical Persian painting. Small wonder then, that for far too long, widely admired and widely neglected, it has remained the elephant in the room so far as the history of Persian painting is concerned. That's a moral there. So let me get some basic information out of the way before we come to the meat of this lecture. The great Mongol Shah Nome was in the Royal Adjar Library in the later 19th century, before being illegally, I stress that word, spirited away around 1910 to Paris. There it fell into the hands of a pantomime villain, a Franco-Belgian dealer by the name of George Joseph de Motte, and at his behest, it was disbound so that the 58 paintings it then contained could be sold separately.  Let's have a look at him, seen through the eyes of Matisse. Pages that had a painting on both sides, on both recto and verso, were split so that those paintings could be sold separately. He glued newspaper onto both sides and went, and of course, there was damage. The damage in one case has been hidden by writing. In the second case, it's just blank. He adds new extra text to disguise what had been done. It's hard to disentangle de Motte's interventions. They are a bleak commentary on the lengths to which greed can drive a man ostensibly dedicated to fine art. Scattered to the forewinds by de Motte and now preserved in 22 locations across the globe, these 58 paintings have offered all too few opportunities to be studied on block, and that's what the art historian wants to have the manuscript in front of him or her and to be able to study the whole thing. No possibility of that here. Five exhibitions in the past century have displayed a score of them, one third. But close prolonged study of a painting while you're standing up in a crowded exhibition is a pain on all counts, believe me. No information has come to light as to the date of the manuscript, where it was produced or the name of the patron. Most of the leaves which had no paintings have disappeared without trace. Conventional wisdom has settled on a date between 1320 and 1335 for it. On Tabriz, the Mongol capital in northwestern Iran, as the place of production, and on either Vizier of Sultan Abu Sa'id, the last Mongol ruler, or to that Sultan himself as the patron. The project came to an abrupt end, unfinished upon the untimely death of the Sultan in 1335, poisoned by his wife. There were later disastrous interventions. I will draw the veil of discretion over them. They're 19th century work. But the word unfinished is, as will become clear, pregnant with implications. So far so good. This brief history is enough to make it plain that the detailed study of this manuscript bristles with too many problems to touch on meaningfully in a single lecture. So I'll limit myself to four problematic issues. These revolve in turn around innovation, the production process, the number of illustrations, and the manuscript's legacy. Much of this material is not dealt with at length in my recent book on this manuscript. Tonight I will investigate the issues around innovation under three interrelated headings, size, propaganda, and teamwork. The sheer size of the manuscript is the most obvious innovation. No other illustrated shah-naame of any period comes close. The original dimensions are irretrievably lost thanks to the radical refashioning of the manuscript in 19th century Iran, which involved encasing the text, the text block in fact, together with its attendant paintings, in a broad frame of 19th century Russian paper. The paper historian Jonathan Bloom has estimated that the original sheets of the manuscript measured some 80 by 60 centimeters. That means that when it was opened, a double page spread of the great Mongol shah-naame would have measured about one meter, 30 centimeters across, including the binding. Now excuse me for using a vulgar visual aid. Imagine a book this size. Imagine reading it. This takes us well beyond the limits of comfortable reading. You could not hold this book on your lap for any length of time. Its sheer weight plus the binding would necessitate a book stand. And it's not only the reader's muscles that would be sorely tried. Actually reading the text would involve constant movement, either of the book itself or of the reader's body. In practical terms then, it would have been physically taxing to come to close grips with this book. Now some of you in this audience will have consulted Kreswell's great volumes on early Islamic architecture. Allow me to confess how much I hate consulting those books and all because it's a physical challenge to do so. How much more I would have consulted them if only they'd been half the size and half the weight. A Kreswellian volume measures some three square meters and weighs seven and a half kilos. But it is a mere pygmy beside the Greco-Mongol charname which would originally have been well over three times as big, measuring some 10 and a half square meters. So this charname in bound form must have approached the weight of 20 kilos, which is the luggage you're allowed onto a plane. It's not certain whether the original was planned to comprise two volumes or just one. The project was never completed. So only the residue, a mere torso, was bound at an unknown date. But it's relevant here to recollect that the copy of the Quran believed to have belonged to the Caliph Uthman in the 650s was kept for centuries in the great mosque of Korodaba and venerated as a relic. And it required two men to carry it. Not many scholars have a sideline in weightlifting. Clearly mammoth size carries with it significant disadvantages. No wonder that this format, like the mammoth itself, became extinct. People in glass houses shouldn't throw stones. And so I must hope that my own book on this charname, which weighs in at a mere four kilos, will not remain unread for that reason. The huge size of this charname is integral to a brand new aesthetic of book production in Mongol Iran that has its parallel in both contemporary architecture in Tabriz and in luxury Korans. But that size had to be justified by something more than huge empty margins on three of the four sides of a typical leaf. In this respect, the manuscript followed current fashion as can be seen in both contemporary and earlier Korans. The option taken up in some of the imperial Mongol Korans was to enlarge the script very substantially so that the sheer size of the letters proclaimed the sacred status of the text. Something similar could have been done in this charname, but it was not. Its text block had 31 lines of text disposed in six columns. And thus its format, though larger, does not diverge markedly from that of contemporary charname manuscripts on the left, as you can see. But presumably the images themselves had to be in proportion to the unprecedented scale of this book. And once that decision had been made at the outset, the next challenge was how to fill the spaces, set aside for the images. That was indeed problematic. The simplest option of all would have been a pro-Rata enlargement of stock images, but that would instantly have revealed their threadbare quality. Imagining that from a small charname. Besides, many of the scenes illustrated in the great Mongol manuscript were new. And even when the painters were constrained to follow existing models, one can sense that they were under pressure to introduce novel features, the sheer variety of tone, of composition, of color in successive enthronement scenes, and also in scenes of single combat or battle. Or monster slains, all cases in point. The extra space now available because of size left ample room for the inclusion of telling details as well as room to recompose the whole idea. Enough of size. Let me now touch briefly on the element of propaganda. It's clear that this was not a book to be read continuously or to be poured over. That would involve too much physical discomfort. A huge dizzyingly expensive book, not intended to be read, must have had some other primary purpose. It was essentially a display copy of the charname, meant to be appreciated at a distance. And the mere size of it was a knockout blow. But it's more than that. It's an attempt constrained by the methodology available at the time to make the book something new, do something new. And its colossal size was part and parcel of that object, for it signalled in unmistakable visual terms the unique importance of this manuscript. It did much more than celebrate Phaedos's text or exhibit the work of the best painters of the time. It was an instrument of propaganda aimed at Persian and Mongol alike. Of course, that audience was very small. One might argue that this makes the very notion of propaganda somewhat counterproductive. After all, propaganda tends to be aimed at the masses. But in this case, the audience comprised the ruling elite of the land. Once they had received the messages of this manuscript, that information could trickle down to many more people. Simply by existing then, this object proclaimed the might and the wealth of the Mongol state in a way that everybody who saw it would immediately understand. But it also honoured Persian national sentiment. And this could be construed as a major step forward by the Mongols. Indeed, it was both an acknowledgement of that reality and a commitment to it by the Mongol state. It was a testament of rapprochement and acculturation. And the insertion of multiple covert references to episodes in the history of the Mongol dynasty in Iran over the previous century could even be termed a hijacking of this most beloved Persian text for Mongol consumption. And it might also have been an olive branch. In 1339, the seasoned administrator, Hamdala Moustofi, described the impact of the Mongol Holocaust of the 1220s as follows. There can be no doubt that even if for a thousand years to come, no evil should befall the country, it will not be possible to repair completely the damage and to bring back the land to the state in which it was formerly. Indeed, the trauma that the Mongols inflicted on the Persian psyche is still remembered to this day. And modern scholars have calculated that northeastern Iran, namely Horasan, has still not recovered today the agricultural productivity it had in pre-Mongol times. Gioveni, the top Persian official of the 1290s, put it more succinctly. Amadand, Kandand, Sohtand, Kushtand, Bardaand, Raftand, they came. They uprooted, they burnt, they slaughtered, they plundered, they departed. Now the destroyers had morphed into creators. And this copy of the Persian national epic was perhaps intended as a monumental memorial of that transformation. A century earlier, the Persians had indeed lost the war, but now they had won the peace. My second sub-theme of innovation is teamwork. It is in this politico-cultural context that the teamwork that made this Sharnameh possible falls into place. For this is emphatically not an example of a luxurious private patron, some patriotic Iranian ordering the illustration of a book by a fine painter with the help of a few lesser lights. It is a state enterprise under state control, the production of a state atelier funded by the state and located in the state capital of Tabriz. All this increased the prestige of the entire project, while also loading a heavy weight of responsibility on the shoulders of whoever was in overall charge. A man we might term the project director, of whom more or none. He had to find the necessary scholars, scribes, painters, illuminators, and other craftsmen. It's likely enough then that dozens of experts, quite apart from the necessary administrators and accountants, were required to run this project successfully. In particular, to execute the enormous cycle of perhaps 216 unusually big paintings was of course a task far beyond one or two painters, however talented they were. The project director had to create a large team with no room for hacks. How did he pick them? He had a very recent experience of illustrating multiple copies of the Vizier Rashida Dean's monumental world history in his purpose-built Tabriz atelier would have been invaluable. Word of mouth about this splendid charname would have spread quickly so that likely candidates would have flocked to Tabriz. And whatever way they joined the team, they would very quickly have learned the need for coordinated teamwork. The size, complexity, and sheer number of the paintings required, and the likelihood that speed was of the essence makes it probable that more than one person worked on a given painting. The necessary microscopic and even laboratory examination required to distinguish different hands has not even begun. Certain features such as the depiction of architecture, costume, armor, weapons, horses, furniture recur so often in these paintings that it would have been sensible to delegate their depiction to painters who specialized in one or other of such features. There are plenty of European parallels for this. A few other painters, presumably the leading masters, among whom were men of genius, would be responsible for plotting out and possibly executing certain compositions. Attempts have been made to name these artists, Ahmadi Musa, Amir Dolot Yair, Shamsuddin, Abdul Hay, on the basis of the 1545 account of the librarian, Doust Muhammad, but their active careers ceased, well, we don't know about ceased. Certainly long post-dated, 1335. I wonder whether the traditional master pupil relationship which has so often been invoked as the model for plotting change is any more useful in Persian painting than it is in calligraphy. Perhaps the great Mongol shanami was more likely the outcome of a massive and prolonged tea method whose detailed interactions, it seems, cannot now be recovered in detail. Obviously, not all paintings were of equal importance. So perhaps there was both an A team and a B team. Probably multiple paintings were being worked on simultaneously so as to ensure that the schedule of work was met in timely fashion and did not falter. Some modern scholars who have guessed how long the project was underway and how many artists took part have differed wildly in their estimates. As to chronology, Sudavar has proposed that it was launched in 1315 before Abu Sa'id came to the throne in 1317, though a date up to a decade later is also plausible. Gray guessed a maximum of six artists working for at most three years. Stukin proposed 12 artists whose output he divided into 18 subgroups working over a period of some 80 years. A theory that no subsequent scholar has espoused. The project, though, was almost four times bigger than either of them had envisaged and that misapprehension compromises their conclusions. It's worth reflecting briefly on what such tightly knit and closely overseen teamwork might imply. Ideas and techniques would be shared either openly or by stealth. The close proximity within which members of the team worked meant that everyone concerned could learn something from everyone else. What to do and also what not to do for young painters to work closely alongside masters of their craft was inspiring. It was an ideal environment for teaching and learning. Competition and emulation were in the air. It was a hot house. The exhilaration of playing a part in producing the grandest book of the age, Bread, Pride and Desprey de Coeur and inevitably, the pace of change quickened vertiginously. The paintings themselves offer convincing proof that their creators were encouraged to put their own interpretation of the text and its implications into the picture that they were creating. So the painter could become the pictorial equivalent of a literary critic. And thus, a radically new approach to book illustration came into being. So much for the problems connected with innovation. These justifiably gobble up the lion's share of the problems posed by this manuscript. Taken together, they suggest that the project was trying to do too much all at once. For there were also plenty of other problems to solve and that takes me to my second main theme, the production process itself, which must have posed some truly daunting challenges. The participants in this project were pushing the envelope on all fronts and some mistakes and failures were inevitable, though we can disagree about which they are. This was a new kind of shaname and it demanded correspondingly new approaches if it were to succeed and the problems generated by the project as a whole were far too complex to be left to the painters alone. Someone had to work out a schedule, not just of time but of content and of people management. The large number of craftsmen involved made it imperative, as I said earlier, that some sort of project director was appointed, perhaps the first of his kind. Moreover, the number and difficulty of the tasks would have been beyond the powers of any one person to perform alone, especially if he were working against the clock. So a supporting team was required, even if he himself bore the ultimate responsibility. Now, who might this person have been? No contemporary evidence exists which can shed light on this question. Two later clues are worth mentioning, one from Timurid and one from Safavid Times. A document, known as the Arzadasht, preserved in Istanbul and dated 1429, was drawn up by one Jaafar Atabrizi, a noted calligrapher for the Timurid prince by Sungur. This document lists in detail the projects on which the various artists in the atelier in multiple crafts and techniques were engaged. It seems reasonable to infer that Jaafar was in charge of the atelier. The Safavid evidence is harder to interpret. It consists of a tantalisingly brief and ambiguous statement by the librarian Dost Mohamed in his preface to an album dedicated to Prince Bahram Mirza, dated 1545. It reads, after significantly paying homage to both European and Chinese painters, that in the reign of Abu Sa'id, I quote, Master Ahmad Musa lifted the veil from the face of painting and the style of depiction that is now current was invented by him. Here is a picture attributed to him. Clearly, this painter was important enough to be remembered two centuries later. But to go further than this, for example, to identify him as the guiding spirit behind the manuscript or even its organiser is to speculate. Similar speculation has identified three painters, Sultan Mohamed, Mir Mousavid and Ava Mirok, as the successive directors of Shah Tahmasp's Great Shah Nome produced two centuries later. And that same Shah appointed the painter Bechzad to be in charge of the Royal Library, though his exact responsibilities vis-à-vis those who produced manuscripts remain unclear. In sum, it seems plausible enough that oversight of the Great Mongol Shah Nome project could have been entrusted to an artist. But it would also have been prudent to appoint others, including professional administrators, accountants and specialists galore. The principal challenge of project management, such a superintendent would have had to face, was drawing up a schedule of work and keeping to it as the project developed, which in turn demanded the most rigorous critical path analysis. That would have included choosing the most appropriate location for the work, ensuring the procurement of materials, selecting the various specialists and determining the choice of illustrations. Let me look at these issues in turn. Choosing where to work was perhaps the least of his worries. This was a manuscript for the court, and there was already a functioning court atelier in a specially built suburb of Tabriz, the Mongol capital, named Urrabi-Rashidi after its founder, the most powerful vizier in the land, Rashida Deen. Here, multiple copies of his great world history were copied, and close by was another intellectual, cultural and educational hub founded by the Ilhan Ghazan Khan. And so it's likely that the manuscript was produced in one or other of these two centres within easy reach of major libraries, which could be consulted by members of the team. At the outset, the provision of materials was crucial to fulfilling the ambition to make this book look like no other, shah-nome. Nothing but the best would do, and presumably no expense was spared. Materials of all kinds, including brushes, galams and inks, were required in abundance. The purchase of pigments in sufficient quantities, especially those that were costly, hard to obtain, rarely used and perhaps unfamiliar, would take time, energy and resourcefulness, as well as money. The extraordinarily wide palette employed for which no other contemporary Islamic manuscript offers a parallel was the result. The provision of paper, elephant folios of outsize scale, might have required special authority. Both artists and calligraphers would have been allotted specific paintings and sections of text, respectively. Other specialists would have worked on sizing and polishing the paper and executing the illumination of rubrics and captions. The project director was perpetually having to look ahead. He had to oversee and coordinate the work of numerous specialists involved and follow a strict timetable. He had to predict and thereby to avoid difficulties and to delegate. Hardest of all perhaps was the selection and supervision of the painters. The art historian, Yvonne Struckin, estimated, as I said, that at least 12 artists were involved in this manuscript. Among them were several of the highest calibre, men who took Persian painting in new and unexpected directions as in this painting. Such artists, even if they did not yet enjoy the status that their successors did, for example, none of them signed their paintings, will have had minds of their own and might not have appreciated being disciplined by the project director if they lingered too long over a given painting. Herding cats is a tall order. If the cats are in competition with each other, the task is all the harder. A further challenge would have been to maintain a consistently high standard as the project progressed. Two centuries later, the successive project directors of the great Tahmasp Shahnomi conspicuously failed to achieve that aim. The images in the later part of that manuscript are further and further apart than they were at the beginning and their quality is often humdrum. The écla of the opening folios, the one on the left, gradually faltered and eventually fizzled out in anti-climax. But some of the finest paintings in the great Mongol Shahnomi are from the final section of Phaedosis text covering the Sasanians. It therefore seems that the Ilkhanid project director did not follow the order of the text in allocating specific episodes to specific painters. The number of illustrations raises the third of the four problems to be considered today. In all probability, it was for the project director, perhaps with some key support from specialists in the Shahnomi text and in the history of the ruling dynasty, to determine what episodes in Phaedosis text should be illustrated and to discuss the interplay of word and image with the artists as a painting gradually took shape. All that required him to have a vision of the epic as a whole and of its meanings. This is a problem that modern art historical scholarship has sidestepped. Of course, the programs developed in other manuscripts, including other illustrated Shahnamis, would help. But the grand scheme of this manuscript called for well over 200 images as many as 216 and thus far, far more than any other contemporary Shahname, most of which had half that number or even a quarter of that number. So the project was venturing into unknown territory with no models to consult. To invent new compositions for so many episodes was hard enough. But that took, that that task had still wider resonances. Was it the intention to have lots of standalone illustrations or were the paintings designed to interact with each other, each with a part to play in some underlying pattern? Such questions would have taxed the imaginations of everyone concerned. Beginning with the project director, was he content to emphasize storytelling or was it part of his purpose to highlight core themes? For example, kingship, loyalty, justice, piety, generosity, adventure, valour or statecraft. In short, was there a program for the entire body of illustrations with some underlying iconographic intent? And could the patron, the person who paid for the whole enterprise have had a say in this? Such questions must remain unanswered for the moment even if paintings that are probably from this Sharnami and are preserved in post-Ilchanid albums are factored into the argument. What remains is a mere torso. There's not enough evidence. The fact that about two-thirds of the illustrations planned for the book were not completed or were not even begun and in any case are lost without trace makes it very difficult to guess how the sequence of paintings was organized. Huge portions of the narrative structure of the Sharnami are devoid of illustrations in this manuscript. There's no other Sharnami that has this amount of gaps. Those gaps include the earliest kings, the reigns of Kei Kaus and Kei Kubad with the stories of Siavush, Bijan and Manijae, the middle and end of the Sasanian period and above all and inexplicably most of the story of Rustam which is the very backbone of the epic. A Sharnami without Rustam is like Hamlet without the Prince of Denmark. The very detailed coverage of some episodes of the text in the sequence of paintings is a further indication of how much has been lost. The stories of Zal, Iskandar, Ardashir and Bahrangur are strongly highlighted in the pictorial sequence and that in itself suggests in the interests of balance that other key episodes or reigns were also richly illustrated or intended to be so. Clearly, some bunching was intentional but on the other hand inconsistent coverage or inappropriate emphases were also undesirable and the project director clearly had a further and utterly unheralded commission which was to find correspondences large and small between Ferdowsi's text and events in the history of the ruling dynasty and to ensure that they were brought out in some of the paintings Zal, Iskandar has argued. This created an unpredictable subtext or subplot throughout the entire sequence of illustrations. It is well nice certain that further correspondences would have been highlighted in the paintings that are now lost and were left unfinished or never even begun and recent work by young Iranian scholars has brought several of these to light. In brief then the decision to have well over 200 illustrations of unprecedented size confronted the project team with some formidable demands that called for unflagging vision powers of invention iconographic subtlety deep familiarity with Ferdowsi's text and plenty of political subwoofer. The fourth and last set of problems I want to consider today concerns the legacy of the great Mongol Shah Nome. Several scholars have expressed surprise at how relatively little impact it had on subsequent Persian painting. Yet it must have created a huge stir in the fraternity of artists who joined forces in creating it and its scale and ambition would have made it a talking point among some sections of the court. Despite this it is astonishingly difficult to pinpoint its legacy though some scholars have proposed that tenuous traces of its manner can be found in three paintings of around 1500. Here's one of them with its possible source the suicide of Shirin. So the question remains how did this Shah Nome come to be forgotten so quickly? Several possibilities come to mind some are based on known facts others are mere are more speculative. The first possible reason is the political chaos that engulfed the Ilhanid kingdom on the death of Abu Sa'id in 1335. The scramble for power was prolonged and bloody. Some of the collateral damage impacted on book painting among other arts. The court atelier ceased to function and was probably disbanded. So the mature artists who were responsible for some of the best paintings in this Shah Nome were deprived of the court patronage on which their art depended. They had to make another kind of living. The very existence of a court atelier under the successor dynasty the early Jalaya Rids has been hotly disputed even if it did exist in the 1350s no major illustrated manuscripts can be associated with it. The principle example of Iranian book painting in the 1350s the Garshasp Nome of 1354 shows only occasional awareness of the great Mongol Shah Nome. It took the best part of a generation for a new atelier to be founded under the Jalaya Rids with an accompanying new manner. And it's worth remembering that their empire was much smaller and less wealthy than that of the Ilkhanids. Moreover, their uncertain grip on power created an instability that worked against continuity. The fortunes of war made Sultan Ahmad move his court from Tabriz to Baghdad and back and then go on his wanderings again. Weak is my arm in war Fleet is my foot in flight equipped. All this points to a severe gap in the master-pupil relationship. Much of the technical know-how and more broadly the experience gained in creating the epoch-making paintings scattered throughout this manuscript was probably lost in the appropriate physical context in which to transmit it. The Safavid Librarian Doust Mohammed in his potted history of Persian painting written in 1544 he is the Persian Vazari glosses over this problem. And then one must reckon with discouragement with the psychological challenge posed by Mongolshanomi. It set the bar too high and that might well of course not just discomfort but a crushing sense of inferiority in those painters of the next couple of generations who might have seen it in plain English it might have led to a failure of nerve on their part. The mere sight of it perhaps even just the recounted memory of it which would have lost nothing in the telling may have helped to persuade the patrons and painters of the next generation or to rethink the art of Persian book painting in radical fashion. Whatever the reasons for it and despite occasional exceptions the illustrated book shrinks in size dramatically. The number of images is also slimmed down drastically and numerous other features either vanish or are for want of a better word tamed the panorama gives way to the miniature the broad brush treatment of people events, compositions landscapes yields to the obsessively careful microscopic treatment of detail understatement a certain distance and a sustained focus on immaculate technique replace the immediacy and violent action of Arshaname. In the world of later Jalalrid and especially classical Timurid painting a world often described in terms of fantasy fairy tale and filigree detail there was no room for the raw power of this Shahname whose heroes wield swords that draw blood in short the great Mongol Shahname stands as a warning about the dangers of overweening ambition on various fronts these include its production process its scale its expense which would have discouraged many a possible patron the sheer number of images and the unmatched aesthetic caliber of some of them all this might have gained it a reputation in some quarters as a white elephant moreover its unfinished state was itself a mute commentary of those who conceived it this might well have constituted a warning not to venture down the same path again at all events painting moved on in a different direction and effectively the manuscript yesterday's style ended up in the attic one particular aspect of its unfinished state deserves special attention in the context of its lack of legacy since it was left unbound in the royal atelier it would not be taken it could not be taken off the shelves and studied as a whole it was not easily accessible this may sound like a small point but it was decisive paintings tucked away in folders in a box are a lot less likely to be consulted so they were not readily available as a model but they obviously inspire later generations of painters moreover when such loose leaves are consulted their unbound state renders them vulnerable to damage that can be the case even when they're in an album if their album is frequently consulted by people who don't know how to handle such material properly it took only a few decades of intense exposure to the public from 1928 to 1963 for the signature of Ahmadi Musa on the trunk of the elephant there to disappear because people would say oh there it is and in the case of the great Mongolshan army the condition of many of the leaves with paintings unlike the leaves without paintings is notably poor now I have a good gander at this this damage cannot be laid at the door of Damat though ideally like to do so the degree of flaking of the surface paint which reveals the underpainting beneath the number of nicks and cuts the multiple creases that crisscross the painted surface speak of a history of casual neglect and even more casual handling these leaves have not been treated with enough respect there is no better protection for a book painting than an unread book and there's more a manuscript preserved in loosely form can't be leafed through rapidly nor is it convenient to go back to check a detail think of using a kindle a bound volume easily located to be consulted makes such tasks simple and a matter of moments so in the case of loose leaves kept in folders or boxes the viewer is constrained to look at them one by one and is therefore denied the cumulative impression so easily formed as you leaf through a bound volume so it was a decisive factor in depriving this manuscript of the legacy that it deserved and that also incidentally made it easier to remove paintings from their folders and insert them higgledy-piggledy brutally trimmed and contextless if they had text the text was removed into albums one can also understand why successive patrons left it unbound given that it was obviously so far from being finished just imagine an album consisting of multiple leaves like that some this way up some that way up some this way up it's mad to summarize then the very ambition of this manuscript on various fronts it's sheer size it's massive radical innovations it's obvious expense which state which was a solemn and eloquent warning to the unwary together ensured that its legacy was minimal time to finish clearly this great masterpiece poses problems galore but the glory is that by and large they were surmounted and to this day one can sense the pulsating energy that runs through the whole enterprise it was that energy coupled with a magnificent self-belief with a deep-rooted patriotism and dare I say a sense of destiny that ensured that no problem was unsurmountable the result is a shaname that is simply awesome and that exudes a unique charisma here anywhere in iranian art from ancient to modern times is the fari qiyani the numinous majesty of kingship made manifest thank you