Can you see it? Yeah, so so we're here at the open server summit. So who are you? Anil Vasudeva with the president of IMAX Research, which is an industry analyst company We do both research and consulting for the IT industry specifically now very focused on software-defined data centers, software-defined storage and software-defined networks But that is coming on very strong in the last two years and it'll catch on fire It's it may be an inflection point for this IT industry and the reason is because it really brings the Operating cost down and and that's what all the data centers are looking for so it's kind of like a required way of doing things to Make things work when there's so much more demand for stuff and the power there's a limit, right? Right I mean let me give you a metaphor a metaphor would be a body is a hardware and Then depending upon your function that you're trying to go if you go in the evening You put a dress you are going in the morning and doing some casual you put a jeans and you go to office And you could put a jacket so those in effect is the at the end of the day Since the hardware is not that we are putting the money in you are creating different personality for different Functions, so some of the functions are you know enterprise functions, which are very heavy-duty and required the robustness as well as require Shall we say you know unforgiving It cannot break down and all that whereas on the other hand you have Videos and movies and all that those are not as needed to be robust and also need very extremely high Bandwidth or you know high speed of data transmission. So those are two different markets So that means if we can use the same computers But let the software make the personalities for very high bandwidth oriented videos Also the same software now switch over all these components join them together to do the enterprise heavy-duty work So now we have reduced the cost with the same computers. So that is the essence of software defined Future of data centers and you were talking about pooling of CPU memory different places. What is that? Right? So the pooling pooling means is look at this way I'm also going to start with the metaphor so spreading you have these jelly beans the yellow color and the blue color and this and So if you want a certain application Running and you need four of blue blue blue Jelly beans and two of these and four of these this is the equivalent metaphor of The kind of CPU you need and the kind of memory you need and the kind of storage So and and the network so because every computer runs on these fundamentals One is the CPUs the other one is the memory and the third one is IO, which is either network or Distributes and all that so now if you if you have a certain application It needs four of CPUs hundred of memories and needs of only two of IOS So you put them together and run the application and then give it back to the pool So the pool is whole bunch of IOS It's a whole bunch of memories and whole bunch of CPUs sitting there as pooled CPUs and pooled memory and pooled IO So you grab some of those depending upon what the application is needy and you run the application and give it back to The pool and that is the most and so the pool can be the most cost effective Why because it's the same models. It's the same jelly beans of one color another jelly beans of another color and jelly beans of third So when you want to build something you pick up two of these four of these depending upon what you need jar So the software defined and the pooling and these things sounds like new way of doing things Is that an opportunity for arm to come in and kind of like customized stuff? Okay, arm is one another incarnation of this whole So CPU it basically what it boils down to is that if you have Very general purpose low cost not not so much low power So you gang them whole bunch of these on CPUs together to create these functions as compared to having very individualized Very individualized very function oriented CPUs that Intel made in the past so and since the power is a very key function of Assembling a whole bunch of so think of this way if you can pull a whole bunch of arm processors and So you can carve them to make a Application to be run or a be application or see application. So that's why the success of it So you think it's gonna Succeed what do you think arm would have a very strong place in a whole bunch of early on not so much into the enterprise very high Demands but more so we're let's say for example, if you look at Facebook and all that they have Thousands and thousands of servers. So and they are all very dedicated to serving your Your information coming in from the web, so they fit in beautifully there And so the company like Facebook and the like e-bays and all that would be doing all these At the expense of some very dedicated high enterprise level Computers that Intel makes so a lot of change a lot of things are happening in this industry Well, the biggest change is not so much the arm versus Intel the biggest change is software So these are the elements that you assemble together and if you can do them on the fly on the fly then you are Serving those functions, then you give it back to the pool then next function comes you give it and serve that Function give it back to the pool. So it's a very new different ways of doing it And it all started with something called virtualization that was the first concept coming out of Stanford And is it Stanford again doing the software defined networking? So the other guys over there They're their hands and many you know I mean if you go if you go to Stanford and look at these evening meet-ups and all that oh You'll be seeing a whole bunch of things coming in from there continuously. It's a it's a It's a building ground for You know the next generation of innovators if you will


Welcome to Russia, the research seminar in Islamikart. I'm very glad to introduce you Tofik Dudley, who is talking to us tonight from Jerusalem, and Tofik is an archaeologist and an art historian. He teaches the departments of Middle Eastern Studies and Art History at the Hebrew University in Jerusalem. His research is on archaeology. He's done several excavations in Jerusalem, Alid, Simnabra, among others, but also his research is in art history, research in Persian painting, in particular Timurid illustrative manuscripts, and in fact, his book, Esoteric Images Decoding the Late Herat School of Painting, was published by Brill in 2019. Today, Tofik is going to talk to us on new finds within a famous so-called Umayyad Desert Castle, that of Herb et al-Nafjah. He's published an article already on this in the Jerusalem Journal of Arabic and Islamikart in 2019, volume 46, but the work continues, his work in progress, and we are looking forward to further publications soon. So thank you very much, Tofik, for joining us tonight and over to you. Just to remind the audience to write your questions and your points in the chat, and I will put them to Tofik at the end of his seminar. Okay, Tofik. Thank you. Thank you, Anna. I will share my presentation. So thank you again for this invitation to this interesting seminar with very interesting topics that I already listened and participated to some of. And as you said, I will talk about some fresco, some wall paintings from Herb et al-Nafjah, and I have to thank Professor Donald Wiltkom, who a decade ago started or restarted, let's say, went to excavate again in Herb et al-Nafjah with the Palestinian Antiquity Authority. And so the Rockefeller Museum, Archaeological Museum in Jerusalem initiated like a project to look for old materials from Herb et al-Nafjah, and they found some files with paintings, and in this way I ended working on those paintings. So I also would like to thank Foundation Max van Burschem, who allowed me to conduct such a research out of generous scholarship that allowed me to scan, to work and to research those paintings. So Herb et al-Nafjah, a well-known sign for archaeologists or art historians of the Islamic period. It was extensively excavated by Dimitri Baramki during the years in 1934-1948. A portion of the results of the 12 excavation seasons was published by Baramki in preliminary reports in Dakota and later in an almost complete report by Hamilton. In that publication, Robert Hamilton discusses the building methods and different decorations, mosaic, stoke, stone relief, and fresco. While his publication said, as the basis for further scholarly analysis of the royal complex, little attention was paid to the frescoes. And here we have a general look on the side, and this is a courtesy of, again, Donald Whitcomb, back when they were working there. So what we have in Herb et al-Nafjah are two main, let's say, units. And one is the bath, and especially the bathhole, and the other, the larger one, is the qasr or the main palace. And the wall frescoes were found by archaeologists on the floors of the palace and on the floor of the bathhole. Although most of the paintings were discussed by Ole Grabar as part of Hamilton's publication on the palace, the recent discovery in the mandatory archive at the Rockford Museum in Jerusalem of two large files containing aquarelle scopes of the frescoes has generated renewed interest in them. So my main focus will be on the palace, but I will also refer to at least stoke and not fresco in the bath or the hamam of this unit. So the majority of the paintings were found in the eastern wing of the palace. So we have here the rooms, one, two, three, and four, and northern portico and southern portico as indicated by the excavate. So most of the pieces I will introduce are from this part, the eastern part. So evidence from rooms in the eastern wing suggests that they didn't contain wall paintings as no trace of frescoes was found on the extant walls of these ground floor rooms. The assumption therefore is that they fell from an upper floor. Hamilton suggests a possible partial reconstruction of this upper floor. And he is like referring also to a caster gharana with an upper floor where you have also private rooms or odian halls. So it's not exceptional to Kherbat al-Masjid that you have such a special room on the first floor, not the ground floor. But before we delve into reconstruction and talking about the paintings, I would like to give a slide about methods. So I'm here following somehow the excavator Baramki and Hamilton method. We have here materials, artifacts, or let's say architectural pieces, first excavated or uncovered, then sorted as you see, and then somehow reconstructed. Of course you have this from the material on the ground and then to somehow imagination. So this I will try to be in between those two lines, materials and then imagining how they supposed to stand or how they were standing on site, let's say. Moreover, as we can see in this picture, we are dealing at least with three elements or perhaps should I say three time dimensions. The time when the remains were uncovered, represented here with not so a pleasant, let's say, man holding a stick with two kids standing in front of them. And they are like part of the workers. So this is the present time of retrieving and excavating back. And then we have a thick earth layer beneath this man. And this is like the excavation. And below it, we have this layer of remains collapsed. And the time when like those remains, the third line is when the time remains were standing back. So we have the palace living, let's say, and then collapsed, and then excavators or archaeologists coming and retrieving those. So it is somehow dealing with, I don't know if you would like to, but it's like having a piece of furniture from IKEA and you have the pieces, but you lost the manual. So this is in general how I like to imagine those materials and how you have to deal with them. More about this excavator, Baramki, especially well documented his work on ground. So you have a sequence of different pictures here, all from the same area. And this is related to the area I will focus on. And this is the main entrance to the palace or the cassette. So we have this pile of stones collapsed with this marble column in there. And Baramki is documenting each step he is doing. By the end, he will clear all of those blocks to open the entrance, but some of those blocks also contained or were plastered and then painted with the fresco. So this method or this well documented process helped me, let's say, step back and enter to some sometimes missing materials out of the circumstances. But it helped me to understand from where those blocks are coming and how should I deal with them. So here I would like to show those pictures how a piece of an Ashlar block was first uncovered, then it's out of context pictured and we see here this like face. And then the third step is to paint those pieces with those aquarelle paintings. And with the aquarelle paint paintings I worked and I will present those paintings. So it's not working with the original. First, let's say the frescoes themselves, like a big amount of them is not there anymore. I worked with the aquarelles. So this is also we have to keep in mind about colors and other things. I'm dealing with aquarelles. So in those files, there are a lot of like, I can say hundreds of such pieces. And it's a huge puzzle that you have to manage and start building and reconstructing. Who painted those frescoes on site? I still don't know. I have his hand. You see his hand. He's now drawing some worker on site. He had his own interest, but I don't have his name, but someone was there doing that. So plus on the aquarelles, I assume that was maybe Baramki, but it was also a big team. Left marks beside the aquarelles. For example, serial numbers and room numbers beside other marks. And those marks helped me also reconstruct. So you see here, pencil marks between two pieces, two lines broken, and then you join the lines and you have maybe two pieces related. And beside the serial number, I have some letters, NP4 here. So it is referring to North and Portico. NP4, North and Portico, and 4 is the area. So this is also helpful. So again, although Grappard made great efforts to reconstruct most of the scenes, it seems that the requisite material was not accessible to him at that time. This mate will be attributed to the 1948 war and the fact that Baramki left Palestine for Lebanon at around that time. Dozens of paintings in the files were thus left out of Grappard's discussion. So we start reconstructing two large blocks lying one above the other, where uncovered in the debris attached to the southern wall of the entrance gate. Fresco remains, which were found on these blocks, are reconstructed here to create a scene that reflects the palaces prestige status. One of the blocks is illustrated with at least four human heads and two frontal faces behind and to the sides of the faces are black pointed lines, which may indicate spears held by the men standing alongside. Spears identify their beavers as soldiers. The reconstructed scene may relate to a military beam. Here I will refer to it as the Triompheal scene. This is the name like I would like to give. So here we build the pieces of this. A two long ashlar stone beating the frescoes described above can be added to at least three other blocks that are reconstructed to create part of the scene. A block beating soldiers holding spears is one of the major blocks. The piece is framed by thick border lines on two sides, one above the heads and one to their side. This border locates the piece as an upper corner of the scene. As such it determines the border of this wall painting. Another piece with a blue sky background beats the head of a man illustrated in profile with an outstretched hand. This piece forms a skyline and occupies the upper part of the scene. As for the lower body of the scene, two pieces are proposed. Both are bordered on the sides and should therefore be placed closer to the frame. One of them appears to be an illustration of three legs around shield and quiver. The lower parts of the legs are covered with grey straps and red bands while the upper parts are exposed. The curve of the shield is painted yellow and the quiver painted the same color as the leg stand. These features suggest two soldiers standing, one frontal and one in movement to the right. One of them is holding a quiver to their back as shield is positioned. The basis of a third soldier can be seen to the left near the border line. Only portions of the legs have survived, one of them partly covered with grey staff. These two pieces, one depicting the heads of soldiers with spears in the background and the other depicting the lower bodies of soldiers, allow us to partly reconstruct soldiers standing on the edge of the scene, adjacent to the left border that is clearly marked with a black line. As for the middle body from the upper leg to the chest, we may learn more from other pieces of Presco found in that vicinity. One piece bordered with a black line, similar to that in the previous pieces, depict different objects that are not fully identified. The most obvious feature is part of a roundish object composed of various pieces. I will point here, this is the roundish on which the palm of, and here is it, a human hand can be recognized. A roundish object bordered with a yellow band is depicted beneath the palm. To the right of the palm is a yellow object decorated with black lines creating a vegetal motif here. I am placing this piece above the previous one below. We see that the yellow line continues at the bottom, suggesting that this piece is part of the middle body of the figure. But this does not make the meaning of the scene any clearer. One possibility regarding the yellow border object is that it is a shield, the soldiers are standing and the shield is behind them. The other possibility returns the yellow border artifact into a wheel and the larger object, a hand depicted on its background into a chariot. In both cases, the yellow object decorated with a vegetal motif could be the lower edge of a tonic worn by one of the soldiers. Another fresco fragment found in the southern wing can shed light on these objects. The fragment bears a warrior wearing a yellow armor and a helmet. He holds a bow and arrow in his hand. This warrior is about to release the arrow into the upper part of the scene to a point which is not longer anymore there because of the fragmentary condition of the fresco. And the warrior is the only clear feature in this scene. So sorry, the archers, a yellow armor is the link that may explain the nature of the yellow objects illustrated in the two blocks referred to above. The archer is wearing what looks like an anatomical warriors with lappets and horizontal band with two horizontal lines on its upper part. This armor may differ from the archer's armor and maybe a palette armor or like from the Roman period. Of course, this is an armor of an emperor and not a regular soldier, but we can see the similarity with the lappets and the other features. So we have here maybe a Roman style of armor. We have other pieces here. The yellow is again the body or the chest armor and the red lines here maybe are bands to hold swords those soldiers had. Moreover, the warrior illustrated on the fresco fragment can shed light on the nature of the reconstructed scene. This warrior, this one, wears a helmet while the warriors in the scene are without helmets. The absence of helmets in some Roman scenes led to the conclusion that they do not depict a martial procession, but rather are part of a ceremonial scene. Such a conclusion would also fit the reconstructed scene. Following this reconstruction, the scene depicts a battle or a triumphant scene where the armor is part of the bodies of the soldiers taking part in the scene. The soldiers bodies or the armor, at least the armor depicted on the long piece below the faces should be placed below the faces. However, we have more faces than bodies depicted on the lower piece. Therefore, the position of this piece is not directly under that face. So I'm referring to those pieces and where they should be placed on this scene. I'm not sure about that. So it's not there. We have, as I said, more bodies than more faces than bodies. So although only a small part of the scene or even the mission alone is reconstructed here, and it is hard to talk about its iconography, something may be said about the style. The soldier's armor is more in the Roman than the Eastern or Sasanian style. This style is determined by the equipment, including the anatomical squares which is related to a Roman or Byzantium army. Moreover, the scales are more indicative of the Roman army than the Sasanian trouser. On account of the scene's fragmental condition, only general impressions can be described here, drawing on well-known examples from the antique era without any documented connection to the artists from Jericho. The projecting spears recall the spears from the famous battle scene of Alexander dated to the 2nd century BC. In that mosaic, the two river kings, the Persian Darius and the Macedonian Alexander, faced each other in battle. One of the clear fields is the skyline where horrified faces can be seen and spears stick out, while the lower part is less obvious, horses and warriors clash, which makes it difficult to distinguish between the different features. The same applies to the lower body of the battle from Kherbat al-Mafzar, where few pieces survive. But maybe we here can recognize the shield and the chariot when comparing the two scenes. Triomphal or battle scenes between western and eastern powers can also be seen in the arch of Galerius in Salonica, which illustrates the triomphal scene of Galerius meeting Narcissus. The reef illustrates the moment when the two rulers meet in the middle of the panel, while members of the two armies are depicted to the sides. As is normal in such meetings, the conquering party stands steady behind its shields, crushing members of the defeated army who collapse onto the ground. Here the Roman emperor wears body armor similar to that depicted in the archer scene from Jericho. Now, the majority of the pieces from which the scene is reconstructed were, as I said, retrieves from the southern porch, more precisely near or even attached to the wall of the southern entrance tower. So the widest walls in the southern wings are the walls of the entrance gate, which measure approximately five meters. The blocks were thus once incorporated in the southern entrance tower. This does not mean that the paintings decorated the gate, rather they probably belonged to an upper floor that was partly supported by the entrance gate. More about the upper floor can be learned from Hamilton's reconstructions of the eastern and western wing. Hamilton proposed the existence of a basilical hall on the upper floor, with a central nave situated between two aisles, the center roofed with a rich timber roof, aisles with a flat roof, and a dome crowning an abse on the east. Following Hamilton's reconstruction of the upper floor above the entrance and taking into consideration the size of the stone blocks decorated with frescoes, we end up with two possible walls that can support the upper floor walls. Those are the southern and the northern walls of the square room of the entrance tower. As the majority of the blocks holding the triumphal scene were retrieved to the south of the tower, and because of their weight which prevented their collapse and desperation far from their original location, it is suggested that the triumphal scene decorated the southern wall of the square room that was once incorporated in the entrance tower. On the same southern and northern walls Hamilton reconstructed niches that were partly found in the southern porch that extended beyond the entrance facade. As the southern wall is long, the painting can be placed on the upper decorative register above the niches. This room had a large window that looked east to the Mu'ab mountains. We can say that the power was not limited to representation, it was also present on site, especially when delegations or envoy visited the palace. Evidence emerging from both sides of the late antique war, the Byzantium and the Sassanian, which the Umayyad inherited, speaks of diplomatic protocol that included military spectacles with ceremonial attire and weaponry. More evidence for such spectacles survived from the Abbasid period too. The historian Hilal Ben-Muhsin Saba'i describes a famous reception given in 970 by Khalif Al-Muqtadir at the Dara Khilafa of Baghdad for an embassy sent by the Byzantium emperor and I'm quoting and reading. An envoy from the Roman emperor came in the days of Al-Muqtadir. God bless him. As part of the preparation, the palace door was covered with nice carpets and decorated with great installations. The door guards and their helpers and the chamberlains were stationed on the gateways, festivals, passages, courtyards and audience halls of the palace. Soldiers from various military ranks were standing in two rows dressed with nice clothes beneath them, the mules with gilded and silver saddles and to their sides the reserve mules dressed in the same manner. Those displayed the equipment and the huge weapons. They were standing from Babel Chammasiya to Dara Khilafa and following them were the chamberlains, slave pages and inocks that were dressed with nice suits, swords and decorated belts. The prayer and post-Omyad ceremonial weaponry consists of two linked chains that the Omyads probably did not break. This helps us visualize the human side of ceremonial life of El-Mafjar why the reconstructed scene depicts the representation of power. However, we have to remember that evidences on the ground are more complicated than that. The entrance hall was lavishly decorated with stocco such as this lady here or those men heads jutting out of the watch. Perhaps those features had some meaning or where we are missing in our reading now so it's just a suggestion for part of the meaning. More about the possible meaning of the triumphant scene perhaps can be added when comparing it to the paintings in Qusayra Amra. The Qusayra is one of the palaces if you cross the Jordan river and like it's it's less than one day journey with a camel to Jordan. So there we have a basilica with a caliph depicted on the wall sorry opposite the entrance. On the side wall of the panel what is called the panel of the six kings by Grabar and we have other names for it but we go with six kings they are pointing to the caliph depiction and according to one reading accepting him to let's say the kings or rulers party. Hence a scene situated not for or on the main let's say access but on the side and somehow is related to this main scene where is the caliph now we we know that is El Walid Ben Yazid at least in Qusayra Amra and the one that Hamilton wanted to be related to also to Mafshar. So perhaps in Khirbat al-Mafshar in the audience hall this triumphant panel is pointing or leading to the caliph himself sitting there in the audience hall or one of the Amids behind the caliph huge window was open giving him high access to the world outside where maybe other kind of audience than the one allowed to enter the hall was standing this caliph standing by the window send us to the bath facade and now we are moving to that second main unit in this palace and this is the bath so we are looking on the facade so behind the caliph a huge window was open as I said giving him high access to the world outside now here on the facade of the bath we have a status of the caliph situated in the niche above the arch of the main entrance to the bath hall following Ettinghausen or Richard Ettinghausen suggestion this was part of the line that led by the end to the place where the caliph or the prince who owned the palace used to sit in the hall where stone chains suspended from the roof held his crown above his head so back to the Qasir audience hall we have to consider also the place of the window and the gaze outside and not only the one inside so we have two directions especially because of this huge window that was open in this basilica so sorry that that was about this triumphal scene its place the basilica and what we can say about that more in this eastern section but in another room not in the what we call the audience hall not so far from this hall we have this panel already reconstructed by Oleg Grabar and this was like so we have this pattern of interlacing circles each occupied by by a composite creature with that dog's head wings and stylized peacock's tail named by Grabar the dragon motif and here I will refer to it as the Psoido Seymour the interlacing circles are drawn in black and white outline and fit with head shaped patterns lined in pink and white and colored pink red and white so we now with the aquarelles bring color and life to this panel so the circles are colored in a mix of yellow and pink and we have here different let's say stines of this creature and at least three types of tails here is one with arches here another one with the rosettes third one only with gray color so it's not one kind of this Psoido Seymour let's say um and in between there are rosettes situated in between the circles line and they are composed of four head shaped leaves so um and this was maybe an upper panel and in the same room and maybe below the the Seymour Psoido Seymour panel we have this panel with rosettes also reconstructed by Grabar but now we have those colors so um if we follow looking for this Psoido um um Seymour we can find it also in other umayyad contexts like we have this symbol or this motif let's say in Pasir el-Hir el-Sharki el-Garbis sorry An Qasir Hallaba um and in both cases it is encircled in medallions um and also rosettes are there and also we have such a pattern in what is called the whole of the impassados in Afrosayab which it's Sogdian earlier than the umayyad in nowadays Uzbekistan we are looking on the German and the pattern on one of the figures there so the latest research on the symbolism of this composite creature suggests that it was used to exalt the noblemen of Sogdian and not as a representation of the Seymour of Iranian mythology that is to say it does not represent the phar or hawarna mentioned in the Avistan now the Psoido Seymour also appears on the Germans of the rider on the lower uh rear wall of the great Iwan of Taqi-Bustan which shares at least three common points with that of el-Mafzir both have a tail composed of one piece and in both cases the feathers of the tail are composed of a series of arcs arranged on one above the other and decorated with a leaf motif two the medallions in second the Psoido Seymour are filled with a hair-shaped pattern and there the rosettes that are set in between the medallions are composed of hair-shaped leaves if the artist of Taqi-Bustan did indeed intended to imitate colors that were originally on the on textiles by engraving their pattern the task of the artist from Khirbat el-Mafzir was simpler as coloring is much easier than a chiseling the examples above demonstrate the rule of the Psoido Seymour motif in the late antique aristocratic world by which time reservoir let's say of different symbols and motifs had been adopted in two different cultures in this way symbols moved between calls including the Sogdian in the far east with its connections to the Sasanians, Byzantians, and the newly arrived Muslims following this line of thought recent studies reject the idea that the figure is the Seymour of Mazdan culture proposing instead that this figure represent divine glory a concept easier for adopting adoption sorry by Islam and Christianity than the Seymour. Matthew Kanepas reflection on cross-cultural motifs may enhance our understanding of their meaning all of these images I'm quoting here Kanepas about those images the winged victory the nimbos and ornamental motifs such as the bird in medallion and the Seymour served as cross-cultural mediators meaningful in both cultural environments and useful for conveying claims about the subversions between the two countries and here he's Kanepas book Two Eyes of Earth is talking about like Roman and Sasanians in this case the motifs did not necessarily originate in one culture and move to the other rather the Roman and the Sasanian subversions were often mutually engaged with a third phenomenon under the watchful eyes of their opponent so end of quote in conclusion even if the Psoido Seymour is not a symbol of the Khawarna or the Far and even if it does not represent the Seymour from the Epic of Zar and Rustam it is still a rare motif evidence for its rarity is its sole appearance on the charmant of the rider of Taco Bustan while most of the figures depicted in the Grotto were garments garments with the different patterns including beards and rams moreover it decorates the clock of one of the center figures in the whole of the impassasos in Afrosyat so there are as I said before I end my talk with this concluding remarks there are a lot of more pieces to reconstruct and together I'm just bringing some examples of human faces imitation marble maybe you can recognize also some Sasanian again with those trousers here some architectural features so things still need to be deciphered or reconstruct so the reconstructed motives and scenes reflect artistic development during the Umayyad period artists active in this period were basically integrating two traditions the Sasanian and the Roman Byzantium the newly established visual language emerging out of the two traditions can be seen in the various Umayyad monuments either religious such as the dome of the rock and the great mosque of the mascos or palatial complexes for instance Osir Amra and Qasr al-Hir al-Gharbi religious buildings be only non-figural motifs some of them similar to the motifs at Khirbat al-Mafsa such as the scrolls and the rosette the palace repertoire is larger and more diverse in compassing animals and humans the wealth of artistic techniques and motives seen in khirbat al-Mafsa is unique the most popular artistic technique is without doubt the stoko which covers huge areas of the palace including the bath hall although the stoko was applied mainly to the walls of the ground floor and entrance porch fresco paintings seem to have been limited to the upper floor the audience hall above the entrance corridor was in all probability planned as the main reception hall in the palace if not the throne room visitors were led to the staircase on the north inside of the castle ascended the stairs that led up to the main entrance of the audience hall once in the audience hall visitors could see the triumphal scene and probably other scenes that are no longer visible the scene facing the entrance of the audience hall of Usair Amra appears what is generally accepted as the figure of the patron Walid ibn Yazid however the figure facing the entrance in Usair Amra is not the only one of importance the figures on the side walls leading to the image of the patron in the main niche contribute to the scene of sorry of to the sense of prestige conveyed by this whole the triumphal scene in Khirbat el-Mafzular probably situated on the southern side wall sends a similar message of power to palace visitors moreover in Khirbat el-Mafzular the pattern patron whom audience is met in the hall could glimpse the view to the east through a wide window this horizontal axis connected the patron with one kind of visitor from one side and possibly another kind of visitor from the other in addition following persilia sushi line of analysis in relation to the decoration of the bath hall facade and its dome festival and her connection between the two ponds we may suggest that the meaning of the audience hall paintings situated above the main entrance probably related to the figures or the symbols that were once on this facade a related message is delivered by other panels that decorates decorate rooms on the upper floor of the palace in one room on the south and wing human figures are depicted in eastern and western custom on the northern wing other royal motifs were uncovered especially the panel with the soido simul which was a symbol of royalty in late antiquity those panels were probably placed one above the other with the scrolls and other panels with the imitation marble on the dadu and by the end to paraphrase let's say gravar we may conclude by saying that the adaptations to the complexes visual repertoire made by the patron or the artists of the mafsar demonstrate considerable intellectual and aesthetic involvement and not merely not merely novoreesh whimsy and i would like to thank here also the curator the previous curator of the archive or keeper of the archive in rockfiller museum who helped me a lot here with the reconstruction silver a car carpoeco and my also a friend slava pilski who also helped me with the plans and work like those to reconstruct and place those pieces in like a good shape and image and thank you all for listening thank you tofik very much for this very interesting talk on this amazing palace and and for bringing the spotlight on the on the wall paintings of course you know herbetal mafsar is much better known for mosaics and not so much for the paintings maybe and also for for bringing the paintings the wall paintings in in connection to other palaces you mentioned kassel here are be and and of course course i remember and i'm i'm wondering whether you talked about you know the royal figures and the battles and you talk about the possible stylistic relationships i wonder whether you work also in the work in progress that you're doing whether you have um you have encountered any scene that can be a narrative scene maybe uh maybe difficult to decipher in the same way that kosa aramra now uh obviously especially in in recent years has been the subject of much studies uh so i was wondering whether you had any insight about that for this for herbetal mafsar i have to say unfortunately i i still do not recognize any narrative that um that it's um and um i also if just maybe there but there is but i also uh i think also the stokos are not really and i have to say also also the stokos still need to be studied like of course they are well published in hamilton book but there are like thousands and now it's not hundreds but thousands of pieces still there in the museum waiting to someone to connect so i'm and there i don't remember also a narrative let's say yes interesting thank you very much there are a couple of points in the chat one is is there a difference from jamie come stock uh is there a difference between simur and samur like i i have to say that i decided to play with the um naming just to refer um that this is like um not the um let's say a vestan or the iranian as as we used to uh refer to it and there are now uh let's say new thoughts about it so this is my point of using uh different naming so i don't know if there is a um real difference between simur or simur like and if i will read from the saname it will be written simur of course uh so and um uh good now heidson splendid presentation are there any pre-seventh century structures on the site pre-seventh century and in recent excavations by donald wittcombe as i said and uh palestinian antiquity authority uh on the northern part and also um not so not far from the bath hole and the bath itself they didn't um um like tackle such like periods so it's it seemed it was um um like everything there is started with the omelette let's say and uh karen pinto please see my article and find the earliest painting of moon in cosai ramra and she puts the link there thank you um yes are there any other comments or questions please write in the chat uh so fatma says what is the technique of these paintings are they fresco or aseko um i didn't like uh examined them uh we have only few pieces still preserved in the museum and they are i have to say we're like not so in my hand reach so i couldn't uh let's examine them and the way they were um let's say applied and which kind of materials used there and as i said at the beginning i worked mainly with the aquarelles so yeah yeah so that's something again in progress to look at um i was actually quite uh uh interested in these aquarelles that you showed at the beginning you said that that we don't know who made them but but you know sort of the connection with of obviously with herzfeld's reconstruction of the wall painting of samara comes to mind and other reconstruction of that period so that it's interesting to see the methodology of reconstruction of wall paintings that was common yeah this is um also um on my let's say uh list to to check um but uh plus there are a lot of um let's say letters um preserved that have some relation to the team working there so you have surveyors and about the painter maybe his name is there but they are not referring to him as you know as as the painter he was doing various maybe words so uh still uh i don't have his uh name still yeah and um if i may add with this uh last um um slide um it is always here in in mafzar is a partial let's say reading um because um of course i divided the frescoes um from the stoccos and the mosaics but um those people who decorated or style like planned the thing they didn't thought about me dividing the methods they had their own idea complete let's say so what is really missing is all to reconstruct but also to uh connect between the different uh motifs um like so this is really uh let's say complicated not something um easy to do yeah sure to see whether the whole program was put together coherently yeah um valeri gonzalez i'm wondering about the symbolization of this iconography that obviously draws from the various late antique repertoire uh do you not think that there was a change of symbolic order in yumaid context because of the uh dynastic the islamic metaphysics that led to a different understanding of the material i'm dealing with these questions in a recent article any insight on that interesting thank you valeri um i have to say i am as you see i was really trying to understand and to uh put together pieces and this is like an advanced level uh valeri what what you're asking about uh because first i want to so those what where does that let's say uh the the features what what what is standing there and then uh i will look for uh let's say those changes inside the dynastic uh or the the yumaid yeah sorry but uh i'm not there still yeah yeah interesting questions yes yes yeah so oh there was a a follow-up from fatma are the paintings well fixed do they get deteriorated when you touch them again fatma i'm not allowed to touch them i have to say this is um out of the circumstances here not much of them are there and for example there is maybe only one piece from this uh triomphal scene what i call the triomphal triomphal scene that is still there most of them are not there anymore they disappeared in or they were lost since back then so i'm not really um touching the materials um but they are in um let's say in a still in a good shape they are dry not falling apart as they can say without touching them they are still the pieces that are there they are still uh standing maybe this uh can answer you your question about the the nature of the material yeah the material is not falling down and back during the forties a baramkes team put some modern plaster on the edges of those pieces so they are somehow kept and not falling down i hope i understood i answered you so apart from your work on the documentation and reconstruction of the of the paintings is there a conservation program um like on site which is jericho nowadays west bank as i said to you before we started this conversation uh there was recently a new roof uh inserted above the bath uh whole like so all the mosaics are exposed and you are welcome to visit this is one side of the thing the other side of of khirbat al-mafsar is kept in jerusalem in the robfiller museum which this museum uh somehow is frozen uh let's say from since the fifties uh out of the status quo let's say uh out of that uh political circumstances so there is no real uh people touching those uh the stoku especially uh they are there and there there is no real um like uh work on them uh here and there there are some uh schoolers coming to scan to picture um um um a agnacio archie came and scanned and pictured the stokos he wanted to create maybe he did uh he joined donald wittcom a team and he wanted to create a 3d models for the stokos and they were planning to uh insert those models uh in the small museum on site so visitors who cannot let's say cross the border and come to jerusalem can at least have um some experience or to see how those frescoes are so yeah long answer for for a simple question but this is those are the circumstances no that's very interesting and as valery says in the chat uh there is definitely a huge archaeological work that remains to be done um i was thinking um you know the what you call the triumphal scene and the reconstruction um so it could have been like those triumphal scene that you showed like the the mosaics the famous mosaics but do you think that it could also be uh reflect like a ceremony that was taking place uh rather than being the reflection of it of a generic triumphal scene there is of course there is the possibility um if we will i will manage to put more pieces this will can give us um like a hint um yeah maybe it is referring to let's say a kind of historical uh let's say ceremony something that really happened or they are referring to it as something that happened yeah uh but still i don't have uh evidences for that yes no as you saw those pieces are scattered there not really um um helping us to to give this and other small pieces um they you you have the feeling and it's uh that they are like decorations uh wall decorations uh as as we have in and now sending you to another world like in Pompeii like so the the architecture small figures uh so um you see that it's it's only decoration like it's uh they they didn't um let's say put a lot of effort to uh give facial features all right it's just like figures in and in the landscape or something like that yeah okay well thank you very much um so then you have a series of thank you in the chat um is is any uh are there any other points or questions for Tophik otherwise is there anything you want to add Tophik about your future research you just continue you will continue with this putting together a picture i am i understand yes yes yes this is well under the big challenges uh which i don't know if i will reach um is to to combine uh between the different uh and i'd say uh the stocco the frescoes that would be very interesting actually yeah yeah so i look on it as you know we have a book and we are reading different chapters um if we want to understand this book we have to really start from the beginning to the end i don't know if i'm the one suitable for that but i at least maybe will add a chapter let's say well thank you very very much and a virtual applause from everybody and thank you Tophik you have lots of thank you in the chat that you can look at from everybody and thank you for joining us tonight thank you very much for this generous invitation i enjoyed it so thank you thank you bye


Hi, everyone. Welcome to CKPI Machinery Advanced Topics, our maintainer's track for Kupken North America 2022. My name is Federico Buantrovani. I'm co-chair of CKPI Machinery and Open Source Kubernetes. Two very special guests joined me today to provide two great presentations. David Eates from Red Hat, who is also my co-chair and tech lead in CKPI Machinery, and Jeffrey Gein, who is one of our top contributors in CKPI Machinery and other areas of Kubernetes. The two topics we are going to talk about today are the power and the danger of aggregated API servers, and Open API v3, which recently graduated to Beta and is making its way to GA. So I will leave it with David and with Jeffrey, and I will give you some information at the end of the talks. See you later. Hi, my name is Jeffrey, and I'm a software engineer at Google and a contributor to CKPI Machinery. Today I'll be talking about Open API v3. Open API v3 is a feature that has gone beta in Kubernetes version 124. To give a quick introduction, the Open API is a language agnostic interface in the format that is both human and machine readable to describe the capabilities for a service, the service in our case being Kubernetes. The Open API is provided in both a JSON and protobuf format, and some of the consumers within Kubernetes include Q-Control Explain, auto completion for UIs, and we also use it for generating documentation. The official Kubernetes documentation is generated from the Open API and also clients. This talk is about Open API v3, but let's just briefly go over some differences between v2 and v3. Open API v3 is a restructure of v2 and allows reuse of API components and also provides some new features. The most notable of these features is the extended JSON schema support. The Open API v3 uses a newer version of JSON schema that supports four additional pretty important fields. These are one of any of default and nullable. For those of you already familiar with custom resource definitions, you might know that Open API v3 is already supported in the structural schema. I like to highlight one key difference between a schema and a specification. A schema represents the definition for a particular type or multiple definitions while specification refers to the entire Open API document and includes the schema but also includes additional things like pass and etc. So one thing to note is that without this Open API v3 feature, we wouldn't be publishing the Open API v3 specification, and that means that we'd still be publishing the Open API v2. What this means is that the API server is able to understand Open API v3 through the structural schema but because it's not able to publish v3, certain fields, certain objects need to get converted into Open API v2. Now, for most fields, that's not a problem. It's a direct conversion, but for certain fields, they're only representable in v3 and not in v2. So this includes things like default, any of. One example here is that on Kubernetes, we have this type called interstring, which the type could either be an integer or a string. And in Open API v2, because when I put to represent it, that just gets stripped down to no type. Some additional examples, if you have any of in your validations, your validations, then the any of gets stripped and also nullable. So one important thing to note is that a lot of these fields are removed recursively, which means that if you have a root object that let's say is nullable and that gets stripped, then the additional properties will also get stripped. So on the left, you can see that we have a true object with two fields, vmv. And by going through the Open API v2 conversion, all those fields and properties are dropped. Now, this is not really ideal for clients who rely on the published schema, the source of truth, because this is just complete information. Now, the good news is that with Open API v3, this is all preserved. So on the left is basically what you get when you see a published Open API v3. It's a pretty much a lossless version of what the structural schema defines. Now, let's talk about the Open API v3 endpoint. So currently, we have two Open API endpoints. One is the slash Open API slash v2 endpoint. And the new endpoint is slash Open API slash v3. Now, if a user of the Go client library client Go, then you'll probably know that we have a function in a discovery client to obtain the Open API v2 schema called Open API schema. Similarly, we create a new function called Open API v3. And the transition should be pretty seamless to use Open API v3 from Open API v2. Now, I'll go into a bit more of the internal details for NonGo clients, and also providing a couple more reasons to use Open API v3 over v2. One improvement that we made in Open API v3 is that we've basically split up our Open API. And essentially what that means is that when you hit the root endpoint slash Open API slash v3, you'll get a list of group versions rather than the entire document. And the group versions, you can then use these group versions to access the particular URL. So let's say the example of apps slash v1 would be slash Open API slash v3 slash APIs slash apps slash v1. And this will contain the Open API schema for only the apps slash v1 endpoint. This is obviously much smaller. And for clients who require just a small subset of group versions, they do not need to download the entire document. Also, a couple additional reasons. Number one, obviously, as we discussed already, the size. An aggregate Open API v2 is on the magnitude of megabytes. And you might think this is not very large, but this increases based on the complexity and the number of CRDs that an API server supports and also number of aggregated API servers. So this can add up to be quite large. Also, let's say there is an incremental update to a CRD. With Open API v2, an update to a single resource will basically cascade into a recomputation of the entire schema. And this leads to CPU and memory usage specs. And this kind of spike, depending on the amount of resources that are available, could potentially cause an utter memory error and causing the API server to restart, which is not ideal. Finally, there is the complexity of aggregation. So Open API v2 has an aggregator that will send a request to all aggregated API servers. This includes built-in types, CRDs, and aggregate API servers. And it will download and merge all the Open API specs. This obviously takes up resources. And Open API v3 basically skips out the aggregator and acts as a processing. So it will be much faster and less resource intensive. Now, let's talk about an advanced feature of Open API v3. And that is called cash fasting. So this is the output that you get when you go to the flash Open API v2 input. You'll notice that we have the list of group versions. But we also have additional thing attached at the end of a URL for fetching the group version. And that is this cash parameter. This cash parameter works similar to an e-tag, but it allows us to skip one pretty critical request. So I'll first describe what happens with e-tags. So with e-tags, number one, you'll send a get request to the Open API slash v3 endpoints or return to 200 with the list of all the group versions. Now, for each group version, you would first check your cash to see if you send a request to a group version. And let's see you have, then you'll have an e-tag associated with that request. And basically, you're not really sure if the latest version and the version you have is the same. So you need to send a request to the server with the particular e-tag that you have. And the server will either tell you, oh, this e-tag is up to date, in which case, you'll get a 304 not modified. And you can just serve your cash version. Or the server will tell you that the e-tag is outdated and send you the response. Now, we're looking at a case that if the e-tag is not changed, then we get a 304 from the server. And this means that we save on the size of the Open API, because we don't need to download the entire Open API again. But let's count the number of requests. We need to send one request to get the Open API root document. And we just send another request only for the server to tell us that nothing has changed. Depending on the group versions, this could amount to quite a lot of requests. And we would do this pretty frequently to make sure that clients have the latest version. So how does this work with cash busting? Well, with cash busting, it's, the beginning is kind of similar. We would have an Open API. So actually three get requests. And we would get the list of group versions, but obviously with an additional hash parameter. Now, the benefit of this is that when we check the cash, now we know the exact URL that we need to hit. And basically the server will give us two additional parameters headers that are important for cash busting. And that is number one, cash control set to immutable and expiry set to a date far from now, let's say one year. And these parameters allow the client to know that, let's say you have a particular Open API with a private hash. We don't actually need to send a request to the server to see if the hash is, or the ETag is up to date because you already have a hash. We chose to go through the hash instead of let's say getting the client to reuse the ETag, because this is actually a feature built in to many HCP libraries that offer caching. So a lot of times this is something that is done with the Go library. There's no additional checks that clients independent on their side, as long as they're able to use the library that supports this feature that looks at the cash control and the expires directive. And this basically reduces the number of requests that is needed to fetch the Open API on incremental updates. As for future work, Open API v3 is currently in beta and will look to move to GA very soon. Clients of the Open API v2 will also be slowly migrated using Open API v3, starting with Q control explain. This upgrade also provides additional functionality with templating, including things like putting a markdown, HTML, or even custom templates. Thanks. Let's talk about the power and dangers of aggregated API servers. We'll start by talking about what API aggregation is and how it works. We'll continue on to what features you can build using it. And then we'll talk about some of the risks in using the feature and what some of the limitations are. So let's compare a CRD flow to an aggregated API server flow. In a CRD flow, the client sends a request to the QAPI server and the QAPI server handles the request locally before writing to SCD and sending a result back to the client. In an aggregated API server flow, the client speaks to the QAPI server as normal, but the QAPI server identifies the traffic as needed to get proxied to an external API server. At that point, the external API server takes over and decides how to handle the request, eventually storing it in SCD or anything else, or perhaps nothing, and returning the result back to the client. The CRD flow is constrained, but safe. It's safe because all the processing happens in the QAPI server, and it's constrained because you're constrained by what the QAPI authors added as CRD features. With an aggregated API server, you are fully unconstrained. Once your server takes the request, it can do anything it wants with it. But there are some risks in using this approach when you get into the availability of the aggregated API server and how it impacts the cluster. So let's zoom in on how this proxy method works. The QAPI server has an API resource called API service, and this API service holds the group and version that is going to be proxy. So this one here saying first.com for V1 means that any request for V1 first.com resources will go to service one. This is a service running on your cluster, and this represents a pod running on the cluster hosting that service. You can have multiple API services for different API groups and for different versions of the same group. Let's work through an example. So here we have a client who is doing a cube control get of foos for V1 first.com. This request is going to go to the QAPI server, and the QAPI server is going to authenticate Bob and then realize that it needs to send the request to this API service. At this point, the QAPI server will proxy the request to the service running on the cluster, and the external API server will receive the request. Let's drill into how authentication is going to happen on this external API server. So on this external API server, we have a delegated authenticator. This is the authenticator that is built in to the generic API server provided by Kubernetes. And when it's configured to use in cluster authentication, which is the default, it will self discover inside the cluster. The proxy client CA used to recognize the QAPI server as a front proxy, and the front proxy will be the first choice. So if the request comes in and it has a client CA that matches our client certificate that matches this proxy CA, then it will be terminated as a front proxy and it will pull the headers out and use that to assert the user. The second thing in the chain is going to be a client certificate also discovered on the cluster, and that can be used to terminate any other sort of client certificate that identifies the user itself, not a proxy. An example of this would be something like a Kubelet. And then finally, it can also handle tokens. So a token review will be sent back to the QAPI server running in the cluster and the QAPI server will return back to the aggregated API server, what user that token belongs to. So the next question becomes authorization. Do you do it at the QAPI server or in the aggregated API server? And the answer here is both. It has to be both. And we're going to zoom in to explain why that is. So when we look at the connections for how an aggregated API server can receive a request, we can actually see there's multiple paths. One path is the one we've been talking about. Client goes to the QAPI server, QAPI server proxies through and identifies the service running in the cluster and the pod handles the request. That's the normal path because that's how clients mostly access APIs. But because we're running as a service in the cluster, fronting a pod in the cluster, anything that has access to the service network or the pod network can actually make a direct request. Because of that, authorization has to happen in both spots. The first spot is at the QAPI server. This is an RBAC check. Usually it can be anything, how you have it configured. But generally RBAC and it will check to see, does Bob have access to run a get on foos in v1first.com? If the answer is no, the request won't be proxied. But if the answer is yes, it will get proxied through or if a client bypasses the QAPI server and goes directly, the aggregated API server will run another authorization check. And now let's look at what it does there. Here we have a delegated authorizer. Again, this is the one built into the generic API server provided by Q as a library. And this one has multiple items in the chain as well. The first is a hard coded set. And this hard coded set you often see used to allow system anonymous to do health checks. And that makes it very efficient for the Qube to do health checks. Keep in mind, this has to be configured by the aggregated API server. The stock library allows it but doesn't specify it. The next piece is the always allowed group. Inside of a Kubernetes cluster, system.masters is hard coded as always being able to run any request that it wants. And this allows the cluster to self bootstrap. And then if a request does not match either of these two categories, a subject access review is done. And that is the API that the QAPI server exposes that allows for web hook authorization. And the delegated authorizer will do it so that the rules match exactly. If the QAPI server says you can do it, the aggregated API server will believe that you can do it. So now that we've talked about the mechanics of how we actually perform the proxying, let's talk about what it allows you to do. The first thing we're going to talk about is binary storage formats. And the OpenShift API server is a good example of this. The OpenShift API server needed performance that was greater than we could get using JSON. And to make this work, we added protobuf, same as the Qube API server uses. Now, CRDs do not as of yet have a binary storage format. Hint, if anyone wants to write a cap, I'd be very interested in reading it. But as of now, they don't. And so an aggregated API server allowed us to use protobuf serialization in SCD and as the wire format and the QAPI server passes the information back. It let us hit much higher performance targets. The next option that we can talk about is a no storage server. You might say, what good is that? Well, if you're familiar with the v1, beta1 metrics, this is the one that backs HPAs, for instance. The Kubernetes 6 project metrics server actually has no storage, no persistent storage. It reads and scrapes values from the nodes, holds those values in memory, and replies to API server requests that way. There's no persistent storage. And the last feature that I'm going to mention is that you can have multiple implementations. And here, again, metrics is a good example. So metrics in memory lets you get to a certain level of scale. But there's an alternative implementation that fulfills the same API, still v1, beta1 metrics, Kate's IO, but it does it using Prometheus. And so Prometheus actually acts as the scraper. It stores metrics values over time. And then the Prometheus adapter will query Prometheus to get the answer. And the client, that client that accesses the QAPI server, has no idea which API server is actually servicing the request. So this allows you to drop in different implementations if you want to experiment or if you have different needs in your particular cluster. So all this sounds really cool, better performance, more storage options, what can go wrong? I talked about some risks at the beginning. Well, let's talk about those. So HA Qube API servers is a very common configuration, right? If you don't want disruption, you normally run three QAPI servers with three at CDs. But when you do this, you can get interesting behavior on aggregated API servers. So if you have some kind of network disruption, maybe the node that is running API server one cannot reach the service network or cannot reach the pod network for whatever reason, then what happens is end clients, those users who are trying to get your metrics or get the openshift APIs from the previous slides, they see a one out of three chance of just not having that API available. And that leads to I'll say user frustration. But it's not it's not fatal. It's just frustration. It's very important to have reliable access. The next class of problem is actually a little more severe. If you lose an aggregated API server entirely, no Qube API server can reach it.  That causes API discovery to fail. And if you remember from our presentation, I believe it was last year, the API discovery is the set of APIs that are used to build rest mappings.  And so if you cannot get your discovery API to work, you will end up in cases where garbage collection effectively hangs for some resources. And certain types of owner references that you go to create, ones that are supposed to block owner deletion, can actually fail if discovery doesn't work. The other thing that can happen is that namespaces can get stuck finalizing and they won't delete. And there'll be a message in status when this happens. It says discovery checks have failed. But as a significant risk and a very annoying problem, there's one special case for namespaces that I'm about to go into that is very interesting. So here we have a namespace deletion cycle. Yes, cycle. So this came up when we were developing operators for OpenShift v4. We had aggregated API servers, remember, hosted in our cluster and they existed in a particular namespace. The example here is an aggregated namespace holding an external API server. And someone deletes the namespace. They showed enough that they deleted it. Now, as soon as this happens, the namespace goes into finalizing state. And that means no creates are allowed. So you can delete content from the namespace, but you cannot create anything new. And deletion happens immediately. So this pod gets deleted. Now, when this pod gets deleted, the namespace lifecycle controller gets stuck because it needs to discover all the namespace resources in the cluster so they can delete them and be able to make the delete calls. But it needs to make delete calls to the external API server, which was just deleted. So what happens is the namespace controller can't make progress without the external API server. And the external API server can't start until the controller deletes the namespace. Now, there is a way out in the cases where you know you're deleting this and you're going to bring it back. And that is to actually make sure that everything has been cleaned up and then manually clear the finalizer. Once you do this in case of OpenShift, for example, an operator will automatically create that namespace again and reproduce the aggregated API server that you need.  But you need to be aware that this is something that can happen. So now the limitations. They can do a lot. Aggregated API servers can do things that no other technique we have allows. But the one thing they can't do is coexist in the same group inversion with a CRD. So if you have an API service for v1first.com, you cannot create CRDs in first.com. So this means that like your thought of, hey, I wonder if I can use this to create sub resources for my CRD. You can't. And you also can't mix those special types of storage, for instance, with a standard CRD either. Maybe someday this will come along. But as of right now, you can't. All right. So that is all I have about aggregated API servers. I hope you found that interesting. And I'll turn it back over to Fede. Hello again. I hope you enjoyed these two talks as much as I did. I'm always impressed by the level of depth and quality of the presentations that my colleagues prepared. Thank you so much, David and Jeffrey for working through aggregated API servers and open API v3. To close the chat, I will just remind you that CAPM machinery is part of open source Kubernetes. We have open meetings twice a month. We have twice a week, three ashes for pull requests and issues. We have two subworking groups. So you can join any part at any time that you feel could be interesting for you and you can contribute. I will leave this slide here so you know where to find us and how to connect with us through mail, through Slack, or through our homepage or in GitHub directly. Thank you again for coming and I will leave it open now for questions to the session. Thank you again. Have a great CUPCOM.


I'm Sarah Backhouse, founder and host of Future 360. Future360.tv is an online video network dedicated to clean technology. We cover cutting-edge clean tech innovation and disruptive business models. Everything from green-building technologies out of Silicon Valley to waste-to-energy technologies and remote parts of the country. So my journey from camera to LA started when a friend of mine found an advertisement for a Japanese-speaking researcher for a television show which ended up being a show called Beyond 2000. That random serendipitous moment led to my career in television. Started in Sydney and then moved to the bigger TV market of London and then the kind of mothership of media Los Angeles. Semantically, I've moved from mainstream TV to covering sustainability and now clean tech and it's so lovely that California is a leader globally in this field so it makes sense that I'm here. It takes a certain something to make it in this city. So everyone that's here is very ambitious, very driven. Right now is a really exciting time to be here. There's a whole wave of innovation in the tech space and media space and incubators like the one we're sitting in right now are sprouting up across the city. So it's a really exciting city to be living in. I did a Bachelor of Economics and Japanese Studies. I vividly remember that walk from the Copeland building to Asian Studies building which I must have done four times a day for four years. There are so many great memories. I co-edited the student newspaper Warowning. So I remember kind of late night deadlines and then the thriller, the newspaper coming out the next day. The current students, I would really encourage them to find their sweet spot. If you think about three circles, one represents your passion, one represents your skill and one represents what the market opportunities are. When those three things align, that's called your sweet spot and once you find that, I think you'll find a very fulfilling, sustainable career. My advice to students will be, if you can't find that, create it. In a really short space of time, I've got to meet some really interesting smart alumni that I had no idea were living in the US and apart from reconnecting, it's been great to network and there's already been opportunities to collaborate. It's so easy to get involved. It's a world of possibility and one that I think everyone should be curious enough to explore.