talks at the Markney Segal Theatre Center, the great Graduate Center CUNY in the middle of Manhattan in Midtown, New York that works in a way fuller than with people on the street than it was the last weeks and months ago. The stores are more and more of them actually are empty. So it's confusing what we see out there. We first time spotted people who have sued cases on wheels and so there must be tourists and not just people from maybe Rikers Island who go to the hotels or people who are replanted. I helped them from the streets. So the mood is great. People are outside. They are talking. Nobody really knows if you put on the mask or not. You take it down when the waiter comes. But all in all, things are moving forward in a good way. We think there's a good leadership. Some news from the world are very troubling, especially from India. It is catastrophic. We heard from our Indian colleagues and Anna Ruparoy, who actually also was with us. She's in the hospital at the moment, struggling for her life. And Abhishek, one of our contacts in India, a great right on playwright, is up all night with his volunteer group to connect people who need corona help with hospitals. The few that remain, they are all volunteers and just two people of his group died. So it's stunning what is happening. On the other hand, we heard also great news about, for example, about Sheila last week from Martin and Marco about this great referendum. The constitution will be rewritten. It's a sensational development of an election that will now help to overturn the constitution since the Pinochet years. So the world is in flux as always. Nothing stays what it is. And we are wondering, where are we? Where are we coming from? And where are we going to? And today we have guests with us who are also part of world theater, global theater, world theater history. It is the form that is called kind of Yiddish theater, a very, very significant us in contribution that was made towards the idea of what theater is, means, where it comes from, what it carries. And so today we have with us Ellen, Yelena, and also Shane, three veterans of the scene. And I think what we really want to thank you all for joining us before I read your bios and we learn more about what you do. Where are you now, Shane? Where are you? I think you don't have your- Right now I'm in Kansas City, Missouri, visiting my mother for the first time in a year and a half. Oh my God, is it your first trip out of New York? No, actually, thereby hangs a tale, which we'll get into, but Ellen, Yelena and I played theater on the road socially distanced in Hackettstown, New Jersey. Fantastic, socially distanced Yiddish theater. And Ellen and Yelena, where are you guys? We're here in front of the computer. And are you in- I'm sorry, I got a little confused. You're in Ellingtown, Ellingtown and Yelena town. If you've ever been to Yiddish theater, you'll understand why we like to do it socially distanced. Most of our audience members like to stay as far away from each other as possible because they know each other and they know that's the wisest thing to do. So you are in New York City at the moment? Yeah, we are very near the CUNY Grad Center, very near the Segal Center. Amazing, amazing. I had no idea. So let me tell our audience a little bit. Let's start with Shane. He is one of the great actors of the Yiddish stage today in the Americas. And he started as Vladimir in Beckett. So waiting for Gojo. I think he's the one by Moshe Yasur, what he directed. Yes, correct. It's Moshe's production. Yeah, one of the great directors of Yiddish theater, one of the old masters who learned himself from the old masters, I hope. He is listening and the New Yorker was very highly full of praise for that exceptional production of the Congress. And where did you do it? We did it at Castillo Theater, the All Stars Theater. We also were invited to tour it. And so he is a Yiddishist. And he is a dedicated part of his life to this great form, that great tradition of Yiddish theater. And they produced also that di-book, the di-book. Very significant, perhaps the most well-known, and perhaps also the best play coming out of Yiddish theater and many, many others. And he also worked as a company that is very significant for New York City, for the history of New York City, for the history of Ongard, in a way even for the history of the New York punk music movement. It's the ridiculous theater company, a very important contribution at the Segal Center. I think we hosted the 50-year celebration and I hope one day we might also get Quinton here or others to talk about how they are doing. And then with us also is Alan Lewis-Rickman. And he is a director, translator and writer. And they did so many, so many works. You can look it up and see it. We also in our email announcement put them out, but everything that is everything in the Yiddish theater he has done. And he really is, together with the other representatives, the soul in a way of Yiddish theater here in New York, a work that also the folks being the carries on, of course, they are very famous. Both of them, he and Yelena were being in the Cohn Brothers movie, A Serious Man. And that's a significant credit. And I think Alan, you also were in Boardwalk Empire. I think I saw you there, somewhere hanging around with the gangsters and many, many more things, even so you didn't even mention it. Yelena Schmulenzon is perhaps best known for killing the book in the Cohn Brothers, Oscar-nominated film, and she is responsible for the miserable life of Michael Stulberg centuries later, so we all have to see it as a fantastic. But she's also in Orange and the New Black, Boardwalk Empire, so many, many, many things. But of course, in the center of, I think it's fair to say of their love for this theater, for the work for this theater is that thing that is called Yiddish theater. She's also Russian Yiddish coach trainer and a translator and she has got many awards for her recorded books in English. So you guys, we of course know, I think, at least a little bit, you know, what Yiddish theater is all about, who for the audience also have international letters, maybe give us a little context about that form. Where does it come from and how do you keep it alive? I don't know who would start, what's that? Jane, you, me, you like it? I'll let the theoreticians do the work. Well, Alan is the great expert in Goldfaden. A pre-Goldfaden. Pre-Goldfaden, there's not a lot of Yiddish theater. He's considered the birth of it, but Yiddish theater before that was pretty much Purim spiel, plays that were performed at the festival of Purim and I believe it was primarily all men and aside from that Jewish tradition doesn't look too favorably on theater, but then along comes Alan, take it away. And then along comes up in Goldfaden, which was really the beginning of, it was a very good book that came out just recently, the birth of the modern Yiddish theater, which is exactly about the Goldfaden era and the Goldfaden revelations. And Goldfaden was, this is at the time of what was called the Haskalah, which is the Jewish Enlightenment prior to that period, which was in the mid and late 19th century. Jewish culture was strictly either religion or religious related culture or people would basically assimilate into the general population, but there was no vibrant Jewish secular culture. That was the Haskalah. Well, literature started, journalism started and Goldfaden basically was the fellow who brought theater into the Yiddish world. And he started it completely himself. He started with tiny gold productions with tiny gold companies for which he wrote. In Russia, right? He started out in Russia. His very first productions were in Romania and then he very shortly afterwards moved to Russia and he traveled all around Eastern Europe. He performed in Russia, he performed in I'm just about positive Ukraine. He went back to Romania up until the latter decades of the 19th century. He was all around that area. He had some great successes in Kiev. So yes, he was in Ukraine and he then came to the United States, went back. He was peripatetic as Yiddish theater was tremendously at that time. Yiddish theater became very, very quickly a worldwide phenomenon which became highly, it was highly popular in any place where there was an immigrant Jewish population an Eastern European immigrant Jewish population which means throughout the Western world in South America, in South Africa, of course all up and down North America and Central America all over West New York. And they took off from there. I wanna have a little quibble with the phrase that you use Frank Hame which is Yiddish theater as a form. The truth is Yiddish theater really isn't a form. Yiddish theater is many, many forms. Every possible form of theater you can think of has been done in Yiddish. It started with very straightforward popular theater and then later expanded to basically and so much in the innovative theater of the 20th century comes from Yiddish theater from various places in various times. Yeah, I'm known. This is quite a remarkable sort of ghost father who came from Romania, was a star in Russia then had to leave Russia then he became a big star in Romania then he was forced out of Romania, came as an older man almost people thought he was dead already came to New York, became a celebrity. I think a hundred thousand people came to his funeral here and so much of also of American theater through Stella Adler. I think the Adler family was part of his company and the idea of musical, the idea of male drama they have melodies and you have a drama the Broadway music of jazz, klezmer music the soul and also of kind of the Yiddish theater of the clagga, the quetching and so many, many other things was an important contribution. Yeah. One specific one, which is acting style naturalistic acting is something in America that came from Yiddish theater. One of the things that was highly, highly prized by the theater audience early on was naturalistic acting and the kind of detail in acting that people would recognize, small bits of business little turns of character, little acting choices that people would make. I mean, realize you're coming out of the late 19th century coming out of the, it's the actor manager era still and we're used to in the Western world more to climatory style of acting where everything revolves around the central figure whereas in Yiddish theater the supporting characters became very important and the nature of their acting style became a real model and many, many, many non Yiddish speakers come downtown to the Lower East Side to see specifically to watch the acting John Barrymore would go to Yiddish theater to watch the acting styles and many of his predecessors would go down to Yiddish theater to watch the kind of acting they were doing, which as I said was filled with detail that an audience would watch and they would quell which means to express a certain kind of pride slash pleasure, so natural, I would say, so weird. And so much of American screen acting then through Actress Union but also for the work of the new group which was highly influenced by the other method in a way and it's the direct bloodlines, incredible in a way, what came out of that was in a way is for immigrants, refugees, people who were harassed in their own countries, came to America and brought part of a culture that then got reintegrated and we mixed, we would say hybrid forms and came out of it but since our talk also is about the time of Corona how do you guys, how are you doing? We had, for example, people who work in circuses and tap dancers also forms who are not at the center at the moment, you know, of attention even so I think they really should, especially circus and also tap dance is a great American invention like jazz. So how are you guys doing? How does Yiddish theater work in the time of Corona? What did you do? How did you survive and accept a trip which you took to New Jersey? So we heard about, so what's going on? How are you doing? Delano. Delano, just like everybody else is doing it was hard the first six months, I think and then I think once someone, one of these two came up with the idea of doing the Dibbuk. Shane, this is all Shane's doing. Shane came up with the idea of doing Dibbuk on online as sort of a supernatural online presence. It really felt like we really got busy because there was all of a sudden a lot of things to do and plan and rehearse and record. And that took a while and it feels like things are picking up now and it's much better than 2020 was for me. We started earlier on in the pandemic while a lot of people were doing a lot of Zoom plays a lot of online readings. And then Shane came up with this idea of doing something more substantial and then we basically, starting with the Dibbuk, we started trying to work towards another form which is to say an online version of a theatrical experience rather than just a reading with everybody sitting in boxes like we are right now on this chat. And this is Shane's doing, so let me throw it to him. Yeah. Sure, there by hangs a tale. First and foremost, I would say that as counterintuitive as this might sound, there has been a silver lining to the pandemic for the world of Yiddish. It has grown, as we say, like mushrooms after a rain. It has enabled, I mean, a few people like Collier Borodoulin at the workers circle with his Yiddish classes had adopted the online format for international learning or even cross-country learning. But it became a necessity once the pandemic hit. And so we had at the Congress for Jewish Culture a Yiddish Seder online. I was very much against anything online. I hate it. I still don't really like it because it's not live theater is just what interests me. Live performance. But I saw this Seder and we had people from all over the world. And I said, okay, we'll do something for Sholam Aleichem's your side for the anniversary of his death. And we got people from Israel. We had every continent covered. We even had a greeting from Antarctica. So with a seal rolling in the background and some Yiddish. So it became clear that, yeah, it is possible. There is an upside. We can get the best actors from around the world for something. But I had been wanting to save this 100th anniversary production of the Dibbak for a live thing. And it became clear that no, this pandemic is not gonna be over in December. So Allen agreed to direct it. And it's really Allen who took that and said, the regular standard Zoom reading will not apply here. So at this point, if you want to talk about that Dibbak and when Yelena says, all of a sudden we got very busy, I'll tell you that part of the reason they got so busy is because we didn't decide to do it online until about three weeks before the actual hit. And Allen worked like a madman to come up with what became a really wonderful production. Allen, do you like to talk about? Thanks, maybe, yeah, I'll try not to repeat what Shane said, but I think that there are a couple of key things is the weird upside because one of the issues in doing theater in Yiddish is look, where do you get the cast from? We have a pool of people in New York, a fair sized pool of people, but it is still, although I say it's not minuscule, it is still limited. And there are wonderful Yiddish actors in Israel, in Berlin, in one of our, in Strasburg, although he's now in Argentina, et cetera, et cetera. So this online situation gave us the opportunity to have people work together in a way that would be borderline impossible for all reasons of practicalities, were it not for this situation. And what I tried to do, something I alluded to before, which is to say, make it vivid. Try to find ways to make the online production feel to the greatest impossible as if people are in actually in the same space, we found visual ways of doing that, to feel like people are sharing a space like they're sharing a stage space. And also to use visual elements that help in the narrative to replace the things, the stage elements that we couldn't normally get. So you can't do a set, obviously, in an online production like this, but we found visual replacements for that so that we could put them into an environment that becomes itself part of the fabric of the story. Just like in Yiddish theater, the language itself is always so much the fabric of this part of the fabric of the story. And the language, language, but just to recap, a theater that started with Abraham Goldfarb and I would say 1870 or 1880, I don't know exactly when, but then more transformed, traveled afterwards through migration, through refugees, through immigration, now all of a sudden, in the time of Corona, a small company and you are nonprofit, if I'm not saying right, like every nonprofit company in New York City is struggling for survival, all of a sudden you do a Yiddish play online with Zoom because you're forced to do it and you seem to like it. You say, we connected to people. People send us greetings from another actor with a seal actress we couldn't connect to. Do you feel for your form and Yiddish theater was so significant? Ellen Stewart, I think first theater she had, it was Yiddish theater, she took over so many on the Lower East Side. Do you feel for that form, even though Ellen says it's not one, but for that idea of Yiddish theater, is it a revitalizing thing or do you think it contributed to the slowly fading of something that is and was so important? Jane? I'm not sure I understand the question. You mean the pandemic and Zoom, if it contributed to the- How the Yiddish theater do you think it will be was one more thing, because if anything, I saw the Leo Bach Institute, what you guys did, you need to see that life, the beautiful comedic way, the representation on stage. And to see something as, you know, as it's also built, something as soulful and playful and melodramatic, but to see it on a screen, on a computer. I cannot think of a bigger contrast. So, but do you think, so this form, your way of working to help you to go on, once it's over to find new ways to add new audiences? I view it as a blip in our history. I think it's something that historians and grad students might study in the future, but I don't think it's going to have a huge change in Yiddish theater for the coming decades. I have to argue against the idea of a decline in Yiddish theater these days. I think that the audience is perhaps not as strong. I mean, it's definitely not as strong as it was. And there's not as large a pool of talent, but I sat down and figured out the other day that there were seven Yiddish theater companies in New York. Now, only three or four of them are actually producing anything, but on paper, there were this number of Yiddish theater. What are the names? What are the names? Well, I mean, I don't want to go to Monty Python on you, but the first two names I'll mention are the National Yiddish Theater and the Yiddish National Theater. Yelena, what did you say? Not confused that you guys are separate organizations. Yes, there's the National Yiddish Theater, which is the Folk's Peanut. There's the Yiddish National Theater, which is the Hebrew Actors Union. There is the New Yiddish Rep, which actually produced Waiting for Godot and Yiddish, the first of all there at the Castillo Theater. There is the Yiddish Public Theater, which really relates to Yiddish National Theater if you see the names in Yiddish. There is the Joseph Papp Yiddish Theater. So there are five. There's the Congress for Jewish Culture with its Royta Kutter Theater that Moshe Yasur is directing and Alan has done some things through it. Then there's Alan's Theater, which is Chabse Yin Bud, which is a production company, but really it's a theater. And there's one or two that I'm leaving out just because I can't remember all these names. Great, well, now we've lost a couple of friends. So now we've lost a couple of friends whoever you're leaving out, thanks a lot. Yes, yes, yes. No, we lost one friend who has two theaters. So all of these theaters have brought together an international audience or performed internationally. I mean, brought together international actors or performed internationally. But what Zoom did was allow us to bring together a whole group of people all at once in a way that was kind of previously unimaginable. I don't know if that will bring in funding that will allow us to do something like a live production of that Dibbak or a live production of Megilla Cycle, which was the second show we did with Mike Burston directing. And in that one, we actually did kind of create sets. For the Dibbak, Alan came up with these woodcuts that we put together as a feed between the scenes and kind of it was eerie how they reflected the action of the play. So then for Megilla Cycle, I engaged someone, Adam Whiteman, to create new paper cuts, a very Jewish form of art that provided the settings for the Zoom boxes which were then cut in by our brilliant editor who never edited a thing before in his life. This guy named Uri Schreter who's in, he's a grad student at Harvard and basically he can do anything he sets his mind to. But yeah. I'm gonna disagree with you on one point in response to Frank's question. I don't think that this, Frank, I agree with you absolutely 100%. There is no substitute for a live audience. Theater exists to be a communal moment, to be a communal activity.  That said, this form that has been developed since the pandemic, very useful and I think will continue in, despite the fact that we're going to come back to, please God, some form of normal life. Because it allows us to do certain projects that ought to be done, that aren't possible to be done without this form. Shane and I have been talking about various scripts from the Yiddish theater canon that would require if we were to actually to do them for real on stage, insane amounts of resources. Because back in the day, Yiddish theater like every other type of theater in the world, a century and a century ought to go, they produced large, they produced cast were always big. There were always multiple sets. And that detail in the sense was very important, et cetera. The physical part of the production is very important. So when you combine the size of the cast and the physical expense of the physical difficulties and the availability of actors, this form is perfect to do if we were to, if we want to do a text that would not otherwise be done. On the other hand, please, please, we do very much want to get back to performing for a live audience that we're all in the room together sharing the experience. Shane. I just want to add what you said, Alan, reminded me of something. I've met no less than three statistician spear carriers, we say in English. People who played with Maury Schwartz's theater back in the day, or say they played with the theater because they went into a couple of shows carrying a spear. At that time, in the 30s, in the 20s, you could put out a call and you could have 40, 50 young people come in and be ready to go on stage for their hour or for their five minutes at a certain point, maybe with no lines or whatever. The last example of that that I've really seen was when we played Carrot O'Brien's God of Vengeance translation at Show World back in the late 90s. We had an open call to Yeshiva Bachrim and people from this Choland group that if you're in Manhattan and you want to see and be a part of this show, just drop in at this hour and we'd make sure they had on a yarmulke and a jacket and we'd tell them, okay, this is what we're going to do. And they'd go in and be the party guests and then they'd leave or they'd sit in the theater and watch the show. But that's, yeah, yeah, it's true. There might be some more things come out of this that would otherwise be impossible. I do think so and it is stunning that many companies or small companies are putting their small spaces, yes, television studios up there that they think there will be an online presence, even if everything goes back to normal, who knows when the next pandemic will come, as many remind us, and many others, maybe this is just a rehearsal for environmental catastrophes that will come and it will be much worse. The significance of this play, the Jansky play the dipper just to point out if I'm not mistaken, Peter Brook did it in 1990. I actually saw it, St. Miriam Goldsmiths was the great actress. It's a fantastic play. It's part of world theater, world history, and let's come to that idea of the ghosts or the undead. It seems to be a bit of a theme, not only in the dipper, which is kind of the undead, but also what in the film, the famous opening of the Colin Brothers film, which you both so masterfully created, the idea of a ghost, Marvin Carlson wrote a lot about it. All of theater in a way is ghosting. You represent dead people on stage or it's young cots that is something between the living and the dead. You don't really know the life goes out on someone pretends to be someone different, but now you have a ghost-like appearances on a screen. You had the ghost come up on our screens or projection. How did you feel as an artistic engagement with it? Maybe also as an actor and being part of it. Was that also a way to engage with the central theme of the play of the undead or something that seems alive, but it's not alive? Something that is there, but it's not really there. The big question, what is real and what is not real? We're gonna have Carol Martin back about it, so much about the theater of the real, so it always has asked that questions. It's a marivo, you know, and other what is actually real? And so we have that here and your play dealt with that and you did it on Zoom, on screens. And so that idea of a ghost in you took it very serious, put it out and created something. Did you work with the medium? Did you reflect all about that? Maybe all of you can answer by yelling on it. Maybe you start. I have something to say. The, it's interesting because you said it felt like because this was basically the first real theater experience we've had, I've had all year in not counting the production in Hackett's Down, New Jersey. It felt like a ghost of a theater production. But you just said a real, it was a real theater experience. Yes, it was, it was because it was sort of a hybrid between live theater and film because we had takes because we recorded it, we prerecorded it. This wasn't live on Zoom.  And yet I'm not because it is really hard to do a performance like that, which is, it's one of the major roles. And it's very hard to be possessed by the dibook on Zoom. It literally felt like not half of, like a ghost of an experience I would have had on stage had I done it for real on a big stage with probably the portion possessing me very, very close and not in Berlin. But it's, and I didn't really want to watch the final product. And then when I watched the final product, it works something even out of my experience feeling that it isn't real, it was real. And it was real for so many people because the feedback we got was absolutely incredible from all over the world. And, you know, this was us in sitting in our homes and boxes, this is not the right word, but pretending we were doing theater. And yet somehow it was real for us and it was real for so many people. One of the things that helped with that production particularly, and this is something I always think is far, far, far too much ignored in modern theater. And it's been the one universal unifying element in all cultures and all societies going back as far as we know is narrative. We have been so fixated for generations on form and so fixated for generations on the artist's experience rather than the audiences and the artist's experience. And the key to that always is narrative. That's the backbone of how we communicate to each other and what we did with the Divock specifically was in the process of adapting it, it was pared down to essentially the narrative and so much of the stuff that was there for other purposes, we just kind of peeled away. And that's part of the reason for success with audiences because that's something that people have been interested in going back to Homer. From another angle, I just want to say that the origin of the online production of the Divock dates back to last summer when I was teaching a theater workshop for the YIVO summer program in Yiddish. And I wanted to do something with the Divock in spite of the pandemic. So I called the class, the ghost in the machine, the Divock in the machine. And Mr. Divock is a spirit, a ghost of an undead who possesses your mind. Right, right. And I took with that class basically the approach that Jeff Weiss used to take with us in Hot Keys and that's how the rent gets paid, which was to give them scenes and I let them work on them and then I'd have them come in on Zoom and I'd give them a couple of suggestions here and there. I did not direct their scenes in any traditional way. And what we wound up with was a wide range of style, quality, whatever, but everyone achieved something. The scene that probably had the greatest effect was the one directed or teched by this guy, Uri Schreter. And that sort of gave the idea later when I approached Alan and said, would you direct this? He said, I need an editor. I need an editor. And I was thinking, we need an editor who knows Yiddish. He knows people who don't know Yiddish, who would do it for the little money that would come in. But then I thought, I know someone who knows Yiddish, who knows how to edit. And that's how poor Uri got looped into it. You know, I think there are more possibilities for a live production of the Dibbak that explore the inhabitation and also the mechanical or electronic elements of our current world. But I also think that this production that Alan really pulled together in an amazingly short amount of time is going to stand as a document for the rest of time in fact, I would go so far as to say that as a narrative and as a representation of Ansky's work of the original author's work, even though it's even more sharply edited than redacted than the film that's so well-known that this is the superior product. It restores elements that are missing from the film and grounds it in a way that just it's quite near and dear to me. Amazing. It's a big thing to say in it. I looked at it and it is haunting. It asks a lot of question and of course puts you to a mind, almost what it deals with anyway, like past, these really obsessions or we carry with us when we try to escape from, as Kafka said, like it's always how to escape, how to get out of something. And by something is holding us back. Is this the first Yiddish production you guys know of on Zoom, which is not just reading, but it was a theatrical, as Yelena said, a real theater experience? I haven't heard of any one of another one. It's so much probably a part of theater history. You created in Zoom time as a small company, the very first Yiddish Zoom. There were a million little productions of things, you know, reviews or music or something like that. To my knowledge, this is the first real production of a play that, and certainly that will have lasting value beyond just a pandemic entertainment, something of service to get us through this time. And I'm gonna repeat something, I'm sorry. I'm gonna repeat something I said before. I mean, the thing we learned from that is that we can do certain things online. So it would, Shane and I were speaking recently about a play that I have a particular interest in. And I've been rolling this play around in my head for a while and it's a very, very strong piece that could use a revival that really needs a production. And we thought of doing it this way online. It would be basically impossible without massive, massive resources to do it for real. Well, now we are coming to the moment where online substitute theater, that we're calling a phrase, is going to be the last thing that anybody wants to see. So that puts us into a weird question-rate timing. Do we go on and do this piece in a time when nobody wants to look at theater online or do we wait six months a year and say, in the meantime, the interim, do something with actual humans in the audience and then go back and do this? Yeah, it's a good question. My prediction is that perhaps it might be like with music, there are the great life acts, but you can go and by the DVD, the old days, you have the cassette, you can have it on iTunes, in a low quality, which actually is hurting the brain, as Neil Young pointed out in his research that we actually might be destroying. This is low resonance things. Our brains for that life, music is healing and life's theater. But, and there might be the play on a TV, you know, screen on the production, which is on Netflix or it's on some online platform. So you have all of it, the life is, and the same will be perhaps like in music. The theater will do the same and a lot of theater companies also in Europe are thinking that the digital leg, arm, head, whatever you will say will stay, there will be next to opera, ballet, drama, there might be the online component because also audiences, as we always say here, Brecht wrote for the children of the technological age, but now we have the children of the digital age, kids who grew up with the iPhone, iPad in their hand, kids who go to a normal book and try to swipe the images and it doesn't work and they're frustrated and until they understand it's a different medium. So it will be interesting. A question to you guys coming to that, perhaps also content of Yiddish with the idea of being isolated, the idea of the danger outside in the world that you can die at any minute the world can collapse the uncertainty in a way the experience of Jewish immigration. How did you guys personally experience or how did your family's experience and your fellow artists, what do you feel it had an additional meaning? We had Carrie Perlov with us, the great director and she said she went back to kind of a spiritual rethinking of her Jewish history. She was surprised by that, that she did that next to acting, directing online, writing book on Stoppard and a pager and online directing, but that was something she went to. How was it for you personally, the experience of Corona, did it change you? Helena? Helena? It was hard, I mean in the beginning it felt like nothing had changed because most of the time when we're not working, we are at home looking for work basically, researching. And I think about two months later this sort of, it became clear that this is not like the real, this is not just, it doesn't feel like a slump, it feels more permanent somehow and a large family and everybody's really dispersed and my parents are a little older and you know, but there's nobody there to explain the technology to them. It was really hard to sort of connect, you know, you could talk on the phone but we wanted to see their faces and then the cameras weren't working and they didn't know how to turn it on. And you were so helpless because you can't help because you're so far away on the other side of the screen.   So like, look, I'm a person, somebody sees me, I see another person, I should let them know I see them. And it's coming now, you know, seeing friends and seeing people, it's like, it's a surreal experience to be able to hug someone, to just, and to stand so close, you know, face to face and have a conversation and who knew we took that for granted? I wanted to, let me expand on something about your response to your question about fear of the end during the COVID period. I have to say, I never, I can only speak to myself, but for myself, for a combination of reasons, I never felt the same level of alarm that some other people did. For one thing, not without going into details, I have a medical history and technicolor. So I've been through, I've had any number of delightful things over the years that I've survived. So my instinct is, you know, this too shall pass. I also have a tremendous historical interest. I've always, always thinking of anything in historical context. And I'm very aware of what went on in 1919. And then of course, in early human history, we've had far, far, far worse things. I'm not to belittle what are losses, but in the big picture of all of civilization, it's a different story. And not that long ago in my own lifetime, we all remember the AIDS crisis and we all lost people who were close to us, but we lost some people, but it did not end civilization. So I never felt inside that this was going to end civilization. I felt that instinctively, that this was going to put us in a weird place for a limited period of time. And what comes out, the changes that this has made to everything we know, what will come out of the other end, that I have no idea. That to me is the great question. But I never feared that this was going to be the end. From my own side, I did see a few people that I really cared for who died from this pandemic, one rather early on a pianist by the name of Rick Winterberg. And it's a great loss. He wasn't too much older than I am, but I don't know how much fear I was living in with for myself with the pandemic. I was worried about my mother. I was worried about some other people that I know, but I was very happy being my own particular character to spend as much time inside doing nothing as I possibly could. And I think that that has been all right with me to an alarming degree. It's a good rehearsal for being dead, I guess. I haven't used Zoom except for working type things, like these events that we talked about. I haven't really like, hey, let's get together on Zoom and hang out. But maybe that's because what I do professionally is what I would do if I wanted to have fun anyway. The one Zoom thing that I really did, the one online thing that I did with any regularity was a salon with Everett Quinton and other people from the ridiculous theatrical company every Wednesday night. We got together through Mark Erson, the pastor at St. John's Lutheran Church. He set this up. And that was actually my first online pandemic show was the Ring Gottfab Lundgett, the ring cycle of Charles Ludlam, that Everett directed for Gay Pride. We had been supposed to do Camille and that was not paused. So he put together the ring cycle. But yeah, that's it. I'm looking forward to a return to real life, but it's a funny, sticky thing trying to figure out how and what. My mother's vaccinated, so am I. So I figured no harm in me coming to see her. But like Elena says, there's a, who can you hug? My aunt and uncle came up as well. Who can you hug? Who do you... Okay, we'll get there. It'll take a little practice. A Yiddish theater will come back more slowly than all other theater though, I think. The audience is a little more health conscious, I think, or careful than whatever audience. Of course, Jews play a great part of any regular theater audience as well, just as an older crowd is a part of most theater anyway. But we'll get there. I know you've touched on a lot of the questions, which is a whole lot of the conversation, which is the Yiddish theater audience. Because we can go on for days about rebranding Yiddish theater and the purpose of Yiddish theater in this historical, ignoring the corona moment for a second, but in this era, in this historical moment, let's say, that's a great big whopping question, but it's a whole separate conversation. Yeah, I know this is, these are all important questions. Also, like what you said, it's saying that it's also experiencing in a way, a low level death, what will happen to us. Someone said that when you die, it's the way of nature to tell you to slow down. And in a way, this virus slowed us down. And everybody said that it was a good thing. So it is a near-death experience, I think, and theater always has also done that. And what does death mean? All great stories here. Lots of many are about them, love, death and life. But is there something you feel that is inside that idea of Yiddish theater and to make sure it's not just only performed in Yiddish, often it's kept in English with Yiddish idioms, so audiences can also, of course, understand it. But is something inside Yiddish theater where you feel this is or could be important for all the companies? What do we think about? What is important to us? Because we are looking, what will happen afterwards? Things should be different and still, of course, part of the same tradition. But is there something where you say, this is something unique to us and I think it's worth looking at? All yours. No, I'm trying to understand it. Something about Yiddish theater that's important for everyone else? Yeah, we know we do something that's, you know, something that, in the soul of it, that is of significance perhaps, especially after a time of corona. What, the wonderful thing about, if we can do something after the time of corona that we can get some nice, significant-sized audiences for, as we did in the past, by the way, it'll be a lovely kind of rebirth story because of all things, people who are not closely connected with the Yiddish world think that Yiddish theater has been dead for the past 60 years when actually, in the last decade or so, last decade or two, it's been more alive than it's been since maybe the mid-20th century. So for that as something to give a feeling of continuance, of rebirth, that's a psychologically healthy thing which everybody needs. So that would be quite nice. Also, there's the value of the experience itself. To see something in Yiddish means to swim in the water of a specific culture, the very fact of hearing the language, which is the, you cannot replicate it in translation. You can do it with translation into English. You can do it with hearing the language and reading super titles over the stage, which is what we do in our productions, but you can't do it when you translate something into English because you lose the essence of the thing. But seeing something in Yiddish and hearing that music of that language for a while is a trip to someplace else, which is the thing that we go to a narrative experience for, whether it's theater, whether it's literature, whether it's a film, what have you. I have something to say that sort of, because we keep talking about the Dibbak and the different worlds and the real and not real and the dead and the undead and it sort of weaves through the entire, our entire conversation. There is a dramatic poem by H. Lavek called A Wedding in Fävenwald written in 1947 after Lavek interviewed the Holocaust survivors in around 1945 and 46 in Europe. And it's about sort of, it's a theatrical piece and philosophical rumination of the nature of what comes after something major, like a major catastrophe like this. There's a wedding. It's a first wedding after the Holocaust and the dead and the living are debating the entire play. Should there be a wedding? Is it okay? Is it okay to move on? How do we move on? The ghosts all and the ghosts are coming and Elijah the prophet is coming and the Messiah is there. Everybody's talking about do we get past, how do we get past this? Do we get past this? And the living see the dead and the dead see the living and it's, there's something, it's a profound piece and utterly worth being done for real. But it's sort of, I feel like we just need, we need a little distance from kind of what we're going through right now because it is a little easier to look back when a little time has passed. And I have no doubt that there will be some kind of Yiddish dramatic response, Yiddish theatrical response to sort of what has been happening. That's a really interesting idea because that's having up to doing a play that has those kinds of themes, doing it for real is the perfect way for people to come together as a community after this time of so much loss and share those feelings and those questions. That's what theater is for. Yeah. Just for what it's worth I put up a little thing with my friend Miriam Hayasegal for the Yiddish New York event this past winter which was a plague wedding which is a traditional Jewish way of exercising a plague from a community. You take the two least desirable characters that you know and you conduct a wedding for them in a graveyard. So Miriam Hayasegal and I did this in drag. I was the bride, she was the groom and we did that at the museum of Jewish heritage, a living memorial to the Holocaust. So that's a little bit of a graveyard. I don't think it got rid of the pandemic but you know through the pandemic there's been a lot of response to the pandemic, the YIVO and I just saw some call from another Jewish organization asking synagogues to send in diaries of how they've dealt with this. And I don't know, I think the response is going on now but theatrically for a response to happen that might take a little time and it might be on some metaphoric level but it's an interesting challenge. Or we might just like try to forget all about it and move on. Okay, nothing. How interesting, so there is a ritual to get rid of a plague and...  Is it a real wedding or is it a performative one? It's a ritual.  Not like an exorcism but you perform it in hope of... Are they... Well, it brings you closer to God. A wedding is a joyous event that brings you closer to God. A funeral brings you really close to God. And so doing these two together is going to obviously bring the whole community closer to God and perhaps drive away this plague, which could be a punishment for some sin like somebody dropped us a for Torah or the community has done something so there's this idea behind it. But yeah, that's the basic thought. There's a movie about a plague wedding and it is a real wedding because it's a mitzvah. It's bringing closer to God because you're putting together these two poor people who could never get married in any way. You're reminding me of something out of it. Something you said trigger this talking about a historical resonance. The Gosset Theater, the State Yiddish Theater in Moscow, which was one of the greatest producing organizations one of the most innovative and daring for at least a good chunk of its existence, Yiddish theater organizations in the world. When World War II was over, after everything that the Eastern European Jews and the Russians went through, instead of doing something like King Lear, which had been their signature production in the mid 1930s or any one of the other heavy dramatic pieces, what did they do? They put on a light musical review, which they hadn't done anything like that possibly. I don't know if they had done anything like that ever before. They did a show called Freilichs, which was literally, which means up tempo, happy tune, happy song, happiness. And they did this in 1947, I believe it was. And it was exactly for that communal purpose. They said, look, here we are, we've gotten through this, it's over and let's get together and be alive. Yeah, yeah. And I think, at least to me also as a bit watching from the outside loving it, I remember Gimpel the Fool by Moshe Yassour, the idea to put stories out there and to look at the world and I'm kind of a fable, the famous, let me tell you a Jewish, old Jewish joke, but actually it has wisdom in a way, these plays are very big, elaborate ones and so much humor in it and so much also sarcasm and as I say, the Jewish humor. And I think something is in there that over centuries I think was developed also to help to survive these catastrophic events to go on to do the next level. I remember there is this joke, a Holocaust survivor comes to God and they meet and God says, I'm glad you're here. And he asked, how are you? And he says, fine, the survivors, I'm gonna tell you a joke. And God says, I'm not sure if that's a good idea. I don't know. And he says, no, no, no, it is about the concentration camps. And God says, well, that's not funny. And the survivors said, I guess you had to be there. You know, so what an incredible level of dealing with something so complex, so undescribable and what families are going through now so many also black actors or playwrights who spoke to directors or making our wills because we are the families on the lines, we don't have health insurances, you know, what will happen to, you know, how do you really deal with this is what seemed to be a year and a half or two years ago unimaginable. So there is something I think in that experience as with so many other cultures and traditions, of course also, but especially in there, I think it's something that is worth to engage with because it's a deep, deep wisdoms of survivors of catastrophes in there. My question also for you, what will you be doing? What if we have projects, we say we're gonna travel or what is gonna happen in your artistic, in your life? What change will you focus on something different before or what's in store for you? This is a perfect note to end on. Something that Shane is responsible for that we're gonna be doing together. And something that could not be a more cheery way to get out of this, get out of the corona year. Go ahead, Shane. Yeah, we're getting ready to take Moshe Yassour's staging of Waiting for Godot in Yiddish to the Royal Dramatic Theatre in Stockholm this November. We're pretty excited about it. It's the first time that the Royal Dramatic Theatre is presenting a show in Yiddish, maybe anything in Yiddish, that's what the Swedes tell me. And so we'll play there and then we'll play one show in Malmö in the South. And I think that'll be a great cap to this whole thing for all of us. Because what could be more fun than watching Godot? Well, in Yiddish, it's a little different than it is in English. It comes across, even Alan, who doesn't necessarily like the play, saw Moshe's staging and said, this I like. Let me interrupt because let me think about the Godot since we're on that briefly before we wrap. It's Shane's translation of Godot and I've seen productions of Godot. I was in an excellent production of Godot in English at Tureva Theatre in New Jersey. It's just never been my favorite play for a combination of reasons. And then in Shane's translation and Moshe's production, I had been asked to be into the original and I passed on it because it just wasn't interesting. And then Yelena and I went to see it in the middle of that first production. In the middle of the first act, we turned to each other in the audience. We weren't eager to go because, you know, we both know the play sideways and it's not our favorite play. And in the middle of the first act, we turned to each other spontaneously and thought, this is terrific. Yeah, they interrupted the show, but it was okay. What Shane and Moshe did, and this is profound, profoundly important, not only for Godot, but for theater period, I believe, is so often Godot has layered on it levels of intellectuality and distancing from the audience and pretense that takes it away from the human essence and what Shane's wonderful Yiddish, which is better than Godot's English, I mean that. And Moshe's smart, straightforward direction did was stripped all that away and we were able to look at these people.  What are we waiting for? What is it about? And that was the genius of that production and I wish all theater of every kind could do that, could cut through the unnecessary and get to the human. Yeah, incredible. I missed it when it played here and we all know that also through Beckett, it is clearly referenced, originated from the experience of the Holocaust. Yeah, by the way, I meant to mention, we later joined the production and we did it here in Greenwich House but we took the play to the International Samuel Beckett Festival in Ireland and we did 10 or 11 performances there. There were almost no Yiddish speakers there. It was in Yiddish with the English translation, super titles. And I cannot tell you how many people came to us and they all said the exact same thing. I have seen Godot many times. I took a word of Yiddish but this is the first time I understood the play. This is my first time. Yeah. Beckett would be turning in his grave that people thought they understood the play but it's true. That's maybe the greatest compliment that I've gotten for that translation or involvement with the production is Irish people saying, it's the first time I understood Godot. Amazing. This is a play, of course, that somehow comes up in these catastrophic times. Susan Sontag famously did it on the crisis of the Balkans. I think Paul Chen staged it because it was the theatre column after Katarina in New Orleans. It's a fantastic audio production where he refused to do inside the theatre. People said, no one will come. Thousands came for those two weekends and what you did there certainly is remarkable. It's important wonders why it's not touring, why it's not showing and it's okay to have subtitles. Why not? We see so many films now. It should also be in theatres, the Yiddish theatre, the National Yiddish Theatre in Romania, which is actually across the National Theatre in Romania. I may be one actor as a laughter who is really a tullish courting water Yiddish actor but they're Romanian actors who learn Yiddish and audience still come and enjoy the plays. It's an incredible phenomenon, what's going on there why not here also in the city? It's such a big tradition and heritage. You know, why are your companies struggling so hard? Why is it so tough to find places and why is it easier to get a recognition in Sweden or on Ireland for work? That is of significance and also connected to New York history and to world drama. So amazing to think that perhaps of course someone will email us, no, there was another production but as far as I know, the serious first series online Yiddish Theatre production happened on Zoom in this year of Corona and questioned everything as Meena Rao said in our talks, you know, questioned everything, we do everything and then go on to what you did and I think this is what you guys did. So thank you for that little update on who you are, what you're doing, the theatre if you keep a life and you put it into the centre of your existence on Earth is the theatre and the art is a way of living. It's perhaps also a big provocation and when you sit at home, we wait for a lot of theatre. People, nothing has changed so much. A lot of theatre people never had jobs. They never have a theatre to play anyway. They don't, you know, so what is the very big difference, you know, because maybe less jobs outside to support it, you know, but it seemed like a lot of theatre already lived under conditions of Corona for a very long time without realising it. So thank you all for joining us. We could go on much, much longer, watch a Yelena and Ellen in the Cone Brother movie as we get a bit more, maybe try to get a copy of the DVD, the Beckett and maybe one theatre, we'll put it on here in New York and invite you after this talk. It certainly is of importance and we go on this week with our serial talks. We have David Godhardt who ran the Great Riverside Studios in London at the time. And the Riverside Studio was the Riverside Studio. Beckett rehearsed there. Giacometti was there. It was the Giacosti filmmakers come to a rehearsal there. An amazing place, something that, from what I heard and so also, you know, something unbridled, even unmatched. So it is quite an extraordinary, I think, hopefully, again, talk that we will have. So join us, be with us, and you guys, thank you for taking the time. Thanks for howling around, for hosting us. And to all our listeners, thank you for taking the time out of your busy lives. So much more talks are out there now. It is important, of course, also as an archive, I think for the future, we are archiving the moment in the history here. So, but still, thank you so much for taking the time and for being with us. And tomorrow we have the great Karen Martin. We mentioned her today is the idea of the theater of the real, the question, what is it? She is a field, she co-created, she wrote about the idea of the documentation of work, documentary theater, a theater that engages in many forms. So this is the real, and we will hear from her and tomorrow about her work, her research and what she thinks has changed in the time of Corona or what has been confirmed, what she has been writing about and what might happen. So this is important and she will join us again with Emily Mann next week, who will talk about her work. We're gonna have an update from the Edinburgh Festival. And so we will hear from around the world, from here from a Turkish producer in Berlin, they did an artwork on the balconies. Artists in the Prenzlauer Berg neighborhood, some of them quite significantly important who lived close to each other, created work on their balconies outside and it was really successful. People loved it, thousands of people watched it. So let's see what other cities did in the world during this time and like you who put together this production. So thank you all and I hope you will join us again if you need great theater, great artwork, but we also need great audiences and it's important to listen and this is what this is all about, a radical listening and it's important what the artists have to say. So thank you guys and I hope to see you in person soon. Bye bye.


So everyone, welcome back to another OpenShift Commons. You can see I've changed my background and this is special for today because we have talking storage. Many of you don't know, but those that are here from Red Hat may have seen a lot of our storage discussions and you spent a lot of time with me here in my dining room, so welcome back. And we are also changing it up today. Today we have an upstream discussion. I'm always excited about those. It's so important that we work upstream with the CNCF and today we have members from Tags Storage talking about a topic that is very important to pretty much all of us, especially enterprise installations and anybody that is concerned with how to get your data back or your applications back if something happens. So today we have Alex Cherkopp from Storage OS. He is the CEO, but also he is the CNCF co-chair of Tags Storage and Tags stands for Technical Advisory Group now. It has been changed from SIG and also Rafael Sarkoli. Who is a tech lead for Tags Storage. So very excited to have you both. Thank you so much for joining us. And Rafael, Alex, if you'd like to tell a little bit more about what you do, especially with Tags Storage and then dive right in. Absolutely. Go first. Thank you so much, Kareen. And so glad to be here with the OpenShift community. So the CNCF a couple of years back had created SIGs and SIG Storage in fact was one of the first SIGs that we created. And the purpose was to help the CNCF with evaluating projects and to create content and educate and educational material for the end users and the community. And since then the SIG has been renamed to Tag just because SIGs were getting confused with the Kubernetes SIGs. And as Kareena mentioned, Tags stands for Technical Advisory Group. So the co-chair of the group together with Xinyang and Quinton Hull. And we also have a number of tech leads of which Rafael is one of our newest members. So I'd like to pass the baton on to Rafael to introduce himself to you. Thank you, Alex. Yes, I work at Red Hat as an architect in consulting so help customer build their clownative solutions and recently joined the Tags Storage to work together on the topic that you see on the screen today which is a clownative disaster recovery. This is a work that I think started about nine, a collaboration that started nine months ago. I think it was fall of last year when we started talking about if this was a good idea or not to explore in this particular tag, right? Or it should belong somewhere else. And then we started sharing some ideas and we are now creating a white paper that I think we're gonna publish soon. And today we're gonna present some of the results of this white paper. Indeed. So this is particularly exciting for the tag because we've had a lot of demand and in fact a lot of feedback in the different tag calls which Rafael has been very patient with and also it's meant that we've been able to iterate on the document. And just for everybody's benefit when we talk about storage in the CNCF and persistence we're not just talking about a volume or a file system that we cover kind of any type of persistence layer including databases and object stores and key value stores for example as well as traditional volumes and file systems. And I think the key thing here and keep me honest here Rafael I think what we're trying to do is to make sure that developers and DevOps teams and the series et cetera have the information and the tools and examples on how to adopt some of these different technologies in a cloud native in a cloud native way because for the first time in a long time it's no longer infrastructure teams that are making these decisions but it's developers and their deployments and understanding the storage subsystems and the different technologies that are available with cloud native technologies is extremely exciting and enables so many new use cases which Rafael will cover shortly. Yeah, that's exactly right. I totally agree. And what we would like to do today is to present an approach to disaster recovery that we call cloud native. Obviously it focuses on stateful workloads and that's the storage workload that Alex was mentioning all of them. It turns out that the hard problem that you have to solve when you distribute a stateful workload is always the same regardless of the kind of interface that you expose. So it doesn't really matter if you are exposing block storage and a block storage interface or a message queuing or a SQL database, the internal sync is really the hard problem that you need to solve and that's what we explore with this white paper. But from a user perspective, the concept that we would like to make people aware is this new concept of cloud native disaster recovery. This is, we call it cloud native disaster recovery. I just said new concept, but the approach, it's something that you could have done even in the past. Our point is really that with cloud native approaches, it now becomes less complex and more easy to create these architectures and this deployment and probably less expensive. And the way we define this cloud native disaster recovery is by contrasting it with traditional disaster recovery. We had an internal discussion in the tag whether calling this traditional disaster recovery was correct or not. By traditional disaster recovery, we mean what you normally would find in many enterprise customers. So not the big web scalers, not the newer startups, but the enterprise customer that many of us work with. So let's go down these columns and rows one by one. I'm going to try to keep it brief because we don't have a lot of time, but and create this contrast and talk about what cloud native DR is. Type of deployment. So are we deploying active active or are we deploying active passive? In most traditional DR scenarios, what we see is an active passive deployment, especially for the state workload. You know, you may have state list workloads that are active active, but they all point to a single active site and then there is a passive site for the stateful part. That's often what you see. For cloud native DR, we are proposing that it should be active active. And then obviously we're talking disaster. So there is a disaster situation, how do we detect it? With traditional DR, in most cases, it's a human decision. Somebody who says, okay, this is really a disaster. We need to trigger the disaster procedure, recovery procedure. It's a human decision. In cloud native DR, we want it to be autonomous. We want the system to realize something is going wrong and react to it. And then the recovery procedure itself, what we see is normally a mix of manual and automated tasks. Maybe the tasks are mostly automated, but somebody is triggering them in a manual way. That is at least in my experience what I see the most. In cloud native DR, we want full automation on the recovery. Whatever the stateful workload needs to do to reset itself or reorganize itself and the replicas and the petition to continue servicing. And then the two metrics of DR are RTO, the recovery time objective, which is how long the system is down and how long, how soon we can get service up and running. And recovery point objective, which is a measure of how much data is lost because of the disaster. And it can also be a measure of how much inconsistency I have created if I have multiple copies of the data. So for RTO, for traditional, we usually have good enterprises that are close to zero, but normally we have hours to recover from a disaster. In cloud native DR, we want this to be close to zero. Essentially, a couple of health checks have to fail and then we realize the situation is a disaster and the system will start bringing the traffic to the healthy locations. And for RPO, again, you could close exactly zero to, again, hours of loss. I usually see, or there are minutes to hours in terms of loss, data loss. In cloud native DR, you have two options here. You can do strongly consistent deployments, which will have exactly zero data loss. And you can do eventually consistent deployments which may have close to zero data loss in normal circumstances, but you're not guaranteed in this situation that the data, once it's reconciled, is exactly correct. It's gonna be consistent, but it may not be correct for your application. So that's a caveat of the eventually consistent systems. And then from an organizational perspective, what we notice is that in traditional enterprises, application teams are normally responsible for the business continuity plan of their service. But what they do is they turn around to the storage team and ask them what are the RTO and RPO that you can guarantee for the storage that we use and then they adopt that as their standard. So the fact though is the storage team is the driver of the enterprise DR strategy. And when instead we are arguing here that with cloud native DR, it's gonna be the application team that will have to choose a correct piece of middleware, a correct product to handle their state, right? And then they will use that product to organize the DR procedure around it. And then one other observation that came up working with this new technology is that DR, for traditional enterprise, it's really the capability that you need to do disaster recovery and we instantiate this procedure are coming normally from storage in the shape of maybe the cap restore or volume replication, whether synchronous or asynchronous volume replication. Instead for cloud native DR, we noticed that this capability are coming more from network again. We see the need for East-West communication between your payroll domains, so that could be regions or could be your data centers so that this middleware that is new generation middleware can, the instances can cluster up, can find each other and cluster up. And then we see the need of global advances in front of the regions or in front of the data centers which we need a way to direct the traffic to the healthy locations. So it needs to be a smart, intelligent global advance. Hopefully we'll give you an idea here of the differences between what we call traditional and cloud native DR. Go ahead, Alex. I was just gonna say one of the key things here is what we're talking about, I think. And again, keep me honest here, Rafael, is we in a climbing to worlds where applications are effectively composable and the infrastructure is declarative. What we're talking about is cloud natives gives you a lot of the tools that are available to automate and to manage disaster recovery just like any other healing process that would take place in a standard cloud native environment. And so we kind of acknowledge the fact that this isn't necessarily straightforward. Some of these technologies are also fairly advanced but the point is that these new cloud native architectures and Rafael, we'll talk about a reference architecture with some of the options available actually enable organizations to have automated failover and automated disaster recovery processes with better metrics in terms of RTO and RPO than you'd get with the obvious manual failovers and manual tasks that you'd have in a traditional system which I think is extremely exciting because what we're effectively saying is we've made applications composable and developers can declare what their applications need from an environment and now we're kind of taking a step further and saying that that also applies to the disaster recovery process. Right, exactly. And it is exciting. I find it very exciting and maybe some of you may doubt that it's even possible or may say the story that I've been told many times is I can reach RTO and RPO as close as zero as I want but the cost increases exponentially. We don't think that this is true anymore. It's actually a matter of composing the architecture in the right way but the cost does not increase exponentially and you can actually reach these numbers with relatively inexpensive deployment. And I use the word relatively but certainly not something that grows exponentially. And so talking about how we can build these architectures, here we are showing a strongly consistent climate where we have a state for workload that is capable of handling this horizontal state sink in the proper way, guaranteeing that there are correct replicas in the correct regions or data centers. We need three failure domains, in this case they're data centers, right? We need three of them because otherwise we wouldn't reach the quorum. And as I said in front of the state for workload, normally we have a front end, probably a state that's front end and in front of it we will have a global load balance. So this is a very generic blueprint that you can then reuse in several situations. I failed to mention obviously the state workload, we'll have storage that comes from the local data centers but as you can see, we don't rely on storage capabilities to replicate across this data center. All the interaction is handled by the middle by the state of workload. So how can we build this state for workloads? Because obviously we are now relying more on the middleware, right on the middleware side. I have a couple of slides to talk about this from a conceptual standpoint. I'm gonna try to go fast here, so maybe we have time for exploring one of such deployments. So we need to define the concept of failure domain, understanding the concept of failure domain. It's a failure domain is an area of our system that can go down or can fail due to a single event. So you could have nodes, nodes could be failure domain, racks, clusters, network zones, availability zones, regions data center, they're all failure domains and if you look at them, they contain each other, you can scale out this failure domain from the smaller one like a single server, for example, up to an entire data center. But the theory around distributed state for workload works the same regardless of the scale of the failure domain. Here in this conversation, we are talking about disaster recovery. So we are talking about failure domain being the data center because that's what usually disaster means. I lose an entire data center, right? There is an event and because of that event, I lose the entire data center. So we're talking about our failure domain or references that are the center. And normally, so for disaster, we mean that we lose the entire failure domain or in particular the entire data center if we don't specify what the domain is. And the disaster recovery is what happens when that happens, what is my strategy? Have a bit, it's slightly different concept. I have a bit is when within a failure domain, something breaks, one, maybe I have one fault. What happens inside the failure domain too? Does the service continue or not? And then we have the concept of consistency. That is the property of a distributed workload that all the instances observe the same state. So the state is consistent across the instances. We need that because when we lose a failure domain, we're probably going to lose some of these instances and we need the state to have been synced everywhere so we don't lose it, okay? Yeah, and I think it's fine if you go into the next slide. On that point, the getting strong consistency is probably the single biggest architectural challenge, trying to ensure you have strong consistency which is one of those key attributes in any storage layer or database layer, et cetera, is effectively a balancing act between maximizing latency or even reducing latency and versus availability. And there's a very convenient capterium here which is one of those things where you can have three things, you have consistency availability and partitioning and you can have any need to pick two basically. And I'll let, I don't steal Raphael's thunder, but I'll let him explain some of the details and how it applies to some of the different systems we're talking about. Yeah, thank you, Alex, yeah. And yeah, you announced the theorem correctly, right? It's usually you pick two of these three, consistency availability and network partitioning, I should improve the picture here. I like to say the theorem is slightly differently because I think it helps understand the kind of choices that are made in the software today, which is network partitioning is not something that you control, it's a fault that happens. So a way of reasoning about the capterium is, assuming network partitioning, what do you want your software to do? Do you want it to be consistent or do you want it to be available, right? Because you can only choose pick another one, the network partitioning is not something that you pick, it just happens. And I'm showing in this little table here some of the common software today that where the choice is very clear, right? If you choose consistency, it means you're building a strongly consistent system. If you choose availability, it means you're building an eventually consistent system. Both at Pro and Cons, right? I have their usages, but that's a very clear design choice that you have to make when you build software. And in reality, some of this system can even be tuned, and depending on how you tune it, they can change the behavior from being available to being consistent, but they are all built around this capterium. There is a corollary to be kept in mind about the cap, from the capterium, which is the Pachelk or Pachelk, I hope I pronounced it correctly, corollary, which essentially means it says, in absence of a network partition, so when the network partition is not there, you can only optimize for latency or for consistency. So this is, I was recently doing some experiment and it was very clear what the Pachelk corollary means, in particular, I was doing for Kafka, right? So if Kafka is one of those that is tunable, if you set Kafka to be consistent and then you spread the cluster across highly, the reason that I have high latency, you get a very high latency on the response for a single communication, whether you are reading a message or actually producing a message, reading is always easier. And that's just the nature of this software work. That doesn't mean that, for example, with Kafka, you can't have still a significant amount of throughput, but each individual transaction will have a significant, high latency because you have told Kafka you want to be consistent. Okay, so that's how this work, and it's very nice. It's incredibly convenient to have a theorem to think about this thing because you can take a software that you don't understand, a piece of software that you don't understand and you can ask the vendor or whoever is the expert to talk about the software in light of the theorem and understand the choices that are made and that are made by that piece of software and just that way you will understand a lot of how that piece of software behaves. Alex, were you going to say something? I was just going to say, just for further information, we also have, the tag has created a more generic storage landscape white paper and in that we define all of the different attributes, like availability and scalability and performance and durability as well as consistency. And it's interesting to me because what we have here is different systems will have different use cases and it's probably worth noting that no one system will handle all of these cases because of course, very high or very strong consistency also typically has, might have scalability or performance implications and vice versa, for example. Right, it's a trade-off that you have to decide on your product. So we said we have several instances of this software, right, running and clustering up, creating a logical instance. How do they sync? We need consensus protocols. I'm going to go quicker in this slide, but there are two types of consensus protocol that they call share state and then share state. Share state is to agree on a state that all of the instances need to reflect, need to have. For these kind of protocols, you can have a protocol based on a leader election, that is the one that accepts all the rights and the other one are just followers and they're validated by the community, the algorithm that are being validated by the academy are Paxus and Raft. In this space, Raft becoming more and more popular. So most of the software that we have mentioned so far and we have another list later, they're all based either around Paxus or Raft when it comes to synchronizing the same state. Then sometimes you have to synchronize different states but you have to make sure that these instances have to agree on all writing or not persisting that state. For that we have the one known two-phase commit or three-phase commit protocols. Another thing to know about is like I was saying at the beginning that the hardest problem that this state full of storage software need to solve is really always the same and that is I need to sync with the other peer and then I need to persist that data. Somehow I need to have a consensus protocol, a list of operation that I've happened and then I have to store it to persist those information. There is an interesting chapter in the SRE book from Google where they explain how to theoretically build a piece of software that can be reused across all of these state full workload products because at the core it's always the same. Naturally you should do it that way you wouldn't get optimized performance. So you have to, this is a theoretical approach you have to still make your optimization out of it. But in reality, there are several companies for example that are using RocksDB or there is another one from Apache that there's a storage layer with some level of consensus protocol to coordinate instances. So if you put everything together, you need to have this reliably replicating machines and then you can create partitions so where you separate your data so that you can horizontally scales and then you can create replicas where the partition is replicated in other instances so you don't lose data when something goes down. And so if you look at this picture here on the right, this is the anatomy of a state full application at least a modern one where you have two replicas here. It's replicas of several partitions so we end up with six instances. Instances as its own storage and then we have the coordination layer. Between the instances with the same replica we use a shared state consensus protocol because we are all storing the same state. And then if this particular software supports transactions whereby I do two operations on two different partitions but I want them to be one logical transaction then I would have to have an inter-partition coordination protocol and for that I can use something like phase commit. So this gives you an idea of the anatomy of these stateful workloads that we can use to build what we call the clownative disaster recovery deployments. I think just on that point, Raffaele, I think one of the exciting things here is that what is effectively happening is we're layering different proven technologies. So you might have sharding for performance and you might have rough protocols for consistency but you might also have a variety of different layers in that stack where for example you might have a SQL layer that's using a key value store that's using a sharding process that's using a file system, et cetera. And so it is really more than ever before it's kind of important to understand the different layers because at the end of the day, the attributes of your system and your failover capabilities and your DR capabilities are going to be an amalgamation of all of those different attributes. Yeah, yeah, I agree. I could agree more especially on the observation you made on the interface layer what is called API layer here. We see more and more products now that offer for example a SQL interface and then also a key value store. And it's clear what's happening, right? They are just adding a new API layer but reusing everything that is below it. So it's relatively easy for them to do that. Maybe they don't always get all the optimization that they could possibly have but it's easy to add additional functionalities to your state or workload. Yep, exactly. Yeah, here is just a table representing some of the workloads that we have analyzed and the choice that they make in terms of the replica consensus protocol so the shared state consensus protocol and the unshared state or shared consensus protocol. Consensus protocol. I would highly recommend if you are considering a new state of workload, ask your vendor or your experts what choices do they make in this space? Because that already tells you a lot about the software that you're about to purchase. Some considerations around strong consistency versus eventual consistency. And they both can be approached for cloud native DR but they behave differently. For example, in terms of RPO, as we said, strong consistency is about consistency, obviously. So we don't lose any data once we create a well-done deployment. So it's exactly zero. I had people that couldn't believe it but it's exactly zero. I never lose data with this. Obviously assuming that only one disaster at a time happens but you don't lose data. With eventual consistency you may be losing some data, theoretically unbounded amount of data but in practicality if the system is not overloaded it's something close to zero because it's just what was in the local cache that the system didn't have time to replicate. Another thing that you should consider is when you lose one data center, one failure domain in an eventual consistent system, like I said before, the rest will keep serving. So the two sides of this deployment may diverge in terms of date and when they come up they don't necessarily agree with the state so there is a reconciliation algorithm that will decide who is right but this reconciliation algorithm may not reason the same way that your application from a business perspective reasons. What I like to say is that eventual consistency does not mean eventual correctness in business logic terms and eventual consistency may pose some design additional design consideration to your developers. If it is at all possible I personally prefer to keep things simple for the developers and choose a strongly consistent deployment. I think strongly consistent is the most predictable and one of the points here about the minimum number of failure domains is that with strongly consistent systems we effectively have an odd number of copies because you remember when we were talking about the CAPTIR and we were talking about partitions if a node is partitioned or is unavailable effectively the remainder of the system still has a majority of copies of the data and can therefore make an automated decision as to who is up and who is down which systems are authoritative for that consistent system whereas in eventually consistent environments it can be a little bit more complicated because effectively some of those decisions can be delegated to the application or some of those decisions could be delegated to reconciliation processes which are not perfect hence the reference to it doesn't mean eventual correctness. In terms of RTO both will react in a few seconds essentially depending on your health checks and the health checks that you set on the global load balancers but also some internal health check that the system has. In terms of latency consistent workloads have a strong sensitivity to latency between these failure domains that could be regions or data centers and by and large your latency will always be greater than true the worst latency between regions that you have multiplied by true because it's always back and forth so that sets your expectation and to me that says not all the use cases I cannot use strong consistency for all the use cases I will have use cases that need really fast responses where I can't use the system so I need to find different solutions but if it's acceptable if the latency that I have is acceptable like I said strong consistency keeps things predictable and simple for the developers. On the other side eventually consistent instead is not affected by latency because essentially the system first writes locally and returns and then it tries to synchronize with the rest of the instances so that does not affect the client latency that was a simplification but it's a way to explain why they are not really affected by their failure domain latency the throughput instead for both is can scale linearly as long as we have workloads that are touching all the partition so the requests are normally distributed and all the partitions are involved more or less in the same way this system scale horizontally sorry scale linearly with the number of instances so you want more throughput just add more instances increase the number of partitions and you get the throughput that you want and then as Alex said strong consistent as another constraint that some of our customers find taxing which is you need three failure domains in other words if the failure domain is the data center like we were saying you need three data centers and we perfectly know that some of our customers have some of the enterprises have two data centers and maybe in the same major area they would have very good latency between these data centers but they don't have a third one so what can they do at the same time instead of the central consistent workloads only need required to yeah so you know there are solutions to get the third data centers one option is to go to the cloud but that is certainly a constraint of strongly consistent system we wanted also to share reference architectures for Kubernetes deployment the first one that we share was very generic but obviously we are looking at Kubernetes with special attention here so not very different we still rely on the state workload to do the horizontal sync we still rely on persistent volumes provided by Kubernetes in this case and then we will have some ingress and a global load balancer in front of it that decides where the traffic goes so here when we lose one side essentially nothing happens the global load balancer should realize it and just send the traffic to the other ones another thing that you should ask yourself when you build this architecture is what happens if the clients can connect to my workloads but there is a network partition between some of the data centers and so one is isolated for strong consistent workloads this is essentially equivalent to losing this data center because all of these instances will become not active because they don't have quorum and so the global load balancers should realize that this side is not responding and send all the traffic to the remaining two so the behavior should be exactly the same we have also a reference architecture for events or consistency similar, right? except you just need two failure domain or two data centers here the conversation is slightly different when you lose well, again if you lose the entire data center the global load balancer can only send you to the remaining one and there is no real-state divergence because there are no rights on the data centers that you have lost different story when you when you lose connectivity between the state for workload but the clients can still write can still have a way to write in this case you can have divergence of the state and that's when we say that's the conversation we were having for this is this is a situation where at some point this fault will be corrected the connectivity will be reestablished the reconciliation algorithm will kick in but the result that you get as the final reconcile state is not necessarily aligned with what you need in your application okay we have some reference material here if you want to go a little bit deeper this is our white paper and some blog posts around building this architecture in practice and if we have time left we can explore one of these environments together so if you think you can do it in 10 minutes I'd love to I think we all would love to see you go into a demo environment but do you think 10 minutes is enough time to just be an exploration of what we have or do we have unless we have questions if we have questions I also would like to answer those questions I have a quick question going from what you were talking about last obviously one of the biggest considerations between strongly consistent and eventually consistent is the cost factor right so as you're working through this in tag storage are you looking at PCO or what do you have you explored that in tag storage overall I don't focus on cost because we try to be product agnostic so my only consideration is data center that may mean more that may be a significant cost depending on how you decide to implement that data center if you go to the cloud it's not actually a huge cost it depends on how much you deploy it's a pay as you go model but if you actually build a physical third data center data consideration is some companies may be running on software that is not capable of being deployed this way and so they may be facing a migration let's say you may have to migrate from MySQL to a new modern database that can actually be applied the way I was describing so this migration projects can create some cost in an enterprise I mean I kind of knew the answer but I was trying to tease out the strong consistency is obviously the best choice all right, dive in into your demo keep asking question I'm going to describe this environment so here in the meantime here I have a three clusters okay this represents my three regional data centers they are in different they are in Google and they are in different region as you can see you assist for your central and west here for example I have one of these clusters obviously I'm from Redats so I'm using OpenShift because it's easy for me but there's nothing in what we discuss that implies OpenShift you can do you can do everything with Kubernetes so here I have Kafka deployed this way you see three instances of Kafka but this is the second cluster three more instances these are all talking to each other and I have for example here a Kafka console in which I can see that I really have a if I go here I have a nine node Kafka cluster you can see from the name of the instances of this node that I have cluster three cluster one cluster two so all of the cluster OpenShift cluster all of the instances that are distributed in different OpenShift clusters come together to create a single logical Kafka instance if you notice this notation here let me make it a little bit bigger this is not the usual service name that you get inside of OpenShift this is a new standard for inter cluster of services and endpoints so it uses this cluster set notation and it also has the name of the cluster in which the pod is and so these are actually the pods that we were seeing before I think I have a Q topic defined here and if we look at this topic look at the partition it has nine partitions so it's well balanced on the available nodes of these clusters and it has I configure Kafka to be strongly consistent so each partition has three replicas and each of these replica is in a different region so if I lose a region one of these will become red but I still have two replicas and I can still continue working so that's how you can set up your workload in I also have in this experiment that I'm running I also have co-crustdb deployment so it gives you a UI where you can go and see what you have so here it has this nice feature where you can see your cluster how it distributes on a map so these are the three regions that I'm using from Google across North America these are the nodes also in this case I'm using I'm using nine nodes three in each regions and then each node within the region is in a different az so that I'm trying to get both local availability and global geographical availability and we can see another nice feature of co-crush that I like of co-crush is that it calculates the latency between the instances and then uses this information to do some internal optimization but as you can see this is obviously a symmetric matrix or mostly symmetric and from west to east is where we have the highest latency around 60 milliseconds so that tells us based on what I said before that latency that we can get from a transaction here once we have distributed the workload the data across the three regions the best latency is going to be around 120 something above 120 millisecond we'll get 120 milliseconds just from the network then there is processing, writing persisting and all of that so you can immediately start reasoning about what kind of workloads this database is so it's a trade-off your workload may go a little bit slower than if it was deployed in a single region but now you get that when you lose a region you essentially don't have to do anything the system will continue the service will continue to be up and the system will continue working that's it I think we don't have time to simulate a disaster and see how the system reacts we would see that here all of this system both Kafka and Cochrane can out-of-detect failures and they will start reacting to it we will see that we lose some nodes we will see that some of the ranges these are essentially the partitions are getting moved around and the system like I said we adjust to the new situation same thing does I think one of the key takeaways here is in a cloud-based world we now have the capability of implementing disaster recovery with different storage systems and different databases and different tools and in fact you get an order of magnitude better automation better flexibility than you do with your traditional systems and I think this is the next logical step for many enterprises as they adopt cloud-native technologies and Kubernetes and OpenShift and cloud-native storage solutions as they look to migrate more mature workloads and more mission-critical workloads that require disaster recovery so my key takeaway here is understand the different layers in your system understand the different attributes like the latency and the performance and consistency requirements of your applications and then yeah absolutely take advantage of the composability and the declarative nature of cloud-native disaster recovery and all the advantages that brings to your application and touch wood sleep better at night thank you thank you and we had a question come in that I think we'll have to take to if you came in from the LinkedIn event longer than a minute and it's a great question what considerations are required for cloud-native disaster recovery in a heterogeneous environment if either one of you want to take it in one minute otherwise we can push that to the LinkedIn chat to answer a quick go ahead Alex yeah I assume by heterogeneous we mean that we don't have an homogeneous cloud provider or infrastructure underneath right like I said this architecture relies on capabilities in the networking space so as long as we can do that it's West communication and we can discover the instances of our state workload on the remote failure domain and as long as we can set up some level of global balancing we will be able to create these architectures and in fact we are doing it and what collaborating with some of these vendors what we're noticing is how do you in order to be predictable in these deployments you would like all of the instances to behave the same but how do you for example provision the same IOPS across different cloud providers they all give this capability or this SLA in a different way or how do you provision the same computing power they are slightly different so those are the things that you may encounter but there is no actually blocker to building these architectures across the regions environment in fact and I'll just take one more second here I would kind of argue that tools like Kubernetes are actually designed to abstract your infrastructure and to give developers the capability of getting the same services from different systems and I think the glue that holds that together then is the east west networking and the load balancing services and things like that but that's it on top of that thank you both for taking that last minute question it's a very very good one and very important thank you again I will post the video recording and the slides at the LinkedIn group and then our LinkedIn event and then we will see you all next time next Tuesday we'll see if we have something scheduled but do you all then and thanks again Alex and Raphael this is a great important topic thanks for having us thanks for having us, thanks everyone bye bye


Hey people, so as you can see I'm at 9,999,223 ability tokens, which means I'm very close to 10 million tokens. And when I get 10 million tokens I can finally claim another badge after I think it's been like a hundred days since I've claimed a badge, probably about that much. So I'm gonna speed up until we're at like a hundred tokens away and then we will look at the badge I guess. Basically like there's too long of a gap between Christmas and Easter, so there's just a weekend, like a long weekend in. Oh, 10 tokens, 10 tokens! Hey guys! So after a bit of grinding we're eight tokens away, so one token and there we go. That's 10 million tokens, it took 132 days, 16 hours and 59 minutes to do it, but I have gotten the 10 million tokens, so now it is time to look at this little notification. I haven't actually seen that in main game, it's time to claim it, so currently my backpack is 3 billion 55 million, so when I collect this, 3, 2, 1, yeah! My backpack went up by 120 million just from that one badge, that's crazy. The quest badge will be even more, but yeah, that's 8 digits, very cool. I'll see you guys in the next video, thanks for watching.