Just what could the Smithsonian be hiding? Established in 1846, it's a tightly knit network of museums and research centers exclusively and uniquely funded by the United States government, nicknamed the Nations Attic, and for good reason. Made up of 19 museums, 9 research centers and even a zoo, it's the official resting place for over 154 million historically valuable items. With an annual budget of around $1.2 billion, two-thirds of which coming from annual federal appropriations, it's safe to say that if the institute needed to hide something, it would undoubtedly have the financial clout to do so. Through the years the meddling in which the institute has been reportedly involved in, can be seen as not only overwhelming but condemning of a hidden agenda. During our extensive research into alternative and controversial historical discoveries, we've often been confronted with such statements as, the Smithsonian people will be highly pleased to get their hands on this. Though unfortunately these sorts of condemning phrases have all but disappeared from mainstream media as the years have passed, they still do indeed exist within newspaper archives, stored within the libraries of earth, and thankfully there are many of them. As time has passed, reports of this involvement have become more and more elusive. This could be seen as a direct correlation with advancements within modern communications, the birth of the internet, along with many other forms of learning subsequently aiding in the distribution of said information, growing awareness of these reports exponentially. As a result, more in-depth and heightened understandings of evolution theory and the protection thereof, becomes more developed and entwined with such institutions. Profiteers from these lies become guardians of secrets which could destroy their status, clearly lending to the possibility and motive for a cover-up. Although reports which hit the internet in 2014 claimed a freedom of information act had revealed that the Smithsonian had covered up the remains of thousands of giants was eventually debunked. The flurry of attention it has created surrounding the topic, an allegation which we personally know to be accurate, has aided tremendously in the search for the truth surrounding these accusations. A source we highly recommend is a book by Richard J. Dewhurst titled The Ancient Giants Who Ruled America, the Missing Skeletons and the Great Smithsonian Cover-Up. It can not only be seen as a go-to resource for evidence of a race of ancient giants, but it also details the thousands of giant skeletons that have been found, particularly within the Mississippi Valley, as well as within the ruins of the giant cities over the past few centuries. It catalogs 400 years of excavations, newspaper articles, first-person accounts, state historical records and illustrated field report, including more than 100 rare corroborative photographs. It reveals that not only was North America once ruled by an advanced race of giants, but also that the Smithsonian has been actively suppressing this physical evidence for nearly 150 years. Dewhurst shows how this suppression began shortly after the Civil War and transformed into an outright cover-up. This being due to Major John Wesley Powell, who was appointed Smithsonian Director, a strict pro-evolutionist.  The evidence for which he claims, and with good reason, is hidden in the restricted access evidence rooms at the Smithsonian. The crystal skulls, a set of the world's most alluring artefacts, possessing the power to create religions, snaring many a Hollywood figure with their mysticism and rumoured possible alien origins. Firstly, how does one tell a real crystal skull from a fake? There are always artists capable of making and selling things that seem old, says anthropologist Jane McLaren Walsh of the Smithsonian Museum. And she should know, Walsh has seen her share of fakes. In fact, she's probably seen more crystal skulls than anyone else alive, subsequently becoming the leading academic on the subject. A stern skeptic with a ruthless ethic, only the most puzzling will convince Jane. Another major player in the skull game, according to Walsh, was Frederick Arthur Mitchell Hedges, an English stockbroker turned adventurer, who in 1943 began displaying a skull at dinner parties which he called the Skull of Doom. His daughter Anna later claimed that he had found the skull in a ruined temple in Belize during the early 1920s, however this was later found to have been a lie. Investigations by the Linnean Society of London, a research institute specialising in taxonomy and natural history, revealed that Mitchell Hedges actually purchased his skull at auction at Sotheby's in London in 1943. How it came to be at the auction house, however, was never established. Which is unfortunate because the Mitchell Hedges skull, according to Walsh's scrupulous examination, is the only one she has ever had to reluctantly confirm as an authentic crystal skull. What's more, it is the only academically accepted original known within the public archives. Smaller than other examples, which under microscope analysis were seen to have been made using rotary drills, the Mitchell Hedges skull is a more finely crafted, yet more crudely designed example, that under the atomic microscope has shown signs of having indeed been an ancient pre-Columbian artefact, which sure enough was constructed using quote, unknown technology. There are of course many examples of crystal skulls around the world, and many more stories surrounding their mysterious construction, elongated examples, stories of groups of these skulls initiating some form of energy field. Ancient laser cutting technology has also been claimed time and time again, however we felt we would approach them from another angle, to experience the rare occasion when modern, specifically funded academic institutes buckled to overwhelming evidence, proofs given by the defeated skeptic to those who pursue nothing but the perplexing truth and a direction for study. Made from a single piece of quartz crystal, Mitchell's skull of doom is unquestionably an exquisite example of an unknown history here upon our planet. Regardless of beliefs or indeed the superstitions which now surround them, there are a rare few which support the theory of lost civilization and ancient visitation. This skull is much smaller than many and crudely carved, leading museum scholars here to believe that in a world of fakes, this one is the real thing. We have in the past covered a vast array of evidence, which suggests the past existence of giants, yet alas, much of what is or has now either unfortunately been suppressed, destroyed, stolen or forgotten about. With the remains of their initial discoveries, now often only to be found remaining proverbially cast in stone in the form of the library archives of the world, and the news reports now digitally preserved within. Often follow-up reports abruptly ceased after the mention of the rapid arrival and insatiable interest of the Smithsonian, among others in said finds. However, now, thanks to the popularity of such subjects, the power and speed of modern technology such finds made during excavations involving a large array of individuals make modern cover-ups difficult and are rarely accomplished. With the only modern, almost openly admitted one of note, having followed the discovery of the supposed tomb of Osiris, when all media was immediately banned from the site. When permitted back, the tomb had already been penetrated and was subsequently claimed as having been found empty, supposedly previously looted, this regardless of its near impenetrability with Gantzenbrink only making it successful with modern robotics, but I digress. Working in cooperation, a team involving the Egypt's Supreme Council of Antiquities, a team from the Penn Museum, University of Pennsylvania, among others, discovered a sarcophagus academically claimed as having belonged to a, quote, King Sobektaheb, probably Sobektaheb, the first dated 1780 B.C. during the 13th dynasty. What we find astonishing regarding the find, however, is a sheer size. Carved from a single-corrid piece of aswan granite, initially weighing hundreds of tons, this finished tomb still weighs a minimum of 60 tons. It was somehow transported to the burial site and placed seemingly with delicacy where it now lay. This resting place in her chamber also some three meters in length. The baffling enigmas of why such size, how were they moved? To explain how these feats were accomplished is far less difficult challenge if one incorporates into their postulations the possibility that the size of these tombs were in fact made to measure, indeed, a match to the height and scale of the civilization who buried them. Could the inclusion of ancient giants into the many other theories surrounding the mysteries of Giza solve the puzzle we still can't solve of how these stones were moved? It is a hypothesis which we find very fitting. Again, in regards to this fascinating subject, we must remark upon the outstanding work of William R. Corleis and his collecting of remarkable tales of ancient giants. But what's more, his general conclusive consensus and in which, due to our own research, we also share. For example, he mentions on page 102 of his source book Strange Artifacts that, quote, the ancient people of most countries seem to have possessed in the strongest degree a faith of giantology, end quote, a feeling we also share having validated our suspicions. He goes on to mention a tale of the astonishing size of the statues, said to have represented the beings who once dwelled within the true ancient Egypt, quote. In front of the portals of the Palace of Karnak are gigantic human statues, and in one of the courts are 12 immense stone figures, each 52 feet high. The adjacent Palace of Luxor has two granite statues, 38 feet high, at the entrance. In the ruins near Thebes are three huge figures now thrown down, one being 64 feet high, and in the Palace of Parthenon of Athens many years before Christ, was a statue of Minerva, 36 feet high. The Temple of Olympia contained a seated statue of a god who rose almost to the ceiling, which would have made it some 68 feet high, end quote. He mentions that entering these places gives one the impression that they are entering the past dwelling place of giants, yet what he goes on to say was found is astounding. Gaius Plinius Secundus, called Plini the Elder, was a Roman author, naturalist, and natural philosopher, and naval and army commander of the early Roman Empire, and a friend of the Emperor Vespasia. He wrote the encyclopedic Naturalis Historia, which became an editorial model for encyclopedias. He states that after an earthquake in a Crete mountain, witnessed the complete intact remains of an ancient giant some 46 cubits long, or 60 to 70 feet tall. Beings of this scale would easily explain how such enormous stones were moved, yet how they could have been hidden from history is an unknown motive, which we find highly compelling.


The study found that there are significant gaps in the knowledge of hypertension and dyslipidemia in Saudi Arabia. Awareness, diagnosis, treatment, adherence, and control rates are all below optimal levels. Additionally, the lack of data on the prevalence of these conditions makes it difficult to accurately assess the burden of disease and plan appropriate interventions. This article was authored by Ashraf Amir, Mivata Lasnag, Raja Awadadi, and others.


Okay, good morning. Good morning to all our cameras. I'm Ron DeLumi, board member of AMAN Israel. We'll open this morning with a session about updates on current diagnosis and treatment. The session, we are honored by the presence of Dr. Sonia Siegman. Dr. Sonia Siegman is the head of the department of hematology in Amsterdam, VU Medical Center in Amsterdam. Dr. Siegman will guide us through how myeloma and other related diseases such as allameiodosis are currently diagnosed and treated. The session will last until 10 o'clock. We will leave time for questions and answers at the end of the session. Without further notice, please, Dr. Siegman, the floor is yours. Good morning. Well, thank you very much for this kind invitation. I'm from Amsterdam. This is our research building. I'm working in close collaboration with Hank Lokhorst and Niels van der Donk and Ingenayov and they will be happy to be later in future meetings with you here. I want to give a short update on the current diagnosis and the treatment of myeloma. Of course, I will leave amyloid to Professor Malini because he knows much more. I have some false slides as a muse and then Professor Malini will talk on that later. The things I will discuss are we diagnosing multiple myeloma earlier than before and therefore treating also earlier than before. I want to talk about the treatment of the younger patients who are transplant eligible, the treatment of the older patients who are in general not transplant eligible, what to do when the disease relapses and also a short notice on the novel developments but that will also be later in the program. So I will have only a few slides on that and a few slides on amyloidosis. So the diagnosis of multiple myeloma earlier than before. So there are now novel criteria for the diagnosis of multiple myeloma and they appeared in the end of 2014 and that leads to an earlier treatment. So there were patients with asymptomatic multiple myeloma whom we did not treat and now we start treatment earlier already. And that is of course to prevent the symptoms of especially bone disease. Probably you know this picture. It's one of the first patients that has been described in the literature in 1844 and you see here the devastating effects of bone disease and especially you and the doctors want to prevent the development of bone disease and you would like to treat earlier in time. On the other hand, when you do the oath as a doctor, the first thing you promise is to do no harm. So when you start treatment earlier, you all know that you will experience side effects of the treatment. So you should have a benefit from the treatment which is more than the side effects that occurs during the treatment. And what we did in the past, only when there were crab symptoms so when you had a high calcium, renal failure, anemia, bone disease, you started treatment of multiple myeloma. But actually from a Spanish investigator group, so the Petima led by Dr. Marivi Matheus and Dr. Jesusa Miguel, they started to treat patients with small ring myeloma, so patients without any symptoms. And they started to treat those patients with lanalidomide and dexamethasone and the other half of the patients were not treated. And what you can appreciate from these curves is that when you start treatment here, the blue line, you see that the progression of the disease is less than when you give no treatment and you would have expected that because when you start treatment, you will delay the development of the symptoms. So every time a patient develops a symptom, the line goes down and here less patients do develop symptoms than when you give no treatment. But the other even more important observation was that when you started treatment earlier, also more patients did survive. So the overall survival was also better. So you did not only delay the disease symptoms, but patients also survived longer. And that was actually very important because then the benefit-risk ratio is very positive. So then we asked ourselves should we start earlier treatment and in whom we should start because not every patient with small ring multiple myeloma will develop symptoms in short notice. There are also patients with small ring myeloma for years, not developing symptoms and you wouldn't like to treat these patients. So what actually is small ring myeloma? How we diagnose small ring myeloma? So whenever you have more plasma cells than 10% in the bone marrow or end, your end protein level is high and you had no crap, you would say this is small ring myeloma.  So what is the natural cause of the patients? So when you look to these patients with small ring myeloma, how the natural cause is? So here you see the patients with small ring myeloma and the risk of the chance to develop multiple myeloma. And what you can appreciate from this slide that in the first 5 years patients have 10% risk to develop multiple myeloma. But after 5 years the risk is less than it's 3% per year after 5 to 10 years and thereafter only 1% of the patients do develop multiple myeloma. So whenever you don't develop multiple myeloma in the first 5 years there's a, the risk to develop multiple myeloma later on is lower. So who are these patients? As a doctor could you predict which patients will develop multiple myeloma with symptoms short notice, in short time because these patients you would like to treat to prevent the development of symptoms. And actually what we did in the myeloma community we said whenever we are able to predict who will develop multiple myeloma in 2 years with a 70 to 80% risk those patients you would like to treat early. So when in the first 2 years 80, 70 to 80% of the patients would have developed multiple myeloma then we would say these patients you really should treat because the benefit-risk ratio is positive. You will have more benefit than risk. So there's several methods to now predict whom will develop multiple myeloma with a 70, 80% risk in 2 years. One is here you see the normal immunoglobulin. So every normal person has antibodies in his or her body in order to protect against infections. Patients with multiple myeloma you know probably have a monoclonal and protein. So only one type of immunoglobulin and the immunoglobulin consists of 2 heavy chains the orange bars and 2 light chains. So what you can see in patients with multiple myeloma that they have an M protein which is increased which sometimes is the whole M protein so both the heavy change and the light change but some of the patients have an additional production only of the light change and when you look to the light change you could have a look to the free light chain ratio so the kappa and the lapda ratio so because you have 2 types of light chains kappa and lapda and whenever the ratio is abnormal you know that these are monoclonal. Now when the ratio is more than 100 so the malignant chain either kappa or lapda is 100% higher than the normal chain and that occurs in 15% of the patients you have a 72% risk to develop symptoms in 2 years so now we don't only treat patients with crap symptoms but also patients without symptoms but with such a ratio of the kappa and lapda change we now start to treat those patients already before the development of symptoms that's also applicable to the patients when you do a bone marrow and you look how many malignant plasma cells are in the bone marrow the patients who have more than 60% of malignant plasma cells in their bone marrow so depicted here are the malignant plasma cells when they're more than 60% in the bone marrow 95% of the patients will develop symptoms in 2 years so whenever you perform a bone marrow and they're more than 60% of plasma cells and they're no crap symptoms now we start treatment already in those patients because almost all the patients will develop multiple myeloma in 2 years some of the symptoms are really devastating like bone disease or renal failure so we start treatment already well the other thing has to do with a football player we have a very famous football player in the Netherlands which is Johan Cruyff who happily deceased but he had very specific remarks and one of his statements was you only see it when you get it and I would say in multiple myeloma that might be too late and now it comes to the next slide because you probably all know that we have several techniques to investigate bone disease and when you investigate bone disease with a classical x-ray like here this is just a perfectly normal classical x-ray of a patient and as a doctor you would say to the patients you don't have bone disease nothing the matter but when you do an MRI of the same patient you can see here that there's a myeloma lesion so you would have said no bone disease when using classical x-rays but when you do an MRI you would say to the patient you have myeloma bone disease and when you perform either a CT scan which is the only the bony structures without the colors you could already observe here also a lesion which is the same lesion but when you also add a radiolabeled sugar you can even see that the lesion is also active so when you either perform an MRI or a CT scan your sensitivity to detect bone disease is much higher and when you have bone disease you would like to stop that and start treatment so that's the third reason now to already start treatment in patients without crap to the classical definition whenever you have an MRI not with one but with more than one focal lesion 70% of the patients don't have symptoms in two years so also these patients are now being treated and that brings me to the crap criteria so we don't only treat patients now with crap, with the classical symptoms so hypercosemia renal failure anemia, bone disease but we will also treat patients now with plasma cells more than 60% in the bone marrow MRI more than one lesion and an epilepsy ratio which is higher than 100 because these patients will develop symptoms in relatively short time there's one other thing I know that in many countries there's no access to the CT scan but actually we would favor to do CT scans in every patients who comes to the clinic because of the suspicion of multiple myeloma because when you look to the CT scan and you would observe an osteolithic lesion like this which you won't see with a classical x-ray you can already observe from here that it is very important to treat this patient because this is already maybe collapsing in several months and you really would like to prevent that so whenever there's an access to a CT scan we would like to do a CT scan in every patient with the suspicion of the diagnosis of multiple myeloma it doesn't need to be a PET CT scan a PET CT scan costs approximately 1200 euros maybe in some countries the price is less but approximately 1200 euros whereas just a plain CT scan is approximately 200 euros and it's also more convenient to the patients so when you have your classical x-ray some of you will know you'll have to do 12 photos which is rather complicated and a CT scan is being done in 2 minutes and in most countries it's not more expensive than just classical x-rays so we would favor to do a CT scan whenever you see one lesion of more than 5 mm you would start treatment already whenever you know everything I tell and you want to proceed faster please let me know so now we have patients who were diagnosed at small ring myeloma a part of these patients we now start treatment earlier because of the high plasma cell level FLC ratio or MRI so still there are patients with small ring myeloma and should we treat those patients and those patients we don't know yet so I think it's very important to promote young patients who participate in clinical trials and so we will know whether other patients with small ring myeloma with other characteristics than I showed here will also benefit from earlier treatment so and there are several clinical trials now going on in Europe in order to learn more about that so that about the new criteria which are now implemented in the clinical practice so the treatment of younger patients being transplant eligible the current standard of care in many parts of Europe and I know Europe is very diverse but when you would have had access to these drugs you would like to start with Bortezumib, a proteasome inhibitor which I think is necessary in all induction therapy for patients who are younger and transplant eligible plus either thalidomide or cyclophosphamide I think the French have shown that thalidomide is superior so you would like to give thalidomide and the Italian colleagues did a retrospective case analysis also showing that thalidomide is somewhat better than cyclophosphamide and dexamethasone and then you perform a stem cell transplantation but when we have the proteasome inhibitors available the Bortezumib and the imids the thalidomide the results were so good the complete remission rates were so high they were approximately the same with what we used to reach earlier with stem cell transplantation so then there came the question in the multiple myeloma society do we actually still need a etologist stem cell transplantation or could we only treat with the novel drugs well there's several trials I will show a lot of abbreviations I'm sorry for that but there are so many drugs so it's good to have many abbreviations because we have so many drugs available but V is Bortezumib that's because it's Valkate K is Carfilzumib Chiprolis, Ixazumibacy so these are proteasome inhibitors I'll come to that later so V, K, NEI, the antibodies Darvatumumup, Ielotusumup so you will read in many papers Mara Illo the H-deck inhibitors so this is Dutch which is Spanobinostat the immunomodulatory agents the imids which are thalidomide lanalidomide, pomalidomide and we have of course Melphalan which is still being used and very effective cyclophosphamide, prednisone and dexamethasone but I will repeat whenever necessary so is a stem cell transplantation Italian study either an autologist stem cell transplantation or cyclophosphamide lanalidomide dexamethasone all patients receive maintenance treatment and when you now look to the progression free survival so the duration before the diseases relapse that's much longer when you perform a stem cell transplantation 43 months versus 28 months when you don't give a stem cell transplantation so you postpone the relapse very pronouncedly by performing a autologist stem cell transplantation so not only novel agents but novel agents plus a stem cell transplantation and also when you look to overall survival we can't say exactly whether that's better there might be a trend that also the overall survival is better and we will do meta analysis of many trials which have been performed in Europe because such a similar trial has been done in France also a similar design with almost exactly the same progression free survival in the transplant arm and in the non-transplant arm so a stem cell transplantation is of added value we performed a study in the Netherlands showing exactly the same the benefit of a stem cell transplantation and also the English the MRC performed a trial comparing just novel agents with a stem cell transplantation and again a stem cell transplantation was of added value so a stem cell transplantation is required and now we are even asking ourselves maybe even too because we did an Italian a German an Italian Dutch trial which was an EMN trial and Dr. Zonnefeldt at the latest Ash and the ESCO presented the data and when you look now to the patients who got two stem cell transplantations so you got novel agents one stem cell transplantation a second stem cell transplantation and then maintenance therapy you see when you look to the high risk multiple myeloma patients and the high risk multiple myeloma patients are the patients with specific cytogenetic abnormalities in their chromosomes not because it's an inherited disease but during the course of your life you develop abnormalities in your chromosomes and then you develop multiple myeloma and the patients with a abnormality on chromosome 17 so a deletion they lose a part the translocation between chromosome 4 and 14 and the translocation between chromosome 14 and 16 so they stick together these three abnormalities we know is very high risk in multiple myeloma and we know the outcome of those patients is inferior as compared to the rest of the patients now when you look to the high risk patients defined as I just stated you can see that when you perform two stem cell transplantations the delay of the disease of the progression is much longer in the patients you transplant two times here you see the curve for two times so you start with 100% of patients and every time there's progression of the disease the line decreases and when you look to the patients who receive two stem cell transplantations as compared to one stem cell transplantation you see that the progression free survival is here 26 months and here 46 months so a 20 month delay with two stem cell transplantation of the recurrence of your disease that's almost two years so we are now currently writing in the Dutch guidelines that when there's a high risk patient with cytogenetic abnormalities we would perform two transplant this is a very early analysis so there are some doctors saying well we'll have to wait maybe a little bit longer before to decide on that so it's an individual balance and you will discuss with your physician whether it's required or not but in general we are really discussing and thinking of two transplants in high risk patients so what about the maintenance therapy so we were not used to give maintenance therapy after stem cell transplantation but that was actually because we hardly had any medication to give us a maintenance therapy in earlier times and I even wasn't at practice we gave malphalan as a maintenance therapy but it was very toxic and it didn't increase the overall survival and the progression free survival was a little bit superior sometimes but sometimes not but here you see the curse with lanylidomite and a low dose of lanylidomite and in it most of the patients will will be able to continue that for a very long time and they will experience complaints like feeling exhausted diarrhea but diarrhea we have some medication for that and it's specific for lanylidomite I'll come to that later but here you see the maintenance therapy with lanylidomite and again the progression free survival is much longer so half of the patients will experience a relapse after 51 months and you have no lanylidomite treatment half of the patients will have developed a progressive disease already after 28 months so again postponent of the progression of the disease and it's not only the English trial also the Italian trials the French trials the Dutch trials have shown similar data and when you combine all these trials you see that the progression free survival is better but also the overall survival is improved so you will live longer with your disease using maintenance therapy as compared to having no maintenance therapy the only hesitation we still have is in the high risk patients whether the overall survival is better but also in high risk patients you will postpone your relapse so now we know that it's difficult to have these drugs available in every country for maintenance therapy but the EMA the European Registration Authority did state that there's a registration for lanylidomite maintenance therapy so that about the younger patients what about the further developments that is the incorporation of antibodies in the first line of therapy and I will come to that later so then about the newly diagnosed patients with multiple myeloma who are elderly and are not transplant eligible it's difficult to state whether you're transplant eligible or not there's a doctor from the USA which is Bart Barlogy and who states everyone is transplant eligible I'm not convinced about that whenever you're over 75 the toxicity will be much much higher so patients over 75 you would certainly not give an autologous stem cell transplantation but 65 was a very limited limit so now we increase the limit to 70 years of age we look through the we have an eyeball test and we think well this is a patient who can receive a stem cell transplantation but there are patients who are not transplant eligible and there are two standards of care in Europe either Melfland, Prednisone and Bortesimip or Lanylidomite Dexamethazone what about the last regimen should you give it continuously or only 18 cycles I think that's very important in clinical practice for patients should you have your treatment all the time or are you able to have a treatment free interval which might be very important because then you won't experience the side effects of the treatment now when you look to the latest update of the first trial and the first trial investigated Lanylidomite Dexamethazone continuously only 18 cycles versus MPT and what you can appreciate from this slide is that the overall survival is better with Lanylidomite Dexamethazone versus MPT so when you have Landex available you would favor to treat patients with Lanylidomite Dexamethazone but when you look to the overall survival there is actually no difference in continuous therapy or 18 so in general you could say only give 18 cycles then you have a treatment free interval you will have a treatment free a treatment holiday they say sometimes and you will have similar overall survival the only thing I would like to say is that when you have a very good partial remission or a complete response with this regimen then your next treatment will only be in 6 years when you continue Lanylidomite Dexamethazone so in many patients I see who have a very good partial response or a complete remission on Landex I would continue because these patients do very well on the treatment and they will have their next treatment only 6 years later so suppose you have a patient in front of you who is 76 his next treatment will only be when he is 82 and the next treatment will be a free drug regimen with many side effects so I would in general give 18 cycles patients with a very good partial response or complete remission I would continue the therapy is it possible also to give it in the very frail patients because many patients are older than 70 and even 80 when they are diagnosed with multiple myeloma and these patients mostly not included in the clinical trials so we don't know how these patients are experiencing the treatment so first what is frail we know actually frail is not age you have this is a very nice documentary which is the autumn goals and it's about the olympics for very elderly patients and this is in the category over 80 I don't know whether I will do that when I'm over 80 but of course we know we have also the over 80s who are like this and I heard Céry Loulin which is a hematologist from France once say and so this is blackade there are 50 shades of gray so it's difficult to say who is frail and who is not frail so we use now in general practice the frailty index so we look to age because over 80 really negatively impact the outcome we know that we look to the activities or daily living can patients dress themselves, bath themselves we look also to the instrumental activities of daily living which is can they handle their telephone can they do their finances and to the co-mobilities and the group of Antonio Palumbo in Italy has made a frailty index and it's not on this slide but it's very easy to do it online I can provide you the link later on so your physician can just answer all those questions then you get the frailty index and we know that the patients who have a frailty index of 2 or more we really should adapt the treatment and when they have a frailty index when they're unfit having a score of 1 you also should adapt the treatment so it's very easy to use in clinical practice and I'm quite interested in trying to predict frailty even more better so what we are also doing in the Netherlands is that we look to sarcopenia so to the muscle fat ratio because then when you have less muscle you might have more side effects because the pharmacokinetics of the drug is very different and we also look to markers in the skin that reflect biological age so we try to predict frailty more better even than with the frailty index but coming back to the French trial and MPT is it also feasible in the frail well we know that approximately 50% of the patients in the study were frails so quite a lot in this study this is not the best slide but when you look here you see the fit patients here the unfit patients here and the frail patients here you still see that the green line which is RD is in all the patients is better than MPT so whenever you can treat a patient who is frail with RD you would favor to give RD but when you look here to the fit patients the unfit patients the blue line and the white line we still have some steps to make for the frail patients because the outcome for those patients is less you see that their overall survival is less as compared to the fit patients but you can use it Lendex also in the unfit frail patients so yes better than MPT but less effective than in the fit patients so we need to do more we in the Netherlands also look to MPV the other standard of care in the Netherlands Melfel & Prenyzone Botasimip and we only looked in unfit and frail patients so 24% of the patients were unfit 76% of the patients were really frail and what we saw in these patients is that almost half of the frail patients half of these have to discontinue the therapy the first nine cycles so it's very difficult to treat frail patients but when you look to six cycles that was possible to give in 70% of the patients and their overall response rate was similar as compared to nine cycles so what in general will be done I think in clinical practice whenever you have a frail patient or unfit patients give a shorter induction therapy so only six cycles five cycles for all we don't know but give less and then continue with maintenance therapy because then you will have more effect shortly about the diseases relapse in general and that will be discussed in the afternoon please consider a second transplant if the first was very effective because then you have a progression free survival which is of benefit and in general you give lanylidomite when you were treated with lanylidomite switch to botasimip and vice versa there are several novel treatments the novel proteasominibitus this is napels when they didn't collect the rubbish and when you look to the persons who were living in napels at that moment you see that there was a lot of stress at that moment and that's actually the working mechanism of a proteasominibitor because when you look to a proteasominibitor also in the cell there's a rubbish bin and the rubbish bin is the proteasome and when you block the rubbish bin in your cell your cell becomes stressed and then your cell will die and that is what the proteasome inhibitor does actually so botasimip can do that only reversible so when you gave botasimip it will block the proteasome so there's rubbish in the cell the cell becomes stressed and will die but it goes on and off and when you have carfilzomip it will do that continuously carfilzomip which is even a more potent proteasome inhibitor as compared to botasimip we have the monoclonal antibodies and it will be discussed later on in the morning and in the afternoon which are daratumumup and elutuzumup and what you see here with daratumumup you see here the myeloma cell which see the 38 on it and when you give a monoclonal antibody like here is the monoclonal antibody against daratumumup it attracts several effector cells and these are cells that are killing the myeloma cell so the monoclonal antibody does something in itself it can lead to cell death directly but it also gets some friends to help to kill the plasma cell so that's the working mechanism of daratumumup in multiple myeloma and elutuzumup does something different it also directly kills the myeloma cell but it also activates the natural killer cell well natural killer cell it's all in the name the natural killer cell will be activated and also kill the myeloma cell so with daratumumup you have friends helping to kill the myeloma cell and here you have also a friend which is the natural killer cell which is activated by elutuzumup and then the myeloma cell will be killed this is the HDEC inhibitor and I already see that I'm talking too long so this is also kind of a rubbish bin blocker because when you block the rubbish bin which is the proteasome and you also give panobinostat you also block the second rubbish bin which is the agregorosome and when you block them both it will be more helpful so panobinostat is also a drug you can use and I think the landscape totally changes these novel drugs because whenever I stayed when you were treated with lanalidomide you give proteasomib and vice versa now you have novel combinations and you can see that the delay of progression of the disease is much better than only with two drugs so suppose you have been treated with lanalidomide before for a long period of time because of maintenance treatment and you develop a relapse normally proteasomib with a progression free survival of approximately 10 months so after 10 months your disease will reappear when you look now carfilzomib the irreversible rubbish bin blocker you see that the progression free survival is almost doubled so much better and when you combine proteasomib with other drugs like panobinostat telotuzumib the progression free survival was also better in these studies and especially I think the use of daratumib will really change the landscape because the progression free survival has not been reached but here you see a number which is the hazard ratio what means that you delay the progression of the disease with 100 minus 38 percent so with 60 percent 61 percent delay of the disease so these drugs will really change the landscape of the treatment of multiple myeloma there's one issue and I think you will discuss that these drugs are very very expensive and I think we really have to work together to get these drugs accessible in all parts of Europe and not only in the rich countries and even in the richer European countries it is a problem to have these available because it's so expensive but also in countries like the Netherlands daratumimab has been placed in the lock by the minister of health so we don't have it available in combination only in later lines but we really should put an effort together doctors and patients and you in order to get it accessible because this will really change the landscape of patients with multiple myeloma well here you see it for the patients who have been treated with botasomib so botasomib then you have relapse when you only give lenelida my dexamethasone progression free survival of 15 months when you combine it with these drugs it's much better again with daratumamab not even reached and you see that also in the patients with high risk which I commanded on earlier the progression is being improved with the addition of several drugs so also here it is much better than before is there something simple because patients also want to have a pill instead of getting infusions two times a week etc I would like to show the data of something which is which is very easy to give and very very cheap and we developed that actually Ingenayov and Nielsen the Donk in the Netherlands in our institution they observed that patients who were treated with lenelida myte and they were progressing on lenelida myte so they were refractory and when you give only one tablet of cyclophosphamide which is very easy to give once a day hardly any side effects it is very very very cheap when you add it to the lenelida myte and we call it Rep lenelida myte endoxan so cyclophosphamide prednisone that 56% of the patients responded again for more than 12 months and when you compare it to the other regimens I showed which are rather expensive I think this is really a good drug to give in every part of the world and we don't know whether you should combine it with lenelida myte probably it does because these results are much better than giving only cyclophosphamide but I thought it would be nice to show that here actually when you compare this 65% of response 30% of patients who respond this is much better much better not head to head compared so it's not good for me to say that it's head to head compared but when you look at it this is a much more expensive regimen 30% of patients and duration of the response of 8 months and think when you have pay for performance models it is very nice because you only pay for the 30% of patients who have benefit and they will benefit for 8 months so that's very worthwhile so thinking about novel models in order to get it funded this is the last slide and then I think I will leave amyloid just here after to professor Malini because he knows much more and then we have some time for questions what about individualized therapy because now we treat all the patients in the similar way maybe not only for high risk it's a little bit different and like I said there are two transportation but this is something else which we can use and I think what will be increasingly implemented in the clinical practice is the depth of the response and I told you we look to very good partial response and we look to complete a remission we did look at stringent complete remission so we looked at the FLC ratio but still we know when we don't see anything by the eye in the bone marrow or by the free by chain ratio there must be some disease because the disease recurs, relapses so there must be some disease so are there very precise methods to detect the limited disease which is available in the bone marrow and one of the possibilities to do so is to do effects analysis so immune for unotyping of the cells and you can find 1 in 10,000 1 in 100,000 cells with such an analysis what you do actually I showed the cell and I showed the CD38 expression on the cell but there are several markers on a cell which are present on a malignant plasma cell but not on a normal plasma cell so when you look to these characteristics with a immunofinotyping with labeled antibodies you can see 1 in 100,000 cells I forgot to implement a graph of that how it looks but the French study recently looked at patients who were treated with botasmic gland in the mydexamethasone they either got a stem cell transplantation or not and then they received maintenance therapy and they had this very precise method to detect minimal residual disease and when you look to the patients who have no minimal residual disease anymore there is the upper line and you look to the patients without progression you see that here these patients progressed much less as compared to the patients who were still MRD positive so MRD determination is a very prognostic marker what we don't know yet is should you now stop therapy in patients who are MRD negative or continue because they might really do well with the treatment you're given you should continue therapy and you should change the therapy in patients who are MRD positive so the next type of trials which are now being developed and are started in some countries in Europe already is to look at the MRD negativity and positivity and then randomize the MRD negative patients with either stopping or continue therapy and in the MRD positive patients either continue therapy or add something different and then we will know what to do with MRD negative patients and MRD positive patients but it certainly will change the landscape and the trials in the coming years it will have consequences for patients because this can only be done in the bone marrow so patients will have to say that they agree with having additional bone marrow because when you want to do this you should repeat your bone marrow after the treatment after 6 months, after 12 months because that's the only way we can learn so that's the problem I think with these type of trials but we really hope that with this we can individualize the treatment and make the world of multiple myeloma patients better and especially know whether we can have a treatment free interval in the MRD negative patients or not so I think it's really important so I will leave this in the talk hereafter I think that's much better because then we have 15 minutes left for questions you could understand thank you so much thank you I complete whatever Anita said because they extremely comprehensive but I would like to say something else I am grateful for your remarks regarding the cooperation for the reduction of medication because in front of sickness and death we are all equal and the medication should not be addressed only to the millionaires but I also have a question for you madam Doctor please you have mentioned the newly emerged recommendation during EHAR regarding the two stem cell transplantation you have also mentioned that for the young people the recommended period between the two is two three years what about the other people people over 50 between 50 and 65 who are also eligible to take to receive stem cell transplantation what would be the recommended period because this has not I did not hear it maybe and the second one you mentioned the benefits of 46 and 26 46 months after the first one 26 has it been established why a reduction of 20 months already are we talking about an overall benefit of 46 plus 26 all in general thank you very much I will show the slides you are commenting on I think the question you had about the two transplants sorry sorry it's here we started with induction therapy and then you do a randomization between either one or two transplants and the patients who you give only one transplant the disease will be suppressed for 28 26 months in the patients you will do two transplants approximately 3-6 months after the first transplant you do a second transplant the disease will be suppressed for 46 months so the patients who got only one transplant after a median of 28 months they will have a relapse of the disease and they will have another treatment your doctor will start another treatment finally I think what is very important to see whether the overall survival is longer because I don't see a flip over but what you see also with maintenance I think is very important because we had a lot of discussions with my patients because when you suppress the disease for a longer period of time and you live for approximately I say 6 years and you suppress the disease for 4 years and then you have 4 years of additional treatment and when you don't give maintenance and you have a treatment free interval and your diseases relapse after 2 years and you have 4 years of additional treatment but in the end you both live 6 years you would be able to omit maintenance therapy because your overall survival is similar and then you have a treatment free interval so overall survival is very important but for maintenance we show that the overall survival is better it would be better with a flip over what I'm saying now I think but maybe you understand so that's about the 20 months difference this is a very early analysis so I would like to wait the longer follow up of the analysis is this an answer to your question? yes thank you very much for the excellent update you already answered my first question so I have small questions for you when you're talking about transplantations you're always talking about אותологous transplantations right? are there any trials with unrelated transplant settings this is the first question the second is do we expect a sort of cure in the near future without transplantation? and what would be because most of the audience is patient advocates what would be your take home message to these people because there are too many different protocols and diverse landscape etc so what would be the take home message for the advocates thank you your first question about allergenic stem cell transplantations either from a family donor or from a match unrelated donor will be discussed later during the day by my colleague so what has been done with allergenic stem cell transplantation first we didn't have all these drugs available we did an allergenic stem cell transplantation very late in the course of the disease and we really know that that's not the time to do an allergenic transplantation because you only had the side effects and almost no benefit although of course there's several patients in the world who did benefit an allergenic stem cell transplantation at that time but when you take all the patients together in general there's no benefit in later lines of the disease what we did then is that we said okay in later lines of the disease it's not a way to go should we combine it in the young patients with an auto so we do first an autologist stem cell transplantations followed directly by an allergenic stem cell transplantation and there have been several studies performed in Europe especially and when you look to these studies the final answer I think is that it's not of benefit in general because the survival curves are similar so it's not a general approach but sometimes we as doctors when patients are approximately 40 years of age you might think about doing an allergenic stem cell transplantation at the first relapse not in the start so auto although so now we know the first relapse is that a place to go and I have regular discussions with my patients at home because when they are young and they only benefited from the autologist stem cell transplantation with novel agents for approximately 18 months you know that you have many drugs available but when someone is very young certainly his life will be shortened so then you think well should we give an allergenic stem cell transplantation at that time what I don't do is whenever there's the relapse already within 6 months after the autologist stem cell transplantation we looked at the patients we did an allergenic and all the patients sadly died so within 6 months the biology of the disease is so progressive that you won't save those patients with an allergenic between 6 months and 18 months I have very long discussions with my patients and sometimes we do and sometimes we don't and it depends on the age on the life whether they have children whether they have a job which they want to continue, whether they want to have the risk of dying in the first year which might be 10% with an allergenic or they say no no my children are 4 years of age I really want to see them grow older and I know it's limited but I don't want to die in the first year so it's very difficult discussion so I can't give a very clear answer not at late stage not fully at the start at first relapse discuss with your patients what is wise to do so that's about the allergenic the take home message I think yeah it's always difficult to take home messages I dislike a world in which we try to say everything in one or two lines and that's the trend that is increasingly being adapted in life because this is such a difficult subject this is so full of emotion this is very hard to state in a one liner but I really think I grew up in the time when there was only Melfelin actually available and it is completely different at the moment so I think this are much better times for patients with multiple myeloma although you really don't want to have multiple myeloma but whenever you're diagnosed with multiple myeloma these are much better times because you have so many possibilities to treat and therefore extend life hopefully also in a good quality of life I really think that there's an unmet need for the treatment of bone disease because it's really devastating so we should look in depth into bone disease in extra medullary disease which is very difficult but we have better outcomes for patients and I think the take home message then also is and I said it earlier already we would like to have access to all these novel drugs for all patients with multiple myeloma and we'll have to work together also with pharmaceutical industry in order to find models to get these drugs paid but times are much better and we are having much better times also in the outpatient clinic when you're discussing because there's a future for patients so that's I think very important yes I have a question which might be specifically relevant to the new and very expensive drugs how is the dosage decided and what kind of studies independent from the pharmaceutical companies are performed to decide the dosage to use myself as an example I get on a special program data tumor mob but that much lower dosage than recommended by the pharmaceutical company and it works very well so that of course also will bring down cost if you can give it a lower dosage yeah I do agree I think of course there is investigation there are investigations how to dose the drug and really also when these drugs are developed in actually all pharmaceutical industry there is extensive research on the dosing because you want to have the drug available for the patients but also on the market for pharmaceutical industry so whenever the dose is too low it won't be good so there is extensive investigations what I think is difficult with the monoclonal antibodies is that there is a difference between the traditional pharmacokinetics and the long term effect because we did a lot of research on our own laboratory and what you see is that there is not only an anti myeloma effect but it really changes the immune environment in patients because it also affects the T regulatory T cells which are suppressing T cells it activates cells which are able to kill myeloma cells and the long term effects might even be just more immune modulation than direct killing and we don't know what the pharmacokinetics is so in this case you can't blame that we didn't do the research at lower doses because there was an unexpected effect of the drug and I think what we really should look for and I agree with you that knowing this it's very interesting to look can we give it once in four weeks, once in eight weeks once in even less and also what we have available in our university and I think that's also very interesting is the immunopat method so what we can do is that you can label the drug with a radionucleide and then you can see whether the drug is actually going to the tumor because we also know in multiple myeloma that it's a patchy disease so when we measure levels in the blood it might not reflect what is available in the tumor so I think also for immunopat there's a role so there's there's a lot to be done it is a little bit more difficult than only blaming industry but I do agree with you that we should look in depth and also look some of the oral drugs you give a certain amount every day but maybe you can do it once every other day and we know that when you buy by example 25 milligram versus 5 milligram that the 5 milligram is relatively more expensive than the 25 so when you can taper those over several days you will save money so it's interesting it's really interesting and I do agree yeah okay thank you very much because look man and now we will have a 10 minute break if you have question you can approach yeah so you have two types of frailty I think where I was talking about was just frailty in in general so frail persons or unfit persons who are not able to do everything in life because of old age but also because of comorbidities for this type of frailty it is important to see whether there are frail and you can do it very easily with the IMW frailty index and I will provide the link and you should adapt the therapy so lower the therapy give less intense induction therapy and then continue with with maintenance therapy so that's the kind of frailty in which you would adapt the therapy frail bones is also important because bone disease and multiple myeloma can be really devastating and I think what has been shown that whenever you got multiple myeloma you should give something which inhibits the osteoclasts because the osteoclasts are the cells that are eating the bone and when you give drugs that lower the activity of the osteoclast less bone will be eaten you get less osteoporosis and we know you will have less fractures and less bone disease these are the bisphosphonates and the bisphosphonates you should give in multiple myeloma patients for at least two years every month and you can either use intravenously permidronate or zolendronate what to do after two years we actually don't know I think whenever there's still active disease you should continue the therapy and by example every three months so lower the frequency but continue bisphosphonates we're really looking for drugs that are also increasing the osteoblast because your bone is eaten by the osteoclasts but being grown by being produced by the osteoblasts and we know by example that the proteasome inhibitors like Bortesmib are increasing also the activity of the osteoblast so for bone disease we think that Bortesmib is a very good drug but it's not really very well investigated but for bone disease bisphosphonates two years every month is I think required and should be done in patients with multiple myeloma I hope I answered your question are the exercises yeah that's difficult but I think whenever you're able to do exercises you go to a physiotherapist and it's good to do you should of course be careful about the spine because it's just exercises in a controlled way not doing weight lifting etc but in which you keep your muscles in a good condition that's of course good because when your muscles in a good condition it starts it's along the spine and it will be helpful yeah okay thank you very much and before the coffee break we'll have a short brief by Ananda about the logistics


Welcome to another edition of RCE again. This is Brock Palin You can find us online at RCE-cast.com where you can find over 100 episodes about research computing scientific computing and other topics I have again here Jeff Squires from Cisco Systems and one of the authors of open MPI Jeff. What's going on? Hey Brock today We actually have one of my colleagues from the open MPI community here today to talk about what kind of started as a Sub-project but has grown into an entire community and project Unto itself, so I think I'm just gonna jump right in here and say Ralph. I wonder if you could introduce yourself Yeah, sure. My name is Ralph Casting. I'm a principal engineer from Intel Corporation And we're gonna open MPI with Jeff for a long time now and we are We started this PMIX community that Jeff asked me to come here and talk about today All right, so Ralph you said the keyword right there PMIX. Why don't you give us the two-minute version? What what is it? Yeah, so it PMIX stands for PMI exascale process management interface exascale and Basically what it is is a it's a standardized way for applications to interact with the system management stack like the resource manager and things like that, you know to request services of various types and And be able to get a response back So what is a history of PMIX because I think there's some other PMIs out there Yeah, there are PMI originally started, you know quite a while back now as a way of having the providing a way for applications to wire themselves up so They'd be able to exchange addressing information basically to say how do I talk to you? But over time what happened was that the the needs grew to where you need the application needed to be able to interact over broader topics and So PMIX kind of grant grew from that as a response to that to say let's give you more ways to interact Okay, so you said a second ago that this is dealing with resource managers I assume you're talking about like torque and slurm and LSF and all the others that are out there But what exactly does that mean? What does an application need to get from resource managers? So there's there's really two things or two types of things that they do First off when you're launching a resource manager can provide you with all kinds of information about your job That's really helpful when you're trying to optimize communications and collective operations So they can tell you, you know, where all the your peers are located What addresses they have etc. That's information you can have at the very beginning So you don't have to exchange it afterwards Then the other things that you know that you want to do is for example You might want to ask for additional allocations of resources or or maybe you want to spawn additional processes So there are these kinds of services You know, what's what's the status of cues and things like that that applications as they're evolving really want to be able to do and so in my Limited worldview here. I think of this is applicable to MPI applications And so when you talk about wire up and communications and things like that you're referring to MPI applications of any scale honestly from, you know, two processes to two million processes That when they start up, they need to exchange MPI addressing stuff to get their ethernet addresses Are there in finnaband addresses or there whatever type of Networking addresses so that if I MPI send to you then I know how to open a network channel to you That's the kind of stuff you're talking about Yeah, in part of that certainly is is in there, but it's also applicable to non MPI processes So let me give you an example say I'm in a cloud and And I'm running an application doesn't have to be an MPI application any application in the cloud Well, there are things like for example when I get started I can use PMI x to communicate to the cloud manager that I'm willing to be preempted And that now can be a service kind of thing You know where hey if you're willing to be preempted you get a different rate on on your charges So I can now announce to the to the to the cloud manager that I'm willing to be preempted And I can use the PMI x mechanisms that the cloud manager can tell me hey I need to preempt you now and Wait for me to say back that I'm ready So I can go ahead and checkpoint my job or do whatever I need to do To prepare for it and then I get to tell the cloud manager back, you know, I'm ready to be preempted now So there's a whole bunch of things like that that you can do. There's got nothing to do with MPI so this almost like a Operating kind of like a message bus where different clients can basically say this is what I'm able or willing to do exactly right one of the Mantras we have it in the PMI x world is that PMI x does nothing All it does is it communicates your request to the local system management stack and returns the response back So that that management stack always has the right to say, you know nice I'm glad you asked that but I don't support it and you have to have it in your application some, you know mechanism for dealing with a not supported response, but But all the things you know people talk about what flexible workflows and the ability to manage their own environment better stuff We just provided that the the hooks by which you can do that so is this done like a core plus extensions like if My client doesn't know about something like how It seems like you could keep adding more and more information that you could be announcing that you're capable of doing or communicating with How do you make sure that things can remain compatible between adding more and more things being mentioned or announced? well, we we adopted an architecture that says we have very very simple API's and It's like the job control who you announced for example that I am preemptible There's just one job control API and then you can provide a an array of key value attributes to that to that API that actually Dick, you know describe what the the operation that you want to do so, you know if we want to Extend the kind of things that you can do like for a job control I want to add something beyond just you know announcing that I'm preemptible or whatever We don't change the API. We don't add another API. We just give you a new key value attribute that you can pass And so that that's how we maintain backward compatibility. We just we don't we have just a policy We do not add API's unless there simply is nothing that fits and then we'll add one So what exactly is the big deal here because way back at the beginning and again I'm admittedly taking a limited worldview of MP of MPI applications, but I understand this applies well beyond that as well, but we used to just do SSH and Pass things on the command line like if I want to start a 32 process or 128 process MPI job We could just do SSH and maybe do that a little smartly maybe via a tree or something so that it wasn't linear Why why do we need all this extra control stuff? Of what used to be a relatively straightforward process Well, okay, so there's there's two different parts to that one is you know if you're looking at an MPI job Which I know is your your focus Jeff the The problem is there's only so much you can pass on a command line And so as your job gets bigger or as the amount of information that you want to pass gets larger You just can't fit it on the command line anymore And so you need some secondary mechanism for for making that for passing that information around what we used to do Was was the PMI approach, right? Which was to say well, we'll take you know Everybody will since we have a limit on the on the command line We only use the command line for what we have to and then everybody simply broadcast that and there's some you know out of band Mechanism by which it gets exchanged And that has all the scaling issues and so that's why we went this way is saying well Let's just get back to that basic thing give a mechanism by which the resource manager can convey that information to you Outside of the command line limitations Okay, so this is a way for applications to talk to the resource manager But how does that address that I'm process a and I want to communicate with process B and Therefore I need to know some kind of network address for process B. How does that work? so Yeah, so the way we did that in the past right was that process a would discover a network address He would broadcast it to everybody and process B would then receive that broadcast say oh, okay I could communicate to you What we've done is that when the resource Manager is getting ready to start the job. We've given them a an API a function They can call that will talk to the network and find out what are the addresses that are going to be used by the different Processes what nodes are they going to be on and what addresses are they going to be assigned to and then we include that in the Information that's given to every process when it first starts up And so when the process first starts up process a starts up it can ask What's the address for process B and it already has that information there? So is this something for end users or is this just something for like creators of MPI libraries like open MPI and other types of higher-level tools that users interact with this just between the creators of those middleware projects and The resource managers or is this something that a user can directly interact with Yeah, it's really both So the libraries, you know open MPI libraries the open Schmem libraries, etc they embed interactions That that use these PMIx interfaces to do their basic, you know wire up and and other operations that they do but then the Application developers themselves are using it because they're the ones who know how they want You know that their application has certain workflow requirements, you know They want to be able to allocate more than any resources or whatever they want to do They use it directly themselves And then you are seeing people embed it in tools So, you know for example if I might have write this tool that allows me to launch jobs It says certain command line options and everything that I really like in the past You'd write that but it would be specific to the resource manager that you were locally using now You can write it with PMIx instead and take that same software and simply move it from one resource manager environment to another Without having to change any of the code Okay, so this is really talking about just having a portable standard API that multiple resource managers and middleware developers can develop to correct Okay, so this does not necessarily Replace something like let's say the task manager, you know TM spawn inside something like one of the PBS derivatives out there Because PMIx doesn't actually again do anything But now open MPI doesn't develop to TM spawn and all the other different interfaces out there They just developed a PMIx That's exactly correct. Yeah, open MPI calls PMIx spawn and PMIx takes care of the abstraction for it So just to nail this down then you're saying PMIx is a library. Is that right? It's actually three pieces first there's a there's an actual standard and And that just defines the API's and and and Defines a set of attributes, you know key value attributes That we all agreed that we would support or at least recognize But that it says nothing in it at all about implementation and people are always free to implement it themselves The second piece is an actual reference implementation. That's a complete library both client and server and That fully implements all the all the PMIx function calls So again, all it does is communicate so the client function calls communicate to the server function calls that then call the relevant back-end Resource manager functions to actually do something Then the third piece that we actually have a reference server It's it's it's about a runtime if you will it looks just like a resource manager That supports PMIx with the exception that doesn't have a scheduler in it And so if somebody wants for example to develop some PMIx based code in an environment that doesn't yet support PMIx itself They can run this reference server in that environment and it operates just as if it was it sitting in a PMIx resource manager Except for like I said, it doesn't do scheduling. So those three elements are what we mean when we talk about the PMIx community Okay, so you're you're then also implying here that the resource managers out there are also supporting Pimx directly. Is that a correct inference? They are some of them already do slurm for example has been doing it for for a year and a half now and IBM has been developing their jobstep manager and that that is you know really based on Pimx Our PMIx all across the board and others are coming along at various stages. So, yeah eventually we we do hope that That most of it if not all of the resource managers out there will in fact provide that support directly Now that's actually a pretty fascinating place then that if applications have Want you know a standard that they can write to to do their large-scale applications like MPI is one of the biggest But there are others out there and then also the run times have a standard to talk to the back-end schedulers Well, that would also be fantastic because as you and I both know maintaining You know the MPI implementation to talk to all these different resource managers who have different abstractions and different APIs and whatnot It was kind of a nightmare. So Since this is a fairly large disruption in this community, how did you manage to pull this off? well the you know being Having some role in open MPI obviously put me in contact with a lot of these people to begin with like you said We had to talk to them because of the infrared interfaces. We had to provide So there was some personal contact involved in that as well But the you know the real thing was that this is a need that All of us that were involved in these communities, you know the resource manager folks, etc We all knew we needed to do something and some of the resource manager folks had already been starting to write proprietary responses to it Which was what it was causing a lot of consternation in the user community because you kind of had to lock your application Then into that environment, which is you know something people really don't like to do So it was a recognized need out there and the real key thing I believe in that way that let us get the adoption by the resource manager guys Was the the stipulation that that the resource manager always has the right to say not supported so So, you know if you don't want to for example say you don't want to support a particular back-end capability You just provide a null in that function pointer and the Pimx server if it gets a request for that We'll see the null on the back end and just return and not support it for you So you don't have to do anything If you if you want to support that capability that function, but maybe you don't support every option that somebody could pass Well, you have the right to be able to look at those options and say, you know, I'm sorry I don't support that and reply back So I think that not supported ability was the thing that was one of the key things so a second part of this inference of You know the the resource manager supporting Pimx natively are they writing their own code their own implementation from scratch or are they using your reference Implementation and then as a consequence of that is there a standardized network protocol that you use because then you can divorce The software implementation from what is communicated across the network So They're all free to write their own code so far. Nobody has done that and there's no indications that somebody wants to do that The we did not we did not standardize the protocol between the client and the server So if somebody writes their own and it's incompatible Well, that's you know, you just have to make sure that you link against this the the same client library that they used You know, they have to provide both a server and a client library, but the API's would be the same to be compatible Let's say so far nobody has gone that route it's a lot of code to write and Nobody has seen a proprietary or a competitive advantage. I should say in Implementing their own the competitive What what's happened is that the competitive basis between the resource environments has shifted from the API's to What level you know, which API's they support on the back end and which ones they don't Okay, so one of the goals of PMI X is you know, you've got X scale right in the name of it Um What exactly does PMI X bring to the table if it doesn't actually do anything that gets you that extreme scale? well the It's really again in two pieces if you will the exo scale Requires that you be able to you know be a first off You have to be able to launch the job in a reasonable amount of time If you just took the current method of broadcasting and sharing things, you know Exo scale machine might take you know tens of minutes to start up a job that size and That's obviously that's something you really don't we would not like to see so Part of that is is the ability to then have the resource manager share the information that it already has I mean we went back and looked at what was actually being broadcasts around It turned out that more than 90% of that information the resource manager already knew But it didn't have a mechanism a standardized mechanism by which it could share it to the application So either the application had to come up with a you know a resource manager specific way of dealing with that communication or We had to standardize it so the the library could be portable and so we took the approach of saying look the resource manager Already knows a lot of this stuff. Let's go ahead and just give them a standardized way of communicating it Then the second half of that was we had to go to the resource manager guys and say there's an additional 10% of the information that we would need and if we had that we wouldn't have to broadcast anything and we had to get them to agree to provide that 10% and And that's we were able to do finally is to get a get a commitment from them And so we created a list. It's on the web That has here's a list of all the stuff we need and the resource manager guys are going through and filling that list in so Aren't you just moving like all the start-up time from the MPI? You know like runtime, you know wire up whatever you guys call that very first step to get everything going Aren't you just shifting that problem to the resource manager? How? You know normally it's not the individual nodes on you know resource whatever the resource manager has run any individual knows It has a lot of this information right like how? I'm not seeing yet quite how this actually benefits if if all the information still just kind of held in one place on one part It's just now into resource manager Okay, well, let's take the the endpoint information as an example so The way it works today All right in the past has been that the application process starts it It discovers a resource. Let's say let's use sockets as an example It opens a socket and gets a socket ID Socket number and then it has to broadcast that because none of its peers know what socket. It's listening on So one way you could address that was be to assign static Sockets to your processes and then you don't need to exchange the socket information anymore because you can compute what socket they're on But the problem is that you might not be the only application Running on that node and so you don't know what sockets you can actually take so What we now do is we say okay resource manager you use the PMI X plug-in for the for fabric that plug-in will Manage a pool of sockets based on its knowledge of what's being run across the across the different jobs Across different nodes and it will use that to assign static socket numbers For this app for this application Those numbers then are included when the resource manager sends its launch message out to the to the compute nodes That information goes along with it So instead of having an exchange The daemons are given Before they even start the political processes They're given all that endpoint information and they just simply convey it down to their local clients and That eliminates the need for each client to broadcast that information So if you go through and you look at what what pieces of information the different libraries are asking for you make at laundry list up and And then you ask the resource manager the workload manager when it sends the launch message out to the individual nodes You ask it to include all that information for every node in that message And so now you just use TCP sockets as an example and that's kind of a baseline, but many more high-end HPC environments for example use other types of networking and they're several available so this I Just want to clarify that TCP was just an example. You could do the same thing regardless of the back-end type of network, right? That's correct, and we already do Support those at least at least all of the the most popular ones But basically yeah, that's right. You know the the network interface support in PMI X and the server side Is this a set of plug-ins so their plug-ins, you know for all your favorite flavors of fabric and We in those plug-ins. We've worked with the network manufacturers to get those plug-ins available And those plug-ins are all now capable of creating those addresses for you and So just to drill down in this little bit more so not only are you making the data available? Let me just provide one thing that was kind of an inference there was that when you say it's sent out to the Demons you mean it's sent once to each Let's just say node or server on the network even though that's kind of an amorphous term But like once and then if you've got 20 or 30 or 40 or more cores on that server that Damon receives the information once and can Locally give that information to all of the processes that start up whether it's 20 30 or 40 using local IPC not networking IPC, right? That's one of the wins. That is one of the wins. That's correct All right, and then additionally on top of that you also do some compression types of techniques in the launch message that sent via PIMX, right? Well, the law PIMX doesn't send the launch message the resource manager does we just provide the information for them that we say look This information needs to go along Here's a payload that has all the information that we need you to take along You'd be surprised That the information is not as big as you might think our typical launch message Our PMI X enabled launch message from like slurm for example, it's less than a megabyte Because there's so much, you know, we compress obviously anything we're gonna provide up to slurm slurm Just gets it as a blob and just sends it along But you know slurm use you know other and the real all the resource managers do they use like regular Expressions to describe where the processes are located in things to try and keep the the launch message down So it really is only about a megabyte in size or so even for an exascale size machine Ah, that was that was my next question. So a megabyte for How many like what what scale how many processes are you talking there? Well, we've been launching In our in our biggest test cases a million processes on like 30,000 nodes and that that's about you know About one one and a half megabyte launch message So it's not very big So what about use cases besides MPI? I know that's where this kind of started But you also talked about you know on cloud how you could say you're preemptible. Has there been any implementations? That kind of touch on one of those other examples Yeah, there are the And I apologize that I cannot I'm not at liberty to give you you know names and details but Because those companies haven't taken it, you know public yet, but but there are people working on You know cloud Interactions like we talked about earlier. There are also people working on different kinds of tools You know that they can take advantage of it You know debugger tools etc that can use these kind of interfaces to do more than they do today For example on a debugger today you get a Node-level representation of where everything is and there's a limited amount of you know of information as you can provide Because the interface is limited So but with a PMI X interface you could ask for example to show the nodes in a network based layout You know, where are they on the network relative to each other the processes? I should say where they are relative on the network to each other You could ask the fabric for traffic reports and show where choke points are Because the interfaces allow you to be able to make that query to the system management stack and be able to get that kind of information back Now you said something a minute ago that I kind of want to dive into a little bit You said you were testing at a million processes. How do you test at that scale? Well, we have Friendly users at who at facilities that have these kind of big machines and They will generally take a little bit of time out and run some tests for us which has been very much appreciated We also have ways to simulate scale So for example, we can launch multiple processes on a given node and make them look like they're sitting on different nodes and so One of our our collaborators was kind of to do that on Amazon where he in fact takes a small number of Amazon nodes But makes it look from a PMI x standpoint like a much much bigger cluster and we can do some scaling tests on that that may not be You know fully Realistic in terms of a cluster but gives some pretty good scaling law measurements So we so we have ways of getting that information even when we can't get a hold of the big clusters So could PMI x help in heterogeneous environments by this I mean, what if I have you know hybrid systems are popping up all over the place where you have sometimes you have Accelerators and nodes you might have FPGAs are coming back on the market as alternative types And you know like for me at the University of Michigan our cluster has Different machines of different architecture types with different accelerators Is this a way that a given application could basically optimize for whatever machine it landed on by asking a system? Do you have these things I know about? Yeah, you could query what we know what resources are available to me There are there are coordination Mechanisms in PMI x that might allow you for example to say well if I I don't see or I have only four GPUs on my machine How many of you got on yours? Oh, well, maybe then we use a coordinate We handshake and you go ahead and run something over there So there are these mechanisms in place I don't think that everybody that we have really fully understood everything we can do with PMI x The community has been trying to come up with you know Just trying to enable people to experiment with it and then the expectation is that people are going to do things with it That are beyond anything we had in mind but that feedback will come in and then we'll be able to perhaps offer better mechanisms for them to do some of those things but I Think this is yeah, we've never had this kind of capability before to be honest with you And we're kind of feeling our way at the moment to say well, you know, is this is useful to you and how might you use it? And and hopefully you know the community to have a chance to try and test those things out So you had mentioned earlier too that you know, you could ask the fabric You know how congested you are in different places and with more complex fabrics Especially on the bigger system just definitely useful, but even on regular systems We might have you know in you know fabric islands or you might have a dual-fat tree or something like that Is there a way or is anybody doing anything where you could effectively choose say an optimized collective operation between hosts based on You know effectively the fabric architecture Which again the application can find out about by asking through one of these standard interfaces I? I don't think anybody has done that yet again, I think that's one of those things that What we're looking to see is you know given the given people these tools that they can do that We expect researchers in particular to start asking those kind of questions And I believe that there are groups in fact that are starting to ask exactly that kind of a question about the collective optimization But I don't think they have published anything yet And This really only goes one direction right now and by that I mean if an application encounters an error condition that is could be system-related Think of like the PBS health check I can't use the PMI interface to send information back to the resource manager or to some sort of metric system or something like that Well, you can there's there's two ways and you can do that one is you could just raise an event and And we support complete binary Payloads and we actually take care of heterogeneity for you So you can put binary numbers in there or whatever you want to do and then the event when you raise it You have the ability to pass however much information you want in there So you could raise an event to the resource manager saying hey, I saw something. Here's a complete blob of information about it And then the resource manager can do something with that obviously that have some kind of a Agreement with the resource manager that a you're going to listen for that event and be you know have some idea What you're going to do with it But but you also have the ability to log so one of these people asked for was they said Well, you know if I see something or if I just want to even log that I have a certain amount of progress And I want to stick that in my job record So, you know every resource manager keeps a record of the job that you could go back and look at see what happened You can actually insert messages into that log From the application and that way, you know, there's a when you get the official Job record of what the job was did you how much resources it used etc Those messages will be there for you so you can record them for yourself or you can try to communicate them to the resource manager Now one thing you've been very consistent about Through this whole discussion is you've been saying a lot of we and the community this and the community that who is involved in the community Well, you know Intel obviously through me and then there's Melanox is with us on IBM Are those three are probably the biggest code contributors in the moment You know, we have a list of others, you know Fujitsu is involved folks from wrist We have Livermore and Los Alamos from the national lab community that have been involved And then there's been, you know, slur the sked MD guys the all terror PBS guys have been there So there's there's a and I'm probably leaving some folks out I'll have to apologize to you afterwards because I There's about about 12 to 15 active members at this point in time And these are members who like you mentioned the first several of them were code contributors to the Open-source Pimx code base itself, but these other members are helping to shape the standard, right? I mean, it's not just about contributing code, right? That's right. That's right the Ever everybody participates all those people participate in the In the standards process which is based on the IETF mechanism So there's always an RFC that that has to be, you know, backed up by a prototype implementation Which is usually in the reference library. It doesn't have to be But it has to be visible and then it goes through review. It goes through Comments and then we have weekly teleconferences and At those weekly teleconferences we schedule and and have a review of those those proposals and then those things get accepted Or rejected usually accepted into the standard. So all those groups participate in that Okay, so this really is quite a bit more than just Ralph's little toy This is a full-on community that is full of all the interested parties across this vertical, right? Yeah, yeah, you know, it really is a lot more than that than me and And I should give credit to everybody that's involved is it's a lot of work from everybody's standpoint You know part of it which I have to understand is that you know, it's you can create this channel by which these two parties can communicate But there's a lot of work behind the scenes that has to happen that the resource manager guys have to agree They're gonna provide and then you know like for example You have to go to the fabric people and say hey guys We need you to provide this this endpoint information blob, etc That's work on that part that they have to do to provide that information Or if you're gonna talk to the file system guys and you say well, you know We need you to be able to pre-cache files for us and we tell you or tell us how long it's gonna take them to Retreat be retrieved. That's work. They have to do so it's really a Collaboration across all these different elements that makes it possible So it seems very flexible. What's the strangest or probably in this case more unexpected use of PMI X you've seen so far The one that surprised me the most was a request really from the from the cloud some people working in the cloud world Where they wanted to be able to loan. I think I miss this really they want to be able to loan resources back So they actually have workflows where they they you know, they need a envelope of resources Eventually at some points in their computation or their work, but there are times when they don't need it don't need all of it And you know, so I had never anticipated somebody actually Loaning resources back to the system and getting them back later That that to me was a surprise That last point was what I really wanted to get across it really involves all these different parties actively collaborating Because that's what's really different in my mind with PIMX versus before is it's the first time I'm aware at least of You know the resource manager the fabric the file system the library guys Language library guys all getting on a weekly telecon collaborating on how they're going to orchestrate this application environment All right, so there's a lot of open-source code here You must have a fairly permissive license that you distribute this under because some of these resource managers are Closed source and proprietary. What what license are you using? We use the three clause BSD So it's it's absorbable by people using, you know, a GNU license as as well as You know proprietary people they're all welcome to use it So Ralph thanks a lot for your time. Where can people find out more about PMI X and get involved? The probably the first place to start would be going to github everything's on github for PMI X so the the reference page for for PMX itself is PMI X dot github dot IO slash PMI X Or you can go to the code repository itself There's a group because we have both the Implementation as well as the reference server there and that's at github.com slash PMI X Yeah, Ralph, thanks a lot for your time. Thanks, Ralph. Hey. Yeah. Thank you very much for having me