Hello everyone. Thank you for joining today. This is a webinar of Myeloma Patients Europe, and it's a very important topic that we're covering today. We often discuss about clinical outcomes, about survival, about how to improve outcomes of patients living with multiple myeloma. But today we have a topic that is equally important, but often not so much in the radar of people, which is about the importance and challenges of measuring quality of life in myeloma trials. So we have a distinguished panel today to discuss these issues from different perspectives. And before I go into the content and also introduce the panelists, just somehow keeping rules for this webinar. So on the next slide, you can actually see that you will see the presenters, but you will not hear or see other attendees because it's a Zoom webinar. If you cannot hear the presenters, make sure that your speakers are not muted and the volume is set too high. You can also later on, when we have the panel, go into the gallery view in the view settings of Zoom on the upper right corner of your screen, because that way you can then see all cameras of the people during the panel. You don't need to do that today, just now, but you can do it later on. And if you have questions, you see the little Q&A button at the bottom of the screen. And this is actually the question and answers function of Zoom. And you can submit your questions there. We're going to see them and screen them while we listen to the presenters. And I'll try to raise these questions then also to the speakers. As a next topic, actually, we're going to have a panel and Q&A session after the end of all presentations. So please write your questions in the chat, and I'll try to ask them to everyone. You can also use the chat function, which you can also find at the bottom of the Zoom screen. So you can chat with other participants. But please, if you have questions to the speakers, please do that on Q&A, because it's much easier for us as moderators to actually see the questions. If you're having any trouble with poor video or the signal cutting, you know, you can also turn off video and just keep audio on. If there are any technical difficulties, you'll also find the MPE team in the chat. So you can actually mention any issues that are there. And finally, we're going to record this webinar. So it will be available on the MPE website. We can see the domain actually here and also on social media channels. All right. So going into the agenda for today, we have in total one and a half hours, we're going to do a bit of introductions in the beginning, but keep that very brief. We're going to have a presentation by Malone Patients Europe in Consilium Scientific Research and then actually go into a panel discussion and also address your Q&A.  So we're going to close this meeting for 30. Okay. So let's move into the panel we have today. And I didn't introduce myself, but probably I should do that now. My name is Jan Geisler. I've been a cancer patient, a leukemia patient and patient advocate for about 21 years. I'm CEO of Patvocats. And we're going to have two speakers in the very beginning, which I'm asking to introduce themselves. But you can see all the speakers here. We're going to introduce the other panelists later on when we go into the panel discussion. So first of all, I'd like to hand over to Lisa Osbienko. She's Chief Executive Officer of Consilium Scientific and she's going to speak about quality of life data and myeloma research results. So Lisa, the floor is yours. Thank you very much, everyone. Thank you very much for the introduction, Jan. So, yes, I have Consilium Scientific now. This is a nonprofit organization dedicated to improving quality of clinical research. But my background is at NICE. And I've been head of scientific advice for five years at NICE. And I looked at quite a lot of trials and work with myeloma trials and myeloma community in the past. So I'm very grateful for myeloma patients Europe commissioning this very important work. It was a very personal interest of mine to see how quality of life data collection changed over the years. And it was an interesting project for us. So I'm happy to present the results. Next slide. Next slide. So we looked at all registered trials for over the 10-year period between 2011 and 2021 across all registries to which we have access. So this is 18 registries. And most trials are of course are in European registry and clinicaltrials.gov. But we also looked at global registries. And there's a lot of duplication of records. So our methodology included cleaning the records, making sure we don't double count the particular trial. We discarded all the terminated, withdrawn, suspended trials for analysis. However, it's very important to understand that the data I'm presenting here is not on completed myeloma trials, but it's on completed and ongoing myeloma trials. So we're here not reviewing results which are published in the paper, but they're actual protocols. So it's a promise of collection of quality of life data either as primary, secondary, or exploratory endpoint. So from our analysis, we identified over 1500 trials with multiple myeloma patients over the 10-year period. Next. And because this project was done specifically for myeloma patient Europe, we've done a sub-analysis on trials that included at least one European country. Here I will be presenting data only on all trials that we have evaluated not looking specifically into Europe. Next. This is the 10-year results, longitudinal data in terms of number of trials that collected quality of life data in relation to the number of trials that did not collect quality of life data. But I think next slide is showing data a bit more clearly. It's the percentage of trials that collected quality of life data in particular year. And I'm sure now you will have a lot of questions saying, but hold on, it all depends whether it's an observational trial or an interventional trial or it's phase three or it's phase zero. So and we have this granularity of data for you. But I think this longitudinal presentation is quite important because I personally been at NICE for quite a few years and preaching the importance of quality of life data collection to every developer really was expecting to see a very strong upward trend over 10 years. And here we do not see it. Actually, on average, it's pretty, pretty stable and quality of life data collection in clinical trials and myeloma is around 36%. Next slide. So we looked at who actually runs these trials. And for myeloma specifically, and it actually differs by clinical area, but for myeloma specifically, you can see that the top shares are run by industry and academia. Next slide. And here we did analysis in terms of also quality of life data collection by a sponsor. And I am really curious of the magic non specified sponsors of trials, they're doing a really good job, I'd like to congratulate them. But when they say not specified, basically, this information is not in the registry, but they seem to be doing a great job collecting quality of life data. But of course, it's very small proportion. And this is a technical administrative error of not entering the actual sponsor. So as you see, surprisingly, academia could have done better in terms of collecting quality of life data. But industry is doing very well compared to other settings, but are they doing enough? Next slide, please. So as you can imagine, most trials registered interventional, there might be running many more observational trials in any clinical disease, but not all trials actually end up in the registries, unfortunately, but most international trials are registered. Next slide. So in terms of this distribution, you see that quality of life data is expected, is collected in more international trials than observational. Next slide. Then we looked at the types of myeloma. It's a very detailed slide, as you see, and most trials are happening in relapse refractory setting, which is not surprising. But there's a growing number of newly diagnosed myeloma patients. And the next slide presents the distribution of that quality of life data collection based on the type of multiple myeloma. And of course, these two slides better be looked at side by side, because here you cannot really look at small during myeloma and say how well it's performing against relapse only because the number of trials is significantly different. But once again, for people with specific type of myeloma or for clinicians looking at specific type of myeloma, this data disinformation is important. Next. We looked at trial phases, because of course, that's a very legitimate question when should you collect quality of life data when it makes sense for analysis, when is the burden for the patient. And of course, as for most medical conditions, we have quite a few phase one trials, quite a few phase two trials, but very few make it further out of the scale. And we have our smaller number of phase three and phase four trials. Next slide. And this is the distribution of quality of life data collection by phase. And I'm very pleased to see that of course, for phase three is the only area where we can see more trials collecting quality of life data than not collecting. So that's a healthy trend. Why is it at 55% only of 56? It's a question and we would love to see more. What's concerning is quite low number in phase four. And at least we have some data in phase two, but it would be good to see more quality of life data in these trials and we'll explain why in discussion. Next. So here we'll look at the primary purpose of the trials. As you see, most trials and multiple myeloma are the purpose of the trial is treatment. Next slide. And we've done this analysis as well. And this is our average statistics corresponding to all the findings that only about 40% of trials dedicated to treatment collect quality of life data. It is concerning because these are international trials. A lot of times patients are exposed to regimens or medications interventions which they which haven't been tested before in these populations. So but this is what we found so far. Next slide. So that was analysis of trials in the registries of all the all the product halls of ongoing trials and completed trials. But we also thought it would be very interesting to see what is the literature saying about this. And of course to get into the literature this is a very different subgroup of trials or studies because to get into the literature they have to be completed and published. We have not looked at any protocols because they were picked up through the previous research. Please. Next slide. So we through systematic review we identify 554 articles which considered quality of life data. Next slide. Next. And there were three types of research. So it's primary research so reporting on clinical trials secondary research which included systematic reviews and economic evaluation. Next. So in the final analysis we selected 266 articles. All the methodologies available through additional information we have from Prisma chart to inclusion exclusion criteria. Next. So as you can see primary research was the highest proportion of the articles that we found. And the question here is also about the years because we looked at the 10 year periods. So these are the articles published during those 10 years. And this is a mismatch with actual trials in the registry which can be ongoing. So some trials we matched that we found on the registries to the ones published. But here there are quite a few published articles on primary research which started prior to the years of our analysis prior to 2011 in order to report during that period. But once again despite the years included I think these are also very interesting trends to see what we found through the literature. Next. I was pleased to see that reporting of quality of life data in primary research has been increasing. And this is also the trend through secondary research with economic evaluation. Very few articles were found. So there's no trend to report here. Next. Very for a publication record I have to say in terms of access to full paper most information was available in either abstract only forms or conference proceedings. And that's a major disadvantage because in abstract only very top level information is given. There is no granularity that can be explored in fully published research. So we extracted whatever we could extract but my only recommendation here is this is important work and this is important work for patients and seeing more publications actually providing further details would be important. Next. So what we found is here in publications quality of live data was collected mostly in primary research and is reported very little in from economic evaluation. But this is the also look at population types. You can see that this is a more aggregate level of analysis. We did not have such details as small during myeloma. For example but it does show you what is the distribution for different types of myeloma populations. Next. This was a surprising finding for us across literature review and across the registries. We actually extracted all instruments we found through protocols through the articles to see what is actually being used in myeloma clinical trials and we found 93 instruments and that is a lot. So there is a prevalence for generic instruments which are listed here. There is use of cancer definitely cancer specific go in the second place and myeloma specific instruments are also used. Quite a few studies use combination of these instruments if they if they collect quality of live data at all. But this variety of approach to quality of live data collection and once again we focused only on the instruments we could identify because as we know there are other explorations through surveys and interviews to understand patient states especially for CAR-Ts or new technologies. This is not included in this analysis. So this is just standardized quality of life instruments. Next please. So there's a overall use of instruments in myeloma research and you can see there's a big prevalence for generic instruments in primary research and also combination of generic and cancer specific. So myeloma instruments feature in some trials in some studies but not as much. Next please. So the last thing we've done is we looked at nice appraisals and how committee and the evaluation group looked at quality of live data submitted by the manufacturers. Next. So in total there were 28 technology appraisals but only 14 were in scope for this study because once again we looked at the 10-year period and some technology appraisals were terminated. So we did the analysis on 14 technology appraisals. Next please. And 26 clinical trials in multiple myeloma were included in those 14 TAs. Next. This is the populations which were analyzed in these technology appraisals. Very equal distribution for newly diagnosed relapsed only and relapsed refractory. Next. Since it's nice technology appraisal this says the company is required to submit the registrational trials and over the last 10 years most trials were in phase three in order to receive a marketing authorization and come to nice. So most trials were in phase three. Next. These are the instruments which were identified in multiple myeloma technology appraisals. So as you see usual suspects we had EQ5D, we had the ERT-CQLC30 and the myeloma specific tool NY20. Next. So where the data came from I think that's quite important because that actually led to debates and criticism from the committee. So only eight appraisals used trial only data so directly collected data in the clinical trial. Four appraisals used literature only data it means no data was collected in a trial at all or the quality of data collected was not acceptable for evaluation and two technology appraisals used the mix of data from the trial and from the literature. Next. So quality of life was dedicated as a secondary endpoint in non-technology appraisals and in 17 appraisals it was not sorry trials it was not collected as an endpoint at all and it's not surprising but just a statement that quality of life was not a primary endpoint in any of the trials but these are registrational treatment trials so it would not be expected. Next. So that's the final conclusion so that there were major issues with quality of clinical quality of life data in the technology appraisal in five appraisals and there were minor issues in nine appraisals. Next. So this is just a summary why the technology was approved at NICE so four technologies went to cancer drugs one because there was not sufficient information on clinical effectiveness of the product and it was expensive to went into the patient access scheme one was found to be cost effective and seven were found to be clinically and cost effective and decision was made right away. Next. So this is the summary of this work and the next slide. Overall quality of life data was collected in 36 percent of registered myeloma clinical trials through the literature search way and our database of registered trials we identified 93 different quality of life instruments used in myeloma clinical trials and clinical research and data from NICE showed that in five out of 14 technology appraisals quality of life data raised significant issues which impacts the decision making process and this is just the final point I have to say multiple myeloma is quite good compared to other areas but 547 out of 1557 multiple myeloma clinical trials were completed but only 63 percent of them reported results within 18 months of the trial completion date and the requirement is to report the results within 12 months so there is a lag and very important to get the reporting of results to patients to the clinical community into the public on time. This is it so thank you very much for your time and looking forward to the discussion and your questions. Thank you so much Lisa for presenting the data that's really brilliant and so interesting from a patient advocacy point of view given that on one hand a lot of trials do not collect the data on quality of life and also that let's say the delay is so long for getting that quality of life data out there because patients don't want old hats and it also needs to influence clinical decision making so a lot of fruitful thought which we're going to discuss in the discussion later on but before we do that now I'd like to introduce our next speaker which I guess everyone knows Kate Morgan co-chiefs chief executive officer of myeloma patients Europe so what does the patient community now think about what are your recommendations so Kate over to you. Hi everyone thanks Jan the introduction and so I'm co-chief executive officer here at myeloma patients Europe and I'd like to thank the chair and the speakers and everyone who's joined the webinar today and hopefully we'll have a lively discussion in my presentation I just want to go through some advocacy recommendations from our perspective and what we've taken from the data next slide please and next one. So just to provide you with a little bit of context as to why we commissioned the report and the research with Consilium Scientific and I think we really wanted to generate some evidence on how quality of life data is measured over time and I think from our perspective quality of life can be quite difficult to navigate for patient organisations it's quite complex and we hear anecdotally that that perhaps it's not being systematically gathered and that that sometimes impacts on access decisions at a national level but we had no data to kind of back up what we were saying and so we wanted to work with Lisa to understand the gaps the challenges and really to to derive some advocacy messages and goals for the community and to sort of advocate for going forward and we'll be publishing the report in full next Monday on the 5th so we'll send that out to everyone who's participated along with the full report but today we wanted to present some of our recommendations that we've generated to stakeholders to have a panel discussion about them and also to hear your thoughts on the ways that we can refine them and improve them and I should say we're not advocating here that quality of life is more important than other endpoints and we just really wanted to shine a light on how quality of life is being generated and how we can improve that for the patient community. Next slide please. So our first recommendation is that we think that researchers should consider the collection of quality of life data in all my lower myoclinical trials and I think from our perspective this should be done at least from phase two and when you're starting to think about generating data for access but I think in situations where in cell and gene therapy where the only trial is likely to be a phase one or phase two when you're bringing it to market then perhaps quality of life should be generated in those clinical trials and I know that there is sometimes an issue around single arm data and we'd be interested to hear the perspectives on the panel on how that challenge could be addressed. We'd like to see the use of two different kinds of patient reported outcome measure. I think NICE and in other HDA bodies often require generic tools to be used to factor into their decision making because you can compare them across diseases but also we've seen from Lisa's presentation that myeloma specific tools aren't routinely used but they should be being used to ensure that we get myeloma specific data. One issue we've been discussing is around the potential wall of ethics committees in facilitating improvements in quality of life so could an ethics committee potentially not mandate but ask industry to present their quality of life plans for the clinical trial and then justify where and they aren't able to provide that. So we're interested to hear perspectives on that topic and I think when industry and researchers have developed their PRO strategy for a clinical trial I think it's really important to run it past health economists and take it to sort of nice scientific advice scientific advice with the European Medicines Agency but also involve patient representatives so at MPE we have a task force for example where we train patient advocates on how to review protocols and how to review PRO methods and so that can be really useful to help reduce the burden on patients because they can tell you whether it's too burdensome and you can amend your approach. And also finally into Lisa's last point as well is that where quality of life data is collected it should really be reported and I think it's very important to assist patients in their decision making and it should also be presented in a patient-friendly way. Next slide please. So the second recommendation is around researchers collaborating to create an online and user-friendly database of myeloma problems so what are the most important problems that we should be using? How have they been developed? Were patients involved? Really a checklist to see whether the tool is suitable and I know that EHA and Sam Salak who's who joined the panel today are working on their own guidelines for this so perhaps we could use their approach as a basis for doing this and really help researchers understand what is and is not appropriate for PRO collection. Next slide please. So the third point is around the potential for clearer recommendations on the collection of quality of life data and clinical development to support regulatory and HTA decision making. So I know there's a lot of work going on on that European level and also with the FDA but is there anything further that we could be doing to make sure that the requirements are very clear to industry and to other researchers on how they should be generating PRO data and also how it's taken into account and also I think from my perspective we see that sometimes industry and researchers might get advice from regulatory bodies, from HDA bodies and from patient organisations like ourselves on how to measure PROs within the context of clinical trials and perhaps there are ways that we can align that and collaborate and really set the clear standards for expectation. Next slide please. And finally I think so building on the last point as well perhaps an international multi-stakeholder steering group could be formed comprising of patients and PRO methodologists and regulators and policy makers to establish a more consistent approach in collecting and assessing quality of life data in myeloma and this wouldn't replace the pan-European level guidance that I mentioned in the previous recommendation but would really add to it and really think about how we tailor the development of quality of life and PROs in myeloma specifically. And again there is the potential to use EHA guidelines as a basis and I think generally we'd like to see a framework through these recommendations for more integrating inconsistent ways of collecting and assessing quality of life. Next slide. So thank you very much for listening and I think as I mentioned at the start please let MP know if you have any further comments about the report and the recommendations after the webinar because we're really happy to hear feedback and whether the recommendations are viable. As a reminder please post any questions or comments in the chat and we can address them in the panel. Super thank you so much Kate very clear recommendations and also a strong call to all different kind of stakeholders to take quality of life more into account. We're going to discuss much more of that and I'm very happy now to welcome our other panelists to this to this discussion which is Professor Sam Sallek from the University of Hertfordshire. He's also an EHA scientific working group co-chair on quality of life Simone Eulemans from the Netherlands Comprehensive Cancer Centre and also member of the ERGC quality of life group. We have Dr. Beata Wieseler head of department of drug assessment at IQUIC and Francesco Pignatti from the EMA scientific advisor for oncology saw a really fantastic panel together with you Kate and Lisa. It's really great to discuss with you about all the different issues that we heard before and probably just to to kick off the discussion. I mean we've heard a lot about let's say how much is or how little is quality of life data being gathered in clinical trials and we heard a strong call for the community how important it is. So probably just as a lead in discussion question I'd like to raise it to Simone so why do you think is generating quality of life data in clinical trials in myeloma so important? Yes Jan thank you very much can you can you all hear me? Well first of all thank you very much for joining this panel it's been wonderful and also like I want to congratulate MPE and also like Considium Scientific for this great amount of work and this great report. So why is generating quality of life data so important clinical trials? I think that well in the past decades there have been a lot of novel agents changing M.M. management and there are nowadays a lot of treatment options and also a lot of treatment combinations and the toxicity profiles differ between these treatment agents. Some chemotherapies are for example known of their possibility to have nerve damage which have an enormous impact on patient's quality of life for example they cannot button their blues or they cannot close their their pens and with this kind of information if you know that some kind of treatments do have these side effects and other treatments do not have these side effects then you can make a decision based on that or you can have that extra information besides the survival outcome of a trial to to base your decision on. So I think that's a very important aspect that with this information you have a more complete picture of an outcome of a trial and another important thing I think in why we should use patient reported outcome measures and clinical trials is that of course we can be clinicians can report on side effects and they can use like common criteria to rate side effects but we have also learned in the past decades that if clinicians rate a patient side effect they rate it less frequently and less severe than if a patient rated themselves. So for example then the view of the clinicians are underestimation of our patient's experience so I think therefore if we really want to focus what are key areas for patients with multiple myeloma we also have to focus on and also have to include patient reported outcome data in clinical trials. Thank you so much. I mean Sam Salik you've been involved so deeply in all different kinds of activities around quality of life and have also been involved in developing HM Pro. So what's your perspective? Why is it so important? Why should we bother multiple myeloma? Thank you very much Janne. I just like before I answer that question I just want to perhaps paint a positive picture on where we are. Now it was only about 18 years ago that I was at a FDA meeting behind the closed door and some of the key players in FDA who were very involved in the CEDA assessment of dossier. It came out of their mouth and they said categorically anything based on patient's perspective is of no value and I remember about 38 years ago when I started to look at the development and measurement of quality of life I used to go to clinicians trying to persuade them to do quality of life studies. They thought that I was coming from a different planet and I was talking double Dutch when I was talking about quality of life. So that's where we were and this is where we are today. I think there are you know successor stories of you know how much we have really come along. It's all to do with the mindset. We really have to try to change the mindset that quality of life is something that is different to the clinical parameters. As Kate very elegantly put it you know it's not to replace it but it's really painting something to the rest of the picture that you know clinical parameters by medical values cannot tell us and that is the patient's voice. At the end of the day they are at the receiving end of these treatment and it's important to really include their voice in the whole assessment of the value of a treatment. So that is really the foundation of why it's important to include patient voice in when we are assessing a treatment in a clinical trial. But the other thing is to understand that you know patientreported.com it we can only secure and improve where we are and where we are heading by involving patients as part of any studies that we do. Patient have to be there as part of the research team. Patient engagement in research is very important and that's how we would be able to improve you know the inclusion of the patientreported.com in clinical trials moving forward. I don't know whether I've said enough Jan but happy to go more than that but I want to not dominate the panel discussion. No that's it that's a great great discussion point Sam and I probably want to turn around that question we've seen the numbers that Lisa presented about let's say the low proportion of trials that are really collecting quality of life data. So what is now cause and effect is it because there's a lack of patient involvement in trial design and that's why quality of life doesn't get the attention in trial design or is it because trial design doesn't look at it and let's say patient involvement is an independent factor of that that comes further down the line. So do we need patient involvement to make that happen? What's your perspective? Well the two-fold one is you need to change the mindset. Still a lot of the pharmaceutical companies academic researchers want to include patientreported.com because it's sexy to do. They want to take you know it's a tick box exercise. Whereas actually believing that it's a good thing to do it's an important thing to do is telling us something that other clinical parameters and biomedical parameters is not telling us and the second is really we need to have more involvement of the patient. The patient engagement in research is very important to be able to I'm only saying that from the experience because now for the last 10 years at least I have categorically said to all the sponsors I'm not going to be involved in any quality of life PRO studies if I don't have a patient as part of the research panel. Not as a participant but as a member of the research team. Now I have seen the benefit, my colleagues have seen the benefit of having the patients as a panel and it's absolutely an eye-opener because they will also get involved in the design of the protocol from the start. They look at the interview checklist interviewing patients. They look at the papers that are written to review the papers. You know from the you know A to Z it's an absolutely valuable asset to have as a patient on the panel of the researchers. Thank you, thank you so much and probably just reflecting and Beata I've seen your hand up I'm going to ask you in a second just one more question to Kate. I mean so now that we have that you've made a very strong call with recommendation one saying that quality of life should be measured in all multiple myeloma trials from phase two. So why doesn't it then happen? So what's what's what's the issue there that actually calls if nine life is not being raised or there's a lot of missing quality of life data? What's your perspective then because you're strongly calling for that change in salmon said we've come a long way already the worst issue? So I think I think there is a continuing perspective that a lot of the regulatory and decision-making is focused around traditional endpoints around survival which is understandable but I think we need to continue the shift towards quality of life improvements. I also think we've seen increases so pharmaceutical companies do a better job than academics than gathering quality of life data and I think a core reason behind that is because they are obviously engaging with the EMA and they're engaging with HTA bodies whereas in academia that they're not designing trials that are going to get them reimbursed at the end. So I think there's some education that we need to be doing with some of the myeloma community around the importance of gathering quality of life data and throughout the trials and I think we'll get there but it's just patient advocacy is getting a lot more sophisticated around the topic of quality of life and I think as you said earlier only through patient involvement in trial design and in PRO selection and showing that we actually can do a really good job when we speak about it while we see increasing shifts to the positive. Thank you so much and I'm Kate you mentioned HTA and probably now Beard it's your call so HTA what's your perspective on that? Yes maybe first of all I would like to agree with Sam's and Kate's suggestion that the involvement of patients in trial planning is one way to ensure that this relevant data is collected. I think HTA actually is another way that increased the availability of this data because I would say over the past 10 years PROs and quality of life has been included into HTA decision-making as a standard part of decision-making. So I think generally across Europe HTA bodies want to see data on mortality but also on morbidity and health-related quality of life. For example in Germany the requirement to provide health-related quality of life or to include health-related quality of life into decision making is in the law and morbidity on the other hand also very often can only be collected by patient reported outcomes if you look at symptoms for example and therefore I feel that because HTA is putting more emphasis on this type of endpoints this has been we have seen an increasing amount of data with that. I mean plus I can only report from from our German context we have hearings on each of our assessments and patient representatives are part of these hearings and I can tell you that they are giving every trialist a hard time that who didn't include quality of life and PROs into their studies so I think indeed that patient representatives do have a role here. And can I can I ask one question to you we are then also to Francesco about the regulatory sphere I mean if we assumed that quality of life data was there does it really help with access to medicine so does it really matter on the decisions that you take in terms of PRO does it really influence decisions you take or is that an add-on to clinical data is it just a tipping point? Well actually we are we are not as an HTA body we are not finally deciding on access that is done at different stages of the process however access decisions are the easier the more robust so I think the artist internet connection seems to be stuck I give her two more seconds to recover probably until Beati comes back shall we can I just pass on to to you Francesco and moving just one step ahead not to access but to regulatory how much role does it play? Yeah sure and I think there is still a perception that regulatory decision which look at let's say benefit risk are mainly driven by a hierarchy of endpoints where overall survival is high at the top and everything else comes comes lower and I think this is somewhat a misconception and in fact there are numerous reports which in the literature which candidly attack us for even not giving enough importance to survival type of endpoints my view in this is is different I think over the years we've tried to develop a framework that looks at all the available data and these are primary secondary endpoints if you look at our benefit risk assessment framework you see a table where we list what are considered to be important good effects and and bad effects and how we put them all together using when we have them patient preferences or something similar so in this more comprehensive framework I think quality of life like survival endpoints play definitely their role and I see this very much from my perspective really not as a debate like Kate was saying quality of life versus clinical endpoints or vice versa but how do we make the best of you know having systematic approaches to put together all of these endpoints all of which are relevant all of which weigh probably differently depending on the situation depending on the on the individual who is making the decision or the regulator in my loma like Simone was saying in many relapsed refractory situation questions about tolerability and and optimizing those and schedules are becoming very important and then this quality of life tool can really serve a key purpose in guiding our optimization efforts and perhaps in first line this is somewhat a a less critical question by no means unimportant so yeah they do have a role young and sometimes a critical role okay super thank you probably I mean I don't know Lisa if you still want to comment but I would find that very interesting because in the data that you've presented you were actually talking about a quality of data and nice technology appraisals and multiple myeloma so how does does that resonate to you what you heard from Beata and from Francesco in terms of what role does it play yes so nice has a slightly different decision-making framework compared to other HD agencies because it looks at cost effectiveness and quality specifically so there it does play I would say critical role and there are particular circumstances where let's say the role of quality of life might be downplayed for example if we look at the product which has unquestionable clinical benefits just the survival is not even a question it's it's very very clear benefit then it's a yes everyone wants this product and unless there are some severe adverse events that would be taken forward how often do we see something like that very rarely so that's why we start looking at all the evidence and nice looks at two things survival and improvement and quality of life and especially now especially in myeloma we cannot run very long trials we have surrogate endpoints we hope for advanced survival we don't have this data and it all comes down to the quality of life so in myeloma it's particularly important it's very very important in myeloma because it's a unique condition where we use triple quadruple combinations of products and the more we add to the patient the more toxicity it adds and even though we may say oh we've been using this product for decades now it's used in the new combination now it puts more pressure and then many many many lines so quality of life of the patient changes and I think particularly in myeloma there are if we look at some of the diseases there there are caveats but that's an important point for decision making and sometimes especially nice it might be the driver and the driver is I would say to the company because we look at company drugs right and what happens is if quality of life data is particularly poor it hasn't been collected they use some paper from 2004 and trying to to get utility values from there the end result for the company is that for them to be cost effective the nice will push them down on price they push them down on price if they cannot demonstrate high quality quality of life data and I mean they might actually there's another side of the story to this is that they might have brilliant methodological approach and the quality of life data has been collected appropriately relevant this shows everything but there is no improvement so that's also of course impact on decision making so we need to see what the drug actually does or the combination of drugs but then quality but what I've showed to you is we've been struggling with actually what is presented by the committee and I will agree with the edit this is my last sentence just to cut it that we are seeing that in phase three interventional studies by industry collection of quality of life data has been improving so that's definitely the case you've seen I haven't shown you the longitudinal information but in phase three we have been improving so it's happening but another concern we found 93 quality of life instruments so very challenges Sam I actually saw you nodding on the issue of quality and improvement over time so what's your comment on that well I think one thing that perhaps hasn't improved is not only just including quality of life in clinical trial but collecting good data you know not not we have to do the right thing but also we have to do the right thing right you know I've seen many clinical trial data quality of life data in recent years that you know sadly they haven't even either not collected at the right time points or not enough number of them during the trial that's very difficult to actually many make any sense out of it so I think what Kate mentioned that is that that's the issue that we are trying to address for the work that we have undertaken under the auspices of the EHS part of our work and the scientific working group to develop guidelines for the use and the reporting of patient reported outcomes in hematological malignancy clinical trials and in fact the first guidelines that we have developed as Kate mentioned is in multiple myeloma and that has been you know the systematic reviews been published and the guidelines actual guidelines have been submitted for publication so I think that's quite important again there are other organizations such as consort I'm sure that the end you know about that they have also produced some guideline for the assessment of the quality of the patient reported outcome reported in the literature and you know I would recommend you know everyone who wants to use quality of life in their trial to actually refer to the consort you know checklist so yes I mean the other thing that I like to perhaps come back to what Lisa said and Beata said what we don't know from the HTA agencies is how much value they actually put on quality of life data on its own not quality of life factored into the economic equation the actual quality of life data that is the benefit considered by the patients to them how much value is put on that in their decision making that is not really clear we know that they take the cost utility analysis where that you include both quality and quantity of life in the equation but we don't actually know how much value they put on the quality of life itself now I think it's important to recognize that the quality of life factored into the cost utility analysis is a different aspect than looking at the quality of life and the benefit considered by the patients and so that's the other thing now as far as the regulators is concerned what Francesco said unfortunately the benefit risk ratio that they look at it has not actually got the patient element in terms of the tolerability of risk the risk perception is very different from you know from one person to the other you know somebody you know would do a bunch of jumping with no problem at all no risk is considered whereas you know I wouldn't do jumpy jump that's very serious and risky thing for me so the risk perception is not actually taken into account by the regulator from the patient's perspective and therefore that also we don't know how they make their decision making based on patient's perception thank you so much I mean you're raising important points on quality and quantity I don't know Beatty do you want to come directly on that and want to comment on that yes maybe for the question of how we value that and and it seems I was kicked out last time so please bear with me if I repeat my anything I say actually it's indeed a different difference whether you go to the cost utility analysis or whether you look at the at the data themselves as at at how symptoms are reported by patients or as how quality of life is reported by by patient and across Europe there are different approaches to that so there are countries which use cost effectiveness analysis and go via the cost utility approach and there are other countries like like France or Germany which which look at the kind of clinical side of the benefit of a new drug and and at that point we directly look at symptoms for example or at quality of life so we have concluded for an added benefit for a drug solely based on symptom data or on quality of life data without going via cost utility analysis so this can have a direct impact on the conclusion on added benefit of a of a new drug and of course this is the more important in the situations where you are maybe not able to show an overall survival effect because it may take very long to to be able to robustly measure that um especially in these situations um an improvement in uh symptoms for example or quality of life might be a valuable addition I mean plus we feel that is a valuable addition anyway because patients want to know that in their treatment decisions they want to know how they will feel if they um if they choose a certain treatment thank you so much Beate I see Francesco's hand coming up straight after that so do you want to comment on this Francesco? No I wanted to comment to something that some said that the benefit risk assessment by regulators is let's say insensitive to the risk attitudes of of individuals and and and I think that is has been definitely true in the past but since quite some time we have tried to to be quite more explicit about what risk attitudes are considered and in fact we have let's say in in general use an approach that is that is modulated and tries to understand what is the sensitivity to the preferences of each decision and in situations where decisions obviously depend on the different risk attitudes whether these are identified and highlighted and in fact there is an IMI project called Prefer that has worked on this concept quite a lot and before this MDIC in the US has produced guidelines which kind of led to also FDA guidance in this area in the area of diagnostics recognizing patient sensitivity so I think why why this was true some time ago I think the regulator nowadays tries to really focus very much on patient perspectives and when we make our decisions we do consider patients who are yes let's say most tolerant to have attitudes of more tolerance to harms and and vice versa that value most benefits even benefits that on a wider scale maybe are not considered to be the most convincing say for other other decisions and again referring to some reports we have been come under scrutiny because there is no universal value given to these endpoints in oncology some progression for survival or whatever but for me what's more important is is how to move forward and I think in many situations we have to recognize that all the regular regulator can do is to describe the effects and then make sure that these are taken into account at clinical level when making the clinical decisions using the individual preferences of the patients and I think that's how we need to work and perhaps even improve our communication of of our assessments to make them even more useful in the clinical context hmm thank you so much um I'd like to just come back to one point that I've heard a couple of times from a number of you and we hear that very often it's about quality the quality of the data if we think about um uh we often hear the methodologies not sound the the quality of live data is not of good quality enough to actually take account of that in regulatory decisions so there's this whole discussion about are we giving enough guidance about the collection of pro data and I know Kate probably first to you because your third recommendation from MP was around recommendations on the collection of pro data what's your perspective on that and then also probably Beata and Francesco to comment on that are we asking for too much regulation or do we have too little I think in in terms of the guidance point um I think as I said in my presentation our perspective in the patient advocacy community is that there are lots of different recommendations on how to to gather quality of life data and patient preference data um and big asks from um for industry and other researchers um and a lot of the direction of travel is set by the FDA um currently and I think um HDA bodies as well ask as well as the regulator for specific quality of life data so you have nice requirements you have IQUIGs requirements and the where the recommendation came from was whether there was the potential to to align on a European level broadly on how how we should be advising people to generate quality of life data in a way that benefits everybody um and I think that's what we would like to see um other things we thought about were around the potential of um through the early dialogues with the um the EU regulation on HDA and whether that could be a place where advice is provided to industry um in collaboration with patient organizations and the EMA on how to um to come up with a quality of life approach um and I think scientific advice is a core area where um where industry can engage with um with regulators and HDA bodies to really think about meaningful data and to make sure it's good quality um so I'd like to hear um Francesco and Beata's perspectives on the potential for guidelines and whether that is a good recommendation or or not. Francesco do you want to respond to that? Sure I think it's it's an excellent point and and like Sam very very nicely described we're going from a from a mindset where people were saying no quality of life data too poorly collected too biased too but yeah there might be some problems with missing data and perhaps even bias but hey all data have problems and we can look at making the best out of them and improving them and I think there is this um strong collaboration going on now uh precisely in this uh I'm sort of referring to the IMI CISACOL project where they are looking at uh harmonizing these uh aspects of data collection design collection and presentation I think it's hugely important much more than than coming up with a sort of rigid rule of what's okay or not okay or which which tools are fine or not fine if you multiply the the aspect of tools with the the analytical uh variance you have practically infinite uh application of quality of life approaches so I think the key is to develop principles that we can all agree to so not not too detailed guidelines but things that can work in most settings and I think together with academia uh this is very important and and yeah just to mention that there is also the the plan to develop some of these guidelines within the context of the ICH which is a context which is you know becoming more open to external organization also and more systematic in considering input of external organization and it's a really it's a it's an invitation to to to ERTC and and and other other organizations who are active in this field to try to to to make themselves visible in these international contexts if not lead them thank you. Probably just asking Simone about this um consistency of quality of life tools and Francesco mentioned mentioned ICH probably as a platform I mean what's your perspective on the consistent use of quality of life tools referred to CISAC while referred ICH and also in the in the chat people are commenting about the use in real world evidence uh which actually is outside of typical clinical trials but it's probably very important evidence even less structured than in clinical trials that could inform regulatory decision making what's your perspective on consistent use of the tools? Yes well um I think it's very important to like you have two different situations and like in the last five years I think there's been more attention to develop standardized core outcome sets and there's also a large project where NPA is involved in the harmony alliance in which we tried to to develop a core outcome set of outcomes what matter most to patients and then you think of survival outcomes is also patient reported outcomes and then set a kind of standard that is a minimal uh uh outcome set that you would like to measure so I think those kind of developments um because there has been two more develops in developments in Europe like that like there's the importer project in Spain they've been developing such a core outcome set for for clinical for clinical practice and also in the Netherlands more attention to value-based health care and also developing ITEM sets and I think those sets thinking with like a multi-disciplinary team uh of course including patients, patient advocates and also all people in the field of expert on a specific disease that you can come to to a minimal important set that it's something that should be collected uh for every patient and of course this the the scope of this was more um outside of clinical trials but I think we can learn of those two worlds what you're being used in real-world data collection but also in clinical trials um then another important thing is and that's a difference also in clinical trials I think that you really have to to look at the research question like what are the side effect profiles belonging to new treatments that you want to compare and really make sure that you include a patient reported outcome measure capturing those side effects so uh what we saw in the presentation was that most patient reported outcome measures that have been used were general outcome measures and I think those are important to get a general understanding of the quality of life of patients but if you want to have if some treatments have specific profiles you also need some uh some tools some patients reported outcome measure to really capture those so I always think that um always go back to your research question and what do you really want to to measure and what what outcome do you expect a difference or change that's also important thing that that's something that we should think of in in in future I don't know Lisa if you have any comment about that coming also from the data that you've looked at I mean you mentioned also in the in the area of the nice appraisals issues with quality of life data in uh technology appraisals did you see any data from your your research that actually supports or drives standardization of use or addressing the quality issues about quality of life data well nice specifically is very uh clear about standardization they want and they're very very simple they just want the q5d so whether it's a good solution whether it's addressing the issues we're discussing here is another question because of course there are a lot of limitations to these generic tools and it's needed for the derivation of utility values for economic evaluation thankfully it's an easy tool to use and it's good to include so but the question is it's not whether eq5d solves all the problems the question is what else do you need to elicit the actual meaningful measures from patients and I think there are some interesting comments in the discussion whether how real world evidence can be used and whether additional prompts should be developed so here I would put a big red button and say hold on hold on we have 93 tools in my loma so let's look into this and let's try to clean it up and let's see what why is this so much who's making this decision where from which shelves are they picking this when they pick the top five I'm clear what is decision-making process when they are going for the rest of 88 I really wonder and there are some specific issues why why but it's already a big mess I also would never say stop never develop new prompts in my loma the field is changing we're getting very new interventions that we don't know what they're doing to patients so we need to stay agile but at the same time I've seen quite a few companies once again let's not specifically focus on my loma here who come to nice with a lot of excitement saying oh we're developing our own tool and that's not helpful either because of the validation because of comparability issues so I would say the field would definitely benefit from further regulation and standardization but within reason because we can't just say okay this is one formula to call these boxes you get through or this is the right thing to do and one other comment is just in terms of quality of the choice of tools the very few situations when quality of live data collection was no it should have been at the right times right number of patients we have comparative data we have something very clear in the reality what happens is sometimes companies or other organizations have to set up dedicated quality of live data studies which are also very important and these need to be very carefully designed as well they actually if you embark on something like that but these can really elicit important information for this group of patients but also there's a good question that I tried to type an answer into whether what do we do when it's open label study what do we do if it's an uncontrolled study is this even useful to collect quality of live data and there's a very legitimate questions because correct the data that we'll get from these studies might be by definition by design of not good quality what do we do with it so and we need to be quite careful not to say oh because this is phase two single arm trial don't collect quality of live data it's very important to collect it because when nice looks at it when other decision-making makers look at it they start looking at pooled data from the trials who involved patients at this stage or with this drug so but it's not helpful if this stage stage two trial used one instruments stage four trial used other instruments so we need to get this kind of communication and standardization into play and care be careful not to over burden patients but being doing things fit for purpose and going back to the suggestion by Kate yes scientific advice joint advice from regulators and HTAs consultation with experts it's important to discuss it that's where companies or academics get those guidance and my worry is actually more academic studies because for them the concept of scientific advice doesn't exist and there is no function to go through that and there is very little dialogue where can they get this input so that there are a lot of gaps basically that's probably a question to use some because academic I mean how can we get the academic world to use more pros which I understand is also a criticism we're always talking about industry and registration studies and so on so how can the academic world actually drive the use of pro instruments do you do you share that kind of view that there is a problem yes absolutely I mean the problem with academia yeah and as you know they all have turned into corporates so the ethos of the academia has changed they're the only way that you can actually do some really substantial research is to be able to get the funding and unlike when you're actually involved in biomedical research getting funds for doing these type of studies is much more difficult than if you're really doing bench research so that's I think the major problem in academia as I can I have been very fortunate because of my contact and number of other activities that I've had you know I always manage to get funding from industry to be able to do these sorts of things so that's the answer to your question but there are three things that I really want to mention in relation to what other speakers have mentioned now I really like the Francesco's idea of influencing ICH coming out with some guidelines regarding you know the use of the PROs in clinical trials and I think that also fits in with Kate's number four recommendations I think if we do develop it the level of the ICH it will have some teeth and I think you know everybody will you know follow ICH as you all know in the major countries in the world now the other issue that I like to mention when Lisa's comment we really got to influence NICE to move away from EQ5D it's a terrible measure the NICE knows that it's a terrible measure and you know we have done a number of studies you know looking at the EQ5D but also it's intended for at least a set generating utility values is not a purely specific quality of life measure so you know we got to you know move away from that now in terms of the number of 93 tools that Lisa managed to find in the study now that's another issue that we need to tackle you know perhaps you know at the level of the International Society of Quality of Life Research we could take that you know our responsibility again we have to come up with a handful of tools and also people should really be prevented from you know going on in developing new tools this really creates an anarchy in the whole field of the quality of life we have to have a few very robust developed you know using the right methodology and making sure that the precision of these measure if not better just as good as the clinical parameters now in terms of the generic measures you know we have to be again look at it very carefully a lot of the measures that are used in hematological malignancies vis-a-vis you know multiple myeloma they have not been specifically developed for hematological malignancy they have been developed for solid tumors and then some questions have been added for you know hematological malignancy at the later stage and now the only one for example has been specifically developed for multiple myeloma is myPOS and of course you know HMPRO which is developed specifically for hematological malignancy now that is a generic measure for all the hematological malignancy and you know there are a number of studies have been done to look at that you know they discriminate between different hematological malignancies now what I'd like to leave you Jan with perhaps we also need a webinar for the use of the patient reported outcome in clinical practice I think that's also equally important as this correct use in clinical clinical trials you know we need to do you know if you like tackle these you know at the same time in tandem because the more patients become aware of the use of these in clinical practice on day-to-day you know clinicians dealing with individual patients then also their expectations would be that that information would also trickle down into the clinical trials and I know Sam you have been a pioneer with HMPRO actually to also try to drive adoption and clinical use and clinical practice use of patient reported outcomes so that's that's very important I mean Kate um what I hear from the group is saying we don't need number 94 of pro instruments we need more effective better quality use of what we have and I remember you presented recommendation number two about creating a PROM platform to on the use of pros and what kind of pros exist because quite often there don't seem to be awareness about doesn't seem to be awareness about which PROMs to use when is that the intent or what's what's your perspective sorry I was on mute of course I was so that was the intention so I think um having an online system so I think there's something in the UK called PROMs where you can um if you want to to use quality of life you can kind of pick and choose like tools that are appropriate to your um research question and I think that's that's kind of how we envis envisioned it is that it would be something where researchers could go on and see the main tools um and perhaps there would be a checklist where it's like patients have validated it patients support its use it's been developed specifically in myeloma and potentially and I'm not sure how possible it is but I'm kind of specifying how it could be used and which treatments are most appropriate for so for example in CAR-T we're struggling to measure that appropriately because all of the tools were developed before CAR-T was around in the side effects are very new and so whether there's whether it could go into that kind of granularity or whether we could link recommendation two to recommendation four where we have a really a really high level consensus amongst clinicians, PRO methodologists, patient groups on the the key PRO tools in myeloma in which treatments they could be used in I'm not sure how likely that is or how granular that we could get but I think that's how we see it is is I think we need we need to steer the ship a bit more in myeloma with PRO selection and at least work kind of the thoughts we had around doing that. Super thank you so much I don't know Beatty if you want to you had your hand up quite a while do you want to comment straight on that on something before sorry. Yeah well actually that was on a on a different issue but maybe maybe for for that point I think I would very much support the idea of core outcome sets and this could then also include the approach to the quality of life and PRO instruments and I can see your point that there might be new issues coming up with new treatments but in the end probably you could break down that to to generic principles that would also apply like probably it would also include pain and fatigue and anxiety and things so so general concepts which you could measure with which could still very be a very good reflection of the situation in which the patient is even if this comes from a from a new type of treatment so that might be the way to go rather than try to specifically figure out what what is new with this treatment. I I actually I wanted to follow up on an on a point Alisa was making and that is concerning your first ring of recommendation in which you say move quality of life at least to phase two I would say ensure that you have studies that allow interpretation of quality of life data and interpretation of quality of life data is only possible in controlled studies so it doesn't help if you move quality of life to single arm studies that will get you nowhere I mean that is in principle true for many endpoints but specifically for PROs and quality of life it will be important that you have controlled studies otherwise I wouldn't know how to interpret this type of data thank you so much Beata and I think let's say robust data is the core of everything so thank you so much for for making that point I know that we're almost at the end of time for this webinar I would love to go on for another hour because I still have a couple of topics that I'd like to pick your brains on but unfortunately I mean people need to do something something else probably after this exciting webinar so I have a last question to each of you because I mean patient organizations need to focus their resources they need to think about where can they have highest impact on what they're doing and MP has actually presented four recommendations first one was about researchers should consider the collection of quality of life data on in all myeloma clinical trials at least from phase two number two was research to collaborate on the creation of a user-friendly database on patient reported outcome measures number three was clearer recommendations on the collection of quality of life data and clinical development to support regulatory HTA decision-making and number four was actually a more international multi-stakeholder steering group comprising of patients clinicians pro methodologies regulators and policymakers to actually allow a consistent approach to a collecting quality of life data so Christmas time just wish one um what would you recommend mpe or the myeloma community to focus on if you have to choose who do I pick first Simone can I ask you first of course and I think I have to answer shawky because you're pretty nice but um I would try to to see if we could raise the percentage of using this very important patient reported outcome the patient perspective in evaluating treatment who's raising death percentage super thank you so much Lisa you want to go next uh yes I'll go with number three potential to have clearer requirements on the collection of ql data because I think it's the most doable and might lead to practical results faster I can't avoid is this also your recommendation francesco I agree with Lisa but I want to move away from the term requirement and more to agree on sound scientific principles on what scientific question to prioritize and what tools we can use to answer them would this been being your recommendation also I biased your response so would have to out of the four would have been that being your preference yes yes okay very good okay very good beate what do you think well actually I would also go for three um and would want to add that we should achieve that we handle quality of life endpoints as we do others because we know how to how to properly collect and analyze endpoints and we would just need to move this type of endpoints to that level super thank you so much and finally sam what's your perspective yeah I would I would also suggest number three but also adding to it I would like the kates group the my number patient europe to also be more involved and all the stakeholders who want to do clinical trials to utilize the expertise that exists within the myeloma patient europe to in terms of you know what tools to use for the clinical trials and what time points I think the industries in particular should utilize the sophisticated expertise that the patient organizations have these days to advise them you know for the use of the pros um in the design of their studies okay super that's great so finally Kate you got the final word you heard all the wishes and and the choice out of the four so what's your final recollection before we close the webinar I think um we'll take three forward because everyone is in agreement we need guidelines and the other thing I was thinking um was around Lisa's comment about academic um trialists and whether there's also as well as having pan european guidelines we need to do something more as a community um in collaboration with the emn and eha to disseminate the the message around generating quality of life um and pro measures through clinical trials and I think that's something we'll take forward um but thank you all so much for um the really helpful feedback on the recommendations and we'll um send you the report on monday super thank you so much you've all been fantastic I can't wait to listen to the recording again and thank you so much for being with us today and see you soon in the quality of life sphere thank you so much


So by now I know that I'm super late to the party on this, but I had to talk about this because Donald Trump is losing his mind And I know it's Donald Trump. So he already lost his mind You can't lose something that you already didn't have in the first place But his attacks on Joe Biden are getting more and more unhinged to the extent that it's forcing Laurie Ingram To call him out for being too conspiratorial or too extremist So he contends in this interview that Joe Biden is being controlled by people in the dark shadows I'd like to mention Biden because he's not controlling anything. Who do you think is pulling Biden strings? Is it former Obama people that you've never heard of people that are in the dark shadows people that what does that mean? That sounds like conspiracy theory of dark. No people that you haven't heard of there. They're people that are on the streets They're people that are controlling the streets We had somebody get on a plane from a certain city this weekend and in the plane It was almost completely loaded with with thugs Wearing these dark uniforms black uniforms with gear and this and that there they're on a plane where so I'll tell you sometime But I it's under investigation right now Okay, I Don't even know how to process that it was already stupid enough for him to Say that Joe Biden is being controlled by the far left if that were true Wouldn't we have him endorsing Medicare for all and agree new deal and pot legalization?  What are we talking about? Well, he was asked to clarify and it gets even more confusing There are people that are on the streets people that are controlling the streets. Okay, maybe the protesters We had somebody get on a plane from a certain city this weekend and in the plane It was almost completely loaded with thugs wearing these dark uniforms black uniforms with gear and This and that they're on a plane Okay, but like what does that mean? What uniforms are they wearing? Was there any insignia on it or logos? Like are we supposed to believe that? In antifa army got on a plane and they were like going to some city to do violence What am I supposed to take away from this? You are the president of the United States What do you want us? To learn from this it doesn't make any sense logically. I can't wrap my head around it. I don't know what you're trying to say Use your words Don.  Like this doesn't make any sense Does anyone believe that Joe Biden is controlled by the shadow people like do we mean like actual Monsters shadow monsters. What do we mean? Do we mean advisors corporate interests?   I don't know what that means. Does anyone know what that means?   You're too far gone. But I mean, we already knew this About donald trump. We knew this But does anybody actually believe That this is going to be persuasive like do your viewers or followers like do they know? What you mean by this is it like you know when when you are talking to a toddler and they're trying to tell you something But you don't understand it, but the mom knows like are your Supporters like the mom where they know exactly what you mean by this. I don't get it like what are you trying to imply?  He's clear. He's saying when I say lizard people I mean literal lizard people people who are wearing human skin underneath their reptilian bodies Like we know what he's trying to say but like with this. I don't even know what you're saying It doesn't make sense somebody got on a plane with thugs wearing black uniforms. What uniform are you trying to convince us that Antifa is like forming an army.   I'm unsubscribing


Ik denk dat deze nog is. Oké, dank je wel. Dames en dames, ik wil een kort presentatie geven over de bezoekers en de bezoekers die een aantal ongeving, monitoring initiaties ontdekken hebben in de Nederlands tijdens een study. De fact sheet was made together with a number of colleagues of mine from TNO also in collaboration with Mr. Driemann of Fubro Geoservices. The study of which I present the conclusions now was performed by the SBR CURNET from the Netherlands it's a Dutch association with partners from building industry that initiates research topics, stimulates innovations develops guidelines and disseminates knowledge and one of the studies performed by SBR CURNET was done by committee 1992 and the question to be answered was how can monitoring be applied and promoted in construction and maintenance. The committee consisted out of numerous parties from TNO but also Delft University governmental parties municipalities and building and consulting industry. So the idea was to take a number of ongoing monitoring projects and have a closer look at them answering questions like are they successful or not are their opportunities are their threats and if so why and can we then extract lessons from that. Now how did we evaluate these monitoring projects we decided to do that based on a scheme for monitoring I guess this is one of the many schemes that you can think of in case of monitoring and probably this one also has some flaws but it turned out to be very useful in structuring our study about the monitoring systems like the point to the top of the scheme where we see on the left an objective a goal that is searched for which should be balanced with the parties on the right hand side giving the supply of information of monitoring systems the flowchart that you see on the screen has a number of blocks and they also labeled with ABC and so on en these labelings will be used later on in presenting the state of the number of ongoing monitoring systems Normally one should start on the left hand side of the scheme in defining the objective so one might think of maintenance optimizations service life extensions gaining insight in specific mechanisms and owners or stakeholders are then interested in how these goals and objectives can be optimized and managed based on proper decisions and measures to be taken for instance optimized maintenance policies operational measures strengthening renewal but also was also introduced this morning this afternoon gathering information for learning so designed by experiments or introducing innovative projects or products coming to the right and once these objectives are looked for you also look for enabling and supporting models that describe the problems at hand and normally these models then also give incentives in proper quantities that can be measured if needed examples of these models quantities are models for threat close or threat close en if these give a need for extra information you can go on to the definition of the requirements of the monitoring so that will be a stage of phase C in our scheme going further towards the design and build of the monitoring system the operational phase with respect to acquisition and we are almost there but then of course it's also a matter of getting the information out of the data and giving the data and information towards the model to optimize then the decision phase stage G apart from the scheme just presented we also looked at some additional aspects that are important in classifying the state of monitoring systems and they are listed in this sheet on top there is the interests and needs of the owner or the user of the objectives that's also including the higher level societal and economic needs and interests then of course opportunities and threats what efficiencies and successes are found but also what kind of factors are found that are hindering a large scale application of structural health monitoring the time horizon is it temporarily or indefinitely in time a cost benefit of course quantitative with respect to for instance single cost or initial cost but also what would it be if one starts upscaling the systems data management is also found to be very important for instance is the organization ready to deal with these large in general large amounts of data is able to manage it and couple it to existing information and last but not least but also the technology readiness level, the real level of the systems to what extent the system is ready for repeated and also organization wide application so far for the scheme that we applied so it's about time to show some results it's probably readable in this matrix we see five projects that we're all dealing with maintenance optimization as an objective so some examples for instance a sense of system to measure the light intensity of a tunnel in Rotterdam with the goal to optimize the cleaning frequency this system came up to phase of stage C so it is not operational not found to be operational yet it's probably due to the fact that it has to be incorporated into the organization there are two examples of joints in highway viaducts again with the goal to optimize maintenance frequencies also these, now one of these two is still found to be in proof of principle and proof of concept stage and the second one is indeed already running and is successfully applied and there is a classical one chloride penetration in concrete we are in a feasibility study with respect to this so phase D some pictures has the picture of the master in Rotterdam with the light system or lighting system we were talking about oh this one I did mention I'm sorry about that but this is about tags or radio frequency IDs that enable the identification of structures but also components within structures and there and thus also decrease uncertainties that can take place for instance during inspections so it's more with respect to data management and a little bit out of context with monitoring maybe a second set of results is presented by this matrix we are now looking at a different objective in this case we are looking at service life extension examples shown here involve for instance on top the moving apartments of a bridge where there was a need of gaining the insight in the mechanisms taking place again a bridge joint whereas we of the people working on it successfully applied the monitoring system and as such could delay the maintenance by and also then successfully have the replacement be combined with other maintenance actions going on on the road there is a steel deck of the van Brienendord bridge in Rotterdam where the crack cross is monitored for those of you interested there is there was a paper also included in the booklet of the first workshop in Copenhagen about this and then again one for concrete silica reaction deterioration also in a highway and most of these projects presented in this matrix are running up and running successfully except for the first one where we did not yet or they did not yet found conclusive answers about the mechanisms looked for so this is a picture of the example with respect to the moving apartments this is the example of the crack cross modeling and monitoring in the van Brienendord bridge in Rotterdam a third set of projects that we looked at is presented in this last matrix it's about getting inside in behaviour of systems and structures first one is an interesting one where we are now introducing synthetic of fiber reinforced plastic lock gates in hydraulic structures in the Netherlands and one of the questions that is to be answered is how can load modeling be properly designed and applied from the regulations point of view again also let's say a kind of innovative path that is worked is with respect to high string concrete high strings concrete applied at the Styx bridge in Blaricum inside in the creep and shrinkage behaviour there are the autotropic steel elements in the helicopter bridge in Utrecht where we want to determine the additional load bearing capacity and there is a way motion system for several places in the highways in the Netherlands with the goal to evaluate the actual traffic loads we don't know they are still in compliance with the Eurocode all of these are basically all successfully up and running in the Netherlands a picture of a way motion system and also a picture of temperature data that is gathered for the synthetic lock gates in Blaricum so some conclusions about this study first of all the scheme or the flowchart it turned out to be a very useful guiding or guidance for appraisal it facilitated the monitoring system design balancing demand and supply it also facilitates the comparison of different monitoring systems getting feedback communication with respect to monitoring systems the monitoring systems themselves most of them turned out to be composed out of the shell components they are in general in general there are a lot of pilots in it by pioneers mostly research driven some of them are also business driven not yet always labeled as successful and what a big lesson learned is that if there are monitoring projects with clear objectives and a good interaction with supply they in general turn out to be more successful with respect to the application and upscaling I think this was also mentioned today that sometimes or maybe too often there we feel or find resistance from a practice a number of courses could be mentioned sometimes it's argued that problems are not urgent enough so in terms of let us proceed the way we always did sometimes the problems are too urgent so then they don't want to take any risk better safe than sorry why start monitoring if we just can place the element and finally the last two they always respect to the costs and benefits it's about who's going to pay and who's going to benefit that's also an important issue that plays a part in the application and upscaling and that's only respect to who and whom but also respect to time often you have to invest a lot of money now and be rewarded much later and maybe you are not the one who gets to reward so these conclusions I think they put some pressure on cost action of what the way that we all together are working on it should indeed enable better the applications of monitoring systems as we are all working on a more structured approach putting the objectives in front with a good balance on demand and supply and last but not least also respect to the costs I think earphones are always a good argument to get things done so the quantification of the value of information is very important for getting monitoring systems applied thank you very much ok thank you very much for sharing this experience from the Netherlands so as for the first part we'll have a short discussion I would like to steal 5 minutes of your time of the coffee break and I would like to invite the 3 speakers here to join me on the stage so in the second part we've even widened the scope of structures