Hello, and welcome to Esmar Conf 2022 and our last special session for today, Quantitative Synthesis with a Bayesian Lens. As always, this session is being live streamed to YouTube and individual presentations have been pre-recorded and published there as well. Subtitles have been verified and can be auto-translated from those individual talks and automatic subtitles will be available shortly for the live stream. If you have any questions for our presenters, each presenter has an individual tweet on our Twitter feed, which is at ES Hackathon. We will keep an eye on questions and endeavour to answer them at the end of the presentations. We would like to draw your attention to our code of conduct, which is available on our website at esmarconf.github.iu. Our first speaker today is Franti Shek Bartosz from the University of Amsterdam, and we'll pass it over to Franti Shek. Franti Shek Bartosz, I'm a PhD candidate at University of Amsterdam, and I would like to tell you about troubles-based meta-analysis, a way of combining multiple methods of publication based adjustment, and drop my R package that implements the methodology. First, I will tell you about an example that compares register application reports and meta-analysis, and I will use it to highlight the problems and challenges of adjusting for publication bias. Then, I will tell you a bit more about different ways of adjusting for publication bias, how we can combine them with Bayesian model averaging, and lastly, I will tell you more about the package implementation and functionalities. So, the example, a requirement in colleagues 2020 published a paper that looked at 15 different meta-analysis and register application report that tried to replicate the main study from each of the analyses. And under some other assumptions, you would expect that the register application report should provide the best possible estimate of the true effect size, and that the meta-analytic estimates of original studies should converge to it. The original meta-analysis were of different sizes, and they range from 15 to around 300 studies. And if you look at the original effect size estimates based on the published studies, you can see a wide range of effects, and since this is a doctoral publication bias, unsurprisingly, you see that the estimates from registered replication reports were much smaller than the original effect size estimate based on meta-analysis. So, this large discrepancy is by many attributes of the publication bias, and one way that you can use this example is to see how well what different publication bias adjusted methods adjust for the publication bias and provide estimates closer to the registered replication report estimates. So, publication bias adjustment is a topic that has been here for many decades, and there have been different methods implemented and developed that try to adjust for it. I like to differentiate them in two different camps. One camp are method that adjusts for relationship between standard errors and effect sizes, for example, term film, bed-piece, or indigenous kink, and the second groups are selection was of p-values that tried to adjust for publication as operating on p-values, for example, 3 and 4 p-sem, 8,9,8,2, p-core, p-uniform. So, I will just go through those in terms of bed-piece and 3 p-sem. So, bed-piece is a conditional meta-regression estimator that tries to adjust for relationship, effect size, and standard errors, or standard error squares. And the idea is if there's no relationship between effects and standard errors in that happens when there's no publication selection bias, then if you fit the regression estimator, the intercept should correspond to the true effect size. However, if there is a publication bias selection, then you see a higher number of small studies with large standard errors and overestimate effect size estimate, and then fitting at your pet model or piece model should provide a much better effect size estimate. The selection models on the other end extend the traditional random effect or fix effect method model between parameter mu to heterogeneous parameter tau and publication bias weights omega. Here, for example, we use a step weight function that specifies different publication bias probabilities for different p-value intervals. Here we can fix the relative publication probability of significant studies to one and we can estimate the relative publication probabilities of the marginal significant studies or non-significant studies. As a result, you obtain a different likelihood function. So the F would be the unweighted likelihood function while the FW is the weight likelihood function that takes the different publication probabilities at different p-value intervals into account. Why this approach is interesting. If you look at the one million published set statistic on Medline, you can see a very similar shape that shows two very large discontinuities, maybe accidentally at the alpha 0.05. So if you use those two very popular methods and in the example of Quaran, we can see the original effects estimate in red, the original publication estimates in blue and then the black circles that are per piece and black triangles that are 3 BSM. And in some cases, all of them will provide the same estimate in some cases, 3 BSM is better in pet piece and in other cases, pet piece is better in 3 BSM. The problem is apriori, it's hard to tell which of the estimates is better. So the question is, how should you base your inference, especially if the methods disagree on the conclusions? So we argue that you shouldn't show or you shouldn't base the inference on a single model. Instead, you should use robust based method analysis and patient model averaging to base the inference of multiple models simultaneously. So instead of selecting a single model, you specify all of the models, you fit them and you base your inference proportionately on how well the different models predict the data. Then use base factors to quantify the evidence in favor of the presence or absence of either the effect, heterogeneity or publication balance. You can use prior distribution to regularize the estimate and incorporate prior knowledge and use Bayesian evidence ablating. That's independent of the sampling plan. So in an overview, the Bayesian model averaging works something like this. You have different hypotheses about the data that are represented by the different demons and some of the demons specified hypothesis. For example, this demon says so, the treatment works. So alternative hypothesis is true or there is no effect, the line hypothesis is true. And you have kind of different assumptions about heterogeneity, the fix and random effect models or different assumptions about the presence or absence of the publication base. So you specify all of those different hypotheses by the different models in your ensemble. You feed the models with the data and the models that predict the data best will grow and their voice will be heard much more. So if the model predict the data well, it, you will base the inference much more often on it and much more strongly. So how we split the ensemble, then you, for example, when obtain the model average estimate, you can look at a different components in a way that model specifier absence or presence of the effect. So you can split the primary problem equally across those two model pairs, then against equally across models, assuming presence or absence of the heterogeneity and then the publication base. So at the end, you end up with eight different model types that specifying all the possible complication or combinations of either the presence or absence of the effect, heterogeneity and publication base. So each of the model types ends up with the same primary problem. But as I said previously, there are different ways how to adjust for publication best. So in our illustration of the demon, this one demon can represent in many ways. So what we do, well, it just surface all the way down. We just specify more models that represent this one demon here. You can specify, for example, the pet model as one way of adjusting for publication best. The piece model or different weight functions. For example, one-sided selection on one-sided p-values on the significant level or selection on marginally significant and significant p-values on the two-sided p-values and all different weights. So if you look across all of the different publication best adjustment methods that we specify in turbospecial meta-analysis, we use the pet and piece, publication adjustment models that adjust for each effect size and standard errors. Then we specify six different weight functions that specify different assumptions about the different possible ways publication might operate on p-values. So, and all of those specifications cover approximately the pet piece, 3-psm, 4-psm, AK1 and AK2 models. So if you look back to our example, we can see that in some cases, all of the methods provide still the same estimate. In other cases, the problem provides an estimate that's somewhere between the pet piece and 3-psm. And in different cases, we can obtain again estimate that somewhere in between, but it's not greater than one of those methods. That just signifies that we are still doing statistics, not magic, and we cannot provide the correct answers all the time. Nonetheless, in simulation studies that are linked at the end of the presentation, you can see it in majority of cases, Bayesian model averaging provides the best possible results. So we, to make this methodology available to the practitioners, we implement it in the ROMR package, and the package uses MCST mission with checks, using the run checks R package, and then computes a margin like that's better for example, R package. And the most things that the ROMR package does is the model specification, some plotting summary function that I will show you in a second, and additional auxiliary stuff. So, the ROMR package, you can use it to specify the default answerable. You just specify the effect size and standard error. So for example, here on the infamous BEMS 2011 data set, you fit the model just with a single simple call, and you can use a summary function to obtain some default summary of the model. Here, you can see that in the first summary table, you see information about the whole model ensemble, and you see that you specify 36 models, 18 of which are some presence of the effect, 18 presence of the eternity, and 32 presence of the publication base. The paramount robots are equal across the component, and you see the possible probabilities. You can also quantify the evidence with the question-based factors, and you see that there is very weak evidence for the absence of the effect, moderate evidence for the absence of the originative, and strong evidence for the presence of publication base. And then of course, you see the model average estimates for the mean and heterogeneity parameter, and then the publication base probabilities and patent piece estimates. So, moreover, the package provides additional summaries. For example, you can look at the summary of individual models that shows you the prior distribution for the effect at differentiating the publication base, the paramount properties of each of the individual models, marginal equals, posterior probabilities, and inclusion base factors for each of the models. Also look at the MCMC diagnostics of the individual models that show summary of the MCMC error minimum effect of sample size and maximum overhead from each of the models to verify that the models were fitted properly. And you can also get estimates from the individual models. So here I'm just showing a print of the last two models that are specified, and you can see the model specification, the parameter estimates, and et cetera. So even if you don't wanna do divisional averaging, and you wanna look at the individual models, you can look at the estimates from the individual models. Then the package also provides plotting functions. So for example, you can plot the model average mean estimate where the spikes correspond to the probability of models assuming absence of the effect. So the effect size is zero, and then the slab corresponds to the density of models assuming presence of the effect. The functions are also implemented in Gigiplot. So if you are a fan of Gigiplot, you can use those. Or you can also look at the prior and posterior distribution. For example, here for the tau estimate assuming presence of the effect and many other combinations. The package allows multiple different specifications and you can modify basically everything about the ensemble. For example, you can change the prior distribution for the effect and specify truncated normal distribution that specifies a hypothesis of small effect sizes to mean zero standard deviation of 0.3, forced to the interval of zero to infinity. Or you can specify different base of adjusting for publication. Here you can specify only prior distribution that specify the pet and piece of models that adjust for publication. If you wanna see more different specifications and customization, I recommend to check the variance of the package that are on CRM. You also implemented the R package in GES with graphic user interface. And the GES implementation allows you to set basically all of those customization also in GES, specify the models for different prior distributions. And then you can create different summaries for inferences, different figures. Here you can see again, the model average mean effect size estimate and also the model average weight function estimate across all selection models. So just to sum up something about the Roost-BasicNet analysis, it can incorporate uncertainty about a selected model with Bayesian model averaging. So you don't have to base inference on any single publication that's adjusted model but on all of them and how well they predict the data. It can provide evidence for either not or alternative hypothesis. It has better preference with small sample sizes. It's the capacity to incorporate expert knowledge and has the potential for sequential reading of evidence. On the other side, there are some disadvantages. For example, it's slow. It requires MCMC sampling and it can also fail under strong P hacking. So thank you for your attention. I hope you enjoyed the talk. And if you wanna learn more about the package, you can also can either look at the Siren where the package is released or my GitHub page where you can also submit a feature request or back reports and look at the GESP. There are some references to the papers that we have written that outlined the methodology in more detail and tell more about the model specification or simulation studies that we further conducted to verify the methodology. Thank you very much. And looking forward to see you in discussion. Thank you very much, Franti Sek. And we will now hear from Christian Rover from the University Medical Center, Göttingen over to you Christian. Yeah, thank you very much. I'm going to introduce the new meta regression features that are now available in the base meta R package. And I first introduced the base meta package briefly. I'll talk about binary co-variables and then go over to different privatizations that you can use. I'll also have an example with continuous co-variables and briefly sketch other advanced applications that you can approach. So first of all, just briefly, so that's the base meta package as it has been up to now. So it implements the normal normal hierarchical model. And what you assume here is that you have a number of estimates and a number of standard errors, YI and Sigma I. And the assumption here is that they are imprecise measurements and the uncertainty you have or the standard error measures the measurement uncertainty here. But the true means here are not necessarily the same. They also have this certain amount of variation or heterogeneity that's quantified by this heterogeneity parameter tau and the overall mean here is mu. So in the end, we have two parameters or two unknowns here that also require prior specification if you want to use a Bayesian approach here and that's the overall mean mu and the heterogeneity tau. And that's, yeah, the model that has been implemented in the base meta package so far. In the base meta package, we are not using MCMC but it's, yeah, the calculations are based on a trick if you want and you get out the posterior densities and quantites and so on directly without having to sample here. Yeah, now the simple normal-normal hierarchical model can be extended to meta regression as well and the extension is sketched here as well. So what we have is again, we have estimates YI and standard error Sigma I. But in addition, we also have co-variables or moderators or study level regressors here that are called X1 to XD. And the assumption or the model looks similar here as you can see just in the second equation here, we see that the overall mean is not just a simple overall mu parameter here but instead the study-specific means are determined by or come about as linear combinations of these moderators here and we still have the heterogeneity parameter and yeah, the change in the model means that now the parameters, the open parameters that require prior specification are still heterogeneity parameter tau and also the regression coefficients beta which is now instead of a single overall mean is this D dimensional vector of regression coefficients. Calculations work similarly as before and the new approach now is implemented in this BMR function here. And I go straight to an example. So the example here is a systematic review that was performed in pediatric transplantation and the endpoint here was log odds ratio of acute rejections. So that's the event that you would like to prevent by the medication and you can see we have estimates of log odds ratios here. They are all on the negative side. So the medication seemed to work and so we have six studies included here. We've got six log odds ratios along with their six associated standard errors. And we can also see that two medications were actually used here. So there's similar kinds of medications but one is the Clisomab, the other one is here Vasileximab. Yeah, and if you want to account for these two medications and estimate individual effects, individual treatment effects for these medications we can do that using meta regression and what we need to do is set up this matrix or these covariates and in this case we can simply set up a matrix as we see at the bottom here. So it's six rows for the six studies and then two columns for the two means that we want to estimate and we can see that we have zeros and ones encoding what treatment group or what medication each study belonged to. Yeah, now implementing that in the base meta package is pretty quick here. So first of all, we need to prepare the data of course. So we need to derive the log odds ratios and standard errors, that's the first bit here. Then we of course need to specify the regressor matrix, the covariates here and that's unsurprisingly it's the matrix that we essentially saw on the previous slide. So it's two columns, six rows for the six studies and two columns for the two treatments. And then the function call eventually is this two line call here supplying the effect measures here that we generated in the first step then the regressor matrix, the covariates here called capital X. And we are also specifying a prior distribution for the heterogeneity here. If we omitted that bit, then we would be using a uniform improper prior which is also okay in certain cases. And yeah, so note that we are running the analysis here. So we're calling the BMR function and we are assigning the results to this BMR or one object and we can use this object later on then. So we can first of all print out the default output here and we can see, I mean, it's a lot of text but the important bit is if you're familiar with base meta output so far this looks similar to the previous output difference now is that instead of just two parameters the overall mean and the heterogeneity now we have in this case two regression coefficients labeled basileximab and we have all the estimates and credible intervals and so on there. We can also go of course and illustrate the results for example, looking at marginal posterior density so that's posterior density for the heterogeneity parameter and then for the two regression coefficients that are labeled according to the regressor matrix that we supplied to the function and we can also generate a forest plot that's probably the most convenient way to all the most familiar way to illustrate things and we can see in the forest plot we also see the regressor matrix reappearing again so we'd see the first two rows sorry, the first two columns here reproducing the co-variables that we supplied to the function and we see at the bottom the two estimates for the two regression coefficients. From this BMR01 object to which we assigned the output of the regression analysis we can in fact retrieve all the posterior densities quantites and so on so we see some examples here so we can compute the 99% posterior quantile here for the heterogeneity by accessing this posterior quantile function here and similarly it works for the beta coefficients we can compute quantiles we can compute the posterior cumulative distribution function at a certain point say at zero and we can also compute confidence or credible intervals for the coefficients here. So that's again similar to what we had before with meta regression now it's often interesting to also infer linear combinations so you want to have something like X prime beta so linear combination of some co-variable vector X and then figure out what the posterior distribution of that is and yeah for that we of course need to specify what our co-variables are so one example would here be for example the difference between ducalizumab and basiliximab treatments so that means we're taking the difference of the two so we multiply one by plus one and the other one by minus one and then we sum up the two and we can implement that by supplying this co-variable vector which consists of a minus one and a plus one here and we're getting out a combless interval for that and it works similarly also we can derive predictions and we can also derive shrinkage intervals for the for the study specific effects these theta i parameters which are also sometimes of interest for certain applications. Yeah so we can use the same coding here also for the forest plots which is often convenient so we can again generate a forest plot and if we're supplying a set of co-variables here so it's one zero and zero one and then minus one and plus one three rows of a co-variable vector yeah three co-variable vectors if you want then we can have them displayed in the forest plot as well so we get the estimates of the two treatment effects and the difference between the two treatment effects yeah and so we can have that in the forest plot or we can also if we're just interested in the in the pure in the near figures themselves we can also use similar arguments for the summary function yeah just briefly so we've used a particular specification of the regressor matrix here but there's always I mean there's usually alternatives setups that are sensible or conceivable one example for example is what you would usually get from the model dot matrix call here which would be a simple setting with a single intercept coefficient and an offset if you want parameter giving the difference between duckly sum up and basilix sum up and there's additional ways that you could use to all different ways of specifying the same regression problems using different regressor matrix setups and they are sketched at the bottom here in general they give you or they should give you consistent results just one note of caution is that in case you're using proper priors for the regression coefficients then the way you code the regression coefficients of course makes difference but there's I mean there's a ways or there's ways to translate prior specifications in one setting into prior specifications in a different setting so again you can get consistent results there yeah so far I've only talked about this one example where we had just binary co variables I just wanted to show that you can also get yeah what you might be thinking of when you're thinking of regression in general so we can also do regression analysis including regression lines and so on so this is one example application here involving 35 studies and we have a number of co variables here including early and late onset of the treatment and then the dose of the treatment and we can fit a model here to these 35 studies including these study level co variables here and in this case we can fit a model using four coefficients there's other settings conceivable again and but in the end we can get out these regression lines as well so what you yeah might be thinking of if you're thinking of regression so in this case it looks like we have an actual treatment or increasing treatment effect with increasing dose in one of the groups here and not so much of an effect in the other treatment group if in case of late treatment onset here yeah just a little bit of an outlook of other things that you may be able to do so we've seen already that looking at the difference between Daclizumab and Basyliximab treatments that was in fact a so-called indirect comparison because it was comparing treatments I mean the studies were all looking at treatment versus placebo and the treatment or the comparison of Basyliximab versus Daclizumab was not implemented in one of the actual studies but still we can try and figure out the difference between the two studies so that's a so-called indirect comparison and that means if we can do that then I mean quite generally if you're doing meta regression then the model can be used at least to some degree for network meta analysis as well and I mean the restriction here is that we're looking at the data needs to be estimates of the contrast so in our case that was these log odds ratios we can only handle two arms trials and we have a single common heterogeneity parameter but I mean at least to some extent we can also do meta regression and network meta analysis, sorry and finally so in the output from the model we can, the output also includes the marginal likelihoods and that's interesting because that means as soon as we are fitting different models we can also compute base factors which means that we can do interesting things like model selection or variable selection or model averaging and so on if we're fitting different models here, yeah. So to sum up, yeah, the meta regression is as an extension from the simple meta analysis my guess is that the most popular practical application may in the end be looking at subgroup analysis or comparing subgroups of the studies just like we did in the first example that we were looking at there's of course a wider range of applications including continuous covariance, network meta analysis model selection, model averaging and so on. Yeah, a little note of caution again, different parameterizations should give you consistent results in some cases, especially if you're looking at or if you want to use informative prior distributions or proper prior distributions for the regression coefficients then you need to be careful to properly translate the different settings from different parameterizations and yeah, the new base meta package version is available on CRAN since this week and if you have any questions I'll be happy to answer them either in the following discussion or also by email. Thank you very much, Christian. I'd just like to invite our two presenters today to the panel and we also are very lucky to be joined by two special guests so Matt Granger and Gavin Stuart will join the panel and I'll open up the floor to questions from Matt and Gavin. Hello. Go on then, Gavin. Okay, great talks, lads. Really, really enjoyed them. Thank you very much. My first question is about the model averaging risk of bias and I thought it was an amazing package, really nice. I wondered whether or not you'd be able to extend that to nested treatments and also to think about network meta analysis as well. So where you've got that problem that you've got the multiple treatments and so you've got the multiple peaks and it just seemed that maybe Bayesian model averaging might be an interesting approach to look at that. I mean, people have tried doing things in net meta, trying to look at all of the different treatments simultaneously in publication bias. Anyway, I just wondered what your thoughts were on those, the nesting and then the multiple treatments. Yeah, well, that's a very good question and thank you for the praise and looking at multiple dependent outcomes or different measures from the single studies is a thing that I've been working for half a year now and it's a tough problem to correct, to be honest. And it depends on what type of models you want to specify. If you just want to look at the pet piece, meta regressions that adjust for the relationship with standard error, standard error squared, that's quite simple because you can still marginalize the stuff out or just use a multivariate normal distribution for dependent variables, but with the selection models, if you really want to model them properly, then you need to get a multivariate weighted normal distribution and this distribution just starts exploding and the number of computations you need to get to get all the proper probabilities. So I think I got up to like three to four outcomes from a single star study. So if you have the nesting and you have three estimates, that's still possible. If you go more, then you are not able to evaluate it. So I'm looking at some approximations, but nothing that I would be really happy about yet. So yeah, but that's a great extension to do in the future and I'm working on. So I've got a bit more of a general question really. A lot of our audience at the conference probably won't know much about Bayesian methanolysis. So can you both maybe give us an idea of why one would use Bayesian methanolysis over any other frequencies approach? What do you think the advantages are? Are there any pitfalls that people need to look out for? Just give people an idea of why you think it's a good idea or a bad idea in certain circumstances. I'll hand it to Christian first. Thanks, yeah, I guess one advantage is that many of the frequent test approaches rely on large sample sizes and topics and that's not the case for the Bayesian method. So they actually work well also for small sample sizes and few studies like the example with six studies or also three studies or two studies without at least any technical problems. Yeah, I mean, of course, I mean the obvious disadvantage of course is that you need to spend thought on the prior specifications that you want to use. But yeah, we've been also working on that and trying to compile the, yeah, obvious choices there or some hints on how you can start thinking about what might be sensible priors, especially for this heterogeneity parameter which is not so obvious when you first think about it. Fantastic. Yeah, maybe I can build one of that and I think the small sample sizes or like small number of studies is the obvious problem for frequentist methods. And as Christian mentioned, the Bayesian methods thanks to the prior distribution allow you to get the proper estimates in that regard. And they also, but they also offer additional abilities for the meta-analyst to incorporate knowledge that's already in the literature. For example, last year we published paper in statistics and medicine where we described different informed prior distribution based on the published trials in the Cochrane database of systematic reviews. So for example, for the heterogeneity parameter tau you can get an informed prior distribution that's based on the previous meta-analyses on similar topics, similar treatments. Now also in just we have a new module that allows you to look at different trials, combine them and get estimates from them. So you can again use this previous knowledge to specify your prior distribution. The same thing then goes for the effect size parameter. Furthermore, this allows you to create informed tests. So you are not testing now against some unspecified alternative as you do in the frequency settings but you can actually evaluate evidence for a point now or some peri-anal if you don't believe in mouse versus some informed alternative that corresponds, for example, to the vehicle treatment or the treatment that you would expect or treatment your founders might be interested in and those tests then have much higher informativeness which I think is very important for the statistical inference. Yeah, could I add to this Matt as well? I think that kind of three fundamentals for me are that a lot of the time with meta-analysis we're not actually trying to say this is the answer, this is the effect, we're trying to express the uncertainty around it. And if you do your normal meta-analysis particularly like the lads were saying when you've got only a few studies that estimate of tau is sitting there in your model pretending that you know exactly what it is. You haven't got a clue what it is. And if you put a little bit of uncertainty on that the credible intervals just explode. So suddenly about little analysis where you've got your four studies and you think, you know what's going on. You do it in a Bayesian framework you can express the uncertainty much more realistically. So sometimes it helps you express uncertainty. Sometimes you get the shrinkage effects where it helps you to reduce the uncertainty by looking at the exchangeability between the studies. And Christian's written a lot on that kind of stuff. You should probably ask him to chat more about that. The third kind of element of it is that it feeds into the decision models. So if you think about network meta-analysis, for example there's all kinds of different ways of doing that in frequentist framework and in Bayesian frameworks and everybody argues about it for the socket being all the rest of it. Whether or not the estimators are biased and we all have all our technical arguments. That's kind of irrelevant to me. The reason why I like the Bayesian approach for that is that it feeds, it gives me the information that as a decision maker I would want. It gives me the probabilities of one treatment being better or not. I can get these cumulative ranking curve for the probabilities and I can get that out of a Bayesian framework. So it feeds directly into the decision-making process in a way that's much more intuitive than the frequentist models. So if I was just having that straightforward should it be Bayesian or should it be frequentist they would be the three things that I'd be thinking about. Fabulous. Christian, would you like to elaborate on the second point where it was highlighted you've written about? Maybe just briefly. So I mean in Bayesian models in general you've got this nice feature that you can sequentially update your information and that's also sometimes handy in meta-analysis so you can do a meta-analysis of your, I don't know, 10 previous studies and then from that derive a prior for your 11th, your future study which is helpful sometimes, I mean for the analysis itself or also for planning the new study and so on. So I guess that would be another advantage, yeah. And that goes under this term of shrinkage estimation. And also from the philosophical point of view like meta-analysis is not really good way or like proper analysis in the frequentist sense because there is no sampling plan a priori. So you cannot really compute p-value because there is no sampling plan for how the studies were to be conducted in the past. So you cannot really, you don't know what's the likelihood. So you really need the likelihood based methods to evaluate evidence properly. But that's not as practical as the other advantages that we mentioned before. Frantisek, I'll just say we have a team chat and there was a lot of love for your demons. Folks are loving your demons. It was really lovely in illustration to illustrate the point. Any further questions from the panel? So I've got another question. This one's for Christian. And it's about, again, it's about kind of extensions to handling nested data and handling the multiple treatments. So I'm guessing that nested data you could handle with your meta-regression framework just by specifying the nesting as covariates. But if you move into the multi-arm NMA you're gonna have problems because of fitting the multivariate distribution. Is that understanding right? You could extend it quite easily to the nesting but doing multi-arm NMA with base meta is gonna be really tricky. Yeah, I guess I know too little about the nesting problem particularly. But I mean the computational trick underlying the base meta function or the base meta package in the simple example it's essentially based on the fact that you only have two parameters, right? You've got the overall mean and you've got the heterogeneity and conditioning on particular heterogeneity value everything is normally distributed and you can, yeah, then numerically try and marginalize over this heterogeneity parameter. And that still works also in the case of meta-regression because instead of the single overall mean parameter that is conditionally normally distributed you have something multivariate normally distributed and that still works. But it's so I'm not sure how you could I guess it would not be feasible to extend the model to include additional variance parameters or something. So I guess anything that's more or less the meta-regression might still work but otherwise you probably need to switch to MCMC or something. Brilliant talks, brilliant packages. So I'm gonna start using them more. Thank you very much guys. If we don't have any further questions I'd love to round it up here. No pressure. If there's any further questions, please do go ahead. Otherwise that's it for this session and we hope that you enjoyed it as much as we did. Thanks very much to Frantisek and Christian and also to Matt and Gavin for joining us for the panel and we'll see you at the next session.


Okay, welcome everyone to, I guess, the tail end of the day. Thank you for joining. I'm here as a representative of Morpher. Morgan Stanley contributed to Morpher a couple years back to Finnaz, and one of the main reasons that we did so was to maybe spur some collaborations. It was our hope that like-minded projects would take a look and realize that there's added value to collaborating on similar kind of technologies. It's interesting that when you go into open source, you quickly find out that a lot of people have already thought of, a lot of the things that you've already thought of. And it can be, I guess, a little nerve-wracking and at the same time it's also satisfying because you realize that it's probably a pretty good idea if people have come to the same conclusions maybe from different angles. And so, you know, we contributed it to Finnaz specifically so that we could increase collaboration. And I'm happy to be up here with Damian Reeves from Capital One because this is the epitome of what Finnaz is really about. We contributed more for a couple years ago and then over 2022, Capital One picked up Morpher as a project and became an equal contributor.  It is actually a pure Finnaz project with multiple contributors. And that's exactly what we wanted to happen. And so, let me get out of your way and then introduce Damian from Capital One. Thanks, Steve. All right. So, hello. My name is Damian Reeves. I'm a distinguished engineer and I work for Capital One. Yeah, that company that talks about your wallet. I love functional programming and languages, compilers, domain-specific languages, and open-source software in general. I've got a real passion for this. And yeah, let's see. I'm also, Stephen just said, a maintainer of Finnaz projects. I contribute to Morpher, Morpher Scala, Morpher JVM. And coming soon, Morpher Rust and Morpher Go. Really ambitious, but what Steve just talked about is we are no longer in the place where we just have a single financial institution leading this effort. I think we've heard throughout this day various people talking about Morpher and the value add. I'm going to talk a little bit more about that. But the fact that we know that there is a place and a space, it means that there's also really excited people who want to contribute and help to broaden the Morpher ecosystem. So today's journey that we're going to go down, we're going to go through Hello, Morpher.  So excuse me, it's not going to be too geeky. But I'll introduce Morpher to you. I'm going to introduce this Morpher IR that has been talked about, show you some of the pieces, what differentiates it from some things you may have worked with already, talk a little bit about the Morpher ecosystem and why I think it's important and it's beneficial to invest in Morpher as a platform. So let's see. All right, let's make a case for Morpher. So really what is Morpher? Morpher allows you to leverage the power of your business logic by storing, sharing and translating and visualizing it. So fair warning, we're talking about business logic. So don't fall asleep. We talked about regulations, so I think if I talk about business logic, we'll stay awake. But are you excited yet? Come on. It's all right, it's 4.15, we had our coffee. All right, what do we mean? So what is Morpher? Morpher is a set of tools for integrating technologies. You can find this on our fact sheet. Morpher is composed of this library of tools that facilitates the digestion of business logic into multiple different languages. So mouthful, we're going to get a little bit deeper into what exactly that means. But I want us to really focus on why is this important and it's important because Morpher makes your business logic portable. And it does it by turning your business logic into data. This enables you to move your business logic across systems, across departments, organizations. It allows you to go across boundaries.  And the compliance back end was in some technology. But you also wanted to have a front end for that. And guess what? You want to maybe before you send over bad data, you want to validate it. So guess what? You code the same business rules in a front end language. That is part of what Morpher is about. It's about capturing the business logic one time and taking out that error-prone re-effort and duplication. What else can we say about Morpher? Well, it's important because business logic when it encoded as data can also be converted to new languages and applications. It allows for the use of existing and future tools. Whatever you mean by future tools, I'm going to get into the ecosystem a little bit more. But tools that you haven't built yet, right? Remember, this is data. And the thing we know about data is once you've captured data, right, if you've got a teenager you know once the data is out there, the data doesn't go anywhere. But once we have something in data, even though we don't have a tool that might be able to visualize, that might be able to convert, today those future tools can exist. I talked about Morpher and Go or Morpher and Rust. It's not just about what exists today. And I want you as I speak today to remember that this is an open platform at an open source financial summit. You can contribute in your own way. So don't feel restricted by any of the tools we present. It's open and it's an open platform. So how does it work? So Morpher defines a standard format for storing and sharing your business logic. And Morpher provides APIs. It provides tools. It integrates technologies. Once your business logic is converted into this developer friendly data, it's easy to integrate and assimilate it into your stack, right? It's stack agnostic. And once again I'm going to underscore, though there may be choices that have been made today, part of why we are a member of FinNOS is that the choices that have been made, it's only because of resources, right? It's only because of someone has to pick something. But I think the platform is open enough. So what's the benefit? Morpher eliminates technical debt. I'm going to go through this kind of quick. It ensures correctness. I'm going to talk a little bit about the Morpher IR and what are some of the fundamentals of it. But I mentioned at the beginning that I love functional programming. And in functional programming we have this idea of purity. And sometimes it gets us in trouble a bit. But it's about what you put in is only, or what you get out of a calculation is only in relation to what you put in. And that's really what your calculations and your business logic is all about.  It is not really the core of your business logic. But we duplicate effort. We reimplement systems. We rewrite business logic all the time because, well, now I need it on AWS instead of Azure. Now I want to run it in Lambda. Now I want to run it in some Java or Python system. And Morpher's idea is let's capture the business logic in a way that, first of all, is strict. I think functional programming is a strict and very direct way of doing it. You don't really need side effects in your business logic. Though we do it all the time, and I'm sure some of you have been involved in systems where in the middle of doing a business logic calculation that you call out to a database, what we've learned over time is if we want to scale, you shouldn't be doing that anyway. And so if you want to embrace things and make sure things are correct, you just capture it once and then push everything to the edge. So it increases agility because you're able to just capture this one time and the knowledge is able to be disseminated. I'm going to, you know, I'm at the end so I have the ability to refer to what came before me. We saw a wonderful talk about how Morpher, along with Legend and Databricks, was able to make a full integration pipeline and I think that's the sort of things that we can look forward to in the Morpher ecosystem. So let's talk a little bit more about this thing that I keep mentioning, the Morpher IR. And it may seem a little intimidating, this IR notion, but let's try to break it down because this is really the centerpiece of what is Morpher. It is the definition really, and Morpher is really about this IR or this data format. IR is a programming language term that means intermediate representation and at its heart Morpher is kind of like the set of tools but in the midst of it is this format and Morpher embraces functional domain modeling and the Morpher IR enables users to model your domain and model your business logic. It means sit with your business users, talk about what it means to calculate whatever you're trying to calculate. Be at the LCR, take a specification and Morpher is about capturing that and then encapsulating it in the form of code and data. So what are the properties of Morpher? It's pure, functional. So it allows front end languages to encode data types as functions and ADTs, abstract data types. What we found are, as I said I'm biased, I'm a functional programming guy. What I find is that you're able to make illegal states not representable when you're able to talk about things in terms of functions and abstract data types, records and option types and you don't have null anywhere in the Morpher ecosystem but you can instead, usually when you want null what you want is optionality. So we support that as part of Morpher. Morpher is strongly typed, right? There's not going to be any loose type because the reason we are able to operate at the data level is because we tell you something and you're able to abstractulate and make a decision based on that type. Morpher is modular. This is a really important thing because you're able to take different pieces from different ecosystems and connect them together. I can allow your imagination to drift if you would where you've got an accountant system and you need to define yet another representation of an accountant journal and an accountant ledger and then you realize why am I doing this because accounting has been around for hundreds of years but I've got to do it one more time. An organization like Finos, right? Could be an umbrella with something like Morpher where you can have your accountant ledger module. You can have rules. You can maybe have people who know what are the regulations that you're worried about and have a module that you can import and it's part of an ecosystem. So that's part of what the Morpher IR with its modularity gives you. It gives you logic as values. I'm going to say that again. It gives you logic as values. I'll key in a little bit on what that means. Logic and values means that it's a differentiator because usually when we're talking about modeling data, modeling services, then you pretty much stop there. You stop at what is the schema? What is the shape? What is the contract? But Morpher allows you to go one step deeper because now you can save the actual logic, how you make decisions, what are the rules along with your schema, along with your data. And it's distributable. So I can package that up and I can ship it. I can put in a GitHub repository. I'll talk a little bit about how we're looking to broaden the packaging ecosystem of Morpher. I'm not going to say it now because I would be going ahead of myself.  What I mean by that is not that we're done. What I mean is that you're able to take the formats that you do know today, formats like Avro, all those other formats that you work with, and those can potentially be generated from Morpher because it's high enough and it's powerful enough. And I'm going to get a little bit deeper into the weeds and what do I mean? I'm going to introduce you. I'm not going to show all the flashy demos. I'm going to go into the details of why Morpher is valuable. So it has, at its type level, and this is where you can do schema, right? You can represent, this is how powerful Morpher is. You can talk about things like a list, but you're not just doing list of string.  And you might not be familiar with some of these other platforms. Generics is something that's usually often overlooked. You can have references to types. You can have tuples. Often those are results that you get from databases. You can have records. Everything has a record. This extensible record thing is almost like functional programmers take on interfaces. And of course you've got functions. Lots and lots of functions. Forget about unit. There's only one of those, but it's needed. And then you've got values. Now, I'll say all that stuff, particularly records, you're probably used to that. When you go to your database, you say, here's my schema, and that's really a record structure. But Morpher allows you to go a little bit further. And how it goes further is it allows you to say, all right, here's all the different values. And I'm just going to let you look at this a little, talk a little bit about. You can do all those things at the value levels. What does it mean to be at the value level? This is the body of your function. And this is the real difference. Because if I take any other format, I'm going to have a little service thing, and it says I take in an int, and I spit out an int. Or I have an add function, I take in an int, and I spit out, and I take in another int, and I spit out an int. What Morpher is going to say is it's actually A plus B that's in there. Stores it in a format that when you're ready, you can turn A plus B into a calculation in Python, in TypeScript, in Scala, in whatever language you want to work with. And you can do that through all the types, lists, records, you can refer to something, which means that you can potentially refer to something that exists in a module. And that's how we support modularization. All those sorts of things, access fields, call functions, lambdas. I show all this to really overwhelm you. I show all this to show that you can do everything in your business logic, because you've probably seen all this, you're like, I'm different. My use case is broader, is bigger, but you've got a very powerful language, right? So you got all that? Cool? It's going to be a quiz at the end. So what am I really saying? Morpher is not just another IDL. What do I mean? It's not just another interface definition language. Morpher is more. And what's at the center of that is also the Morpher ecosystem. And that ecosystem is really why I'm here. So the tool in story with Morpher is you'll hear it all the time if you talk to someone from the Morpher team or engage with a project, that you take a front end, you do this process, you kind of make Morpher IR, and then you do transformation. And when you do your transformation, you can transform the Scala, transform the Type Skip, transform the Jason Schema, transform to a fancy visualization that looks really good at a conference, or you can even do more. Whatever it is, right? You can do reports, you can do all sorts of things like that. All right. So with Morpher, you model your problem domain, you transform what you've modeled into data, now you store that, your business models, and then you use it. And I'll talk a little bit about the ecosystem, then I'm going to talk about why I think we should invest. The ecosystem is ever broadening. I remember in 2020 when Morpher was open sourced that we really had Morgan Stanley and a strict set of tools, but more, there's more in the works. So with Capital One and Team of Capital One, we've been working on Morpher Scala. Morpher already had Scala Tooling, it allowed you to generate Scala, but we're looking to do deeper integration. We're actually working on a compiler plugin, which will allow you to model, if you use Scala as your programming language, that you'll be able to write Scala, you'll get Morpher out just like how you can do currently in the Scala ecosystem, you write Scala, target the JS platform, write Scala, target Native platform, and we're working on write Scala, target Morpher as a platform. We already have Morpher Elm, a lot of the ecosystem is there. Morpher Go, it's one of those things, along with Rust and even .NET that will allow you to do things to target those ecosystems and eventually, we hope, WebAssembly, and then we even got this other repo to the side, which is about Morpher Editor or whatever, but what it is, is this ecosystem of tools and needs exist, and in the open, right, because it's not just one company funding all this, we're able to do that. So why? Yeah, particularly Go, Go is not really good with those constructs, so there's some design going into how Go does support generics as of this year, by the way. I'll get, I'm going to try to release some time for questions. So why invest in Morpher, and why is Capital One invested in Morpher? Well, I'm going to tell you a little bit about Capital One as a company. We believe in open source. I think the term we use is we're all in on open source. As an organization, we believe in standards and standardization. We've got business rules all over the place. In Capital One, your organization probably is the same, and we believe in innovation, and because of all that, I've actually been allowed to work on a business problem, but to work on it in a way where I'm able to use an open source technology like Morpher. And so we invest in Morpher because we know it gives us increased power. So I work on a platform where we had our own DSL, and we wanted to give more power to that DSL. We wanted to expand the capabilities that our business users could have, and we had a choice. We can maybe use one of the systems that exist that maybe isn't as pure didn't really follow the principles that we believe in. I work on something where being correct is really important. So we already embraced functional patterns, and Morpher's focus on correctness was important for us, but we needed more power than what our DSL provided. And we could have started from scratch, but being open source first, one of the things that we like to do at Capital One is look, is there something that does what we want to do? And Morpher fit that build. We want to expand on modularity. We wanted to be able to have one part of the business define rules and procedures and calculations and have another part of our business use that and wrap it, extend it to incorporate it in what they're doing. Security is important. Because it's closed. Because it's not just like, it's not just a scripted engine, right? I've talked about all this, it's not just a scripted engine. With scripted engines, you suddenly find yourself where you've got a scripted engine and maybe you've got some login and you find that in December you're suddenly trying to squash some sort of bug because you're able to do scripting. Anybody remember last year? We said, what we wanted to do and what we wanted to do then is what you get out. And we wanted discipline constraints and we wanted an ecosystem. And we found that in Morpher. And now, because of this, we looked at this, we said, listen, we can build this from scratch or we can contribute and we can get all the power of an expanding ecosystem because if we did this earlier, we wouldn't get it. But we recognized that if we contribute back in the spirit of open source, then we can get benefit and we can benefit others and we can build on the ecosystem. So I'll talk a little bit about what my team and what myself personally is working on. We're working on Morpher Scala. I talked about it. We're going to bring the Morpher IR and all its types. We're going to do that Scala 3 interpreter which allows you to take Morpher. Right now you've seen Morpher running in web applications. You've seen it maybe running at a command line. You're going to be able to run Morpher directly in the JVM. Morpher, just to be clear, right now you can compile and generate Morpher code that runs on the JVM. But this is about directly taking Morpher models in real time running them on the JVM platform in a secure manner because you don't have to deal with all the things of jars or things reaching out to libraries. That's really important to us because with Morpher's data what it means is that if you've got not just hundreds but you've got thousands of variations of business rules you want to version all that stuff you want to be able to load it on the fly if you have maybe millions of customers that have little slight variations on how they use, I don't know something like a credit card that's really important and this is what Morpher allows us to do other things we're working on workspaces which are around packaging and the extensible CLI and yeah, that's basically it I just want to say thanks to the Morpher and Finno's teams Steven Goldbaum, Tilla Mahali the entire Morpher team and Finno's team and I want to also say thanks to the great team of engineers at Capital One and the leaders at Capital One and a very open and awesome open source program office and thanks I know I don't have much time but Q&A Go for it, you had a question Yeah, so that's one of those things where we've looked at and for someone like me I love languages, I love programming languages so I'm involved in a lot of those projects and yes part of why we look at languages that kind of have principles or at the heart the same tenets as Morpher which is why Scala was one of the first ones I'll be very honest Go I was hesitant for but Go has a pretty big presence at Capital One so I kind of begrudgingly started the effort of seeing what it would look like to integrate Morpher what's also interesting is remember Morpher has basically a front end where you can author something and then you have a back end which is about execution so you can actually not even author it in Go but just use Go as an execution platform and Go and Rust as execution platforms they actually open up the door for you to run on WebAssembly which allows you to run basically anywhere I think it's going to differ a little bit based on your use case but there is a lot of use cases that we've seen where you do want to just dynamically load but from a security standpoint the minute you say dynamically load there's all sorts of issues that get unlocked so the idea then is if we have this runtime interpreter of this very closed loop then you're able to have dynamicism without security holes so I do think it's a big future it's actually the principle behind why WebAssembly has taken off so I think that Morpher can benefit and learn lessons from that it's also an age old balance across all programming we've always had interpreted systems users defining query rules at runtime versus compiled systems with unit tests and there's always a balance of when do you want to use one or the other and it really comes down to the context in use case what I'm saying is that the tension between interpreting a runtime versus compiling is that you can make changes to things like regulations at runtime and push those out whether that's a good idea is really up to the business for example you might have traders who would like to create rules is that really the best thing to happen or should we actually unit test the kind of rules that a trader would build to make them push those out those are really business context questions I think the important thing is that you've got the option for both you could take that same model and interpret it at runtime or you could compile it down we've seen examples where there's a desire to separate the platform on which the thing executes from the rules and the logic that it's being executed and once again I've probably said security a lot of times in a safe manner being able to do it is not any huge achievement you can take a scripted engine you can deploy that in your application and just let the hackers and security holes creep in or you can do it in a very secure sandbox environment and that's what the interpreted play I shouldn't say play but the interpreted position I think for Morpher it adds a value in that you have a sandbox safe execution environment and it's something that for your organization if you got a security group that they're gonna ask oh can I talk to a database you can't do any of that for Morpher it doesn't even have the concept of that and that's part of the power you still have all the power to do everything at the business logic level but you can't do the bad stuff there's no launch and missiles in the middle of your calculation and I think that is all my time I can get one more question that if that is an oversight on my part just to repeat the question oh sorry I just want to make sure that everybody here so the question was are there some types and some types being you go ahead and explain it some types being like unions being able to express the idea of a choice like A or B or C and sorry that's an oversight Morpher actually includes some types it's an important part actually the IR is full of them yeah I agree we totally agree we love some types some types are the ability to say hey look I'm getting a message that's of this format like a trade request message which has these ten fields versus maybe I'm getting a trade cancel message which just has one field the idea of a field and through pattern matching your code goes and figures out what do I do with that a lot of languages don't have that it's a very powerful functional feature yeah and I skipped over that but I went through it quickly but we support pattern matching and all those things as well well thank you so much for your interest and of course you can check out morpher.finneaus.org thank you


This is an eight items energy smart home performance and I want to show you a really typical thing that happens in knee walls I can see it down here to see where this is sticking out. Those bats are only three and a half inches thick So if it's not touching the wall, it's really not doing anything and Same things going here. I pulled this out So This is not doing its job at all And that's one of the reasons that a lot of people don't really like fiberglass insulation if you spray foam this or Net and blow it Or some other method. It's a much more likely to actually contact the surface needs to be insulated, which is the drywall in there


Last year, the WIPO Arbitration and Mediation Centre had a very strong performance once again, particularly in respect of the administration of internet domain name disputes. Over 2,500 such disputes were filed with the organisation and with the centre representing over 6,000 individual domain names. One of the things that's happening in this space, of course, is that the context in which these disputes take place is rapidly evolving. It's rapidly evolving because the Internet Corporation for Assigned Names and Numbers, ICANN, which is the body responsible for the domain name system and for the technical management of the domain name system, has taken the decision to expand, potentially infinitely, the top level domain name space. So they are in the process of introducing or authorising the introduction of new generic top level domains. That is what you find to the right of the dot, such as COM, is to the right of the dot. So this space is expanding very, very significantly. And they have already approved some 160 new top level domains. What is the impact of this? By the way, that decision to alter the domain name system in this way and to modify its operation is a decision which is entirely within ICANN's responsibility and WIPO as an organisation has nothing to say about that and it's taken presumably for navigational purposes, that it is to improve the navigational possibilities of the domain name system on the Internet. But what we are concerned with is the impact of those decisions on branding, on the capacity of consumers to identify with reliability, dependability, the true owners of products and services that are sold over the Internet. So we see with the expansion of the generic top level domain name space that the opportunity for misuse of trademarks has increased. So while there was a limited number of generic top level domains, the opportunity to register trademarks without the consent of the trademark owner as a second level domain that is to the left of the dot such as brand.com, the opportunity was limited to that number of domains. When you increase the generic top level domain name space by the addition of potentially hundreds or thousands of new generic top level domains, the opportunity for misuse is increased significantly and this will have a major impact, a major impact on trademark owners and on their capacity to monitor the misuse of trademarks. And they will have to form their own views about what strategy is to adopt about this. Are they going to register their trademarks in every single new top level domain which is potentially a very significant burden? Are they going to attack every single misuse that occurs in all of those domains? These are significant questions, the answer to which we do not know at this stage. But we are watching very, very closely this space and trying to ensure that the branding system that is legally protected through the trademark system remains a reliable instrument for consumers to associate products and services with particular origins.