Hello and welcome to EsmarConf 2022 and this session on network meta-analysis. As always, this session is being live streamed to YouTube and the individual presentations have been pre-recorded and published there as well. Subtitles have been verified and can be auto translated for those individual talks and automatic subtitles will be available shortly for this live stream. If you have any questions for our presenters, you can ask them via the presenter's individual tweet from the app ES Hackfond Twitter account. See them on our feed. Presenters might have time after their talks to answer some of those questions or at the end of the session if time allows and we'll endeavour to answer all the questions soon after the event. We'd like to draw your attention to our code of conduct which is available on the EsmarConf website at esmarconf.github.io. So our first speaker is Sylvia Matelli who is a post-doc drill fellow at the University of Paris. Sylvia, over to you. Hello everyone. In this talk I will present NMA Studio which is a new online and interactive web application to produce and visualize results from network method analysis. This is all joined work with Anna Haimani also from the University of Paris. So the main reason behind NMA Studio is that we wanted to provide a tool that could not only enhance but also facilitate the interpretation of the main findings from a network method analysis. We know that network method analysis by integrating direct evidence with what is called indirect evidence can simultaneously compare many treatments and for this typically there is a very large number of results and outputs being produced. So visualization can be challenging especially when we have large networks with many treatments. Also more recently it seems that we are moving towards the context of a living evidence so with data collected weekly or monthly so we need new software to keep up with this fast production of evidence. However existing software in the field so far are not fully interactive in the same way we hope NMA Studio can be. So a bit more specifically NMA Studio is a Python application that connects to R to produce the NMA results using the library net meta then those results are imported back in Python where the app is built. In NMA Studio users upload their data and interact directly with a customizable natural plot by clicking on different nodes so the treatments or different edges so the comparisons and based on their selection different outputs would be displayed. So NMA Studio will follow all of the key steps which are typically performed in any published NMA starting from the very first assumption of and fundamental assumption of transitivity and then once this is met typically a large part is dedicated to the reporting of the of a summary of the treatment effects in the form of forest plots or league tables then importantly we need to assess what the consistency holds in the network so if there is a statistical agreement between direct and indirect evidence and also to assess for the presence of small study effects which is a phenomenon related to the problem of publication bias. Finally a ranking of treatments is usually provided so NMA Studio will assist the user in each of these steps and we will see this how with the demonstration. So our data come from a recently updated systematic review about chronic plates rises which comprehends 158 RCTs and compares 20 different drugs. There are two primary outcomes that will be analyzed one for efficacy and one for safety. In this case both outcomes are binary and risk ratios will be used. So we can just have a look at the app. Okay so this is the main home page of NMA Studio and first thing you will need to upload your own data from here selecting your data file for example and data format or type of outcome. Here I've already uploaded the data and this is permanently loaded actually this example into the app but once the data process is completed your network plot will appear here on the left. This is a completely interactive object so you can move it around zoom in or out or also drag every item. Also there are additional settings here starting from the layout so you can change layout from list you can change edge sides or node sides for example and also a few options about coloring the nodes for example by risk of bias you will have in your data or by class of treatment if you have that and also just choosing your desired color and same for the edges. Then you can download the plot from here or look at an expanded version from here. Still in terms of network visualization we added an option to look at the evolution of the network over time with the slider here on the right so for example if we click on the very first data available for this data in 1963 we can have a look at the first trial and then we move on over time and we can see how evidence is added and also you will see that the data table is filtered in real time accordingly. Also in terms of data filtering you can filter your data set just by clicking on a few comparisons or edges in the network or a few nodes and the data table will be filtered accordingly again. You can also have a look at the expanded table and export the data in CSV. Okay so now we are ready to start all of these key steps of the anime that I was referring to before starting from transitivity so to check transitivity we need to check whether the distributions of the potential effect modifiers that you have in your data set are similarly distributed across comparisons. You can choose your effect modifier here for example we have a look at age which is typically provided and the corresponding plot will appear here. Also these access labels and this is true for all the subsequent plots are editable so for example here you can just choose your label and also if you want to highlight a few comparisons of interest you just have to click on the corresponding edge in the plot. Moving on we have the summary of the effects so starting from forest plots here we have three options an anime forest plot the typical pair was forest plot and the bidimensional forest plot in case of course two outcomes are given so starting from the first one here of course we will need to choose a reference treatment and we do so just by clicking on a node on the network. We can have a look at the plot also you will have to choose the correct direction of the outcome to interpret the plot correctly and we can also have a look at the plot for the second outcome in this case the network plot will be automatically updated for outcome two then for the pairwise for forest plot of course in this case we will have to choose a comparison and you can resize a little bit the objects and also again you can have a look at the plot here for outcome one or two and you can save it all the plots will you can save it from from here then moving on again we have the bidimensional forest plot so you will have the forest plots for outcome one on the x-axis and for outcome two on the y-axis again we just choose a reference treatment the forest plot will appear here and in this case you will have to pick the correct region of the plot to interpret the results correctly based on the direction of the outcome but also you can click on the legend to remove sequentially a few treatments if that's needed then again we have the part dedicated to the league table so league tables report all of the possible treatment effects between any possible pair of treatments in the network so they tend of course to be very large and here you can see that you can scroll the table we also allow for two different options of coloring the table first and default option by risk of bias in the direct comparisons you will have the average risk of bias but also coloring by report from cinema assessment so cinema is a software that reports an overall rating in the confidence of the evidence which is graded as very low confidence low confidence moderate or high in each comparison so what you will have to do is to upload the results from the from that you get from cinema for both outcomes of course and then have a look at the corresponding coloring here on the lower part of the triangle we have results from outcome one and on the upper for outcome two also you can have a look at the expanded table here and you can export it maintaining coloring and formatting but what is important here is that as we said league tables tend to be very large so you might want to have a look just as a subset of nodes which are more of interest so you can do so just by clicking sequentially on a few nodes and you see that the filter the table will appear here then moving on we have other checks so starting from consistency in this case in NMA studio we allow for two different options a global test for inconsistency which is the design by treatment interaction model and a local test for inconsistency which is the node splitting approach you will see that you have all of your results here in the tables for the node splitting for all of the different direct comparisons you can scroll down and you can see that suspicious values are flagged in red or yellow also here you can filter the table just picking a few comparisons of interest then we have to assess for the presence of the small study effects and we do so using the typical funnel plots in this case comparison adjusted funnel plots here also we have to pick a reference treatment so again we just do so clicking on the metric and the corresponding plot will appear here for outcome one or outcome two then finally we said we have the ranking of the treatments in NMA studio we allow for two different plots a heat map reporting p scores for both outcome or for one outcome only and a scatter plot of the p scores so p scores are the frequentist analog of the more common sucre values and what is important here is that you will have to choose the correct direction of the outcome so for example outcome two for us is a harmful so we change this here and we will have our heat map here with treatment sorted from best to worst then we have our scatter plots reporting p scores for outcome one on the x-axis and for outcome two on the y-axis again you choose the correct direction and you will have of course on the upper right part of the plot treatments that appear to be the best in both outcomes but what is important here is that you can use the net or plot to assist in the interpretation of these results for example if we just look at some more details we can see that the best treatment is actually a treatment which is okay at low risk of bias but also not very well connected there is only one trial assessing this treatment so it is always important to use the network to assist in the interpretation of the findings so this is pretty much it for functionalities of the app we also have a documentation page and the news page so just to conclude we have seen how anime studio is a full interactive and flexible application and that it can simplify the full anime process while also assisting in the interpretation of findings however as all softwares it comes with many benefits but also with many risks so we always really highly recommend to use anime studio following advice from inexperience statistician there are many features that we will we would like to add to anime studio just to name a few here we would like to add more options to customize our network plot and also a more robust system of alerts or warnings for example printing the errors from the r console directly for sure we will add an option for performing Bayesian enemies and also we are looking into ways to provide each user with a permanent link to their project anime studio is also a campaign by python package which is currently under development but that will be available soon of course this is not an exhaustive list so we can add more and if you have any suggestion or ideas that will be more than welcome so you can get in touch with me at this email address here if you want to discuss more thank you very much everyone wow such a useful software for anime users thank you sylvia and we look forward to seeing you again for q&a at the end so if you have any questions for sylvia or any of the other presenters please do post them on youtube or on the tweet thread under her name and i'm delighted and we just our next speaker who is virginia kira who is a doctoral researcher at the university of burn virginia over to you and kira i hope you can all see my screen and can you hear me as well we can okay so thanks for joining my name is virginia and today i will present the development app so development is the first tool to assess the risk of bias due to missing evidence in network man analysis and the tool and underpinning framework been published in bmc medicine and in a few words the framework works in a way so that it evaluates the risk of bias in network man network man analysis first by evaluating the risk of bias in each pair was compared in each pair was comparison that can be made between the intervention in the network considering both within study and across study risk of bias assessment then the risk of bias in each of the pair was comparison is combined with the contribution that each star comparison makes to the network estimate to get together with any presence of small study effects and any bias in the unobserved comparison so that we reach a final level of risk of bias in the network estimate and we give up a level of either low risk some concern or i risk of bias but due to time reason i'm not going to go into the details of the framework but i want to show you what we have developed as a app that semi-automates some of the assessment this is because due to the nature of network analysis such assessments obviously are more labor intensive than a standard pair was met analysis so this is the rationale for developing an app the semi-automates some of it so the app is already available and first i'm going to just show you the data then an example of network the data that i'm going to use to demonstrate the app it's a network of antidepressants from head to head studies and the network graph it's shown here so the first thing the user is from to do is obviously load the data it has to be loaded as a CSV file and then the user can view the data here it's also important as you can see there are some studies that do not have the outcome of interest they have a blank cell so not missing values here and this is important because these studies can be informative about the selective outcome reporting bias which will be assessed in the app in the tool so the next step is to move to the data analysis tab where the user has to select the parameters for the analysis that are run by the app are required by the tool to do some of the evaluation some of the assessments so and then you can start the analysis by pressing the button depending on the size of the network obviously the analysis can take a few moments a few minutes and for this reason i've already run the analysis before and this is essentially what you will get once the analysis is completed so first of all you will get a data summary with a network graph and there is other characteristics of the network interventions and comparison as well as the outputs from both frequentist and by Asian network meta analysis which are run respectively with the net meta and the box net package as well as also the output for the patient network meta regression but i will move straight to the main output of the tool and of the app which are the tables where to record the assessments so first we will have the pairwise comparison tables here the all the possible pairwise comparison are automatically grouped according to whether they have data for the outcome of interest as shown here in group A or only data for other outcomes as we can see here in group B we have two or whether they've been the comparison are totally unobserved so there actually is no data no study identified for those comparisons another thing that the app does automatically is calculating the total sample size and the total number of studies in each comparison so this is important for the first assessment for the within study by assessment of bias which is commonly known as selective outcome reporting bias so specifically here the this obviously this assessment is only done for those comparisons for which we have found extra studies that do not report the outcome of interest but only report outed outcomes because we need to assess whether the reason that these studies are not reported the outcome are due to the results found in those studies so if the outcome was not reported the outcome of interest was not reported because we think that the authors didn't like the magnitude of the direction of the results then that it's informative selective outcome reporting bias but the users also need to take into account the effect that this will have on the total synthesized result so that's why in this case the sample size is important for example here we see in comparison six there are two extra studies and we see the we have judged the most suspected bias they bring to oxidin because we think that the two extra studies report did not report the outcome for the reason that I just mentioned because of the magnitude of the the result the direction of the results and we think that's enough to affect the synthesized result but you can also see in comparison nine there are five extra studies that either we thought that the reason that in our report the outcome was related to the magnitude to the direction of the results or the extra sample size was not enough to affect the already quite large sample size that's why we adjusted as no bias detected so obviously as I said this assessment is only done for the comparison for which we found extra studies also for those in group B obviously cannot be done for the unobserved comparison but the next assessment the across study assessment of bias which is also known as publication bias can be done for all of the pairwise comparison this assessment is done primarily considering a qualitative techniques qualitative considerations such as whether the the reviewers were able to get data from great literature or whether there is well-known publication bias for a comparison or that field additionally if there are comparisons with at least 10 studies then some quantitative consideration can also be taken into account such as tests for small study effects and final plots which are also calculated by the app as you can see here the data analysis stop but as I said these are only in addition to the qualitative consideration once again the user has to then select no bias or suspected bias with the relative direction for each of the comparison then to arrive to the overall bias judgment the user can simply press this button here which it says that it applies an algorithm the algorithm is actually really symbolic just check whether there is a in either of the previous assessment as suspected bias in the case then it's suspected bias also in the overall judgment once these level for the overall bias are done then we can move on onto the main output of interest of the tool which is the robin table where we record the risk of bias assessment for each of the network estimates once again the network estimates are automatically grouped into mixed only direct or indirectly depending again on the availability of the data and the first thing here is the evaluation of the contribution coming from the evidence specifically the app calculates automatically the contribution matrix which you can see here and you can also download which shows how much each of the direct comparison contributes to each of the network estimates so this contribution is used then in the robin table together with the previous overall bias assessment to see how the contribution of evidence with suspected bias it splits in the sense of like in which direction it's going and again this is done automatically as you see in the first two columns what the user has to do is evaluating this contribution and this direction so here the user has to make a some sort of like subjective decision for example here we decided that if there was at least a difference of 15 then that will be enough to count as substantial contribution I'll give you some example you can see here that if there is at least 15 percent going in a specific direction then we will go for substantial contribution from bias favoring a specific treatment if it's not 15 percent then it's no substantial contribution from bias but there are also cases like for example let me find some yeah here do loxidine versus fluvoxamine sorry versus mill nasifram it's 30 and 36 so the difference is less than 15 and that's why we selected substantial contribution from bias balance so the next assessment is is a bias assessment for indirect evidence this is because the unobserved comparison has obviously no weight in the previous part in the part of the contribution from the evidence but we had judged the unobserved comparisons in the previous stage in the previous table for risk of bias and this can lead to bias if the reason that the studies are missing is related to the results found in such studies so what the app does here is essentially just considering the previous assessment for the unobserved comparison so the overall judgment that we had in the previous table and copies in this column here as you can see is grayed out for the direct or mixed estimates this is because we have already there's no need to do that for the mixed estimates since the unobserved comparison we have already counted the direct comparison in the contribution assessment so we only need to consider them for the indirect estimates and that's that's what the app does it's just going the previous assessment in this column the final assessment is the evaluating whether there's any evidence of small study effects and again automatically the app reports the unadjusted estimates from Bayesian network meta-analysis as well as the adjusted estimate from the network meta-regression they use the smallest observed variants as a covariate and what the user has to do is check the point estimate as well as the overlap of the confidence the credible interventions to see whether there's any evidence or of small study effects for the network estimates and so I can tell you without going through all the 153 estimates that there was no evidence of small study effects because there was a good overlap between the confident credible intervals and the point estimates were not that different so we can also just set them all as no evidence by pressing this button here and finally we can then go find our overall risk of bias level and the we can easily do that by pressing this button that applied the algorithm that we propose in our paper I don't have time to go through the algorithm but what I want to say is that if the user does not agree with our rules our algorithm rules so I want to use stricter rules or less strict then you can also change this manually as you can see here the app allows you to change it it will just show you with a yellow warning in this sense but also these manual changes can also be done in the previous table and finally both tables can be downloaded which can be useful for reports for example as you see here and we are at the end of our demonstration so I will just show here the link for the app which is as I said already available to use the paper and I also want to mention that we are planning to we are drafting a paper which is going to serve more as an actual as a manual for the app so it's going to be more technical paper there's also going to be a live webinar for organized by the Cochrane team for the Cochrane training and it's going to be on the 5th of May so it's going to be obviously longer than 15 minutes so there's going to be more details about the assessment and also show all the functionality of the app I also want to mention that the Robman is now integrated in the Cinema Framework and Web application specifically in the reporting bias domain where the user is prompted to upload the Robman table I also want to thank the two people that really helped me with the development and improvement of the app so Rory, Papa, Gustantino and Alex all the way and finally I'm happy to take any question at the end of the session or otherwise you can contact me on Twitter or by email for questions or feedback which is always appreciated thank you very much. Thank you very much Virginia for your great time keeping while presenting live but especially for showing us how to use this tool to critically appraise the studies across this evidence base and so our third speaker in this session is Clarice Neville who is a research associate at the University of Leicester. Clarice over to you. Cheers. Great so hi everybody I'm Clarice Neville and I work as part of the complex review support unit based at the University of Leicester in the UK and today I'll be sharing my work around developing a novel multifaceted graphical visualisation for treatment ranking in an interactive network meta-analysis web application. So treatment ranking within network meta-analysis is a really powerful tool the results can often be misinterpreted or used or presented inappropriately as a consequence we as a team wanted to improve the ranking outputs within our anime web app Meta Insight therefore we had two aims firstly ascertain the current methods and visualisations for treatment ranking within anime and secondly develop a novel graphical visualisation for meta insight. To make sure we're all starting at the same place I'm going to quickly introduce anime's so firstly the standard method for gathering evidence systematically whilst also appraising and synthesising the evidence is called a systematic review. To then obtain a quantitative pooled estimate of the outcome of interest in your review one can run a meta-analysis to quantitatively compare healthcare interventions or treatments. The next step is to simultaneously compare multiple treatments which can be done with the network meta-analysis this is done by forming a network of studies and treatments and then the anime uses both direct and indirect information to estimate relative treatment effects and it's through anime's that treatments can then be ranked. Our first step was conducting a targeted review of the literature since January 2011 looking at papers that introduced discussed or compared ranking methodologies of visualisations. This gave us in total 29 academic papers and articles within that 29 there were two articles that were particularly useful. As part of their paper Veronica and authors produced a wonderfully concise summary of common ranking statistics and their primary characteristics and the paper by Cosmire and authors is a brilliant detailed record of graphical visualisations currently used within the field of evidence synthesis. So from these articles what did we learn about treatment ranking statistics and which ones are important? I'm going to bring forward four key points I found. Firstly most ranking statistics are based on Bayesian methods where multiple simulations are run thus giving rank probabilities based on the rank distribution outputted from these simulations. Secondly a popular option that appeared was the surface under the cumulative ranking curve aka the sucra and this does what it says in the tin. It gives a single value but incorporates the entire ranking distribution but naturally still has limitations. There does exist a frequented alternative the p-school however some authors have indicated that its interpretation can be a bit more challenging. Whilst simple methods may be easy to understand such as the probability best they are often more unstable and don't encompass the entire analysis. The opposite is also true i.e whilst more complex methods are more rigorous they can be harder to understand. And finally interpreting ranking in isolation is not advised there are many things to consider alongside ranking results and not doing so can lead to exaggeration of results or misunderstanding. Literature also introduced us to lots of visualisations there are too many to mention in this presentation so I'll focus on those that I felt worked well and fitted within the remit of a single outcome as this is what meta insight currently works with. So first we have rank grams which are a popular choice with lots of variations and they essentially plot all ranking probabilities for all treatment rank combinations. They can present the results on the same or separate axes, can use lines or bars and can present absolute or cumulative probabilities. Cumulative rank grams are also known as sucra plots due to their direct relation to the sucra statistic by definition. Generally rank grams were found to provide informative balance summaries of the distribution of ranks however some people have stated that comparing treatments can become difficult. An alternative that is less commonly used is a radial plot. Generally radial plots have an outcome measure for example sucra marked radially around a circle. Some variations use a donut shape and use colour coded sections to indicate the value. The benefits of this layout include the ability to present concentric circles to consider multiple outcomes of subgroups and including other information nested in the middle such as a network plot. However a limitation of working on the radial scale is that generally humans struggle to discriminate between radio lengths. Next the ability to display multiple elements of an analysis in the same space a multifaceted display allows researchers to present the larger picture and this is great for treatment ranking is allows users to form robust and sensible inferences by drawing conclusions from a combination of sources. However one does need to be careful to not overload the reader with large amounts of information. And finally with advances in technology a natural movement in the field of data is incorporating interactivity. Interactivity allows users to drill down further into the data or analysis as they wish and can even evolve users choosing their own settings. For example this interactive visualization allows users to specify the relative importance of outcomes regarding treatment choices. So that was just a snippet of the literature that we found and it provided great direction and knowledge when developing our own graphical tools for the treatment ranking within Metaminsight. A few things to mention regarding how we developed these tools. Firstly we decided to start with just Bayesian analyses due to its popularity in the treatment ranking field. Secondly designing the tool was very much an iterative process of creating designs, sharing them, getting feedback and going designing again, starting with pencil and paper and moving to mockups of different software. And the final designs were created in R which is what Metaminsight runs on and the GG plot 2 package. Now before sharing what we came up with let me quickly introduce the exemplar dataset for this presentation. This is the in-built example data set in Metaminsight for continuous outcomes comparing pharmacological interventions for the treatment of obesity and the outcome of interest is BMI loss three months from baseline and the NMA was on 24 studies including five treatments plus a placebo. So our first plot we're presenting we've called the Litmus Rankogram. The base is a Rankogram which we chose due to its popularity and easy to understand nature. The aid interpretation we decided to plot cumulative probabilities and have all curves on the same axes and with cumulative probabilities one can then say that the nearer the curve to the top left corner the better the treatment. It was reported that comparing curves can be difficult so to further aid comparison we added a litmus strip of sucre values which has two functions. Firstly to act as a key through the colors and secondly gives the sucre which is easier to compare. So for this example one can see that the ranking analysis indicates that ribbon band performs well as we can see the green coloring high sucre value and probability curve being near the top left. For the converse reasons placebo performed the worst. It's harder to discern between the two curves for metaforming and all estates debatrimen which may seem like a limitation however I believe it actually emphasizes the point they performed equally well. And next we have the radial sucre plot. We decided to develop a second plot for two reasons firstly there's the number of treatments increased a rancogram like before can easily become crowded and secondly we really liked the radial plot with the nested network plot that I showed earlier and so we created this at the price of losing some of the granularity that the litmus rancogram gives. So for each treatment its respective sucre value is plotted radially in descending order we stuck with the sucre to not overload users with various statistics and the sucre considers the entire ranking distribution whilst outputting a single value. To keep things consistent between the plots the same color scale was used but the colors of the nodes also indicating the sucre value. We are therefore aware that sucre is represented in two manners and that could be seen as wasted ink however we believe it further aids communication and impact. So we took the plot by cider et al and took it a step further by overlaying the network plot rather than nesting it. Therefore the size of the nodes represent the number of participants in each treatment arm and the thickness and presence of connecting lines indicate the number of trials that directly compared the respective treatments and we felt that overlaying the network plot meant less work for the user and more impact. The benefit of including the network plot in this way is easy to see if I unpack this example. So one can see that rim and bound perform the best due to its high sucre and green color however it's actually quite hard to see rim and bound due to its small node size from having a few participants in the treatment arms. Furthermore one can see that it isn't very well connected to the network. These immediate factors should tell the reader that the result for rim and bound should be interpreted with caution. On the other end of the scale the large red node for placebo immediately shows that there is a good amount of evidence supporting that placebo did not perform well against the other treatments and that we can be confident of that result. We noted that having the treatment network overlaid could become messy and hard to read for large amounts of treatments or connections. So in such cases we've created a simplified version of the plot which goes back to the original nested design. However we've kept the nodes colored and sized according to their sucre and data. As I mentioned earlier a key message that's present in the literature is that we should avoid presenting rankings in isolation. In this spirit the final graphic ranking tool was designed to be a ranking panel. So here's the design the central pane contains the ranking plots that I've presented as these are the focus the headline act. Then on the left we present the relative treatment effects. This will aid interpretation of the ranking and reduce potential exaggeration as an inherent risk of ranking is accidentally putting undue emphasis on treatments performing differently purely because they have different rankings. Then the aim of the right pane is to give users the opportunity to explore and interrogate the data behind the ranking analyses. Currently this is just a network plot but we have other plans that I'll mention later. Finally I'm going to show video illustrating how the ranking panel looks and performs within Meta Insight our NMA app created with R in the Shiny package. Firstly we run our Bayesian analysis and then click on tab 3c to access the ranking panel and this is how the panel initially looks. The user can then switch between the two new ranking plots as they wish including the simplified radial sucre plot. There is also an option to have the plot in a colorblind friendly scale. All of the plots can be downloaded and the ranking results exported as a CSV file. Furthermore as is the case throughout Meta Insight there is functionality to run a sensitivity analysis where you can remove studies and compare the results. So this new treatment ranking panel is currently live in a beta version of Meta Insight so please do have a look and play around. We do have future plans and ideas including ones that I alluded to earlier in this talk. Firstly there are plans to extend the contents of the left and right panes within the ranking panel. Regarding the left pane we plan to allow the reader to switch between relative and absolute effects. With the right pane we very much intend to increase the number of visualizations available to the user. Current plans include incorporating some form or risk of bias or grade visualization so users can see the quality of studies being analyzed and we also anticipate including David Philopo's bias interval plot where users can ascertain what level of bias could change the results and thus how robust they are. We have lots of ideas for making the panel more interactive including options to easily change the reference treatment where applicable and we plan to look into using the plotly package to enable further information via hovering and unfortunately it isn't just a simple thing to convert these ggplot objects to plotly objects as they have multiple layers. And finally we plan to wrap the newly developed plots into an R package so that other researchers can create them with their own data outside of Meta Insight. So thank you for your time today, I just want to acknowledge Will Stahl-Timmins, the bi-statistics research group at the University of Leicester and the rest of the series U group for their input into developing these designs. And here is the link again to the apps you can explore for yourself. Thank you. Thank you very much Clarice for such an interest in presentation and it's really great to see accessibility in the Meta Insight app as well. You're certainly given Mark Ladrenay's a run for his money with your presentation skills this year so Mark you have a challenge to meet here ahead of your session tomorrow. So our final speaker in this session today is Tasnim Hamza, who is a doctoral researcher at the University of Bern, hands them over to you. Today I'm going to talk to you about CrossNMA, it's a new R package to synthesize the cross design evidence and the cross format data using Bayesian network meta-analysis. So standard network meta-analysis synthesized aggregate data from randomized clinical trials because it is easily accessible from the published literature. However, heterogeneity may be present across these trials and we include participant covariates or effect modifiers as aggregate information to explain some of this heterogeneity. Including being covariates could however induce aggregation bias so ideally we would like to have or to include the covariates on the individual level from every study but more typically we only have IBD from a subset of studies and then aggregated data from the rest. In terms of study design, most NMAs include only clinical trials in the analysis because they are typically at lower risk of bias but on the other hand they are conducted for a restricted set of population and this makes their results hard to be generalized to the general population. On the other hand, the non-randomized studies or observational studies or observational studies reflect better the reality but it comes with high risk of bias. We might need in some situations though to combine these types of evidence while taking into account the biases on each design. And this is what the Crescent MA model is doing. It's a recent extension of the NMA model to synthesize mixture of data formats, IBD and aggregate data and different study designs. We built the Crescent MA model by integrating these four different approaches which combine clinical and observational data into the three-level hierarchical model that synthesizes IBD and aggregate data. How does the three-level hierarchical model work? First of all we define an individual level regression model to include participant covariates. Here beta zero captures the diagnostic effect. Beta W is the within-study treatment covariate interaction and beta B quantifies the interaction between the relative treatment effect and the mean covariate value on study level. And then for aggregate studies we have access only to mean covariates so we can only estimate the coefficient beta B. Then we combine the evidence across IBD and AD for relative treatment effect, delta and covariate coefficient beta B, while beta W is combined only from IBD studies. Now how do we integrate the four approaches to combine clinical and observational data into this three-level hierarchical model? So the simplest approach is to not differentiate between the two types and fit individual regression model for both the clinical and observational studies and do the same for the aggregate studies. But we know that each type has different level of bias and this model doesn't account for this bias. And this brings me to our second model which adjusts the relative treatment effects to the bias on each study. For individual model and aggregate model we add this highlighted part, the bias effect of each study, gamma multiplied by the bias indicator of that study R. The bias indicator R takes values zero or one based on the assessment using the risk of bias tools. However, we know that the judgment using these tools is subjective and carry many uncertainty and we reflect this uncertainty by assigning a Bernoulli distribution to the bias indicator which will provide zero or one to R. But now and to estimate the probability of bias on each study we either assign a beta distribution, lower risk of bias studies are given, this distribution is Q towards zero and high risk of bias studies Q towards one or we could estimate the bias probability using study characteristics through this logistic model. There is another way to reflect study bias by directly modeling the bias adjusted relative treatment effects theta which is estimated by a weighted average of unadjusted effect delta and the bias adjusted effect delta plus gamma. In bias adjustment model one in the slide before this theta is either the unadjusted effect for lower risk of bias studies or is a bias adjusted bias adjusted effect for high bias studies and not both of them. But here we allow theta to be weighted average of both parts but for lower risk of bias studies we give a greater weight to unadjusted and a little weight to bias adjusted part and vice versa for studies at high risk of bias. The last approach in combining clinical and observational data is using observational information as a prior to model the clinical evidence. It is a two-step approach. First step is to conduct network meta-analysis only for observational data using individual aggregated data or mixture of both. In Bayesian framework we will get a posterior distribution of each relative treatment effect. Second step is to conduct network meta-analysis for clinical data but as a priors we use the posterior we get from the observational data. Now some people could argue that usually observational data has a greater sample size compared to clinical trials and this makes the observational information dominate the estimate. So to control this potential high influence of observational studies we downweight the contribution from these studies by increasing the variance from the posterior's hairs or make these distributions further. These models are implemented in the croissant amée package which is a suit of tools for performing network meta-analysis and meta-aggression with individual participant data aggregate data or mixes of both and each format can come from clinical trials or observational studies or mix of both. Behind the scenes models are estimated in a Bayesian framework using Jackson R the package allows for conducting standard network meta-analysis when we have aggregate data from RCTs it allows the inclusion of IBD of these RCTs and it also enables each format include observational studies. So a workflow in croissant amée model is as follow. First of all you start with your data sets participant data and aggregate data or only a single data of one of them. There should be both of them in a long format which means one row bare R, bare aggregate study or bare participant in IBD. We then call the croissant amée model function to construct a JAX model and reformat the data. We might visualize the network and then we run the analysis using the croissant amée dot run function and once the model is fitted we then kind of reduce summary of results, leak table or check the convergence. So let's apply this to an example. First of all we have some individual participant data at the top here. So one row bare each individual and you can see that we have in this example a binary outcome and then we have aggregate data below this one row bare each R study with their outcome and coverage. On each data set you need to have at least these highlighted columns study ID outcome the assigned treatment and the design of the study clinical or observational and for aggregate data you need additionally to provide the sample size column in combining these together in a croissant amée model. First of all you need to indicate the name of your individual participant data and aggregate data. You need then to indicate columns name of these variables and you then choose which model to be used to combine treatment effects across studies either random or common effects model. You set the reference treatment in this example relative treatment effects will be evaluated versus a and finally you indicate which approach you would use to combine observational and clinical data. Here goes one of these four approaches that I talked about before. To conduct network meta analysis meta regression you could set a covariate here we use H to adjust all relative treatment effects and also you could adjust the relative treatment effects to study bias using bias adjustment model one and here there are too many arguments to be set as well all will be described in the help file and the vignette. You can also plot the network using the net plot function you could pass to this function different arguments to control the colors the thickness and many other features which are very similar to the net graph function from net meta package and then you can go away and fit the model using the cross-anime deuteron function it takes the model we created using the cross-anime deuteron model and you need to set the settings for mcmc samples iteration burn and etc. Once we fitted the model I can then print a summary of the results so here it's showing us the mean standard deviation etc for each of the parameters we have here the regression parameter b the relative treatment effects d and the bias effect plus the two heterogeneity parameters also to provide the convergence information the Gilman-Robin statistic and the number of effective sample size you could also produce the tracer plot to check convergence of each parameter and finally you can you can create the leak table by cross-anime leak function so here the values in each cell it shows the relative treatment effects and the 95 percent credible interval of each treatment on top compared to the treatment on the left so I really just gave a brief summary today if you would like to find out more and you can check out our package on on github website soon we will submit this package to Cran with the illustrator documentation and detailed example we also submit our methods paper thank you very much thank you very much Tasman for your excellent presentation on cross-nma and how it can be used with both individual participant data or aggregate data and I'm really looking forward to that paper and its arrival on Cran so that's it for this session on NMA we hope you enjoyed it as much as we did and thanks so much to all our presenters we'll see you at the next session which kicks off with multiverse meta-analysis and we'll start in just over 30 minutes from now thank you


Today, the entire globe is a potential theater of operations for the United States Army. For these officers, this is more than just a statement of fact. It is their working context, the basis of their mission. They are members of the test and evaluation command of the Army materiel command. What are the effects of jungle heat, humidity, the other natural enemies of materiel found in the tropic environment? What can be done to minimize their destructive effects? The test and evaluation command must answer such questions for every type of natural environment on Earth. In the frozen wastes of the far north, in the world's barren desert areas, as well as in the humid tropics, the Army constantly seeks answers to such questions as how can clothing, rations, field shelters, vehicles, radios and weapons be made more resistant to the ravages of environment? How well are American soldiers equipped to live, move, communicate and fight in the jungle? The Army's key center for getting these answers lies in the Panama Canal Zone. This is Panama City, capital of the Republic of Panama, modern, clean and helpful. It is a place rich in history. Here stands the old city destroyed in 1671 by Sir Henry Morgan in his ruthless search for Spanish gold. Majestic cathedrals, reminders of an age that has passed, still serve the city's faithful today. As seen from the Gulf of Panama, the latest in architecture dominates the city's skyline. An up-to-date airport accommodates the sleek jet aircraft of many nations. Modern recreational facilities are plentiful. Panama is unexcelled for either salt or freshwater fishing. The long-famous Panama Canal still stands as one of man's great feats of engineering. Yet for all its modern aspect, Panama retains the flavor of its past heritage. The colorful folk dances of the Panamanian people. This center of commerce and culture lies the primitive jungle, setting for the United States Army's Tropic Test Operations, the Big Green Laboratory. Since 1962, the United States Army Tropic Test Center has used the canal zone's wide variety of natural environments to test men, materials and equipment in the humid tropics. The crucial need for such testing is clear when you remember that some 20% of the world's land area has a hot and humid climate. The Panama Canal Zone is uniquely suited for tropic testing. In its relatively small area of 500 square miles, nearly every feature of the humid tropics can be found, and three different types of tropical environment exist. For example, the Atlantic coast of the Isthmus is covered with wet evergreen forest, here from 100 to 200 inches of rain falls each year. Even during the driest period, February and March, it rains on about 15 days of each month. Temperatures range from the mid-80s in daytime to the mid-70s at night. The relative humidity hovers at 100% almost every night and often during the day. These are the same conditions that military equipment and operations would face in southern Burma, Malaya, parts of Central and South America, and parts of Africa. In general, it is a region teeming with all kinds of plant and animal life. On the Pacific coast, there's a second major type of tropic environment, semi-deciduous jungle. From 60 to 100 inches of rain falls each year, and there are distinct wet and dry seasons. In February and March, the driest period, less than one inch of rain falls per month. The ground bakes hard and often cracks open. The grass turns brown and some trees shed their leaves. All conditions found in much of Vietnam, Laos, Cambodia, parts of Central and South America, and parts of Africa. During the wet months, however, there's ample precipitation to qualify this area as a humid tropic environment. Southwest of the zone at the Rio Hato Military Reservation, broad reaches of tropical savanna or grassland provide a third major environment, typical of the humid tropics. Rainfall measures only 40 to 50 inches a year. The dry season is longer, and daily temperature and humidity variations are greater, as in the drier sections of Central Thailand, parts of Central and South America, and parts of Africa. Within these three major types of environment, there's a wide variety of terrain, vegetation, soils, and moisture conditions useful in testing equipment and materials. For example, there are sandy beaches, mud flats, coral reefs, sea cliffs, hills, steep stream banks, mountains, mangrove swamps, fern swamps, and several types of palm swamp. For mobility tests, there's an ample variety of soil types. Players include the Chagres River, plus many smaller streams, and the Big Gautun Lake. So much for the Big Green Laboratory's natural resources for tropic testing. Now let's take a look at how these resources are put to use. The Army Tropic Test Center at Fort Clayton in the Canal Zone is one element of the Test and Evaluation Command Army Materiel Command. As part of TCOM, the center's mission is to plan and carry out tropic environmental phases of engineering tests, service tests, evaluations, check tests, and confirmatory tests. The center also does research in support of its own testing mission and coordinates and supports research and development test projects of the Army, Navy, Air Force, and friendly nations. The center's permanent staff includes more than 20 military and civilian specialists in such fields as electrical, mechanical, aeronautical, chemical, and human engineering, as well as in geography, microbiology, meteorology, and soil sciences. The center also administers and supplies a locally-based Army meteorological team. The team in turn supplies meteorological data for support of the center's projects. Administrative and logistical support is also given to the Air Force Systems Command's Scientific and Technical Liaison Office, which arranges for Air Force support of test center projects. The headquarters building of the Tropic Test Center also houses its Soils Laboratory. Soils are tested at various locations in the field and samples taken for detailed laboratory study. The particular test being conducted here is to determine penetrability. Other characteristics of interest to the center's scientists include temperature content, stratifications, consistency, texture, color, acidity, alkalinity, compressibility, strength, plasticity, and remoulding properties. The soils are grouped according to a scientific system by which hundreds of thousands of soil types from all over the world are classified into 50 major groups and within these into many more orders and suborders. Much classification makes possible a scientific comparison of soils which is of great value to both military and civilian research worldwide. In the laboratory samples are analyzed for both chemical and microbial content. Special attention is paid to those properties which affect the corrosion of material and the evaluation of vehicular traffic ability. As part of all tests, behavioral scientists consider the human element. How well does man match machine in the tropical environment? Nearby, Tropic Test Center troops are quartered in barracks typical of tropical Fort Platon. A few miles away adjacent to the mirror flores locks are facilities for chemical, microbiological, and botanical work in support of the center's research effort. Inside this building is one of the world's most complete collections of Panamanian plants. Thousands of plants from the zones, test courses, and research plots have been identified. Many of these plants serve as hosts for microorganisms which speed the deterioration of material. To study these microorganisms and those found on surfaces and in the tropical atmosphere to determine their role in deterioration, this is the laboratory's primary job. Much new understanding of the deterioration process is being gained by center scientists and this too benefits civilian research efforts as well as military. Support headquarters are located at the Las Cruces Logistics area which also accommodates munitions test grids. Also located here is the center's motor pool. There are open storage sheds for deterioration testing. Also a small photographic lab and supply rooms. About half of the center's projects are conducted at Fort Sherman on the Atlantic side of the Isthmus. In this building the center operates a field office. Inside air conditioned space is available for visiting test teams to use as field laboratories or offices. Billeting space is also available. At material exposure sites in each of the various tropical environments hundreds of samples of material are tested. For instance various plastics, paints and other preservatives of kind used for both the military and civilian community. At this breakwater site materials are exposed to a heavily salt-latent atmosphere. Test items at all sites are regularly inspected and the results recorded. Civilian producers of the items tested benefit from these findings and the resulting product improvements benefit both military and civilian users. This site provides an open country inland environment. Periodic comparisons are made of the effect of exposure on various materials in the differing environments. Four main vehicle mobility courses are in use. Two on each side of the Isthmus. These courses afford a variety of terrain types, physiography, soil characteristics and vegetation as well as such features as abandoned gravel and dirt roads, grassland, jungle, streams, lakes and rivers. At a 14 acre chemical grid area such munitions as smoke, white phosphorus and other types of hand grenades are tested. As in all tests results are carefully recorded for detailed analysis. In the Cerro Tigra area environmental exposure tests of chemical material are conducted. Also tested in this area are such items as chemical agent detection kits, protective clothing and protective masks. Location the center operates a small chemical biological laboratory primarily in support of chemical deterioration tests. As part of a long-range environmental data-based program several 150 foot towers are being set up in each of the tropical environments. The objective is to compile year-round environmental data at various heights from ground level to above the treetops. This white filter blackened by particles from the atmosphere has been exposed for only one week. The particles were collected by high volume samplers as a part of studies on atmospheric contamination. Bubblers collect chemicals from the atmosphere at various levels of the tower. An air sampling device is used to draw off microorganisms. In general the environmental data-based program calls for quantitative analysis of the microbial, chemical and particle content of the tropical atmosphere to determine the influence of region, season and height above ground level. Laboratory analysis of airborne contaminants are throwing new light on the nature of the deterioration processes which so violently attack materials in hot and humid climates. Also as part of the data-based program facts on soil, vegetation, animal life and microorganisms will be recorded for a period of years. Thus testing agencies will eventually be provided with a body of knowledge on tropical environments far more extensive than ever before. Here are some typical examples of tropical deterioration suffered by plastic, glass. To combat such destructive effects the Tropic Test Center has conducted or taken part in a wide variety of projects. Concepts, ideas and theories, as well as materials and hardware have been rigorously investigated to determine the effect of the humid tropics on the soldier's ability to live, to move, to communicate and to shoot. For example, how well can a soldier hear the enemy in the dense jungle? What sound frequencies carry best through thick vegetation? How well can the soldier pinpoint the source of such military type noises as troops marching, truck convoys or undergrowth being cleared? Just how good are such ideas as soft landing devices, beacon lights to guide paratroops to assembly areas or glide angle indicator lights to assist in safe night landings by aircraft? Liquid are certain items of equipment. For example, jungle boots, collapsible canteens, plastic water containers, tactical radios, protective masks and clothing, armored vests and shipping containers. What is the effect of tropic humidity on Army firepower? Guided missiles. And how well will the Army's newest weapons survive the tropic environment? For example, the redout, Sheraton-Shelele system, the chaparral. These have been highlights of the work of the Army's tropic test center. The center is relatively young in years, but it is dedicated to a time-honored tradition that the American fighting man deserves only the best. Keeping this always in mind, the tropic test center works unceasingly to provide a kind of soldier's warranty to assure his ability to live, to move, to communicate and to fight in any of the world's humid tropical environment. All this is a vital part, but only a part of the vast work directed from the headquarters of the test and evaluation command at Aberdeen Proving Ground, Maryland. Since 1962, at more than a dozen TCOM test centers, virtually every item used by the American soldier has been evaluated in every possible climate and environment and in every phase of development. Engineering tests. Can a vehicle take the punishment that combat terrain may give it? Is its water capability up to the standard set? Can it meet the needs of the soldiers who will have to depend on it? Where dependability is often literally a matter of life or death. The answers are found in advance. For how about a multi-unit design which is intended to function as a rapid assembly assault bridge when the units are linked together and also serve as individual power ferries for vehicles and armor if the need arises? Will the concept really save hours of precious time at a vital water crossing? TCOM experts, check it out. Will a new design for radio communications within the rifle squad do its job in actual use in the field? Intensive troop testing by TCOM will reveal the answer before any combat unit is asked to depend on it. Weapon systems which today stand guard over key installations in the combat theater were first tested and evaluated in depth by TCOM. Weaknesses searched out, strong points revealed and reinforced. The ability of a new oven designed to produce hot biscuits for hungry troops in the field or of gloves to prevent frostbite or of field boots to resist rot or of the fabric of a field uniform to stand up to abrasion or of rain garments to keep a soldier dry. These things and thousands more must concern the people of the Army's test and evaluation command. Because these thousands of items are destined to become part of the vast catalog of equipment, supplies, arms and ammunition on which the effectiveness of the United States Army directly depends, each single item, every last component of every one of the tools of a soldier's trade must be as efficient, durable and reliable as intensive testing and evaluation can make it before it is delivered to the combat zone. When weapons and ammunition reach this point, the time for testing is over. TCOM makes sure that testing has been done right. Here the time for evaluating the effect of a dusty environment is passed. The ability of a vehicle to overcome a terrain obstacle must already be proven before it faces that obstacle within artillery range of enemy forces. In cloud-shrouded mountains and in the dripping jungles below, the American soldier has a job to do. No part of it is easy or comfortable or safe. He must be able to have perfect confidence in his equipment, his weapons, the modern means of transport that makes him the most mobile, hardest-hitting fighting man in history. The American soldier has that confidence and it is well founded thanks to the professional skill and dedication of test and evaluation experts half a world away. The people of the United States Army's test and evaluation command.


Hello, everybody. Good evening. I was gonna say good morning, but it is definitely the evening. Welcome back to another vlog. I've had quite a long day of work and now Drew's off work.  I need to return a couple of things to UPS. Drew needs to get something notarized and he went earlier, but the notary is only there at like 3 p.m. onwards. So we're gonna do that and then maybe get some groceries. We leave in like three days, I think. So we need to just have like a couple little things, but anyways, I need to gather my returns. All right, got the goods. It's a bra that didn't fit. I've already exchanged it for a new size. The new size does fit, but now I have to return the other one and then I have one in my car and then I think we're good to go. Let's go, adults. I did not sign up for this. Super quick grocery haul. I feel like I always need to disclaim because if I don't say this isn't like a full grocery order, people are like, where is, I'll just like grab a few things that we need until we go out of town, one of which being oat milk. So we got that. We're trying to get through this lactose free milk that they accidentally gave us, like our Instacart person, but it's disgusting. So we had to get oat milk. We got these Biscoff, these ice cream sandwich bar things. They're not sandwiches at all. I don't know, ice cream bars. They have a chocolate and then we also saw that they had a white chocolate. These look delicious. I'm not like obsessed with white chocolate, but it looks really good. So we had to get them. We got an avocado. I love avocado at lunchtime, like on rice cakes. It's like my favorite thing. I got some strawberries to put on my yogurts in the morning. We got a couple things to bring to Disneyland just to have for like snacks. So we got these peanut butter crackers. We got some beef jerky to take with us. The teriyaki flavor is the best flavor of beef jerky. We got some pistachios to take with us and then Drew's mask. And then some other things that we were out of was spring mix for my salads. We got some spaghetti sauce. We just got like the basic kind. We got some salad dressing. This is just Italian. I don't know. I like to change it up sometimes. And then we got some spaghetti. There's our grocery haul. That was expensive because these two things total were about $30, which is illegal. It should be illegal to clarify. He's kidding. Okay, bye. All right, finished my workout. I was very sweaty. We're watching escape to the country, obviously. And we're going to try these bisque off ice cream sandwiches. I'm very curious.  Well, I'm not going to try it. I've had the regular chocolate, but we're going to try the white chocolate. So somebody hadn't gotten on the couch for about two years and then just started getting on the couch again. So that's a new thing. We're going to have to get some kind of like blanket or cover or something like that for that couch because the girl's a mess. You have never had, you've never had cookie butter. You haven't lived. And this is not nuts because I don't like nuts in my ice cream. This is chunks of cookie. It's delicious. Well, let's see what yours looks like. Oh, well, mine looks a little better. Mine's not very attractive, is it? Can I try it? You want to try it? I've never had it before. All right, let's see. This one? You do. Whoa, I did not expect that. I mean, this one's still phenomenal. Whoa, that one is better. It's the next day. I just finished a hot girl walk. I also finished taking Layla on her hot girl walk. I can't go for as long as I can. So that's separate. It is currently 7 20 a.m. Literally 7 20 in the morning. So feeling good. I just opened an airy package and I was like starting to open it. I was like, well, I guess I could open it with you guys because I don't know of anyone else's this way. But I've been having a hard time finding good underwear. I end up just buying it from Target or Airy. But I need to do a video where I test out some underwear, like a main channel video. But anyways, so first thing I got is this cute little bralette. I've been needing a more sand beige colored bralette just to wear under white things. And this is not padded. It's just kind of comfy. I really like bralettes that I can also sleep in and picky. I got another bralette. The thing that like upsets me about Airy is that they know they can make plus sizes American Eagle included and like they don't, which is really annoying. I also got this bralette like tank top. It's got like a built in bra, but it's also like a tank top like a long line. And I thought that was really cute. I'm definitely gonna like sweat in this but whatever or I guess I mean my sweat is gonna show. And then I got some shorts. I had a feeling I had a feeling these were sweat pant shorts. See I don't know like they are cute and they look really comfy. I got them a size up and they look too big. So I might need to return them but I just wanted some like high waisted shorts to wear around the house that were like a little bit longer and these aren't all long enough. So anyways see about those I'll try them on. And then what is this? A tank top? I had like a $75 like reward to use or something like that. So I used it. This is cute. Okay this is like a crop top. I'll wear this with jeans with leggings.  Very adorable. And then I saw these bike shorts which like you might be thinking don't you have enough? No I don't. These look really cute. So these are what are they called? I'll fly an OG bike short. These are the seven inch ones. I don't think it says but these are just like a ribbed bike short and like they have a really cool ribbed material but then on the bottom they have like a lettuce hem. How freaking cute are these? Adorable. They're pretty thick too. So I might actually pack these to wear to Disneyland this week. And then I ended up getting three pairs of underwear because I wasn't sure which style I was gonna like. So I got three different pairs to like see if I like any of them and then if I do I'll like order more. And I think they have like it's like three for 20 or something like that. I'm not sure. Okay so first we have this pair. This is like a more high rise pair. I think these might be mid-rise or something but they have some ruching on the butt. Kind of cute. Loving the green. And then this pair is both of these are actually the same material and they're like the same material as this tank top. Like it's really really really thin. So I'm like wondering if it'll eventually get holes in it. But this pair is the high cut bikini. I'm just trying out different styles because I pretty much wear the same like low cut straight across. And like when I wear high waisted stuff it doesn't look great. So high cut. I don't know I'm gonna try that. And then this is called the boy brief. And this is just more of like a loungey. I love this like neon kind of lime color. Is it weird that I'm showing underwear? Whatever. I don't care. So there's my airy haul. I need a shower. I need to wash my hair. I need to get ready. I'm filming today. Do you need to go outside? Okay hold on. I also started my period this morning. So it's gonna be a rough one but we're gonna get through it. Definitely need to get some Tylenol. But first I'm gonna eat breakfast. It is a little while later. A lot later. My hair is mostly dried. Now I washed it earlier. And I kind of want to give this like red one blow dryer thingy another chance. I've tried it a couple different times. I just can't seem to have any love. But I've seen some feedback from people on how to like really turn it when you're using it. I was just kind of doing it straight. I'm gonna like see if I can give myself a little blowout moment. But honestly I don't really have a lot of time because we have plans tonight. And it's already like the afternoon. And it's just taken way too long to procrastinating today. But like let's see what we can do. I'm gonna give myself five minutes. See what we can come up with. Okay you know what it could be better. But I'm honestly not too mad at it. Like it does look like a little bit of blowout. I was about to say this is a fail but now that I'm looking at it it's it's not that bad. Okay sure we need to do a quick makeup routine here. I am manifesting that by the time I leave the house tonight I will have said yep I filmed two videos got them out of the way so wish me luck I've got three hours. All right I threw on the bike shorts the new bike shorts that I got. These airy ones so comfortable just because I'm gonna be like sitting on the floor and I threw on this like scarf top that I got in a mystery thrift box. Drew is really likes my hair. I don't know how I'm feeling about it like in person it doesn't look as good as it looks on camera. Maybe that's just in my head I'm not sure but either way we did it we got ready let's film some videos. Oh dear god imagine if that just fell over and broke I would cry. Forgot a memory card as always. Okay we've acquired the memory card and we'll catch up with you in a bit. Aw that's that's a painting that Drew's niece made me and she told me to put it up in the window for when I'm filming my video so there they are. It's a little while later it's actually four twenty and I need to be done by five doing this so that Drew and I can cook dinner and like get ready and leave and stuff by six o'clock so I'm doing pretty good. I'm hanging in there with one more video to film and hopefully it'll go by really really fast. Let's see if we can crank this out in like 40 minutes 39 now. In case you were wondering I did it I finished both of my videos Drew's brushing his teeth I just finished brushing my teeth we're all ready to go we're gonna head off the door and go to our fun little karaoke night we can't stay out too late because he goes because Drew has to work at seven in the morning I have a bunch of stuff to do this week we leave in like a day and a half to go to Disneyland so we just have a lot of stuff to do before we go but we're gonna like hang out for a little bit.  Are you ready? I have to pee. I am wearing actually besides my shoes and all thrifted outfits so this shirt is from Thread Up it's so comfy and then this skirt I actually recently just got when I was thrifting with Alexa and then of course my white cowboy boots because it's a personality trait at this point both my necklace and my earrings are from Amazon and I cannot believe this but we're going out on a weeknight who are we by out I mean we're gonna be home in like two hours okay let's go. This one is sad we're leaving I'm sorry you can't come it's for people only but I'm sure Sierra would love to see you. We're back we have our drinks Tuesday night we're at the club but everyone's like on their way they're almost here but uh we've got a little backstreet boys coming in hot this tastes like a capri sun it does we got a drink called a kinky cooler and it is um basically a kiwi capri sun it's making me kinky look we did promise I'm excited I was thinking the same thing they're like I've been searching my whole life to find the whole chocolate fun too this one's for Frankie yeah this is for the bonus Jonas gotta go what decisions are happening over dinner we're mostly off camera but singing along it's always so good to catch up and we'll come visit them in San Diego soon so you'll see her soon yes we're gonna try to do a trip with the boys yes everyone else okay love you bye my battery is going to die it's flashing but we just got home it's just before 10 p.m. it was such a fun night I'm so glad we went I'm drenched in sweat like it was so hot in that room it's crazy but that's gonna do it for today's vlog I hope you all enjoyed it thank you so much for being here and I will see you in the next vlog bye


Oh my God, I thought he had already the worst wind in the world, but no, this one here is the worst. And I'm Oli, and this is our hero, Bumblebee. Together we are attempting a Guinness World Record to become the youngest pair to circumnavigate the globe by motorcycle. Join us for season 3 here in South America. Good morning world, welcome back to the channel. It's day number 267 on our circumnavigation around the globe by motorcycle. We're here in the small town of El Calafate in Argentina, and we're staying in this little cabin where we've been for the past six days, I think, trying to catch up with some editing. Yes, and unfortunately we still feel a little bit sick. We really try to recover and try to get healthy as much as we can, but it's not 100%, but we are ready to hit the road. And it's really crazy because this here is our rent for two days. I mean, I want to throw it in the air. I don't do it, but it's pretty crazy, hey? But let me show you guys where we're heading today. So we are here, and today we are going to be making our way across to Glacia Perito Moreno before coming back through El Calafate and up this road round to El Chalten. Yesterday we are heading to the star of this region. The whole reason that El Calafate is such a popular tourist town, it is for this glacier, Perito Moreno. I really can't wait to see it, and it's already nine o'clock, so better hit the road. Let's go. Look, here's our little cabin. That was actually really nice. Yeah, I mean, you don't really need more than that. Oh, hold on. Oh, damn it. The battery again. Hold on. Sorry. Please just start this time. Come on. Otherwise, we have to turn it around and push you down the hill. Come on. You can hear it clicking, and if you go to the battery voltage, 11. I think it will start. Just give it a minute. Okay. So stand is definitely up. Come on. Come on. Come on. Come on. Come on. Come on. Come on. Put it in neutral, perhaps. Okay. Oh, come on. Yes. Yes. That's strange, because it looked like it was something to do with the first gear, but you know, I have to stand up the whole time, so. Oh, man. Yeah. I think when you leave it for five days or six days like that, you know, without starting it, it can be a little bit dodgy. Heart attack in the morning. I was excited to try out our first jump start. Me not. So it's about 50 miles to the glacier now, but it should be a quite nice ride, because we will have a nice view of the Lake Argentino. Hopefully, let's see. And then hopefully a nice view of the glacier. Yes. Back on the bike. What a view in the morning. Hey, I just really love the colors. The colors of the lake is really light, bright blue. Looks incredible. Yeah, stunning, stunning landscape. And look at these rock formations and this rocky cliff here. So we're just coming up to the entrance of the National Park. UNESCO World Heritage Site National Park. Cooler. Because this glacier is inside a national park called Park Nacional Los Glaciarres. So like basically the Glacier National Park. Gracias. Going in. Woo. So we actually read online that it was 8,000 pesos for two tickets. So there's like 20 pounds a ticket. But when we got to the entrance, apparently they've actually just increased their prices. So it was five and a half thousand pesos per person, which is about 27 pounds per person. Wow. So yeah, in total it cost us over 50 pounds to get in this national park. Yeah, it's not a cheap trip if you want to go around and see all these natural wonders. But you know, you've got to do it. Oh, it's getting exciting now. The glacier is just on the other side of this headland here. So we should be getting some glacier views soon. Glacier, glacier, glacier, glacier. Yeah, it's a pretty cool ride here as well, just along the lake. Yeah, and I looked on the map and actually this is the same lake as on the other side of the headland. So it actually wraps around this bit of land. And the entire lake is basically the meltwater of this massive glacier that we're about to see. Wow. So there's a lot of water going on here. And did you know that 95% of all the earth's fresh water is actually frozen? Some info in the morning. Oh, wow. There you go. Look at this. Wow. And it's so blue. Whoa, there we go. Oh, that's bright. That's crazy. The rest of it is very dark and gray, but the glacier is just like insane. Wow. Awesome. Let's ride a little bit closer, shall we? Wow. I never saw a glacier like that. Never ever. It's so big. That's crazy. It's so impressive. Oh, we're coming up close now. Look at this. Wow. Nice. Wow, it looks unreal. Hola, buenas. So we were hoping to do a kayaking trip up to the front of the glacier, but we just messaged the operator on WhatsApp and they said that the weather today is not good. So actually they said that they're not running any kayak trips today. Unfortunately not. Yeah. Super cool because this guy, he just opened and we were like, are you selling boat tours today? And he's like, yes, yes. The next one is in 20 minutes. And we were like, okay, we take this one.  Okay, we've had our safety briefing and now they said we can go up to the upper deck and have a look. How magical, hey, how magical. You can see all the massive, massive chunks of ice that have fallen off the glacier. But actually it's really interesting because this glacier hasn't really changed in size for like 100 years. So like most glaciers have been retreating and retreating. This one was actually advancing and it's pretty much where it was about 100 years ago. And the ice here is actually flowing down from up on the Patagonia ice field, which is up in the mountains. And it's flowing at around two meters a day. It's insane. This glacier is massive. So at the front of the glacier here, the height ranges from around 70 meters to 40 meters. So I think the bit right in front of us here is 70 meters high. That's a huge, huge wall of ice. But further up in the Patagonia ice field, the glacier is apparently something like 700 meters deep, which is just a lot of ice.  It's really hard to get a sense of perspective, but I mean, it is huge. Bye bye, most beautiful glacier in the world. What an incredible experience, hey? The best, absolutely the best. I think that's a really good way to see the glacier. There are trails you can do, hiking trails and see the glacier from the land here. But I mean, you can't get much better than getting on a boat and getting up close and personal to it. That's incredible. Yes, I think so too. So now we're going to make our way back over to El Calafate. We're going to grab some empanadas in town and then we'll hit the road north. Conno, the empanada place is closed. We'll have to find another option for lunch. Yes, we found a solution for lunch. It's not empanadas, but it is a nice vegetarian burger and fries, so that'll do. That was such a nice veggie burger, good on your YPF petrol station. Yeah, when you can't find something in town, you can always rely on going to a petrol station. So the bike is fuelled up, we're all fuelled up and now we have 130 miles to ride basically around Lake Argentino and over to El Chalten. Yes, I'm super excited. I checked some rooms on Airbnb and booking.com and they are so super expensive, so it must be a very, very, very nice town. Well, as El Calafate is famous for the Perito Moreno Glacier, El Chalten is famous for Mount Fitzroy, a pretty epic peak and the town is like right next to it, so it's going to be really, really nice. Yeah, our navigation says it takes about two and a half hours. It's already 340 years. Yeah. We'll get there, we'll get there. Yes. Goodbye El Calafate. Bye-bye. It was a really nice day.  Okay, it is actually bordering on dangerously windy out here. Yeah, I mean that's really not funny how windy it is. I don't know if I've ever seen the bike at this angle before on the road, like the amount of lean I have to do to push back against it is ridiculous. Wow.  This one here is the worst. And I'm only going 45. Lake here, on the river. It's been very blue river. Wow. Oh, I didn't realize we're actually going over it. Cool. Look at this color. Wow, this must be the river coming out from Lake Argentino. Yes. Look at that. Wow. Oh, wow, it's amazing when you have the wind behind you. Oh, you can't even feel it anymore. Oh man, why it can't be behind all the time. Wow, look at this river. Amazing. Wow. That's so beautiful. And no wind. Jesus. Okay, this is going to be a tough 100 miles. Yeah. Yeah. And the atmosphere is hazy from all this sand, from all the dunes and everything being swept up in the air. This is like the Sahara all over again. The peaceful ride I was imagining. And that is, when we were riding the Sahara, the wind was coming from the other side. So that was wearing the, that side of our tire. And now the wind is coming from this side. It's actually wearing the opposite side of our tire. Perfect. So it worked out really well, actually. Puzzle the twisting and turning of the roads. Press the winds coming from one direction, then the next, then here, then there. Yeah. Okay. Looks fine. Yeah. Okay. Look at this place. Yeah. Is this one like a little dog or something? I don't know if it's a dog. That's definitely some human excrement in here. It's basically an abandoned toilet block, right? But people are still using it as a toilet block. There's so incredibly windy around here as well. Literally, this is the only place where you could pitch up a tent. And even here, with this kind of little wind protection, it's not really ideal. Wow. It's so windy here. Have a look over here. Pretty cool views. Okay. So after a bit of deliberation, we've decided that it is just too windy, too rocky, too exposed to camp there. And as Lavi said before, camping in El Chaltén is prohibited, like wild camping. So it looks like our only option is to go to a paid camping in town. Be careful here, Ollie. Yeah.  Okay. So look both ways. Yeah. Try and get straight. Hold on. Okay. Okay. It's just super windy. Yeah. Wow. That's not the easiest task. At least the paid camping in town is a little bit cheaper than the rooms in town. Apparently, we saw one on iOverlander, which is only 2000 pesos per person per night, which would only be like 10 pounds each. This is not too bad. But anyway, it's basically between that and having the windiest night of our entire lives. Wow. What a place. Yeah. Finally, coming into El Chaltén, this cute little town nestled inside these hills in the lost glacier in its national park. Wow. Awesome. Oh, wait. This is the petrol station here. This is the petrol station? Yeah. Go on there. I think so. Yeah. YPF. This is like the craziest petrol station I've ever seen. Look at this. Yeah. Totally. I hope they have some petrol here. Yeah. Yeah. Wow. Ha. That is a remote petrol station. Yeah. It's like they sort of brought it here in a truck. Yeah. Hola. Hola. ¿Qué tal? Well, we were incredibly close to running out of fuel. I think the bike estimated we had less than 30. Frio. Frio. Oh, frio. Mucho. Viendo, viendo. Crazy. Loco, loco. Vendo. With the motto, pooh. We can see nothing. Muchas gracias. Muchas gracias. Okay. Wow. That was nearly the full, full tank that we just filled up then. I underestimated how much the wind would affect our mileage. Yeah. Totally. Because we filled up and then we only did 130 miles and that literally drained nearly the whole tank. Insane. Really insane. Yeah. Look at this. Wow. Beautiful. Totally beautiful. It might be windy and it might be raining, but I can appreciate this location. Okay. Yeah. Over the bridge here. Real Fitzroy. Oh, pooh. And now you sort of going to the left, roundabout and then left. Bienvenidos El Chalten. Awesome. Wow, what a town. Looks epic, man. And then in 600 feet you turn to the right. 600 feet. Yeah. You got a feet set up on there. Apparently. Okay. I've forgotten how much a feet foot is. I don't, I don't know. No. Not, apparently not here. Okay. Apparently the next one. Okay. Whoa, look at this cliff. Wow. Wow. It's absolutely crazy, man. That is an absolutely insane cliff. Apparently it's here. Like this place here. Yeah. Okay. El Padrion. Yeah. I think so. Okay. Let me check again on over all of Orlando. You don't need to because there are tents here. So yes, they have space for us here. They said we can pitch our tent somewhere around here. Perfect. So they said basically anywhere from this pole this way. So basically we were just picking this little spot here in front. You can see how many tents there are here. Look at this. Crazy. Good evening guys. We made it. Happy Nalai. And it's even windy here in the camp. Can you believe it? But look at the view in front of us. We are next to some major cliffs. Wow. Epic. Epic. What a day from a massive glacier to a windy, windy, windy road. Like the windiest ride of our lives. That was boarding on dangerous. I was being... Wow. Man, that was crazy. Yeah. I'm ready for bed. But the forecast looks better tomorrow than today. So fingers crossed we can get some nice mountain views tomorrow. So that's it from us today. We did at the end 280 miles, which is about 370 kilometers. And we hope you enjoyed the video. If so, please give us a thumbs up, subscribe to the channel, share the video with your friends and family. Comment below and we will see you next time.


What's up everybody, Jason here for jasperjasecasebrakes.com. Last break of the night, giving away five makeshift spots for tomorrow's mixer. 1920, mosaic, fast break, pack, random teams, number eight. And again, 30 total spots, randomizer, customer names, teams, distributed hits out of the cello, or sorry not this cello, this fast break, mosaic. Just remember no veteran base commonship. And at the end of the rig, a randomizer, customer names again, and a separate dice for a randomizer. And it looks like top five get a spot in the $299 summer heat basketball mixer. So there you go. All right, smokey. Have a good night, man. So let's roll the dice and let's do the break first three and a five eight times. One, two, three, four, five, six, seven, and eight. Eight, eight times. CJ down to Sean. And then eight times here. One, two, three, four, five, six, seven, and eight. One, two, three, four, five, six, seven, eight. CJ with the hogs, Ryan with the jazz, Kevin with the suns, Kevin and those others. Patrick with the T-wills, Jacob with the blazers, Patrick with the magic, CJ with the Celtics, Darren with the nets, Jacob with the pelicans, Mike Tower with the calves, Tony Barkie with the Bucks, Matthew with the Clippers, Joel with the Pacers, Ryan with the nuggets, Kevin M with the bulls.  So again, no one's going to trade guys. We'll close the trade window, distribute the hits out of the pack, and then we'll see who wins those five mixer spots for tomorrow. All right, here you go. Good luck guys, a little fast break. Here we get out of here. We got Furcon, Little Disco, Vince, Sturkover will chamber them. So there you go guys, quick sell, I mean quick fast break pack. Now let's see who's going to win spots in the mixture. There you go. Top five get in after six and eight, three, nine times. One, two, three, four, five, six, seven, eight, nine, nine times. Joel, you're in. Jacob, Stephen, Sean, and Ryan. Look at that winning spots and back to back pack breaks after nine times. So there you go. Top five in. Congratulations. Another one is on the website already guys, number nine, and we can get another one going, jazbystcasebricks.com.


Good afternoon, everyone. This is Melissa with thestockswish.com and welcome. I thought I would do a review of the gap I did, actually two days in a row now. So it was a gap yesterday. There was a new gap. And today I did a continuation gap because it just really set up very nicely. And this is just bleeding down now. The target for this, the overall target is $6. It's a lot to expect of that in one day. The stock has had a big move in the last two days here and had a nice trade in it yesterday and today. So here it is. So looking at the overall picture yesterday, the stock gapped up. I mean, I gapped down, but rallied first. And that's okay. It rallied and still held a resistance area. In fact, the highest resistance area I had written down for this was $7.50. And it ran up there and held. So there was a nice short setup in here to play down with a good risk to reward into the afternoon. I was out of this front lunch. I did not redo this. But this, I was looking at this and I said, crap, I should have redid this. Had a nice setup yesterday afternoon and then dropped into the close. I typically do not trade in the afternoon. So I saw it, you know, as soon as I got out of it and then I was rallying lunch, I saw it, but I didn't go back into it. Was a nice trade though. And bled in the broken new low into the close. So many, many times what happens is when these break the first day, if they don't go to the full and target the first day, you can watch them to trade this next day. And so that's what I did today. And today did something beautiful too, which is what? Held, look, this area of resistance right in here. Here's the open of today. This is 9.30. Open and dropped. Had a beautiful, beautiful entry in this today. If you were aggressive and did it first thing, but the volume was low. I waited until the volume came in and then did it. I had a five cents thought and it's continuing to break. So the low right now is 671. I get out of this already. And again, bigger target on this is $6. It's certainly going to get there. The next target actually is 664. That's a number. 664 is a number 650 and $6. So this is one you could actually be in for a swing trade. You know, that's the day they got, which was yesterday, beautiful trade. Somebody said something to me once about going back and I do video reviews. I do have a few tapes, actually two. I have two tapes of the room online on YouTube. If you ever want to listen to me, I had taped the room. They're four hours long, but I just taped the room twice for people to listen to.  That doesn't make sense. I have to focus on what I'm doing and I'm running the trading room. But someone said something about me doing trade reviews after the fact. Well, I run a live trading room. There's a substantial amount of people that are in my live trading room. So when I do trade reviews, it's some trades that I did because I'm putting them on YouTube. So since I run a live trading room where people that are in there, they would know if I put a video out there to say that I did a trade that I didn't do because there was people in the room with me and quite a few. So seriously, you know, when I do a trade review and say I did a trade, I did a trade. And I did it in the room. And the people in the room know that I did it because they either did it with me or watched me do it. Now, there's times that I will review different charts that I did not do, but I never say I did a trade in a video that I didn't do. So I did this today and yesterday. It was a nice trade. So if anyone has any questions or would like to learn more about trading and trading gaps and how to find gaps like MDR. MDR rated very well. In fact, what was MDR? I don't know. I'll have to go back there and look at the sheet. They're a little disorganized here today. It was over 21 though. I know that. I don't remember. It's either 20, I want to say 22 or 23. I don't remember. This got a lot of points. This was a great get. So it's going to get to the $6. It's just not going to get there in a day because obviously this is a little bit of a snail, but that's okay. Anyways, if you'd like more information, please feel free to email me at Melissa at thestockswish.com. And reach out to me if you'd like to learn how to trade gaps. I teach a class on how to trade gaps. I teach my rating system in the class. It's a two-day course. The next class is coming up the August 17th and 18th. And if you'd like more information, email me at Melissa at thestockswish.com. Thanks everyone. Have a great day.