So welcome to the Dual Anonymous Pure Review Town Hall for Planetary and Exoplanets Research Programs. My name is Delia Santiago-Madarice, the lead for the Dual Anonymous Pure Review process under the Planetary Science Division out of NASA headquarters. And we'll have program officers from the programs that will be doing Dual Anonymous Pure Review and they'll be introducing themselves shortly. First of all, please submit any questions you have during this presentation via the link here. It is also posted in the chat. You can upvote questions already posted as well. In addition to that, you can ask questions in the chat of the presentation. Due to the large number of people, we're going to field questions in that manner and you might get some of the answers back to you from the other program officers in the chat but we'll also be answering some of the questions at the end of this town hall and we will also have, we already have some pre-prepared answers for the questions that were posted earlier. So this slide is going to be a quick overview of Dual Anonymous Pure Review, which we fondly call Dapper. And I'll be going more into depth into all of these points later on. But overall, we have this great website on Dual Anonymous Pure Review. Also, note that these slides will be posted there later on so you can access all this information and a lot more at this website. The goal of Dual Anonymous Review is to reduce implicit or unconscious bias in the evaluation of the intrinsic scientific merit of proposals. And the way we do this is that in addition to the proposers being unaware of the review panel member identities, the reviewers themselves are not told the identities of the proposers until after the evaluation of intrinsic and scientific merit. So the process for proposers, and again I'll go into this more in depth later, is that proposals are written to exclude any personally or organizationally identifying information in the scientific portion of the proposal. And they must upload a separate expertise and resources not anonymized document, which contains all of the personally or organizationally identifying information. For the reviewers, this means that they evaluate the intrinsic scientific merit of anonymized proposals without knowing the proposing team who they are. And after the scientific evaluation is finalized for proposals, the panelists will review the expertise and resources anonymized documents to assess whether the team has the qualifications and capabilities necessary to do the proposed work. So we'll be going over what programs are going to be converting to Dapper dual anonymous peer review, what exactly is dual anonymous peer review, and why is it important, how to make your proposal compliant, and how it's going to be evaluated. So which programs are converting? So first of all, we've already done a number under roses 20. These were all inspired by the work of the Hubble Space Telescope Reviews by the Space Telescope Science Institute starting in 2018. But under roses 20, SMD did a pilot with one program from each of the visions. So the astrophysics data analysis program, earth science US participating investigator program, habitable worlds from planetary science, and the heliophysics guest investigator program. So habitable worlds is still under review, but the other three have all been completed. And I'll be showing some information on that. In addition to that, the astrophysics guest observer, guest investigator programs are permanently converting to dual anonymous peer review. And a note on the programs I just mentioned that were piloted under roses 20, all of those will be continuing under Dapper under roses 21, habitable worlds is not being called under roses 21, but the other ones are going to be continuing under Dapper. For 2021, of interest are the columns on the right of planetary science and cross divisional. We have our DAPS here, and the exoplanet research program, which is largely managed out of planetary science. So to go more in depth about the planetary science division programs, first of all, under roses 20, we have habitable worlds, which is managed by Lindsay Hayes and Becky McCulley-Rench. And I believe Lindsay is here, if you want to say hello. She might not be able to say hello right now, but Lindsay is here, and they've been working on this review. And the proposals have been submitted, the review process is underway. Just a few proposals were returned without review because of egregious rule violations. But most other non-compliances were allowed through PI's were given a warning letter, but the proposals are still being reviewed, and we expected this under the first year. So I'll go more into depth about some of those violations, non-compliances later on, once you have a more clear idea of what one should be doing, but really quickly, the most common issues are failure to follow the reference numbering scheme that you're supposed to follow, and then just slips, basically.  So just small little slips like that. Under roses 21, we have all of the data analysis programs, as well as the cross-divisional exoplanet research program. So here are the list of program officers, and we can start from the left if folks want to introduce themselves, starting with Cassini Dap. Hi, this is Henrik Schrupp on the Cassini Dap, a CDAP, and also on New Frontiers Dap. Thank you. Hello, I'm Doris Thao, and I'm the program scientist for DiscoveryDap. Hi, I'm Shisha Noida. I'm the lead for the Lunar Data Analysis Program. Glad to see so many people online today. And I don't know if Mitch Schulte or Adrian Brown are here, but they lead the Mars Data Analysis Program together, and I believe our XRP leads here. Hi, everybody. I'm Megan. I'm the lead from PSD for XRP. It's good to see everybody. Thank you. So now we'll talk about some of the motivation. In summary, this is also called Double Blind. We tend to say dual anonymous now. Dual anonymous review is to address issues related to implicit bias. So a classic tale is that in the 1970s, the top five orchestras in the U.S. had fewer than 5% women. When blind auditions were instated, that went into the 30s, 30%. And another example is with behavioral ecology. They switched to double blind review, which resulted in a significant increase in the number of female first authored publications. So first, it's difficult to completely interrupt implicit bias through training. It's also referred to as unconscious bias because we aren't always aware of our biases. So structural changes are needed, which is why we're doing this. We want to thank the Hubble Space Telescope team for pioneering dual anonymous peer review. And here's some data from them. So this was when they, before they switched to dual anonymous peer review, you can see the proposal acceptance rates between male and female PIs. And so this is the percent of male PIs that were accepted and the percent of female PIs that were accepted for a given cycle. And you can see that there is a difference. And theoretically, they should be, there's no reason they shouldn't be accepted at the same rate. So when DAPR or their dual anonymous peer review was introduced, this difference decreased. It didn't completely go away. And you can see for cycle 26 that females were slightly overrepresented in the next two years. They were slightly underrepresented compared to the percent acceptance rate of men of males. But this difference decreased. And that's the overall point. And one point is some preliminary data we have from the astrophysics data analysis program. So this is a slightly different chart. This is the percent proposals with inferred female PIs. And the total pool is the leftmost group. And then on the right are the top two and top three proposals from every group. And blue is non-DAPR. And you would, theoretically, there's no reason that the, you know, between year to year that these wouldn't just go straight across. But you can see that the percent proposals with female inferred female PIs were underrepresented in the top two and three proposals in each panel before DAPR. But with DAPR it reached more parity with the percent that submitted. So gender is just the tip of the iceberg. Another thing we're always looking at is success rate by institution type. These are the success rates for the ROSES programs in the SMD pilot under ROSES 20. And you can see that government contractors and NASA centers are selected, have their proposals selected at a higher rate than non-R1 or R2 academic institutions or minority survey institutions. And there are a lot of factors at play. We don't claim that bias is everything, but the idea is to play, to level the playing field for everyone. It's not a silver bowl, but it doesn't solve everything, but it's one tool to help level the playing field with everyone. So what exactly is it? Again, the proposals are unaware of the identity of the members on the review panel, but the reviewers do not know who the identities of the proposing came during the scientific evaluation of the proposals, of the proposal. And the intent is to eliminate the team as a topic of discussion, not to make it impossible to guess who might be on that team. We know that people have very specialized fields and there are, you know, you might feel like you can narrow it down, but the idea is not that you can't guess at all, but the idea is we're not talking about this. And we want to create a change in tenor of the discussion, away from the individuals on the proposing team and toward the proposed science. So it's not completely blind process. We have proposers submit the anonymized proposal, but also a not anonymized expertise and resources document. And the first step is the mayor of the proposal is assessed and separately the qualifications of the team are assessed. So the qualifications track record and access to unique facilities is indeed part of the evaluation. It's just separated out. Feedback from Hubble panelists is that proposal discussions were characterized as more collegial and efficient. Focus was squarely on the science rather than the scientists. There was a noticeable shift in the depth of discussions as well. It was clear that the reviewers had read the proposals very diligently and that without the distraction of names or institutions, there was no recourse but to focus on the proposed science. Discussions at both the panel level and the attack, that's the time allocation committee level, focused predominantly on whether the science was novel, impactful and feasible with HST and not on whether the proposers had the expertise to carry out the proposals. Several TAC members noted that they felt the discussions at both the panel and the TAC level seemed more collegial and less emotionally charged than previous TACs perhaps because either positive negative feelings about the people involved with the proposal were largely removed. So how do you make your proposal compliant? We're giving you lots of information but first of all step one is not anonymized. It's submitted per normal requirements and step two is anonymized per the guidance that follows. So first the inspires program page. Each element will have a document entitled guidelines for anonymous proposals and it has a lot of detailed information about submitting the anonymous proposals and a quick start tutorial and FAQs are at the website listed here which is the same system. So first of all you need to exclude all names and affiliations of the proposing team including anything in figures or references to personal websites. Do not claim ownership of past work. So instead of saying my previously funded work or our analysis shown in whatever you would say previous work shows or previous analyses in this reference you would just have it be you would have no ownership of it and references are cited in the passive third term. So you can see the yellow this prior analysis brackets one indicates that and that just kind of draws attention away from you know who's frequently cited. You can still say we when you talk about you know the work you're going to do you can say we propose to do something or we will do something like that you just can't say it in terms of ownership of past things and you'll include the separate not anonymized expertise in research document. So how do you publish how do you reference unpublished work for proprietary results? You might need to do this occasionally or maybe non-public software things that haven't been presented in public but they're not citable that could imply the investigators but we want you to kind of again refer to it without attribution or ownership. So you can use language such as obtained in private communication or from private consultation which people can do anyway but you will do that in reference to yourself as well. And just again recall the goal of dual anonymous is to shift the tenor of the discussion not to make it impossible to guess the team members. Regarding institutional access to unique resources another common issue is that you know you might have unique access to certain facilities to do the work and you can still talk about this in the STM the scientific and technical management section of the proposal you just should do in a way that does not identify people. So you can say the team has access to this ultra high resolution IRMS which will enable the required measurements. And in this kind of situation we would recommend you can support that in your expertise and resources document. And this is an example of anonymization and I'm actually going to just go to the next slide where we highlight so the yellow goes to yellow so instead of saying in Rogers 2014 we concluded say prior work concluded if our model from this is correct you say if the model from this is correct and we will compare with our first epoch it's a first epoch was obtained in 2007. Those are examples of how to translate that text. And a really big thing is that many people obviously resubmit proposals. This is where it gets a little trickier. We really ask you to really check things when you're resubmitting a proposal or reusing text from something else. Just double check that you're not claiming ownership of password. Do not include the names of personnel associated with the proposal or their organizational affiliations. Write the references in the format as indicated use third person neutral wording. And then again you can go to this website for information. But how is the capability of the team to execute the investigation account? I know there's kind of a concern that somebody is going to write a proposal but they don't actually know how to do that and it's NASA can be funding people that are not qualified but we're going to make sure that they are evaluated for that. So we have the expertise and resources document so it's a second document you upload and it is distributed to panelists for a subset of proposals. Typically the top third it depends on how the program decides to do that. But it's more than we would presumably slightly more than we would potentially select. And it will have the following elements. So you have the list of the team members and their organization and their roles. A brief description of their expertise and what they're bringing to that. A discussion of the contributions of each team member will make to the specific investigation. A discussion of the specific resources required to perform the investigation. So access to laboratory equipment, specific samples or sites, etc. A summary of work effort to include the to include an anonymized table of work effort. So you'll have you'll have a anonymized and not anonymized portion for that and that's outlined more in the guidelines document. Biographical sketches as required. Statements of current and pending support as required. This is all in the expertise and resources document. And any letters of resource support. So if there's some facility that you need to have access to or letters to the PDS things like that that all goes in the expertise and resources document. And how will your proposal be reviewed? So we have the two parts. We have the science review and the expertise assessment. The anonymized scientific review will take place and everything is finalized. The evaluations are written. They're signed off. They are done before the expertise and resource document reveal occurs. And then once that occurs it'll occur for that subset of proposals and panelists will assess the team and resource capability to execute the proposed investigation. And this is how what the panelists are instructed to do. They're instructed to consider proposals solely based on the merit of what's been proposed for the first part. And they're instructed to not spend any time trying to identify the PRA of the team. And even if they have an idea they're supposed to discuss the science and not the people. And there are NASA appointed levelers present to ensure that the integrity of this process that are going to be present in each panel. And they're also instructed to make sure that their language is important discussing the proposal. So they should use you know neutral pronouns such as what they propose the team has evaluated data etc. And the NASA appointed levelers which in some cases are program officers but it's always somebody that's you know job is to be listening to the conversation. They're going to be there to support all of this. They are there to ensure that everyone's focusing on scientific merit. They're not necessarily listening for issues related to science just more that the they're focused on the discussion itself. And if the discussion ever becomes inappropriate and trying to guess anything about the proposers the leveler can refocus the discussion and they have the authority to just stop the discussion of the proposal. Then the discussion of the resources of the expertise and resources document occurs. So again the scientific evaluation is completed. And then these documents are distributed to the panelists for the subset and they assess the team capability using the scale. So proposers are either uniquely qualified qualified or not qualified to do the proposed work. We set the expectation that the vast majority of proposals are going to be qualified. And there actually is a requirement that if the if the reviewers find that the team is uniquely qualified or not qualified they actually have to write something about it and justify that reason because we do expect the vast majority of teams to just be qualified. And uniquely qualified does have a very both of them have a very high bar. So for example to be uniquely qualified you have to be exceptionally capable to do the proposed work and also have singular access to resources to complete the work. And being not qualified means you have severe deficiencies in the necessary expertise and resources to execute execute the proposed investigation. And when this is distributed we instruct panelists to base their assessment based on what's written in the text and not say things such as oh so and so is clearly qualified. So we understand that DAPR is a major shift in evaluating proposals and preparing them and there might be some slips in writing these proposals. We do reserve the right to return without review if there are particularly egregious violations and non-compliance in terms of identifying the proposing team. So we also know that some work is so specialized that even if you're you're trying to anonymize the proposal that identify identification of the PI and the team members can be discernible. But as long as you follow these guidelines we're not going to return your proposal without review. That's not our goal at all. So a quick review of DAPR experiences to date. We've only had three egregious violations on the anonymization guidelines which resulted in proposals being returned without review and that's out of hundreds. Here are some common pitfalls that we see maybe 10 to 15 percent of the proposals claiming ownership of password. So our previous analysis PI has established record including metadata. This is kind of a tricky one. That might reveal the name of the PI. So bookmarks for example might have some PDF bookmarks that might have the name of the PI. When people recycle proposals prepare piratidual anonymous peer review and they kind of skip a sentence or something and they don't anonymize everything. Providing the names of the investigators and the contents page we've had I would say we've had some like wonderfully beautifully anonymized proposals but then they just accidentally left in a title page or contents that had their name like right at the front. So trying to that's something to be aware of and double check. Providing the origin of travel for professional conferences that's something to leave out in the anonymized portion of your proposal. You can just say that you're traveling. You don't need to say that in the anonymized portion you don't need to stay your starting location for example. Mentioning the institution name and the budget narrative. So this is something to double check with your budget people. I know that a lot of times the people who prepare the budgets are working you know they might not always be aware of the latest guidelines here and their default template has the institution so that's something to check. And including PI or CoI name on budget tables it's another thing where we'll see like a perfectly anonymized proposal and then the budget table just like names the PI because again it was prepared you know a different person prepared that. And then from panelists the reviewer surveys thus far have indicated that the discussions were more focused on science content than on non-dapper reviews and there was overwhelming support to continue Dapper and Future reviews and that's those are results from NASA SMD so far. So I'm going to answer some submitted questions. These were submitted as of midday yesterday. Why did the planetary science division not go with open peer review? So the goal of Dapper is to reduce implicit bias which revolves removing the identity of the proposer from consideration when evaluating the intrinsic of a proposal. So open peer review might address other concerns for goals. I know it's been credited with more you know collegial interactions and such but it's not directly designed to address issues of implicit bias. And we're basing this off of HST's process which suggested that removing the identification of the proposer that inferred female PI's were selected at a rate more on par with their submission rates than in previous years. So this process has been shown to address some issues of implicit bias. How does the dual anonymous process work with continuation proposals? So this is some language from the guidebook for proposers which you know the kind of key point as at the end it says therefore NASA will consider all proposals received in response to a notice of opportunity for funding notice of funding opportunity as new proposals and will review them impartially. So proposals submitted to under a Dapper program are like other programs reviewed as new proposals and reviews as such. So they should not refer to previous funded work in the anonymized portion of proposal. You can refer to that work in your E&R document. How do we refer to relevant previous work without outing ourselves? So I think I just addressed most of this in the presentation but you but to just emphasize you are free to refer to the work itself you just cannot claim ownership of it and you should discuss it without attribution. For unpublished work we recommend saying previous work instead of our previous work or obtained in private communications. And then there are a couple of questions. So how do we describe the use of analytical labs to be sure the reviewers don't find out where the work would be done? And the second question I will confess I did not completely I was not completely sure of the intent of it but it was that dual anonymous makes sense when everyone was proposing the same use use the same equipment. How can it work in an open proposal call? If if I'm not addressing that here please feel free to use the chat because I just wasn't quite sure what the nugget was of that but I believe the the questions are about how do you how do you talk about specialized equipment? And you can talk about these aspects you just can't discuss them with attribution towards your group. You can refer to having access to specialized equipment facilities and if that's required the panel is going to flag that and then when it comes time to go to the for the ENR review the expertise and research review then they can check for that. So you know the panel might say wow this is really you know they're going to have access to this like not many people have access to that and that's not going to be a weakness or a strength they're just going to flag it and then in the ENR it'll be apparent if the group actually does have access and so they're they'll perform that full analysis using the three-point scale and again just remember that the goal is not to make it impossible to guess the identities of the investigators but to shift the focus away from the individuals and toward the science. And then there were kind of a group of questions that kind of addressed the same thing but the idea was that a scientist track record is an excellent indicator of future research but shouldn't this be part of the evaluation. So the proposers should be able to make their case through the description of their proposed work that they have the necessary skills to achieve research success and the track records are addressed in the ENR document and they're going to be voted on and we're not trying to make it impossible to guess who it is we're trying to shift the focus of the conversation and this is a follow-up is how do we evaluate proposals without knowing their track history and there were several questions all related to this and there were in the past there were programs that funded purely on track record but it's not something we're currently doing and we evaluate the proposals new every time and I would also like to note that to date no highly rated proposals evaluated under DAPR have been submitted by a team where they were not deemed at minimum qualified and this is out of over 500 proposals submitted to NASA absentee how do we handle proprietary restricted access resources the example given in the guideline was not convincing so we just asked to use the guideline given we'll flag it and we'll verify it later and I keep saying the same thing you know we're just trying to ship the conversation of toward science not people and if you follow the DAPR guidelines we're not going to return it without review and if if we are seeking to fund or use a specific observatory where the set of PIs are known and sorry I cannot completely see this can we no longer reference that observatory you can you can still reference it in the STM you just can't write it in a way that you're identifying the team ever so you can say the team has access to telescope time on the WNK observatory which will enable follow-up of the galaxies in the sample and in this situation we encourage you to follow up and say how you have access in the expertise and resources document and yes that's amazing so some final remarks here oh actually this is a good place to bring up any other questions I want to list all the program officers here you can ask any of them for questions about this and if it's very DAPR specific you can always reach out to me but I encourage you to reach out to your program lead as well this is Henry Throop I'm going to read a couple of the questions Deli that have come up in the on the web page they've been voted up the first one here is for technology development proposals often build on previous work and with a small field proposal teams are identifiable how does that work so are any of these I did not think any of these were technology development proposed programs I mean I know they're data analysis programs we so that is an interesting point we don't have any technology development explicitly in any of these programs so I can I definitely see that point but we're we haven't also extended DAPR to those programs at this point in time so that will be something you know we'll need to decide if that's the right match for that thank you next question is can you address how the institutional and management sections of the proposals should be adapted to the two DAPR that was this last slide that I thought was missing and I think I just clicked over it for it went really quickly because I actually remember spending time on this question and then I was all of a sudden at the final remarks page okay I have a slide for that so this is a couple of things a lot of this is from the guidelines but I'm just breaking it out so the STM portion itself is written anonymized the summary of work effort is included in an on my fashion you can say Hawaii one quite to in the main document and then in an anonymous non-anonymized fashion in the E&R document so you'll have the table twice once anonymized another time non-anonymized and then as usual roses proposals include a redacted budget so so the redacted budget that is provided to the panel doesn't have salary fringe or over doesn't have salary fringe overhead but it doesn't have you know names of people etc but it should you should have a very generative that discusses financial support for the CoIs you just don't you can have it anonymized so there's it's the same thing where you've always had a redacted budget it's it's just anonymized and then in the unredacted budget you can use names and all of that and the facilities and equipment section is not included in the main proposal document it's a shortened version of this information including letters of resource support are in the expertise and resources not anonymized document so I think that addresses most of the components of that question thank you next question is similar to the question addressed about labs and access how do we anonymize a proposal that uses a code that is not open access and I can imagine for instance a proprietary code might be linked to one institution or might even have the institution's name in that code how do we address that you can um you can say that you have access to it I mean you can consider while an institution might have access like um you somebody can share code right like you could have a collaborator who share who shares that code who's not that institution for example so the point is that you just you can refer to it you just can't say you can't say that you can just say that you have access to it it's the same basically the same as facilities you just say that you have access to it um and and again I mean yes it does that possibly give a clue but that's you know it's within the rules and um you know the point is you're saying you don't say that you created it but you can say that you have access to it um on the review does Dapper in any way in any way affect how conflicts of interest are identified and managed um so to some extent um I need to double check so we we are having to the onus is more on the program officers to manage these um but we do we have changed some of our internal processes um and I think that's all I can say about that moment um will the proposal be sanctioned in any way or penalized in any way if proposal accidentally reveals too much information about the key members and works they have previously done um so the first year we're trying to be more forgiving we understand that slips may occur and we've been able to draw a clear line so far of agree just versus not it's been pretty straightforward um where you know it it was clear that somebody didn't do it at all anonymous a proposal at all versus like oh they had like two slips or something like that um so if we in general find that there's been a good faith effort um we you you might get a warning this year um or but you could still it could still be reviewed um but if it's if it's free just it will be pulled but that's has not happened extensively this year we had a couple questions about formatting references uh has NASA provided any templates for formatting references in the correct way with numbers rather than the pi's names no um so there's in the guidelines document basically it's numbered brackets and then you number it so it'll just be like a numbered list and you refer to it in the numbers in the proposal so and presumably there are standard latex templates out there uh or other templates that one could use to format the references that way um question about the review panel for the flow of the review panel does the evaluation of expertise extend the time that the panel will meet for does it take uh longer to discuss this than it does normally um it's pretty brief actually um it's um I would it's it's not a substantial it adds minimal time to the panel I would say like maybe yeah this is a very rough you know it's going to depend on each group but maybe an hour is that and um but there are other parts of it that are you know you're actually spending less time discussing some of those things during the actual discussion so it kind of evens out in some ways we have a question about external reviewers uh will external reviewers uh also see only the science portion of the proposal over those where they see the uh the non-anonymized section as well they will just see the anonymized version um let's see for facilities uh what if you have uh sort of private 100 ownership of the facility and identifying that facility would definitely identify you I mean it would so in general I mean you could it's you would still you would just say that you have access to it I mean you don't need to say I mean you would just say we have access to it um you don't need to say we have access to this and only the people that own the lab have access to it so you would just refer to having access to it and and if somebody happens to know that it's you know something that only those people have access to then that's what it is um but the point is to just say you have access to it and one more here uh may have PI volunteer to be identified as author of the proposal I don't understand what so I think that the question refers to whether someone may opt out of the dual anonymous system and identify themselves by name in the proposal I mean from what from what I'm hearing that would be outside the guidelines so I'm not quite sure if I'm understanding the question I think that's what the question is yes okay okay um and just a note too on the um the very specialized equipment issue and I know this comes up a lot um is that you know we don't know there could be this could be a lab of five people and there could be postdocs or people who graduated from them and we don't know who is who and that's that is still you know anonymized and um anecdotal information is that um a lot of times people guess and they're wrong and they think they know who it is and they don't always so um again and the point isn't to try and guess and see if you're right or wrong and the point is to to not be talking about it as much uh let me pass it over to Doris now okay she has questions press on uh yes most of the questions that were asked are already have already been responded to but there's a question here about uh can we have any anonymized description of expertise in science uh uh in stm e.g the team has decades of experience in counting craters along with recently developed experience in numerical model that's something that just um by default belongs in the expertise and research document um and it's um yeah it just that's where it belongs it's not that it's um that's where it goes like by structure that it would go in the enr section there's another question about the conflict of interest which you've addressed and also I want to let you guys know that then Evans also uh addressed that question so if the answers are not uh uh full for you uh feel free to uh pose it again there's a question about uh the program officers when we are preparing selection document will we be doing this blind as well uh uh with anonymized proposals so we know the identities and that helps us manage the process uh there was a lot of conversation and I think that already has been addressed about what do we do if we have access to facilities or past past papers publications so I think unless I see something new that's about it for for the questions on the chat let me have one question here okay I have one question here is there any change to the page limits and what can you say about the page limits of the expertise document um there is no page limit to the main part of the document um you I mean you kind of end up having a little bonus space because you're excluding some of the stuff that is normally in that portion um and the enr document does I haven't looked at the call in a long time because we're like way off cycle but it does it should not have any page limits um and also Dan Evans is uh who I believe joined us during this he is the the lead for SMD doing the dual anonymous peer review and so thanks to Dan for pioneering this and if you had anything to say you're um it would be you're welcome to chime in and say hello to everyone no you're handling it beautifully already deli and that needs me to chime in and there is no no limit to the enr document but typically speaking um a few pages um maybe up to five pages is about right including you know biosketches and stuff like that but for the written material um you know lists of expertise and so forth that's typically about a page or two we also had a question about the slides for this town hall um would those be posted and available yes they will be on the dual anonymous page um and actually if somebody is able to post post that page in the comments that would be great so that people know where to find that it's been referenced numerous times but obviously if you don't have the slides you can't access the page so do it all right deli if you have time I have a couple more questions here okay great all right so a question about the citations if the citations are fully listed at the end then what is the point of changing the in-text to number so so the idea is to just draw attention away from that um and it's kind of a compromise right because because if we say if we had put the references you know if the references were in the redacted portion people are gonna be like oh they're citing these crazy references that are completely invalid so we might have a way for people to properly cite things but it's not upfront in center um and we you know we do rely on the honor system of their viewers to not make it you know a like a hunt to figure out you know who cited a lot but um another question uh asking if there are any statistics on the proportion of proposals with excellent to very good science ratings but very low qualifications rating so we've only done um the reveal for the top rated proposal so like if if something was rated fair or poor we weren't doing the reveal um and then of those that um were made it to the expertise and resource reveal um none in SMD have been deemed not qualified and so it's just those three categories not qualified qualified and I don't know how many have been deemed uniquely qualified um but my understanding is the vast majority have been just qualified all right uh I have one more question uh for oh for a follow-on proposal do you still need a section on accomplishments from the previous award how can that be anonymized or should this be left out so um I think that was hinted at in the continuation question um so that should be in the expertise and resources and whether or not it's required at all you can you can talk to the specific program lead for that so I would talk to the specific that person to see if that needs to be included at all but if you are including it it would be in the expertise and resource section and it would you see it's considered part of your expertise and there's a question about um let's see would the review process differentiate between the new proposals or and the follow-up proposals yeah so you don't refer to your previous work it's it should just be a standalone so you're not making any reference to previously funded work including you know your your predecessor proposal I think that's answering that right okay one more I have I think this is the last one I have here and it's quoting the questions on the Q&A page as an important sub-questions on referring to previous work how do we talk about preliminary work for this proposal could we say we have done the analysis shown in figure one since this is still anonymous but demonstrates its capabilities which I haven't heard discussed basically yeah if it's if it's not published anywhere then yeah I mean if you obviously say if you published it in LPSC recently then that's kind of your your owning you're making that connection but if it hasn't been published anywhere and that can be attributed to you I see no reason why you wouldn't be able to refer to to your preliminary results thank you Delia I think that's all I see in the chat if I missed your questions please type it again I do any of the program officers have anything they want to chime in and they might just have to say a lot of people are putting thank you notes I hope you'll get a chance to read them after this well thank you everyone for coming here and there are a lot of resources available and feel free to reach out about any questions I know that this is a big change but I we're excited to work with everybody on this and and I'll say thanks to my colleagues for being here and supporting this event and thanks to the community so have a great day everyone and I don't know Shoshana if we stopped recording but


Code and Closure is organized into namespaces. Each namespace is known by a symbol name, and at any one time during evaluation, one namespace is the current namespace. Again, this current namespace is the one used by the evaluator when resolving symbols. When the closure runtime starts, the current namespace is a namespace called user, which conveniently contains everything from the standard library's closure.core namespace. So right at program start, a def form will modify the namespace called user, but we can also use the functions and macros of closure.core without having to fully qualify those symbols. The closure.core function inns as an in namespace changes the current namespace. Here, we quote the symbol foo to pass it to the inns function. If the namespace foo doesn't already exist, this call to inns will create it, and either way foo will henceforth be the current namespace. Be clear that this new namespace is completely blank and so does not contain the elements of closure.core like the user namespace does.  There's a better solution once we understand namespaces a little better. A namespace can actually have four different kinds of mappings. Symbols to vars interned, symbol to vars referred, symbols to classes, and symbols to namespaces. Interned mappings are the normal kind of mappings, the ones we create with the def special form. Here, for example, we assign foo to a var map to the symbol bar, and we use the def and macro to assign a function to a var map to the symbol zeros. Referred mappings, though, are created with the refer function. This function takes a symbol naming another namespace, and every interned mapping of that namespace is added as a referred mapping in the current namespace. Here, for each interned mapping in the namespace foo, this call to refer adds a referred mapping in the current namespace. Be clear, refer does not copy referred mappings of the other namespace, only its interned mappings. One way to think about this is that interned mappings are the true members of a namespace, whereas referred mappings are just borrowed from other namespaces for convenience. By using refer, we can conveniently use vars from other namespaces without having to fully qualify their names. Here, before referring the foo namespace, we have to qualify var of the foo namespace, but then afterwards we do not have to qualify var. It is common practice to refer closure.core into the current namespace so that we don't have to write closure.core every time we want to use one of its functions or macros. If a namespace we refer has a mapping of the same name as the current namespace, the refer function will throw an exception. We can get around this by resolving any conflicts in a map after the keyword rename. So this call refers the interned mappings of foo into the current namespace, but it creates a referred mapping called bob for foo slash alice and a referred mapping called david for foo slash carol. And this way we avoid a name collision if the current namespace already has things called alice and carol. Mappings to Java classes are created with the import macro. Here, this call to import maps the Java class java.util.date to a symbol of the same name in the current namespace. And understand that because import is a macro, the symbol argument gets passed un-evaluated so we don't quote it. Mappings to other namespaces are created with the alias function. When the evaluator resolves fully qualified symbols, it looks for the specified namespace in a global table of namespaces, but if not found there, it will also look for a namespace alias in the current namespace. So after we create this alias super, we can use it in place of the full name supercalifragilisticxpialidocious. When we start the closure runtime, we specify one source file to run. In our code, we can then load and run other source files with the load function of closure.core. The load function reads and evaluates the code in a file specified by a path relative to the Java class path. The file is expected to have the extension .clj, but that part is left implicit in the specified path. So this here loads the file bar slash baz.clj relative to the Java class path. By the way, from here on out, we'll always assume that closure.core is referred into the current namespace. A closure lib is a source file located at a path, relative to the Java class path, that corresponds to a symbolic name. For example, a source file stored at richard slash millhouse slash nixon.clj has the lib name richard dot millhouse dot nixon. By convention, a lib should create a namespace of the same name as a lib itself. A very large lib might create additional namespaces, but this is not a very common thing to do. If a lib source file becomes too large, you might wish to move some of its code into separate files. Common practice is to place these other files in the same directory, and then use the load function to run them from the main lib file. The require function loads a lib by its symbolic name. This loads the file closure slash java slash io.clj relative to the Java class path. When running a source file, the code in that file might change the current namespace, but unlike the load function, require ensures that the current namespace at the end of the call is set to the same namespace as at the start of the call. The require function takes several different options for convenience. We won't go into these options, but here's one example. This example requires the lib closure dot java dot io, but the imported namespace is known by the symbol bar, not closure dot java dot io. The use function is like require, but also refers the required lib's namespace into the current namespace. So this here loads closure dot java slash io.clj relative to the java class path, and then it refers the closure dot java dot io namespace. Rather than use nns, require, use, and import directly, it is most common to use the ns macro, which bundles this functionality together into one macro. A typical lib will start with a single use of ns. The macro has many options, which we won't go into, but here's a simple example. Here, the ns macro switches the current namespace to foo.bar, into which it requires closure dot contrib dot sequel, uses closure dot test, and imports java dot util dot date and java dot util dot timer. Note that because ns is a macro, we don't quote the symbols which specify the lib and class names. Now, let's take a look at the ns macro. The ns macro switches the current namespace to foo.bar, into which it requires closure dot contrib dot contrib dot. The ns macro switches the current namespace to foo.bar, into which it requires closure dot contrib dot.


Can you smell how pristine the rocks business shirt is? Hey guys, this is my review for skyscraper and first off I want to say to all of my friends who worked on this film It must have been an amazing experience not only to work with the rock would work on such a huge big-budget film in Vancouver and it's just helping build Vancouver more and more as a big place for making movies So good on you guys to everyone who got to work on that that must have been a great time But now onto the movie It dumb it's not super dumb It's not Jurassic World 2 dumb, but it's a little dumb The film is definitely a giant homage to die-hard and the towering inferno with a few intricacies of its own But enough smoking check-offs guns that you're going to have to find a storage cabinet to hold all the guns in this The film is about Dwayne the Rock Johnson's character who's a securities consultant for this big-ass giant mythical futuristic looking building called a pearl and He and his family are the first residents inside this building but on the day that they're about to think about opening it up for the residential areas of course terrorist people come in and take over the building, but also at the same time start burning the building and The terrorists are not that great they're Blandi McBlandi terrorists and it's unfortunate because Clearly they want the main villain to be a Hans Gruberland character There's a lot of similarities in his actions and some of his dialogue moments, but the dude is terrible I don't know why they thought this guy would be a great villain because he looks menacing ish, I guess but in all honesty He's one of the weakest parts of the movie the best part of the movie for sure is Dwayne the Rock Johnson he brings a bit of a humanity to this character in terms of his Motivation his actions with his family as well as certain moments through the climax, but the Problem is while they're trying to make this guy seem to be you know a family man and whatnot Yeah, sure.  It was right at this point. I was like, okay I know what kind of movie I'm going to get that Campbell is not Sidelined she does have a few action moments here and there which does help vary up the film But she is basically the wife, but who can do a move here and there Admittedly the film is an entertaining piece. There are some great sec pieces There's one thing that I actually thought that was really interesting was his adherence to using a gun and how that is a Constant thing throughout the movie the rock himself. It does really push this film He is definitely your driving point as he does have some good funny lines in this film But his motivation is definitely true. You see that this is probably his best performance of a character since Be cool He does a great job of really putting his character on the line through the action sequences through the emotional moments And he definitely will be the draw for you guys But in terms of just rewatch ability Because the fact that it is diehard and the towering furnace put together it has Just as much resemblance to this movies is just saying the titles really the motivations for the terrorists is Very very bland and the terrorists literally you only know two of them Instead of the whole group kind of that whole diehard-esque theme was while John McClain was one man He was trying to figure out who these guys were. Yeah There's the whole family aspect that he's trying to do but in diehard dr. McClain was trying to find Holly Gennaro his wife he was trying to save her all the while taking down the terrorists for the terrorists focus and Skyscraper his family is his focus which right at least so I guess but in terms of how these villains function They're idiots.  It doesn't seem to be marking Dwayne the Rock Johnson shirt at all like near the end of the movie He's covered in soot yet. His shirt is still that color.  Some people may enjoy it mind you this theater was completely empty and this was a Tuesday night I think there was eight people in the theater beside me and my boss and a friend But in terms of the rocks movie collection, this definitely channels a few of his other films of the rundown walking tall Definitely hobs in terms of superpowers, but I guess this is a fun time if you're into something like this It's not as stupid as San Andreas was But it's not as fun as rampage was rampage was dumb fun, but it was fun This is just dumb amusement. I guess you would say but that's just me I'm not entirely a big fan of this film, but I know some people who will be so in the end I'm gonna give Dwayne the Rock Johnson skyscraper a three out of stuff. Yeah, it's got some pretty good moments And there's a lot of parts where he's like Anyways guys, that's all for me I hope you enjoyed this review if you did leave a like and if you're interested in more maybe subscribe Otherwise, I'll see you guys next time


How do we create speed? There's two key parts, and we're going to talk about it in this video, so check it out. Everybody, it's Eric Johnson in today's video. We are part three of our rotational shot put one-on-one series, and one of the things we've been looking at is kind of introducing you to the throw, understanding you have to have a structure. The first video, we started to look at the entry side. Now we're going to look at that thing everybody wants. You want to have that speed. How do you create speed into the middle? And one of the key things is if you messed up that entry, you're going to be toast.  So you have to come into that throw on balance. So we're comparing our throwers one through four. Remember, thrower number one, 10-year-old first day throwing. So the movement is very good for the first day, and as this athlete continues to go on and apply the six pillars, he should continue to have more and more success. When we look at again, and we compare, and we kind of review, and we're staying nice and long, and we're getting that foot, now right here is the drop-in. So let's just focus on thrower number one. You're going to see what we call is our pillar three, is we have to drop into the throw. So sometimes there's a drop, right? We don't want to drop. We don't want to turn in place. We want to drive into the ring. So we're rotating, and turning, and dropping in. And if you don't drop into the throw, you're going to fall, right? And that's what we don't want to have. We're going to fall in the throw too, in the rotational throws. If you don't have the axis set up on the entry, and drop it into the throw, the hip falls to the side, you're basically trying to turn, and you're falling into the throw. And that is going to put you off balance. And then you're going to be decelerating. You can't increase your speed. So if you apply speed, even when you're falling, can you kind of potentially salvage that? Right. But it's a salvage. It's not an optimization of your technical positions. Therefore you're going to be losing distance. So when we look at thrower number one, and we see him, he's dropping here, you're going to see that nice little turn, right there. You see that knee kind of hinge, and now he's just passed nine, and he's pushing that knee in. So he's moving everything into the direction of the throw. You're seeing this, right? Now the thing that this thrower is going to be learning in is back in our entry, we didn't talk a whole lot about the sweep leg. We just talked about kind of the position of where it comes up, but we are going to be learning how to keep this leg active and open and keep those knees at. And so right here is where you set in, and you kind of create that wide sweep. So again, this athlete's been throwing for literally a matter of hours. You got a pretty nice position. He's not entirely sure how to use that left side yet. So you notice his left arm's a little bent a little early, but he's got a pretty good sweep. It goes a little narrow. But again, for day one, right? This is really, really good movement. And you're going to notice right here that that foot keeps moving. And so he's not quite over rotating. He's pushing and his hips are moving, which is real good. So what we're going to be doing is coming into pillar three, we drop in, we're creating speed. Now what are we going to do? We don't want to just keep blowing past the middle. We've got to get this guy, he's got to wrap in his arm. So that's where you see kind of this arm come in here. And you want to see that you see the right side and the left side kind of come together so that you can accelerate rotation. And then you're going to come open long again. You're going to see how this arm comes out. I've always talked about creating that long path and position. And that's going to be key. So as the thrower comes through, he can keep turning.  So he's used to kind of pushing instead of rotating all the way through and getting that side so he can push behind it. So now if you just kind of back it up a little bit, and we're going to kind of connect how that dropping in and applying speed in the rewrap affects that length through the finish, right? You see that. Again, when we look here, what's going to be the key is do we push the knee? We have to be driving in. And this is where we talk about that if the upper body opens too much, you're creating rotational speed, you're going to be just rotating in place and you're not going to be driving in. So it's a combination of creating an angular rotational linear position, right? And that's why this is tricky because you've got to learn how all that puts together. And so when we look at thrower number four, you're going to see how he's going to come around. You're going to see how the lower body is really moving this way. He's got a nice left foot. He's got a nice sweep leg path is starting to be established. And you're going to see how that leg kind of comes up. And then he's out here nice and wide and you're going to see how he kind of opens this around. Now, if you looked at best male throwers in the world, you've looked at Joe Kovacs, Tom Walsh, Ryan Krauser, Darlin Romani, when you look at those four guys, each of them are doing essentially the same sort of mechanics. There's some key nuances, differences. Walsh and Krauser and Kovacs and Romani, they're all going to stop that arm and they're going to do what we're showing you here in this pillar four position. So when you watch this and you drop in, you see like this, this thrower number four, you're going to see how he creates that rewrap right there. And so what we're looking for with the rewrap is we want to create this nice vertical axis. And the closer this shoulder can be to this line, the more torqued up the upper body. Now, when I say that, we're never trying to keep the upper body or the shot back. We've always taught rotate the right side around into the left side. And so we're creating this line between the two sides. And that's the key. We're not trying to hold the shot back. We're trying to accelerate. So we're bringing the left side and the right side together to create faster acceleration. So when we look here, like this is again, our thrower in week one, thrower number two, you notice he does, he does a pretty good job of creating his rewrap, right?   And then you're going to notice thrower number three, same thing right here, you can see when we put him in position and we go right here, you're going to notice that his shoulder is actually kind of the furthest away, right? We want to be more in this position. So by the time we're there, so when we get it, and we get that left arm here, you're going to see how it's opening just a touch. So this is going to be factors that are related to what are we doing? We're dropping in and how we attacking the throw. So thrower number three, he's got his chest open a little too much to the throw. And so therefore it's going to pull him into the throw and it's going to create a little bit of shift to come through. So here's the thing that we're going to focus on. We are trying to teach how do we move this sweep leg in our pillar three and how do we have. So if we have everything lengthened, we're going to feel the long path. The left arm goes short, the sweep leg tends to go short. So let's look at thrower number two real quick. And so you're going to see he's got a pretty good long path and then the upper body starts to follow the arm a little too much. Whereas if you look at throwers two, three, and four together, right? Watch when we go back and we look at three. So see how we're all right basically at this point here. And now watch two, he's upper body starts to lead. Whereas you notice three and four continuing the long path and they're not allowing the shoulders to turn, they're turning the arm. And this is a really important cue for you guys. So as you do that, now how are we going to do that and how are we going to break that down? This is where again, inside the throwing chain reaction, we go through this in detail and we really break down specific drills and we start putting it together in pieces and we put it back together in what we call pillar connection. And then we apply it to our throwing and we're always being able to find the individual areas that need the most attention. Your throw is going to be the result of your weakest link. And so if you can get all of your links in your chain nice and strong, your throw is going to become really good. And then you're going to be working on elite level stuff before you know it. Today's video, we are covering part four of our rotational shot put one on one, right? So one of the things that we showed here and I thought it was really good to go through showing a beginner, right? We've got our thrower number one, who's day one, we've got thrower number two, who's in week one of the rotational shot. He's been throwing for multiple years, but he made the switch and a lot of you guys are going to be in that situation. Thrower number three and thrower number four are multiple years and they've been getting into more and more details or more comfortable. They're past thrower number one's phase, right? And for that matter, past thrower number two. So today, what we want to review that we've obviously gone through the entire throw. We have explained that thrower number one, when we look at the speed of thrower number one, he's taken just over two seconds, thrower number four is taken just under a second and a half. That's the thing. By the time that shot's done to out of your hand, now we're in our pillar five. And this is where you're going to see. And this is where I'm very impressed with thrower number one. It says a lot about the athlete, but it says a lot about what we're teaching at our events and how we're going through the process. So this athlete had gone through our biggest update on our throwing chain reaction. So if you've been to a 2019 camp, we kind of shuffled things around, put it together. And so what you're seeing, what we had set up for 2020 was this more integrated mix where you're getting to a lot of things and you're moving through and you're actually learning and understanding the connection and how the system works on a better standpoint. So here's a perfect example. Again, first day rotational thrower, 10 years old, and there's some really great positions here. So when we get to this power position, this is one of the things we want to do. We want to be on the ball, this foot, and you're going to notice that all of our throwers are there. And you're going to see that knee pushing out. And you're going to see, here's our 70 foot thrower. You're going to notice he's got the longest strike path. So again, one of the things that we discussed was we have to get the lower body and the feet. So we have the legs in the foot position. So we're going to understand how those things put together sequentially. And then we have the trunk, right? And we refer to the upper body. So we look at our center mass and we look at the path of the arms as we're going to come through. And again, if we have a short, a short path, or if we have a long path, and that's going to be the key. So the key that we talk about in pillar five, right? This is the beginning of the power position. So we want to come out of the throw. So we're going to go here, and you're going to notice we're going to open the arm is coming, opening out, because we want to stay as long as possible through the finish. And that's what our throwers are learning to do. So look at thrower number one. He does a great job of really staying long. Thrower number two is shortening that just a little bit. Pretty good path. Thrower number three is doing a pretty good job as well. You're going to see that. And but you're going to really notice the length on four and number one and number three. He's pulling it in a little bit. But if you looked at our other videos, we kind of explained what's going on in his chain reaction and why he's going to wind up pulling it ahead a little bit earlier. The thing that we're going to do is as we move here, this is what we set up. We set up the right side to be moving around. Okay, we stay on the ball of the foot. We take the left side nice and long and out. We're not smashing it yet because we're getting the body set up to hit what we refer to as our pillar six. And that's where you can see we want to stay on it and come around. And that's that long path that leads you into the throw. So you're noticing that when you see this, you see the block arm stop. See how the block arm stops. Again, block arms. Nice stop on the young guy. Again, but he's kind of pulling down. And that's where you're going to see his elbow is going to kind of drop. And he's going to lose a little bit of strike. And then you're going to see thrower number three. Pretty good.  And you're going to see how we get this position as we go through. So here we go. So you can see that. Look at the extension. So you're going to notice the extension with our most advanced thrower. Pretty nice extension here on number two. Again, good extension here, but you're going to notice how he pulled around. He's not, he's day one. He doesn't know how to hit the finish quite yet. And then you're going to see thrower number three, a little too much jumping. He kind of pulled off of it and he didn't allow himself to come around. Again, we're more proponents of what you see the best guys in the world doing. And that's kind of what we teach the movement. We don't know exactly what they're saying, but we're teaching the same sort of patterns to kind of create the same types of finishes that you're seeing with these throwers. So let's clear out all the lines. As we look, the one thing you're going to notice in common with all of our throwers is you're going to see this nice long counterbalance. And again, you notice our day one thrower toning around 25 feet. So we got thrower number two who's been throwing about roughly a week. And truthfully, this was about maybe his third training session. So this guy was absolutely tapped, but you can see how much better he was moving just in a matter of a handful of hours, right? Four or five hours. And it's, it's a huge difference. Now remember four or five hours, that's a lot of throwing, a lot of training, a lot of really specific work. And that's where the throwing chain reaction system comes in. Like I said, we got the link below. Check that out because the idea here is what we want to do is show you, you need to understand the direction. That's first and foremost, if you don't understand what you're trying to accomplish from a technical standpoint, you're going to be out there working on God knows what, and it's going to be an inefficient use of your time. This is a perfect example. Thrower number one, one day, multiple hours, hitting great positions. Now what are we trying to do? You see that his foot's a little low, but look at how nice the left arm, the block arm is. He's opening this foot up. He's got his knees loaded up. Everything's working. I mean, that's a really great position for a day one thrower and he's 10 years old. He's got to stick to the principles that we talked about in the first videos on how do we set up like our pillar one and two, pillar three, four, which is enabling this pillar five, six. So as everything stops, now we're into pillar six. He's going to be able to start engaging the block side. You can notice we don't see that block arm yet. Now we see it. It's stopping. He's pushing. He's a young guy. Look at that position. Really great. And again, look at our weak one thrower, right? Same thing. You see him coming through.  He's not used to it yet. There's no timing yet. He's been traditionally a glider. Kind of hit this plateau with the glide, which is going to be real common. Now he's throwing as far in a matter of a few sessions with his rotational shot that he was with his glide. And that's a huge sign that he's absolutely going to be throwing much further. Now, thrower number three and four, you can see have the best positions. Thrower number four, again, one of the best all-time high school throwers in the U.S. 22 meters. You can see how he squared up elbow in position, shot, great block arm, hips coming through. And thrower number three, what you guys don't know is because I coach him, I see where are his strength deficits. And that's where you can see some issues. And you can see how he's pulling away. He's jumping and pulling away. And what we're going to do is that's going to be largely due to some reaction to strength. While he's setting up some good positions, there's some technical, there's some strength issues that are inhibiting his ability to move efficiently. And that's one of the key things we want you to do. So big summary. Again, when we guys take it back and we look, we set up pillar one, there's very specific action. You're going to notice different actions here. I really like what day one, again, day one thrower, as I mentioned before, he's a perfect example of the updates in the system. And so you're as good as what you focus on. Here's this guy. This is the first time he's ever done it. And he's hitting great position. So I really like the setup and the rhythm and he's coming here. What you don't know about pillar thrower number three is that he came from kind of a different technical model. We've retrained that and it's more efficient and he's doing a lot of really good things. And I think he's going to have a really big year coming up. Looking at thrower number two, you can see of all the throwers, he's going too fast with the upper body in and it's following the shoulder. And that's what's creating some of his technical issues. So we're looking as we go pillar two, three, so it kind of shortens up. He gets a pretty good rewrap. Throwers one, three and four. You're going to again notice that notice the rewrap on thrower four. Again, he liked to style wise. He liked to have the arm low. So you see throwers like Darlin Romani likes that. You see Crousers up a little higher. Kovacs is higher. Walsh tends to be lower. So again, this is going to be some style things that you're going to see. And that's what you have to ultimately do. And that's what our system allows you to do is to find the best things that are going to work in terms of what are the biomechanics and the physics say that we need to do. That's what the throwing chain reaction is all about. We're showing you how do we create those positions. How do we connect those positions. That's what we call pillar connection. And that's how we facilitate the chain reaction. Hopefully you guys found this helpful. You can see how all those things yield and set up these final big throws. That's the goal. Again, if you guys are unfamiliar, be sure to check out the throwing chain reaction system. Click the link, visit our site. We go step by step on how to teach you the right things. So we've been able to help our coaches to help thousands of athletes throw further. So be sure to like, hit that subscribe button, share this video, comment below on anything you'd like to see in the future. And we will see you on the next video.