Good morning again everyone. Thank you very much for for coming this morning I see lots of faces who have come from from from near and far, and I know it's a slog to get in This time of the morning too so thank you again for for making the trip and I would be remiss also as I was in my report not to more personally thanks those of you who participated in the working group series and Gave a lot of your personal time to help Educate as I said a liberal arts major about robots, which is no small task So I see in particular Colonel Bill Tartt who's in the audience Thinking about his next career phases and had the opportunity to go to your retirement a couple months ago and really drew home also everyone in the Air Force who has worked so hard after the over the past decade and in the other services thinking about Unmanned systems and I tapped a lot of you who have that hard one Experience and harvested your thoughts and hopefully reflected them in a in a helpful way. So thank you to all of you who contributed We are privileged to be joined on this panel by three qualified highly qualified experts to discuss with you What we've framed as the topic of US leadership amid commercial sector growth and changing global supply The panel opens for discussion a few of the major themes in our report The first is to consider how relatively important Unmanned systems technology and unmanned aerial vehicles systems RPAs whatever we'd like to call them How important is that technology not just the way it's being employed today? But the way that it might be employed going forward and not just by us But by other nations and non-state groups as well Is it transformative or revolutionary and does it deserve the attention? It's receiving including the proposal that we discussed earlier to elevate it to its own office within the Department of Defense Is that a is that a good idea? Is that helpful or is it not helpful? Related is doD effectively organized currently to harness the available benefits of unmanned systems This came up a lot in the panel But but going forward are we going to be able to take full advantage? And I think Dan's question got to the part of it Which is how else might we use them? Where might they be most beneficial and how is unmanned systems? Technology again being adopted by foreign militaries What role can and should the US play in shaping a more favorable outcome in this diffusion of military innovation? Stealing a line directly from my correlates What are the arms control considerations? Let me introduce the panel to you in the order in which I will invite them to offer their opening remarks Depending on time I might follow up with a few hard questions of my own and then turn it over to you all for a Q&A session Very pleased to be joined by dr. Lynn Davis who's a senior political scientist at the Rand Corporation and also serves as Rand's director of the Washington office From 1993 to 1997 as many of you already know dr. Davis served as the under secretary of state for arms control and international security affairs We are delighted. She's here today To discuss among other things the findings of Rand's new report armed and dangerous UAVs and US security Of which she was the lead author This is I will tell you if you haven't read it the best report I've read on the arms control Context and implications of UAVs and I would strongly encourage you to pick it up if you haven't and read it alongside the CSIS report Of course great compliment And while I'm assigning UAS reading let me turn to JJ Gertler Mr. Gertler is a specialist in military aviation at the congressional research service with a long and distinguished resume of government and research Service before that including a stint at CSIS Where he was a senior fellow and in January 2012 he wrote a report on US unmanned aerial systems That's a foundational to understanding on the topic and was very helpful to us as we conducted our research And finally, let me thank Mike Horowitz for making the trip down from Philadelphia to join us here today Dr. Horowitz is an associate professor of political science at the University of Pennsylvania And he is a non-resident scholar here at CSIS In 2012 he served as a council on foreign relations international affairs fellow at OSD policy Where he toiled under people like Ken Handelman in the audience there And worked on among other things UAS export policy issues And he has just published in the current issue of foreign affairs a provocative piece titled the looming robotics gap and Another another must read that I will assign coming out of this panel and with that let me turn to dr. Davis For opening remarks. Thank you again. Thank you. Thank you Sam and also for the plug for our recent report. That was that was kind Let me just start by saying I'm going to take you to a slightly different level and pitch For the presentation that I'm going to make but we can come back to some of the more specific issues. I Will be talking about armed UAVs RPAs Drones UASs, you know what I'm talking about. So that's it's but it's the armed systems And I'm going to just describe to you a couple of the questions that provoked us into the study that we've just we've just completed And that is folks. We're talking about these weapon systems being transformative Talking about these weapon systems proliferating around the world with dangers that one could only imagine People were talking about Having to find arms control solutions that is to keep that proliferation from Happening and then others were talking about the arms control system non-proliferation system that's already in effect the MTCR is standing in the way of all that ever we ever wanted to do in terms of kind of developing these systems and Selling these systems to to friends and partners and so that was sort of the set or the context that Let us to want to kind of step back and say, you know, what do we think about those questions? And how would we answer those questions? And so in a just a couple of minutes I'm going to lay out the propositions or findings and if I don't provoke you along the way to a question or a Response I'll be a little surprised But basically I just want to share with you some of the thoughts that that you know came to us as we work through some of those questions and Issues and the first sort of I think important thing to say about this whole subject is That it's really important to distinguish between Shorter range systems longer range systems low technology systems high technology systems for this Audience I don't think I need to say that but I think the debate often is at cross-purposes Because people don't start by saying I'm talking about short-range systems or I'm talking about unarmed systems Or I'm talking about long-range arm systems And so I think it's important as we think about how we answer some of these questions that be very very careful When we look at how these systems are going to proliferate There's no question that short range UAVs are going to proliferate There are lots of civilian applications the technologies for those are not that sophisticated And so we're going to see a proliferation of those kinds of systems but if you then focus on the longer range systems and Those that are armed They're really kind of reduces the set of countries and places that can develop those and will be likely to acquire those And I think in addition what we have to look at is that many of our Partners but also our adversaries when looking at these systems may actually find other systems that are more attractive Either a militarily or in terms of cost and so that sort of sets the boundaries if you will of What it means to say that these systems are are going to proliferate So then turning from the fact that yes, they're going to be around of different kinds and and different characteristics Are these systems transformative that is are they revolutionary are they game-changers you will you I didn't know Why we weren't asking that question quite frankly when we started but people said well people are out there saying that and so we better Try to answer that questions. And so we answered the question by saying no it's not transformative in the sense of nuclear weapons or even airpower is it sort of developed at the end of End of the First World War and on into the Second World War So we basically you can find the argument there So I've actually seen a couple of you know people shaking their heads So maybe I'm having an effect already in terms of what it is that we're that we're finding That's not to say that these systems don't are not very attractive That's you know trans it's a different sort of question as to how you would find benefits from employing these systems And there are clearly benefits and we go through the different types of benefits that one can see starting with long-range armed UAVs clearly they are very attractive to us in the circumstances that is the United States and the circumstances that we've been using them Against insurgents and insurgents without air defenses. And so they are in a very attractive System for that particular set of operations and we can and we would sort of move through other places where we could see their Attractions, but then we step back and say but then there's some sort of you know if you will sort of Sort of not Necessarily things not to make them useful, but they have their own sort of vulnerabilities and limitations And of course the the vulnerability of the longer-range systems is that they you know that air defenses are counter to those to those systems Again, I'm not really telling most of you what you don't already know But I think it's important to kind of set the stage for this for this conversation Short-way short-range systems That's what we were kind of talking about for the most part this morning and the first panel are you know have enormous sort of Opportunities for the mill for our military services for others. We could see their potential value the terrorists Not only just to have a sort of an effect a military effect, but maybe a psychological effect So it is the case that they are you know sort of you you have to think of the sort of Benefits and limitations of these systems Thinking about the particular missions that they might be Undertaking and so before I take up all your time on the panel I'm gonna take take the point of arms control something that I've spent a fair amount of time in my life thinking about and non-proliferation and so you know These these are are going to proliferate But they are not going to kind of create the same dangers That we saw with the proliferation of nuclear weapons or even with missiles long-range missiles In terms of the need to think about a regime, you know in terms of Their non or preventing their proliferation I think it's a very different situation With the respect with respect to UAVs armed UAVs Unarmed UAVs and I think it's important to see that distinction in my own mind These systems are more like conventional aircraft than they are like the other kinds of systems that we have sought to Design non proliferation regimes, you know to prevent the trade or the the transfers Now given that's where we are It's also the case that these systems find themselves already in a non proliferation regime in the missile technology control regime They are already part of that regime and they they are part of the the controls or the The ways in which we seek to prevent their proliferation and and so I'll make two sort of final points about About that and if I haven't provoked you by now I expect that I will in the next next couple of minutes one I see enough flexibility in the MTCR the missile technology control regime for us to be able to share and sell These systems to our partners and to our allies I think the regime itself has that flexibility built into it And I'm not one to say that somehow we undermine the whole regime by precedence in terms of the potential sale of those systems To our allies and to our friends But what's good about the MTCR is that it already has in place You know a consensus in favor of keeping those systems out of the hands of those that we find dangerous So we can use that regime to prevent the proliferation of these systems to potential adversaries and to Potential groups that could be dangerous to us So in a sense we have the the ability to to kind of make the policies that we need with respect to sales while controlling With our with our allies and those other members of the regime those that could be dangerous to our security So let me stop there Thank you. Terrific JJ And let me pick up from there in a certain sense I can't certainly talk about the arms control aspects But more on the technology side and how widely this stuff is likely to proliferate Both Dr. Davis's report and Sam's and I'm happy to endorse them both having spent time at both Rand and CSIS My sister went to Penn. I can't claim any personal experience there but Both of those reports Indicate that the technology. We're talking about is not all that difficult The United States has been getting heavier than air Vehicles to deliver ordinance with some guidance to a remote location since 1917 Active guidance during the course of flight followed not that far afterward and hobbyists all over the world every weekend are flying devices larger than many currently fielded US UAVs some of which are jet-powered some of which can drop objects and They're flying them quite some distance from themselves. The technology is out there for the shorter range And it's not that terribly difficult to scale for longer range But there's a big asterisk on all of that and it is that while the technology of the air vehicle itself is not that hard The communications infrastructure for true global operations is reserved to a very few countries Colonel Callahan could sit in his office somewhere in the western United States and fly a UAV Halfway around the world because we've got a satellite constellation that spans the globe and provides connectivity Russia can do that China arguably can do that and that's the list of countries that can do it when we're talking about Transferring these devices We're talking about the ones that can be controlled basically line of sight Some other countries are developing the ability to do communications relay whether through aircraft or through other UAVs But that in some ways is the more challenging and far less developed technology than unmanned vehicles Remotely piloted vehicles themselves Is it all coming? It's coming all over the world There are over 70 countries that are developing one flavor or another of UAV a Smaller subset that intend at least or have declared an intention to develop armed ones One of the countries that's maybe not doing quite as much is the United States of America And I apologize for the small chart especially to the folks with cameras in the back The drones are coming the drones are coming well. This is DOD's plan for procuring UAVs through the current FIDUP We go from over 1200 a couple of years back to a running level of less than 30 per year Why is this it's because the United States is Quite consciously taking a strategic pause in UAV procurement really two reasons One for the types of environments. We've been operating in for the last 10 years We've been buying plenty of UAVs that are suited. We have a surplus on hand We have enough to provide the requirement of 55 full-time caps with a surge to 65 If any of you run across General Dave Deptula in your wanderings by the way You can get an earful on just how relevant a metric caps is but We have that technology already well in hand and fully bought the next wave of technology Faster stealthier There's currently not a procurement plan at least not one that comes out in the public budget documents to acquire such UAVs DOD is consciously waiting until that next generation is ready Before they go and start buying more UAVs in order to operate in different kinds of environments than we have been for the last decade That raises the question really of whether DOD is well organized to go ahead and do that You've seen the moves in Congress. You've seen the proposals to concentrate all UAV procurement in a or at least development in a central joint office From the Congressional Research Service perspective where we don't make recommendations or conclusions There's some merits to that. There's some demerits to that but When you had multiple services buying essentially the same platform or a couple of platforms from a very limited universe of Manufacturers, it's reasonable to ask whether there really need to be separate development Organizations doing that as opposed to those that are developing their requirements and subsequently fielding them So other countries are going to have UAV capability They are going to be able to drop things at least within line of sight control on adversaries How we go about controlling that regime? I'll leave to dr. Davis But whether the United States decides to take Action to begin defending our forces and whether that's more a function of something like network attack than physical air defense systems to guard against these Might be a good thing for a joint office to discuss or at least for think tanks Thank You terrific Mike Thanks a lot. Thanks for having me Sam. It's a honor to be back here where I first started after college as an intern I I find myself in violent agreement with much of what the two previous speakers Said and in the interest of getting to questions. I will I will try to keep this at least reasonably brief especially for a professor I Think that it's a mistake in some ways to think about this from a proliferation perspective is just a question of Reaper proliferation to start and Essentially asking the question will others how many people will be able to do with the United States can do now because I think that Misses essentially the military robotics forest for the the CT slash ISR Reaper trees and That we we have this current generation of UAVs that we've used to to great effect around the world and that we're now concerned with others Using but as the I think the previous speaker Mentioned the the the more important issue in some way isn't what we have now It's what it's what might happen next and in that way. I think that the trends from a proliferation perspective or not promising if you think about the Factors that make military technology a lot more likely to spread and spread pretty quickly Two of the most important are whether they're underlying commercial applications for the for that technology and what the unit cost of that technology looks like and In the case of most military robotic systems of which UAVs are a subset You're talking about a technology that that as others have said is actually pretty simple to build And where the technology is likely to proliferate further It's not that surprising that is thinking about it that way that Google purchased I think 10 to 12 robotics companies over the last 16 months and in fact is taking those that worked in the defense sector and pulling them out of the defense sector Think about how big the US defense budget is if if Google thinks that the US defense budget is such small potatoes That it's worth redeploying their robotics engineers to work on commercial applications What does that say about what the size of the commercial market is likely to look like and what the spread of this technology Is likely to look like I think what it suggests is that this is a classic example of a type of technology With lots of commercial applications. That's likely actually to spread very far And I think it's also likely to spread very far due to international military investment International military exports and there was the report that surfaced just a few weeks ago For example about Saudi Arabia allegedly purchasing an armed Chinese drone That I think is sort of fascinating and I'll get back to the implications I think for US arms export policy in a moment But I think what that suggests is that the the proliferation of longer range armed systems And and maybe I'll try to square the short versus long circle here and say maybe medium It might happen a little bit faster than we had anticipated Because it's not just that others will have the ability to build many of these systems because the underlying Technology is easier. It's pretty easy to get It's that there are others out there countries like China and Israel and others that seem willing Certainly more willing than the United States to export a lot of these technologies and in a combination of military necessity and in probably the the wow factor from US usage over the last decade has Generated a lot on the demand side with many countries around the world interested in acquiring advanced UAVs and For many of them Shorter range armed systems may be enough I think I think the whole panel shares the view that the proliferation of those kinds of systems is is inevitable But I would go a bit further and say that I think that the proliferation of longer range systems Maybe a little more inevitable than we've realized Keeping in mind of course that the that the ped tail and all of the processing challenges Involved in doing exactly what the US does is something that that others probably It'll be very difficult for for many people to do But that doesn't mean that especially as this technology gets cheaper and proliferates further That others won't have longer range systems able to do more and able to do things that are militarily relevant And that's why I think it's important for the United States to do two things One is I think that the as the last panel hinted at the Ongoing debate about the capabilities that the new u-class system will have I think in some ways is the the canary in the coal mine for exactly that that chart the hesitation that the the United States military has demonstrated when it comes to investing in next generation military robotics especially military robotics that get out of the coin CT ISR kind of box and those are the technologies of course that are the most challenging to existing constituencies interest groups in the service in the services those are the ones where we would therefore expect there to be a lot more resistance and So I think continuing to push the envelope on those technologies Especially in an era of fiscal austerity will be difficult, but something that the United States needs to do and And the second thing I is that I think the United States actually needs to lean a little bit further forward When it comes to our UAV export policy And I think that the United States should do this of course within the confines of the NTCR in a way that doesn't abrogate America's arms control commitments, but in an age of fiscal austerity Building partner capacity to substitute or supplement for American capabilities is an important national security priority And there's a strong demand signal from America's allies and partners This is a technology that that many are interested in including some of our closest allies around the world and I think it's possible for the United States to design a responsible export policy to close out You know that involves allowing more armed systems out the door to our closest allies and partners But the alternative here is not that if we don't do this they won't get these systems We like to think that Sometimes these are unique and special snowflake But given that we know that the US monopoly on on armed UAVs is over if it ever existed The alternative is not that if we don't sell armed UAVs Others won't get them the alternative is either that there will be incentivized to build them themselves To invest more in those sorts of development capabilities or that they'll simply buy them from others and I think in a world especially when it comes to the present generation of UAVs Well, we're interested in promoting responsible usage of this technology in a safe way that that complies with with the law of war international law, etc We have a much better chance of encouraging that with responsible exports Then by you know, you know throwing our hands up and saying we don't want to play this game and letting countries buy Systems from China or design them themselves instead and not getting access To the training and other things that will allow the US to shape the way that the rest of the world uses these systems I'll stop there Thank you very much a lot to chew on and before I turn it over to you Let me ask dr. Davis to follow up on a point that that Mike raised So if we assume that the proliferation or the spread of these systems is is somewhat inevitable At least in in terms of shorter range armed systems What are your thoughts about? How the US has an ability to set the standard on how they might be used or or do we have an ability? Will how how are other states likely to use them? Will they look to use them in the ways that we do should we expect? surprises and and then After after your thoughts on that JJ, maybe if you could pick up the question of Defense If these are spreading rapidly have we done enough to think about our vulnerabilities in in that environment? Sorry, I'm gonna go back again to say and I think we tried to keep the distinctions between short Fuzzed over a little bit to longer range But I think we think about arms sales and export controls We really do need to think about the distinctions and also when we think about sales to to partners allies the distinction as well I As I thought I sort of began to suggest is that you know sales of these particularly a shorter range systems Maybe even over into the longer range systems ought to be in my mind thought of more like Conventional aircraft that it is like other kinds of non proliferation Types of sales and if you do that then you're thinking about those in the same ways you think about You know a sale of an aircraft to to a partner and is that the best use of their money? Is it the best use of our? Sort of sales to them if we want to think about building partnerships or interoperability and all those all those different kinds of things So if you have I think that sort of if that's on your mind as you think about this And I think you have a way not necessarily, you know Simple way but a way through the issues having to do with export controls and and transfers And that's the way I think the debate ought to be to be structured as we think about arm sales Of course the argument is made that we would potentially have more Control like you say over the use of those if we are actually the folks selling them rather than they're developing them on their own Or they're coming from Chinese the Chinese or whomever And here the the last part of our report really does talk about how we ought to see the own our own use of these systems As a sort of motto for their potential use to others which takes the additional step Beyond how we might be able to sensitize our allies and and friends as to their use if we're in the arms sales business But then beyond that to how our own use and we talk about the different ways or the different Foundations that you might set for our own use over time so that others would use them Following along those same sorts of principles and I think that's a discussion that started by the the way the Administration has started to talk about its own use and sort of you know post-Afghanistan now in a world in which You know terrorism is more you know more more global But still with the value of these systems do that particular set of missions and contingencies And so as we as we try to think through our own use and how we place it in the larger Environment to start thinking about you know whether there are ways in which the US can take a leadership role in Establishing international norms for the use of these longer-range system arm systems And whether or not there are ways that we can gain a consensus among those who would be operating those into the future Thanks Thank you With regard to defenses United States forces don't tend to like anything flying over their heads that isn't ours and fortunately we've operated in a lot of environments In the last 50 years where there hasn't been anything flying over our heads that wasn't ours And we are optimized to defend against many types Take the whether it's an inhabited vehicle out of the question if it flies at a certain altitude at a certain speed We know how to shoot it down What is unusual about the UAV challenge in the air defense is first you have some that are very small and fly Very low in an area that we haven't traditionally optimized our defenses for But second you also have some other tools that are applicable Electronic warfare becomes a much more relevant tool if you're trying to disrupt an unmanned vehicle Network attack becomes a much more relevant tool if you're trying to Disrupt an unmanned vehicle so there are opportunities of available to us One of the reasons in fact this may be a decent argument for that joint Organization is because the threat to each of the services is different from UAS But some of the tech underlying technologies that you would use in addressing that threat are very common Whether that would come out of a UAS office whether it would be charged with defense as well as offense Is a separate question? But when you start looking at things like EW and network attack You're looking at basic technologies and then refining the application service by service rather than having to develop the technologies in parallel Of course our adversaries have those same opportunities against our systems as well So you're in the sort of classic way of doing it and then we get to the question of should we export that? Mike did you want to jump in? I've got another one for you if you if you okay great the the question for you is based on your research on historical diffusion of military innovation and watching how Militaries do and do not successfully integrate new new technologies What is your assessment of how DOD is doing with this technology? How how's it how's it going and in light of sort of next wave technologies? We anticipate to be intersecting like autonomy What what what are the keys to? Maintaining our edge That's a great question and and in some ways I think it gets back to the the great point that the issue for each service is somewhat different that the challenge is presented by unmanned systems and even unmanned aerial systems or vehicles or somewhat different the you know what we all we all know what we don't want we we don't want the US to end up you know in the way in the sort of stylized military history fact sort of like the British with the aircraft carrier Where they they're the inventor of the technology even have some systems that work pretty well But essentially don't figure out the force employment They think about it more as a spotter for the battleship rather than as the the mobile airfield that we've come to think of the carrier as and In you know in some ways the you know to throw a little sci-fi out there Hopefully, you know sci-fi the the worst case for for the country like the US is Imagine a you know a future contingency in you know in a place like East Asia where the United States US is fighting against another country's Air Force say to use the Air Force as an example and we Toss up F You know fifth-generation aircraft F 22s and F 35s and they shoot down our adversary systems at a ratio of say 8 to 1 or 10 to 1 but suppose our adversaries deploying unmanned systems that they build at a cost differential of 15 to 20 to 1 That's a cost curve that the United States would be on the wrong side of and despite having what would still be the best pound-for-pound Aircraft in that case in the world That would be a conflict that I would be extremely nervous about and That's the outcome that I think a lot of people you know I think in in this room and we think about these issues seek to avoid in That I think that the DOD the Department of Defense has done reasonably well up to this point in investing in some of these technologies But the big challenge moving forward is going to be integration and I mean integration in two arenas I Think the first is it's no doubt and in some ways think the comment on the previous panel that that you know You take you know our kids sitting on a couch and they are quick to twitch reflexes and computer skills over anybody else in the world Point well taken the the challenge is taking the technologies that are being invented and Moving them into the services making them regularized parts of how the services operate I think that's both true for unique military capabilities for platforms the services develop and I think it's especially true for the integration of commercial advances Think about how long it takes us to build anything. I think that Developing a faster ways to integrate commercial robotics Especially as that market explodes over the next several years will be crucial to sustaining America's edge Yeah, and I should say following up on both the the defense point and on the exports point We have a lot of experience and understanding of air systems We haven't been talking nearly as much either here today or in the community as a whole about Underwater vehicles and land vehicles. What are the potential controls on those? What is the desirability of controlling those? What is the United States interest in defending against those? My perception not being an expert in that field is that the international development of those systems lags behind us even more Significantly than in the air, but it might be some since the subject is unmanned systems and not just aerial We might want to give some thought to those Great, and let me open it up to audience questions, and I will take two of them at a time This is a great innovation because then we can pick which ones we want Jake Carney might start doing this too So sir in the front and if you wouldn't mind just waiting for the mic Sam's bringing it to you and identifying yourself Hi, I'm Richard Whittle. I'm an author and a fellow at the National Air and Space Museum And I'm wondering especially based on Dr. Horwitz's last remarks Going forward is part of the reason the Defense Department is investing less in UAS That it's investing so much in the F-35 the F-35 is supposedly the last manned strike fighter and The u-class I guess is the first unmanned strike fighter So are those two programs complimentary or competitive? Great question, and we can take one more if anybody has one sir Bill Greenwell American Enterprise Institute. I want to follow up on the commercial side And and obviously there is a lot of investment a lot of interest going on at what point you know And obviously we still have the FAA to figure out how we're actually going to employ these things and that could take a Few a few years to say the least But at what point does the MTCR become a barrier to the deployment of say Unmanned cargo aircraft it seems there's going to be a range and a Capacity that's going to inhibit not only cargo aircraft but possibly some of the things that Google is thinking about in networking Thanks, who would like to start it might even keep Amazon from delivering my stereo Well, we'll have the tenured professor take the F-35 question sure the I think the I think the F-35 and u-class should be Complementary rather than then competitive I'm not sure if it's necessarily good to end up that way if you think about you know service cultures and the potential for For people to think that they're sort of competition between them and this gets to whether the u-class will look You know more like a reaper on a boat or more like a you know more advanced You know sort of next generation system, but I think that we tend to I mean there I think there are basically three different ways that we can imagine the integration of unmanned systems And I think this is true both in the air and air and beyond the first is the way that we've often been thinking about them Which is essentially one-to-one substitutes, you know either you'd have an F-35 or you'd have a u-class But the second and I mean Boeing demonstrated last summer some of the possibilities here through the remote piloting of an F-18 is essentially man-unmanned teaming the possibility for say An F-35 up in the air with a bunch of unmanned fourth generation systems working together And then the third is of course the you know swarm Which means something we know it's important, but we're not sure exactly Exactly what it means And I'll leave the MTCR question to the arms control expert JJ did you want to add in first on the first question, and then we'll go over to dr. Davis here? Only two things one is in a competitive in a limited budget environment such as we have today every program is competitive with every other Program I haven't seen signs that DOD is consciously trading off F-35 against unmanned vehicles or the other way around but You're trying to fit all of these programs through the same soda straw, so naturally there is some competition And the other oh the other part was regarding the notion of controlling other UAVs Those of us who've been around a little while remember when the army Comanche was going to be the quarterback of the digital battlefield If you conceive of an F-35 Controlling for UAVs in concert those loyal wingmen we heard about in the first panel Isn't that the real definition of the quarterback of the digital battlefield? So I think you were were you discussing our own development or the potential transfers of systems So let me come back and say it's really important to and we try to do this in the paper And for those of you really wanted to get in the details. I that's what we were we were trying to do Because you're right to say it really does depend on both the range and the you know the capability of the system to deliver a Payload so you have to start there and you have to see whether it falls within those parameters or not In order to get to the MTCR kind of categories, you're not talking about line-of-sight kinds of Vehicles you're really talking about these more sophisticated higher technology vehicles so the ones that Amazon may be deploying or the ones that they're deploying over movie sets and all that probably not gonna fall within The categories of those but notwithstanding that The regime itself is an export control regime It's not like a treaty that that bans weapon systems like on the nuclear side So it's an export control regime in which each nation has it's you know keeps its own prerogatives to sell and transfer and In my view there's technology. You know there's flexibility within that system to make those kinds of transfer sales You know as they if if they seem sort of warranted in our national security interest And so I think this whole idea that MTCR is a block is Really not the way to think about the regime Bottom line Great next question Patrick Tucker Thanks for the call out. Yeah, Patrick Tucker defense one so To me it seems like the cost effectiveness of these machines goes up as autonomy is introduced and there are a lot of things to talk about in terms of Increasing autonomy. There's ethical considerations. There's clear cost considerations anyone in AI will tell you there's technical considerations It's really hard to do and it might be years away having said that that's where you're going to perhaps realize the greatest amount of Bank for the buck in the future development of these systems So I was wondering if you would address Briefly, what should we think about when we talk about spending more to increase the autonomy of armed UAVs? What are we missing and what should we be focusing on? Thanks? Why don't we just take that one because I have a feeling that's gonna inspire some spirited response who would like to take it first Mike Not gonna say no to talking the I Think how the United States deals with the autonomy question writ large and I'd push that beyond UAVs Is an enormous question and I'd say I say beyond UAVs very explicitly because I think the the underwater realm is What is one that potentially? particularly ripe for The use of autonomous systems, but I think it's important to distinguish between Automation and autonomous weapons and that automation means automating Automating more and more tasks say if a UAV loses satellite connectivity. What is it programmed to do? Whereas autonomous weapons is you know, we all start getting worried about the terminator on the on the battlefield The I think you know as the as the DOD directive on autonomy states the United States I think you know is not it's not deploying any autonomous weapons right now I think the the United States will probably Go slow on these systems pretty deliberately precisely because they raise these very complicated ethical and Technical challenges and that very simple autonomy if you think about something like the Israeli harpy Isn't all that hard to do it's building an autonomous system that can strike a target Precisely with the level of confidence as an even an unmanned system that that I think is extremely Is extremely difficult But I think there are reasons to to imagine that lots of countries around the world may be interested in this and that One could imagine for example autonomy as a potential potentially attractive Place to go for countries concerned about about anti-access area denial environments about a to ad and what do you do in? General in a world where you think you might have to continue fighting even though you've lost connectivity So I think this is going to be up. I think this is the the bleeding edge of where we need to be thinking I think it's important as we there's another distinction that that I think it's important To remember and that is you know when you're talking about Autonomous systems we have some of those a cruise missile rate So we have some you know that go on a one-way mission and just go and then there's those that go and then come back and so and The going and coming back is what's very difficult if you just take a something That's not very high technology you want to crash it into something and surprise somebody or you know have sort of Psychological effect those things you know has belies used something like that you know against the Israeli So I think it's again to make you're not talking about that. I know but just again for purposes of clarity What it is that you're actually meaning? Sort of right here and just one moment Sam will bring your mic her Jim Katke with National Defense University Following up on this Fully autonomous theme here one of the things I find most interesting in this whole area in the private sector is The do-it-yourself community So you're having you know these communities now people sharing plans for different things. They're 3d printing them. They're sharing control systems It's very rapidly evolving. I wonder if I could get your thoughts on potential threat spectrum going ahead from non-state actors using these types of systems terrorists or even lone wolf terrorists And let's take one more right in the front, sir Evan Wallach from GW law school. I teach a course in this stuff um Let me ask dr. Davis. Would you consider it transformative in your area? if you looked at these things from the viewpoint of command responsibility that is eliminating the fog of war and the ability to go in afterwards when you investigate and Determine exactly what the device did and how it was directed to do it. Would you like to start with the response to that? uh We're looking at the this is a you know kind of a first defense Which is that we weren't looking at the specifics but more that general sense of you know Is it like the weapons that we we called to be transformative in the past? And is it a useful thing to talk about these systems as transformative or more like the norm you know kind of weapon systems that you understand and we've dealt with before And therefore having done that we can sort of apply the ways we've done that before to these systems And so I I can't I can say we didn't address that specifically I'm I'm thinking that you think it is by asking the question And if so then give him back the microphone for a second and let him tell me why Sure, um The former Yugoslavia tribunal Worked on imposing command responsibility on civilian leadership because They said well you had the ability to control and investigate now supposing When mili takes place that Lieutenant Kelly can be downloaded and Costa and president johnson and everybody in between Can determine exactly what was done under the treaties certainly under the protocols They have an obligation a criminal obligation to then investigate and punish and if they don't do it It's a war crime. That's what i'm thinking But then i'm not an expert But my colleagues are as to whether that's any different than from some of the other systems we have as well My interest is eliminating the fog of war though that you can't say we Until we get to a regime that Where autonomous vehicles are being employed where they are making their own judgments and revising their actions based on the circumstances around them Somebody programmed Somebody steered it to a particular place somebody told it go here do this That is one of the really confounding things getting back to the autonomy question about When you start to give devices The ability to change their own orders on the fly based on the combat environment they find themselves in Yes Presumably you could still go and download it But somebody along the way programmed the logic that makes the machine do what it does So there it is always traceable back Somewhere How far back you start to get into legal questions that are beyond certainly my competence and i suspect Those of most of the people in the room except perhaps yourself, sir Who would like to take the diy community slash non-state actor question the threat question Only that we've already seen it There was the gentleman arrested for planning to fly uav into the pentagon. I believe It's exactly the example The other the other side too is that if you think Think about the different ways that terrorists could employ or Gain either military effect or psychological effect There are a whole lot of other ways that they would find easier and potentially more effective than this one So you're actually I think and we write through that so you can go find that as well I think it's just important as we think about each of these uses to think about the alternatives that exist to the use in this particular way And we've got time for one last question if somebody has one and they can keep it short Hi, i'm major ryan sims headquarters air force. My question is regarding the um the mtcr again Uh in the past we've seen a similar scenario occur with the development of satellite technology with regards to the us controlling export imposing, you know those types of prohibitions and then it inherently drives a side industry of Other countries like the french for instance who who sort of take pride in the fact that they can develop an itar compliant satellite and sell it to Other countries and so thereby, um, i'm wondering if you are seeing any parallels with regard to this technology Um the proliferation of uas technology across other countries that are taking interest in it Um the united states imposing strict regulations and then thereby inherently creating a side market Which we no longer can control or have any oversight of Great question mike I think the the potential analogy between unmanned systems and in the satellite industry is is potentially harrowing from an american perspective No analogy is ever perfect But and in some ways this gets back to the previous question about cargo aircraft In some ways and that if the united states makes the decision to take itself out of the market That doesn't mean that the demand won't exist And we can be comforted by the fact that right now we're ahead but That market will exist and other countries will work on the technology to fill that vacuum and the profit motive is very strong And so other countries technologies will get significantly better Which in some ways i think is why it's important for the united states to To lean forward a little bit when it comes to unmanned exports right now to lock in our advantage and And in some ways discourage some some other actors for maybe even getting into the game In the first place because we know from the satellite case that if we step back Others are going to get involved and their technology will end up being pretty darn good And we know that from the uas case as well I mean while the united states has exported to britain, italy Looks like france and is considering japan Germany bought israeli The russians bought israeli And the chinese are Evidently willing to sell to just about anyone who wants so The market's out there already for the non-us systems saudi and ua allegedly have bought chinese Final word final word I think the asat is different in the sense of Where it falls in terms of regimes and how you would think about arms control or non-proliferation So I think that's why it's not analogous you are making an analogous as my colleagues are as to How it is that if we don't sell others will will take the market That's a debate that goes well beyond either space or well beyond uavs But to conventional aircraft and the debates we have about what kinds of sales we make on conventional aircraft What kinds of technologies we associate with those who will be in that market if we're not in that market You know, we have a lot of experience about that. We have a lot of processes that go in place to try to make those decisions I've sat there with a pen at times making those decisions. They're not easy decisions But I think that's the route that we should be taking with these systems and thinking about it sort of individually and in that context Thank you very much and thank you. You've been a great audience. Thank you for the mini conference Experiment we've done here two panels back to back and please join me in thanking the panel


Hello my friends! I thought I'd give a little update as to the status of things with our flooded home. It actually looks pretty good on the outside. We did have pressure washing done but the inside, oh, I'll walk through real quickly. But here's the good news. These are the things that I tucked away upstairs before the flood waters came and I am just so excited. Some of my art supplies and I do have our other home is being renovated and I'm very close to being able to put things back into an art studio. It's an old house but it's newly renovated. So look at these prize gyms here that I've got. Yay! There's my precious little pastels again. I'm so excited. I need more greens. I actually have more greens in another box. But anyway, this is just so exciting for me. So there is a rainbow after the storm and the rain. But in case anybody wonders, hi! What a house looks like after a flood. Let me turn on some lights here. Again, we've done nothing but tear out a little bit of flooring and look at that. This is all the yucky flood waters that come in and water came up to about two to three feet in the house. Anything that was left in the house is ruined. Of course, all doors, baseboards, flooring, kitchen cabinets, bathroom cabinets, everything has to be redone. So yeah, it's a mess. But you know what? It's okay. Like I always say, we had too much stuff anyway. But it's a beautiful home when it's restored. Oh, we're not a Chili's restaurant. We had that in our garage. But anyway, yeah, so look at the pool. Don't you want to just take a swim? So we're gonna slowly get things back together here and it's going to be better than it looked before when we get it renovated. So this was the former Monet cafe. And of course everything's moved out. I am gonna steal those track lights to put in my new studio. But this is the little upstairs area I had where I just urgently and very quickly put everything that I put up there to save it. I've got all of my artwork is up there too. But anyway, so these few little pastels that I rescue today are going to the future Monet cafe. And I am just feeling all of your love and your prayers. And I can't wait to get painting again soon. But until then, you guys are blessing me with your beautiful works and your beautiful hearts. Wow, you guys are so helpful for each other. So until then, happy painting.


In today's video we're opening the new Legend of Blue Eyes reprint until we get every four. Legend of Blue Eyes White Dragon is the first set ever released in the Yu-Gi-Oh trading card game. It premiered in English in North America on March 8, 2002. This set contains 126 unique cards that were originally from the Japanese sets Legend of Blue Eyes White Dragon and Phantom God. The most desired cards from this set are as follows. In the super rare slot, Rygeki, a powerful spell card that destroys all monsters on your opponent's side of the field. This card was banned for years and is still usable in the current meta today. Flame Swordsman, a terrible card when it comes to playability, but an iconic monster from the anime used by Joey Wheeler. In the ultra slot, Red Eyes B Dragon, also known as Red Eyes Black Dragon. Another Joey Wheeler card with an entire archetype built around it. Fans of Red Eyes still try to make this card playable to this day. Dark Magician, a card also known for its anime prowess in Yu-Gi's deck, actually has had somewhat competitive decks recently. Even being popular solely because of the anime, the continued support added to the archetype has helped this fan favorite card thrive. Exodia the Forbidden One, a monster made up of five parts, each with its own card. This card wins the duel when each piece is in your hand. The nature of the five pieces being different cards also makes this card very collectible. It is one of the most prominent Yu-Gi-Oh cards of all time, due to its major role in the anime's first episode. And finally, the Blue Eyes White Dragon, the face of Yu-Gi-Oh. This card was used by Kaiba in the anime and is well known for only having four copies, one which was ripped by Kaiba. This powerful engine of destruction is a card everyone wants in their collection, and the LOB version is the first print to release in the TCG. What's up guys, we're back with another epic, Legend of Blue Eyes 25th Anniversary opening. This is Legacy Week 3.0 essentially, because we will be opening these legacy packs for all five sets that just came out until we get all the foils. So, in today's video, start with Legend of Blue Eyes is going to be really fun. Before we actually get into it, we do have a giveaway. I'll be giving away this booster box of Legend of Blue Eyes White Dragon. All you have to do is like the video, be subscribed, turn on notifications, let me know down below which LOB foil you want to see pulled the most, because you're going to see all of them. First pack, let's go. Legend of Blue Eyes White Dragon, 25th Anniversary. Can we pull every foil in this video? That's the goal. We have the 13th grave. Okay, cool stuff. We could have gone for all the commons and rears too, but that's just too much work guys. Too much work. We don't care about them that much. I mean, there's no petite moth in here, so let's go. Pot of green, nice. That's a good rare. Okay, no foils yet. Lots of potential in this video. Probably going to be a longer video if I had to guess, but based on the video yesterday, it seems like the pull rates might be a little bit higher in terms of the Ultras and secrets, so I think the secrets won't be a problem. I think the Ultras are what are going to take a while. That's probably what we're going to end on having like, you know, four Ultras and having every other like foil at that point. Okay, we don't have anything so far. We're starting over. There's a lot of packs to be open today. I do have an entire case of this product if we have to go through it. I think that that should be good enough in polymerization. Our first foil, and now you can see on the screen, boom, we have pulled our first of 22. One out of 22 down. All right, good stuff. Good stuff. LOB 25th anniversary. Very fun to open this up. Tomorrow we will be opening up Metal Raiders in this same way, so if you're excited, let me know down below in the comments. Of course, I'm dragging two super rares already. Two down, 20 to go. We are looking good because there's 22. Two secrets, 10 supers, 10 Ultras. Just clarify. All right, guys, we're crushing it so far. Two supers, we're looking good. Let's see what we can get. We have Dark World Thorns. We have Two Pronged and the Fury Seeking. Okay. Two supers down, no Ultras, no secrets. We got to pull some big LOB cards. Let's go. We have a Stop Defence and a Beaver Warrior. The Beaver Warrior has been pulled up. Fortunately, not a foil. Should have been an Ultra Rare Beaver Warrior. Would have been epic. What can we get in this second pack? Here we are. This is the next pack. We have Gravedigger Ghoul and a Lesser Dragon. No Ultras so far. So I really don't know how long this video will take. With ratios, we might start pulling four of the same Ultras. It could take forever. Oh, the right arm of the forbidden one. Let's go. Our first Ultra Rare, first piece of Exodia has been pulled. Very nice stuff. Looking good. We are three foils in. One Ultra, two supers. Hopefully, we get a few of those triple Ultra boxes like we saw yesterday. Celtic Guardian, another super, four foils down, 18 to go. Feels like we're on a great pace, but the problem is once we start pulling a bunch of foils, we'll start getting doubles. That's where it can really slow down because you pull foil, but it's the wrong one. Teen Angel, we have Numeruco, we have Goblin Seeker Remedy and a Metabat. Meta Knight, Bat Friend. All right, come on. 18 left to go. Can we pull them? We have Valdrum, Beast Fangs, Fusionist, Feet of Reflection, number two. All right, good stuff, good stuff, good stuff, good stuff. Will we be able to pull that Exodia the forbidden one? Dark Magician, Blue Eyes, White Dragon, Red Eyes, Black Dragon. Not right here. Okay, we got a dragon. Who's the lesser dragon? So many epic cards in this set that could potentially be pulled. Let's see if we can get them here. We have Tion, Laser Cannon, Hanehane, Flame Manipulator, good stuff, but not good enough. Let me know in the comments. Are you guys excited for these new releases? It's pretty exciting to be able to open a booster box at L.O.B. because they have not printed booster boxes unless you count Korean boxes, but like these English shaped ones since like 2003, right? So Dark Hole, very nice. Another Super Rare, that's five foils so far, four supers, one Ultra. No secrets yet though, but yeah, they haven't printed boxes in so long. They've reprinted a lot of these packs, but in a booster box, it just feels a little different. Pot of Greed and Druelling Lizard. The Druelling Lizard has been found. Mystical Sheep, Terror of the Terrible, Tyho, and Fusionist. The Fury of Seeking, come on. Give us another Ultra. Give us a Secret Rare. Let's go. Red Eyes, or not Red Eyes, Red Medicine, different, Betty, Peacefangs, Metal Dragon. We have a Soagon, Mammoth Graveyard, the Living Arrow Card. Come on. Larvis, Umi, Mizorizame, Miscal Moon, Flame Ghost, and an Exodia, the Forbidden One. We pulled one of these yesterday, and now we pulled another one. Exodia and the right arm. We're two pieces of Exodia down in our very like first two Ultras. That's pretty impressive. Very good start for the Exodia set. Will we complete Exodia first or the other five Ultras? That's the question. Will we get Blue Eyes or Dark Magician? I don't know. We'll find out. Arm Ninja, we have a 13th Grave. Beautiful. Can we pull something else? Two good Ultras. That's a good start. Yami, we have Terror of the Terrible, Silver Fang, Silver Bow and Arrow. Spirit of the Heart, and a Flame Swordsman. What do we don't have? Do we have six Supers already? No, five. Five Supers. We are halfway down with the Supers. No reprint or a repeat so far. That's nice. Looking good there. LOB. Very nice. We are completing the set as we go. We have Tripwire Beast. We have Man-Eater, Hino Tomasoul. Okay, good stuff. So far, a lot. Seven different foils out of the 22 we need. Ah, there's a repeat. Celtic Guardian. Okay, it's got to start eventually. Eventually, we're going to start pulling the same foils in the Super Ears slot. No secrets yet. No succubus knight, Umi, Vodger's Arms, Arm out. We've got Arm Ninja, and a Fiend Reflection number two. Okay, okay. Fun stuff. But where is the card we are looking for? King Fog, Mystical Sheik number two, Turtle T-Grey, Tyhome, Giant Soldier, Sandstone, Dark Energy, Armored Starfish and One-Eyed Shieldwreck. What are we going to get now? Exodia is the only Ultras we fold. Exodia and Right Arm of the Forbidden Wall. Let's see if we can get another one. We have Carbon Hollow Warrior, Tripwire Beast, LOB Galore. I wish there was a petite moth in here that we could look for, but there's really not a great common for us to search for an LOB. It's kind of weird. Final Flame and Flame Manipulator. There's some cool ones, but nothing that's really lashed on throughout the years on our channel. Umi, we have Mystical Sheik, we have Mesheen, we have Charibbon, and the Furious Sea King. All right, Sea King, he definitely looks different when he was in the Pokemon franchise. So, Heart Armor, we have Stop Defense and a Dark Hole. Okay, that's a repeat. Okay, still at five supers. We've got a couple repeats in a row. Can we get another Old Trove? That is not a repeat. We have Ran Temperatura, Raised Body Heat, Numeruco, Flame Ghost, Man with Graveyard and the Living Arrow Card. Lots of packs have been opened, but yes, we are still far from our goal. Will we get a Blue Eyes, White Dragon, Dark Magician, Forest? Will the packs we have be enough to complete the Hollow Set? We're about to find out. Dissolve Rock, Move, Trap, Silver Bone Arrow, Beast Fangs, Fusionist, Thirteenth Grave. Still many packs to go. Still a lot of chances. We have pulled a total of like 10 foils or something so far. A few repeats as we've gone along. Dragoness. Oh, it's a normal. Gaia the Fierce Knight. First Ultra, that is not an Exodia piece. We'll take that. Very beautiful. Another Legend of Blue Eyes pack. Surprise, surprise. By the way, if you like this LOB opening, don't forget to check out the video where I did every print of LOB ever made. Go check that out. It was really fun. Open two first edition packs on that video. Go check it out. It was good times. We have a Flame Swordsman. That is another repeat. Okay, we've got three repeats so far. We've got five Supers. Three Ultra. Surprise that the Supers are not doing that much better than the Ultra so far. Arm Ninja and a Fiend Reflection Number Three or something. I don't know what number it was. Number Two, not Number Three. Did they ever do Number Three? I guess they did Sasuke Samurai. We have a Tion Flower Wolf and a Drilling Lizard. Cool stuff, but still no secret rare, surprisingly. No secret has come out Monster Egg with Niscal Moon, Masaki, Pot of Greed and a Mountain. We once did, I think, a 100 pack opening of the 2017 versions back in the day. It was like in 20, no, 2020. No, Dragon. We have a Sandstone, but we have never opened. Probably the amount of LOB that we're about to open today. It is about to be the most we've ever opened in one time. We have Hard Armor, Carbonala and Mountain. Where are the Supers, man? The Supers are straight hiding right now. I guess we've pulled eight so far, so we've pulled like two boxes worth. A couple of repeats though. We have a Trihorn Dragon, our first secret rare. Now we just need that Gaia, the Dragon Champion. Trihorn Dragon looks pretty amazing. Nice secret rare pull. All right, that puts us at nine out of 22. We're almost at the halfway point, which I feel like halfway will be a little bit farther in because it might be hard to pull some of these last few, but Rygeki, another one we don't have. That is 10 out of 22. Not a bad start. Six Supers, three Ultras, one secret. Interesting. Those numbers are working out. We have the Silver Fang, Aqua Midor, Beva Warrior. Very cool. Still searching for more. Can we pull more Epic cards? We have Silver Boat, Laser Cannon, Armor Fissure, Leicester Dragon. 10 have been pulled so far. The other seven Ultras are going to be tough. The seven other Ultras, that's going to take a while. We have a Dark Magician. I said it's going to take a while and we pull the legendary Dark Magician. That is beautiful. All right, Dark Magician and Exodia have been pulled, but we need blue eyes and we need red eyes for the big four. All right, beautiful, beautiful pulls. 11 out of 22. That's the halfway point technically in terms of numbers. We have Gravedigger and Immortial Number 2. I'm glad we haven't gotten any repeat Ultras. I'm sure it'll happen at some point. Those are the ones that are really going to hurt later on. Gaia the Dragon Champion. We pulled most secrets. We're at 12 out of 22. Wow, this is going pretty quickly so far, but there's so long way to go. We got 10 more foils we got to get. Gaia the Dragon Champion. Beautiful looking card. Maybe we should give it in Dark Magician a couple more seconds because we didn't spend too long on it. We have Dark Magician. We got Gaia. I just love the secret around these new ones. A little bit of extra time because they are pretty Epic cards, but I'm on a mission right now. We got it. We got to complete the 10 foils or the 10 final foils. Let's see if we can do it. We have the Hard Armor, Dark Fire, and Sandstone. Very cool. Will we get that blue eyes? Will it be the last one we pull? How crazy would that be? I wouldn't even be surprised the blue eyes was the hardest to pull. My hardest, I mean, hardest today. The ratios are all going to be the same. A lot of people will think whatever the last card we pull is the actual hardest, but it's not. It's just how the ratios work out. Arm Ninja. We have the Forest. They didn't short print anything on these. I would be very surprised. We have a Red Temperatura. We have a Celtic Guardian for the third time. A few repeats of that guy. That's okay. It's a cool card. Now, what can we get the rest of the way? We have a Pot of Greed and a Firegrass. Very cool. Still a few packs to go.  I mean, I have more after this, but Final Flame. We have M Warrior Number 2. We're doing whatever it takes to get these 22 foils from Legend of Blue Eyes. Succubus Knight. We have Hard Armor. Charubin. Flame Manipulator. Also pretty cool. Where are you? Exodia pieces. Where are you? Monster Reborn. Oh, Exodia piece. There you are. I guess. There you are. Right leg. So, we have the entire right side of Exodia and the Exodia head so far. Let's keep it going. We're looking good so far. So, we're up to 11, 13. How are the supers not getting pulled? This is interesting. We have Drilling Lazard. This card in the back, I think is bent. I don't know. I couldn't tell. No, maybe not. Maybe we just warped or something. Okay. Supers. Where are you? We're missing four Super Rare still. We have a Swords of Revealing Light. There's a new one. Okay. Seven supers. Wow. We're getting close. 14 out of 22. Okay. I definitely cut this pack before I didn't cut the Blue Eyes White Dragon. Let's find out. He might be upset about that. We have the Man-Eater Bug. Back to Max Supers that we need. Eight Supers. Let's go, Man-Eater Bug. We're looking good now. We're at 15 out of 22. Making serious progress as we go in this video. Can we complete the Super Rare's very soon? Older's would be even better because those are going to be the hardest, I think. We've already completed the Secret Rare's. Hilariously, they're the easiest ones to pull because there's only two of them. And we've already pulled both of them. So, no big deal there. Secrets will probably be the easiest to pull. It's something that a lot of people don't realize about old school is when there's only one or two options or if it's Hobby Box, there's only one. It's one every two to three boxes, but then you look at a Ultra Rare. It's a lot harder than that to pull because there's so many different ones. You only get two per box if you're seeking. So, specific ones can be harder to pull. All right. Let's go. Dissolve Rock with Silver Bell, Laser Cannon Armor, Beaver Warrior. Can we get them? Umi, Remove Trap, Silver Bone Arrow, Stop to Fence and Mountain? Lots of Epic cards, but not the cards we're looking for. Where's that Blue Eyes, White Dragon? The 13th Grave. No, we're missing Moth's Fruityborn, missing Blue Eyes, we're missing two pieces of Exodia. What's the other one? Red Eyes. Red Eyes. Don't forget about you, buddy. We have Carbon Dollar Warrior and Warrior Numerotos. Lots and lots of LOB cards being pulled today. We have Sparks, Arc Magician. Our first repeat, but this is a nice one to get doubles of. Dark Magician is a repeat Ultra Rare. So, not any closer to our goal, but hey, good stuff. Good stuff. Prenzipan, Electro Whip, Tyone, Laser Cannon, Final Flame, Funeral Reflection Number Two. Also good cards, but we need some new Ultra Rears. We have Flame Swordsman, that is a repeat. Couple Flame Swordsman, Three Celtic Guardian, Couple Dark Holes. We got a couple Dark Magicians. Good stuff, but we need better. All right, Blue Eyes, White Dragon. Let's do this. We have the Tear of the Terrible. Dark Hole, another one that's three Dark Holes so far. Give us that Blue Eyes, White Dragon. Of course, it would be one of the ones we're struggling to get. Two Pronged Attack. What are the supers that we need? Mystical Elf? We don't have yet? We pulled that one in a video yesterday. We have not pulled Mystical Elf, and I'm trying to figure out, oh, Trapple. Those are some of the lower end ones, to be honest, which is kind of funny that they're so difficult right now. We have Beaver Warrior, of course, classic. We have been through quite a bit right here. I think once we finish these, it'll be about four boxes have been opened. So we've opened a decent amount so far, and we have lots of go. We got two, seven to go, seven left, seven more. There's the Trapple that we've been looking for. Make that six more that we need. Okay, one more Super. Five more Ultras. No more secrets, and we are done. We are done with the challenge. We have Drilling Lizard, cool stuff. I believe that we can do it. I believe we can do it. We just need some Ultrarairs. Dark Energy. Okay, come on. Last few packs. Give us some luck. Give us one of those Ultras that we need. And Warrior Number Two. We got the Blue Eyes, White Dragon. We're gonna need the Left Arm, the Forbidden One. Right Arm, the Forbidden One. We need Monster Reborn. We need Red Eyes, Black Dragon, Red Eyes, B Dragon. I don't know. Will it be Black Dragon in here or B Dragon? I don't actually know.  Red Eyes, Black Dragon. That's what it'll be. They updated the text on this print. Very cool. The Sixth Ultrarayer has been pulled. That's awesome. We need four more Ultras. One more Super to finish this set. Not bad. Not bad. We're making good progress.  This has been going well. We have a Drilling Lizard, cool stuff. Give us some luck, guys. In the comments, send us. There's definitely a bank card in the back. You can see that. Oh, another Try Order. Okay. Only our third secret pulled so far. It looks like so far we have not had a three Ultra Box, which we, two of our five boxes yesterday were three Ultra Boxes. So I'm interested to see how many we get. Lesser Dragon. So far, I think we've pulled seven Ultras out of almost four boxes. So this'll be eight if we get one right here. Hopefully we'll get two. That'd be great. We have a Miserizame, Flame Ghost, and a Drilling Lizard. Okay. So only a few packs left in this final box, which isn't the final box, but final on the, on the table right now. I'll add more Goblin Cigarimini, Beaver Warrior. It is always in that slot, the Beaver Warrior. Come on. Blue Eyes, White Dragon, Exodia Left Side, and Monster Reborn. That's all we need. Monster Reborn, Left Arm of the Forbidden One, Left Leg of the Forbidden One, and Blue Eyes. Rygeki. I thought it might be the Monster Reborn. Okay. Double Rygeki. That's cool. Always like the Rygeki. Five packs left. Can you bless us with the cards we need? Second best knight. Come on. We have Dragoness. Sands down. Okay. Four packs left until we move on to the next set. Here we go. Or the next amount, not set technically, the next set of packs I should say. Not actual set because I'll do another way. Here we go. Three packs left here. We have a Hainan, a Sogan. Oh no. Tell them that we didn't get one Ultra Box. That'd be rough. Not like this. Come on. Make it happen. Ray and Temperatura. We have Hard Armor. We have the Left Leg of the Forbidden One. Okay. Not a double. We'll take it. That's awesome. We need one more piece of Exodia. One more piece. The Left Arm of the Forbidden One. Then we need the Monster Reborn and the Blue Eyes, and we have completed everything. Wait. Do we get all the Super? No. We need Mystical Elf. We still need Mystical Elf. Okay. Mystical Elf. Make your appearance right now. Here we go. No. Not right now. All right. 18 out of 22 down. We're very close. We need three Ultras. We need one Super. We are done here. We need Mystical Elf being hard to find. That's crazy. Mystical Elf playing hard to get. All right. Let's see if we can find it. We have the Polymerization. That is, you did pull that. I was like, I don't remember that. Yeah. We've only pulled one now, so that's the first repeat. What else can we pull? Will we get more Ultra Rears that we need? Let's go. We have the Dark Gray. Good stuff. Good stuff, but not good enough. Four cards remaining for L.O.B. If you guys enjoy this kind of video and you want to see more like where I try to pull every foil or something in the future, then let me know and we'll try to make it happen. We have Grave Digger. We have Meta Bat. This is kind of the perfect set to do it on because, you know, with the Starlights, we kind of do that except, you know, the Starlights, it takes so many cases to do it. When there's not such a high rarity, it's more possible with these old sets. And hopefully it is possible that we can get all four here or the ones that we don't have at least. We have a Lesser Dragon. Cool. Still have not had a triple Ultra Box yet, though. We got a couple of those yesterday. Hopefully we'll get more Trap Holds. Another repeat. Where is Mystical Elf? The repeats for the most part have been supers, which has been good. I'm sure we'll start running in some Ultra Repeats here soon. If we don't, we get really lucky. We have Flame Manipulator. Lots of L.O.B. open. So if this is our fifth box, that means we're going to be opening over 100 packs already. So this is over a 100 pack L.O.B. opening. Just pretty cool to say. I mean, that's before it's always been difficult, even in 2017, reprints to be able to do that. It's been expensive. So glad that we can do that nowadays, or at least for now, you know, until these become scarce again in a few years. We have Sparks. We have the Dark Magician. A triple on the Dark Magician. That's a good one to triple up on. Dark Magician's really cool. We haven't doubled on anything, but we have tripled on Dark Magician. Very interesting. Let's see if we can get the Left Arm here. Left Arm, Blue Eyes, or Monster Reborn. Still have not been pulled. Let's go. We have the Dark Hole. Still haven't pulled that Mystical Elf either. Where are you at? We need that Elf. We need that Elf. We need some Ultras. We have Ray and Temperatura. We have Fiend Reflection Number Two. A lot of chances left in these boxes. Can we make it happen? We have the Lesser Dragon. It's been soft so far. It's been soft. I mean, it started off hot, but we've definitely slowed down here, and it's getting a little bit harder as you go to get, you know, now we've pulled 18 of the 22, so it's getting really, really hard. Not a lot of progress in the last few minutes. Let's see if we can turn that around very soon. Nope, not there. Still a lot of potential left in these packs. We have two Mildark Ruler, Curse of Dragon, and another double. Can we finally get that Mystical Elf? The Supers, man. I can't believe how hard the Supers have been so far. I figured they'd be really, really easy to get all 10, Fire Grass, but I guess the repeats are really good into this, because we pulled like probably 20 Supers that we just haven't pulled the 10 we need. Another Gaia, so we have two secrets, two of each secret, so we've pulled, and this is our fifth or sixth box, something like that. We've pulled four secrets so far, so interesting to see that. We have Metal Dragon, Mountain. We need another Ultra. We got one Dark Magician since we started this new box or boxes, I should say. We have a Left Arm. There it is. The Left Arm or the Forbidden One. We have officially completed Exodia. Obliterate! Two Ultras left. One Super. I can't believe there's a Super left, and we only have two Ultras left. Gaia, of course, it's Blue Eyes, White Dragon, and then Monster Reborn is a good one as well. But of course, it's Blue Eyes we haven't pulled yet. Come on, Blue Eyes. What if it only comes in 25th anniversary? It doesn't. It doesn't. It'd be nice if we did Man Eater. Another doublet. Three Dark Magician, no Blue Eyes. Interesting. Now, people will watch this video and say, Dark Magician's easy to pull, but it's not actually true. It's not. It's just we're just happening to pull. Doubles to the Dark Magician. Keep that in mind. King Falk. We have the Hanein, the Kumutoka. How have we not gotten a Mystical Elf? That one is shocking to me. Blue Eyes, not surprised. Not surprised. It's just Blue Eyes. Of course, it's going to be the hardest one to pull somehow. We have a Two-Mouth. We have the Drilling Lazzard. So a lot of packs to go, but still a lot we need. We need three foils, Lesser Dragon. Wow. Where are you, Mystical Elf? Where are you? Larvis. Trap hole again. Man, we didn't have that one until our ninth super. Now we have like three of them. Mystical Elf. I believe that you will be next. You can do this. Metabat. Okay. Cool. Cool. How fitting would it be for the Blue Eyes to be the last one we pull? It probably will be. It's going to be the Blue Eyes somehow. Or what? If it's a Mystical Elf, I'm going to be shocked. We can't get a super rare. That'd be pretty insane. Okay, you're ruling Lazzard. Give us some luck. We have A. Oh, this could be it. No, it's the guy that fears nine. Another double. Okay. This is going to start happening. You're going to start getting doubles. It's going to be really hard to get those last few. Okay. So that box had all double ultra. No, wait. No, no, no. That's not right. We had left arm. So the first box had one and one. The second box, it seems like, is going to be one double and hopefully one new one. Maybe. Right. Gecky. Another double. Okay. We just need that Mystical Elf. Just saying. Can't believe we haven't pulled it, to be honest. We have a dark energy. Come on. Where are you? We have a fire grass. Man. This is pretty tough. We had four when we started these next three boxes and so far we've only gotten one new card. Spike Cedra. His set completions can be pretty tough, which makes them kind of fun at the same time. Sparks. We have a Lesser Dragon. Oh my goodness. Lesser Dragon. We need the greater dragon. That's what we need. We need the fire grass because the blue eyes of our dragon would be considered the greater dragon, I would assume. Legend of blue eyes. The legend is that you can never pull a blue eyes of Mystical Elf or a monster reborn. That's the swords of revealing white. Man, the same supers. I want to count how many supers it takes for us to pull a Mystical Elf when we're done. One, two, three. Why am I counting? I started counting like who doing a pack trick. Let's go. Remove trap. We have a Gaia. That's a third secret rare. Okay. Feels like you're getting secret rears pretty often in these. That's five of them and I think we're on like the sixth box, something like that. Red Medicine. We have a giant soldier in the 13th grave. Cool. Not many packs here. Man, are we gonna, how many packs are we gonna have to open and get this Mystical Elf? There's a right leg. More doubles. More doubles. We just need that one Mystical Elf. We need two ultra rears. Can we do it? We have a man-eater bug, another double. Come on. Come on. We get a crimped card. We got a crimped card. Who remembers crimped trihorn dragon? Is this the foil? Tell me it's not. That would actually be kind of cool about crimped for us. Okay. Okay. Nothing in there. That's pretty cool. Bring them back that crimped secret rare from back in the day with Vintage Yu-Gi-Oh. That was pretty crazy. So crimping is not a thing of the past. It is still happening. I've seen it many times in the new sets. Mystical Elf. We're looking for you. Make it happen. Kumutoko. That is not an elf and it's not Mystical. Terror of the Terrible. We have dark gra- Oh, more crimpage. Another crimped card. How many crimp packs are there? Interesting. Crimp city over here. We have Hainhain, Sogan. Still three away. We're eight or 19 out of 22. 19 out of 22. We have a trap hole again. Oh my goodness. Trap hole central out here. Where are you? Bless us. Beaver Warrior, no. Come on, Monster Reborn. Blue Eyes, White Dragon. Mystical Elf. Metabat. I pulled a Mystical Elf yesterday out of one box. No, we can't pull it. We still have a few packs left. Come on. Give us something we need in Warrior Number Two. Okay. Good stuff. Good stuff in Warrior Number Two. Umi, we have a Monster Reborn. Let's go. Of course, the last Ultra Rare is going to be the Blue Eyes, White Dragon. Monster Reborn, number nine on the Ultra Rares. We have nine Ultras and nine Supers. Somehow the Mystical Elf has still been out of mid-bull. I don't understand this. It's a Mystical Issue. That didn't make sense. Doesn't make sense. Mystical Issue? It's a mystery? I don't know. Trihorn? Okay. So I think that we're going to pull a secret almost every single box, it seems like, at this point. We're six out of seven right now. This will be the seventh box, I think. Reaper of the cards. Cool. Come on, baby. We need some luck. Come on. Feet of reflection. I think we should have at least one more Ultra. Hopefully we have a triple Ultra Box or something. That'd be cool. Can we get the Blue Eyes here? If we get the Blue Eyes before the Mystical Elf would be absurd. Either way, this is going to be weird because we're either going to end on the Blue Eyes, of course, or we're going to end on a Super Rare, which would be really surprising. We've had nine Supers for the last three boxes. We have another bent card in the back there. We have a Mountain. Okay. What will we get? We have a Rygeki, Super Rare. Good stuff. Coming down to the end. We opened a lot of packs today. For some reason, I didn't think it would take this long. I don't know. I just felt like it wouldn't take this long, but the more I think about it, the more it makes sense. Ma'am, the graveyard, three packs to go. Here we go. Skull Servant. We have a Dark Energy, two packs left. Can we get that Blue Eyes? Wait, have we pulled all the Ultras here? No! But not the final cards. We haven't pulled a Mystical Elf yet somehow. Okay. That is insane. The last two that we needed were both in that same box.  Let's see. Can it happen? Super Rare. Let's go. Oh, the Mystical Lack! No way! That is insane. The last Super and the last Ultra and the last two packs, plus the Monster Reborn was in that same box. What an ending. We have officially pulled all 22 foils from Legend of Blue Eyes. If you guys enjoyed this video, make sure to subscribe to the channel. And if you like this type of video, there will be four more in a row like this with Metal Raider, Spell Ruler, Pharaoh Servant, and Invasion of Chaos coming up over the next four days. And there'll be a live stream after that. So make sure to subscribe and I'll see you guys then. Shout out to Tomefo Show, Daxer, JT, Cho, Puffins of Doom, Ernesto D'Endam, The Cycle, America Doidster, Supreme Stage 21, An Undertale Show, Ian Musa, Junior Barting, and Mimic Echo. Thank you guys for supporting the channel and I'll see you guys in the next one. Peace.


Hi, my name is Michele Neute and in this video I'll talk about the free tool StatCheck, a spell checker for statistics. I'll talk a little bit about what the problem is that StatCheck is trying to solve, how it works and how you can use it. So let's start with the problem. This snippet of text you see here is a paragraph from a published psychology article. And we notice that most conclusions in psychology and related fields are based on statistical results like the one I highlighted here. So if we read it, it says simple effects analyses within each of the two levels of valence were conducted, revealing a significant main effect of subtype upon the proportion of positive words falsely recalled. And then there is an F test reported. There is something wrong with this result. And we can find that if we recalculate the numbers that are here, because these statistical results have the interesting characteristic that these numbers should be internally consistent. So if I have the degrees of freedom and the test statistic, I can recalculate the p-value. And if I do that, I find that the recalculated p-value is actually 0.06 and not 0.05, which would make this result not significant anymore. Now we find that in psychology, roughly 50% of published papers that report statistics contain at least one inconsistency. And in about one in eight papers, so 12.5%, roughly of psychology papers, there is at least one decision inconsistency around p is 0.05, like the one that we see here in this snippet of text. Well, why do we care so much about this? Well, three main reasons. First off, it can make results uninterpretable. A statistical result is a very important reason or sort of ingredient on what we base our conclusions. And if we can't trust these numbers, can we still trust a conclusion? Furthermore, specifically relevant for meta-analysis as well, we often need these reported statistics to calculate effect sizes. But if we can't trust these numbers, how do we know if we calculate the correct effect size? And finally, this could also be a symptom of underlying problems. Maybe there are additional problems in the underlying data that we cannot even see. So this might be the tip of the iceberg. This is why we developed StatCheck. So StatCheck is effectively a spell checker for statistics. It's an R package and it has a shiny web app as well. So let's first take a look at what StatCheck does. So StatCheck looks in articles for statistical results to see if they are consistent or not. So you can feed StatCheck an article, it will search through the text and it will look for statistical results that look like this. So it needs a test type. So this is a t-test. It needs degrees of freedom if it's relevant for the test type. It needs a test value, 2.20 in this case, and a p-value. StatCheck can recognize t-tests but it can also recognize f-tests, z-tests, correlations, chi-square tests, and especially relevant for meta-analysis again, q-tests for heterogeneity. It takes these ingredients, these degrees of freedom and this test value. It uses them to recalculate the p-value. Now in this case you see that if I feed it the degrees of freedom 28 and the test value 220, it actually recalculates a p-value that is consistent with the one that is reported in the text. So in both cases the p-value is 0.036 and that means that StatCheck will label this result as internally consistent. So in other words these numbers belong together. However it can also happen that the computed p-value does not match the reported one. So in this case it may be because of a simple typo. So instead of 0.036 you accidentally typed 0.046. In this case StatCheck will flag this result as inconsistent and that means that the three reported numbers do not add up so to speak. So they don't belong together. StatCheck distinguishes between inconsistencies and decision inconsistencies. So if the reported p-value is larger than threshold for statistical significance 0.05 but the computed p-value is smaller than 0.05 or the other way around, StatCheck classifies this as a decision inconsistency. There are two main things to note about how well StatCheck performs. First it can only find statistical results that are completely reported and in APA style. APA is a specific reporting style that is commonly used in psychology. Second StatCheck can sometimes make mistakes in classifying results as consistent or not. Luckily the sensitivity and specificity in this classification are very high. So 85% to 100%. It differs a little bit what settings you use. I'll talk about that in a bit. But even though it's high it's not always perfect. So take care in interpreting StatCheck's results. So how can you use it? How can you use StatCheck for your work? Well it depends a little bit on what your goal is. StatCheck can mainly be useful for self checks, for peer review and for research. And for the self checks and the peer review we created an easy to use point and click web app. You can find the web app at statcheck.io. It's effectively it's very straightforward. You upload an article it will give you back a list of statistics it could find and whether or not they are internally consistent. So if you click on the browse button that you see here in your screen as well you can upload the paper and the result will look something like this. So here I uploaded a Word document but you can also upload a PDF or an HTML document with four statistical tests in it. You see the little table below the browse button with the four tests that StatCheck could find. You also see the computed p-value. So which p-value StatCheck looked for or which p-value StatCheck computed. In this case it's the same for every statistical test because it's a fictitional document that I just created. Usually these are different numbers. You can see that in this case two of my results are consistent so that's great. One is an inconsistency and another one is a decision inconsistency. So this is my own manuscript I was checking. I might want to go back to my text and my statistical software to see what went wrong. Now if I do that in this case I already noticed a potential reason for the inconsistency so the third row in our table. So if I look at the full text of where this result was reported I noticed that I said furthermore we performed two additional t-tests just in case. This test was one-tailed and then a test is reported but this one was not. If StatCheck doesn't realize that this highlighted test is one-tailed it expects the p-value that is about twice as high as the one that is reported. But here I'm very explicit in identifying this test as one-tailed so it seems a bit unfair to count that as an inconsistency. So in order to take this into account there is a setting in the web app that you can use to correct for this. So it says try to identify and correct for one-tailed tests. Now if I check this box StatCheck will rerun on this article but now it searches for keywords in the text that indicate that tests might be one-tailed and you can see that in this case this third row in the table is now counted as consistent. Now for details how this works I'll refer you to the frequently asked questions but this is a quick way to resolve any potential causes for errors. Well the web app is mainly used for self-checks and peer review mainly if you just want to upload one article and quickly see what is wrong in it but you can also use StatCheck for research and if you want to do that the R package might be the best choice for you. So StatCheck is an R package it is published on Chrome so you can install it in R. So note that it's slightly different than most of the other R packages in that it also needs a separate program to convert PDFs to text but the details on how to install this and how to go through the step of the installation are explained in the manual. In R you can do additional things as compared to the web app such as checking just a string of text directly. You can check separate articles just as in the web app but you can also check an entire folder of articles. So if we sort of look at the differences between the R package the R package and the web app a first difference is that you can scan a large body of literature in one go. So for instance if you want to learn something about the general inconsistency rates in an entire field. Another thing that you can do in the R package is there is more options there are more options to specify what you consider an inconsistency. So for example you might have different people might differ in their opinion about whether p equals zero is okay to report or not or maybe you want to assume a different alpha level instead of the 0.05 that is often used in psychology. You can also play around with the latest developments in the R package. All the code is freely available on GitHub. Two of the things I've been working on that you might want to check out is a new PDF reader that doesn't require this additional program to install when you want to use the R package and I'm working on additional code to also look for non-APA reported statistics so that we have a little bit more flexibility in the types of statistics it can find. If you want to know more about StatCheck or if you have any questions you can find everything you need on the StatCheck website so statcheck.io. Here you have all the resources in one place you have a frequently asked questions if you look at the top of your screen there's different tabs the home tab looks like the page that we saw before with the browse button and you can upload your document and get the results but the next tab is the frequently asked questions it explains a little bit on how StatCheck what StatCheck does how it works if there are things that are unexpected how what could be a reason for that. It also has a list of additional reading materials so the manual where you can find the R package a scientific paper that we published about this the validity study and the github page with all the code. So thanks for watching this video I hope that this was useful and that StatCheck can be a useful tool in your work as well.