Good evening and welcome to another candidate forum here on channel 17 town meeting television My name is Howard Wood and I'll be the moderator tonight tonight. We're doing Democratic primary candidates for Chittenden County Sheriff and I want to introduce our candidates. This is Michael major and Kevin McLaughlin. I Mispronounced your name earlier, but I think I got it right this time you did so Gentlemen, we're gonna start with an opening statement. I'll start with you Michael if that's okay We'll give you a couple of minutes to basically tell people why you're interested in running for this position And what what qualities you bring? I appreciate that I've been a state employee And working with the sheriff's office for 34 years during that time I was also able to accomplish a couple of degrees I received the associate's degree in criminology through Champlain and also a bachelor's degree in business also through Champlain Also during the years I've worked for other agencies. I've worked for Heineberg police Richmond police I also worked for the Addison County Sheriff's and Franklin County Sheriff's for a term each Since 1992 approximately. I've also been a field training officer for the agency Where I've actually trained the deputies that the sheriff would hire brought him in showed him the Valcor system our computer systems paperwork any kind of Legal documents Pay-per-work, etc. We would have to be done also during that time. I've done some volunteer work. I did 20 years With the kids and cop program special Olympics, I've also done 18 years worth of Boy Scout volunteer work I tried to be in the community as much as I can over the years and that that has definitely Come to fruition I also spent 10 years on the South Bronx and Reptile Board dealing with inmates and and What they call a direct referral which would go to the board instead of somebody actually going to court first on a different charge That we actually come to the board We would discuss it and then we would act as a judge and actually determine What people would have for penalties or any kind of punishment based on their first-time charge? I've also done Work with the federal and state grants. I was the coordinator. I applied for the grant under the sheriff's office since 1995 I've also Been involved in political work. I've done a lot of testifying down at the state house On different committees. I'm currently assigned to the governor's marijuana bill as a committee through the Vermont police association So I've done a little bit of everything in my 34 years and Recently education the experience that I have I think I'm a good candidate for the position of sheriff. Great. Thank you Thank You Michael and Kevin. Let's Why do you want this position? Well, my name is Kevin McLaughlin I'm the current sheriff in Chittany County, and I've been with the sheriff's office now for 45 years And I started out as a deputy and worked my way up all through the ranks and became the chief deputy and then in 1984 80 486 I went to work as a DEA drug task force agent came back and then that year in 1986 I ran for sheriff was successful, and I've been sheriff ever since Basically, I'm gonna give you some of my professional developments that I've done over the years I'm a graduate of the FBI National Academy. I'm a graduate of the National Sheriff's Institute on management University of Southern California, Los Angeles Also graduated of the senior executives in state and local government Kennedy School of government in Cambridge Then with the Vermont police academy and many different functions over the years about 30 years as a training the training coordinators what we do is we actually Call the TAC committee training advisory committee what we do is actually Plan for the training of the police academy for the 16 weeks that they go through DEA Drug schools department of Homeland Security go on and on as sheriff It's very unusual position because you are hired all the staff You also are the human resource director. We have to take care of all the payroll systems The health care systems the retirement systems you prepare plan develop and oversee the state and county and Department budgets We hand to all the contractual services the sheriff of the county is the legal contractor for the department We do all the civil process in the county, which is between four and five thousand people yearly Also oversee all the transport state budgets all the seizures and share of sales that we do in this in the civil side of the house Put together all the rules and regulations standard operating procedures and on and on and on and basically my priorities for the next four Years are to maintain a DEA drug task force agent that I have assigned at the DEA drug task force for the last Spend there since 2006 and he is currently the longest serving task force agent what he is doing He's assigned there to fight the opioid drug issues that we're having in the state and around the country because they go beyond the state of Vermont We have a new governor's highway safety person coming on hopefully in in October and that would be a full-time position He will oversee the grants for all of Chittenden County all the police agencies in that area So this will be a first for us Continue good training for deputies and staff One of my missions in the next four years is to support the video arraignment system That has kind of gone in and out of the court system and to continue all the programs that we've developed Over the years school resource officer officer child support. We contract services for their Investigative unit civil process Department of Health to present the children and families patrols in three towns Court security and drug tape back stuff. So that's pretty much What I've accomplished over the last 31 years and I want to continue to do that for the next four I think we're lucky to have to such Accomplished people going after this can I ask a question? I'll start with you Kevin a little bit about For those people that don't realize that we have a county sheriff's department as opposed to the local How do you see your role or you could have helped us to explain to the viewer? What the sheriff does or the sheriff's department does as opposed to the local police department? Well, there's three layers of government state county and local We contract for services for a lot of towns that do not have police departments can't afford a full-time police department So what they like to do is they'll contract with with the sheriff's in Vermont Whether it's for five hours a week or 50 hours a week. We currently can contract with Westford, Jericho and Underhill We do that. So the people out there probably see a lot more of the law enforcement deputies in market units doing traffic stops things like that The prisoner transportation we do that all over the state of Vermont because we have the only female jail in the state of Vermont here and the only Juno unit so many times when they call us to transport We may have to take somebody from South Burlington to Bennington or Brattleboro Newport So they go all over the state. So they're always moving and in the civil division the the deputies that work there They work in plain clothes and plain Plain cars and they serve civil process throughout the whole county and what they do is those are summons that complaints Court orders seizures assets. We do share of sales from time to time We seize as it because of the court order. So this is kind of a thing that we're all over the place People see on traffic control. You'll see the officers out in traffic control when we have those type of contracts also That's great Michael. You mentioned that you were part of the governor's Kind of task force on marijuana Tell me a little bit about your involvement in that and what your views are and how do you think that's going to impact? Chittenden County in the future where you think that's going to affect when when you become sheriff Well, I was part of the legislative board for the Vermont police association And part of the governor's request is that they have a Member of the association on the actual board So I've been on it for a year. It's going to continue for at least another year and we've done a lot of Research we even had a couple of experts come up or actually do it by By email or by computer from colorado You know these people have 30 and 40 years plus And they have a state experience absolutely and absolutely and they're studying it, you know day to day So we're taking that information And as the board And looking at it, but we're also looking at it from different views We have the taxation part of it. We have the medical side of it and then we also have the public safety side of it I'm on the public safety side so we meet Currently we've met, you know four times and then we meet At individual units and then we meet together as an accumulative unit to discuss all the issues And then we end up making a report to the governor, which we did in january So we have another report that has to be done by december 18th Um, and we're still doing, you know more research as you know Legalization of marijuana is a huge topic and colorado is actually finding out some of the issues that it's causing They're having more homeless move into the area People are actually, you know mailing marijuana through the postal service Which is you know, having problems for the postal service and And in chinti county, we're going to see Now that it's legalized, we're going to see a lot more issues. I think Basically with the thc levels being uncontrollable as far as regulations The actual marijuana thc levels. Yeah, it's been Homegrown so there's no regulation on the quantity Right So that's going to increase issues and problems And then you end up with car accidents, you know domestics Property damage That's all going to cause issues for the local law enforcement And and we really haven't seen all the issue that it's going to cause yet I think it's going to get worse before it gets better We've already seen one issue where they're giving away free marijuana with the sale of a t-shirt You know, that's come on really you give them five feet. They take a mile So i'm not too sure or where it's going to end up in six months or a year But the board is working hard. We're trying to get the right information to the governor That's great. Kevin, you must have you know, pretty clear or some ideas on where this is going for As well, what are your thoughts? I have many friends in colorado who are sheriffs and and when I go to the national sheriffs conferences We meet we're on the i'm on the drug enforcement committee of the national sheriffs association I've been there for almost three years and basically we talk about that We talk about the drug issues nationwide where it's coming from where it's going what's happening And the sheriffs in colorado tell me to tell the sheriffs in vermont get ready. It's coming your way And I think one of the questions is, you know, does it complicate your job in any way? I think it's way too early for that But I think the complications would be on drug while driving. I mean, is there any testing for that? We haven't got that yet And I think that that would be the first issue to come and we would have to get more dres Which is a drug recognition expert That's a police officer who gets a elaborate training in this area where he can actually See certain characteristics that occur that a judge Judge would accept as a dy kind of thing. So these are issues that I think we're just at the beginning of Uh And you can just look at other states to see where they were day one week one year one year five that kind of thing And they say it's pretty going to be pretty much the same Go ahead. I the sheriffs thing about the dres The process for the dres is pretty lengthy. It's pretty involved They're finding out that Certain areas of vermont have more dres than others like orleans county. They might have one So the response time for that dre is longer and anything over an hour They're not responding based on the time and the purse. It's a level of intoxication What some of the other issues they're finding is that officers are not applying To be dres Well, like I said, it's a hard course you have to apply And if and once you're Rejected as such, you know, some people just don't want to go back Plus with people retiring Like state police have a lot of people retired this year. So some of them that are dres are taking that training with them, obviously That's a big issue. Also, we don't really have good field testing equipment To give to the officers in the field So you're sitting there wasting an hour hour and a half waiting for a dre to come over Then the dre easily takes two to three hours to do the processing. So it's not it's not short sweet and quick It really takes a long time to get the process going And because there's so few They're called out more often and think about it if you're the only person within 50 to 100 miles On a weekend and this is going to be an issue. How many times you're going to get called out And after a while they go this is this is enough. You know, what's the answer to this more dres and better Testing equipment and until we get the testing equipment the dres is Yeah, pretty much yet. So what just What is the what is the process when a dre You have someone and you suspect something what happens then? Well, the dre will show up and they'll just literally take you over the stop They'll just start talking to the person Sometimes they'll take them back to their office and there's a whole sheet of questions that have to be asked and answered first And then based on their scientific knowledge of the drugs They'll go into like physical testing looking at the eyes of the sort of speech But the process like I said, it's it's extremely long It's really tying up officers on the road and then depending what happens in court if the case gets dismissed A lot of that time is wasted. They really don't want to show that an officer did something wrong Because it goes against the tar program But like the state is tasked with buying all the equipment that money has to come from somewhere Well, what about, you know in terms of Not just marijuana, but in terms of things like opioids and various things It's the same it's the same thing. It's the drugs. I mean certain things with your eyes and and they're all they're all They're all trained into doing that. So it's very similar the dre officer Has trained in all of those different aspects of those things. That's interesting, you know And so there's appears from both of you that they're we have a real need for Many more officers that are trained in that as well as the equipment that goes with it Where does that where does that funding come from working? Well, the funding would come from most likely the state of Vermont and Either through governor's highway safety or federal grants most kind of things Not less the state department of public safety can get it in the grant for them And then deal it out through other agencies throughout the state. Do you see that coming in the future? Is that Well, the site I mean the uh, the state lab has some equipment But as we develop with more dearies It takes more equipment to do the same jobs because of the uh, the volume Yeah, but there So that obviously the opioid thing and other drugs, you know besides marijuana seems to be on the increase anyway All of the all of the law enforcement agencies seem to be dealing with it, you know Um, where does the sheriff? How do you see the sheriff's role in that type of thing? Well, as the sheriff said, we have an undercover DEA officer, but in talking with him With the opiate issues that we currently have in the legalization of the marijuana taking a back seat Now it's going to be even further back The bigger issue is the opiates, obviously, but with the legalization it's going to make it a lot harder What do you think? Well, I I agree that it's going to be a lot harder and at the same time I think that more people are going to be carrying it Because you can carry up to a certain amount and not getting into trouble if you don't have any problems And because more people are carrying it I think what worries me the most is that it's going to be easier to say Oh, you don't have any here.  I'm coming over. I worry about all that Well, let's veer a little bit into the whole notion about the you know the gun control kind of legislation I'll start with you Michael a little bit about what your view on the recent, you know, legislation Legislators just passed, you know Gun control legislator legislation What are your thoughts on those things and how is that going to impact your your viewpoints in the sheriff's department? well as far as Like high-capacity magazines, obviously law enforcement is exempt I don't really Personally, I don't see how the capacity of the magazine will save lives There's equipment out there that you can carry four and five 10-round magazines in your gun you just swap them, you know left to right up and down So the number round is still going to be there along with pouches that you can carry on your on your waist So I'm not sure how that's going to save lives The other aspects just like somebody being under 20 and one without taking hundred safety I'm actually for that I've made sure I have five kids all five of my kids have taken the hundred safety course Before they were able to look at a gun buy a gun or even go hunting which is law anyways But to me the education part of it to me. It's not a punishment. I think everybody should know about weapons I think by training them they'll hopefully respect them a little bit better You won't have accidental discharging In you know, somebody just walking into a store and buying the gun and walking walking out Not knowing anything about it. That's just not good business Yeah Well As a law enforcement officer in the state of ramaa, I mandated to And sworn to uphold the laws of the state of ramaa. These are now laws Some of these laws will affect a law enforcement officer Like the extreme. I think it's called the red flag one Where law enforcement officer if a court orders it could go to a seize a weapon From somebody who it shouldn't have it for a year and those kind of things So that that would affect law enforcement that way and that's my job is to enforce that I think some of the other things that are law probably law enforcement won't get involved in But we as sheriffs could get involved in when I look at the law the way it's changed if somebody has the Stuff that they're not supposed to have And it gets that way where maybe it's a civil case if somebody takes the civil case to court And all of a sudden there's an order to go seize it so we as sheriffs may get involved in that also So it's it's you don't know how it's going to turn out But my opinion is Is is uh, it's a statute I'm mandated to do it and I can't take sides either way because if I do Something's going to say he's biased against or he's biased for so we have to remain I have to remain neutral on that This may get me in trouble for asking this question and you feel free not to answer But is there a need from a law enforcement? aspect for More or more Gun control at all. Do you think? Well, as you know, vermont is one of the safest states in the union. We have had very few if any gun laws Um, I think what happened recently is an eye opener Where that young man and In the down south in the school district down there where he was saying that he wanted to um Hurt and kill more people than the virginia tech did that that's an eye opener It's like wait a minute. This is vermont. This is not supposed to happen here So when you step back and take a look at that saying, oh my god, this could happen here, too So it's an eye opener. I think that we all have to pay attention to it We have the mental illness thing going around where we have that that is really causing a lot of issues. We need more Uh help in that area and police are getting it to some degree With people working, but the mental health professionals really have to step up and take over that I'm not trying to try to anybody's second amendment rights of those things. I just wonder from a, you know, You know a law enforcement perspective what you think is if there's a need for more or something What do you think michael? Um, I think uh, you know hunter as well and you know, so I think uh, it has to be we revisited Um, I think some some of the decisions were too quick Um, I really think we need to back up a little bit Um, like the sheriff said, you know When you're talking about the mentally ill have access to guns, you really don't know what's going to happen How do you prevent that? There has to be something out there we can do You know better background checks Who's really going to apply for a background check probably the honest person You're not going to have the bad guy or the mentally ill person going to a gun shop and applying for a gun They're going to get it from our friend Black market. It's just not going to happen legally What about the 3d gun stuff? That's you know, they're talking about you know, that seems like that's going to be a whole another issue That we'll get into another time. I read I read uh, I read a Story not too long ago and they were talking about problems facing vermont and they did a study And the first one with 27 was the economy jobs and the cost of living 19 was drugs in the opiate 11 was taxes and almost way down the bottom 2 guns in marijuana But once once again guns in marijuana Haven't really surfaced in vermont yet. And if that does happen How those are going to be 27 percent 19 five six years from now, right? Something to watch and be careful about what do you think that issues or If you and when you get elected What are what are the things that you want to bring to the table and and really focus on when You know day one My priority wise is to maintain a financial stability for the office By hiring more deputies Going out into the community looking for contracts More outside yearly contracts. We only have a couple right now And we depend a lot on part-time officers By having the full-time contracts full-time patrol like the sheriff said We can increase our staffing and by doing that Having more staffing and do more contracts as far as profitable type contracts Bringing in the money to pay off, you know, the debts re-accumulate Because we are self-funded There are only certain positions in the office that are paid by the state of mind the sheriff's position Mine as the state deputy along with five other officers The two secretary staff are also paid by county But everything we do is self-funded In order to maintain that financial stability we have to have officers on the road And work it Kevin I agree with that and like I said the some of the issues you have is you have your good years You have your bad years and you have to plan for those in over the years I I saw a huge thing happening around 2008 I saw our civil division going way up way up in demand for services civil process or whatever And and I look back during that time and basically it was a credit card companies They were coming after you when you were 500 dollars behind not 12,000 and they were coming That's when the economy collapsed back then and now in the last 12 months We've seen a huge decrease in civil process the economy is good People are paying their bills so our civil process is going down So it's an up and down ride and you have to as sheriff plan for this because You have people working for you on a full-time basis And you have people who work for your part time and they and they like to have the 10 to 12 hours a week extra And so as your as your years go up and down You have to plan for this because of the fact that we don't get funding for motor vehicles We have 25 cars imagine what that costs just right there And then uniforms for all the staff are not supplied by the county or state and those kind of things So as I look forward to it as forward in in in the contract world Yes, but at the same time law enforcement generally nationwide in vermont We're having problems finding qualified candidates Because you read in the paper every day what's going on and you know, who wants to be a cop anymore? I mean, that's an issue that we're seeing and no in the candidates you get aren't the ones that maybe would have wanted five years ago So these are the issues that you have to to look at and You know the millennial generation is a little different from the instant gratification generation from this generation It's like well, I have to work weekends. Well, I have to work nights. I mean Sometimes that happens and we're down to just a few minutes So I want to give each of you a minute or so to make a final statement Michael will start with you if you can Just you know, let let the viewers know You know, let the voters know and by the way, I want to make sure that everyone realizes go out and vote but Michael um Michael major what is give us a minute or so of your statement there. Thank you over my 34 years of service I've worked and prepared myself for this day The sheriff Obviously has been in his position for 34 years But over my time, I've been able to increase my education and knowledge of the sheriff's job and responsibilities I've testified at the state house on committees. I've increased my knowledge of the political field I've been unlike I said the the special legislative committee for the governor I was also on the law enforcement advisory board of the chair The chair of the even more police administration 2015 currently the treasurer now But with all the assignment I've done over the years the details the education the training I feel that all that combined Really shows that I'm ready to take over the slot of sheriff Great. Thank you. Yeah. Thank you. Kevin Well in my 45 years In the chinning county sheriff's office and 31 years as sheriff on all those committees that we were just discussing I've served on those and actually I think in many of them. I was on the original Founding members like to coosie in a chinning unit for special investigations I was one of the founding members 31 years ago when the chiefs got together and said we need To have a unit specialized in these kind of things because we're not getting good good cases because they're so few and far between Um Having transformed My staff in 1987 to around from around three or four full-time positions at 21 positions today over the years modernizing the agency Working with all of the law enforcement agencies in chinning county and around the state of vermont There's so many things that have happened in my career that are so successful that I love this job Like uh, I was I was chosen to be a grant reviewer by president clinton's staff On uh americorps. I went to washington and actually did grant reviews grant facilitating Back in the days when brady bill. We were established as the only sheriff's office in the united states of america that actually did brady law checks Gun checks for the whole state of vermont So those are kind of things that I've been involved in I was on the national sheriff's conference review committee Which assisted in writing the manual for court security and transportation and prisoners that was a national study This was for all sheriffs around the nation. I was on that committee And chosen to be on that committee based on my experience in in my in my office tenure as sheriff So i'm Looking forward to it. I'm also looking forward to the new governor's highway safety program We're going to implement this year. Hopefully on october 1st waiting for a few things to Fill in and basically i'm running for reelection and I um asked for your vote and Well, I thank you for your continued support Thank you to both of you. This has been a great conversation. We did not have enough time We were just kind of touching The issues here, but I can tell that we are blessed to have two great people running for On this side and i'm i want to urge our Our viewers if you want to see this again, you can catch us on cctv dot Um on on the internet on cctv tv dot org


So you have awesome animations but they are in different files, not a problem. In the next 20 seconds you will have all of your favorite animations together in a single file. Alright, let's do this. Now in object mode select your rig and go to pose mode. Select the bone in the armature then drag select all the bones. Now in the timeline left click the keyframe you want to copy or you can drag select multiple keyframes if you want to move more than one. Make sure that all of the frames and keys that you want to move are highlighted in yellow, otherwise they will not be copied. Right click to copy frames or control C. Go to file, open the file that you want to move these keyframes to. Now in object mode select your rig and go to pose mode. Scroll up and down on the mouse and middle click to get a better view of the timeline. Then drag select all the bones, scrub to the frame that you want to add them to and select paste frames or control V. And now you can see the keyframes from the last file have been transferred to this one. Quick note you can't copy and paste from two separately opened blender files. They have to be opened within the same running blender. Alright, that should be it. Hope that helps. Thank you so much for watching. If you enjoyed this video please don't forget to like and subscribe. Hope you have a fantastic day and I'll see you around.


All right, let's get this started. My name is Ryan Davis. I am known pretty much everywhere as Nspider, except for Twitter, as you can see. I am an independent consultant in Seattle, and I'm a founding member of SeattleRB, which is the first and oldest Ruby group in the world. So setting some expectations. This is an introductory talk. It's very little code in it. I'm not going to teach testing or TDD in this talk. I'm going to be talking about the what and the why, not so much the how. I have 218 slides, which puts me at just under five and a half slides a minute, so I have to go kind of fast. So let's get started. The simplest thing that we can ask is, what is Minitest? Minitest was originally an experiment to see if I could get Test Unit replaced for about 50 of my projects that I had at the time. I'm up to about 100 now, with as little code as possible. And I got that to happen in about 90 lines of code. It's currently available as a gem. We didn't have Ruby gems back then, when it was originally written. It now ships with Ruby as of 1.9.1 and up. It's meant to be small, clean, and very fast. It's now about 1,600 lines of code, which sounds like a really big increase over 90, but still, that's very small. It supports unit style, spec style, benchmark style, very basic mocking and stubbing, has a very flexible plugin system, and et cetera, as we'll see. There are six main parts of Minitest. The runner, which really kind of is nebulous now. Minitest, that should say Test, which is the TDD API, Minitest spec, which is the BDD API, Minitest mock, Pride, and Bench. I'm only gonna be talking about two parts, Minitest Test and Minitest Spec. So let's jump into Minitest Test, which is the unit testing side of things. So test cases are simple classes that subclass Minitest test, or another test case, and tests are methods that start with test, and they make assertions about your code. It's just classes, and methods, and method calls all the way down. Everything is as straightforward as you get, and it is magic free. That slide is two years old, so this wasn't my only barbit null. So Minitest Test includes the usual assertions that you would expect from XUnit, plus several added beyond what XUnit and TestUnit usually provide, methods marked with a plus are new to Minitest, and methods marked with a star do not have negative or reciprocals, as we'll see in a sec. Unlike TestUnit, Minitest provides a lot more negative assertions. It doesn't provide some that you might expect, which I'll go into later, but one of the questions is like, why are there so many pluses? I really want my code tests included to communicate to me and other readers as clearly as possible. Not only does the code communicate better, but the error messages are much more customized, so when there is something wrong, I get better information about it. And finally, AssertEqual is enhanced to do intelligent diff mode. It allows you to see what's actually changed instead of a huge blob on the left side and a huge blob on the right. You get to actually see what's different between the two. But I said that I would describe why some negative assertions are missing. This is something that I hear a lot more than I'd actually like to hear. The question of where's refute raises or assert not raised? It's the same place as a refute silent. So let's look at that. Let's highlight the key components of refute silent. So refute silent says that this block of code must print something. What it is, I don't care. That is a valueless assertion. What you should be asserting for is a specific output that you want. In the same vein, refute raises says that this block of code must do something. What it is, I don't care. It's a valueless assertion again. Instead, you should be asserting for the specific result or side effect that you actually intend. I've heard the argument, but it's useful. No, it isn't. It implies side effects and or return values have already been checked or aren't important, which is always false because you wouldn't be writing code otherwise. It falsely bumps your code coverage metrics and it gives you a false sense of security. I'm an ex-lifeguard. I have a lifeguard in my high school days at various lakes in my county. And one of the things we really feared were parents with kids with water wings because the parents think they've got flotation devices and we'll just talk to our friends and ignore our kids and watch them drown like this. So in other words, this only makes it look like something has been tested when in fact it hasn't had any tested applied to it at all. I've heard it's more expressive. No, it's not. Writing the test itself was the act of expression. It's an explicit contract in every single test framework out there that any unhandled exception is an error by definition. The test mere existence states there are no unhandled exceptions via these pathways. And I've been having these arguments for years. In fact, I had this argument last month and I know that some people will never be convinced and honestly, that's okay. You can't win them all. But it doesn't mean I can't try. So hold on your hats, stand back. I'm gonna try one more time to convince all of you. It's like if you call it, it must be okay, right? So I wrote all these extra assertions to verify that everything is okay. And if you'd like to license this code, come please see me after this talk. If only that were possible. Our jobs would be so much easier. Next up is MaytestSpec, the example testing side. In short, where MiniTestTest is a testing API, MaytestSpec is a testing DSL. Instead of defining classes and methods, you use a DSL to declare your examples.   Here's an example that is equivalent to the previous one. So we have describe instead of class, we have it instead of def test. But in reality, describe blocks really are classes and it blocks really are methods. This same example one to one transforms into this code where describe makes a class, it makes a method, there is no magic. All of your normal design code tools exist and they work as normal, which is really, really important. It means that include works, def works, everything is as expect, as you expect, you're just using a slightly different language. Similar to MaytestTest, MaytestSpec has many expectations defined and a similar set of negative expectations in the similar set of missing culprits. And all of this is gained for free because each one maps from expectation to assertion directly. Underneath MaytestTest and MaytestSpec is the infrastructure to run your tests. And it does so in a way that helps promote more advanced and robust testing. Maytest has randomization baked in and has always been on by default. It helps prevent test order dependencies and keep your tests robust and working standalone. By rule, every single one of your tests down to the lowest level should be able to run by itself and pass. If it requires another test to run before it, it's not standalone, it's not a unit and it's wrong, it's buggy. And as far as I know, Maytest was the first test framework to have randomized run order. There's an opt-in system that lets you promote a test case to be parallelized during the run and that takes randomization to a whole other level. It ensures thread safety in your libraries and absolute robustness. Cause if it can handle the parallelization, it can handle anything. The original Maytest was a tiny 90 lines of code and over time features have been added that you get to choose to help you enhance your tests. It's still really small in comparison but it's incredibly powerful. So what was my reasoning for Maytest design? Maytest is not special in any way, shape, or form. All of my usual tropes apply. If you've heard me up here, ranting and raving, Maytest is no different. First and foremost, it's just Ruby. It's classes and methods and method calls. Everything is as straightforward as you can get. I believe that less is more. If I can do something in less code, I will, absolutely. Method dispatch is always gonna be the slowest thing in Ruby. More importantly, less code is almost always more understandable than more code. So let's take a look. Here is assert in Delta, which is the equivalent of assert equal but for floats. Never ever use assert equal on floats and never use floats for money. There, I've done my usual caveats. It's as simple as possible with minor optimizations. In this case, we use a block to delay the error message rendering. In the case that we don't have an error, we shouldn't have that cost. And really, this just boils up on up to assert. So you really only need to know about 15 other lines of code to understand how this works as a whole. Indirection is the enemy. I want errors to happen as close to the real code as possible. I don't want things delayed. I don't want layers of indirection in between. I kind of feel like Noel is making my point because he just kept talking about the layers of indirection that our spec adds over and over. I want to strip all of those out as many as possible at least. I want the responsibility to lie in the right place. No managers necessary, no coordination going on. I want objects to be responsible for their own duties. This may not be the best example. I don't know how to get a good example of indirection as the enemy because it's rather hard to show what you don't do. Must equal is an expectation that directly calls assert equal on the current test context and then assert equal is three lines long if I remember right and that's it. No magic allowed. Even test discovery avoids object space. It has minimal method programming in it and it uses plain classes and methods to do all of its work. I originally wrote Minitest in part to see if I could because I was the maintainer of test unit at the time and test unit terrified me. But I also wrote it because I was working on Rubinius and I wanted Rubinius and JRuby and other implementations of Ruby that hadn't finished being a full Ruby to have the simplest implementation of a test framework possible so they could get feedback quickly. And finally, it has a thriving plugin ecosystem. I designed Minitest to be extensible so that Minitest itself could remain minimal. Here are just a small amount of the popular plugins for Minitest. Okay, so what does this have to do with Rails? Well, the official Rails stack uses Minitest. Each release, it peels back the testing on in encouraging better testing practices except that peeling onions makes you cry, right? Hopefully not in the case of Minitest. So Rails 4.0 was the first version to cut over from test units to Minitest and they did so on the Minitest 4.x line. At the time, I had already declared that I wasn't going to keep updating standard lib Minitest to Minitest 5. I was only gonna maintain it at level version 4 and that's because test unit is built on top of Minitest and has a lot of hooks into the internals and it just made it really hard for me to ever update and not break their stuff. So I put a freeze on that. So Rails updated to Minitest 4 to remove a layer of complexity and indirection but also to allow them a migration path to Minitest 5. Because test unit was already wrapping Minitest, there was basically no impact on anyone. Arguably there may have even been a almost imperceptible performance improvement but I'm not gonna claim that there was one. Rails 4.1 switched to the Minitest 5 line. This was painful for Rails itself because of crufty tests and the number of monkey patches that they had on Minitest itself. This got Rails onto the newer code base, the active development line of Minitest and it made things easier to do like exec based isolation tests. There's a number of tests in Rails or each test actually forks a process to run a separate Rails app by itself so that if they're running in parallel they're not gonna infect each other. As painful as it might have been to get Rails switched to it in passing, you hopefully never noticed them. However, Rails 4.2 turned off the alphabetical order of the testing and to remove a monkey patch and that started to run tests in random order. This is solely to improve the quality of Rails's and your tests but it may have had some impact on some people. We did get some reports that after updating people started having tests that previously passed failed. I had to isolate a number of test bugs in Rails itself because of this and I'm gonna say honestly, despite the pain that it causes, this is a good thing. Test order dependency bugs are problematic and they're incredibly hard to track down. I'm gonna talk later about a tool that can help identify these bugs and hopefully future versions of Rails, they should be tracking many tests. Aaron Patterson and I keep those things in sync and he lets me know when things are coming down the pipe. So as a Rails dev, what does all of this mean? Hopefully if I've done my job right, means nothing. You shouldn't even have to see many tests most of the time unless you wanna enhance it with some plugins and that's because your subclassing Rails's test cases like active support test case or action controller test case, there's about six of them if I remember right, there might be more now. And the basic architecture looks something like this. You write your own test class that subclasses active support test case and active support test case subclasses may test test.  Aaron and I wound up adding before and after set up and tear down hooks to make it easy for Rails and other libraries or frameworks to extend many tests to do extra wrappings that they needed to do. It provides things like fixtures to load test data, declarative forms if you like that instead. If you don't like those, you can just use def. And it provides extra assertions like assert difference, assert valid keys, assert deprecated and assert nothing raised. Wait, what? Don't worry, this is the actual implementation of assert nothing raised. It's only there for compatibility's sake and personally I think that it should be deprecated and removed, maybe that should be for Rails 5. And all this means that you can write simple tests that describe your current task, things like database transactions don't have to clutter your test code and you can focus on what the test is really trying to do. This is a very simple example and I think all of these test examples come from the testing section of the Rails guide online. Action controller test case is another Rails extension except that it subclasses active support test case so you get all of those goodies layered on. And it extends it to include more like all of your usual HTTP verbs, simulate web server state, and provide assertions specific to handling requests, which lets you easily write clean functional tests like the following. I love air conditioning, I'm so dry right now. I'm just gonna leave the lid off. Active dispatch integration test provides full controller to controller integration tests. As usual it subclasses active support test case so all of the usual stuff is there. It provides a ton of assertions that I cannot fit onto one slide and allows you to write comprehensive integration tests that span multiple controllers. If you want more details about all the stuff that Rails adds on top of many tests you can get that here and it's described in pretty good detail. It's actually a really good read. So Rails' approach to subclass main test test leads to a very simple setup that remains very powerful providing you with everything you need for unit tests all the way up to integrations. It leverages main test power, providing randomization, optional parallelization to provide better testing, make your tests that you write better and more robust over time. But what happens when something goes wrong? Perhaps you want to use spec style. Turns out the DHH disapproves of our spec so much that he wouldn't allow us to switch the test framework in Rails to main test spec. He reverted that commit. As I understand it he simply doesn't want people to submit patches using spec style so he didn't want it easily available. But that doesn't mean that you're stuck. If you prefer spec style that's totally fine. You can use Mike Moore's main test Rails or Ken Collins' main test spec Rails to do varying degrees of this type of code. The two libraries are not the same. They suggest slightly different styles. One tracks our specs style a bit more than the other. Or perhaps you upgraded to Rails 4.2 and now you have failures.  I'm sorry. Unfortunately that's not as simple to deal with. It is a bit harder but again, it is a good thing to catch and fix this sooner rather than later. Why are my buttons not working? It is a test order dependency bug. And quite simply, that simply means that tests will pass when they're run in a particular order. Say A before B but not when B runs before A. And if it was only three tests that wouldn't be a problem. It would be pretty easy to find and fix. But you probably have hundreds and hundreds of tests. That's not so easy until a few months ago. I wrote MiniTest bisect to try to isolate the problems that we were having getting Rails onto MiniTest randomization. It helps you isolate and debug random test failures. In short, it intelligently runs and reruns your tests, whittling it down until the culprits are minimized. And here we're gonna see a simple example of it running. Ignore the fact that I'm pre-specifying the seed. Pretend that I'm running this anew and we get this failure. We get the failure. We grab the random seed and we rerun the test using MiniTest bisect instead. That's gonna rerun the entire test suite to ensure that it is reproducible. And once it is, it starts to bisect the culprits space down to the minimal subset. And you can see that it speeds up dramatically as it goes on. Getting it down to two tests run in a particular order that will cause the repro every time. I see I'm not alone with this problem. Or maybe you're just used to kitchen sink development. For starters, just try it. It might work. Things like Mocha and a lot of testing libraries are ready to work in MiniTest and they're just fine. Otherwise, you're not alone. And someone probably beats you to it. So look for the existing plugins that are listed in MiniTest. Read me or search on rubydems.org or Stack Overflow or whatever. But my suggestion, and I know this is going against the flow here, try less complicated testing. Only bring in plugins once you've decided that you really, really need them. Otherwise, start fresh and clean. Try it. You might like it. But the problem is that change takes time. Just remember that you might want to measure the before and after in order to be more objective about this change. I've only got anecdotes of projects speeding up when they switch to MiniTest. I would love more data and if you would submit that to me, that'd be great. I have heard that people have halved their test times by switching from R-Spec to MiniTest, but again, I don't have anything objective. So, all of this MiniTest stuff sounds interesting, but why should I bother? The first argument is this. I'm not gonna bother with that.  There's plenty of data showing that the benefits of testing, and if you can't get past that, I'd rather help other people. I'm gonna pick my battles, thanks. Like this one. This is a battle I wanna pick every time. Obviously, not everyone uses it. The official rail stack uses it, which means DHH uses it. Tenderlove uses it. Jeff Kazemir and his cohort teach MiniTest at the Touring School. Nokogiri, Hamelgod, New Relic, SQLite, and a bunch of other very popular gems use it. In fact, there's more than 4,000 gems that declare dependencies on MiniTest, and unfortunately, because MiniTest ships in StandardLib, there are plenty of gems that don't declare their dependency on it. So I'm sure there's plenty more. So what are the real functional differences between MiniTest and RSpec? Being test frameworks, there's plenty of overlap, so I'm not gonna go over that. Where they are unique, though, is where it gets interesting. To be fair, RSpec provides a lot more than MiniTest. Things like test metadata and metadata filtering, more hooks like before and around. Implicit subject described class, fancier mocking, et cetera, et cetera. Basically, it's fancier. MiniTest by definition doesn't offer as much. Some of the stuff that made it unique has been adopted like randomization, but benchmarking, parallelization, and speed are the main distinguishing features. Basically, it's simpler and more pragmatic and snarky. So it's the cognitive differences with RSpec is where things really start to diverge, and this is where I think that Noel's talk actually proves my point quite a bit. Myron Marston, a few years back, wrote a really great response on Stack Overflow comparing RSpec and MiniTest. It was a bit biased, but honestly, I think that it was rather fair. The problem is that it's really long, and the meat of it is in this first paragraph here. But even it's pretty long, and apparently I've been working with a tender lab for too long, whose attention span is that of a pherodon methamphetamine. So there's so much there that I have a hard time dealing with it all at once, so let's focus on that one paragraph. I'm gonna color code it red for the RSpec points and blue for MiniTest.  I've read it too many times. So Myron thinks this is why RSpec is great, and I think that this is everything that's wrong with RSpec, and we're both right. Philosophically, we're both right. We have different goals, and we have different perspectives on what good is. So back to that paragraph. Let's try to boil this down even more for the ADD. Even that's pretty long, so let's break it down to a simple table. Example groups. MiniTest compiles blocks down to simple classes where RSpec, and I'm paraphrasing this here because he didn't say what it did, reifies testing concepts into first-class objects, and that first word I think is the big red flag for me with regard to RSpec. If you're using that word, you should be coding in Haskell. Examples, MiniTest compiles, it blocks down into simple methods, whereas they use first-class objects. MiniTest uses inheritance or mixins for reuse, where they use shared behaviors for first-class constructs, and simple methods for making assertions for expectations versus first-class matcher objects. What even that's pretty long? So let's boil it down further. MiniTest uses a class, they use a first-class object. MiniTest uses a method, they use a first-class object. MiniTest uses subclasses or include, they use a first-class object, and MiniTest uses method calls, they use a first-class object. First-class simply means that you can assign something to a variable, and you can use it the way you can use any other value, and it just so happens that everything that MiniTest uses is Ruby, and nearly everything in Ruby is first-class, so that's not a good distinction. Everywhere that I could use Ruby's mechanisms, I did, and everywhere that RSpec could reinvent, they did. So let's try to take a look at this and see where this starts to get more cognitively complex. I think this is best illustrated by examining how RSpec works. Here we have two nested describes. Each one has a before block, and each one has a single example. Yet the before blocks seem to be inherited, and the examples are not. There'll be exactly two runs here, where the first one uses one before, and the second one uses two befores. So our A and B classes is nesting like subclassing. What's the analogy we can use to understand this? Well, if their classes, and nesting is like subclassing, then we need this undeath method to ensure that we don't inherit any tests from our superclasses.  But that's the runtime behavior that RSpec users expected out of MiniTest spec, and that was something that I wound up having to put in. What about this analogy? Are before and after included modules? If that's the case, then we don't have to undeath any methods, but instead we need to generate a bunch of inner modules and have a bunch of includes, and setups need to intelligently call super, and because of the complexity, this sucks, too. This is basically another object model. To effectively use RSpec, you need to learn a whole separate object model that sits on top of Ruby's object model. This is doubly confusing if you haven't already learned the Ruby object model, or if you're just trying to learn both at the same time, you're gonna get overwhelmed. What winds up happening is it encourages users to hand wave. Noobs are just using, just learning Ruby. They don't have the time or the ability to dig in and learn both object models, and it encourages users to hand wave the oddities away. What else are you gonna do? So it encourages them to not know what describe and it or actually are or do, and if you didn't see the previous talk, there's a lot of stuff that goes on inside those described knits. Basically says, here's the magic incantation to do X. So when you're a beginner, any sufficiently advanced technology is indistinguishable from magic, thanks to Arthur C. Clark. So now I need to rename my talk. Sorry, Noel, but RSpec is magic. So from that previous post from Myron, he said that many people find it to be overkill, and there is an added cognitive cost to these extra abstractions. Indeed, here are the raw numbers of that added cognitive cost. Don't bother grokking these numbers, we're gonna visualize them in the next slide for both the FLOG line and the comments plus code line. FLOG is a complexity metric proportional to how hard something is to test, to debug, or even to understand, and here RSpec is 6.6 times bigger, and this is the combined meat of the project. This is code plus comments, and it's basically how much you're gonna have to read to understand each library. At 8.5 times bigger, it's akin to reading Dr. Seuss versus James Joyce. Do you like my Dr. Seuss font? I think I faked that really well. So back to that added cognitive cost. As we're gonna see in a second, it's not just cognitive cost. There are performance differences as well. All those abstractions reinventing the wheel. It has a real cost. Here we have some fairly complex plots. This shows the runtime and solid lines and the amount of memory allocated in dashed. The green lines are 100% passing, the red lines are 100% failing, and the others are in between that. And as you may notice, there's a slight problem to these charts. Can anyone guess? They're not using the same scale. So now you can see that there's a severe and painful difference between RSpec and main test. Failures have exponential growth in RSpec. Runtime is always near zero in many tests, memory is linear, and it's always lower. Brian, who cares? Passing RSpec is fast enough. Indeed. I actually agree with that. Everything is bread and roses as long as everything plays nice. But what happens when it isn't? You have a bug or you refactor or anything else goes wrong. Oftentimes when I'm refactoring stuff, I'm gonna make a change and I'm gonna have 100 tests fail. All of a sudden you pay. For completeness, the speed of the actual assertions in both systems are purely linear. The speed of running those tests at the method level is also linear. And because they're linear, please do not try to regain any speed by reducing examples or expectations or otherwise reducing the quality of your tests. If you want to speed up, Minitest will always be faster than RSpec pretty much by definition. So you can switch to Minitest or you can never ever refactor or have any bugs. So in summary, at the end of the day, as long as you test, I don't actually care what you use. Use the best tool for your job. Hopefully I've shown the technical merits of Minitest. Choose what works for you. Not because it seems popular. Oftentimes I hear that they chose RSpec because there's more documentation available for it. But maybe there's fewer articles about Minitest because there's less need for them. Minitest is much easier to understand. You can read it in a couple hours and understand it head to toe. So maybe Minitest users aren't missing. Maybe they're just busy getting stuff done. Choose what works for you. Who knows? Try it. You might like it. After all, it's just Ruby. Thank you and hire me.


about order from chaos and computation in neural networks. Jonathan. Thank you very much, and thank you for inviting me. So like Chingiz, I'm trained in physics and kind of find my way into theoretical neuroscience. But while Chingiz is just advocated for why we should understand deep learning in order to understand the brain, I'm going to offer a different perspective of why perhaps we should look elsewhere and not just understanding deep learning. And the basic argument behind that, which can follow through this talk, is that our brain does more than just classify cats and dogs or identify digit, or at least the brain of most of us. And if I'm trying to maybe pinpoint what's different, what's different than just this simple feed forward image classification is the basic element of time. Because when we operate, we do different tasks of cognitive tasks and computation. We interact with the world. We're getting feedback back from the world. And our actions take time. So in those deep networks, you lose the time perspective in the same. Another thing is when we look at the brain, it's very dominated by a highly recurrent connection, not feedback connections, which are more suitable for doing computation or dynamics during time. So the framework that I'm going to talk or advocate to use is what we call computation through dynamics. And the idea is this is we have some task we want to do.  This can be solved using some algorithmic solution, computational solution. Now the main idea or assumption here is that this algorithmic solution can be implemented as a nonlinear dynamical system. But basically that's what the ball is. Living quantum aside, basically we are all some kind of nonlinear dynamic systems. And this nonlinear dynamic system that solves the problem can be implemented as a neural network. So if we had this piece of cortical network, we get some input. The input could be modulated by time. It could be a temporal input. Something is happening, and we get some output. And from this, we can get a variety of different cognitive and computational abilities. We get memory. We can do integration, temporal pattern generation, temporal correlation, Bayesian sampling. And it allows us to do different cognitive processes, as evidence accumulation, decision making, language processing, motor planning, and control, sensory processing. So this is a very rich framework. And what I'm going to talk about today, it's going to be very general, a high level. But you can think about it as more temporal generation, pattern generation, and motor planning. Because this would be the easiest and most stressful way to think about this dynamical system. So how do those kind of recurrent neural networks that some people in machine learning work on come to neuroscience? So a lot of times they use in-cylical hypothesis-generating framework. There is some network we train it, usually using back propagation through time to perform some task or to match some activity of neurons. Now this is very effective, training with back propagation is very effective, but there's no reasonable biological implementation for this thing, for many, many reasons. The brain just does not do back propagation through time, or at least as we use it. Nevertheless, this has been used in a variety of neuroscience studies where they're trying to emulate the brain and then study this network, and actually they've been used pretty successfully. We have learned a lot on the brain by studying those networks. And because of this huge success, it's very tempting to try to understand why, how can we do a more biological back propagation through time? And there have been such studies of trying to do it, but I feel that's pretty much not what you should be looking for, which find to solve something, not by forcing it to be something that is not. So instead I'm gonna try and kind of propose and a little bit study a different approach. So first a few observations on the brain. If we take an isolated neuron from our cortex and inject currently to it first, I hope all of you know or heard that neurons are basically activated by spiking. There is this kind of discreet action potentials. But if we insert a stable disinclined neuron fires like clockwork, very stable. But when we look in the brain, or not only in the brain, but even in cultured neural networks, we find something very different. We can have the same input to that network again and again. But each time we do it, and here on the Y-axis, you get a trial to get a very, very different behavior. And we can measure the variance versus the mean count and you can see that the final factor here is about one or even larger than one. So from this very accurate behavior, we get this very noisy behavior. But when we look at behavior, behavior is pretty reliable. If you think about what you need to do every day and especially a specialist like tennis player, cellist, they need to perform again and again the same action with high accuracy. So this is a bit of a puzzle, right? Why do we start with something very dependable, go to something completely undependable and noisy and go back to very reliable behavior? Now, we're in good company, we're not the first one who's thinking about it, so John von Neumann asked this question. Way back, especially in related to biological computation, how do we get reliable computation using unreliable components? So the framework I'm gonna use here to understand relies on reservoir computing. It came from basically from type of machine learning. So I'm gonna go do a quick overview of what does it mean. Imagine we have a network, so this is some just recurrent network and neurons just going the day doing about what they wanna do. And then we put some input into it. Let's say we perturb it in some way. And we wanna make sure that when we perturb it, the net we can read out specific output. You can imagine this input to be some higher order in the place of the brain telling us to move the hand and the output would be the actual control signal to the muscles and we wanna learn it. Now we can take a readout unit and it just connected to randomly, obviously we'll get nothing. So one way to do it is just do it many, many times again and just slowly learn, do some kind of regression and learn those weights so we can get good approximation. Basically, we have kind of the activity of those very noisy neurons X like a sort of a basis function. And from this basis function, we can construct any function. Of course, if this was infinite and have enough mass properties, this could be the case, but the networks are not infinite. So this is pretty much limited by the basis functions that we can get from the way of the world. So in order to improve that, there's what we call as computing with a closed loop. Basically, we can close this loop back onto itself and that actually allows us to learn much more. I'm not gonna go into the details, but the first, there are some issues of stability, but the first algorithm that we're able to solve it very nicely was suggested by some students in Abbott. It's called the force algorithm and they were actually able to teach a stickman to walk. What does it mean? What does it mean here? Basically think of every point here on the body as contained of three outputs and then giving an input, they just train each of this point to do the specific trajectories which translate to this walking man. So we're able to do, to teach a lot of things, this network's dynamical patterns. Now most of the sufficient is at the edge of chaos. I'm gonna talk soon about why or what's specific about the edge of chaos. It's important, but that leads to some issues. First, once we learn, chaos is a train. Meaning once we learn something, the network is no longer chaotic. It's part of the, what this machinery does, which leads to some problems with continual learning. If we want chaos and we get rid of it, once we learn something, how can we learn something new? Also there's issues of task inference and most importantly, which I'll touch just a bit today, which is, this also is not biological possible. This is a false algorithm. So things I'm gonna talk about today and hopefully get through today. So first is how can chaotic network faithfully encode signal computation? We wanted to be to have a network. We wanted to be chaotic or noisy like we observed the cortex, but still get out something very reliable computation out of it. And the second part, which we'll probably get on it to do briefly is how can this network learn efficiently and still remain chaotic? So a good place to start is close to the beginning, Alice. So first, how do we get this chaotic network and what's so special about it? And this goes back actually all the way to 1988. Some policemen, Sandy and Sandra, some else, so that if you define a nonlinear recurrent network and that's what we have here. In my age is the input into each neuron. We call it something like the membrane of the potential of the neuron. It follows this differential equation where R is the output of the neuron. It's just some nonlinear function of it inputs and they're all connected through them through some disorder connectivity matrix. So this is ordered JJ. It's random. It's case like one over N, very much like spin glasses. The only difference here is that J does not have to be symmetric because this is not really spins and neurons are not generally the connection between neurons and all of your symmetry. So what they found that, and so we have the parameter G here, so sorry, I said some external input in that book. They found that if you have this parameter G here which basically says what is the amount of the disorder in the system, the magnitude of disorder. If you increase G enough, it becomes chaotic. So for small g, we look at the different neurons so each blue line here is a neuron. We see that this network settled into some fixed point. Nothing is happening. But if you increase the disorder in the system, basically there is a transition and the system goes to be chaotic. And by chaotic, I mean every neuron is doing, is fluctuating internal generated fluctuation and you can actually calculate the positive life of the exponent. This is actual chaos. Interestingly, we can actually find this. This is, and we find that this is sort of not exact but looks like a second order phase transition. It's a continuous transition. And with a second order phase transitions come some critical effects and those critical behavior of the system is actually what's favorable a lot of time for computation. For example, long time constant, strong correlation, all sorts of things. And this is one of the reasons that this edge of chaos that just mentioned before is favorable for computing. We also found, and this will be important at the end of the talk, we'll go back to that about the dimensionality. So we have a network and it starts to be chaotic but when we look at the dimensionality of the chaotic activity, it does not take the full space. It's just a small fraction of it but while it's still for the small fraction and this fraction increases, we increase the disorder in the system, it is extensive with the system size and the fact that it's extended with the system size will be important at the end. Okay, so we saw how we take those very clockwork neurons. There's no noise in here but we still get chaotic behavior. And the first thing when we get a chaotic network and we want to use it is how to get rid of that chaos because as we said chaos is not really good for computation, it's not reliable. So how can we remove this chaos? So to try to understand this, I'm going to use the most simple complication we can do which is an autoencoder. What is the autoencoder here? This is a network receives a signal X, it's a low dimensional signal. A lot of time I'm going to use just a one dimension but everything I'm going to take can be done in several dimensions. We're going to inject it to the network, each neuron is going to get some projection of it and we're assuming we're going to train something to read back the signal out of it. Now, no matter how well we can do, if you can see on the right, basically we'll never get exactly the target. Why? Because the chaos brings noise. And so we get some approximation of that. And the question is, how can we make this more exact? One way is to take infinite number of neurons but again, the brain and the circuits are limited. So how can we increase that? Another way is to increase the signal. So we can just push a stronger signal, the signal is stronger than the noise and then we can read it out. But this is not a very good plan. First, if we increase the signal, we increase the firing rate and this is not very efficient. Second, if neurons are saturating, we're going to move away from the dynamical regime and it's going to hurt our computing. So how can we increase the signal but still not increase the firing rate? And easy solution is basically take the output and send it back into input. And so it can't cancel the strong input. This needs some synaptic balance and I'm not going to go into the details of how it becomes. Basically, synaptic balance means that the input into each neuron that feed forward input and they can input kind of balance and cancels out. So overall, the neuron does not increase its firing rate. What do we get here? And what we get the same thing that we saw before but now the input is some W. W is the input projections of not of the input X but of the difference between X or N and X and our estimation. So this is, and if we take B to be strong and stronger, this is what we call predictive coding. Why is it predictive coding? Because what the neuron now are coding are not X but deviation for the expectation of X. Deviation between X and X hat. And this is known to be very efficient energetically wise of firing level wise but we've also been found in different animal systems that we actually found and chose it's happening. And we can see what happens on the right. We'll see what happens when we do that. We can increase this effective B, make B larger as we go down. We can see that the readout becomes much, much more faithful. We have less readout error but the underlying system remains chaotic. So this is a system but now that we have, we have that the question is, let's do some math, let's do some theory which we've kind of avoided until now and how do we solve it? This will be the most technical slide of this talk. I'm not gonna solve the entire main field but I wanna give you the main idea of the main conceptual idea of the solving it. So we have this high non-linear dimensional system of the Hs and we're gonna divide the Hs into two spaces. One is a coding subspace which is basically where all the code lives or where the input and the output lives and the rest which we call the bulk. And we'll assume that the dimensionality is much more than the number of neurons and we also assume it's that the Ws are basically orthonormal and for large network, random would be good enough for example for that. So now we can solve for those things separately. If we look first on the bulk, basically what we have this addition that we have here by removing the low rank or the low dimensional thing is just a very small correction to the activity and if you look at that, that looks like the original chaotic equation from a few slides ago. So basically this is what just give us chaotic dynamics in this bulk space. If we look on the coding space, we can define those U which are basically the fields in the direction of the coding and we get a lower dimensional equation for the behavior. And now this is where the trick lies of how we can do it because what I've done to now, I've taken a nonlinear dynamical equation and I've broken it up like it's a linear dynamics. Obviously we cannot do that but this is where the trick lies because D is very small. So we have interaction between those two spaces but in the bulk space it barely fills the interaction due to the coding space because it's so small and all the interaction of the bulk space goes into this noise term. So in the mean field we can just look at it as noise and just start it as properties. The first thing we can, the last thing we can do to solve this equation just look at U at this field U and divide it into its mean and the mean here is like a thermal mean or mean over the noise and you can look at it mean over time and the fluctuation and then we solve static mean field or first order mean field for the mu and that will give us the bias in the estimation of excess eventually and we can use second order mean field or dynamic mean field to solve the fluctuations and give us the other, the fluctuating error of the readout and that's basically the framework for the mean field. What we get from that, so this is, I'm showing you here the solution. So first you see the delta H is the order once meaning that the underlying neurons are all chaotic and they're dancing and they're jumping about but as we increase B we can see the decrease of the overall error and the error is, here it's not the U but it's already the fluctuations in the actual readout and I've put here for reference both from theory and simulation what happens instead of chaos we would have noise. So the blue one is what in the case of a chaotic network just inject gargling noise and it's an interesting thing that this framework is much more effective in reducing this chaotic noise than just thermal noise and this helps computation and another reason why we want to use chaotic networks it's actually have a very interesting reason due to the time constant of the system and you can look at this paper if you want to understand more about why the difference here between especially in the appendix of this paper. Okay so one more thing, you can say great we can remove now the fluctuation we can get good reliable readout why not just increase B to infinity and remove all the noise so we have the most reliable output but that's not the case and we went back to John von Neumann who said every network or nervous system has a definite time lag between the input and the output so the time lag will become important and why that? Is that because of hipsters? So why hipsters? I'm gonna use here, I'm gonna quote here Jonathan Tubal on the paper with non-conformist try and stray for mainstream trend they oftentimes end up making the same choices because they are too slow to spot no longer popular trends too slow, that's the key point here and it's too slow basically is our delay imagine that whoever calculated this readout there or in this case it's just linear but it could be something more complicated it took time for the feedback to come back so where does this feedback enter in our equation? You can figure out yourself that it's pretty much in the coding subspace because in the chaotic subspace there's chaos, chaos is symmetry over time so the delay doesn't really matter there but where it matters is coding subspace what does this thing do? We can write the characteristic equation for this ODE and look for its stability and its stability is when this gamma here disappear becomes zero and we're left only with the imaginary part and that basically depends on B and B times this thermal average over the derivative of the non-linearity we can plot and we can see a phase transition below that we'll have stable dynamics and above it we're going to develop strong fluctuation you can see it here below it we have some noise that we've seen before where we can actually follow the output but if we go to the non-stable oscillatory regime we are developing strong fluctuations and this depends on the function of the B of the amount of feedback but so what's new here basically we can see that there's global instability and oscillation in the mean field so all oscillations come in the mean field while the underlying, if you look at the underlying neuron they're still chaotic, they're still very much chaotic so like we've changed the B also and it depends not only on the delay but it also depends on the noise and the noise goes through the fluctuation through this thermal average over phi and we can change the amount of chaos for example the G from before the G was the amount of disorder in the system we can change it and actually we can see that we can stabilize the chaos if we add more chaos the system become more unstable sorry, more stable while we remove the chaos it becomes more unstable the intuition behind that basically is this delayed feedback trying to synchronize neurons like those hipsters while noise and chaos trying to desynchronize neurons because each one gets some independent noise so these are the two opposite effects and every time we get two opposite effects we get trade-off and we get trade-off we get optimality and we can actually calculate that so here we see the calculation for the mean field so we look at the fluctuations of the output it all comes from the source that comes from the chaos but it depends on two terms the first term is what we talked about before when we add the balance it goes like 1 over b and larger balance so it's not larger feedback and more balance is the synapses makes the movement this is what we can see here without any kind of delays in the blue but once we have delays and there is a critical level of balance there is also some resonant point which attracts those fluctuations and we can see as we increase that something, the fluctuations and specifically around this frequency increase and when we have those two things we expect to find optimality so we can see here that here I change the amount of noise, g and we can see as we increase we can find the total fluctuation there is an optimal point going back a little bit to biology so what we actually now predict that we at optimality we should see some oscillations in the brain and indeed we see oscillation in the brain normally that we can say that the oscillation basically also oscillation basically depends on the amount of the delay and what the amount of delay is depends on this whole feedback chain that we saw for example if the feedback is just a small feedback within the network we would expect only external delays we would see fluctuation in the high gamma regime that we record in the brain but if we look at if this delay has to go to several assynapses or several networks until it comes back we'd expect oscillation closer to the theta regime and one thing we're trying to check now with experiment is to verify this kind of finding if they are unrelated okay so what did we get so far and I have to run I guess because there's not much time left is we got cortical activity which is a basic pattern generator but with this kind of we still get a low error decoding subspace where we can use the code and we can go on arbitrary linear dynamic system I haven't shown that I just an auto encoder but with this simple setting we can do arbitrary dynamic linear system and at the time we find find level disorder and global oscillation so going back to what I wanted to show we said one thing is how to encode with chaotic network the second how can we learn I'm going to go very fast about that and because I'm out of time so remember that we looked at this force algorithm so we kind of understood maybe how we can solve the problems of chaos but you learn it because you keep the chaos after learning but what about task inference and more biological learning so for that we look again at the brain and we're looking for this kind of feedback mechanism that we have in the brain we actually find them and the key thing for that is a cerebellum cerebellum sits here on the back of our head and while it's sometimes called a small brain it actually holds more neurons than the rest of the brain altogether so it's a very important part of the brain that's been developed with the neocortex from the beginning actually we can find something similar even in the brains of flies so beyond that we know that it has a very distinctive structure unlike the cortex which is a random neural network or something more interconnected it's more layer wise looks a little bit more like a deep network but we know that damage to the cerebellum leads to deficiency in learning especially motor but not only and it's layer wise and it's connected to the cortex as we look and it gets input from the cortex and put its output to the cortex so if you guessed it the kind of idea is to take this output loop that we've done for the force but replace it with something like the cerebellum that can actually learn this output in a more efficient way so one thing you can see this is very huge layer so this is a layer in the cerebellum called the granule cell it's a very very huge layer that's very conservative and very typical to the structure of the cerebellum that is a huge expansion like it's a feed forward layer with a huge inspection as it comes from the cortex so actually we're going to focus about what this layer does to the learning and I'm going to run through that so the idea or kind of the intuition of what it does we can think of imagine that this is a cortex now let's ignore the recurrent connectivity but there are some blobs of activity each blob could be something that we want to learn or activity in some way or representation of different tasks and what we have seen in before in previous work that as we increase the size of this cerebellar layer and the sparsity activity net we can read things from that much more easier the kind of intuition is that if we look at the cortical layer and then we have those different blobs and each one of them again is a task representation of dynamical types but the cerebellum is going to look something like that so it's much easier to read out from intuition basically cerebellum looks with a magnifying glass into the motor cortex and it has to learn more effectively how can we show that again without theory but just some numerics let's take two networks and see what you mean by effective learning so we have this recurrent network like the force that we just said and it gets different input each one is a different static pattern that it gets in the pools and we want to teach it for each pools to do a different target so it gets different neurons get activated we want it to produce different output each one would say one would be move your arm this way one would be move your arm that way and we ask how can we learn it what the effect of you learn it and how do we how do we what do we do with efficiency how we define efficiency basically what happens when we add more neurons so in one way when we have this force like we can take and add more neurons into the cortical layer while on the cerebellum kind of architecture we keep the cortex straight but fixed but we add neurons into the into the cerebellum as as you can see that if you look at the without the expansion or without the cerebellum if we add more neurons we don't really improve the efficiency of our learning but if we add neurons to that big layer we can see that that the error drops like one of the end like what would you expect without going into the theory why do we see them now I'll go back to the beginning and what I said about about the kerosene and the dimensionality is basically that when we have this chaotic system in here when we have some dynamics then it's a low dimensional but extensive with a size so as we add more neurons here we're actually adding more dimensions to the error what instead of we're adding in here we are able actually to remove out the error okay so to finish up basically what we're doing now or going next is doing to a more full cerebellum like structure of this and this actually maybe relates to other works here or what you're interested about deep learning is how to learn those dynamical system in cortex but using the power of somehow a deep learning or deep learning with wide layers which we know a lot about but we can now connect it to the cortex one more thing that we've done but I didn't have time to show that using this expansion we can actually start using or at least in some cases start using local learning or something like heavy and learning meaning that we can move away from this forced learning and use this kind of mechanism to learn in a biological possible way to summarize so we have shown that the disorder and chaos makes cortical networks good pattern generation data and it's a good and we can use it for learning this diversity of patterns feedback is a lot of reliable low dimensional task encoding and cerebellum like expansion allows efficient learning of low dimensional tasks and representation thank you very much these are my group members and collaborators bolded are the one that are that I'm working with something that is related to this work and if anyone is interested in that please come talk I have vacancies for PhDs and postdocs and thank you for your time very nice, thank you very much I'm sure there's going to be I'm sure there's going to be questions I think Ali was the first one do you have any idea why force kills chaos? why it kills chaos? yeah why doesn't what is the mechanism? yeah what is the mechanism? so when you increase this you create this loop basically you can change the point or increase the loop with the input you can change the transition to chaos that's basically the assumption actually I haven't tried to look exactly what did happen as a function of learning but the assumption is that especially if you are in those nonlinear networks in ton H and it works less well with value but the strength of the input of this loop changes the goes into the calculation of the transition to chaos so it can change this and basically and that's why it works close to the transition if you're at the edge of chaos you don't need much to move to become more stable it has a bias somehow like it prefers to not use chaos in your way well you try to try to do something that's not chaotic I mean you have a very output is very reliable thank you Stefan thank you I probably missed how did you perform the analysis in the second piece of your talk when you started to have a hidden layer before the readout after the reservoir I didn't miss I didn't talk about it okay can you comment on that on which part you mean on that here um okay this one okay so the analysis we've done here is what I'm showing here it's for static patterns so forget now the recurrent activity and we just look at the static pattern imagine that here we get an input some set of static patterns and what we want to do is just basically train a readout so if you think about it in biologically this would be some kind of cortex this would be like a granule cell and it would be like a pockinji cell that needs to learn out of them and the idea is to ask okay let's have just the pockinji cells active only for some of the output so basically it correlates those different tasks so all we need now to do is to be able to ask what happens if I have a readout and I want to ask to train a network so our this output is positive only for an arbitrary uh... subset of the labels just arbitrary of them and then we ask how can make this arbitrary classification so each output can only listen or transmit some of them and for that you can look at those I mean there's work with a Bakhtosh Babadi with Sampolinsky and we have a I think it's a new paper from 2016 that we use this kind of ideas and basically what we're doing is just implementing that not now we're doing it implementing it with the current connectivity I will check the paper probably and there's another question from Alada and I have a feeling I know in which direction it's going to go so concerning dynamic of main filteri when you talk about the two subspaces so the coding subspace and the bulk subspace how you do determine the noise in a self-consistent ways it correlated to the other parts or okay so uh... the eta term yes so the eta term comes from from the bath now the contribution of this part of the coding subspace is negligible so I can solve uh... dynamic main field for here we're just ignoring subspace because the contribution is just of order d o square d over n and it's much much smaller than so here we get chaos and once we have the chaos here and I have the autocorrelation function of of of the activity in the bulk basically eta is the autocorrelation so it's not gaussian noise okay this was my question so it's related to the autocorrelation what I was referring here it comes from the fact that the autocorrelation of the difference between gaussian noise is autocorrelation is not gaussian thanks thank you I don't see any more immediate questions so let's thank the two speakers of uh... this afternoon session once again the coding stopped


This is a cipher message This is what they wanted me to tell my audience Someone is trying to make contact with me. Which number represents a letter A is 1 B is 2 C is 3. You can hear him too, right? G U R T H E a Shift value of 13 It's a rough 13 algorithm Today we're gonna be talking about passwords and specifically how I built an AI that can crack them Thank you to AI camp for sponsoring this video if you're new here My name is Josh Beasley and I'm a recent graduate from Yale University where I study computer science Now I live out in LA where I'm active duty military and I make videos about college tech and anything else I'm interested in before I start blabbing on about neural networks and adversarial training loops I thought I would give you all a little 60-second introduction on how you go from a string of characters that represents a password To an AI that can crack them So first off when people say AI or machine learning they mean a lot of different things But this project focus specifically on one type of machine learning called supervised learning So as supervised learning you have a lot of data and attach that data is Ground truth values that you want the AI to predict given the data So if I wanted to build an AI that would recognize handwritten digits The input data would be the actual handwritten digits and the output data would be what the digit actually is So this would be a 7 and this would be a 9 Supervised learning involves feeding a program a bunch of these handwritten digits Letting it spit something out and then telling it if it's wrong or right over time If you have a good model and you do this enough times the computer will begin to catch on to some of the patterns in the data and begin Getting a higher and higher percentage of predictions correct now I use this word model and that's the tricky thing about AI because we're not explicitly coding a program You're actually coding a model that has a bunch of attributes that can be adjusted based off whether your prediction is correct or incorrect There's a ton of different models to choose from the one I use for this project. It's called a neural network I won't get in the details, but this type of model supposed to simulate neurons in the brain So you have a bunch of neurons and when you feed data through them They can either fire or they don't fire and neurons are connected to other neurons and these things called layers and the neurons that are Connected to each other each have a weight associated with that connection now That weight is one of the attributes of the model that I was talking about that She can wiggle up and down to improve your prediction accuracy And as you feed the model a bunch of data and it wiggles all these attributes up and down That is what is called Training and I think that was definitely more than 60 seconds if you're interested in learning some of these AI techniques I highly recommend checking out the sponsor of today's video AI camp AI camp is an immersive Project-based summer camp for motivated students under the age of 18 It allows them to explore the field of AI before going to college and getting locked into a career You'll have the opportunity to work in a small group of like-minded students to build and deploy an impressive AI product Like those seen here with no prior coding experience necessary testimonials and success stories from alumni Really speak for themselves, but your experience could lead to a internship at a top startup in Silicon Valley The best part is about 60% of AI camp students are on scholarship and 10% attend completely free I really encourage you all if you're interested to check out the scholarship application at the link in the description It can all be completed in one sitting back to the product I coded this AI as part of my senior computer science thesis at Yale and I'll just say it was very involved In fact, I had three completely different models that I had to build and run a bunch of different experiments And it's all detailed in the little thesis booklet that I printed out But we're not going to get into that I'll save you guys the research and say out of the three different models that I tried The recurrent neural network was the one that actually performed the best Recurrent neural networks are best used for sequence data and in the case of passwords It's sequences of characters So all the network does is take in a single character and then predict what the next character should be As it receives more and more characters it begins to build up somewhat of an internal memory And that internal memory is what we're going to use to predict passwords So we have a recurrent neural network. Now we need a data set Luckily cyber security is a really really hard thing to do right And there's tons of leaked password data sets online Now all you have to do is feed your recurrent neural network Thousands upon thousands of passwords as it reads a window of characters through the text and predicts what the next character should be And then it builds up that internal memory and develops its own sense of what the structure of a password should be And then all you have to do is wait a couple days because training takes really really long Even when I was using the Yale super computing cluster after all that training You can test your model on a reserved portion of those passwords Just to make sure that it's performing up to par and then start cracking It's literally as easy as supplying the recurrent neural network with a single character and then letting it Just run for as long as you'd like and generate as many characters and therefore as many passwords as you'd like At the end of the day, you have a text file with a ton of potential passwords that you can use to start cracking Those password cracking techniques rely on a dictionary of passwords that they modify Switch around and then basically run through all of them and test to see if they work Now this model is a great way to expand upon an existing password dictionary And ultimately improve the success of cracking a given password So me personally, I train the model on 80% of the data. That's what we call the training set I reserved 20% which is the the testing set and then I let it run for a while generate thousands upon thousands of passwords And then I checked how many passwords that were generated Lined up with passwords that it had never seen before in the testing set And it actually cracked quite a few you can actually see here some of the passwords the recurrent neural network was able to crack Based off training on different sizes of data sets and that my friends is how I built an ai that can crack passwords If you enjoyed this style of more informative video and talking about some cool You know c s tac ai things that i'm interested in Love to hear what you thought in the comments down below as always my instagram dms are open if you have any questions I'll drop a like on the video and it helps me and supports the channel a lot more than you think And if you're new, please subscribe and if you're new, please subscribe and I'll see you all next time