In starting, or kicking off this panel, one thing I want to make sure everybody knows is that we actually in advance, so a bunch of volunteer hackers, which is what we are, right? This is a volunteer operation, none of us make a dime off this, we actually lose money. So we raised our own money to pay some interns, these poor souls, to spend three months building a list of every election official in the country. Over almost 7,000 people we got contact information for. We then paid to do a snail mail, U.S. post office, mailing to every single election official in the country. We then followed that up with two emails and did 3,500 live phone calls to local election officials to invite them to come here and let their staff take part in the training, loan us equipment if they want research done on it, and so on. And because of that, our attendance from local election officials is up several hundred percent from last year. And what we're really excited about is that we have some people here today who are truly great Americans, who are deeply committed to protecting the votes of their constituents and citizens, and have been spending the last two years, and for many of you, even before that, really thinking deeply about how to better secure our elections. And by the way, showing up here, well, you know, we're not the answer to this problem, but we are hopefully a piece of it, and we are really appreciative to have the election officials in Homeland Security here to try and, you know, learn from what the hackers figure out and each other. So with that, let me just do some quick introductions, and then we'll turn it over to them, because I'm very interested in what you all have to say. So to start, Secretary Padilla is the Secretary of State of the largest state in the country. California is here today. I want to give him a round of applause for showing up. Thank you. He's been working on this issue for years, deeply committed to it, and has been involved in election security issues well before the Russians came at us. Amber McRenalds is from Denver, Colorado. She is actually working on a whole host of interesting things, including auditing and open source technology that we're really excited about. We know that this community loves open source. Jeanette Manfra is the Assistant Secretary at Homeland Security. We are incredibly appreciative to have Homeland Security here. We've got a bunch of Homeland Security guys in the room as well, so thank you guys for showing up. You know, as we've been saying for years, well, for two years, you know, this is, in no way, shape or form, some sort of like criticism on election officials. Frankly, it's not election officials' job to fight off existential threats to the United States. That is the national security industry's job. And the national security industry is represented here by Homeland Security. They're the ones who kind of have taken the mantle and been assigned the task to work with state and locals to help secure their elections. Noah Prietz from Cook County, Illinois. He put out, I believe, I think the first kind of revamped election security plan after 2016. He actually was one of our few RSVPs last year from local election officials and braved the DEF CON for the first time last year. And we're deeply thankful to him for coming. And then Neil from Orange County is here. He also, we were talking last night actually, has put out an incredibly impressive election security program for Orange County, California, which we're going to be highlighting this weekend at the village and then later when we release our report of all the vulnerabilities. So, with that, I will shut up and turn it to Assistant Secretary Manfred to say a few words. Great. I'll be really brief. I talked a lot this morning. So, I just, first of all, thank you for coming, for hearing us. And, you know, I just, I've learned a lot about our electoral process over the last couple of years, things that I didn't fully understand. We have worked for a long time with things that you might more traditionally think of critical infrastructure, whether that's our electric grid, our financial systems, emergency services, those sorts of things, which are just as complicated and tricky to defend as our election systems. But, you know, I guess I would say just a couple of things is it's, for DHS, we really see ourselves as sitting at the intersection between, you know, individuals and organizations that participate in things like DEFCON, the academia, the private sector, state and locals, and other federal agencies. And we sort of have this set of unique authorities that allows us to sit in this place and be a convener and drive progress on reducing risk across our countries, or across our country, and frankly, been working internationally.  But what I would ask for you all is, a lot of the questions I get is, you know, well, just now the elections community is thinking about cybersecurity, and, you know, the Russians woke us up that we need to secure our elections. This is just not a fact. These folks, and many of those who are not represented here, have been thinking about this for a long time, and they do a lot with not a lot of resources. And now they're on the front lines trying to deal with a lot of these issues, and they can't do it alone. We all have to work together. And I think this is incredible that we can bring the different communities, sort of maybe a community of folks who aren't used to working with government like you all, and folks from federal, state and local working together to figure out how to address challenges collectively, because we're all in this together. So I challenge you all to listen and learn from them. They're here to also learn from you, but really try to understand a little bit more, it's a little bit more than just a voting machine. There's a lot more that goes on in an election process at the state and local level than just the individual voting machines. So challenge you to learn a little bit. And with that, I will pass it on to the secretary.  So we have to like, they're just yelling at us. We actually have to take all the chairs out of here. So can we break for like five minutes? These chairs can stay. And then we got to move the chairs. Sorry. It's good news. Everybody cares about democracy, so our room is overcrowded. Okay, now we're back. So, Jeanette, here you go. I don't have to say my remarks again. No. So, no, I think we were just about to introduce Secretary Padilla. So, all right. Secretary Padilla, everyone. Well, thank you. Good afternoon, everybody. I am excited to be here. I really am. This is my first day of CON. I confess, but I am here to listen and I am here to learn. But I also understand that some of my colleagues signed off on some statement that went out yesterday, the National Association of Secretaries of States. That's the first question I got when I walked into the hotel. So let me just acknowledge that up front and then tell you just a couple of other thoughts that hopefully inform this panel and the conversation. Like, I kind of get where they're coming from. For as much attention and emphasis there is on cybersecurity and election integrity, a big piece of that for us as secretaries and elections officials, too, is making sure that voters and the public in general have the appropriate confidence in the systems when people go vote. Right? If it gets into the mind of anybody that maybe my vote is not going to matter, so why should I go vote? That in and of itself is a form of voter suppression if you look at it that way. So just trying to strike the right balance of cybersecurity and integrity with confidence in the system. Some of my colleagues and I'll admit I too sometimes are still a little traumatized from the headlines from last year's conference, right? Voting systems hacked, voting systems hacked, voting systems hacked. Well, my background is in engineering. I'm not a programmer, I'm not a coder, but I think I have a proficiency for technical stuff and like any good engineer, right? You always start with your knowns and your unknowns. You want to understand methodology. And if there's distinctions between what's happening downstairs and real world conditions, that doesn't mean that there's nothing to learn from a convening like this. But it does mean that to be informed about what the takeaways are. So that's where I think some of my colleagues are coming from.  Another initial observation. This is sort of a good handoff, right? From the Department of Homeland Security to a Secretary of State. Because if you look at general dynamics from the last couple of years, the whole coordination and collaboration that we are now participating in is relatively new. I remember vividly when the buzz first came out and my first call from the Department of Homeland Security under the Obama Administration came out in the late summer of 2016. And the initial conversation about whether or not to declare our election systems as quote unquote critical infrastructure. What we have experienced since then is to kind of simplify it. The intelligence community, with all their expertise, having to take a crash course on how elections are administered in the United States of America. On the flip side, elections administrators at the state and at the local level, and you have some of the best from across the country here at the panel having to take a crash course on cybersecurity. Does it mean the intelligence community wasn't looking at the election space before? Does it mean that the elections administrators weren't thinking about cybersecurity before? But boy, there's never been such a spotlight and emphasis as there has been since 2016. Through today, through this November, onto 2020 and beyond. It's our new reality. So that being said, I do want to just offer a couple of points, maybe tee up some questions and conversations later. I mentioned I'm here to learn. I mentioned our comprehensive look at cybersecurity. It's not just replacing equipment, upgrading firewalls, and what's the latest encryption technology for us? It's also about professional development and training. You can have the best protections in place, but if you still have state or county employees clicking on a link sent by that long-lost uncle who just won the lottery, right? What's it all for? It all gets compromised, right? So training and cyber hygiene is an important part of our comprehensive strategy. How we not just secure our elections infrastructure in our processes, but counter misinformation and disinformation. This information, that's a big part of what we're grappling with in this comprehensive look. So much more than that, but just to give you a flavor of how it's a much more comprehensive approach and strategies that we're taking in California and I think across the country if I can speak for my counterparts for a second, and last but not least in my opening remarks. While I thank the United States Congress for appropriating $340 million last month, let me be abundantly clear, we need more resources, right? All the things that we know we have to do, all the things that I'm going to learn and observe, because I'm going down to the village after this panel, to implement and act on all these findings, recommendations and discoveries, we need additional resources. So the money that came to states by Congress last month is not new money. It's the remaining help America Vote Act dollars that were just appropriated last month but authorized 15 years ago in the wake of Florida 2000. I call that money, after I say thank you, that's butterfly ballot hanging Chad money, not cyber threats, 2016, 2018, 2020 money. We need more regular, more consistent support for constant increasing of our cyber defenses if we're going to be serious about this conversation. Cyber security and election integrity is not something that we should invest in only once every 15 years. And so again, a thank you for last month's appropriation, but we need more. And on that front, I do speak for all of my colleagues across the country, both Democrat and Republican and local county elections officials throughout the country as well. So we're going to need your support in that. We're going to have some enlightenment going on, some lessons learned going on today, but when we all leave this gathering, this convening and we go home, I need you all to be advocates for more investments in election systems and integrity at the county, at the state, and especially at the federal level. Thank you very much. Thank you, Secretary Padilla. And with that, we'll turn it over to you. No, I pray it's a good county. All right. Hey, good morning everybody. Good morning. So thanks for doing this. Our community is trying to figure out how best to engage. I was asked yesterday, why are we here? And sure we can learn some specific technical stuff, but I think more importantly, I think about four years ago when a couple of guys from here took over a jeep wirelessly. And then they went to work to help Chrysler make sure that those exploits aren't possible anymore. We cannot pay in our community for the expertise that you all bring. And so we're going to mature a strong relationship between the voting community and the security researchers. So we're in the beginning stages of that. I'm excited to see how far we've come in the last year, the folks that are here. And anyway, we're all committed to the same goal. So anyway, my name is Noah. I'm the director of elections in suburban Cook County. Like Secretary Padilla said, we've been securing votes, voter records for a long, long time. It's not our first rodeo, all right? Prior to, how many of you guys were around doing this stuff before 2000? Okay, so back then, I like to say we were logistics managers, mostly. It was a wedding planner era of election administration. We bring together a list of people, put them in a place on one day holding an election and it's done. Obviously 2000 exposed serious flaws with punch guard technology. There was a significant disparate impact in some communities. And the federal government got involved for the first time in elections, spending significant, something like three and a half billion dollars. And it ushered in a whole new era of technology, some of which is problematic now, touch screens without paper trails, certainly. But we all had to switch from logistics managers to become IT managers, legal compliance managers. 2016 was another sort of inflection point. Because now, given the probability of attack, we've got to become cybersecurity managers. So spurred by the need to defend our systems against foreign actors, the federal government and the states, and many of us locals have been sort of negotiating a relationship. Secretary Lawson likes to say it was an arranged marriage, and it's going as well as any arranged marriage could be. But the states have zealously guarded what has traditionally been their domain of managing elections. And they've been very helpful. Secretary Padilla is a great spokesperson. In the run-up to the 2016 election, my boss as a Democrat pointed a lot to Secretary Hew said from Ohio because he was out front saying these systems aren't rigged. It's important in elections that we're able to maintain a nonpartisan approach to what we're doing. So the secretaries of state, state election directors certainly deserve a lot of credit for their efforts. And at the risk of being a little overly broad, though, local election officials, like myself, there's 108 in Illinois, 8,800 around the country, we bear the brunt of running elections. We lock the warehouses, we program the machines, we review the tapes and the logs. We push the equipment out, program it, audit it, count the votes, and release them. And it's tough. Somebody said we're like, with a nation state actor coming out of small county, it's kind of like Andy and Mayberry being sent out, sent out to defend against a foreign attack. These are shadowy adversaries that we're facing and we're all coming to terms with how best to partner with the states and with DHS as sort of force multipliers for us to help us in our efforts. In Cook County, we've studied this a lot. As Jake said, we put out, after last year's DEF CON, a white paper, we focus our efforts around three things. It's defend, detect, and recover. It lines up pretty closely within this cybersecurity framework, but it's easier to remember. Three instead of five points. We partner with the Center for Internet Security when they published their election handbook. We worked with the Belfer Center. A lot of great contents being made. I sit on the Government Coordinating Council. It's a construct of the Homeland Security of the critical infrastructure. There are eight secretaries of state, eight state election directors, nine locals, the chair of the Election Assistance Commission. And we're working hard to sort of help DHS prioritize the investments that they're making, that they're making in our space. So what's become clear to me as we study this is that each election office needs somebody to own security. There are 8,800 of us. We're one of the biggest. We've got one and a half million voters, 100 employees, $20 million budget. And we're able to sort of specialize some resources. And even we decided that we needed to make another position and hire an InfoSec officer in our office. We partnered with the Chicago Board of Elections to do that as a shared resource. And we've been pitching this idea, been pitching it to the Secretary of State and the state election directors, that this money that was just given, the leftover butterfly ballot, hanging Chad money, it's not nearly enough to do a technology refresh. But what it can do is if it's employed right, is the states can hire staff to go partner with local election officials with this expertise. We're not yet cybersecurity managers. Most election officials have one or two people in their office. They outsource most of the work they do. And it's really difficult to conceive of the idea that we can absorb the 20 emails we get from the ISAC every week with listing every vulnerability. The idea that we can dig deeply into the Belford recommendations or the Center for Internet Security without a partner focused specifically on this. So it's interesting to see some of the states stepping up in Illinois. Our legislature required half of our HAVA funds. So about $7 million be spent on a, they call it a cyber navigator program. So we're putting 10 or 15 people on the street in the next few weeks partnering with like adopting five counties and going in there, helping them increase their defenses. Not by creating sort of new material, there's plenty of great material out there, CIS, Belford, specific DHS recommendations. But to help us defend our systems, helping sure we've got the best detection techniques so that when a successful breach occurs, we're able to find it and to make sure that we've got mature disaster recovery or business continuity plans. So that's, I think, a big focus in our industry right now. Obviously, defense is very difficult. I mean, you can ask Uber or Equifax, HBO, Sony, it's just a very, very difficult thing to do. So the key for us is elections administrators to make sure we're resilient that we can overcome any successful attack. Obviously, that's pretty easy in most of the country because there are paper ballots or VPATs. Increasingly, there are great auditing techniques which would indicate when something went wrong and establish the ability to put out results that are trusted and true. So anyway, y'all are on the floor. I'll pass this along to Amber, but I really appreciate your focus on this and appreciate the sort of maturing sense of nuance that there is in elections. It's not a binary question. We're wrestling with our ability to provide accessible ballots to everybody, and that isn't always lined up with the easiest systems to defend. So anyway, appreciate your time and Amber, McDonald's now. Hi, well, I'm super excited to be here. This is, again, my first DEF CON. I couldn't come last year. I have a five and a seven-year-old, so I need to ask somebody to get another one of these badges because I cannot go home with only one. It's very cool. I'm going to need to take one for both. So I've been director of elections in Denver for seven years. I've been in the office for 13. I started as an operations coordinator that oversaw the mail ballot process and then I moved to a management role. And then I was deputy director starting in 2008 and then director starting in 2011. So 13 years I've been administering elections and touched various points in the process. And the one thing that when I came into Denver, Denver was not known for running good elections. Most of the systems were completely outdated, pretty backwards in a lot of ways. And as a 26-year-old coming into the office at that time, now you know how old I am. I kept asking why to all of the people that had been there. And the answer was always we've done it this way for 15 years or we've done it this way for 20 years, so we're going to keep doing it this way. And I'd also come in kind of after Florida 2000 and the one thing that I always say is about elections is elections are about people and process, people and process throughout. Technology supports a lot of those things, but it's ultimately about people and process. And the problem that happened sort of after 2000 is nobody asked questions about how do voters want to vote, what should the voting model look like, what should we do to change policy to make it easier. It was just basically a money dump into various systems that now nobody's using anymore because there's various issues that were identified with that. So there was sort of this rush if you will to purchase equipment and deploy systems that actually do not have any benefit to the voter or respect voters in terms of what they want to do. So in asking all those questions and for many, many years every day I'd go home from work thinking how can we make this better for people. And so we've tried that's been our mission in Denver is to try to redesign the process and make it more effective for voters. So a couple things about Colorado, we deliver a ballot proactively to every voter that's on the rolls prior to the election. We have same day registrations. You can literally come in on election day to a vote center, any one of them and you can, if you're not registered to vote, you can get registered to vote right that day. So this, your name not being on the poll book or you not knowing where to go with your polling place or any of that is eliminated. So since we've done a lot of those reforms, that also means that more than 99% of our ballots in most cases are a paper ballot that the voter hand marked and then the remainder are marked at vote centers on a ballot marking device but still a paper ballot. So every ballot in Colorado is counted in a central place. We don't tally anything at vote centers. We don't tally anything at polling places. We don't have cartridges. We don't have USBs. We don't have equipment moving around in the field. We literally transport all the paper ballots that are cast in the field on ballot marking devices if it's at a vote center or if it's using the mail ballot we mail to the voter. Everything is a piece of paper. We had the fourth highest turnout in the country in 2016. We have the highest, we have the highest voter registration rate as a percentage of population as well. So Colorado has a lot of from a policy perspective doing things very well. The other policy that we just implemented and we were the first state to do this and I'm going to call out two people from Colorado that are in this room that had a lot to do with it. Dwight Schellman, I don't know where he were. Oh, there he is, see? He's way over here. Dwight Schellman is from the Secretary of State's office and the risk limiting audit that we deployed as a state would happen without Dwight Schellman. So Dwight Schellman is, if you want to know about a risk limiting audit process or anything with that, he's here and he's amazing. And then Jennifer Morrell, who was the director in Arapaho County and then now has gone to be a risk limiting audit senior advisor at Democracy Funds where she's now helping everyone else deploy audits across the country. Both of them are Coloradans, both of them were leaders in terms of getting this policy deployed for us. So we have all these great things that are happening in Colorado and a lot of it has been literally organically driven by voters. Voters started requesting their ballots by mail. They started asking us for that. And so we got to 2012 and we were 80% plus people requesting to get their ballot by mail and so then we decided okay, let's just deliver a ballot to everyone because the 20% are all calling us asking us why we didn't send them one because we didn't want to send them one. So we did that and it was all designed and centered around the voting process and making the process better. It wasn't a technology decision, but there's outcomes that have benefited technology but it was about people and process and making that better. In terms of cybersecurity and making sure the election is secure, cybersecurity is not our only vulnerability in the election process. That's bomb threats, all kinds of other things that election offices face. We have challenges that happen all the time whether it be fires at polling stations or vote centers and we have to move everybody or any of these sorts of disasters. This one is one that election officials and you heard this, there's 8,000 local election officials across the country. Cyber and technology is not a strength that most of them have. In Denver, and I don't either. That is not my graduate work. But in Denver, we're sitting in a county as a whole. We have a centralized technology services department with a security team. Four or five years ago, I went to them and I said, look, I want you to help us figure this out. We've been part of that jurisdictional security plan, the elections doing penetration testing, doing all this prior to the election but we've collaborated with our technology services department and the chief security officer for the city. And that collaboration and that commitment and that coordination are all keys to making this better. And then the final sort of C word that I'll throw out there in terms of ways to make this better is continuous improvement. When I came into Denver 13 years ago and I was asking why, it struck me that no one was curious or creative about solving problems. We were not at all in a model where we could continuously improve what we were doing. And that's exactly what we have to be doing as election officials to make this better over time because the threats we face today, as you all know, are going to be very different tomorrow and are going to be very different five years from now. So we have got to get to a place where the elections world is agile and get in this mindset of continually improving and having curiosity about how to make things better. And we've got a lot of good examples of that in Colorado and it's been an honor and a pleasure to be the director of elections there and election officials are committed to doing this. They work extremely hard to make sure that you get your vote delivered to you in some way that's at a polling place or mail ballots but they're not technologists and they need people in this room and there needs to be collaboration and coordination amongst everyone that's involved. It's a community effort and voting should be a community. Voting is the quintessential community effort and so it does take a broad community of people committed to make this better. So with that I'm going to turn it over to Neil Kelly. He's amazing from Orange County. He visited his website. He's got all kinds of data analytic tools and he's done a whole bunch of awesome things in Orange County to make it better. It does not look today like it did when he got there. So he's one of the premier election officials in the country and always happy to share a panel with him. Likewise Likewise, thank you Amber. First of all I'm going to sit here not because I'm very proud to be in California under Secretary Padilla's leadership which I am but the Venetian is a lot further away than it looks and I would advise not running here because the last headline I want is election official passes out at DEF CON so I'm going to sit right here. Yeah I know a little bit about Orange County 1.6 million registered voters I've been the election official there for 14 years. The average tenure I think of election officials in large counties is not generally 14 years so I'm glad to say I think we're doing some things right in Orange County. We're more diverse than I think a lot of people think. The stereotype of Orange County is that it's heavy Republican. We're actually kind of split between Dems and Republicans now in Orange County. So the Reagan era of what you thought about a long ago in the 70s and 80s it's much more diverse. We support seven languages in Orange County in the election office so definitely a diverse office and women by the way are registered at higher rates in Orange County than men and they turn out at higher rates in Orange County than men and I thought well maybe that's because men tend to not live as long as women but it's the opposite in some other counties so it's interesting to me. I think it was teed up very nicely by the way and I just do want to say thank you Secretary Padilla because under his leadership he really has been focused on elections in California and I'm very proud to be a part of that partnership. Like Secretary Padilla said I was contacted like he was in the spring summer of 2016 and everything changed for us as Amber said we were focused on security before that. We were hyper focused on security after that because things definitely changed and I kind of want to walk you through some nuts and bolts of what we're doing in Orange County related to the security side. So previously you would think of big events and incidents in elections would be acts of God and some other things but not necessarily the security side and that certainly changed overnight for us. Prior to 2016 you would think of fires and we're dealing with that in Orange County right now we've lost a polling place because of the fire that's going on right now so those things do happen but after that spring time of 2016 we saw a voter data theft, phishing attacks certainly were on the rise doxing of political campaigns and then scanning of systems. The scanning of systems as I'm preaching the choir here goes on all the time, thousands of times a day so that really wasn't news for us, what was news for us is where they were coming from and looking at those IP addresses very closely. I sit on the Government Coordinating Council with NOAA and I'm proud of the work that DHS has been doing in this space and looking forward to continuing that effort. So specifically for us on the physical side we've changed a lot of our physical security can't talk completely about that but you think of the building and how ballots are transported and the chain of custody side of this, we have really enhanced that. On the cyber side we essentially have a three layer approach to that security in the county and Orange County I think as a whole does very well at that but there's no finish line to this process, it's ongoing and we're going to continue to work on that and the one that I'm concerned most about is the social aspect because the phishing campaigns are a big concern and like Secretary PD said you can have one individual in one office can click on something and can cause problems so the training side has increased tremendously for our employees and yet we still see on the social side employees doing things that we need to continually train against because this is going to be an ongoing struggle for us. We have added and I know many of you are aware of these sensors, Albert Center to our system because I believe the voting systems definitely are tremendous vulnerabilities there and we need to keep plugging those but also the voter registration systems are a concern because that's one of the things I lose sleep about is what can we do to continue to protect the voter registration system so that our sensor is something that we have put in place recently and that end user training and awareness I think just has to continue because that's going to be a problem. The second is third party review and I want to talk about auditing in just a second but third party review I think is also very important because I can't sit up here and say I think our date is great and take my word for it we need that third party auditing and review and they're going to have a yearlong partnership with us to scrutinize our data and to look at what we're doing and to have a third party review that I'm not afraid to be transparent I think we need to open it up. I think all election officials need to open it up to be more transparent and as Jake said earlier we recently released a cyber well no it's not cyber it's election security playbook and that election security playbook is on our website I put it out there for the public to the things that we can talk about publicly here's what we are doing to protect your vote because I will tell you one of the biggest questions that I get is what are you doing to protect our votes and I want to be forthcoming and transparent in that process so just real quick the auditing I think is the biggest piece to this can I use this analogy on you just for a second so the commercial aviation industry the systems are both people and technical if the system fails God forbid you can have a disaster but there's auditing in that component which is the flight data recorder and you can go back and figure out what happened what did occur same thing on the auditing side there is a bill in California right now I'm sorry I don't remember the bill number but that bill thank you very much that bill is moving forward I suspect continuing to move forward I think it's going to be I think scheduled for a senate vote pretty soon to allow risk-limbing audits in California not mandated but to allow it in lieu of the 1% audit that we currently do which is 1% of the precincts that we audit by hand I still think that's helpful because you're physically auditing those ballots but risk-limbing audits like they were doing in Colorado they are doing I hope to have that in California we just did a pilot in June which is one of the most important tools that we can do because at the end of the day if we do all the things that we're supposed to be doing on physical and cybersecurity enhancements and we still have a problem how do you detect it we need to be able to detect that and so I am an advocate for auditing I'm an advocate for transparency we need to continue this process and finally paper is very important so we in California maybe there's two counties still left in California that are running electronic systems in the polling places and we have paper backup on that system which I think is absolutely critical and we have about a million vote-by-mail voters that are using paper there are debates in the industry about whether paper vote-by-mail is the right way to go from a security standpoint I happen to believe it is but still you have that paper backup and that audit trail so I just want to share with you again I'm here to learn and I appreciate the invite Jake very much and I believe in transparency and I think we need to continue to improve that process thank you very much alright thank you very much and thank all of you for coming here like Woody Allen says half of life is showing up and so the fact that you guys showed up and then also these talks were very informative and shows I think that people are taking that many election officials are taking this stuff incredibly seriously so with that I want to open it up to questions anybody yes sir does anybody want to take that so the question was it was noted that the vendors that are not up here and not present and do we think that the vendors are taking this seriously I can speak to one vendor that I know my observation and sort of the way that I've approached things in Denver is we rely very little on the vendor so we don't have them program any of our any of our files we don't have them involved in anything we've purchased software and we use their system and it's all cots it's a ballot marking device and we constantly give them feedback and they actually listen to us and make changes and so we've had a very good kind of relationship in that way I am not easy on them which is one of the things that I think is important for election officials like we you have got to hold vendors accountable for your needs our vendor that we use came to me five years ago and kind of showed me what they were thinking in terms of their next generation of their system and I told them that I wouldn't buy most of it and so they actually went and worked with us and redesigned some things to be cots so the vendor community is critical to this not every local election office like Orange County or like Cook County or like Denver because we all program our own stuff we probably rely very little on vendors to do anything but that is not the case for most of the local election offices across the country and they rely on vendors and the one thing that has been sad for me to see especially in small counties they don't most rural counties if you have a thousand people in your county or you have even 10,000 people often don't have full time county attorneys to help them with contracts they don't have full time technology staff so I believe the vendors some vendors have probably unfairly targeted many of those election officials and sort of gotten them into very expensive contracts or very expensive services it's hard for those local election officials to get accountability so that's really where local election officials are the ones actually doing all the process of the election but this is really where the states can really help the local election officials especially in California especially in Colorado and other places but that's really where the state can get involved with certification or how testing happens to make sure that those kinds of things don't happen to local election officials so vendors key Jake mentioned this we've started to look at some different open source types of systems not necessarily different from other things that we're doing the RLA tool the riskal main audit tool that Colorado now has is open source so I think that is something that we as a community need to continue to engage on and figure out what the best path is forward one thing real quick so I wanted to just add one piece we've talked a lot about some folks mentioned the government coordinating council the department uses to bring everybody together on the state and local side we've also established a sector coordinating council which is our term for bringing industry together so we have these these authorities that allow us to have non-public conversations with industry our focus was the priority is state and local but it is also important to bring the vendors together we they are all now together they've signed their charter which is I know that sounds like bureaucratic but that's actually really important to be able to bring a bunch of different companies together that are competitors and being able to say look we're all committed to working with each other and working with our federal state and local partners it's very important so yes they take it seriously and I know many of them are here but yes there is a lot more work to do and similar to the work that we do with other industries whether they're in the electric sector or the medical device sector we need to continue to work with them we need to make sure that they have information that the government might have to better protect their systems but we also we need to continually challenge our critical infrastructure community to improve the security of their products so we're continuing to build it they're at the table so that to me shows me that they're committed but that's just the first step just to add briefly I agree they can be helpful and should be helpful there's a lot of expertise and experience to tap there but it's clear in my mind that it's contingent upon states of what security standards are, what they need to be including in there by the way paper boughs, paper boughs and at a minimum a voter verified paper audit trail so I just hope that the vendor community hears that loudly and clearly and comes forward with products that reflects what we are sharing as best practices and standards that's the point of all this collaboration on a related note I will call attention to a new law in California I don't know if we're the only still or for the first in addition to all the measures we take to help protect our voter registration database which is different by the way than the online voter registration availability in California many states have online voter registration separate apart from that for, there's been a big misperception when there was this brief commission that the administration set up to look into massive voter fraud which does not exist whether or not the voter database is public information the voter database is not public information some voter information is made available for certain uses like campaigns for their outreach, journalistic uses research, academia, etc we now have in place in California a requirement upon third parties if they have access to some of this voter information and their systems are breached or they're compromised somehow to notify us, that wasn't the case before and they're not required not just to notify us but to cooperate with us in any sort of investigation or audit to figure out what happened if there's any exposure there so I know that's not vendor specific but I think pointing to an example of the third party responsibility here which we have quickly gained an appreciation for okay anybody else yeah go ahead I'm concerned about allowing wireless connectivity allowing modems allowing barcodes and it has concerned me that I haven't seen more of the presidents from DHS or from election officials in that process because those voluntary voting system guidelines are going to shake all the initiatives that we have like for the next 10 years and we want to be nipping those security problems in the bud not after the machine takes certified and my question is to DHS and election officials are you on other calls that I'm not aware of like involved in the shaping of those voluntary voting system guidelines or can you be more involved letting them know what you might be concerned about thank you okay and if we can be quick with our answers because the fire marshal is mad at us again and I'll take one more question after this but whoever wants to yeah ditto but I I'm going to go to miss for years now we have not we were not previously involved in the voluntary voting standard guidelines but yes we're very involved in providing security expertise the concept is to like much of the standards development process where you're looking for expertise the working group is trying to understand different perspectives but then there will be a review process and sort of coming together and we will work very closely regarding EAC and NIST and we have folks a lot of whom are here will actively be involved in that review process just real quickly I came in on the tail end of the approval of VBSG 2.0 on the committee for EAC and so not only am I involved but where the rubber is really going to meet the road and I think you've touched on it really well is the requirements right you have that framework now with VBSG 2.0 but you've got to make sure the requirements are adequately established and so I am very involved in that and I think your points will meet well we'll go with the guy by the door since we all need to leave after this sir here you can use the mic given that we know that Russia's interfered in some elections recently probably we'll do again do you think that the Trump administration should declare it as active war to meddle in elections and then immediately put Russia on notice as a representative of the Trump administration I will say I active war I think it is no I don't think it's an active war I think again remember this is not new other nations have been trying to undermine our democracy for decades and this is just a new way of doing it and and then I would say I believe we have we have issued many sanctions and you know we continue to look for all the tools that we've got in our deterrence toolbox but sort of what I talked about earlier is I'm still a believer that you know defense a stronger defense can be the strongest deterrence and it's not easy to continue to use the tools that are unique to the government and we'll continue to do that but that's why we're here because we've got to make it harder for them and we have to do that together so hopefully that answers your question okay so with that how much time we got so look I appreciate the question and while you know cyber warfare is different than traditional physical warfare I think a threat if not an outright attack on our democracy needs to be recognized for what it is respectfully we're working together but I am not a representative of the Trump administration right one of the things that you and all of your colleagues will say it takes what we face in 2016 and our continue to face now requires a whole of government response right that's not classified you've been saying it publicly requires a whole of government response last I checked the person who sits in the Oval Office is a part of our government and as great as we're working together we still need the right words to come out of the mouth of the sitting president of the United States of America and it has not and when he comes close he always equivocates and that sends the wrong message we're working behind the scenes to buttress our defenses I leave it to those with the appropriate clearances to figure out what we're doing in response or proactively etc but I think from an election official's perspective we need three things unequivocal recognition by the president on what happened in 2016 we need ongoing resources and commitment to constantly invest in election systems and it would go a long way to hire to designate a well-respected coordinator out of the White House on cyber security because that position is currently vacant and that vacancy speaks volumes alright with that thank you to everybody who came we really appreciate your participation thank you for everybody who has questions and now we have to get out of here because the fire marshal is pissed off at us


What is up you guys welcome back to another one if you went into the channel I am Goldpony on doing new car truck SUV reviews on YouTube and today We are in the new 2021 Nissan Sentra courtesy of Hanover Nissan in Hanover PA for more information on their inventory Please feel free to check out the link in the description box below And so it is a beautiful rainy day here in Pennsylvania and I am in the new Sentra reason being is because there are some tech upgrades actually for the 2021 model year This thing is priced extremely well And so in this video I will be testing out and going over everything about the new century including acceleration braking steering Feel ride quality sound system exhaust clip all of that fun stuff So having said all of that what do you guys say? Let's just go ahead and jump right into it and as always Let's start with pricing and so there are a few different trim levels for the 2021 Sentra first one being the s starting at $19,460 sv which is the one we have today starting at $20,520 then the sR starting at $21,800 but regardless of trim level that you go with the power plant on the Sentra is going to be the same Powering this little beast as a two-liter direct injected inline four cylinder putting out a hundred and forty nine horsepower at 6,000 rpm 146 pound-feet of torque coming in at 4,000 rpm power sent to front wheels through a CVT Zero to sixty time comes in at approximately eight seconds flat with MPG numbers coming in at 29 in the city 39 on the highway taking regular Unlit fuel and so before we do any kind of fun acceleration tests here in our new Sentra I did want to mention to you guys that there are actually a few different drive modes Of course, there is normal that is what the Sentra naturally defaults to but by the driver sign Left knee there is an eco button that is another one and there is a hidden sport mode on the Sentra I say hidden because it's not labeled sport It's simply a horizontal line button located on the back side of the shifter here And so if you press that like I just did it did it immediately downshift for me So it is gonna hold the RPMs at a much higher level It's also going to adjust the throttle response then as well So do what I mentioned that you guys because that is kind of the hidden drive mode Then not a whole lot of people know about unless you're watching my videos of course to go ahead and hit the subscribe and the bell notification By and say you stay up to date on all that But anyways now I've got all of that out of the way What do you guys say? Let's go in and find a straight away here and let's see how quickly we can get our new Sentra here up to speed in the rain. All right, here's our straight away guys in three two one Little bit of slippage one is wet out, huh Okay, I gotta say I okay I was focused on the acceleration But what I was kind of more focused on to be honest was the CVT Because I gotta say that actually simulated a traditional automatic quite well Which is a very good thing in my opinion because CVTs are traditionally known to be somewhat Emotionless because it's like one giant rubber band kind of acceleration But when Nissan did there with their CVT on the 2021 century at least it actually simulated an automatic quite well I didn't mind that acceleration for that particular reason. So well done Nissan that kind of surprised me and the acceleration was I don't know it's pretty much as expect not the quickest thing in the world But it'll merge you onto the highway without any issues But to go along with that acceleration as always breaking is equally important And actually the brake configuration is going to differ amongst the trim levels believe it or not For example, if you were to go with the S trim level, you're gonna get front disc rear drum brakes However, if you jump up to the SV or SR, you are going to get four-wheel disc brakes So I wanted to mention that and if you do go with that four-wheel ventilated disc brakes You will get a 60 to zero stopping distance coming in at an impressive 114 feet and so a lot of times with sedan's you're in the 120 So 114 feet is dang good if I'm being quite honest So well done Nissan for that and as far as the braking feel goes. It's literally perfect. There's no brake pedal delay There's no soft spots. It is a very nice braking feel if I'm being honest there But continuing on touching on suspension and handling up front You're gonna get an independent strut type front suspension in the back independent multi-link rear suspension Of course front and rear stabilizer bars as far as ride quality goes. It's actually not that bad I was one of the first things I noticed a lot of times with compact cars You do of course feel a good bit more of the road But it's not bad comparatively speaking to some of the competition that I've driven as well So ride quality is perfectly acceptable in the center without a doubt steering feel now That's the first thing I noticed starting in the 2020 model year for the Sentra the steering feel got noticeably heavier Which is a brilliant thing because it better points you in the direction that you want to go It's a much better feeling of being in control here of this compact car So love the steering feel in the new Sentra without a doubt well done Nissan especially for that The touching on cabin noise a little bit you do hear a little bit of the engine but as far as that goes That's pretty much about it There isn't a whole lot of exterior wind noise coming into the cabin as we ride over a bridge right here still not all that bad It's pretty much as expected the engine noise is the only thing you get when you really hit it But other than that is perfectly fine touching on visibility There is no flaws to the shape of the Sentra so therefore visibility is perfectly fine I can see perfectly fine out of my rear view mirrors who definitely no issues there, but That about rounds out the performance segment of this review you guys Let's now go ahead and take a look at the exterior of our brand new 2021 Nissan Sentra all right so here she is you guys the new 2021 Nissan Sentra finished in scarlet ember In case anybody was curious of our exterior color name Let's go ahead and start up front on this one first thing I want to mention is a feature that you find on BMW and a lot of other luxury manufacturers But active grille shutters actually do come standard across the board meaning whenever the engine needs any additional cooling Those shutters are going to open up and then they will close back up Then when that additional cooling is not needed of course so as a pretty cool feature I'd like to mention to the sides halogen headlights do come with the S and SV trim levels They do come with the automatic feature meaning when it starts to get dark at night They will turn on automatically for you there LED daytime running lights also coming standard You will actually get LED headlights though if you were to go with the SR trim level Of course, that is not what you guys are looking at right now Fog lights also coming with the SR trim level as well just below in the center though Chrome V motion front grille is going to come with the S and SV And then you will actually get a dark chrome V motion front grille Then if you were to go with the SR butt pretty much rounds out the front Let's go ahead and make our way now to the side of the center here So since we are around to the side first thing I want to mention is there is actually a two-tone color scheme That is available for the center like a black roof for example It's a pretty cool look to it So I did want to mention that chrome belt line molding does come standard on the center You can find a floating roof line along the C pillar towards the back there I think that's a pretty cool look take a look at the side mirrors They are power adjustable side mirrors for all trims They will be heated for the SV and SR trims body colored for the S and SV and then gloss black side mirrors for the SR Then an SR is also actually going to give you LED integrated turd signals So I did want to mention that as well Let's take a look down at the wheel setup 16 inch steel wheels with covers for the S trim level 16 inch aluminum alloys for the SV So therefore that is what you guys are looking at right now and 18 inch aluminum alloys then for the SR trim level I did want to also mention though that SR trim level is also going to give you essentially a Lip kit around the entire exterior of this thing So a little added side skirts for the SR trim level at least as well But now let's go ahead and make our way to the back of the center So since we are around back of this one body colored shark fin antenna all the way to the top You guys can see that just below that a rear spoiler is actually only going to come standard with the SR However, it is optional of course for the other trim level So we do have it here on our SV just below that you can find some trim level badging as well And just below all of that you can actually find a body colored rear diffuser Which is pretty cool coming with the SR and SV trim levels and just below it all There is actually a single exhaust outlet. It is tucked away, but having said that I do believe you guys know what we have to do next as always here is that exhaust clip All right, so now since we are around back of the center to go ahead and open up that rear trunk There is a button that you hold on the key fob that is one way There's also a button on the trunk itself and there is a button by the driver's side left knee then as well But once opened up cargo capacity comes in at 14.3 cubic feet if that was not enough space There is a 60 40 split meaning the rear seats do fold down bumping that up quite a bit then Also wanted to mention to you guys in that cargo area There is some cargo lighting back there And if you were wondering if the center had to fix the flat versus a spare tire If you lift up underneath through that cargo floor, it actually does have the full spare tire back there So that is pretty cool. That's what I prefer personally But the make our way to the rear leg room that is going to come in at 34.7 inches. So for reference I mean even six feet tall. This is how much space I had in the second row there Do you want to also mention though? I was able to find a rear center armrest with cup holders, which is pretty cool And there is a single usb charging port back there as well for the kids to stay charged up So that is definitely a necessity in this day and age, of course But so the make our way to the front seats manually adjustable cloth seats do come with the s and s v trim levels You will find a sport cloth with orange stitching then if you were to go with the sr Quilted leather is possibly the best very best setup that I've ever seen in the center That is going to be optional for the s v trim level only So this trim level that we have today you can actually get quilted leather in so that is a pretty cool Look if you guys want to look at Nissan's website for that one That's pretty cool heated front seats then are going to be optional for the s v and s r trim levels then as well And overall though as far as the seat comfort goes that we have today in our cloth seats I will say it is perfectly fine. I've had absolutely no issues with the seat comfort I could easily see myself going on a long road trip here in the new center So no issues there then take a look at the steering wheel It is of course tilt and telescoping and is leather wrapped if you were to go with the s v or s r trim levels And then heated steering wheel is actually going to be optional for the s v if you wanted that as well Then make your way to the startup Let me first start by showing you guys the key here Essentially all of your buttons are located on one side of the key Nissan logo at the top lock unlock button pop The rear hatch the circular button near the Nissan logo that is going to be your remote start which comes on the s v and s r trim levels only But then if you were curious as to the push button start that actually does come standard on all trim levels So therefore all I am going to do here is simply put my foot on the brake and press that engine start button located just in front of the shifter and so Once started up tachometer is on your left speedometer is on your right There is a fairly large digital display front and center to control what is on that digital display There are steering wheel mounting controls found on the left side of the steering wheel Gives you a ton of different information like sure bait sure be of course But also how many miles you have left until you hit empty which when I started driving this one said 365 so that is pretty darn impressive Especially in a compact car digital speedometer you could check out up there as well average fuel economy at any given time Of course tire pressure information radio setting safety information the list goes on really just about everything could possibly want up there But then make your way to overall interior quality if you wanted a power mooner for the center It is going to be optional for the sv and sr trims It doesn't come standard on any particular trim level But you can still get it if you wanted to go that route dual-zone climate control coming standard on the sv and sr So both driver and passenger can set their own temperatures there Auto dimming rear view mirror is going to be optional for the sr And I did want to mention to you guys We actually do have a package option that gives us a Frameless rear view mirror with homeland controls to up to three different garage doors I found that pretty cool. It is a $440 option But I love that because the alternative is the clunky rattling garage door opener that clips to your sun visor So I do like home like whenever that is available for sure Also another option we have here is something called the lighting package that goes for $500 that gives you ground Illumination at night and also interior ambient lighting then as well But overall interior quality is pretty much as I expected it to be there are some hard plastics But I do like the contrast stitching found just above the passenger side glove box I like the circular air vents. They're very outy-esque. So that is a pretty cool look for me Just in front of the shifter you have a 12 volt power outlet usb charging port phone charging port an auxiliary charging port Kind of touching all bases there just beside the shifter You have dual cup holders just behind that a little bit of storage there as well And within this center armrest you can find a very deep amount of storage actually for a compact car I will say that so overall the interior quality that we have here in our particular spec today It's not bad But like I said, if you go with that optional quilted leather look for the sv trim level That is going to be the very best interior quality that you can get in the center if that is important to you But then take a look at the tech display. It is a 7 inch color touch screen display for the s 8 inch color touch screen display for the sv and s r bluetooth and audio streaming coming standard either way But here is the main new feature for the 2021 centra Android auto apple carplay is now standard for every single trim level Whereas it didn't come with the s trim level in previous years It now does come with that base trim level of the center So you got to love that essentially with that is if you have a smartphone simply hook it up to the center via usb cable And you have free navigation displayed up on that tech display without having to buy any factory navigation system Also, you can like just like your pin door songs. There's a bunch of different compatible apps So that is definitely a big deal to me at least but anyways You can also check out your radio information up on that tech display And by the way when it comes to the sound systems Four speakers are going to come with the s6 speakers coming with the sv and s r And there is one more optional sound system available. That is going to be an eight speaker bow sound system We don't have that today. We do have the six speakers. So what do you guys say? Let's go ahead and turn on serious xm here. See what we got playing today and let's test out the clarity of this one All right, so that was kind of a mixed bag for me I got to be honest the base was pretty good for a six speaker sound system But when you turn it up a little bit louder the clarity does get a little bit muffled the higher in the volume that you go But then again, it is a six speaker sound system That's kind of what I expected but not the clearest six speaker sound system that I've ever heard But last thing I wanted to mention to you guys on that infotainment screen at least is When you do put the center in reverse you will find a rear view camera that takes up the entire screen Of the infotainment screen here, which isn't always the case. That's why I emphasize that which is always though Is going to lead us into safety and so to start IIHS top safety pick If you go with the s r trim level because it needs those led headlights in order to get that designation But front side side curtain airbags do come standard driver and passenger knee airbags as well in the back You're going to have latch aka lower anchors to the tethers for children for the rear car seats rear child door locks tire pressure monitoring system That's all pretty boring at this point But the good stuff that does come standard across the board by the way Forward collision warning autonomous emergency braking with pedestrian detection lane departure warning rear automatic braking a blind spot Monitoring system with rear cross traffic alert rear parking sensors the driver attention monitoring system high beam assist and lane trace assist as well That is a good bit of standard safety even for the base trim even for the s trim level I should say so that is wonderful Well done you some for that But then if you were to go with the sv and sr You're also going to get adaptive cruise control then as well So overall when it comes to my final thoughts here of the new center a great steering field That's always the first thing I noticed when I got in the 2020 model year and the 2021 very heavy So I love that great braking 60 to 0 and 114 feet is quite brilliant if i'm being honest Also great pricing great price points for every single trim level really a lot of times when you look at other manufacturers Let's say the civic it can get quite pricey But I will say the center is priced quite right also in that quilted leather That is available has a very upscale look very outy-ish look So I love that although we don't have it today of course But as far as the room for improvement goes one thing that i'm starting to notice more and more And i'm sure it's going to come to the Nissan center as well is wireless android auto apple carplay I love that it comes standard on the base trim level now But the next step is going to be wireless You don't actually have to hook it up and you don't have any cluttered wires everywhere So that is one thing I would look for in the future and of course it's not the quickest thing in the world It doesn't do bad when you put it in that sportman that I told you guys about remember That's the horizontal line just underneath the shifter there But still could be a little bit quicker, but anyways that about rounds out this review guys Thank you so much for watching feel free to follow me on social media at the bottom of the screen there If you wanted to see what's coming next on the channel before it actually gets to youtube Be sure to hit the subscribe and the bell notification button if you're in a new car reviews That is what we do here on this channel after all Do appreciate you guys watching more than you know when I will see you guys all in the next video Stay gold


Arms feeding war. They need him to get busy on the box Jump hook counting remember these two kids wesson at woods Just up because you think you know so much about a player because you played against them CJ Jackson nice time by Kyle Young Here's a double team for war Winston reverses it McQueen pure bullet McQueen one-point lead for the Buckeyes Winston To the hole the floater goes down. He's got a lot of tricky stuff Cassius Winston Top of the arc till Ward snatched it down goes the opposite side and lays it in that's a rebound No, that's a that's a grown man rebound right didn't some feel that this Michigan State team can win it all this year Backdoor Washington for first Jackson pushing it Jackson a three Major work Jackson Andre Western is crowd ready to go crazy Washington Wesson to screen for Jackson Western steps out Three 18 first half points Winston cross down traffic nice box pass With the layup That's the aggressive nature, but a key shot was kind of fell asleep and aren't either would take a debate take advantage of it He has five points now need picked up Winston the other way hesitation gets to the right To the mast and that was all Cassius Rotation that time by the Buckeyes opened up the post pass Yes, he did Winston a lot inside and in nicely done as Xavier Tillman finishes Here's Winston hesitates and gets to the cup again Cassius Winston at seven and half time Winston 20 footer Cassius Winston 18 now That's a big fella again. Here's a double tee Leaves his feet wait open It down Andre Wesson denied the baseline skips across the lane As it takes it up strong Washington trying to shake up hit can't do it more to go Washington D Washington Contestant shot Nick Ward doesn't box out ends up on the poster Cal Young making his presence from not only in the defense event But not giving up on the play Winston What a pass by Cassius Winston it's about patience and wrapping it around how about the kitchen gathered then by Ward? Caleb Wesson with it on the box guarded by wick Nick Ward. Oh No, oh school big ten basketball Wesson can't finish gets it again any strip by Henry Call it good for Wesson with it baseline 18 seconds to go Washington Rims off the plate hauls it down and that'll be it Michigan State will walk it into the front court and the Spartans will hand the Buckeyes their second loss of the season Michigan State improves the 13 to 2 4 and 0 in conference play Ohio State suffering their first conference loss of the year 86 to 77 the final school


Welcome back everyone, the Cube's live coverage in Las Vegas, I'm John Furrier, host of theCUBE with Dave Vellante, we're here with SAS, at SAS Innovate 2024, we're back with two great guests, Brian Harris, Chief Technology Officer of SAS, Marinella Profi, Global Product Marketing Strategy for AI and Genet, both stage performers today on the keynote, Brian, Marilyn, thanks for coming on theCUBE again, great to see you. Thanks for having us. Great to be here. So, we were just at your event last year, we said, you said some things, put some things out there, this is what we're going to do with AI, Genet of AI, it was all the hot rage, chat GPT, really innovates on the user interface, check, we see that. Getting things done is a whole nother ballgame, you guys got some huge announcements, the models, I love it, let's just get into what's going on right now, what's the key things for your transformative technology around Genet AI? Let's just get to Genet of AI out of the gate, because everyone's talking about it, Huffin and Puffin, which is again, it's a really exciting technology, but it has to be put in context to actual business outcomes, and so, I think for me, I'd like to actually give it over to Marinella a little bit, because she's a day-to-day data scientist out there, and on the ground, meeting with customers, hearing the feedback, and so Marinella, why don't you talk about, why don't you talk about what you're seeing on the ground? Absolutely, thanks, Brian, yeah, so from my perspective, I'm a data scientist, and I think the number one lesson that I've learned, and there is a lesson that I've learned, well, the past year and a half has been, LLMs, Larging with Models alone do not solve business problems, like, this technology is so cool, and no matter how cool it is, it's still imperfect, and it's all going to come down to solving real-world business problems for specific industries and specific people, right? We're seeing a lot of companies just saying, we have this technology, let's go and build it, let's see, here's how we're going to do it, so a lot of it's around how do we build something, but that there is very little around why, why are we even building it, who's going to serve, but what's the value that's going to deliver, right? And so, for me, from my perspective, what I've been seeing is that, you know, Genevry.TVI and LLMs in this case are like the 20% of the entire value chain, if you consider a real-world use case, and I think, you know, what Brian shared today with Georgia Pacific was an incredible example of that, I don't know if you want to add more. Yeah, definitely, I mean, it creates some point, right, that, you know, Georgia Pacific is on stage talking about their use of a SAS Viya as part of our Genevry.TVI workflow, I mean, there is a lot of moving parts, and there is no more kind of validated feedback on a production use case at Genevry when they're actually trying to run their plant, like their plant operations is leveraging Genevry.TVI, and as part of that, they're getting corrective actions through this Genevry.TVI experience that is leveraging SAS Viya to orchestrate across LLMs, orchestrate across quantitative reasoning needs where we're interacting with enterprise systems, so all of this orchestration is happening ultimately to give a human language response back to an operator who is three, five years' experience in their job, and they're trying to convert them and upscale them to a seven, you know, five to seven year employee. There were a lot of components to that stack. Yes. And you think you get a picture of it here. You had Bedrock on there, you had SageMaker, you had SAS, you had another couple other pieces, and so you see Genevry.TVI as the orchestrator. Yeah, for us, at some point, you got to integrate all of the tech, and what you're really getting to is you're making a final decision to give a response back to a user. And as we know, a lot of these cases right now are very shallow. They're just prompt engineering to an LLM and come back. And yes, RAG architectures are there, but we got to get things done in sub-second response time here. So there's very, very challenging integration and expectations of responses from Genevry.TVI workflows that we have to get done. So for us, Viya sits at the center of that, and even Sam Coyne mentioned it today, that we really bring all of these things together. We can augment prompts, we can orchestrate against other traditional models and things like that and bring that data back in so that the LLM can reason over that and send back the response back to the user. An answer to your question, I think Viya is orchestrator here, not Genevry.TVI. Genevry.TVI is a piece that can augment existing processes and Viya, our data and AI platform functions as that orchestration of the entire positioning flow and that's exactly what we were announcing today. We've extended Viya to allow developers to use our API to govern, orchestrate, and build real world use cases with Genevry. And I think the point about Genevry.TVI is an element of the stack, the higher end of the stack. I have a picture, I want to get your reaction. Okay, you built on AWS, so got some hardware, got some cloud. There you go. Sass and intelligent decisioning, that's your layer, right? That's the- Decisioning system. Decisioning systems and the multiple, real time data knowledge bank process model. Okay, that's the meat and potatoes, that's the stuff that's the key jewels to make that thing work. And then you got the G&A on top, managing it, and then you got the outcomes. Yeah, and so like, yeah, so do you think of it as that something has to bring all these things together because right now, a lot of the reasoning capabilities in text for LLM is around, you know, textual fact-based reason responses. But a lot of the decisions in the enterprise are quantitative. So that's why we pride ourselves on being in the quantitative reasoning engine inside of Genevry.TVI workflows. And intelligent decisioning is such a flagship product for us because it allows us to, we can bring four or five different models to actually integrate across text, across the LLM itself, across the actual enterprise systems we're integrating with. I want to connect it to something you said in the media briefing, if I could, which is about ROI. I mean, if you look at how people are using Genevry today, it's very chat GPT-ish. Yes. You know, so it's not literally blowing away the cash flows. I mean, it's not really paying for itself. And so when you start talking about these industry-specific models, that's where you're going to get return. And people are pushing out their ROI expectations based on the surveys that we have, right? And so, you know, they're being a little bit more conservative because they're saying, okay, this is fun, this is cool, but we need real dollars. So when you open up a prompt to thousands of people in your organization, the quality of your questions is now your biggest concern, right? And now people are paying for this. So now, okay, well, that's why you saw us talking about stored prompts because our ability to say, all right, look, there's not an infinite set of questions like in the public domain with like chat GPT. In an enterprise, there's a finite set of questions. So why don't we understand those questions you want to ask, make an inventory of those, use Viya to kind of do the semantic analysis on this and then say, when three, four people ask a question, we're going to get the question that the right question you're asking. And then let that then drive, reduce the error to the LLM, reduce the error to the vector database, reduce the error to the quantitative reasoning with Viya as well. I thought that was a very nuanced point. I thought one of the things that jumped out at me was the stored prompts because Dr. Goodright was actually bragging about the compiler versus the interpreter on stage. If you think about that, as you get smarter with these stored prompts, you can learn, and that's like compiling away the LLM risk. What's funny about it is if you remember, what did database providers want to do? They wanted stored procedures. And stored procedures was their way to govern the way to build optimized queries for databases. So stored prompts are really just an extension of that entire model. People are doing this in all kinds of use cases to get smarter. You make a prompt, you get an answer, you come back, it's back down to the architecture, why I like that graph and why I called it out was, that's a production workload. The only thing that's outside there is the customer's environment in Amazon and your SaaS stuff. So that brings up the kind of the big picture here. You got training and inferences, the big discussion on how to deal with data, and then you got prompt response, and then you got reasoning in the AI paradigm, all contextual and behavioral. So you're essentially rolling all that up. So, okay, prompt response, that could be real like IO. Reasoning could take a little longer maybe, or so I got to run that somewhere. How do you see that applying to known end-to-end workloads? Because this is the theme that's popping into this AI world and it rises. I got known workloads. I can scope them, I can qualify them. I think the way it rolls up is that we're seeing the, definitely the agent design pattern is emerging. In fact, in our Viacopilot demo that Maranella was showing today, we're actually doing some iterative work where we have one generating the code, right? One part of the orchestration is generate the code underneath it, execute the code, if it fails, tell the L, if it failed, it'll make the corrective action on it and execute it again. And then we'll get the actual- No human. And then it's why you saw like a map on the screen or you saw some of the extra follow-on visuals or suggestions. So that's why this orchestration is so important because it really is an iterative process. It's not a single request, single response experience. And so for us, this reasoning thing, if you think of reasoning, reasoning has to span, you know, text-based, fact-based things, media, like images and video. And, but most importantly, in the enterprise, quantitative reasoning is really the majority of the workload challenge. It's not about just, you know, I mean, customer service, yes, I got a knowledge base, I can give you answers on it. But a lot of the enterprise-class problems are, how do I look at numbers, analyze numbers, and make a recommendation on the next best action? And that's why I asked that dumb question in the media thing. That was not a dumb question, by the way.  Well, I was trying to get it. I came out wrong. I've seen a lot dumber questions in there.  But the point is, customers today have to go to different models for different models. So they want one place. They want to have the reasoning and all this. It sounds complicated, but put it in one place. You can do vector embeds for unstructured data. I can bring a table in. I can bring in a DNA sequence. That's the mode, right? I mean, all kinds of data can be structured that way if it's in one place. I don't want to go five different places. And then do you have the technology, I think, that you can talk about, why don't you give the example of the banking one we talked about today? Correct, yeah. Where you can really talk about all the tasks that were required to really make it work. Yeah, so this is one example that we were working with a global bank where they had, they were doing complaints management, it's a simple use case. Like it's not revolutionary, but like the question was, how can I augment what I'm doing with generative AI with large language models? And the integration of the LLMs is not just easy as, oh, give child GPT the complaint and generate a summary. And then based on that, it'll ask to generate a reply to the customer and send that reply to the customer. Like that's not how it actually would work because you have so many other pieces in the middle. And in this case, the flow that the developers were able to be able to actually productionalize that use case involved other things like, okay, the summary is generated by the LLM, but how do you explain if that's accurate? How do you explain how you generate that summary? And so that's where natural language processing techniques, which is actually a method that has been proving to be really critical and fundamental in explaining some of these, the output generated by large language models. That's, and then how do you integrate your customer data within that summary? The LLM doesn't know if your customer has a high credit score, is a good payer, is not a good payer, like the credit, those are your enterprise data. The LLM, and unless you want to give that to the LLM and expose them to the LLM, which I don't think that's what our customers are asking to, you need that quantitative decision engine that augments that information and sends back that information for the LLM to actually generate that summary and to generate that email in a more reasoned and a way that makes sense. I mean, if you think about it, we compress all the complexity of the quantitative side into a much more consumable fact that could be presented back to the LLM so we can use the strength which is to generate natural language response. Which, so that might be a score, for example? Yeah, yeah. And then the LLM will take that score as opposed to trying to generate the score, which it's not good at. It's actually horrible at that. Exactly, that's horrible. You said it well, right? I would say it exactly that way, it's terrible. Dave, he's a SaaS user in our, he used to program in SaaS and that wasn't bad. We appreciate it. We appreciate it. I was born in that year. For those about the proc, we salute you. Yeah. And if your customer is complaining about your credit, your service, the LLM is not going to be able to generate the next back action for your specific customer because that's to know your customer, doesn't know what the next back action is. Since you're a customer, I want to get more use case information of what you guys see in the marketplace. As Dave and I always talk when Dave does some research on the monetization, that's really going to be the scoreboard. Are you, is it adopted in production when you guys are showing some examples of greater than others? And the monetization of AI, you see the early adopters, some are monetizing really well, so that's also a tell sign. Monetization, uptake, and then customer success. So, you mentioned you're doing well with the business performance. And you spent a billion dollars of investment and spending more on the tech. So you got tech investment and monetization. Good job. What do the unit economics look like? What's the use cases that are emergent? You can point to it and say, we're starting to see the things line up in the market, the fogs lifting in the enterprise. What is it out there and how is it being monetized? Beyond the chit-chat, right? Yeah, yeah, yeah, I'll get it there. I mean, I would say that we see a lot of customers worried about yet another metered challenge to the business. I mean, there's CFOs saying, look, we're trying to run a profitable business. And look, if you can tolerate metered billing when you know you've got enough room on the financials to tolerate it. But when you start to, remember, we're lowering the barrier to interaction for data and AI across the entire enterprise. When you do that, you're actually encouraging usage of these metered services, which is fine. But you have to make sure that the use cases are driving an ROI that can support those metered interactions. And then we have other customers are saying, can you take me off the meters? Can we just run? It's my data. I want to bulk upload my own data. Yeah, yeah, exactly. So we want to respect all of those, right? Many customers, we're partnered with Microsoft. We want to leverage their open AI services where possible. We're working with Bedrock and Amazon and AWS will do that. But there are customers who are like, hey, I want to run with Mistral. Like I want to go and I want to run that in a GPU that's hosted in their environment. So the same cloud spend conversation, it was really... They want optionality, but they want to have confidence on what the spend's going to look like. Yeah, absolutely. Have some scope. Yes, because what's happening is everyone, again, the FOMO that was happening about, just go run to this thing, is now being realized in the financial outcomes of these businesses. These are big customers. I mean, can they get GPUs? I mean, if you're not spending 10 million bucks on video, you have to get it. I mean, I think when you're training models, it's one thing, right? If you're just looking to go and do the, hosting them from scoring these models, I think there's options there, right? There's a lot of things coming down in size on it. Now we're just going to make sure that our stuff can react to any one of these scenarios. Like if you want to use a cloud service provider, we're happy to do it. Some of the customers just want, that's what they want to do. So we'll leverage it. But we're seeing others now starting to say, let's bring it locally. Is it tracking like DevOps did? Do you get a use case and you experiment with something? Or is it end-to-end workflows that are emerging? What is some of that? Or is it the data that's on? I think first and foremost, I think there's a little bit of a hammer and nail problem where everyone's saying, let's start with generative AI. When that really, generative AI is a subset of AI. It's not a superset of AI, which is a big problem because it's been presented in the market as if generative AI is this new thing. It's actually a subset of the larger AI umbrella. In fact, the problems being solved by the rest of the AI space are much larger than the problems being solved by the LLM and generative AI space. And some of the best models are small. Yeah, yeah, exactly. I think if that's going to be the future, where the smaller language models are, yeah, smaller language models, that would say almost auto fine-tune themself for specific industries and specific use cases are going to be the future. And that's because they're more accurate and faster? I think it's like anything else. If you go to the public, the large language models are out in the public are just, they're kind of generalists in these things. And they're powerful for many reasons. But they get vanilla. Right, exactly. If you want to use an example here, and I mean, Google was trying to, remember when Google was so huge, they wanted to get into the enterprise space, right? How could we put that Google engine? People were going home and they were having a Google experience for search and they would go into the business and say, why can't I have that same search experience in my business? The use case was different, right? Google's indexing engine was about finding information authorities. We, while in the business, it was about quantitative analysis. So they never could really penetrate that market. We're having the same challenge now with generative AI, which is that you can have a very public domain question and answer experience with LMS that are very powerful. And you want that capability on the enterprise, but there's more work to be done. And what people are realizing now is how much more work there has to be done. Especially for us, we're in regulated industries. So we can't be wrong. So I can't just go and run. Yeah, I can't wing it. Yeah, I can't wing it. We've got to go and work meticulously with them to understand here's what happens when you do this response. And if that response gives an answer and you need proof of it. Yeah, there's no space for hype. We need lineage. You need all the things that go behind it. No space for hype for us. And what I would add is also that, I mean, my question is, until what point this gigantic large language models providers are actually realistically going to have data to keep training these huge models? Yeah. Yeah, they're going to try it. Meaning they're going to run out of data. They are. It's called strip mining. I think so. And either it's because it's going to become a legal issue or a point where you can't take my data anymore to try and whatever you want to train. Or there's just no more data. So we have to go back to synthetic data. Can you believe we're saying that? I know. We're running out of data. Adobe is charging $7, I think, a minute for the video, right? Yeah, yeah, yeah. And that's why a data maker for us is a big announcement. Correct. So that's data maker for us. It allows us to attack the structure, the gap in structured data for synthetic data. And by being able to do that, we can go into regulated spaces and give proof to the statistical congruence of synthetic data that can bootstrap models that will allow them to run effectively on real-world data. You got to have explainable AI as table stakes and security built-in. Great to have you guys on. We're out of time. Final question. They got the hard stop. Oh, that moment. I was enjoying this. That's what I got. I'll get one more question in. Good. You got one more. Maybe two. So next, great step up from last event. Thank you very much for bringing all the goods to the table. Next year, when we're sitting here, what are we going to be talking about? What's going to be the biggest discussion with Brian? We'll share your thoughts with Brian. Yeah, I think that what we're about to achieve with that factory, right, we're still, we're building our own solutions with it, but the productivity of that thing is incredible. When we start wrapping some generative AI experiences on building applications that are, the treatment on that's going to be incredible. I think what you're going to see on our Viacopilots work that we've just done and demonstrated today, I mean, the ability for our Viacopilots to be a very practical approach to the AI lifecycle is, I think it's going to be world-class and second to none. That's where I think I'm right, we're at. What I am really excited about is to continue. So the strategy that we have taken over the past two years is to have purpose, like building something with a purpose, right? And I am really excited to come and see that come in action. Like we're not just a startup that is going around what looking for funds from a VC. We're building these things for actually being used in the world and creating real world value. And I can't wait for that to continue to be and have even more, like I want to come here next year and talk about other 10 customers that we've done and how all the models that we're going to sell. And so that to me, what I'm really excited about. That billion dollars in investment, you're monetizing well, which is proof. And you got production workloads. I mean, just keep it coming. Yes, and I would say too we will have a significant amount of progress on penetrating the market with selling our models, which we're really excited about. I got to ask one more question. We got time. Ask for forgiveness for you. You said in the media briefing, nobody's looking at the impact of AI the way SaaS is. And today it's very clear you're talking about very focused on solving a lot of business problems. How far out are you looking? I mean, What do you mean by that? In terms of the impact of AI, AGI. If we have AGI by the end of the decade, I mean, are you thinking that far out? And then if you think about exponential growth of computational power, mid-century. I mean, this, when you make that statement, I have to ask you like literally how far out are you thinking? Is it more? I think we'll have tools that we'll be able to evaluate. We'll have capabilities that can evaluate the reality of these models.  So there's like certain kind of workloads we put against them. We try to measure them on grading these things, but they're not measured on the enterprise's actual workloads. So how do we think about what does the answer look like in an enterprise? And how did that model react to it? And then being able to go and say that work, that didn't work, starts bringing some true in-customer validation endpoints to what are we doing with generative AI? So, I mean, I think that's the area where you'll start seeing what these things will have to be validated independently of the model creators. Well, the stars of the show, you're at high demand, you got to go, we're getting the hook, you got to leave it there. Okay, we're going to continue the conversation here on theCUBE, we'll be right back after this short break. I'm John Furrier, Dave Vellante, we'll be right back.


The proposed method uses a local binary pattern histogram and hard cascade classifier to recognize multiple faces from a single image. It achieves high accuracy rates when tested against both local and real time streaming data. This method outperforms other methods in terms of recognition rate, making it suitable for applications such as surveillance or biometric authentication. This article was offered by Ari Solis-Nanto, Aidean Rakhim, Dania Aridani, and others.