All right, we're recording Hello, my name is Adam Miller. I am the newest member of the Thorough release engineering team I currently work on Thorough engineering before that I spent three years working on OpenShift Online for new ways through with OpenShift. It's it's a lot of fun But today we're talking about next generation tooling in Thorough release engineering and that's a half lie Because it is true that we're gonna talk about that But before we talk about that we're gonna talk about some background topics and some of them are gonna be kind of fundamental to understanding What we are today what we were yesterday as a project is a distribution of what we're going towards tomorrow And then what that means to release engineering in terms of things that we need to cater to goals that we have tooling that we need to either Alter refactor or come up with completely new in order to deliver those things So background so we're gonna start off with what is fine release engineering We're going to define what an operating system is and that seems very kind of ground layer like but just base level but When we add to that defining what application isn't discussing the separation or possible separation of those things It gets it gets interesting and then what is a package? What is an distribution and how these things correlate to both Fedora's project as well as to release engineering from there We're gonna talk briefly about Fedora rings and where we're going with that if from a release engineering standpoint from a Compose and build tooling perspective what we have in place today what we're gonna work towards tomorrow And I see Ralph in the audience and I realize I forgot something one of my slides that we were talking about for release tooling I'm gonna mention it and I'll add it later So background to kick it off. What is release engineering and I shamelessly stole a nice formal definition for Wikipedia So it is a sub desk the sub-discipline of software engineering concerned with compilation assembly delivery source code to create products and saw other software components and The goal for Fedora anyways, and and I put in there that surely I just because I do and this is There's two URLs here and because of the contrast It's kind of a difficult to see but I believe slides will run up on the Internet at some point You put them read them and there is the overview of what for a release engineering is In a more detailed context and what that means in definition of roles and responsibilities And then as well Thorough release engineering philosophy and that breaks down Basically what these two lines come up with is maintain sanity in the pipeline going from upstream source code into Something that is consumable by the Fedora community in the user base And produce and maintain tools that facilitate in that goal So in in a nutshell that that is kind of what our our aim in life is to be and what we want to deliver to the community and and to the project and It rolls around this whole concept of build sanity and build reproducibility and other attributes of that so Before we talk about The tooling and what it means to release engineering these different concepts. I want to kind of ask a few questions and define a few things So what is an operating system? Operating system is software that manage your computer hardware and software resources provides common services computer programs Also shamelessly stolen from Wikipedia And so at a fundamental standpoint This is what allows us to interact with our hardware. It provides the core components that we need to do everything else And what is everything else well generally everything else is considered an application at least in current standards of definition and nomenclature and common I guess paradigms of thought process around what a computer experience is not just you know the operating system But everything and that you know kind of branches and bridges from what we do with like servers and big iron Mainframes down to what we do on tablets and cell phones. It's it's What does that mean what does it mean to be an operating system? So that kind of brings us to the next one was an application Applications a set of computer programs designed to permit again. I stole this from Wikipedia I'm just shamelessly like I have my credit lines and I'm happy with that So does I permit the user form a group of coordinated function tasks or activities application software cannot run on itself But it's dependent on software execution and what I love the most I Felt like it need to be said again is dependent on the system software to execute. So it's dependent on the operating system and That raises the question That I first heard from Alex Larson I don't know where it originated entirely But I heard it from him in a talky cave and I thought it was fantastic and I like to bring this topic up again Because I think about a lot Probably to an unhealthy extent Just like I think about Self-hosted versus non-self-hosted operating systems and primary languages and these are things to keep me up at night because I am Different and that's okay, and I'm okay with that and it's healthy or not healthy But one of my favorite tag lines because I'm a packager and I came into Fedora as a community packager And I love packaging and that's very weird and I get strangely passionate about packaging software systems In in different language, you know framework So, you know like in Python and Ruby and NPA like node.js and this like kind of clutch that is kind of go packaging That's anyways What's a package? What is a package if we have an operating system? We have an application and there might be some definition of where one in the other stops What is the package? Well the package is is effectively a build artifact from source code that be that can be consumed and that can vary in terms of what it is based on content versus code because if it's content It's like icon to wall papers or documentation, but if it's code then it becomes An application, but it could be part of the operating system because the base OS is built out of packages But everything else as well. So when you get into the world of packaging systems At least as a distribution is built and we're next slide is what's a distro In a distribution is you have a package format. So for Fedora, it's RPM for rel is RPM percent of RPM That is the thing that we centralize on it's the common component and it's very powerful in nature And there I don't see any reason necessarily get rid of it or try to Change what it it does it's you know functional level of the ground zero, but What about things that aren't rpms? What about things that people in the community? actively use that are flourishing on their own things like Python pip and Ruby gems and no JS npm and Maven and others that I probably am not very familiar with. I Know like the c-pan is still a thing You know, there's peckle and pear and like all of these different You know systems out there that very large groups of people in very vibrant communities in their own right In their own piece of the technology world are very interested and enthusiastic about All also the point that I'm enthusiastic about RPM. I'll be not as much. I get a little Animated in my excitement and that's that's that's good, but Where you know, we're kind of in the stack. Do we differentiate how much of that really needs to be brought into things and? In in a Fedora context Actually, I don't want to get myself in our context We kind of roll towards though like what's a distro so a distribution a Linux distribution is an operating system made as Software collection based on Linux kernel and often and a package management system and this rolls back to that So from Fedora, what does this mean today versus women yesterday? So yesterday? The distribution was a one-stop shop. I mean when when the when source forage rain supreme nobody Who ran a package of an operating system or a distro based on a package management system would just randomly grab things and compile it Like nobody ran that production. That's not the way we did business. It's not the way that anybody like who really had a good Set of standard practices in place did things however What we're starting to see in this kind of shift people are running things from npm in production people are running things from Ruby gem production Is it a good idea a bad idea? I think is up to the merit of the people doing the work to debate and that's fine whether or not they do believe or don't believe but then you add containers and you have these Docker environments and you have Rocket and you have run C You have you have all these different things that allow you to kind of put a contextualized sandbox around This environment that may or may not be what you traditionally thought it was so a long time ago The young repository was the place to be that's where software came from that's where you got your software period period the end full stop And that's kind of changing we're seeing we're seeing a fundamental you know branch out in you know avenues to to consume software so It used to be this one soft shop, and it's no longer. It's moving into this thing where people aren't doing that as much One of my favorite actually one of my favorite examples of this Has actually isn't fedora But Jesse frazzle she's a Docker core contributor, and she's also a debbie and maintainer But she is notorious around the internet for giving these talks about running just wildly interesting things in Docker containers And like one of the most recent things she did was like ran quake for something In a Docker container on her laptop on stage at a conference Just kind of showing everything and she has this github account where she's got like something the ballpark like 40 different applications That are just standard desktop applications Things that she uses as a day-to-day basis, and she runs like her entire desktop in a set of Docker containers You know like everything is alias and things so that when she runs in her So time when the manager I use i3. She's also an i3 user, but yeah So there's a system launcher called D menu, so if she runs a thing in D menu will actually map to a Docker retainer It's very interesting to see this Did you know that's completely different software delivery mechanism pulling directly from the Docker hub for your registry and you that's where you get Software from and if you read the her Docker files, it's not actually coming from a distro package Like they're all built from source on inside of that container and then like pull down I don't know if they all are the few that I looked at where and it was just I thought it was quite interesting but just that that's a an example of Where this is wildly different from how it used to be done like that if you if you proposed that to I think a room of system administrators or a room of traditional willing to share users five years ago They would have laughed or thought you were crazy But that's not that far fetched anymore like it's just not like it's just because of how things have progressed and changed and I think a lot of these different avenues for consuming content or consuming code or programs have matured as time has gone on It's not as Wild West crazy idea to do so Thinking about Fedora Where does the opera system and the application begin? I got a little bit of myself on the slide because I just really like that particular question. I think that's a It's an arbitrary concept But it's one that you have to define and I think it needs definition at some point because reality is is there is a difference and I think good examples of this are generally mobile operating systems Android iOS Firefox OS from OS They have a very firm definition of where the operating system stops in the application begins such that you can you can Completely upgrade in it and maintain each one independent of one another and completely separate life cycles so for an app an Android application as an example, so I I'm a science. I'm on user. It's a community distro of Android I'm a big Fanboy of all things community and open source and things like that in their community. I'm sort of free build of ASP A lot of things But anyways, I run the nightly builds because I'm crazy. I also actually run raw hot on my laptop they They put out these nightly builds and I can upgrade my operating system underneath my applications every day of the week And my applications don't need to change nothing about them changes the cash gets erased and it read as a the art VM caching optimizations when it reboots, but you can maintain these things separate in separate life cycles and those kind of thing and if you take that concept into you know a more I guess Large-scale operating system for lack of a better vocabulary term How do you take that concept? Down there. Well, it's containers. I mean, we're it's already being done right now. Like you have containers I think a prime example of that is and is rel 7 With certified rel 6 containers You now have applications that ran on an older operating system that you can now run at a different life cycle than the OS Who's actually touched on the bare metal? And that's very it's I think it's a fascinating thing in the whole concept of what we're able to do with that moving forward What we can do in fedora, I think is is very special there I think going back to my comment about rawhide. I I love the idea of rawhide I want the hottest bits at all times. I want them on the laptop yesterday today and tomorrow But right now if I do a DNF update and I pull in 130 packages This is actually something happening. I pulled in 140 some odd packages something broke And I started going to Koji because we don't keep you can't do downgrades You don't keep history. It's just rawhide. It's the latest for the pole for the compose So I go to Koji and I grab packages one by one I downgrade them and about 30 packages in I didn't find what had broken it I just gave up on a reason on my laptop to the current stable version for the time and I just kept moving on with my life But if we did a No s first application split And the OS was I don't know an atomic image and I were to just switch Do an atomic host downgrade and I switched to yesterday's tree and I reboot all my application space is still there It has not modified, but my OS is different It's we have a lot of advantages there We can make rawhide a more attractive option for developers and contributors within the community who want to work on the late-breaking most cutting-edge technology With that you know kind of safety blanket easy rollback and not modifying or altering an application space. Yes Well because So that's actually an interesting question let's say for example, there's a post a post install or post uninstall or trigger in The RPM transaction to where some artifact is left behind when I do an upgrade or a downgrade and now the functionality Something has changed and that doesn't happen often, but it has happened Whereas with an with an oh it with an RPM OS tree. It is a literally a full tree It's effectively like a git snapshot of a file system with this set of packages So you have almost a guarantee that the tree that worked for you yesterday is going to work for you today because it is unmodified whereas With your file system if you did a young or a dnf upgrade and then a dnf downgrade You still have modifications on the system like it's state changed good And I don't disagree with that and I think that's a good point is is a lot of functionality like that does work in well, and I think it's because of Probably the engineering effort and the extra QA effort that goes into those things and and the support team behind it However in Fedora space, I'm thinking more about the cutting edge of the raw hide like in raw hide nightly builds We test zero of that and we have zero guarantees that those kinds of things work And there could even be a bug that was introduced that would break some of that functionality And that's and that's kind of more of what I'm getting towards because in release engineering standpoint We're always looking towards the next release We're looking towards maintaining the current one getting updates out the door But also the next release the deliverables for that and so from from the release engineering perspective I'm thinking more of those concerns as opposed to long-term maintaining a running system That may need to live around Yes, okay, so I don't disagree with you But I also I want to kind of move because what I'm getting towards is part of what the like Fedora's a project wants to do so Just very quickly if he seemed more tightly coupled right now today, we have docket Docker rocket XGF if anyone's not familiar with XGF. It's this kind of very interesting concept about running containerized desktop applications inside of Effectively a container sandbox such that it is shipped holistically as an OS tree I was actually just about to mention that there's a thought like the right after this It couldn't have been more conveniently time. There's a talk on that and everybody should go I'm actually I will be there I'm very excited to see because I very cursor like on a cursory concept know what it is But I've seen it. I've only seen it work in like a demo on the Internet. I'm very excited to see Anyways, so for a reason This kind of brings around to like the third or next concept has been kind of mulling and manifesting and coming to light in more recent time and so if we were to think about it like the face design would be our operating system and then Everything out here could be our application space and that's that's very interesting From a release engineering standpoint because what we need to build and how we build it might change Dramatically because right now it's all it's just one big clump. So it's a giant package set. It's a giant repository It's a giant composition of things where we generate metadata and have tracking for what needs to go into ISO images for the different products and those kinds of things. So Thorings question mark. What does it mean to release engineering? So we need to be able to adapt more rapidly and this is a big thing is because we have this we have we just we have build artifacts and our Purposes to deliver them as quickly as possible to cater to the needs of the community to cater to the needs of the contributors However, we need to maintain Sanity within that we can't allow things become the Wild West I think a great example of this that kind of has enabled a lot of people to do a lot of things more rapidly is Cobra Cobra has allowed people to build Anything and they can do it as rapidly as they would like to on any release of fedora without breaking The bulk user base and that's great. I think I'm a big fan of that It it caters to that need without breaking the core deliverable From a releasing standpoint that defines The product of for our station or the product of our server or product for our cloud It lets these things iterate independently and people can optionally install it on top So it has it has its own independent life cycle. I understand like its delivery mechanism is still very similar to the traditional setup that we have But it can optionally override components that are within the core, you know the core environment so In terms of water at what is actually delivered We need to maintain this kind of concept or this set of Facilitated sanity and and stick with the release engineering criteria things need to be reproducible audible final and deliverable And and what that means to different people is varying and the link that I mentioned earlier has formal definitions Of what we mean by that but basically reproducibility is we need to have and if anybody attended the reproducible build talk Yesterday Okay It was amazing talk. You missed a good one But it discussed a lot of what this means to us and such that if you have the same build environment the same Build repository the same set of point-time snapshot of all the packages with the same inputs you will get the same outputs It's not like a bite-for-bite thing because if you do check sums on like Modified like m-time and a time But you would end up with the same output We need auditable Build tooling in such that we can trace it for security auditing CVE's that kind of thing. There's some interesting things going on right now in Docker space where like Docker images are just kind of like this black box You may or may not know how those builds are created and what's in it You can unpack the tar ball and stare at binaries all day, but then It's difficult to audit there's people working on it. There's actually an auditing toolbox that's being worked out on get it up I'm looking forward to that which I'll actually mention in a minute and something that we're working on so built tooling today And yesterday so we have Koji and Koji Original format was RPM centric. It was built in a time where the RPMs was the delivery mechanism It was built in a time where the disher was the one-stop shop and in case you're that very well Somewhere along the way image tracking integration was added. It allows the churning of things like cloud images and And then we also have the live CD creator, which I understand is being replaced at some point I'm not I'll be very I'm all of me. I don't know what's going on with live CD creator I'm punchy. So punchy is a composed tool And that's that's been around for a while. It's currently iterating being rewritten and adding new functionality Lorax works does tree compositions Boaty is the update mechanism that everybody knows and loves and then the Wild West is copper so tooling for tomorrow Tomorrow the arbitrary tomorrow coach, you don't know so coach you don't know We'll be content generator centric and if anybody went to the Koji to talk yesterday We'll have a lot of more insight on this, but effectively a content generator is a build mechanism or a build system or tool that will Take a set of inputs and provide a set of outputs based on a standard definition of what metadata is required to Define and create those things and that metadata and the bill artifacts can be passed within those systems RPM builds would then be a first-class Content generator and there could be other things now the content generator is going to be specific to Koji to we're supposed to have that in Koji 1 to 11 But in to it will be kind of a first-class data type for lack of a better term and and that will allow us from Relationary engineering standpoint to pivot more rapidly on these newer types of built artifacts that need to be generated and produced Based on community command, so I think one of the big things that I will admit that we're like behind the right now is Docker I think I think Docker has exploded and it's very popular and from a fedora tooling standpoint In terms of what we as a project officially release. There are certain things that I think We're a little bit curved on And that might that might actually not be fair from a distro standpoint, I don't even know if anybody else is doing it at all What we're working on so maybe we're ahead of the curve But I like to think that we We are we you know features friends first We're we try to travel. Anyways, so One of the things that you know, we could we could add is whatever the next Docker is, you know, if it's not containers You know could be some other piece of technology gets really hot in five years And it's just exploded overnight and the whole world's excited about it It's very revolutionary and and you know kind of Changing in the way that we traditionally do things or paradigm shifting that kind of thing and we should be able to adapt that better and I think having something that is Centered around this concept of this content generator. We could then have a system that would integrate very Tightly, but be loosely coupled in the sense that it doesn't have to be Integrated into the Koji code base to function as though it was And that that's gonna That's gonna lend itself to us in a big way So punchy for it's currently in use It's constantly being iterated on that actually took compose time I think from something like 12 hours down to four hours like it was the huge performance Increase it allows a lot of different building compose tasks to be farmed out through Koji Sort of things that should have been able to be run in parallel, but weren't able to based on punchy three like Design something or other they've been resolved a refactor to read you know reinvented in such a way that you know Parallel tasks can be executed parallel and it just sped things up a lot for the composers Which is going to lend itself to allowing rawhide to look so rawhide nightly builds are gonna look more like a traditional Release so when there's a test candidate or a beta or alpha beta build or release such that you have all of the everything artifacts the isos Those kind of things the rawhide nightly's are gonna look identical to that and that will hopefully lend itself to More easily integration with qa processes because all all of the composed outputs from The release ensuring standpoint will be identical. So we're not messing with different batches of Bill artifacts put out So layered image bills for containers This is what I was kind of alluding to a little bit that I thought we could could have done better on but Anyways, so maybe we're head In Docker, there's this concept of layered image. You can have your base image just be a core set of honor packages then Let's say your next layer will have Python and then well, you know because you know your next layer has What's cool this week?  So you have multiple layers underneath Let's say layer two has a security exploit Let's say for example the build system has 40 of these that share layer two How do you audit them? Well the concept of the layer build image? the layered image build service will allow number one the Docker files to be integrated into disk it number two the build to be executed through Koji to the Fed package Fed PKG client and then number three these automated rebuilds of things based on audited notifications for CVE's so that people who use and consume applications and Containers that are layered images out of the Fedora project can have confidence in the fact that they're They're maintained at a certain amount of reliability a certain level of quality Much like the distribution itself. So it's kind of a new type of deliverable from Fedora It's being worked on right now It is being worked on right now. I actually really want to have a demo for everybody today, but I found a bug in a thing It's OSBS stuff. Yes. Yeah, so I have a slide on it a couple in like two slides But it's OSBS. It's the open ship build System build service service so we should build service client actually so open ship version three Basings of on top of Kubernetes adds a lot of abstraction for things like builds build integration source image building And depth tooling and entire lifecycle chain The entry point into the ownership depth and build tooling has a profile for a custom builder The open shift builds service client tools call into that API entry point and use a custom builder to afford us all of these features of Of the more verbose build tooling So that's something that's kind of being worked on right now We're aiming to have something available and staging in the next few weeks And so that we can kind of announce it let people start getting the tires on it More details fall on that there is we have the team is working on it We have a public con bon board But it's in a system called Tiger that may or may not be long for this world. We're discussing We actually will probably tomorrow as a group get together and discuss a Handful of tools and try to decide on something for project project management type things within the floor community And that's probably end up being a very long-winded discussion But anybody who is interested in project management style things In the door space We should organize and get the group together and talk about that because I know the QA teams in fabricator reason tiger We need to I would like to just as like a quick little tangent I would like to try and find a project management style tool set the standard eyes on it as a for our community Such that there's like an entry point for new contributors coming in who want to look at what's going on all over the place And you could just be like, oh, I'm curious what's going on here. That'd be great And integrated into Fedora hubs everybody see the floor of stock. Yeah one. It's gonna be amazing I'm like I'm so stuff who like okay. Who has an Android device Okay, who uses Google now With cards like bull swiping imagine that but better in web format and widgets and things that pop up based on like Yeah in for Fedora all things Fedora. It's You should have gotten stuck. All right Two week releases another big thing that we're working on right now is for atomic the project atomic group is pushing the envelope They're doing a lot of iteration. They do two weeks release two week Cuts for upstream components. They work on they work with us and actually requested that we start churning out two week releases on that So for floor 23 and beyond hopefully we get it in for further 23 final The plan is to actually have that be a new life cycle deliverable within the Fedora space and that kind of starts That's gonna be in I think the first thing that we start delivering outside of the general six-month cycle To start kind of like start paving the way to allowing different components or different products within The Fedora space so workstation and server and cloud could later maybe pick their own life cycle It was in Fedora and actually set up a release tooling to allow that the Wild West is gonna be copper doper is actually copper But for doctor which is already out there and staging it's being worked on. It's it's it's very it's very good Tech you should go check it out other so the other is just we don't know like we don't know what's gonna come next What's gonna be the new interesting thing that everybody wants to check out and do I'm Standing in the way of the slide deck in the camera thing. Hi everybody. Sorry So we don't know what everybody's gonna want to work on it We don't know what's gonna come out in the next few years, but we want to try to cater to that and be as quick to pivot as possible So really quick atomic two weeks We actually have a plan like there's a plan to make like tested automating things and gets out the door. So effectively So we will trigger an OS 3 kickoff build based on Detecting a change in an artifact that goes into that right now today. That's just set of RPM So there's an update to an RPM that lands in OS tree Koji should right now Bodie does but based on a fed message it will kick off a build No, that build happens It will automatically pick up a test and that test be marked pass or fail If it fails it will go down to a bug filer or yeah bug filer if it Passes it can be marked as a release candidate Once it passes the set of tests and the tests are not yet up on the bird right foot the girl to IO PAG you are into IO. It's an open source get-up phone. You can sign with your fast account. It's good stuff There's gonna be a lot to talk on today. You should go So Pierre a pingoo on IRC he wrote it. It's good stuff We've been using it has existed for like a month So if you don't know about it, it's not like a huge like you missed the boat It's very very new. We've been using it release engineering for our stuff and it's it's been very good for our workflow but anyway, we're gonna host the The atomic test up there that will run in two near well. I went to Kushal's to near talk earlier today Yeah, one. All right All right. No, it's it's a very simple testing framework the language for it is is based on the premise of Exit codes of commands so you can use any test suite unit testing whatever Cucumber anything if the test runs and fails then it that step in the test fails There's things to modify, you know non-zero is not a failure, etc But it's very simple So we're at first pass the test Then it will become a release candidate and then at the point of release the The latest release candidate will go in it will be uploaded uploaded to cloud providers will be uploaded to FTP There will be an automated kickoff to update the website That does not yet exist, but we're working on it down the wiki if you want to check out the status of that and then update links send out an email to announce and maybe roll back if we break something and then Our we'll have quick docs links download launch instructions those kind of things So this is the base idea and a very shameless rip off of this. I stole this diagram from Matt Miller I want to thank him Exponentially for writing this up because it's it's amazing. This is the plan in visual form and I'm bad at diagramming So this is my diagram handiwork This is why So this is the layman build system So if you're just get with that package Disgit is where the Docker files will live and that package where you will set your commands off much like you do our cam builds today The build will call about the Koji cogs links into the open-shift environment via the OSBS tooling The OSBS tooling will actually do the build inside of our open-shift environment and then we'll export all the build artifacts and store them in Koji and then we can also auto upload them to a Docker registry of a choice Whether or not we host them ourselves or externally that has yet to be decided But that is on the road map So this is the layer built kind of Visualized versus me just hand waving and blabbing about it So what next so containers the morning also your container formats is something that we've had requests for and Is kind of on the road map for some indeterminate amount of time in the future, but it is something that we're looking into So rocket run C is new in the open container foundation or whatever they rename themselves again. I think I don't know So run C and then freight agent who is not good with freight agent. It's a it's a new It's not really a new it's a new delivery mechanism for container Root file systems that has system DN spawn on the back end and it like is all kicked off with system The unit files and then other but we did I mean what's the next big thing? We don't know I mean there's there's new container technology pop-up all the time The comic workstation though a ton of workstation proposal to hit the mailing list recently the desktop mailing list Which I don't know Somebody in the working group wrote that at some point so at some point we're gonna have to Figure out what that's gonna look like and how we need to build that how we need to deliver that from a release engineering standpoint So that's something that we're kind of trying to pay attention to as it develops and as it progresses over time I for one I'm like really excited about it because I I've almost just built my package set as an OS true as a atomic image anyways, but I already have Ansible playbooks from my laptop some weird So no, it's not familiar Newlycule is an app as a container application specification such that you can link together multiple containers to comprise an application Such that you don't have multiple services running a single container each service would exist in its own Provides a metadata for linking it that kind of thing the build system So the layered image build system will at a later date support this and allow people to actually pipe in a new Coal application builds into the into the environment and then pop out the containers and people can pull this back and it'll If we do it right it'll work And then the next new hotness we just we don't know but these are kind of the things in our general You know future unknown Timeline roadmap that we want to work on from a listening standpoint in the next generation tooling On top of and to the side of currently continue to maintain everything that is today Questions that's either really good or really bad. Thank you all


Hi, everyone, I'm Brinley Zabriski and I'm going to walk through how to use the REMR package. So REMR stands for Rare Event Meta-Analysis and it's a package that implements the method in this article here, so it's a permutation-based approach for heterogeneous meta-analyses of rare events. So this method can work well when events are binary and rare adverse and when there's heterogeneity in the dataset. So the main benefits of this method is preserving the type 1 error rate and having good confidence interval coverage. So I'm not going to go into detail about the method and about the results but I did want to show a few things. So here's just some plots. The first one is showing confidence interval coverage on average and this yellow method is REMR. And so we can see that that comes really close to what we want, in this case 95% coverage more so than other methods and then similarly we preserve the type 1 error rate more so than other methods and kind of these challenging cases where we have rare events and heterogeneity. So those are the benefits of using this method and obviously there's consequences. So we're going to have larger confidence interval widths than other methods but again that comes with the benefit of maintaining coverage and so forth. So how this method works, it's based on conditional logistic regression where we condition out the nuisance parameters instead of trying to estimate them but instead of using asymptotic inference we use a permutation-based method for inference and that is useful when we have rare events or sparse data. So what we get is something that looks like this here which is the permutational distribution of the test statistic and the way the test statistic is computed can be found in the paper but for this example data our observed value of the test statistic was 37 and then we permuted our meta-analysis data set a ton of times with some constraints to get all possible values of the test statistic so we get a range from 27 to 53 here. So then to get like a p-value we could just say okay well everything more extreme than 37 for a two-sided p-value would result in our two-sided p-value there. So that's kind of the gist of the method so again based on conditional logistic regression but we used a permutation-based approach for inference instead of an asymptotic-based approach. All right so with that let's jump into the actual package REMA. So we're going to use this very clean data set so it's a smoke pain cessation devaculatization drug and the goal of this meta-analysis was to determine if there's serious cardiovascular adverse events with the use of very clean so very clean uses is here and then a placebo is here and so you can see the the patients who received the placebo total and then the number who experienced a serious cardiovascular adverse event and so forth so there's 22 studies here so for REMA we can go ahead and load the package and then if we look at the documentation we can see there's just a few arguments here the only ones that we need to specify are these four so just the data information so we're going to start with that just in a really basic meta-analysis so because we're doing a ton of permutations this can take a while to run depending on the number of events so the number of events is what largely drives computation time here we're looking okay this you know set of 10 here adds add some computation time so this takes just over two minutes to run so I went ahead and ran that in advance so we don't have to wait for that so if we look at the results we see that we have a combined odds ratio of 3.4 95% confidence interval that is pretty wide right that's a pretty big range and so that's just again one of the you know potential downsides of this method is that we're going to get a little bit more conservative results but I mean we're pretty much guaranteed that we do maintain 95% coverage which is nice and our P value that we got is 0.046 so that's just kind of a really quick meta-analysis using Rema but I want to go through a little bit more detail about what's going on so if we plot this Rema object we're going to get a plot that is the permutational distribution of the test statistic so in this case where this asterisk is that is where our observed test statistic is so we have an observed test statistic of 34 and then a permutation distribution ranging from 20 to around you know the over 40 and so to get the two-sided P value we can add up everything that has bars of heights less than this value and that's how we get the two-sided P value so a benefit of this is that we we're not assuming normality here in this does look fairly normal but you can see that the tails are not the same right and so you can imagine you know different distributions that look a little bit differently than this one that are more skewed and we kind of maintain that exact distribution if you will which is it's kind of nice so let's look at some of the details here so rare event tether genius analysis method great our two-sided P values returned and mid P equals true so mid dot P is saying that we're going to use the mid P correction when we're competing our P value and what that that does is instead of taking this entire bar and adding that probability to our two-sided P value we just take half half of that bar and that reduces conservatism and it's been shown to work well and so that's what we implement here in this package and then we'll go over the conditional maximum likely destination in just a second I did want to show you everything that's in this remote object so the you know traditional things that you put in are in there as well as your arguments T is the odds ratio CIS is 95% confidence nobody can pull out P value is the P value and then the distribution contains the test statistic and the probabilities that we're in that plot so you can get the actual data there and then T side is the observed value of the test test statistic so that's what to kind of just straightforward Rema implementation looks like because it takes so long to run let's say we decided we actually wanted a you know 10% alpha level instead of 5% so instead of having to run this whole thing over again and take another two minutes in this case we can just pass Rema a Rema object and it can almost instantaneously update the results if we change let's say an alpha value so here you can see you know our confidence intervals a little bit narrower because we changed our alpha level alpha level there so that's nice we can pass a Rema object in instead of the whole data so we don't have to recompute the permutational distribution we already have that and so we can just change alpha and then similarly if we didn't want to use the mid P correction for some reason we could turn that off without having to rerun everything and then you can see that does make our results you know more it's conservative in this case. Okay so you really just need to put in the data and let Rema do its thing but again you can actually pass a Rema object if you need to change things afterwards. Okay I did want to show you just two more things but that's really the basics of this package we have a few more obviously updates we want to do but it's it's very functional now in it and it maps well to the paper so you can get more details there. So here's just a made up data set just for illustration so if I run this you'll notice that we get an error and it says that there's only one value in the permutational distribution of the test statistic and so this method can't work so that same as in that you know picture there'd be only one bar here and so that's just the trivial case where we the value is the only value in the distribution so in case like this Rema simply won't work and so you'll need to use a different method to analyze your data set so here's an example where it doesn't work and then I also wanted to just show a quick example where we use a different estimation method for the odds ratio so again I just made this data set up if I run this and then we can plot it take a look so this only has two values in the permutational distribution so we have a test statistic and our observed values 24 so whenever we have an observed value that's at the minimum or the maximum of the permutational distribution then we cannot use conditional maximum likelihood so if we go back to this first one we did you can see that we use conditional maximum likelihood to get the odds ratio compensating great but we can't do that in this case where we're at the in this case the maximum of the test statistic distribution so instead of using conditional maximum likelihood estimation we instead use the median of bias estimate so when we do that you'll see get some slightly interesting results so here it says we're using the median of bias estimate great and then our odds ratio we actually get an odds ratio which is nice and we get a lower bound in this case for the confidence interval and then we go to infinity for the upper bound and so that'll always be the case when we're at the minimum or max we'll have infinity somewhere in there we do get results we get a p value it's just a different way to do that estimation and again more results are in the paper but that that'll be the two different options you'll have for this method part and that's that's basically it that's REMA so if you guys have any questions please feel free to reach out if you want to kind of check out more of REMA here's the CRAN and the GitHub repo this is the article which the package is based off and the data set that we use for illustration is linked here as well so again please feel free to reach out I hope that was helpful.


I feel like I have to whisper now. So essentially, this is about community health and vulnerability mitigation, and one of the things that has become really sort of flavor forward in open source in the last year, year and a half, are the dependencies that are related to projects. And the problem or the challenge for open sources that, first of all, identifying what all the dependencies in any given project are, and then deciding how far down the dependency chain. We do have folks on the back end, actually. We do have folks on the back end now, yeah. Can you turn down the volume a little bit? Can you turn down the volume a little bit? I think I'm almost causing feedback for myself. OK, that's probably better. So effectively, anytime you're importing something or have a dependency, often there are then transitive dependencies and tertiary dependencies and circular dependencies that sometimes exist. And so managing this is important because suffer vulnerabilities are injected in each of these things that I depend on and maintaining an awareness of where I am possibly going to have a dependency and where an actually explicit vulnerability is important. And so just knowing your vulnerability is often very difficult because these dependencies are not easily surfaced or visible. Does everybody agree with that? So kind of the birds with feather is looking at some brainstorm about where or how we can start to address this challenge in a way that isn't utterly overwhelming. I know, for example, on this effort project and in some of the embedded systems, software builds a material right into the way that the code is compiled. So you have a very complete inventory of everything, both at development time and at runtime. But most open source is not held to that level of safety critical, safety criticality. And so we have to go back and try to discover what our dependencies are. The things that we've started to use with chaos are a metric called upstream dependencies, which are essentially, these are the dependencies that your project has on other projects. We also use a metric called Libya, which is how old is the, I may be running air traffic control for the airport right now, I'm not sure. So Libya is, you know, so if a library was released yesterday and the version that you have was released two weeks ago, then your version would have a 13-day Libya calculation. So how old is the version that you're using of a library compared with the most recently released version? And so that is a useful, heuristic measure for, you know, do I have a lot of out-of-date dependencies? The more out-of-date dependencies you have, more likely you are to have a vulnerability that's in your code somewhere. And obviously there's a vulnerabilities list, but a lot of projects aren't paying close attention to that. And then there's also the OSSF scorecard, OSSF, I always feel like I say too many Ss or not enough Ss. The scorecard, which we've implemented, and we use that to get a sense of how rigorous the different components of our project are and the different components of the projects that our project depends on are. That's also a useful measure to try to get our heads around this dependency problem. But the core challenge kind of is from this sort of set of complex interdependencies that exist. And so I'm just curious how do you think about these problems or how do you think about addressing these problems on the projects that you participate in? So I'm looking at some DevOps metrics. DevOps? Yes. Okay. And maybe there are some of the capabilities that include dependencies that can help to measure. So in DevOps metrics, you try to measure the performance of the software delivery and then you measure in terms of the technical capabilities. And some of the technical capabilities include testing the dependencies. So maybe we could look at the daughter research program that has a lot of the technical capabilities of DevOps from Google. Okay. And then they have some metrics for dependencies. There is not only measuring the dependencies itself, like the amount of dependencies. It's like how do you deal with the dependencies? When do you test? What do you do with the test? What triggers the test of dependencies? So maybe those kind of dealing with the dependencies metrics can help you to analyze the risk of having the dependencies. Because the dependencies will exist. So maybe only measuring the amount of dependencies is not going to bring you good understanding of the risks. Maybe measuring how do you deal with the dependencies can add you more value to compose an understanding of the vulnerabilities. From a DevOps perspective, I'm not a DevOps person. It's okay. Forget the DevOps things. It's like I just wanted to bring where did it come from. But it's like the idea of understanding how the team manage the dependencies and not only measuring the dependencies itself, the number of dependencies. So when you say the manage them, one of the things that's come up in the discussions we've had on the CAS project are open source program offices. For example, they don't want to be overly restrictive of what dependencies a development team can incorporate. But they also want to try to build an awareness that each time that you include a new dependency, you're creating something that has to be checked and managed. And some effort ideally would be made to know that that dependency, for example, is a going concern. So it's in this line yet exactly this. And instead of only measuring the amount of dependence, you measure the practices that the team do with the dependence. When do they test? How do they do that? What triggers the test? What do they do after the test, the dependency? Yeah. When you say test, do you mean just an ordinary software test and that it passes with a dependency in it and a new version? Or go ahead. I have an idea related to what Bianca was saying. So things like the dependabot and other kind of spots. Can you measure your dependencies and if they have a dependabot kind of thing, how quickly they close those pull requests and things like that. So you can actually measure how actively they're maintaining their own dependencies, which are those tertiary, right? Yeah. Yeah, that's a good idea. So look at the dependabot pull requests. I know on Augur, I get dependabot alerts all the time. And off, I would say maybe half of the time, give or take 10%, there's a fix that won't break anything else. But the other side of that coin, half give or take 10%, is there's actually a substantial amount of work because the dependency change introduces breaking changes. That's especially problematic in different ecosystems. I mean, I'm sure you've all seen this. In NPM, breaking changes seem like almost automatic with any dependabot update. Python environments, less so. So it seems very ecosystem specific. I like that idea. Of course, the project has dependabot turned on, but is it still optional? I know I signed up for it voluntarily. But I know also sometimes GitHub starts to include things. OK. All right, so it sounds like maybe they're starting to turn it on automatically. OK. Hello. Oh, Steven Jacob from Rochester Institute of Technology. How much? All right, OK. Good to see it. Steven, this is Kelly Blanco from Auckland University, New Zealand. Bianca and Marcos from Northern Arizona. So it's a very academic crowd in the room. No one's threatening the show. There's a mic no one's threatening the show. Oh, OK. So dependabots, things like that, that's a good idea. Lib year metric that you have. Yeah. You could use that in like a hierarchy as well, right? Like what are the Lib year of your dependencies? So maybe some metric. Yeah. For each of your dependencies on how updated their dependencies are. Yeah, and we have like, we have pretty good data on that. Like we can, when we run a project, we calculate the Lib years for every import. So it's like, if I run it every week, I've got like Lib year data. You can see trends and things like that. But when it comes to fixing these things, I still feel like most open source projects that are not large scale CNCF type things struggle to have the resources to actually resolve all of these dependency aging problems in a timely manner. What about this is a question like, have you combined any of these types of dependency metrics with looking at your community metrics that you have, like the community health of your dependencies? Does that matter? It's not necessarily from a vulnerability standpoint, but more. What is the relationship between community health factors like pull request acceptance rates and issue response times and the extent to which the dependencies on a project are aging continuously? Right? Yeah. Yeah. I haven't done that, but that's a really good idea. Like I have all the data. Yeah. That's a good idea. That would be a really interesting correlation to do on a large scale. It's a good suggestion. In the community architecture assignment, we have our kids do? I haven't, but I wasn't accepted to RIT. No. I just struck me that I should probably send you that to show them what we have them look at. Do you describe it a little bit for the people? So I teach online. I forgot the repeal. There's a microphone by you, I think. Yes, I see that. I probably wouldn't have hit you if I remembered there were people online. So we have this, I've been teaching open source contribution at RIT for about 14 years, and then we opened an Ospo two years ago. And so we have students run large scale community architecture assignments. So I'm just going to try to find it as I sit here with my phone. But it has them look at not only like these days, a bitergia scan, but we have them go through maybe 10 or 15 pieces and we have them do blog posts on what they found and we have them do that three times. Since working in public came out, we have each group do one on a stadium and one on a federation and one on a, not the toy, but what's the third or fourth one she does? Stadium, toy. What is it? Do you know working in public at all? Oh yeah, and not egg balls? Yeah, she has these four categories. So we have them do that.  It's probably going to take me a minute to find this thing because I don't have a laptop and because my brain is essentially dead. But essentially you take the four categories that are not egg balls. We have them run through 15 different categories of stuff and we have them do one each on a, so they see the difference on that and that's their training to then go do their own when they're trying to figure out what projects to contribute to by the end of the semester that will have run through it with a group three times so they can say, oh that sounded like a great project but now that I look at it, it's a train wreck so maybe I don't want to try to contribute to that. Oh that sounds like a pretty interesting way to have students start to explore open source. Yeah it's because we're not, my students aren't exclusively CS or SE students right there, they're all across the board so the traditional, if you will, teaching open source community way of getting students lost, productively lost in a large code base doesn't work if some of your students aren't SE or CS students. So in general, those classes and the summer of code that Said started that other people are going to start to adopt, they pick one partner project and they will hop onto stuff and do team assaults of various upgrades or improvements or fixing dependencies or whatever. My students learn in a group how to figure out what would be a good group to work with and then they go and do it on their own. So the idea of looking at community health to see how that corresponds with dependency maintenance. Or even just is this a community I want to work with.  So that's kind of like the thought process behind that. The Callaway coefficient of fail is part of there. Do you remember that? I don't. Oh God, Tom Callaway's spot. Did this huge... Tom Callaway, he was at Red Hat, right? He's now at AWS, but he was at Red Hat for a very long time. I met him at Red Hat, but I didn't know the Callaway, what is it called? The Callaway coefficient of fail. Okay. This is a real thing. This is a blog post he put together that was kind of a tongue-in-cheek how you know you're doing open source right or wrong. And it is old. So that like one of the pieces of it, in Windows you get X points of fail right away. So we don't use that quite as one of the things we do anymore, but... Well, I mean, yeah, certainly certain languages are not very friendly on Windows. All right, here we go. Comm March public assignment. I finally found it. It's opening up. So we run Qualdron. They calculate Callaway's coefficient of fail. They look at the project's IRC channel. They look at the hazard. They look at the source code repository. They look at the mail list archives. They look at the docs. They look at what are their other comms channels that the project uses. And they look at the project website and our blog. And then they try to answer a whole list of questions that we have. History, purpose, and goals of the project. What was the initial commit? The latest commit? Is it a benevolent dictator for life or another structure? What's the structure? And obviously now that we use these egg ball categories, some of that's already pre-answered. What's the turnover on the core team? Right? Is it the same 20%? Are they staying the same over time? You know, who's got where to the unique knowledge, highest amounts of unique knowledge line of project? Who has commit access? You know, how many total who approves patches? How many people who's responsible for quality control and bug repair front and back end developers? Are they the same? Are they different? I'll read through the Callaway coefficient. Does Callaway's coefficient have anything about dependencies? It does. I think so, yeah. What are the major bugs or problems that have risen during development? How has the project participation trending? Would the project survive? The BDFL was hit by a bus yesterday or now. Would the project survive? The core team are the most active 20% left project. Do they have official onboarding in place? Do they have documentation available? How good is it? These do sound a lot like the things we measure in KS. I guess it's all from Callaway. No, not all of that is from Callaway. The Callaway thing was kind of satirical, but to give you the Callaway coefficient of fail, if it will load... Of course, it doesn't load. It's something you can Google. That's what we have them do. We do cauldron because it's easier for them. They just put an URL. Boom. Because we're not looking for them to develop. Modern things over time. We're looking for them to do a quick scan to see whether this makes sense or not. Okay. So kind of what I'm... The things I've heard are looking at depend about sort of as a level of responsiveness to the dependencies that change. There's a system called CVEs, which most of you are probably aware of. And so some of these metrics are intended to sort of not rely on the CVE process, but simply to help a project understand where they're at and by maintaining sort of up-to-dateness, long-standing issues that appear later, things like Herk Lead or the log4j issue that occurred. If you were maintaining a certain degree of currency, you wouldn't have been susceptible to those specific issues. And there's a sort of a hypothesis that's untested that if you do maintain a certain degree of currency with your dependencies, that you're less likely to be tripped up by some long-standing vulnerability that only is identified years after it's already been around. But of course, if you can be aware of CVEs, if you have a full software bill of materials and you know what all of your dependencies are, then any time a CVE is issued, you should be able to query that list of dependencies and know if your project is susceptible, which is obviously dependable to some of that for you. It's kind of to some degree what dependable does. Dependable doesn't catch stuff for every language, though. How a coefficient of fail original blog post is now in your email box. Oh, thank you. I'm feeling like a broad blog post, like the Goggins proxy of fail subsuming Callaway might be in order. Anybody else have any thoughts on dependencies? Anything else you want to bring to the table? Is there anyone, any questions online or is anybody still online? Okay. Yeah. Well, I think if there's no further discussion, I will just say thank you for coming and you all come back now. Thank you.


Hi, I'm Chawla Jain, as he said. I'm from Charlotte, North Carolina, and I'm a GIS DBA, so I think I'm one of the few ones here, or rather I haven't met anyone really doing what I do here. So GIS DBA basically deals with spatial and non-spatial data. And another peculiar thing which I found was that we use PostgreSQL for most of our production databases, but the actual spatial element which we use is not post-GIS. We use a proprietary database there or extension which is called Spatial Database Engine, which is made by Environmental Systems Research Institute known as ESRI. There are a few people here from ESRI. And these GIS software we use is ArcGIS, which is like the most well-known proprietary GIS software in the world. Mecklenburg County, and again I'm one of the few people here from public sector too. Mecklenburg County is located in the southwestern region of North Carolina. It's bordered South Carolina and North Carolina, as you can see. And Charlotte is one of the largest cities in this county. The present population is about a million, and it's also as many of you may be knowing that it's the second largest banking center in the US, home to Bank of America. So the GIS department is about 40 people, and we managed both the city of Charlotte databases and also the Mecklenburg County, which is where city of Charlotte is located. There are many business units which use this data, some of the most known to people are 9-1-1 services. Then our land records department, where you get all the property information and maps. The local department of transportation and schools and so on.  We are about 40 to 50 people. 20 of us are the editors who do the actual editing of the data, the land records and the addressing data. About 10 or so do the actual mapping and analysis, and the remaining are programmers and DBAs, and I belong to that group. So what is a GIS data source? I don't know. Is anyone familiar with GIS other than post-GIS? Okay. The GIS data sources are generally SD feature classes. In our case, since we use SD with PostgreSQL, so SD feature classes, or you could be calling it post-GIS feature classes if you are using post-GIS. Then there are personal geodatabases because most people will have to pull these databases from the enterprise database server onto their desktop, and very often they have to manipulate it for analysis and personal use, so that's why they have something called personal geodatabases, and that's mainly in access format. Then the shape files were the first GIS files which were introduced way back in the early 90s, and they use the DBAs database format, and they can be easily transported from, you know, even via emails. Then there is text files because we get a lot of our data in lat-long form, and that's always in a text file which can be converted to a spatial format. Then the drawing files from AutoCAD and other similar software is also converted to, you know, geo-referenced data, either in polygon or point or line, whichever the case may be. A lot of the data is in raster files. Now raster files are generally stored on a separate database server. They're never stored with vector data, but raster files are a big source of GIS data, and then JPEG files, which are generally, they don't have spatial data, but they are sometimes linked, you know, like a little bit of point. Now the role of the GIS DBA, as I said, is, you know, you are actually in charge of both the spatial and non-spatial data, and, you know, this is just a general definition which I picked up. Most of our data is stored on data servers, which are either in SQL server or PostgreSQL. We started way back in the mid-90s with SQL server and STE, which is from ESRI, the early versions of STE, and we still have like three production servers. Two are in SQL server 2008, and they're on their way to 2014, and we have one major production server in PostgreSQL. This is thanks to very open-minded directors and other people in our local government, which have allowed us to move to PostgreSQL. So we have our major production database, which serves all our web applications in PostgreSQL. PostgreSQL 9.3, and ArcGIS, which is the client software, is also 10.3. We're always two versions lower because for all the testing, which needs to take place. And we also have like shape files, JPEG files, as I said, LiDAR data, all this is stored on file servers. So this is a chart of our production servers. We have, as I said, two in SQL server and one in PostgreSQL. And then we have a script server, which hosts both SQL server and PostgreSQL server. And that server runs all our major ETL packages. And it's really complicated the way we run this because we have all these different formats of data. And that's true for most GIS units in the country. So, I mean, you can't get away from it. Then these are, of course, the same servers. We have development servers. And then we have a testing platform on the other end, where we try out all the latest versions of Postgre and ArcGIS and SQL server. And half the time, most of the latest and the cool versions of Postgre don't seem to work with our ArcGIS software. But it gets there after two versions. Other data we support is a big file server, which that serves a lot of our other websites and business units. Then we also have a SQL server 2005 server for our air quality data. This basically collects all JSON feeds from different air quality monitors all over the state. And those are converted to spatially referenced data. Data maintenance and creation is done by our land record staff who edit directly on our SQL server, ST server through ArcMap, which is a part of ArcGIS. ArcGIS is a desktop software. It has two parts. One is ArcMap and one is our catalog. ArcMap is the actual tool to do analysis and build models and also to make maps and edits. The other part is called our catalog from where you could load data, view data, but it doesn't do much beyond that. Then we have a mapping group which makes maps and does the analysis like if somebody wants to set up a new business in Charlotte and they may come to them to find the best locations. And that's a big thing as many of you may have, you know, be already aware of that. So that's our mapping staff. They can directly upload sometimes data to our catalog. Some of them have permissions to do that. That means it goes straight to our Postgre server. We also have an intermediate development. Another one. This is a Postgre ST server where some of the mapping staff directly edit on that and that is moved over by our catalog to our production ST server. Most of our major automated data updates take place via the following methods. The ETL jobs could be the SSIS packages which is SQL Server Integrations packages or Penteho that's open source done with Spoon. I don't know if any of you use Penteho or Spoon. Then of course we have plain SQL jobs, Postgre SQL jobs which is run via the extension PGA agent. Then STE has its own language. So we have to run some conversions through command line STE and of course they are scheduled via back files. Then ArcGIS, the mark map which I talked about earlier, has a big model builder and that is where the actual GIS data conversion and modifications can be done. The models can be exported to Python script so you can either write all this in Python with ArcPy class or you can do it through model builder. Then all this, many of this including the Spoon jobs and the models and pythons are all scheduled via Dos command back files. Some of the spatial files which we update daily is like our address points. The whole county has, every address has a point and we have the land records file or the parcels file, lots as many of you may know the streets file, building footprints and the tax annals. These are very critical data to any jurisdiction in the country because that's where how your 911 services work. If this fails or if it is outdated, we get calls from the police department and all kinds of people. The other non-spatial data files which we use, which are joined with these spatial files are the real estate files which have information on ownership, sales, etc. These are actually maintained by our IT staff and they're on a SQL server box and we move it to our SQL server box and then finally to PostgreSQL. This is just an example. I'm going to show you some examples of our spatial data flows on a day to day basis. The master address points I edited on a SQL server through ArcMap. They are either edited or created or whatever as new addresses come in or changes come in. A text file is generated from that and that text file has XY coordinates. That is pulled into this model builder which you see to the right over here. That text file is then converted through this Python script or model builder to a shape file. The shape file is then moved over to file servers because a lot of people still use shape file for desktop manipulations and it's also moved over to our production PostgreSQL server. Now that's on the right shows the same model builder in Python format and that's scheduled via a BAT file. The parcel file that's again edited by our land record staff on a SQL server box. We use this Python script to just extract a feature class which has only few fields which we need for our production server which also hosts one of our major websites which I'll show you at the end if we have a few minutes. This parcel file is extracted via this script and then there we use another VB script to check if there are any errors because sometimes this extract doesn't work well. If somebody has entered maybe a parcel number is eight digits and if they've entered more than eight digits it fails or if somebody has entered some apostrophe or some strange things even while editing the lines. Once we had a line drawn all over the county and none of the records went through. So things like that, human errors. So we first check for that and then if everything is good we load it into our production server and those are some of the Python scripts to the right. The parcel layer is then also joined to a lot of those spatial files which I talked about which came from SQL server to Postgre which have all our ownership and sales and all that to create different views, parcel land views, sales, the KMA data and that's what our view looks like in our catalog with the parcel files and the data showing up on the right side. These are those non-spatial files which are used to create the views. They have land information like the basic land value, the assessed value, the building information whether it's commercial, residential, how many bathrooms, fireplaces, etc. It has sales value, historic and present sales location which is the site address and then of course basic info like the deed page, deed value and things like that. These files, this is an example of an SSIS package if you have ever used. This is SQL server integration services and that moves it over from SQL to SQL. You could use this to move it to Postgre SQL but I haven't had much luck with it so I generally just use this for SQL to SQL and then I feel that this kettle from Pentaho is much more easier to use when you have to go from database to database and so I use this to move from SQL to Postgre SQL. The screen to the left is the actual transformation and to the right is a job in Kettle and then it's scheduled via back file like that. This is just an example of our task manager from our script server, just a snapshot from one of our servers showing all the various jobs we've done. Now just like how is Postgre used to display GIS data? You could either use PostGIS which many of you are familiar with and or you use ESRI special database engine component. At Mecklenburg County we use both. The PostGIS we have an open source server like that with Postgre and PostGIS and it supports all our open source apps but since most of our database is edited by our catalog, our map and most of our end users are still using our GIS we use SD for our major production servers and who knows maybe in a year we might get away with that and go to PostGIS.  Next you create the SD schema and other schemas which may be necessary. Now prior to Postgre 9.3 at this point you would have to install STE and that would install all the STE tables into the Postgre database but now we can do it through the Arc Toolbox.  So at that point all these tables like STE table config and STE table registry all this gets installed into the Postgre database. This is just some samples of data. Restore and backup was done through command line till 9.3. Now it can be done directly by a PG admin tool and the only thing I think which is a little different I don't know about for our data is that first we have to restore the public schema and then the SD schema. If we don't do it in that order the data doesn't come through right. This is a snapshot of some of our major online applications and I can show you an example. This is our website. If you enter like it's called Polaris 3G and I hope it works. I just typed in an address and it will bring it up and also the pictures. This is all part of our GIS database. This is one of our major websites. It gets almost million hits a year. All reactors, even nosy neighbors everybody is looking at this all the time. Anyway that's the main thing. It can be seen. I thought of it. That's our website. I just typed in that address on the top and this is what you get to see. I'll do it again. It has these photographs. It also has historic pictures and back side and front side. If you have any questions at this point let me know. Windows. We use Linux only for the open source just post GIS and post that's done on Linux. This is all Windows because SD won't work with Linux. All the SD software won't work with Linux. A kettle. It's from Pentaho. It's also an open source I think. It's Pentaho and it's called Kettle. It has parts and it's called Pan and Spoon and Kitchen. It's got all these strange names but it works really well. If you look at Pentaho you should be able to see it. Yes, you can do it. You can view it but you can't do much beyond that. That's why we have these two groups. The ones who use post-GIS are just our web app people. None of the desktop users are really... But then it's also a matter of getting them... Even getting them to use our GIS is a big thing sometimes. To switch them to something else might take a long time. Oh no, we don't. No, we do post-grade authentication not Windows. For most of the readers we have a generic account and then we have few for those who edit and the database has a little different. When we had the SD SQL server, when we first started, we had individual accounts and that was so we moved to just... We have ten accounts. That's it basically. It's very odd that ESRI, I don't know if you've heard of ESRI, ESRI has this big international conference in July in San Diego and like about six years ago when we moved over, I was in the post... administering post-grade with SD. There were six people sitting but this year there were at least 200 people sitting. So there are a lot of people now doing this. There are a lot of cost saving especially in the government sector. Don't they edit SD SQL server? They are still on SQL server. No, the other stuff is all the other users mapping and all our websites, many of our websites, we have a lot of online applications.  But I see no... They could move to post-grade. It's just that we have some other databases in SQL server which are especially all our real estate data which I talked about. Those people don't want to work with post-grade. It's a matter of mindset. They don't want to change. And for all SQL servers we have our whole IT group. There's a lot of database administered but this we are on our own. It's like we are told if you have issues, you're on your own, you're the ones who moved to this. But we've been with this for almost six years and I feel there are less issues. Yeah, that's true.


I'm here to talk about a really devastating disease. It's created a huge impact on basically everyone's lives. It's killed some of the most well-known celebrities, the most noble Steve Jobs recently. It's called pancreatic cancer. And pancreatic cancer is a devastating disease. Only 5.5% people who are initially diagnosed will survive after five years. And one of the main reasons due to this is that usually once it's diagnosed, the cancer is spread around the body when the patient has a very small chance of survival. And in order to prevent this late detection, we need a method for early detection, meaning a simple, rapid, sensitive, versatile, and minimally invasive sensor for the early detection of pancreatic cancer. And so there are a variety of proteins that are found in the blood of patients who have pancreatic cancer that are at elevated levels. I show society one called mesothelin. Mesothelin, it's highly overextressed by pancreatic cancer but not by healthy tissue. But in addition, it's highly overextressed by these things we call non-invasive pre-cursor lesions. Meaning if you can detect this protein, you can detect pancreatic cancer before it becomes invasive, possibly making the survival rate jump above 50% rather than a measly 5.5%. So in order to detect these ultra-low concentrations of mesothelin in your blood, I employed these carbon nanotubes, atom-thick tubes of carbon. And they're really the superheroes of science and material science because they have these broad applications which lead to inexpensive biosensors. In my case, I was looking at how the distance between neighboring carbon nanotubes highly impacts how electricity is transported along that. And so I am really fascinated by carbon nanotubes. I was reading this really interesting paper in my biology class. And all of a sudden we were learning about these new things called antibodies, organic molecules that actually bind specifically to one protein. In my case, it would be to mesothelin, kind of like a lock and key. And so then I thought, in my biology class, I was just sitting there behind my desk looking at this little paper, I thought, what if I put this antibody in a network of carbon nanotubes just wildly on a whim? And then it hit me, amazing. I was very, very happy. My biology teacher wasn't as happy when she found me reading a paper instead of writing an essay on biology class. So then, what you had to have then is a substrate that you can basically support this network of antibodies and carbon nanotubes. And I thought it had to be porous to take up this awkward solution, but also it had to be inexpensive and readily available. Filter paper, it fulfills all of these. And basically all you have to do is create new from dispersion of these antibodies and carbon nanotubes that are just flowing around, chilling out.  And so here's a visual depiction of the sensor being created and how you test it, basically, it only requires a sixth of a drop. And you just put it on and then you just measure the change in electrical potential, just an electric parameter. So then in order to detect pancreatic cancer from the detection of this protein, basically what you have to do is you have to establish what we call cutoff level, where anything above that level of mesothione is considered positive, anything below we consider negative. And so usually it's 10 negrons for ML for mesothione, so that's why I chose. And so then I ran in vitro study just to make sure it was working pretty well and it worked spectacularly, actually. And then we went into actually testing human samples. We had patients who had pancreatic cancer and then also the non-vasive pre-care solutions. You can see they all fall in this range of 18 to 28 negrons for ML. That's well above the 10 negron per ML cutoff. All of these would be found positive by my sensor, meaning that it had 100% sensitivity towards pancreatic cancer. Then we had patients who did not have pancreatic cancer, they're either perfectly healthy and skipping around, or they had cryopancreatitis, meaning in inflammation of the pancreas, the disease was most often mistaken in diagnosis of pancreatic cancer. All of these you can see are below 10 negrons for ML, meaning that they would all be found negative by my sensor, meaning that it had 100% selectivity towards pancreatic cancer. So it was giving near perfect diagnosis of pancreatic cancer, thus revolutionizing how we look at cancer diagnosis. Thanks. So then what we had, we were looking at another application of the sensor. We wanted to see if it could effectively look at how well a treatment was doing on a patient. What we had is five chemotherapy drugs were given to patients over five weeks, and you can see for each of the five chemotherapy drugs there were distinctive drops. So it could reliably monitor how effective a cancer treatment was. And so what happens is the sensor, it's really revolutionizing how we look at cancer treatment and diagnosis. Because compared to the current gold standard of protein detection and cancer, it's 168 times faster, over 26,000 times less expensive at 3 cents, and over 400 times more sensitive. And also it has limitless applications. You switch out the antibody and you can detect an entirely new protein. Meaning you can look at other forms of cancer, but also you can look at cancer treatments effectively. You could look at cancer drug resistance because these are caused by unique mutations in the DNA producing a unique protein, which you can pick up with my sensor. Also you can look at the effectiveness of a cancer treatment, meaning that we are revolutionizing how we look at cancer treatment, personalizing the treatment at a patient so you don't have that conventional guess and check method. Also you could look at other forms of disease, such as in food and water safety, E. coli, salmonella, rotavirus, and then also you could look at other blood pathogens, such as AIDS, STDs, and all of those. And so it could operate in any setting, such as a regular checkup, but also in developing nations, revolutionizing and possibly saving thousands of lives in cancer diagnosis, treatment, and the biomedical field. Yeah! Thank you.


You find one more lock nut. Oh, this boat is awesome having used it in a while. It'll be good to get it going again Definitely have to fill in floorboards too These ones have survived pretty well all the other floorboards were starting to rot so I threw them But anyway, look how good that looks And that is excellent paddle wheel Hits up here a little bit Just slide it back before I screw it in place Totally fine Cool Both and washer No bolt and washer No, no, no, that's a net in the washer. What happened? You just did this right like four times Both wash right. Yes. Now. What's next? Washer net next to the tools and do not drop these Hey, don't drop those be very careful.  Yeah, I'm trying and these are marine grade bolts Shouldn't rest Now push the little button on top of the camera. There's nowhere to sit yet Well, I can't currently steer without the motors, you know, just pedaling because the motors were what steered before If I turn this yeah, that thing at the back there moves. This is always the coolest steering Remember when I made this So many people were like, oh those strings are gonna break like the minute you start using it, but they've Been on there for years So yeah, that's ignore the bicycles bicycle tire but that piece that moved back and forth I would connect to this pipe and then like turn the motors So maybe I can put a rudder back here Connected into that and then that'll work Well, maybe I'll put the rudder on that side since That motor is obviously not being used And then I can have one motor and a rudder And some pedals I'm not sure what I'm actually gonna end up doing for motors on this boat Well, I'll worry about it later. I'll just get the I well I definitely want to be able to go Unpowered with just with pedal power and a rudder. So I'll do that first Then I'll worry about the motors But I also want to change like the whole roof in here and like I Don't know. I mean maybe I don't I mean it's kind of amazing in here, isn't it?  They're mostly in the shade anyway. You got to move some surfboards out of the way Man, this is an awesome boat.  I made it have standing room for me The thing is If I'm driving this in high wind conditions I'm gonna want a lower roof More streamlined some kind of thing. It's not it's not terrible because it's all open the wind can get through here And I have driven this in some really Terrible windy conditions And it made it through and everything It is just whenever I'm in terrible windy conditions. I'm always just thinking oh, I want the boat to be shorter and more aerodynamic and but I mean I Don't know Before I worry about the roof and all that stuff. I just need to get some kind of steering and Then I can drive the boat out into the ocean and Go down and scrape off all these barnacles that have built up Yeah, I gotta get the barnacle. You want to go scrape some barnacles? Yeah, you are getting pretty good at swimming under but these things are sharp you can cut your knuckles on them really easily Well, let's just give this steering wheel an inspection and make sure everything is working right. Yeah, the rollers are good in there This string which turns the main steering column Still in good shape and I made the steering column go all the way to the front So you can stand on the front of the boat and still steer because sometimes you know I want to be up there and still steer and then the steering column goes all the way back To a string there that's wrapped around and down through some pulleys and to that thing at the back there Maybe we should put a little bit of grease in here a little squeaky, but everything seems to be working I also have to clean up all these leaves and crap. This is a gas mask. Yes For doing fiber bus Are you asking me if I can put a steering wheel in front of the boat? Well, it doesn't need a steering wheel. Well, you wouldn't be able to reach it But I can just reach up and grab that and turn it and steer the boat Man, this is such a cool design. It's it's such a It's such a good boat for like working stuff and cargo You can put a lot of cargo on the front there, you know when the floorboards are in obviously and back here too a few times I've had like this full of stuff and up in the front full full and Yeah, I can take some pretty big stuff One time we went somewhere all of us not you you weren't born yet But you and your sister and your mom were all inside here and we were on our way home You guys were all huddled up in there I was driving and the rain was coming in sideways, and it was it was so wavy But you guys were in there. You said you were fine Actually have a video of that five years ago or something. I'll see if I can dig it out Oh, you know what? It's a video of when I was doing building the fiberglass domes So I could find it pretty easily Maybe I should make a couple more Pedal wheels or well even one more pedal wheel with the pedals and everything I can put one right here And then I could have a paddle wheel going here and then one going back there And then I can have you know a whole bunch of people pedaling at once. I can make a kid-sized one Well, you you could do this one. I think actually Bellichicks go sit on this one. I Think both sides the pedals are the same size. So yeah, pick whichever one you want.  Yeah, I don't worry about just uh Yeah, so you can just sit right on this thing. I need a chair. That's a little further back Okay pedal forward Yeah, that way that's actually backwards, but it's forward for you Yeah, pedal pedal hard like try to like you're like you're trying to go somewhere You got to move a little bit more forward so you can actually reach the end Keep going keep going keep going don't stop don't stop I guess it'll be it'll be a little smoother when the boat's actually when the boat starts moving because right now every time that The paddles are in the water. They're in stationary water. I'm gonna get you and your sister on this thing Okay, I want to see how convenient it is for two people to pedal. Can't something to sit on that No, no, I want two people to pedal on the same one Like can we both get our feet on this one reasonably to do that? I need I need to make make actual pedals. All right, you pedal there I'll tell you on the on that boat over there Well, you want to go forward or backwards the pedals are actually wide enough that you can get two feet on Oh We're getting fun. We should be getting a little bit on the solar panels. Let's just Sparks, oh it doesn't work. Yeah, it sounds I haven't driven it in years It's maybe the oil slowly dripped out or something. I don't know. Maybe it's just got some I don't know I'll just check it out Yeah, there's my super cool oversized fiberglass But filled with concrete anchor What's really why there's my little windows for when you go inside that zone I just have to figure out. Do I want to do I want to lower it make it more? aerodynamic Or I want to keep it like this Which is better for You know just just being a workboat It's definitely a consideration. I Could just keep using this as the transport boat The problem is that boat can hold like four times what this boat can hold I could put You know fifteen hundred pounds on this boat Before I start getting well like over I can put a thousand pounds of cargo plus myself on this boat without getting nervous I could put a couple tons on that boat. No problem. Yeah, I could put the kids on you guys don't wait