Thanks everyone for joining us. We have with us today, Anupam Jain. He is going to be talking to us type reification with PureScript. So we're looking forward to hearing about that. Thanks. Hi, everyone. Welcome to this talk. Let me just share my screen. And we'll get started. So yeah, so the title of my talk today is. It's pure script typeable, which is a library that I wrote for PureScript, which basically provides type reification. And I'm going to explain what that means throughout this talk. But if you wanted to check out this library, you can check it out on my GitHub, which is AJ NSID. And you'll find a library with this name. So what is, you know, typeable typeable is a type class. And I'm going to be talking about that. But why do we need it? What does it do? So I'm going to start with a motivating use case of a very real world problem that people might encounter. And how does typeable help us solve that, right? So let's say you're building a database for a company. And it would look something similar to this, right? You would define all your data types, each data type representing a particular aspect of your data model. And these are, you know, very nested data types. So a company has multiple departments, a department has multiple subunits and so on, all the way until you have information about a person itself. And the person will have salary, name, address, those kind of things, right? Things that you might actually want to perform some operations on. So this is a very common situation that you're in that you have this very deeply nested data structure.  So in this case, we want to do something like this. We want to give everyone in the company a raise and we want to define a function called increase, which takes a salary modification function, which is this salary to salary function. And it basically takes in a company and returns a new company, which has all these salaries modified according to the salary modification function. So how would you write that? This is really straightforward. This is basic Haskell programming. But this is what it would look like. You would define for each level in that data structure, you would define a function that would do this modification for you.  But if you look at everything inside this red boundary here, you would see that all it's really doing is that it's going into that data structure and applying this modification function K recursively. So let's look at this. So increasing the salary across company basically means that we apply this function, which is a department modification function on all the departments in the company. So it's just a map. Then IncD is a department modification function. It takes, again, a salary modification function and it returns a department modification function. And again, we basically go inside the department data structure and we apply IncE, which is an employee modification function, and IncU, which is a subunit modification function. And we basically apply that. So we basically keep on going, keep on drilling down into the data structure until we reach what is the meat of this entire operation, which is actually modifying the salary, which is right here outside the red boundary. So let's focus on that for a little bit. So this is what that function looks like. And it's just identity because once you have a salary modification function, you know how to modify salaries. So this is the meat, the actual logic of what we wanted to do. We wanted to change a salary when we already have a salary change function. But instead of writing the simple bit of code, we had to write all this rigmarole around how to modify salaries. If we were actually doing this in an untyped, dynamically typed language, it would be very easy. We would have something like this. We would have something like this. But here we could just check if the value that we were trying to modify is a salary. If it is a salary, then apply the function. Otherwise don't do anything. And that would have been very easy. But because we want to do this in a typed safe way, we want something like this. We want to define a typed function that basically can take any type A and a salary modification function. And it gives us that type modification function. And this is what pure script type will allow us to do effectively. So this is a motivating use case. This is one of the use cases that pure script type will allow us. So let's look at some ways in which we could accomplish that. Given the problem statement where you have a class of types and you want to do something specific for each type, you might automatically reach out for type classes. So you might define a type class called map salary, where you have an increased salary function, which basically takes a salary modification function, and then it returns that type modification function. So before I continue on, there's going to be a lot of code on these slides. And if you have any questions, you can put them in the Q&A. And I would, if I think something needs to be answered immediately, I would answer it immediately. But otherwise I'll continue on. So you would define a class like this. And then you would define an instance of that class for every part of that data type. So let's say we wanted to define it for an employee, you would define it something like this. And it's a fairly straightforward implementation. You would call that function the salary modification function F. You would call that salary modification function F on every subpart of that data type. So you would call it on P, which is a person. So you would change that person's salary. And then you would call it on every, on the salary itself. And P is the personal data of that. And here we would also like to be able to handle cases where we don't have any salary information. So we also have a generic fallback instance where for any data type, which we haven't specifically defined an instance for, if you just pass that, then the salary modification function does not do anything. It keeps the data type as is. So this is one way of solving this problem. And it will solve the problem in a very specific use case where you want to just call map salary increase directly. If you wanted to call increase on a particular data structure directly, and the compiler has all information about what that data type is, then it can go through this instance chain and figure out what to do or how to do it.  And the problem is that we had to define an entire class hierarchy for that specific function. We had to define an increase salary function. And we had to define a specific class that only applies to that increase salary. So it's a lot of work just to avoid some work. So this is just as much work if not more. And we wouldn't like to have that much boilerplate. What we would like to do is have a general solution where the traversal through the data type is separated from the actual operation we need to perform on that data type. And we're going to look at how we can do that. So this is the kind of thing that we would like to do. We would like to have a function which takes a salary modification function, checks if the value that you were provided is of type salary. And this is not Haskell syntax. You can just check for types like this in Haskell, because all types must be known at compile time. And then, but let's say we were able to do that. And here salary represents the type salary. So if we were able to do this, that if A is an instance of salary, then just apply the function. Otherwise return the function as is. So let's try to write this. And here is what the signature of that function, instance of, would look like. It would take any type A and it would take something that represents the type of that A or some other type. And it would return a Boolean, which basically tells us if that value A is of this particular type. And as soon as we write this, we know that we need something, some value that represents a particular type. To be able to perform this logical operation, which compares the types of these two things, we need to represent types at the value level in some way. So what is that? So that's what we want to solve. And the standard solution for this in both pure script and Haskell is something called a proxy. Where proxy is basically a value that has a phantom type parameter, which represents a type. So instead of, so if you wanted to provide type information to a function, you can pass an a proxy instead. And this usually works, but it doesn't work for our case because proxy doesn't have any information at the time value level. So at the value level, it has only one data constructor called proxy, doesn't have any arguments. And all the type information is at the type parameter level with a phantom type. So if we wanted to actually start writing this, we would be stuck immediately because we know that it's a proxy, but there's no way to actually get the type. And we need to be able to get the type to be able to compare it with the type of it. The proxy doesn't suffice for us. We need something better. And in pure script, pure script is a language that compiles to JavaScript and it has very good FFI with JavaScript. So a standard solution that I tend to jump to is just punted to F5, punted to JavaScript. So you can define, so we don't know what the type representation is going to be, but let's say that JavaScript, the untyped language has that representation and it's able to somehow represent the types of every type in the type system and we are able to do a bunch of operations on it. So the way to do that in pure script is that you define a foreign data declaration. So we just say that there is some foreign data called type rep. We don't know what it looks like, but we can just access it directly. So we define something called type rep and then we need to be able to create values of that type rep. So we, again, in a foreign function interface definition, we define a function called type rep which takes a proxy, which here represents the type that we want to create and it returns a type rep for that type. Now the proxy A part of it is basically just to let the type system know what type we want. So it's not carrying any value at the type level, at the value level, but it's just for the type system. So we have a type rep which basically gives us a type rep for any type. So an example would be defining a functional Boolean type, which gives us a type rep that represents the type of Booleans and we would create it like this. We would pass it a proxy that is of type Boolean. This here is pure script specific syntax where, and I think this would be valid in Haskell also, where we basically want a proxy and the type of this proxy is proxy Boolean, right? But because this data constructor already tells us that it's a proxy, we just don't know what type of that proxy was. So we can omit the proxy type constructor here and we can just say that there's something of type Boolean. So this makes it slightly more concise and more pleasant to write. So to me it reads like it's a proxy of type Boolean, right? But actually it's a proxy of type proxy Boolean, right? So yeah, so this is how we can define a Boolean type rep by calling the type rep function. And then we want to write instance of, right? So we, one way to do it is to have, is to basically create a function called equal type rep, which can compares to type reps and that returns a Boolean, right? So instead of taking a type and a value and then comparing the types equal type rep basically compares to type reps and it tells us whether they're equal or not, right? So here to check if some value is an instance of some type, let's say Boolean type, we basically get the type rep for Boolean, right? And we compare it with the type rep of that type. So here this is how we are doing it. We basically are given a value A and a type rep T and we want to check if the type rep T, the type of value A is the same as the type rep T and we do that by first extracting the type rep of that value A.  So this way, if we already had these functions defined for us and this opaque data type, we are able to come up with an instance of function like this. So this could be our API surface, right? It's a very generic API surface, which has just three simple things. It has a data type declaration that's opaque to us. The users cannot create new values of this type rep, except by calling the type rep function where we specify the type that we want using proxy and then it gives us a type rep, right? And the type reps that we get have this property that if the type of A and B are the same, then the type reps of A and type rep of B also have to be exactly the same. So if we have a type rep true and type rep false, they both should give a type rep representing booleans and it should satisfy equality, right? So we should be able to just compare those two type reps and get through. So that is important. And we don't use the Haskell equal operation for this. We use equal type rep for this. So the equality is also an FFI function, which we use here, right? So it can compare two type reps. And then as I said, we can define instance of as a separate utility function, which depends on EQ type rep and the type rep function. So this API looks pretty good if you can implement it in FFI and we can. So I'm going to go through the FFI bits of it at the end, but this is one way of doing it. But if you wanted to do it manually in pure script or Haskell itself, then we would do something like this, right? This is one way of doing it directly in pure script where you basically create a data structure which enumerates all the types that are available to us. Right? So you can define a data type rep, which has a constructor for integers, a constructor for characters, a constructor for array. And now we need to also know the type of the elements of the array. So we have a type rep as an argument. Then a function takes two type reps because the argument and the result types also have to be specified. And similarly, tuple takes two type reps and so on. So we can enumerate all the types we have in the system, right? And then we can write an EQ type rep function, which basically does a direct comparison. So if it's tnt and tnt, if you're comparing two tnts together, they're obviously the same so it can return true, right? So there are some limitations of this approach, which we'll get into. But this is one way of doing it. You could actually implement this yourself. And then you want to create type reps. So with this manual data type approach, one way of creating type reps is you create a typeable class, right? And then you can define instances for each data type that you support. So you create a typeable class which has a type rep function. And then you define an instance for integers. So type rep when called on integers will give me a tn and so on. You have to define an entire chain of these. This else thing might be confusing for people who are coming from Haskell and don't, are not familiar with PureScript. PureScript has a feature called instance chains, where you can basically provide a bunch of instance declarations to the compiler and provide an order in which they should be checked, right? So you can define something like typeable int else typeable a. So for any type, it can check if, if it's an integer, it'll call the first, and it'll use the first instance. Otherwise it'll use the second instance. So this is a very useful feature for resolving instances. Right? So given that you can easily create the increase function as we wanted. You just check if a is an instance of type rep of salary, we want to check the salary, then do f a, otherwise just do a. And this will be a solution, right? Not quite because we immediately run into a problem. At this point, we have two branches, right? The first branch checks if the type of a is salary. But in the body of that branch, the compiler still doesn't know that. The compiler still thinks that a is some opaque type that it doesn't know anything about. And it doesn't know that it's salary, right? So this particular check is, needs to be propagated to the compiler somehow, right? We know it and one way of fixing this and not really fixing it, but when we're working around it would be to just unsafe coercive because we know that they are the same. So you can easily define a course to salary function, which can take any value and unsafe coercive to salary and then use that here, right? And this is obviously unsafe, but it's not any less safe than what you would do in a dynamically typed language, right? But this is not good enough for us. We want it to be safe, right? And that's one thing that type bill allows us to do. So we're going to continue down the path of our manual type representation and see how we can change that. So what we really want to do is, at this point when we do the instance of function call, instead of returning a Boolean, we want to return something that allows us to convert type A into a salary, right? So instance of will only return true if A is the same as salary. So it only makes sense that when you do instance of, it also returns as a function that allows us to do that conversion, right? And let's call that a coercion, right? And this is known as a type witness. It basically says that you have performed a value level operation which got the value of a type. And you know that this type is a salary or a Boolean or whatever. Now you also want to type witness that can tell the compiler that this thing is true, right? So in this case, our type witness is a conversion function which converts A to salary. And if we had such a conversion function, we could write it like this then, right? We can say that if instance of returns a value, a conversion function, and we indicate that by wrapping it in maybe so it can fail also if A is not a type of the type salary. So if it returns a just, then we also get a conversion function called coercion and then we can just call F by first core second, right? But if it returns nothing, which basically means that A was not of type salary, then we just return A as is, right? So this would solve the problem. But again, we start writing instance of and we immediately run into a problem. Instance of is supposed to return a function that converts A into salary. But the arguments to instance of, they don't have salary anywhere, right? So instance of should have a type, it should take a value A. It should take a type rep and the type rep here is entirely opaque. It doesn't say that it's the type rep for salary. We want to convert it to a salary, right? So it just says that there's some type rep. And if the type reps match, then it wants to convert it into the value represented by this type rep, right? Into the type represented by this type rep. But it doesn't know what that is. So there's no way to actually write this function and have a type check. We can't just come up with a type out of 10 error, right? Which matches this type rep. So there's no way of actually writing this. To solve this, we do something which is known as index type reps, right? So until now what we were discussing was unindexed type reps where the type rep is just an opaque type without any parameters. But to solve this problem, to be able to carry the information of the type with the type rep at a type level, you basically add a phantom type parameter. Oh, it's not really a phantom, not necessarily because it's completely opaque. So we don't know what it is. And we add a type parameter A, which represents the type that the type represents, right? So the type is now tagged with the type itself. And the API remains the same except now we have these types coming along with us. So let's look at instance off. Instance off will now have two type variables, A and B. It takes a value A. It takes a type rep of type B, and then it returns a coercion function where A, and we represent this coercion with a tilde. So it returns a coercion function which converts A to B, right? And ideally it should also return a function that converts B to A. If A and B are the same type, then we can go either ways, right? So let's represent, for now we let's represent that with a tilde. And again, this comes from EQ type rep, which represents that type coercion, right? So this is why we need index type reps rather than just normal type reps. And we haven't lost the unindexed representation because you can always use existentials for this. So type rep has a type parameter, but if you just wrap it inside an existential, you can get an opaque type parameter type which does not have a type parameter, right? And this is how you would do existentials in PureScript. PureScript does not have existentials built in, and this is how you work around them, right? So a lot of this talk is about how you work around some things in PureScript. And this is a library called PureScript Exist, which basically defines these definitions. And this is how you would do it. So it's just unsafe coercing it. It just forgets, right? It forgets what type it has. And then, so make exist basically just the unsafe coerces, a type representation here, F would be type rep. So it just converts a type rep A into a sum type rep without a type parameter A. And then when we want to get it back, we still want to make it type safe, right? So we have this higher order type here, which it's higher order because it has a for all in the type. So what we get is, so run exists as a function that can take, you can think of it as a callback, right? So it's a callback that can handle any type A, right? So it will basically be given a type A and it can handle any type A, right?  And because this is a higher order thing, it's the for all is inside the type of the callback. It's not possible to misuse this. You can't actually leak out the A from the callback. So you can't know outside this callback what the type is, but you just know that it's some type, right? That you can handle. So this makes a type safe and this is how you do existentials. And you can recover the unindexed representation using this existential, right? And then this is what the API would look like for the indexed API where type rep represents a type rep A. And here you will notice that we have gotten rid of the proxy argument. We don't need it because the way you're using type rep, just by the type of it, like here, we wanted a type rep of type Boolean. So we will just say type rep is a type rep Boolean, right? Because now the Boolean is a parameter to type rep. So you can just say that I expect the type rep to have a type Boolean and you don't need to pass anything to it. And EQ type rep now takes two type, type reps, A and B, potentially different types, but maybe the same. And we don't know if A and B are the same type and it returns a coercion wrapped inside a maybe, right? So we can use this coercion to convert between A and B. So this is what our API surface would look like and we can represent it. So if you wanted to do the same thing using our manual type representation, we'll have to use something called GADTs. With GADTs, you can constrain the type that you get from the constructors, right? So now type rep has a parameter A. So our Tint has to specify that this is not just a type rep, it's a type rep of int, right? So if we've constrained the type that this type constructor Tint returns, similarly, Tcar is a type rep of char and so on. So we have to tag all of them and then we have to implement EQ type rep like this. But the problem is that pure script does not have GADTs. So we can't write this actual code, but there's a very simple workaround where instead of using GADTs, we use normal ADTs. And we say that along with the data constructor Tint, we also carry again a coercion between int and A, right? So we are saying that it's any type A, but I'm also carrying a way to convert any integer to A and any A to integer, right? So they're isomorphic. So effectively this is the same as saying that A is an integer, right? And this is again our tilde, which is a type coercion. So if you use those type coercions in our type rep, we will basically be able to define this in pure script. And this is just a handful of constructors. You can obviously extend it to as many as you need. So for example, you would need constructors for employee salary and all those things to be able to use it in our example. And then we start writing our EQ type rep using this kind of a constructor, right? So here we have two Tints, right? IA is a type coercion between integer and A and IB is a type coercion between integer and B. And we have to return a type coercion between A and B, right? So how will we do this? We know that because both type rep A is a Tint and type rep B is also a Tint. We know that they are both the same because they're both integers. But how do we create that coercion between A and B? So this requires us to actually look at the internal structure of that coercion, right? So coercion is just an isomorphism. As I said, it has two functions. If we are saying A and B are isomorphic, they are the same, then we can convert between A. We can convert any A to a B and any B to an A, right? So let's define a data type called same. It basically carries two functions A to B and B to A, right? And this same happens to have a category instance. You can obviously define an identity function for this, which basically says that any type is equal to itself. Effectively, it's saying that. So we can define a same AA instance for any type A. And our conversion functions are basically just identity, which don't do anything. And we can also define this category composition function, which basically, if we are saying that we have a same AB and a same BC, which means A and B are the same and B and C are the same, then we also should be able to say that A and C are the same, right? And this is how you would write it. It's pretty simple. So the first same AB means that we have two functions AB, which converts every A to B and BA, which converts B to A. And then similarly, we have a function that converts any B to C and any C to B. And now to get a function that converts from A to C, we first go from A to B and then B to C, right? And similarly, when we have to go the other way, when you have to come back from C to A, we can first go via B. So we can convert from C to B and then B to A. So this value that we create is an isomorphism between A and C, right? And this falls pretty nicely out of this data structure that we created. So it's, so we have these two manipulation functions. And these are basically kind of like proofs, right? Which, as I said, ID is basically saying that each type is equal to itself, which is the identity function that we just defined. And then we can define a symmetry function that says, if you have a proof that says A is equal to B, then you also have a proof that says B is equal to A. And it's extremely easy to define if you have a function from A to B and B to A, if you just change the order around, then you will get the isomorphism between B and A, right? The other way around. And then we have the transitive function, which if we say A is equal to B and B is equal to C, then A is also equal to C, right? And this is just the function that we defined in the previous slide. So we have these tools that allow us to prove different things. And then using these things, we can define our function, right? So here, we come back to the question of how do we define an equiti prep with our manual type representation? So if we have a T end, which has the isomorphism between integer and A, and we have a T end, which has the isomorphism between integer and B, then we can get the isomorphism between A and B by just composing these functions, right? So to go from A to B, we first go from A to I, and then I to B, right? And this is composed, so it reads the other way around. First we do A to I, and then we do I to B, and we get A to B. And similarly, if you want to go from B to A, then we first go from B to I, and then I to A. And we have our conversion functions between A and B, A to B and B to A. And then we can compose our same AB like this. Or using the functions, the manipulation functions that we defined earlier, we can just say, if we have a proof that integer is the same as A, and we have a proof that integer is the same as B, then we have the proof that A is the same as B, because A is the same as I, and I is the same as B using our transitive proof, right? So we can compose these two things together. And we were not given A is the same as I. We were given I is the same as A, right? Where I is integer. So we just use sim symmetry to invert the type arguments. So we can just say that if you know integer is the same as A, then A is also the same as integer. And then we have the arguments in the right order to compose these two things together. So this is an example of how you would manipulate the proofs that you're already given into the proof that you actually need, right? And you can define functions like this for all types, for all the data constructors. Like the integer is here. You can define it for any type that you want, right? So, yeah, so this is again, we'll have a class typeable, which will be defined for all types that you need. So for an integer, our proof will just be identity, which is this identity is the categorical identity that we defined earlier, which is this one. So it's basically same identity identity, right? And then we can define it for array by, again, just doing identity like that. So we can construct type reps, typeable instances like this. And now we have a way of writing this function. We will say that first we check if A is instance of type rep. If it is, then we got a coercion, right? And the coercion is that same data type that we defined. And recall that the first parameter here is the conversion from A to what we, what is our desired type? So A to salary. So this coercion is from A to salary. So we can just coerce it to salary, and then we can apply F on it. And if we got nothing back, that means that A was not of type salary, then we just return A as is, right? So we can define this function finally like this. So an alternate API for this would be defining a cast function instead of a type checking function like instance of or EQ type rep. We can also define a function called cast. And the cast is basically a way to take any value A and maybe convert it to B. And the way it does it is it first checks if A is the same type as B by doing an instance of, and then if it is, then it just converts it, right? By using the conversion function. And then we can write our increase function in, I think slightly nicer way using cast. So we try to cast it to salary. If it is a salary and we got a just. So here N is a salary now. So we can just apply the function F to it. Otherwise we just return A as is, right? So this is a slightly nicer API. The typeable library provides both cast and EQ type rep and instance of, so it provides all of these functions. So we have been working with the same data type, which is here, but the library actually does not use this. The library uses something called libanese equality. So I'm going to go with that for a couple of minutes. Basically, instead of having two separate conversion functions, A to B and B to A, libanese equality states that for any context F, you can convert from F A to F B, right? So you have A wrapped inside some context. You can convert it to a B wrapped inside the same context. And this is just a single conversion function, but it's of the same power as that isomorphic isomorphism that we defined earlier, right? But it's a lot more flexible and a lot easier to work with because it's a single function. So we define our libanese data type like this and we define a type alias for this also, the delta. And then we define a run libanese function, which basically unwraps the libanese and gives us this conversion function back, right? So if we have a libanese equality between A and B, then we can convert it any A wrapped inside the context F to a B wrapped inside the same context F, right? And it just unwraps. So this is what it actually uses. There's a library called pure strip libanese that does this. I think that's what the name is. And again, libanese is a category, but you will see that the instances are much easier to write. So libanese, because it has a single function, so it's libanese, the identity for libanese is just identity, right? If you have an FA and A is the same as B, then you don't need to do anything. You already have an FB. And composition is just, again, function composition. If you can go from FA to FB and you can go from FB to FC, then if you just compose those two functions together, you will go from FA to FC, right? And that's what libanese is. It's a lot easier to write instances for. And this is the proof. You've got just a couple of minutes left. Sure, sure, sure. I'm just going to quickly wrap it up. And this is the proof of how you know that this is the same expressivity, because you can convert a libanese equality, a live data type into a same data type, right? And if you just look at it, this is how you derive it. So you know that they're the same. And I'm just going to skip over this. And finally, we come to the solution we hear this. This is a technique called scrappy boilerplate, which basically generalizes over traversals, over recursive data types, not recursive nested data types. So a GMAPT is like a generalized traversal over any data structure A, which, so using this, we can define a function called everywhere, which can apply a particular function to all leaves or all stages of a nested data structure, right? And when we pass it, our salary increment function, we basically get what we want, right? So this will take a company or any data structure which has nested salaries inside and it would increase salaries everywhere, right? So this is a general way of writing it without having to go into specific nested traversals, right? Some of the use cases for typable is, for example, you can have dynamic types. They're dynamic values. So the way it's done is basically you basically have a data type that both has an A and the type of A, wrapped up inside an existential, right? So the type does not actually specify the A part of it. The type just has the context T, but when you match over the existential and you get the typewrap out, you can use the typewrap for comparisons and figure out what the type of A is, right? And this is the way you do it. You basically pass it an expected type. You say, okay, I have a dynamic T. Is it a boolean, right? And if it's a boolean, it will give me a just that boolean value. Otherwise, it will give me a nothing, right? And you unwrap it and you check if they're equal and if they're equal, it does the coercion and returns it, right? Another way, another use case is serialization, deserialization where you basically store the typewrap itself. Typewraps can be serialized. So you store the typewrap. You serialize it with the value. And then when you read it, you can check, okay, I was expecting an integer and the typewrap I got was a boolean. So type store match, right? So you can have errors. Let me show you an error. So here we get the deserialized typewrap out. And then we compare it with the expected value. And if it didn't match, then we can throw an error which says expected type was this, which was the typewrap that was passed. But I found something else, which is the typewrap that we actually read from the serialization. So yeah. So this is the library. The library basically makes it extremely simple and safe to create typewraps. You don't have to do this manual representation thing because that's unsafe. And it's not very extensible. So the library basically just says that you can define your own takti instance. Takti is a class that the library provides. And it's a very manual instance. And the compiler also makes it impossible to get it wrong. You cannot write an invalid instance for this. It will not compile. So you just define an instance like this. And it will automatically generate a typewrap for you. And it does it using F5 hacks that I'm not going to go into right now because you don't have time. But basically it makes it very safe to actually generate typewrap and typeref for types. So yeah. So this is the F5 stuff. I'm not going to go into it. But this is the library. Please check it out. And I'll be happy to take any questions. Thank you so much for sharing about this with us today. It's been lovely to hear your insights.


Everybody forced here up in 22 23 prison basketball choice 10 box number 14. It's here random teams There are the results there link is in chat. You can find them anytime elate sports cards.com. Here we go Yeah, is it Dennis or Denny either way? Yeah, great stuff It's not built for this sort of event Not the type of like traditional home run derby got a kind of guy Also said the pitches were like inconsistent like I don't know and after a certain point He was just like well Well, this is this is not gonna happen But yeah, dude someone pitches were like up.  I might get I might get all lemon cookies. I might skip everything I just get the lemon a dozen lemon cookies I've probably half a dozen and and That would push me all the way in in diabetes Wow with a bang a silver max truce for the heat and Ocha egg Vaji jazz and no feet well Paolo Ben caro rookie signatures magic There we go pulled some nice palos tonight in the breaks Goes to Scott F with a magic Congrats Scott Get that Magda for you, buddy try colors of marketing for the jazz Matthews for the Rockets got purple nice Davion Mitchell Kings color match for 20 for 20 Purple 420 for the Kings Luda 49 Caruso bulls and red 88 K. Love calves. All right, Jay rods up here. We go Jay rod and Pete I don't not like super reading interest, but I kind of want Jay rod to keep going because he's the home guy Got silver Kendrick Nunn Lakers Ejela Dell rookie pelicans Sam Cassell on the signatures out of the Bucks Cold strange 420 card. What why is that one? Why do you that? Try color sixers hardened Paul for the Suns. Oh green RJ Barrett Nicks numbered to eight three of eight Nicks on the RJ nice card Luda 49 Steph Curry nice Warriors color match to 32 of 49 Dubs that's sick and red to 88 drew holiday bucks Yeah, I see I see Vaughan. I see your motives We're all three. Don't worry Sean calls people three, but he also plays for it nice. Oh, yeah What are you gonna do? That's silver McAlbridge's Suns Jane Hardy Mavericks rookie Warriors auto Anton James. Oh, we got two autos. I think we're just auto Anton Jameson and Hornets rookie auto Mark Williams Bonus auto action there got tri-colored assuming bulls tiger is a Wiggins for the Warriors Purple number to 20 norm Powell Clippers 5 of 20 blue to 49 siacum raptors and red to 88 McDaniels wolves so nice low numbers in this one lot of number of cards We're good to see guys nice swing going here pretty good pretty good. Oh, yeah, he's he's getting ahold of some That's got to be to 440 already almost almost almost Silver poku for the Thunder Powell for the Mavs Sean Kemp sonics auto going to the Thunder. Yeah, he'll get in it. He'll get in a minute here This is the longest one so far Yeah, there it is try color okoro calves uber a Hornets Clark Grizzlies purple to 20 Marvin badly the third Pistons and Red rookie to 88 is Grizzlies Vince Williams, Jr. Yeah, I think you're right rich I think I think Pete and Jay rod gonna light it up Jay rod knows he needs to hit a bunch though. I think he's out there and Pete's over there Okay halfway through he's got 14 that's pretty good actually that's pretty damn good He still has the bonus a fan is loving it. I like that silver perks for the Pistons cappella Hawks Pistons red choice out on Grand Hill It's cool when the home guy does well and everyone's like yeah Try color on a newbie Raptors The iron Fox Kings Ray Allen sonics red to 88 Zion for the pelicans and a Tyler hero for the heat Yeah, yeah, Denny. He was the first round pick for the Patriots last year Polar bears in his cave getting ready. Yeah Got silver for the Cavs Jared Allen and spurs Trey Jones Green choice auto Pacers Roy Hibbert to eight Roy Hibbert three of eight auto Pacers Tricolor THT for the jazz Fultz magic Diabata Clippers red to 88 K. Love Cavs and Blue to 49 John Stockton jazz Well, they're all guys that have that bat speed And they have the swing built for it like movies a fantastic player, but he's more of like a doubles guy and like Got a hold of some homers here and there he can hit some long ones, but he's much more of a rounded player than some of the other ones Jayrods like Mookie with who he had more power Silver Dwellin beat Sixers and Bo-Champ buck Shurkey Nuggets Dan Issel red choice auto go nugs try color for the Thunder SGA Mavs Tim Hardaway and T Wolves went the more junior Red 88 Dame for the Blazers and blue to 49 Vince Carter Raptors He's hit for oppo as well. It's pretty good. Oh That's this year. So you don't really hit oppo home runs in the very your pictures all over the place You're just swinging outside pictures. That's silver KCP nugs moody warriors Thunder Auto Trayman Yeah, Alonzo has got some work to do No, he still has a minute. Wow, he's got 30. He might have more than that try color Tatum Harrison Barnes TJ Warren red to 88. You've got Demba sell spurs and Jake Crowder Sons This is all unfair. That was awesome. He was the eight seed You know, I guess he only has 30 13 home runs this year. He got a tough draw He got screwed Oh see Jayrods going for the going for that Vlad record Got silvers of Kaminga and McDermott King's rookie auto Keegan Murray, there we go Kings got tricolor progden Celtics Mitchell for the Kings and a green Demba sell spurs to eight There we go six of eight spurs blue to 49 Jimmy Butler heat and red to 88 Jimmy Butler heat Ali was easy. So Lou Bob was the ones you know, okay, okay? I had a wrong then they've three and 60 no three and six is mookie and flat They're two and seven All right quick recap here. We had autos of Keegan Murray Trayman Dan Issel Roy Hibbert to eight Grand Hill Sean Kemp Mark Williams Anton Jamison Sam Cassell and Paolo Ben caro. Thanks again everybody moving on to football


Come on. Defconn, welcome. Mowder. Okay, I'm James Kirk. No, I'm not, but he is. Is James Kirk? So, let's see who's got. Alright, so I am James Kirk. I am the captain. So I'll be giving a talk today on defense industrial base controls. It's basically how classified systems are used in defense contracting companies. So, I have to put this disclaimer in because I've already been catching a little bit of flak. So I'm going to tell you what this presentation is and isn't. There's nothing classified in the presentation. Everything that's in it has been obtained through a Freedom of Information Act request or is publicly available on the Internet. But the FOIA requests do take a long time, so I've done everybody the ease of including everything that I have in this presentation is included in PDFs on the DVD. So you don't have to go hunting or filing your own FOIA requests. I'm also not giving any vital secrets to help the evil terrorists. You know, we don't want Alcoa to get in there. And the data we're talking about is purely collateral classified information. So in the classified world, there's three different types of classified information. There's collateral, which is basic confidential, secret and top secret information. There's also information called SAP, which is a special access program. And that's things like the stealth fighter program, things that are so super secret that there's only a small amount of people that know about them. And those are sometimes exempt from even congressional oversight. So that means that not even Congress knows what these defense contractors are working on. And the last type is SCI. It's special compartmentalized information. And this is mainly intelligence information. So things like that are under the oversight of the department of, or DNI. So none of the security control gaps or deficiencies identified are classified.  This is specifically to contractors. In the Department of Defense and the NSA and the different departments throughout the government, it's completely siloed. So in the DOD world, everybody uses Diacap. And, you know, if you go to NSA, there's going to be different requirements there. So this is purely information that's derived from an agency called the Defense Security Service. So a little background on who I am. Well, my name is James Kirk. I'm currently a senior security consultant for Rapid 7. But this talk is solely my own opinion. And it doesn't represent the company at all. I have to put that disclaimer out there, of course. I used to be a special agent for the Defense Security Service. I was with the agency for a year and eight months before I just couldn't take it anymore. So this topic is mainly going to be about the DSS, how they implement controls, how they develop them, and some of the major flaws in their security controls. Now, unfortunately, it's going to take me a little while to get there. So I hope you guys hold out. It's just because I want to give some background. A lot of people are familiar with how classified controls are put in place, how they're developed, and how they're enforced.   Last day, I want to go through, you know, the structure of the agency and some of the federal standards that are in use in the other agencies. So how are these controls developed, or what enforces them? So there's a group called the NISP, and it's, I don't want to say it's kind of incestuous, but it is. So the NISP is actually, it's run by the U.S. government. It's, that's the National and National Security Program. But the NISPAC are the people who have the influence in it. And so you have multiple agencies, 16 representatives from defense contractor companies, and one representative from the DSS. So you have these 16 guys who are, have invested interest in what these security controls are. So if you're, you know, I'm not going to name names, but if you're one of the biggest defense contractors in the industry, am I going to say I want the toughest security controls in place? That's going to cost me, you know, two billion dollars next year to implement. Or am I going to have the most lax security controls?  So it's, it's very, it's a very invested interest. So, you know, I don't think it's really, I don't think it's a good to have such a, you know, dominance of defense contractors in place when you only have one representative from the DSS. And lastly, the NISPAC and the NISP work together and they develop a document called the NISPOM, which is a National and National Security Program operating manual. And this document is the Bible for how controls are put in place. And the last time this document was updated, it was 2005. So, not really keeping track of changes in security controls. So, starting with the structure of the agency, there's four main directorates, IS, which is industrial security, CI, which is counterintelligence, Disco and CDSE. So if you have worked in the, in the classified world before and you have a security clearance and you're not in the government, you've got your security clearance most likely through Disco. That's what most people know about DSS is that, hey, they got, they did my security clearance, they did my background check. That's who they are. Well, the funny thing is it's only like a small percentage of what they do. CI is just how it sounds. They do counterintelligence. If a defense contractor gets hacked or if there's a foreign agency that's trying to target that company, a counterintelligence group will go out there and interview them. I've got some slides at the end of the presentation that go over some of the information that has been aggregated from counterintelligence. And industrial security is the primary portion of the agency. And they're the ones that do the inspections of contractors. And so ODAA is a subgroup of industrial security. It's called the Office of Designated Approval Authority. And these are the guys that handle just the computer systems that process classified information. There's four regionally dispersed DAAs. These are designated approving authorities. And these are GS-15s. And if you know anything about military SQ, they're the equivalent of a colonel or a captain in the Navy. And so they're pretty, I mean, they carry a lot of authority. And they're the ones who derive the security controls based upon where your company is in the country. So it's west coast, southern part of the United States, east coast, and then the capital region. The agency is run by a few SESs. And these are senior executives that are pointed by the undersecretary defense. And so the main thing I'm trying to get here is the DSS is ultimately responsible for enforcing everything that's in the NISPOM.  So how does a company process classified information? It goes through a certification and accreditation program. And so when I worked at the agency, basically what would happen is a contractor would say, okay, we want to stand up based upon our contract, you know, 50 or 100, you know, let's say their secret systems. And so what they do is they fill out this, you know, 15 page document. They click on a couple things and they send the document to me. I have 28 days to review it. I look at it.  I'm just looking at what they put in there and make sure that they've signed it, that it's correct. And then I send the document up to my boss who approves it. And he doesn't review it, he just makes sure that I have all the appropriate documentation sent to him. So after it's approved, the contractor can then start processing secret information or whatever level they've asked for for up to six months without any oversight, no validation if they have security controls in place. It's just based upon their trustworthiness. Now, and then they can ask for an extension and get another 180 days. But usually we have to look at them between six months and a year. And when I'm working with a company, I'm only working with one person who's been designated as the information system security manager. And this can be, I actually have seen it a few times where this guy graduated with a master's in business, worked at Enterprise Renekar for a year, and now he's been appointed to the ISSM. He has no security experience, doesn't know anything about information security. And he's the person who's ultimately in charge of, in this case, a Fortune 500 company running, you know, thousands of computers are connected doing top secret information. I don't know why, but that's the way it is. I don't get to choose who they pick. But the only thing that stops that is there's a regulation in the NISPOM that says they have to be trained to a level commensurate with the company, or with the level of the information system. And that can be, I mean, that's so ambiguous that you can never really enforce it unless they do something, you know, disastrous.  So my role was an ISSP, information system security professional. During that six months period, we schedule an onsite visit with the company. And so, you know, I e-mail back and forth to say, hey, what's a good time? And we go out to do a visit. Now, this is for the accreditation portion. And this is within the six month period, we go out and I spend maybe a couple hours with the company. And we don't have any tools to do the evaluation. We're not allowed to run tools on the contract or systems. So all I can do is ask them to click through things and show me that it's configured correctly. And, you know, it's based on a simple checklist that says our password is in place, our passwords, X characters, et cetera. As for applications, we don't really care what applications are installed in the system. We don't care what patch level the applications are at. And it's a pretty generic accreditation. So after I do that, and then I do some more paperwork again, and I send my boss and he says, hey, they're good to operate for three years. So, okay. Then within the, so these companies, if they're not processing computer systems, they're still subject to an annual inspection. That's where we go out and check, make sure that they have access to classified information in the safes, actually have clearances. So we now add them under as part of their annual inspection.  And this is a lot different than the accreditation. This is when I go in and we just basically tear apart the company. We open up all the safes, all the drawers. We have basically unfettered access to anybody who might have access to that classified information or the systems. And that's where it was a little more fun because then you go in and you just get to sometimes tear apart the company. And so if we find any discrepancies there, then we fail the company or give them the lower rating. I'll go into that. So, and that's kind of out of place here a little bit, but what I'll do is they'll do the security controls first. So these are security controls that are companies are required to implement. And starting with Linux, I kid you not, this is straight. I have the reference here. This is the only requirements that they must implement that are specific to Linux. So if you have experienced Linux, I hope you understand that this is not sufficient. So when I first got to the agency, I knew this was a contractor already. I was kind of on a mission to when I got to the agency I was like, well, there are security controls are crap for Linux at least. And so I'm going to try to work to implement some better controls. So my partner I within the agency, we spent about seven months when I got there developing security controls for Solaris 10 and Red Hat 5. And these were supposed to cover both the Unix and Linux variations. We put some generic remarks in there so that if it was Solaris specific that they could implement it, but it was just basic Unix that could implement it as well. This document ended up being about 156 pages. And it sounds like a lot, but it was mainly like, I mean, like we're talking about enterprise rental car people here. So it's like, click on this button, type this command in. So you have to really baby step people through it because if we ask them to implement something that they can't, our agency gets blamed. So this is, so when I was talking about the NISPOM being the Bible document, that is very true. But as you see the reference is, it says ISL. And so the agency at any time can issue a thing called an industrial security letter. And these clarify sections in the NISPOM. And their attempt to, the NISPOM didn't address anything in Linux at all. And so they came up with this ISL in 2007 saying, we're going to make sure that we protect Linux appropriately. And so here are the really important things you need to protect. You need to turn Audition on. So here we go. And then in 2009 they decided to clarify more. And that will be on the next slide. And in 2009 they came out with another ISL that I thought would cover Linux but instead it just covered Windows. Assuming that, I guess they're assuming that every defense contractor purely uses Windows but that's actually the exact opposite. So I did a quick comparison here. I hope it came out okay on the screens. But DISA, this is another agency in the government. It's called the Defense Information Systems Agency. They're actually come out with controls for like SIPRnet or the NSA net or FBI net. These are really, I think they're pretty secure. I mean as you can see they cover a lot of different things. I've compared, a quick comparison between what the Stigs call for and what the NISPOM and ISLs cover. So we don't cover discretionary access controls at all. We don't care if you use privileged commands.   And we don't care if security personnel do anything either. And so it's kind of funny to me because some of these things are the things that we should care about the most. Because if I'm processing classified information in the company, I'm not caring about some ninja trying to sneak in, break through a secure room. We have locks on all the rooms. We have motion sensors in the rooms. These are all requirements. And I can't recall a time when someone actually broke into a classified room and stole stuff. What we care about is the insider threat, the disgruntled employee or some foreign nation person who's been infiltrating into the company and is on a mission to steal stuff inside. And the only way that's going to happen is if we're tracking and auditing people, printing stuff off or exporting the media. And I'm going to get into it at the end, but this is a perfect example of the WikiLeaks. Whether you're for or against it, this guy was easily able to take a CD and burn stuff all day long.  But the DSS did not agree with that. They said it would cost the contractors too much money. One second here, sorry. So in 2009, like I said, they came out with Windows baseline standards. This was a similar size to the document I wrote, about 150 pages. It came out with a document called the standardization of baseline technical security configurations.  Here's the thing, though. I mean, a lot of companies have fought this because the NISPOM is the only document they have to follow. And the baseline standards clearly in the front of the book says this process manual is not directive in nature. It just says that if you want to get approved in a timely manner, you have to follow the document. But what if I'm a company and I don't want to get approved in a timely manner?  And then lastly in 2011, they actually worked on a document that modeled NIST 853 standards. And it does pretty good. I mean, NIST is also based on what the STIGs are based on. So I thought they did quite well with that. However, it doesn't matter when these documents came out. If you are approved in 2009, you don't have to update until your next accreditation date, which is three years. So you can still run unsecured for three years, in my opinion. Of course, Linux is still left out. I guess they still think those seven different things are the, you know, and I'll be all of Linux's controls. Some of the major changes in the June 2011 one was they've moved to 14 character passwords finally, which has been a standard and in place for quite a while with NIST. However, NIST POM still overrides that because it says they don't have to use eight character passwords. So we've been, when I worked at the agency, I was fought all the time. They finally addressed patching. So in the past, you never had to patch in your applications. It wasn't required. And so now it just says that if the ISSM, which is your enterprise Renekarp guy, thinks there might be vulnerabilities resulting from software, they have to patch it. Now, it doesn't say, it says expeditiously. But as a special agent going to do an inspection, what does that mean? I mean, how do I hold a company to an expeditious timeline? It's not really fair to the company. And it's definitely not fair to the agency because you can't, you know, that's not an enforceable standard. They also sort of addressed USB drives. They said that they must be tracked and accounted for. What that means, again, is up to interpretation. And they should be disabled if possible. And that is an unenforceable action as well because all the company has to say is, we run USB keyboards. Okay? Well, cool. You don't have to use USB drives. But oddly enough, there's an item called DivNet. And this is for, if a company's been hacked, they report this information through a system called DivNet. If they're a big contractor, no one cares about the small ones.  And so if you do have a DivNet console, you are required to have USB storage disabled. Now, why that matters between a system on DivNet and not? There's no, there doesn't seem to be anybody who can tell me the answer to why. From a wrist standpoint, I'd like to know why, I do know why. It's DC3 and they do care why they don't want people using USB storage devices on their system. So. And the last one, oh yeah, the things that are major changes are successful log-ons, et cetera. They kind of went into more what auditing capabilities are. The last one is security seals. And I kind of find this funny because of the big tamper evident contests every year. They have to be approved tamper proof seals that are pre-numbered. Now, I can tell you internally, there is no approved list. So when an approved tamper proof seal is, I don't know. We just go out there and if there's a number on it, I've seen people put marker number on it. Cool. Works for me. And there, it's, so when you process, you're supposed to make sure no one's put a, you know, key logger in your keyboard. And then you just look for the tamper seal. Okay, so how do these controls even get developed?  Well, I can tell you, there's a GS14 who works at the agency headquarters who used to work for a defense contractor.   Like, that sounds cool. So he puts in the document and sends it out for 24 different federal agencies and they either give it a yes or no. And if they don't answer, it's approved. And then they issue this document out and here we go. So I mentioned earlier that my friend and I, we developed this document. We sent this guy and we also sent it to, as a GS15, the head of the ODA. And the answer was thanks but no thanks. Like they just didn't, they didn't want it. They didn't want to deal with it. They said that they had sent out a previous document that was, I kid you not, it was a copy and paste from Oracle on how to implement Solaris. And it was just a copy and paste from their implementation guide. And the 24 federal agencies, this is, now this is an honest thing. If you, if you know Linux really well and you want to work for the government, you will get hired. Because within the entire agency I worked there, there was only four guys who could do an Linux inspection at a company that has primary Linux systems. And so when we set this document out, two of the four guys were the only ones who could review it. So it didn't get reviewed. They sent to the agencies and the agency said, we don't understand what to do with this and so we're not going to say yes or no. So the agency that I worked for said, we're just going to pass. So there's going to be no Linux standards yet. Okay, so when we were, when they're developing the security controls and they're saying, well, how do we test these before we say a defense contractor has to use them?  Well, we use a lab environment which consists of a laptop from 2006, a hub, and another laptop from 2006. And, well, when I first got this lab environment, it had windows on it and I didn't have access to the BIOS and I couldn't install Linux unless, well, we just pop them, board out and reset it. But that's basically, that's what they have headquarters as well. So honestly, the test resources are limited and one of the biggest complaints about the agency is that the controls aren't developed. So back in 2009 when they first came out this huge document on how to implement windows, they did and basically the people who actually follow the document line by line ended up locking their systems out. Because it makes them turn a shut down, shut down of an unable audit which is a good setting except that they turn an auditing feature on that logs like 10,000 audits in like two minutes. So as soon as they turn it on, it shuts down the system. Hey man, it's the federal government, what can I say? I'm not, I'm sorry, I'm not trying to make this into a bash thing for the federal government. I mean, the whole point of this is to, I mean, I just want them to adapt. I mean, there's great security controls out there. You know, if they just used them it would work fine. So how does enforcement get done? It's through a credentialed special agent, it's through a 0080 which is the guys that do with the physical security and there's also 2210s which is what I was, IT specialist. So when you get onboarded, they do a phone interview and they say you're hired. And the phone interview was about 10 questions. The one question that referenced Linux was how do I show running processes? And then I was a certified Linux guy.  These special agents, they don't carry guns, they have no law enforcement authority, they just carry a badge and the most we can do is throw it at somebody. So it's kind of, the only cool thing was I got an undercover car when you get pulled over. It's registered to the attorney general of state of Texas, so it's kind of fun. So for training, they onboard you and they send you to the DSS academy.  Cause it is on Quantico. But you get there and it's a two week course and they teach you how to do an inspection. Which is basically getting on a ladder and looking around in the ceiling to make sure no one's hiding up there. And I'm like, okay, so when are we getting to the computer side? I mean I'm an IT guy, I'm not here to do physical security inspections. Like oh, it's coming, it's coming next week. Okay, so go through the first week, cool. I'm now certified to do physical inspections. Second week they're like okay, so here's how to do a computer inspection. They put a Windows PC connected to a hub with another Windows PC and they give us a checklist that checks to make sure that you can log in and that there's password and now I'm certified to do PC inspections. They call them peer to peer lands. And so I brought the question because I used to work for one of the big defense contractors. I said well what about like a 5000 node WAN? I mean we had that at my company. Oh well, you know that's just OJT. So well I work in Texas in an office by myself, so how much does it get OJT? Now I already knew how to do the work. You know, rhetorical questions to see what their answer was. And they don't have one. And that's the extent of the training. And the thing is that this training is to encompass either 0's or 8 0's and 22 10's. So you have these guys who have never touched a computer to do inspection. These people that are actually physical security specialists that are now doing computer inspections. They're the experts out there certifying systems. And so this is a job requirements you have to be to be a GS13 in the agency. Which is actually a quite high and highly highly paid position. There is no Linux requirements you can see that.  But you don't have to have much experience at all. And the people that are interviewing for these positions, basically if you have a CISSP you're hired. Because the people within the agency have such a hard time getting terminated. So if you have your CISSP you're golden. Because it's a 8570 IA level 3 position. Alright, so what happens during an inspection? I kind of went over a little bit. If we go in it's depending on the size of the company there's a few different sizes. There's double A's which would be like a company that makes airplanes. And they have a huge, I mean they're building strike fighters. Well I'm not going to name names but these are big facilities. So we have like seven guys that do IT inspections there. And it's, you know sometimes it's complex. They put the guys with the best skill sets to do the portions of the inspections that they're good at. I always get stuck doing Linux and network controls like firewalls. So, partners with industry. So this is kind of a new term that happened since I got there. Now, if you remember with the oil rig out in the Gulf of Mexico the mineral management agency was responsible for doing the inspections of these oil rigs. But they're kind of incestuous as well with the oil companies. It was very lax oversight. Not very strict controls. Sometimes they didn't do the inspections. And they were partnering with them because we want to make sure that they make money. And so that's the same thing. When I got to the agency before it was always, they're a compliance oversight agency. We have to hold, you know, the company to a standard. Well, when I got there there was a lot of complaints that defense contractors thought we were being too hard on them and we're not letting them make enough money. So, we had to partner with them now. And so what does partnering with mean? It has, it's another ambiguous term. All I know is that I had a GS-15 inspection at a big organization and before we could give them the daily findings that we found, we had to review them with a GS-15. And I had two different findings, just an example of auditing not being done. They have to audits weekly. And passwords weren't complex enough. And he's like, well, why can't we combine those findings in the same one? So, well, they're not the same thing. But because the inspections are done by them, and if they get 15 findings, for example, then they might fail. So, if we only give them 10, then they're good. And we don't want to fail one of the big five contractors. And so, if you're not compliant, and I've actually done a few failures, then we it's noted in the security log, they get enough findings, then we give them a marginal or an unsat rating. A marginal just means they have to do a reinspection. And if they fail it again, then would they go unsat. And if they're unsat, they get two inspections to fix it. And if they don't, then they're, I think their contracts get suspended for a certain amount of time. All I know is that I went to a location and they had a top secret. It's called CINWIDI, critical nuclear weapons data. It was a document that contained basically all of the missile silos that are hidden, that are secret in the United States. And it was a filing cabinet. And so, we're like, okay. So, we take the document and move it to the Air Force base and secure it in a skiff. And the director of that company had already called the director of my agency and was pissed off about it. I guess they thought a filing cabinet was appropriate storage for a top secret document. And by the, I mean, if you're not familiar with it, top secret documents that be stored in a room, within a safe, and the safe has to be alarmed. So, here are some common vulnerabilities that are actually pulled from the ISS website that are found. Not auditing is the top one, which is kind of funny. Not reporting classified compromises. That's great. Poor safe combination, security, processing on under credit. This is great.   We would take the hard drives and put them in the safe and make sure they didn't use them. It's a good thing to submit that document. Can't tell them when they have to submit it, but they should submit a document for accreditation as soon as possible. Okay, so an overview of some of the inadequate controls that are in windows. Patching, like I said, it's very ambiguous. And I mean, we can't, I mean, if the document's not patched, I can't tell them to patch it. I mean, I just have to tell them that they should expeditiously patch the system. Okay. USB, that's how stuff gets taken off the system. Virtual environments. If you're running VMware or, you know, any other virtual system, excuse me, there are no security requirements for it. Do whatever you want. UAC, you know, with Windows 7 and Vista, not addressed at all. Classified data is not audited. That's another thing that bugs me.  I just care if you're taking classified data off the system. Training is another one that bugs me, is that we're not required to have these guys, you know, sufficiently trained. I think that there should be a firm training requirement for the ISSMs, and it shouldn't be a person-firm enterprise. So for Linux, honestly, there's just too many to list. I kind of did an overview here. The problem is there's a lack of expertise in the government. You know, if you drive down I-35 in Texas, you'll see signs everywhere. It says, no Linux. We'll hire you. This is a good thing to learn, if you're not familiar with Linux or you don't feel comfortable with it, there's job opportunities out there all over the United States. This is the bottom one. It's kind of funny. I went to a company and they didn't have user finding. They fought me on it, and I was told that nowhere in the documentation does it say that they have to have the audit.rules file configured. So they don't have to have any auditing flags. So they're not going to audit anything, and it's not a requirement. So same issues that affect Windows as Linux, patching, USB, virtual environments, and tamper controls. Like I said, there are no approved tamper controls. So there are no approved tamper seals. So they can use whatever they want. I've seen people use, like, not tape, but like kind of laminated stuff, and then they put a number on it, and that works. Okay, so kind of moving through this pretty quick here. So the whole point I think is if you do work in this world, you know, I'm not trying to bash the companies. I'm not trying to bash the agencies. But the point is that I think there should be a partnership. There shouldn't be this term of partnering with industry. There should be a real partnership with if the government won't come out with controls, you need to not stick your head in the sand and have an ostrich effect. You shouldn't say, well, they didn't tell us that we had to have 14 character passwords on Linux systems, so we're not going to implement it.  I mean, if you really are implementing the security controls and you have the experience, I think it's your responsibility to make sure in the absence of security controls. And, you know, with the government, you know, help them sorry to say, you have to help them develop the controls. Me giving this talk is not going to make them come out with a Linux document tomorrow. It'd be nice if it did, but I don't think it's going to happen. So, why does this talk matter? So whether you believe it or not, there are real governments out there, nation states and even people that are targeting the U.S. technologies. And so I've actually taken, this is the counterintelligence group within the agency is fantastic. And the only downside to counterintelligence agency is that now if the NSA detects that your company's been hacked, the counterintelligence agency or counterintelligence group will go out there and talk to you about it. And so will a representative that was like me. Except we can't tell you who hacked you. We can't tell you what they hacked it. You've just been hacked. Sweet. So, here's some actual real data of what this is for East Asia and Pacific. If this was a classified briefing, I could give you the real countries, but it's not. So we have to be generic here. Just, you know, put it on a map. This was for FY09. So from October 2009 to October 2010.  Doesn't work so well. But solicitation is seeking employment. Now this is only 6%. And I think it goes up in the next slide. But what they're doing is they're sending these resumes out and they're like, well I'd like a job and work in a classified world. And sometimes you have the HR people who are like, well I don't know where they're from. Excuse me. So I hire them on. They do a background check. And hopefully DSS sees that they're from China and that they're not using China's. I'm using China's again. But it can be anywhere, you know, where they have people that work for part of the PLA or work for the Kremlin and they're being paid to infiltrate a company and steal the classified information. And so that's what the seeking employment is. Suspicious Internet activity? I honestly don't know what that is. But I guess it's probing the network outside network. So now it's great that it's a suspicious network activity I guess is going up. It's where they're actually massively targeting the company. I guess seeking employment did go down. I think that's because a lot of the times when they do get hired on Disco does a pretty good job of weaning out potential infiltrators and they just deny them clearance. Conference and trade shows. There's a lot of there's a lot of defense contractor trade shows where they fly to France and they try to steal airplanes. And so there's a lot of foreign nations that go out there and target these. So this is for Near East and this is FY09. So employment's quite large there. Direct requests and then here employment's gone down academic solicitation. This is interesting where we've gotten a lot of suspicious reports where a there's a professor. We used to do inspections at we do inspections at big like universities do a lot of research for the government. It's actually pretty cutting edge and so you have guys that are like, hey I'd like to come to your school or I'd like to do a research project with you and they miss Billy Guy's name. It's like a Nigerian email scam sometimes. But they're trying to work with this guy who's working on a classified project. This is for Europe and Eurasia and then this is FY10. And then this is South and Central Asia and then FY10 as well. So South and Central Asia they have a higher percentage for solicitation to seek employment. Now when I first did this talk I didn't think I'd be in track one so I actually brought the book with me to share because there's more information in it but it's actually quite a large room so if afterwards you would like to see some more information on it I'll be in the QA room and I can actually let you take a look at the document where this is derived from. So wrap up and I hopefully have some time for questions in here as well. So why the talk, right? So like I said the talk's not meant to bash the agency, it's not meant to bash the companies but I think honestly the federal government's been kind of not necessarily federal government, I'll say the DSS has been remiss in developing appropriate security controls to protect classified information in multiple areas and I don't want to say that it's primarily because of money but it really is and what happens is when we when we start developing new documents like the new NISPOM is coming out I believe in 2013 is you have 16 representatives from different contractor companies that are they're the ones who get to you know they're the ones who get to help draft this NISPOM and so it's you know like I said in the beginning is it their best interest to develop the most hardened security controls that are going to cost them most money and hire the most technical people to implement or are they going to go for lax controls that allows them to just be willing to with implementation and hiring people from enterprise so you know that's I think that's why I mean this is my personal opinion too so you know you can think whatever you want of it and I also think like I said in the beginning that it's important if you are a citizen to know how they're implemented and I mean because ultimately this is your data right I mean the federal government should work for us not the other way around and you know I have to put some buzzwords out there too so stuck set in flame right so these are malware that we now know that have been developed by the U.S. government with in conjunction of other countries and how are those how are those put in place well I mean you have to have like these systems are not available on the internet they're not but you have to have an insider that puts this malware on the systems and I mean what's stopping that from happening here there's nothing there's absolutely nothing stopping people from coming in an insider and putting you know malicious malware on our systems and harvesting data now I'm not saying that security controls will stop it but at least at least to help audit it it'll tell us what happened or it'll slow them down and you know we can stop you know we can stop you know like I said you know either forward against it I think that some of the information that's been released is good but you know a lot of the information is I mean it's classified for a reason so I think sometimes the government overclassifies stuff but you know anyways I do have all my references in the document too available on the CD or the DVD and you know I guess if anybody has any questions I'd definitely like to answer some uh oh do we have one here I'll give it a shot sweet so I've worked on government projects of various sorts and one of the things that's frustrated me is that between the NIST guidance and the DISA guidance and the NIST Palm stuff not only are the guidance for auditing kind of not great the tools suck and there's no decent way of creating a good build a known good build do you think there's a lot of room for that do you think that the industry would respond if you said alright here's a kickstart for Red Hat Linux 5 or 6 that that builds it secure and go with that well I can definitely answer that so back in actually about two years ago they came out with this it's called a NIST tool and it's supposed to work quite well it configures your system for you and you're good to go but it only works on windows of course but that's another tangent but anyways what happened was we went to I didn't go but it was part of another inspection at a large defense contractor and the guy accidentally hit configure instead of verify like the DSS guy this is when they let us use tools on systems to automate and help our inspection process so hit configure instead of you know inspect and so reconfigured their system and totally destroyed it and so that was the last time we were ever allowed to use tools or put them out so the answer to that is I don't work with the agency anymore so I'm not sure but I would say no thanks so have you heard so have you seen difference between different security systems security agencies between levels of security as example like at least in the middle of the picture between like someone like CIA versus FBI okay so um it's so the to be honest the FBI is like they don't like to play well with others like they do their own thing so they have their own requirements now DISA and the NSA do work together on items called STIGs and so you can just google DISA STIG and it has configuration requirements configuration settings for almost any device any system any operating system these guys have a huge well I know yeah I don't to some extent they use generics but they uh like they've done they've done a lot of testing with them they work really well the problem with is that they just they're afraid to use DISA STIGs because they're really harsh requirements we say they like the agency is the DSS okay and the actual their official answer is that we are we're using the national industrial security program operating manual we are not the Department of Defense so we're talking about like the corporations so they're a national entity not a federal entity so I think it's a crap answer thank you hey thanks for coming out here and giving us talk I totally appreciate it so your recruiting speech has been amazing knowing that their bar for employment is so low and their pay is so high I had a couple of questions related to that taking a look at the job listings first off are they hiring in the bay area are they paying and third off how much actual experience do they have because I have a friend who's not technical at all that ran out of a job they could use one so it's kind of a self centered question so that's pretty much what I'm asking well I applied so I applied to a company I applied to the DSS in February I never heard anything back so I applied to another defense contractor and got hired on in almost nine months later I finally got back and I said okay whatever so actually for GS13 the starting pay is 81 or actually it's probably 82,000 now which I mean I'm saying high so I mean this is for government position pretty good benefits in the bay area it's probably going to pay around 95 and you just got about like six to nine month lead time but I'm sure they're hiring actually when I checked the job postings yesterday before I updated my slides they didn't have any positions opening but the 2210s are considered critical fill so they're they're going to be hiring all over the agency I guess excellent thank you very much yeah go for it yes what well I told them I wasn't going to move unless they paid for it so they paid for me but most of them are no PCS so I was going to say feel your pain I know it's like starting the support group for people like us meets in the bar after all the talks I'm in but I just want to comment you know the diet cap process is written by and for bureaucrats a lot of times you know it's just paper drill and they say oh well we'll accept all these risk because we just want to make it work and you know see it time and time again and CNA packages really are nothing more than telling adversaries how to get into your system I know and so DSS their answer to taking the CNA packages off they took them off the internet because they said well we don't want our adversaries to know about them then why the hell can I request them through a FOIA request an adversary who really wants to get into an agency into a company is not going to do a FOIA request well let's just ignore it you got one yeah actually I just wanted to say I can't believe you just dropped the bomb that is a feel the pain oh my gosh that's amazing talk I just want to say I couldn't help but get the feeling here are you trying to ask DEF CON to do something for you is there a hidden message or something you know I was told there might be some guys from the agency here so I mean if they are in the audience it's not like I said it's not a bash on you but you got to do something about it I mean it's shit thanks everybody for coming


I'm O'Brien, welcome folks. This is Tom O'Brien of TFNN where you have five days a week. We go seven hours a day. We go 24 hours a day in the internet at TFNN.com. Always remember, folks, whatever you think about, you bring about whatever you focus on, Garose. Hope everyone's having a great day, safe day. Let's make it a great night, folks. Don't take anything personally. Your truth is personal to you. Your own opinions and point of view reflect your own agreements and are personal to you. It's no one's truth but yours. That's a tricky card, man. Mug it wise. Let's take a look at it out here. We have the Dow Industries down 193. Nasdaq off 62. S&P's down 26. Gold. Gold contract up $15.60 straight into 23.66. We have Silver up 35 cents. $28.16 announced. Light Street crew down about 13. $85.30 a barrel and notes and bonds. Ten-year note, up 12 ticks, trading $109.20, the 30-year up $27 at $117.31 and $Kingdala. $Kingdala trading up 49 ticks at $104.189, Euro 108, yen 151, British Pound, $126 to $1 U.S. dollar. iPhone numbers<PHONE_NUMBER>, give us a call, folks. Want to know what's going on in y'all world, in the world of the S&Ps. Let's take a look at them. What do you have? Well, bottom line, you're pulling back yet. So yesterday, folks, what we did, we tried to go higher yesterday with light volume. Today we go down with light volume. Okay, that's telling, that's saying that, guess what, this thing's not over yet, meaning it's going to try to go back up tomorrow on the CPI. So we're down 270. You're doing 41 million shares, 41 million is coming into last Thursday, which was the downdraft of 96 million. We go into the, actually, let's go into the futures. And just see, Louis, we think we're going to lay out the rest of this session here. Okay, so I'm going to pull up the S&P futures. Well, well, well, you have a high volume low, folks, so we're going to lows. Yeah, that number there is 5204. No, 5209. Oh, that is, well, 5208 is the low. So what we did here is this. You came down, and then the last time that we had big volume here, watch this, because we're testing this area right now. This is kind of cool, actually. See right there? That's the last time we had any volume. That was 106 million shares, contracts. What we just did there, 21, 106,000 versus 21,000. Still wants lower price. It's going to be interesting. Let me go to the NQs. So right now, this thing, NQs, looks to me like it does want to go to the lows. And it's the third bar before the low, actually. That was the last time we had big volume, but they're too close. It's just like when you do an ABC structure up or down. If you have a bar that is right before it in order to stay the most conservative, that's what you should take. In this particular case, it's going to be the same thing with the NQs. So the NQs are down 75 right now. The last time we had any volume is right where they're trading right now. But because of that, this thing's going to go after it's low. So the NQs once got to 18,160, that's going to be quite a move, man. That's what we get because we're at 18,218 right now. Let's go over to the dollar and take a look at the dollar, because the dollar was giving the market breathe-in room earlier and then it rejected lower price. So the dollar, we went to 103,880. You're at 104,195 right now. And the dollar rejected lower price again. So the last bounce was anemic on the dollar. We'll see what this one brings. And then we go over to the gold contract. Gold just keeps going, man. It's on the C to D of an ABC structure up and it's been a straight line move and continues to be a straight line move. We got to 2884 today. You're $20 below that right now, 2364. See this volume, 264,000 contracts. That is just monster contract volume, monster. So this thing wants to keep going higher. Let me just see this for a second. Yeah, you can see this volume after volume after volume, man. So the bulls and bears are fighting. There's no doubt about that because when you take a look at it intraday, you can see it quite clearly. And then we had silver on the move and some of these silver stocks are really taken off. So we take a look at the silver market, the contract first. You hit 2840 out here. Let's see. S1, because it's 30 bucks that this thing is on its way to silver. You know, the silver bulls look to me like they're going to be very happy because what has happened here, let me put this on, is that silver has picked up on volume as gold did. That being said, what does happen with silver is that we're coming to the top of the consolidation. The top of the consolidation is 30 bucks. And you break the 30. It looks to me like we're going to break it. This is saying that silver's going to go to $35,000, man. You get silver $35,000. You're going to have, my God, some of these companies are going to make so much money. It's unbelievable because what does end up happening is that like pan-American silver is finally on the run. Took a long enough, but it's on the run.   Let me just look at this for a second. I'm curious as to. So the next time they come out with numbers is May 10th. What I believe we're going to start seeing. Okay. So they're looking to do 2.6 billion. That's double the amount they did five years ago. How's that? And bring 37 cents to the bottom line. And they're still, they're still growing. That's pretty, they got some growth here for a mining company. You don't see mining companies on a three year basis. They're growing by almost 8% a year. Stay right there folks. Come right back with our man, Mr. Basil Chappan.