How Cinegon and SAPI can help you move towards a more sustainable future for packaging. As a brand manager in the chocolate industry, Ben is facing challenges on all sides. From customers and competitors to government regulations, his own company's ecological footprint and retailers selling their products, the pressure concerning sustainability is increasing. But Ben has forged a plan. Paper wrapping, combining a renewable resource with the best recycling stream available. The engineering manager Charlotte knows that the transition will be challenging. How can they handle the delicacy of paper while guaranteeing smooth production as well as sufficient product protection? Luckily, Charlotte has just heard that the market leaders Cinegon and SAPI can provide a solution. Time to take the plunge. Experts help her review the current best-by-date and define the needed product protection based on product properties and logistical demands. Together, they find the best paper packaging solution to work with. That's where Cinegon takes over and invites Charlotte for a test run at their test center in Switzerland. A great success. Back at Charlotte's company, they are getting the machines ready. Having a Cinegon, formerly Bosch packaging, bar line system already in use, all they need is the new upgrade kit, paper on form. With specific forming shoulders and ceiling tools, this retrofit kit enables the transition from conventional to paper and Charlotte can keep producing without compromising speed for cold ceiling. Out in the shops, the new flow wrap packaging stands out on the shelf with a great look, beautiful colorful printing and an excellent touch and feel. And the story doesn't end here. After serving its purpose, the paper still has other lives ahead. Do you too want to help your company get on a more sustainable path? Then get in touch with us.


There's a difference between a criticism and the complaint. One carries a judgment and one carries a request. When we are criticising, we are judging. But when we are complaining, we're really appealing for something better to happen. When we get these two things mixed up and we do all the time, it creates a necessary conflict and division between us. And often a complaint is heard and felt as a criticism because we didn't say it well. And I've been guilty of that and I guess you have too. We had an unhelpful energy attached to our complaint. We used the wrong words, didn't phrase it well, picked the wrong time. So it was felt as a criticism and so the request for something better to happen wasn't heard and didn't take place. I'm appealing to us in this video to have a greater emotional, relational radar and intelligence for the difference between these two things, criticism and complaint.


So today I'm going to discuss analytics in Postgres and I'm going to talk about it from the standpoint of not just using Postgres as a data store but using Postgres as part of your analytics infrastructure and really your engine for running predictive models. So the traditional way that analytics has been done is you have separation of the model and the data. You use the data to build the model but the models coded in Python are what have you or frequently it's coded in whatever application language is being used for the application in particular. And the execution model for predictive analytics data for the models themselves is basically the cycle of okay pull some data together from the database or whatever data sources you're using. Perform some computation with that data and then make a decision and then possibly repeat the process as you step through a decision tree. If you notice half of this formula that I just described gathered data that's a data problem and frequently the data half is the majority of the cost that you invoke when you're running your models in production. So the main idea here is instead of bringing the data to the model why don't we bring the model to the data because bandwidth just network network infrastructure is one of the highest costs in any data center and it's one of your big performance bottlenecks. So instead of shuffling a whole bunch of data to your model run the model where the data lives. So I'm going to do this in the form of a case study it's it's don't worry it's not going to be some deep dive advertisement for my client but I did I did want to describe this from a standpoint of a system that I've actually built that is a real-world application and not something that's just theoretical. So just to give you some background on my client they're a financial company they're a near-prime lender and really the key thing that you need to understand about that is that means the FICO score that all of us worry about ooh do I have a 720 well that FICO score for near-prime so people in like the low 600 five the high 500 range they're not quite as concerned about what their FICO score is and that means that the FICO score is not a good predictor of are they going to repay their loan. So for somebody that's doing lending in that environment they basically have to build their own version of a FICO score they have to look at all the information about people applying for credit and make a decision on the fly of is this person a good credit risk or not. So having the ability to track and control their analytic process is absolutely critical especially because with online lending where there's no collateral involved and they're not a car lender or anything like that these are strictly their installment loans but you can think of it kind of like a credit card. So if somebody comes in and you agree to give them a loan and in fact they were somebody who is intentionally trying to defraud you you can have millions of dollars waltz right out the door that you're never going to see again and obviously that's something that a company cannot tolerate for long. The other aspect of this is as a lender they operate in a regulated industry so they have to be able to demonstrate things to regulators such as we are not making lending decisions based on gender and things of that nature and again that speaks to the repeatability and the trustworthiness of their analytics becomes very important so that they can show regulators what they want to see and not come on their regulators well what do you what do you mean you can't show me what how you made this decision what do you mean you don't know what model you use. So the addition to that you know they need to be able to protect sensitive data obviously they also need to be able to rapidly evolve the analytics models because as somebody comes up with a new way to try and defraud them they need to respond to that very very rapidly and finally performance is a consideration. A lot of the leads that they have coming in are done on a affiliate purchase program so a lead will come in and they'll have a number of seconds to be able to decide do I spend the money to purchase this lead or not and those lead purchase costs can be very expensive so by the time you get around to purchasing the lead you want to have a really really good idea is to whether this is a good potential customer or not so they have to do real-time analytics as leads are coming in. So let's talk a little bit about traditional the traditional workflow in more detail or as I like to call it death by a thousand paper cuts except that now you know today is the era of big data so it's not just a thousand paper cuts. So traditionally the way that that analytics models are developed is you pull a bunch of data from a source system into some kind of analysis environment whether it's Python or SAS would have you everybody's got their own flavor or maybe you do some amount of searching for patterns on the source data system itself you find the correlations you build the model you train the model with a subset of your captured data and then you validate the models performance on the remainder of the data so if you have a data set of say a hundred thousand loans and you know ten percent of them defaulted you want to you want to be able to identify what is it about the ten percent the default so you would run a sample set through the model of say you know 20 percent of the defaulted loans and 20 percent of the non-defaulted loans you'd use that for training and then you'd run the model against the remainder of the data set to see hey does the model do a better job of predicting or not that's the analytics side then you get to the development side or as I call it throw it over the wall so the model's thrown over to the developers the developers complain about the code possibly mock the language that the analysts are using well why aren't you using rails because that's the cool language and then the model gets completely recoded now you have to test the new code or you could just throw it out in production and cross your fingers because that's always a good plan and then finally after this entire process has has gone through the analytics folks get to see the real results of the model running in production against real data and one of the issues here is this process can take weeks to months depending on how efficient the organization is as you can see there's some problems with this first of all data wrangling is not the same as analytics there's two separate problems that are involved here the ability to track models with this system is extremely limited basically you're only track traceability on the models themselves is whatever is getting checked into the version control system for the application itself the analytics department may or may not be tracking their stuff you know and what they've done for the models tracking the execution the actual performance of the models in in real life use is ad hoc it's it's pretty much whatever gets implemented for each individual model and a lot of times there is no cohesive system in the application there's a lead model and then there's a credit limit model and then there's a loan amount model and maybe there's some models for who knows what else and and they're all separate and and they don't know about each other there's no commonality to the code and finally this us versus them leads to bad morale and massive cycle times now to be fair I'm kind of describing some of the worst of what the situation gets into hopefully none of you are stuck in that kind of environment but even if it's not that bad this this kind of disconnect that you run into between the analytical team and the development team with this well we'll hand you the code and then you have to get it out in production it's it's not a useful paradigm and it adds a lot of inefficiency so the solution that we created for Kairios is based on recognizing that there's there's basically four separate challenges in predictive analytics there's gathering the data to construct the model there's constructing the model itself there's the model execution phase and there's the data that you need to execute the model but the commonality here is you've got two things that are data and two things that are code so how we approach this as we said let's look at these two problems on kind of on their own but from a cohesive standpoint we'll recognize the data problems as data problems and recognize the code problems as code problems we use the right tools in each case but we'll do this cohesively instead of ooh the data is here and the code's over there so data you want to make it as easily accessible as practical because inaccessible data makes the job of your analytics team trying to build the model more difficult and ultimately the data that they choose to use that they say this has significance you know this these data points here have statistical significance those data points need to be readily available for the production model to make use of if they're scattered all over the place buried in XML documents or what have you that impacts the quality and the performance of your production models so instead of taking a point by point approach of let's expose this data point oh and now they want this one so we'll expose this one and now they want this one so we'll expose this one we wanted to try and make as much of the data available as possible as practical so there's two main data sources that Kyrios deals with one is third-party reports and generally speaking those come in the form of XML you go you make a call to a web service and here's your gigantic XML document that comes back the other is the the metrics on the lending portfolio itself so the problem with the XML is that hand-coded XPath like that stinks have any of you ever dealt with that yeah all these sheepish grins going yeah this is a horrible time in my life I felt dirty you know hand-coding all that stuff and I've seen this from working with other analytics teams where they've got just these expressions that stretch this long and then they need this for a hundred different data points so huge expression huge expression huge expression huge expression and trying to maintain that is a is a complete nightmare let alone that the pain of actually creating that thing so we said why JSON's the cool new thing yes we'll follow the herd into the JSON but it doesn't take much thinking to realize well the JSON operators in Postgres make it nicer it's not as bad as the XML but who wants to hand-code all that stuff so really what we did right from the get-go as we said look everybody knows sequel sequel makes it easy yes these are structured documents or I'm sorry they're no sequel unstructured JSON but this is a document that it doesn't change you know so we can easily take the JSON and turn that into a set of views that represent the hierarchy that's inherent in the JSON document so if you're if your top level has keys a b and c and each of those is an object you have view a that has its fields and view b that has its fields and view c that has its fields now when you're dealing with it from a sequel standpoint all you need to know is how to join things together and everybody knows that so that results in data being accessible via sequel very easy to deal with the portfolio metrics this is this is a different problem so the third-party XML you've got this well-defined structure and the third party vendors say hey you pay us five bucks here's the day that that we're gonna give you they've already identified for you the kind of data that you probably want to look at the portfolio metrics it's up to you now to try and decide what are the things in our collection of data that we've captured from our clients that could be useful predictors of different credit risk behaviors some of these are going to be kind of obvious like people that just they get the loan and then we never hear from them well obviously that's bad so people that never pay okay that's one metric that we want do people pay on time that's maybe important maybe not and and this is one of the places where the FICO score is not so handy because the FICO score is based on predicting are you going to be late on a payment most lenders don't really care that much if you're late they care if you don't pay a lot of them and they won't say this a lot of and maybe I shouldn't say this since I've mentioned who my client is and I'm not saying my client does this but you know discover card and certain other folks you miss your payment they get to charge you the $35 late fee do you think they're crying in there you know crying in there in there in their glass over that no as long as you pay back what you owe them they're going to be happy so what this boils down to is you need to identify what metrics are going to be useful some of that identification can be done strictly as part of your analytics modeling process I mean that that is part of predictive analytics is you look at a whole pile of data and you let the machine do the hard work of identifying hey I found this pattern over here it seems to be important some of this is just going to be driven by well look we need to measure portfolio performance so part of that for a lender is being able to accumulate a loan loss reserve so as loans are going through their life cycle you need to be able to predict okay these are the loans that are okay and these are the ones that we think there's a risk that we're not going to see the rest of the money so we need to start planning financially that this chunk of money is not going to be coming back once you identify these useful metrics you want to code them cohesively and this is this is an area where a lot of companies I think do a really really not so great job they look at things like tracking portfolio performance as oh hey that's the responsibility of the bi team or it's the responsibility of the finance team and then maybe finance has to go talk to the bi team and then the bi team might kind of try and complain to the coders that hey it's hard to get at this data and somewhere along that chain you know things just break down what you really want to do is come at this from a holistic approach of metrics that are going to be useful for predicting things are probably metrics that you should be tracking and reporting on anyway and if you now have two uses you know your business reporting and your predictive analytics that are saying hey we need the same data point well let's just say this data point is important and we should worry about it all the way through the data life cycle so starting with the customer interaction on the website how do we collect information in such a way that we can do analysis on it and that starts with everything from how are you collecting customer addresses are you sanitizing that data do you keep the original data that was unsanitized because sometimes you can glean information about things like did they use all caps did they not use all caps that may be a predictor I honestly don't know but that that's the kind of things that you need to worry about on the front side and by doing that now when you're collecting data down the road it's not this enormous nightmare to try and measure these things it's not a problem for your bi team and it's not a problem for your analytics team finally the the performance aspect of collecting the portfolio metrics that's something that certainly in the finance industry for portfolio metrics you generally don't need to worry about too much up front and have any of you heard the axiom that the first rule of performance tuning is don't that's what this is about so what you don't want to do is get all worried about oh well if we don't materialize it and do this and that and the other thing then you know it it won't perform well and then you end up doing nothing and now to calculate that metric you just have to run a bunch of horrible sequel by hand which by the way is going to perform horribly anyway so focus more on getting the metrics to find getting them implemented and then where there's a performance problem you can go back and address it at that point once you've built some infrastructure and you know yes this is the metric we want it's a metric we need and by the way we need this report to run in less than 10 minutes and we use it for the lead purchase so we need to be able to pull the data in less than 500 milliseconds the really important key part of all this though is everything I just talked about is is built on this concept of bringing the analytics folks the business intelligence folks and development together to define this stuff and work on it together now that that doesn't mean all of a sudden you can't create a single data point without having 15 people in a conference room but you want to have that cross communication and most importantly you want to have a feeling of shared ownership so that it's it's not this throw it over the wall mentality so the second aspect of this is the code portion this is the stuff I know you're all waiting for so creating and trading models that is frequently a computationally difficult task executing the models is usually not depending on what you're talking there's exceptions and frequently when executing the model is computationally expensive expensive it's not the CPU cost it's not the computation itself it's getting access to the data that the model needs so if it's necessary consider if you do have a model that is very computationally intensive to any of you work with predictive analytics with image processing image recognition one okay all right work up work with GPUs or if it's if it's too expensive to run inside the database you can consider using an external you can use an external storage mechanism obviously you want to avoid duplication of code and traditionally this is this is a big problem because your analytics folks like to use languages like are in python and developers like to use things like rails used to be hot and now it's more node and you've got this dichotomy that doesn't really mix by running the at the the model code inside a postgres you can let the analytics folks create the code in the language that they're comfortable with because postgres supports are and it supports python and supports a bunch of these other sort of procedure languages and now you're not you completely eliminate this step of throw it over the wall and let's rewrite it from scratch so you want to support that you want to make the model development environment as similar to production as practical so what we did with Kyrios is every model has two database functions and that's really all that there is code wise to each model is these two functions one function is responsible for pulling all the data together now because we've got the XML information separated out into a series of views the data gathering functions are really simple it's it's just hey here's a sequel statement pull these fields from this view join to this view pull these other fields join to this view pull these other fields just bring it all together and we bring it all together in a compound data structure of a postgres composite type the execution function that's responsible for running the model can then be coded in any language that you want and the only restriction on that execution function is that it accepts a compound data type and it returns a compound data type the reason for doing that is that allows us to predefine here's the structure for this model for both its input and its output also the framework then has these compound data types that it knows about and it can just store those into a table so every time that the every time that a model executes it just inserts a record into a table that says here were the inputs here's the output two fields that's it you're done the framework doesn't need to know anymore about it now obviously to use the output you will need to understand what the structure of that compound type was here's the here's the credit score portion here's the margin of error indicator here's you know any warning flags that we saw whatever you want to do yes it is a structure so compound types in postgres the syntax is basically create type give it a name parentheses and now you essentially have a table definition it doesn't have the full support you can't say foreign keys and all that stuff but you say here's the field name here's the data type here's the field name here's the data type and in fact postgres under the covers creates one of these types every time that you create a table so if you say select star from PG class and you get those rows back you can actually treat each of those rows as a field and you can store that inside another table or you could treat the entire record set as an array of composite PG class and and if you wanted to actually code that you would just say here's a field and its type is PG underscore class square bracket square bracket and that tells postgres oh it's the type name is PG class and it'll go and look up and say oh PG class that's a composite data type I know what that is and then the square brackets just indicates that it's an array so the the model code that kairos uses is typically a pi a python function so the model itself ends up coded as a single python function if you're familiar with python python has this nice ability where you can define a function from within a function itself so the more complicated models you'll have a top-level function and the top-level function may define it additional functions but at the end of the day the model itself is nothing but pure python and all that that python function does the only constraint on that python function is it accepts a single dictionary which is the equivalent to the postgres compound data type and it returns a dictionary which will then get turned back into a postgres compound data type so the database function the pl python function then is nothing but a simple wrapper around the python function this means that doing the model development the analytics guys their job is easy because once they know what the structure of the dictionary they're getting is they can produce test data they can do whatever they want and as long as they're returning the dictionary that's their interface and they can go off and do their thing and they don't have to get hey mr. developer I need this no let them do their work let the database people do their work now as I said the key one of the key constraints here is that you do have to know what these compound data types look like and this is where you want to get the people working together to again on this data problem of hey here's the data that it looks like we need for this particular model and by by doing that in advance what that allows you to do is you get the collaboration again going so that it's it's not you have analytics guys that aren't necessarily database experts and they're crafting these horrific this is how we'll pull the data no you get them involved early with the database guys oh you need this well hey we'll do this thing over here and it'll make it a lot better but by by having the separation of the code that means that integrating the code into the database becomes very easy and in fact we can do this there's two ways that we can do this so as models are being developed we can create the data structures in the production system and go ahead and put just the data structures in production not the models but just the structures and provide the support for hey this is how you gather the data for the structure the analytics guys can then take that use those data structures and actually we can create the data gathering function they can use that on a Postgres replica to say okay give me the actual data set and they can they can get a data set that could then be pulled directly in Python so they could take a Python console like Jupyter Notebook connect the console to the replica they get the data streaming back in exactly the same format that's going to happen in production and they can do whatever they need to you know analytics wise operating in their tool set in their environment that they're comfortable and productive in and they don't need to back and forth with DBAs or developers hey can you push a new function you know a new version of this function the other option that we have is we can give them just a snapshot and say here's how you create the Python function and you can now you can just go run your model so you create the function you know you create your model you stick it in in a PL Python function and you say hey okay here's the loan portfolio select Python function go and that's how they can do the model verification the third option because of this feature is you can actually put a non-production model into your production environment they have to be careful obviously that you don't start accidentally calling it from the web front end and all of a sudden bad things are happening but you can actually do this to support your development and your model validation efforts to say okay here's the model we think it's in its final form let's load it in production we won't use it yet but once it's in production on our streaming replica select model function from all these loans run basically run the the model against the existing portfolio see what the performance is all the same code base there's no recoding here there's no magic anything it's all the same exact framework working from within the database so tracking the way that this framework works is the actual function source code for the data gathering function and for the model function is captured in the framework so every time that you run a model you have pointers back to the exact source code there is no question there is no well let me see what git looked like on that day oh wait we're doing a deployment at the same time so I'm not sure which one we use now every single execution you know exactly the code that was put in place the because of the composite types the input and the output those get recorded with the individual model execution as well so every time you run the model you know everything there's no questions and by the way the table is not updateable and it's not deleteable so once the model executes you have a permanent indelible record in Postgres well super user could really go in and mess with it but short of that you know this is a record that's not it's not going away it's not going anywhere and you can show that to regulators and give them the confidence that they know now that hey you know what you're doing this isn't you know this isn't guesswork when when you provide data to the regulator they know that it's the correct data so all that stuff that I have been talking about you know that sounds kind of complicated and hairy what does it look like in practice that oh but that's client code please don't share that unfortunately I I would love to open source this I am going to be talking to my client about potentially doing this but obviously this is a competitive advantage to them so I don't know that I will be able to open source the framework itself but I have given you the valuable insight here today of this is the idea you know this is how you can build a system that executes this stuff from within Postgres itself and if the code simplicity aspect so that that seriously is all that you need to do to run the model the for those of you that are not familiar this the kind of funny thing with null and then casting it to a data type that data type is a table that stores the actual model execution so every every major version of the model that makes major changes to the data structures it does get a new table created but we're using Postgres table inheritance so that when you want to see just show me every model run we've ever done that's easy you just query model dot model run I think there's a table no it's model dot model execution I think is the table name but you query that and it's going to show you every single model execution that's ever ever happened and then if you want the details for a particular kind of model well I want the the lead purchase model query the lead purchase you know call the table lead purchase or lead purchase model you query that table it's got the results so by casting null to the data type of the table that tells the function oh hey we're running this model and the function goes and finds out here's the details about the model itself and then we just have a single surrogate key called identity ID we use that to identify individual people so that tells the model okay well the identity ID gets passed to the data collection function so it knows when it's pulling data out of the database yes hey it's Bob so we're running the model for Bob so go get Bob's data and then hand Bob's data to the model function that we were told about earlier and the model function provides its result and then the model run will ultimately provide the the actual record that's that's logged for the model execution so that's what the code looks like of course the other question especially for real-time analytics is performance so using just a very simple really a do-nothing data gather function and a do-nothing model execution function these are just test functions that we use for unit testing the framework you can see that the two functions take less than a millisecond to run the important number is that other which indicates that the overhead for executing the framework itself is just under seven milliseconds so this idea that you can't run analytics in postgres I don't buy it so much and unfortunately I didn't think ahead of time to run these tests I could have run these tests for some of the production models but in reality using the real models we're still seeing sub-second performance so this is something that will easily scale out to quite significant workloads in production without you know causing a performance problem so key takeaways use your data tools for the data use your code tools for the code and this doesn't mean keep the team separated you want the teams collaborating together but you want them to be productive so if your model guys want Python let them use Python let them use it in its native environment or are and let the data folks use their data tools don't ignore this idea of using code Postgres as an actual code execution platform a more traditional way to do this would be oh yeah we hate all these things you know all most of those bad things about analytics you can say well alright we're gonna build analytics as a service and we'll do service oriented architecture and you'll make a call to the service and the service will know and it'll do all this stuff and the problem with that is the service is still going to be shuffling data in and out of your database so it's better but you still have this data movement that becomes a significant performance problem so use Postgres as a platform Postgres is a service or Postgres providing a service and as I said get the teams working together and we've got about 10 minutes for questions yes yes so the question was because we're tracking everything that's involved with the model execution that's the input data the output data and the code itself has that created a data size problem not yet because they're not in production however from a code standpoint we don't store a copy of the code every single time the code gets normalized so so when we go to execute the function we pull the source code for the function out of the Postgres catalogs we create a hash we compute the hash of that we use the hash to look up in a table to say hey do we know about this and if we know about it we just say oh yes we're good here's the hash and actually just to be paranoid because I think the hash is actually that the hash is the Postgres four byte integer hash function so there will be collisions so we don't use the hash as the key we just use the hash as an easy fast way to do the lookup so we grab the hash once we use the hash to find the function we just do a comparison of the actual code that's pretty fast because it's just a mem compare so that you know C level standpoint it's just running through two arrays at the assembly level and then finally we say okay here's the surrogate key and we just record that with the execution when it comes to the data itself the nice thing about the Postgres compound data type is that it uses the same mechanisms that are used for defining a table so if you want to know what the internal specifics of a compound data type are you run the exact same queries that you would run as if you wanted to say hey what are the contents of this table so that information is stored one time and then the storage for the compound type itself is just it's a Postgres internal format that's essentially just a stream of bytes it's the same as if you had created a table and said oh I want to store this and this and this and this in the table and then essentially however that table would end up on disk that's what gets stored in the compound data type so the only disadvantage to this from a data size standpoint is we are making a copy of all the input data in our case at least right now all of that data already exists in Postgres the data gather function which by the way the data gather function because we have PL Python and what have you we could go call external services from the database we're not doing that but we could but we do have all that data already in Postgres we are now making a second copy for every model execution and that was an intentional design decision because going back to the regulated industry thing yes we could have set things up so the data is you know we pulled it from here and here and here and here and the tables are immutable and now we just said you know what it's not worth it in this case the bytes are cheap enough we just make a copy we store the copy and the important thing from model executions is in your production environment when you're running the models that table is basically insert only you're you're not if you were to refer back to the table you'd be referring back to an execution that happened extremely recently so it's all going to be in cash so we are not we could it really we were early enough now that we haven't really we don't really have data sets to do the model training on so right now the models are basically well okay this we think this is a good starting point and then we'll start collecting data if we were to do that we have two choices for for running running the model as if it was up in production we could either put the model in production not call it from production code but then once it's in production we could run it we could run the models by hand on the production database not a good idea the second option that we have is if we create a snapshot or if we have a logical replication slave we could then run those model functions and actually go create those records that I think is going to be a matter of what's convenient to start off with when we first start doing model training we may well do that because it may just be the easiest thing to do oh hey look it's it's here's the code in production and just there's your overhead yes yes and and everybody please understand this was developed with a specific use case for a specific client in mind we're obviously very focused on the regulated industry portion of this as well so should you just blindly go take this model and implement it where you work well unless you work in another financial company maybe not so yes if you are talking about this is a huge chunk of data no you're not gonna want to do that my experience in the financial industry is that typically the the model input data is only going to be maybe a third of the database in its raw form but then in its compacted you know in its composite key form I don't think that it's it's going to be terribly significant right right so the the question is if your analytics folks are using something that there is not a procedural language for could you still use this this concept the answer is yes because you mentioned shell actually there is a pl shell so you could fire up a shell from in the model execution the the one downside to that now I wouldn't necessarily recommend shell if you want to do that what I would suggest is that you treat it as oh that portion of the model is a standalone service so you use one of the procedure languages like PL Python or PL Pearl would be two good choices or or maybe v8 and you use those to call out to that service and you pass the data to the service and then you get the results back and then you can you know for the rest of that you can you can do what I describe so I'm being told we are out of time I think is next the lightning talk oh yes next the sponsor talks so I will stick around here for another four minutes but then we all need to be next door so thank you very much


Either a slow drift to the downside by a lot of names that have pretty big runs. Welcome to Access a Trader, the number one community for those who are committed to taking control of their trading in order to achieve success, profitability, and longevity. Thank you for joining us. Here's Dan Shapiro to help you find your edge, master your process, and own your future. Hey guys, good evening everybody. Welcome to another edition of the Access a Trader dot com. Let me wrap up show. Hope everybody's doing well. Okay. Hope everybody had a great weekend. Hope everybody had a pretty decent trading day. So we started talking about on the weekend update that, again, you want to give the bulls the benefit of the doubt. There was a lot of good setups coming into today. And you'll see, especially off the watch list, you'll see the numbers that we talked about pretty much throughout the weekend, the game stop to the downside, AMAT, NVIDIA, Tesla obviously had just a ridiculous run. And the one thing that I kept on kind of reiterating over and over again is you want to make sure you start looking and start watching for some signs of the market getting tired.  August got to a pretty aggressive start. The Dow was up. The Nasdaq was up. Everything was up. And slowly but surely the market started losing a little bit of steam. Now, again, that's not a case for this is the market top. The market's going to zero short everything in sight. This is a case that we talked about even when we went to all the way back to, you know, a couple of weeks back. When you saw the market started getting tired after, you know, five, six straight sessions and started losing a little bit of steam. Again, not because the market is short, not because this is the top of the market. It's just because the market, when it goes up too aggressively, too fast, it starts to kind of get to the point that it needs to get its feedback under. We kind of need to breathe. You kind of need a little bit of rest. And when you look at a lot of names, and we started talking about this on the weekend update, we started looking for clues, right? What happens after a stock takes out of a range? What does it do the rest of the day? What happens when a stock puts in an opening range highs? Let's see what it does for the rest of the day. What happens when a stock takes out all time highs? Let's see what it does for the rest of the day. And these are all signs. And this is all data that we continue to talk about and continue to kind of reiterate the point. It's not just price action that matters. It's price action that matters off of important channels. So we kind of got our answers to a lot of those questions. So for example, Tesla broke out three days ago. The breakout is not at 700. The range is at 700, but it broke out at 656. And then again, over that 680 level, this is the last high here. And today, again, took out that 700 level and exploded to the 727 level. The question was, what was going to happen after? And if you look what happened after, it's literally a six hour straight of lower highs and lower lows. Like literally six candles in a row, lower highs, lower lows. Then you get into a name like Moderna. Again, not that this is the end all be all, but the point is, again, we're looking for more clues. Again, if you look at with the CDC reported, it said it's seven day average of daily US COVID cases surpassed the peak of last summer. It's just the headline. Okay, it's not meant to be a conversation that we're about to have. It's just the headline. So the question is, well, what was it going to do after it took out its 52 week high or all time high? So it took out its high 362, went to 365 and a change, and then had a really, really aggressive reversal. Again, things we're looking for. If you guys remember NVIDIA, right? I've been talking about NVIDIA now for, you know, for about a week and a half. And I said, look, that 99 areas gotten rejected three times, right? It finally got above the 99 area, went up about 60 cents or so, and then went below. Again, these are all kind of telltale signs that stocks are getting pretty tired. Again, doesn't mean the market's going to collapse. It's not going to go to zero, but it's very, very tough to wake up for tomorrow's session. And after you do your research tonight to turn around and say, look, I'm 100% bullish. The markets looks great. I think we're going to rally. Yeah, it's great. You can say whatever you want, but the point is when stocks get tired, just like you, right? Run a race, you know, take a deep breath. You need to kind of reset and then you run another race, right? That's kind of the whole point of trading, dissecting what's in front of you. And if you start gathering information about, well, look, stocks are strong, but they're not following through. Stock looks great, but they're not following through. They're getting good price channels and aggression move out of the channels. But guess what? They're not following through. If it happens once, okay, I get it. It happens twice. That is a theme, right? So going into tomorrow's session, you know, I'm not really in love with anything, right? I can't really turn around and say, I love this. I love that. Yeah. I mean, look, is there some longs that I like? Yeah, there's a couple of longs I like. Is this some shorts? I'm like, yeah. Some of these meme stocks that, and we started talking about GameStop over the weekend update, our nice little, nice shortening this morning and it stopped perfectly at support. We'll get to the pivots in a second. But I think more chances than not for tomorrow's session, you're probably going to be faced with a slow drift, right? Either a slow drift to the downside by a lot of names that had pretty big runs. And again, remember, they're not shorts. They're just stocks that are drifting lower. Any news, any rise into rising 60 minutes support that you're not aware of and you're going to get run over right back to the upside. So you can't go into tomorrow's session and say, well, Dan says the market's short. No, specific names that are going to possibly break down ranges are short setups. But the overall market, I wouldn't call it a short. I think it just needs a break. And I think if you're going into tomorrow's session, guns blazing, bull bias on one side of the market, I think you're going to be very, very disappointed. And just because kind of the three examples that I gave today just a few minutes ago are probably going to be a pretty good landscape of the overall market. What's going to happen? Just think about this logically. If a stock can't confirm, excuse me, if a stock can't not resume an uptrend after it confirms a channel, a macro channel, well, that's the first sign of the market being tired. Is it an isolated situation today that we might resume tomorrow? OK, we'll see. Right? Let the market tell us then that's the case. If we start seeing especially early morning buyer strike, that's the best way of saying if we see a buyer strike on any spike of earnings, any spike of futures tomorrow morning, that's, I think, we're going to start looking a little bit more to the downside. Obviously, still be buy bias. But just remember what we talked about in the weekend update, the Nikkei, although it rebound, I didn't see what it did towards the end of the day. But although it rebounded 1.8% throughout the session, or at least in our morning session, keep this in mind. They've been gradually going lower and lower and lower. So I think you have to be mature tomorrow. OK, I don't think you need to be, as they say, bolster the wall, pedal to the metal. I think it's more of a concentration day. I think it's more kind of gathering data. See what's weak. See what's strong. You don't need to trade like a lunatic every single day. You don't need to trade full size every day. Just see where the market rotates. See where, if there is a theme, that's going to be possibly measured out for tomorrow's session. And if our favorite names kind of wake up, like in the videos of the world, like the apples of the world, and start confirming challenges, then maybe we can get a follow through on some big aggressive beta names. But until that happens, I would not be surprised if 24 hours from now, we're having the conversation that says, well, the market played out exactly how we thought. There was a very, very tight channel, constriction of channels, and there wasn't really a lot of room. I wouldn't be surprised. And you know what? That's healthy, OK? The market needs to do that from time to time. Again, the market doesn't care that you're waking up with last night's erection that wants to trade everything in sight. The market's going to do what it has to do. And if it's tired and it's giving you clues that it's tired, well, guess what? It's tired. If it looks like a duck and walks like a duck, I promise it's not a rhinoceros. So it's very, very important to kind of understand what we're looking at, OK, for tomorrow's session. Understand that we might not have one of these explosion of channels, both of the up to the downside. And I think tomorrow, just, again, learn to be prudent, learn to sometimes take the little bit more defensive approach. And again, if there's good setups there that confirm, yeah, of course we're going to trade them. But if not, again, always remember there is another day. So let's talk about today's pivot. Not a lot, right? Not a lot. You have to kind of focus out of the box. A lot of the moves came in on one individual channels, and then they kind of faded for the rest of the day. But again, you don't need many trades. So let's talk about it. So obviously, this was last night's focus list. You had Tesla, that's 700 area. Everybody saw what it did by now. But remember, guys, again, the word breakout. Breakout is a very, very subjective work. For me, Tesla broke out at 655. This was three days ago. Today it confirmed that $700 channel. I took NVIDIA, went up 50 cents, went down 50 cents, went up 50 cents. I finally took a small loss on this thing. It was about 60 cents or so. I'm going to revisit it tomorrow. Again, the one cool thing about NVIDIA, and we were seeing this now the last two, three days, they are coming for the 205 weeklies with a lot of repetition. It's not just one or two bets. The aggressive nature of the bets is not exactly where I need it or like it to be. But I'm definitely, definitely continue to watch that channel. NEO, you know, not a nice move today. NEO was a play literally off of Tesla 4530 needs to build. Here was NEO off the daily list, right? NEO took out 4530. You went to about 47. Again, short term, $50 call buyers coming in. Apple, I still like. mRNA, not a big move at all. 6263 needs to build. Went to like 65 and change. AMAT, I forgot to look at. For some reason, I forgot to look at today. Guys, I totally apologize. I forgot to look at if you took the trade. Congratulations. I completely missed it. 4150, 142 needs to build. Here was AMAT. AMAT really had a nice move. I can't believe I forgot to, I literally forgot the stock. AMAT took out the 41 and a half, took out the 43 and traded all the way up to 45 and a quarter. For some reason, I just forgot to look at it again. FISV did nothing. GameStop was actually pretty good. I actually shorted at 158. It was the opening range low. Covered it right near the bottom of the range in the 55s. See, I think the stock goes lower, right? I think the stock goes lower here. The only reason it stopped, I actually shorted off this range here and it stopped right at the 55 level. So I shorted off the 58. If it could get below this 55 area, guys, there's a lot of room to the downside. They were coming for short term expiration in the 140, 130 weekly contract. So let's keep an eye on this thing for tomorrow. At least there's definitely a value trade set up there. And this thing has been kind of like grinding lower, grinding lower, but if this thing loses the support, I think there's a lot of room down. So let's keep an eye on that. And again, the national pivot is 99. Again, only went up like 50, 60 cents. Apple never got to the 47 level. Moderna is spiking up a little bit, right? Spiking up a little bit off the 63. Like $3, nothing there. Amazon for experienced traders watched the green to red. Only went down like $7, $8. That's actually nothing for Amazon. AMD was actually a pretty good watch. Watching for a blow off top in the morning session. Incredible three day run for experienced traders only after the first move up. Let's see if it goes green to red. Note, this is not a pivot. This is just exhaustion. And it does a big move. I mean, really, really big move on AMD. You could see the, hold on a second. You could see what happened here at the open, right? You see here what happened at the open. And here's the market, you know, the market opened up and just got destroyed.  It actually went back to red to green and spiked up all the way up to 110, just some monster, monster stock. But again, if you took that green to red, wash out this morning was really good. FISV never got to 117. BTBT, nice moving. For all you guys who trade these smaller cap names, 99010 needs to build. Here was BT, what is it? BTBT, right. BTBT, so here's the whole 99010 area. Went to 11 and a quarter. Nice move there for all you guys who love these smaller cap names. Nice move there. You know, great reversal on AMD. We're going to take on the way up on BTBT. Game stop, right. So 158 if it builds below it can flush. Again, take on the way down when all the way down to 55. I covered in the 55 as my last piece. Nice move there, nothing wrong with that. Again, here comes the weekly 205, right? And it got above the 199. That was kind of my case. You know, things are getting tired, right? Like really, really tired. And this thing can just kind of just stalled out here. So nothing there, but I'm going to still watch this thing for tomorrow. Tesla for a bounce never got down to the 706, 707 level. And I think that's it. So, look, going into tomorrow, do I love anything? I don't love anything, right? I think maybe tomorrow there's early weakness on Tesla. Maybe they could trap off the bottom range, maybe give a trade for a red to green move. See, I think Tesla, just because it had such a huge run, just over the last three days, it needs rest. It really does. It needs to, you know, consolidate, maybe even back test the bottom range here. But it needs, it's tired. It really is tired. And the moral of the story is for this thing to go, and again, there was some really, really aggressive call buying for the August, for the September, 750, 800 calls. So the money flow is there. I just think it needs a kind of a couple of days to kind of relax, to kind of get its win together. And just like mirroring the market, I think the market just health wise needs to kind of, you know, digest a little bit to see where the next directional buying is. So guys, have a great night, everybody. God bless, and I'll see you all tomorrow. Take care.


All right, I think it's about four o'clock or so. So I'm going to start talking. This is identity management. If this is not what you were expecting, yeah, I'm not sure why all of you are here, but why not? So can we talk a little bit about identity management? Yeah, this should be fun. I was going to do the whole Johnny Long intro, but I'm not as cool as he is. So who I am is not really relevant. Yeah, you guys. So we're talking about identity management. Hopefully a few of you guys know what identity management is. Let's go for a raise of hands identity management suite. How many of you guys actually use some sort of identity management in your organization suite? And how many of you guys use Novel Identity Manager? Wow. All right, so this is going to be pretty centric around Novel Identity Manager because that is what I used to do a lot of consulting on. And I know it pretty well. Knew some of the ins and outs of it and found some ugly things while I was using it. And I'm going to talk about those. So what this talk is not, is there anyone from Novel here? Sweet. So if Novel happens to be downloading this talk later, what's that? Is there any press here? Is that what you mean? I'm sure there's press here, and so I'm, oh well. So if Novel happens to take a look at this talk anytime in the future, this is not a fix-it list. Like I'm going to talk about some specific things that I see as issues, you guys can decide. Maybe there are issues, maybe they're not. But this is not like a fix-it list for Novel. I'm sure there are other things that I have not found that Novel should address. And so I really hope that Novel takes just a serious look at their Identity Management scheme, their architecture, kind of what they're doing in that space. Since it is built as a security tool very often, I would prefer that they go and look at it as a security tool and kind of look at some of the security implications of that, do some documentation around how to not set it up insecurely. Just got my picture taken, so they now know where to find me. I do have very high respect for the people at Novel. Crispin, go to his talk tomorrow night. He's very smart. He's a great guy. And Ed Reid, who was formerly there, great guy. So Novel has some great security people. I just think that they have pushed out some of Identity Management pieces a little faster than their security could keep up. So what is this talk? Basically an attempt to convince you that Identity Management is basically just another asset on your network. It needs to be protected just like any other asset. Most of you should be aware of that for the not quite white hat side of the crowd. Hopefully you'll see some things here to take an interest in. And yeah, I'd like for people to go out and start doing research in how to evaluate the security of Identity Management systems as they are becoming a lot more prevalent in corporate environments, as we saw by the show of hands. All this information is pulled directly from public resources straight from websites. So nothing I'm saying here is acquired in any way that's not public. So this is all public knowledge. OK, so Identity Management, most of you seemed like you knew what it was. So basically what is it? Or actually, let's start with what's being said about it. This came out, I think, yesterday or the day before. ISC sent this little quote around. It's a front burner issue for lots of organizations. Basically it's becoming a big security issue. People in the security space are starting to look at Identity Management as a security tool, which is great. It's very useful as a tool. But like any tool on your network, it has some liabilities. And so that's basically what I'm trying to raise awareness about. Theory of IDM, basically you have two systems, generic systems, basically that hold user identities and are connected in some way by some meta-system. And that's your Identity Management system. The Identity Management system continuously manages your identity throughout the whole lifecycle of that identity, provisioning, updating your authentication tokens, granting you authorization within those applications or systems. And it should all be done with the audit and proof repeatable. That's basically what IDM is. Specific products should all be familiar with them. Who's right in Identity Management? This is just Novell's customer list off of their publicly-facing website. So there's a few of them, universities, banks, hospitals. Lots of people are using it. You had to do a search on Identity Management on the RFPs. Lots of government entities are using it. So worst common configurations, typically when you think of Identity Management, a lot of times people will think of an HR system connected to some sort of enterprise directory, which is then connected to other systems. And it's all automatically provisioning identities, controlling where there are. So HR database, going into your directory. These are some of the common ones that Novell pushes. And they are kind of important, as we'll see later. Because it really does depend on how you set up these things as to how secure they are. So what are the issues? When we look at Identity Management system, kind of what are the issues that come up with regards to security? Complexity. Bruce Schneider said yesterday, hey, complexity is the enemy of security or something like that. Don't quote me directly. But I think that's what he said. But basically, if you have any big complex system, it's going to be pretty difficult to secure it. And Identity Management systems are usually very complex. You look at Novell's Identity Manager, for instance. You've got lots of pieces within the system itself. By definition, you're talking about at least two systems. Entire server stacks. It might be a Windows stack and a Linux stack. It might be on the same server, but a database system and an operating system, or a database server and a directory. But basically, you have at least two systems, because obviously you wouldn't be using Identity Manager if you're not trying to manage two systems. You also have management tools, because you probably want to manage the system somehow. So in the Novell space, that's designer, iManager, formerly console one. You have different ways of doing this. User-facing applications, very, very commonly, you see Identity Management being used to connect into some sort of user-facing application, some sort of portal, user-self service, what have you. Very complex systems in and of themselves. And then you add on to this in Identity Management framework, and it becomes even more complex. And usually you have auditing systems in all this. We're dealing with identities. So naturally, they are going to be fairly high value, because this is usernames, passwords, or other authentication tokens. If you're synchronizing certificates, this is where your authentication tokens reside. This is where they're being synchronized. So to break into the system, or any of these systems, is going to be a high value target. It also deals with authorization. So I want to make the distinction there. But these systems are automatically, based on business rules, determining what you as a user are authorized to do within a system. So if I can grant myself full access rights, then that's cool. I can break the system by doing that. And of course, you're also very commonly dealing with HR systems, which contain things like social security numbers and other good things like that. So high value targets, very complex. And to be quite honest, a lot of administrators just kind of go, oh, it's Novella Secure. Oh, it's Novella Security System. Therefore, it is in of itself secure. And they don't really take a step back and say, hey, how secure is it as an application, as a tool on our network? How secure is it? Best practices are out there for securing identity management. You can go download that off of Novella's website. Sometimes they conflict, and oftentimes they're very incomplete. I've noticed, as I've gone around set up systems and as people have downloaded the best practices. So a lot of times people just don't really lock these down very well. So basically all I'm saying is it's a liability on your network, not a control. So let's look at the security of it. High target for attack. So, okay, so if we're actually going to exploit this, by the way, I should say I'm trying to get through all the talking about things in about 20 minutes and then well about 30 minutes of some demos. And then we'll be done with it. So just some theory of exploiting it. Basically, how do you leverage complexity? Or I'm sorry. In a complex system, you want to leverage that to your advantage. If you have lots of systems, these are usually in very large environments with very strict change control. So really the defender is at the disadvantage. The attacker is obviously at the advantage. Nothing new here. It was very much, or I didn't imagine it very much is kind of a hot topic right now. I think last couple of years has been kind of one of the emerging technologies. And therefore it's been rushed out by companies like Novell. I'm picking on Novell, but honestly, go and look at any of the other ones. They're just as bad, if not worse. Actually some of them are much worse. So yes, I'm picking on Novell and yes, Novell rushed out their product a couple of years ago. They've done a lot of fixes to it, but it's still, you can definitely tell the code quality is not there. Or at least it's not where it should be in my opinion. When we look at an identity management system, we can attack it at a bunch of different layers. Obviously, most of the time you're gonna be having a network in between your systems. Sometimes you'll be doing all this on one box, but I'm not really sure why you would do that in too many cases. So most of the time you have a network layer, you can attack that. You have the connected systems. Basically, you have one of your systems might be a Windows system. So you have the entire Windows stack you can attack. Might be a Linux system. You have the entire Linux stack you can attack. Database, directories, what have you. At the application layer. So the actual identity management engine that's gonna be running all this. The engine's just code. It has vulnerabilities. It has bugs in it. So do all the agents that run all these connected systems. And the management tools themselves also have a lot of bugs. So you can attack it a lot of places in the actual system itself. And then the next layer down, the rules themselves.  We need to codify them. You've then coded them into some sort of logic that the identity management system can understand. And often these can be exploited. We'll show you some examples of that. So, now I'm gonna start picking on Novell, because they're what I know. But again, I do wanna be very clear. Novell is a great product. I do actually like a lot of their products. I'm running Cecil Linux right here. Microsoft's identity management is nightmare. Anyway, we won't even go there. So anyway, I pick it on them, but these are just some examples. And I really do want Novell especially to better their security. So that's why I'm picking on them. Why am I presenting this stuff? Basically, I think Novell has made a few poor decisions in their architecture. Whatever. They made some decisions that aren't really clearly documented. I find a lot of people, a lot of customers will go, they'll implement the system as they think is best. And they don't really have any instructions on how to set up things securely. There's very few best practices. And so they end up just setting it up horribly insecure as a result. And even when they do follow best practices, there are still a lot of ways that the system can be exploited, whether or not the best practices have been used. So we'll look at a couple of those. So, minimal Novell identity manager system.  I think I'd go into that, yeah. So I'm gonna cover all these more in depth. Meta-directory engine is basically a set of Java libraries or a native code libraries, depending on the exact platform.  Within the identity management engine, there's the concept of drivers, which are pieces of rules. There's XML rules, along with a native code component that translates the output of these rules to native code or a native application calls. And as I said, the driver rules and a shim, which is basically a Java class, which has a publisher and subscriber. All right, remote loader, because it is running our remote system, it does have a lot of interesting characteristics. So the remote loader is basically a miniaturized, actually it's not a miniaturized engine, it's an application that loads the shim in so that it takes XML and translates it to native application protocols. And basically it connects to the IDM engine through over the network. Password authentication for it. It uses an XML-ish protocol to communicate with the engine. So hopefully as you're sitting there, you're kind of getting an idea of how complex these systems are if you weren't aware of it. So how are these systems typically used? Basically, you kind of your standard HR database, off and running on a Windows platform or an Oracle platform, something like that. Usually has a remote loader out there, which is basically second things out of HR. Putting that into some sort of enterprise directory, or putting it into your e-directory in Novell's case. And from there you're synchronizing out to different directories, different applications, that sort of thing. And there's usually this flow from HR to the identity vault, and then you synchronize from there, which has its own security implications.  Another common configuration, basically you have Novell admins that do all their work in your legacy NDS tree, that gets synchronized to a meta directory, and then that gets synchronized out to some sort of self-service portal, what have you. This is a fun one, bringing Novell into an active directory environment. Some CTO says, hey, we're gonna start doing Novell stuff. And so the Windows admins keep doing everything in active directory, and it automatically synchronizes over to your Novell directory. But this has some common, I mean, there's some problems with this, right? Because is active directory and e-directory equally secure? They can be. I think there are ways that you can set up active directory to be secure. I was back when I was working for a consulting company. We went to a bank where it had this exact setup we didn't know at the time. It took us about 45 minutes to compromise active directory, get an admin there. That's cause, 45 minutes, cause I had my firewall on on my TFTP server. Took us about half an hour to figure that out. So we got domain admin on active directory, couldn't figure out how to get into e-directory. We're sitting there banging on it. So we tried the admin user that we created in active directory. We thought, hey, why not, let's give it a try. And lo and behold, it was there. Automatically synchronized across. So we actually had an admin user in e-directory as well as an admin user in active directory because identity management system had very nicely synchronized that all across. So that's just something you have to keep in mind of, when you further upstream you're going with your identities, the more carefully you need to be about how the systems are gonna be broken into and protect them better. Or put in business rules to protect yourself when they do get broken into. So choosing your targets, obviously, aim high in the identity flow because you're gonna have better payoff. You compromise the HR database, then you can basically create any user you want in the system. Whereas if you're just compromising one way down at the end, who knows, maybe it might synchronize back up, might not. But this is gonna be different from every environment. So I'm not saying, I'm not making any blanket statements about how you need to set out your network. But you just need to make those decisions, keep those security implications in mind as you're laying out your network. Just real quick, I wanted to go over the security best practices from Novell. There are some really good ideas here, things you really need to do. And some things that really don't help you at all. So their number one thing, use SSL. Absolutely, use SSL. It makes it a little bit harder to do certain types of attacks. It makes it impossible to do other types of attacks. And yeah, it's a good idea. It protects the confidentiality for someone just sniffing your network. Modern control access to your driver sets in your tree, obviously by default, very few permissions on those objects, so you should monitor them. Don't allow too much information in your password hints. If you're doing password self-service, force password changes. Follow industry best practices for security measures, such as blocking unused ports on the servers. Wow, that's a nice statement there. Designer, if you're using Novell's designer, which is kind of their design tool as well as a management tool. In that case, you should limit your consultant's rights in your tree. Not sure why you only do that with a designer, but that's where the best practices stay. Control your project files. Basically, this is a fake application running on your consultant's laptop. It stores the passwords, we'll cover that later. And don't use encrypted attributes because designer can't understand them. There's a great security best practice. Track and changes to sense of information, basically audit things well. All right, so that covers the theory exactly 20 minutes. Let's talk about how we would actually go about doing some bad things to one of these systems. What would be the goals that we're trying to do? I mean, we talk about exploiting a system. What are we actually trying to do? Someone yesterday was talking about how the goal of your exploit is the data. I think it was H.D. Moore's talk, excellent. The goal really is your data. I mean, it's whatever data you're trying to get at within the system. But to get at that, you need identities, you need authorization, and that's what you're gonna be attacking the identity management system to get. So, one goal is to just gain an identity. If you can create your own user, great. If you can get a user created in one of the connected systems, great, not as much access with air, but you still have access. Exceeding your authorization. If you're an employee, the identity management system says, hey, you're supposed to have access to these files. If you can escalate that up and exceed what your authorization is based on system rules, great. Stealing someone else's identity. If you can steal their authentication tokens, you can become them on the network. Not that difficult to do. And basically violating the integrity of the identity system. So, that's our goal. You come into a network, how are you gonna actually find out whether or not it's running identity manager? Maybe some of you guys work at companies and you don't know actually if you're running identity management at your company. Or you've broken in over the internet and you don't know if this network has identity manager installed. First thing you're gonna look for is if you can publicly browse the LDAP directory. You can search for the object class. Object class is by default, readable by anonymous. So, anyone can anonymously connect to your LDAP ports and look for object class equals direct smell driver. That's a dead giveaway.  If they don't have identity manager, which formerly direct smell, they don't have that installed. They don't have the plugins to manage their identity management system installed on that version of iManager. Then these will come back with 404s. If they do have the plugins installed to manage identity manager, you get 200 coming back. Easy way to find it. Remote loader. You're remote loaders, so there's little applications that are basically just translating XML to native code. By default, they run on port 8090. That's kind of the default start one. And then if multiple remote loaders are running on a system, it'll just increment up the ports. So, looking for open ports on 8090 and above is cool. But they are only open when a remote loader is listening for a connection, but the engine has not actually initiated that connection. Once it does that, then the remote loader only talks to the engine. And it doesn't listen to the network. And that does not return, it returns a closed port, in that case. Lastly, file system. I don't know how many people have access to their Linux servers that are running directories, but if you do, userlib.directs.mil or cnovell remote loader, just two of the places you can look to find the installation. So, these are complex systems, lots of systems involved. Obviously, you can do Windows exploits. I mean, name an exploit, and you can probably exploit a component of an identity management system. But nothing new there. Exploits in the actual identity management system itself. In the actual code that manages the identities. That was interesting to me. Saw some things that looked a little bit ugly. As I mentioned, code quality looked a little bit poor to me. So it was something I kind of took a look into. So again, you have med directory engine, you have some shims, you have a remote loader. Lots of good stuff to, lots of good targets to look at. So, three slides back, I kind of mentioned a very trivial denial of service, which, very simple, basically, we have remote loaders running on a remote system, listens to the network. But as soon as someone connects it to it, it turns off. So, you know what? Netcat, great denial of service tool. Just connect it to the port if you see it open. And it, basically your engine can no longer connect in. As long as you don't send it anything, it will hold that port open for forever, just waiting for you to initiate with your password. So, if you open the port, basically the whole system can never, can't talk to the engine. Why is that a problem?  I was gonna demonstrate that, but yeah, oh well. Ranifuzzer on this, you know, we're connected over a network. Let's look at some fuzzing statistics. Basically, it took about a minute to five minutes till you crashed your remote loader. When we were doing a pure Java remote loader, so the remote loader was pulling in a pure Java library, basically Java would just get so confused and couldn't tell, you know, one side thought it was open, the other side would try to connect, and it was closed, and all sorts of confusion there, and basically we'd just come to a halt, not be able to resolve itself after about five minutes. If you're loading an actual native code, like the Active Directory driver, you actually crash the application in about one minute. I do have dumps if anyone knows how to write good exploits, talk to me. And about 20 minutes to crash the engine. Basically if you're fuzzing the engine side, it took about 20 minutes for the engine to just go to all weird. Again, very heavy use of Java, so I don't think any of them can be used for a remote exploit, but you can definitely throw into weird states where it just doesn't know what it's doing. And I did not get around to fuzzing the XML definitions that are actually held up in the directory, so directory objects that contain all these XML files. Been trying to do that, but I have to write my own fuzzer for that, and haven't gotten around to it yet. Oh, DX command, which is kind of the command line utility that uses NCP to control the driver functionality. Yeah, it basically choked on the first packet that got fuzzed, so it would just quit with the Java dump. So not real reliable. For something that's really managing all your identities, not all that reliable. So, let's say we can't fuzz things. If we own one of the systems, if we actually control your identity management, control the meta directory, so the e-directory, we can just go and change business rules pretty much any time we want.  So, not too useful, but obviously, someone who's attacking your identity management system, they can do this, and what controls do you have in place to prevent someone from doing that? That's a question you can answer. Modifying the system libraries and the applications that are up there. On a Linux stack, running my demo system, I'm running OES here with e-directory. Good default permissions in the directory, great file permissions are very tight. We then go over to Windows side, and full control is granted to everyone by default after a default install. So, I had a little fun with that. Basically, I thought, hey, if I can create my own shim without anyone noticing, since full control is given to everyone, so me being everyone, I could just log in and change out the shim. Why not? So, in our test tree here, basically we have a remote loader, remote console here, which is just controlling the connections to our different remote loaders that are running here. One of them is a completely named driver to be Trojan, which is just a little mid-text driver. It's up there in our tree, I can, wrong way. So it's just a driver running in my tree. So, what do we do? Basically, when the remote loader starts up, it reads the novell remote loader lib directory, pulls in every single jar file, uses that as your class path. And then, if you look back at any of your, the properties of any of your remote loaders, the Java class that's supposed to be called to initiate this shim, it's listed right there. So, two things we need to do, we either need to swap out what that class points to within this directory, which is a little bit difficult. You get some files being locked and stuff when you try to swap that out, or you just change them all. So, that's, you know, just wanted to demo that real quick in case someone doesn't quite believe me. Basically, it went and downloaded the driver SDK from novell, compiled my own driver, added in a little bit of code that wrote a file to a, writes out a file to the C, you know, the root directory. I'm not a Windows person, so if I talk a little bit, UNXE, sorry. So basically, it writes a file out to your C directory, and then starts up like a normal driver. So, we'll just, you know, copy that over since we have full control on the lib directory. That's cool. So now we have to make sure that when the driver starts up next time, it points to the right class, it points to our hostile class. So, we'll go conveniently, our remote loader told us what the config file name is, which was config. So, there is our class. Just swap that out real quick, because that's fun. We'll save that. And obviously this is loading memory, so, you know, it's not looking at the config file right now.  So, you know, causing a denial of service on these is far too, far too easy. So, but for demonstration's sake, I'm just gonna stop it. So, for our C drive, you know, no weird files there. So, let's start up a driver and demo gods do not. There we go. So, woohoo. Basically what it did, start the driver, driver, my custom driver, write out a file to the C drive. And it's running as local system, so basically you can do whatever you want.  That's about it. So, that was fun. Go build your own by go downloading the SDK, just use ant to build the default examples there, and add in wherever you want. So, we wrote our own shim. Can we write our own remote loader? You know, that might be possible. I looked at that, said, you know, hey, basically, the engine is going there. It's, you know, it's given the IP address or, you know, the DNS name of the remote side. Can we write our own? That's gonna mimic that and do some password stealing. I tried it out. One of the things I'd overcome was, you know, basically they recommend that you use SSL on this, but kind of get to that, they use it in kind of a weird way that basically SSL is used to authenticate the engine, but it's never used to authenticate the remote loader. So even if we're using SSL, we can get around that because it's not cryptographically protected. It's just authenticated by password. And there's basically two different passwords that are used, your driver object password and your remote loader password. And your remote loader password is actually used in two different ways. One for the command, the little command window that actually controls the driver, and it's also used for the engine connecting in. So, pictorially, hey, your remote loader's there. Listen to the network, cool. Password authentication, when the command piece goes in and tries to connect to your remote loader, it does this great cryptographic challenge response. It takes a timestamp and it hashes it a couple times, verifies it both sides and know the password without ever passing the password over the network. Beautiful, that's how it should be done. When we're talking about the connection between the engine and the remote loader, your passwords get sent in clear text. I don't know why. But anyway, they do. Because of that, we can write our own little trojan. One thing I just want to point out. Many people use the same password for these multiple passwords, and nowhere in the Novell documentation does it say, hey, use different passwords for these. So, every Novell training you go to, most Novell consultants I've talked to, they always use the same password for the driver object and the remote loader. Bad idea, because your engine is saying, hi, here's my password, now authenticate to me. And if you're coming back with the same exact password, that's just too easy. So, it's easy. So, because it's easy, I decided to write a little tool to do it for me. Have a little server, I'm gonna start up. And then the other thing I need to do is basically, there are many different ways of ARP spoofing DNS redirection, plenty of ways. You can get redirect where this engine is talking to. For demo sake, because the demo gods hate me, I'm just gonna go in and change the IP address if it ever decides to load. So, formally we're connected to our remote loader directly, I'm just going to change that to go to my server, which, and so as soon as you make a change, it's gonna restart the driver, woohoo. And let's go see what we got. We didn't get anything, there we go. So, basically we was just listening to the network, the engine said, hey, here's my remote loader password and please authenticate me and start talking to me. I basically said, great, thank you for that password. I'm gonna turn around and use that, just try and use that for my driver object password. And so I passed it back using the right protocol and it gave me a success message so I know now that my passwords are the same and it prints out here. This little tool here will be up on, or will be pointed to, I haven't quite figured out a good place to host it yet, but I hate novell.blogger.com. I'll put it up there. SSL, again, I'm just gonna mention, novell does some great things with cryptography and then every once in a while you just wonder what were they thinking. So they've made it very, very difficult to talk to remote loader as a fake dry smell engine, but it's really easy to talk to it as a fake remote loader because there's no cryptographic authentication of the remote loader. It's physically impossible to load a certificate up on the engine side. And so, that makes it all possible. However, you do use a really funky version of TLS so I'm still kind of working out that to get this tool to run under SSL, but it should not be a problem. Like I said, it is possible. Okay, using the rules. Basically, to actually figure out how you're gonna use rules to your advantage, you have to know what rules are up there. Fortunately, you can just do an LDAP search anonymously on your directory tree and by default you can get back the names of all the drivers because it's in your CN, it's in your RDN. So, pretty much anyone can go there and read what rules you have defined for your RDN installation and they can compare those to the default rules because most people go, they download the drivers from Novell, they use most of the default rules, they change a few and so people can basically read what your business logic is based off of just an anonymous search of your tree unless you turn that off, which I highly recommend. So, for example, if I go up and read that you have an Active Directory driver up there and I know that Active Directory driver by default matches on full name and I can troll one side and obviously I can start exploiting that by exploiting the default business logic that Novell gave you. And obviously if you have some custom logic in there, I'm not gonna know that as an anonymous attacker but as a reasonably tech savvy person in your corporation, I can probably make some good guesses at it. Exploiting the rule processing. This is another fun one. Basically your shim is taking XML code, translated over to native application calls and different drivers handle it differently every single one of these shims is written by a different person, you have vastly different code quality. I happen to have done a lot of work with the JDBC and doing database connections, so I had some fun with it and yeah, basically by default, if you just do a default install the JDBC driver, this nice little rule comes in there where it takes your common name and uses it in an update statement on your SQL server running as DBO. So hopefully most of you get where I'm going with this and within the next 10 minutes, we will quickly demonstrate a SQL injection in direct-small processing. So have a little table here. This is my default database. Basically just a default install of the database straight from Novell so you can go and duplicate this yourself. I added in this little table for my little proof of concept. I'm now going to add in a user, there we go, to my directory. That one guy had a few special things added to him. And let's see, so got a few errors. And in this table, hello Defcon, yes we have SQL injection in your driver. This is default driver from Novell. So anyone using the default driver from Novell should check up on this. SQL injections, last one, passwords. A couple of places that we deal with passwords, obviously a lot of people use identity management for passwords. Don't use passwords. Okay, I can't say it enough, do not use passwords. They are the weakest form of authentication, but unfortunately a lot of you have to use them so you don't have a choice. Let's be careful, at least know where your passwords are going. Using identity management for Active Directory on the window side you're gonna have a password filter which is basically just a little DLL that you've put in Active Directory which says, hey every time someone changes their password in Active Directory, write it out to this registry key. It does encrypt it, so if you go and look at it, Novell has not, I couldn't find out from Novell publicly how that encryption worked, so all I can say is it is encrypted. Go and look at those keys yourself if you can find a way to decrypt it. Cool, write a utility for it. Universal passwords, so this is actually where my dislike of Novell started and I have a rocky history with them as a result. But basically they decided as a company to go with this whole idea of universal password which was, hey let's store our passwords in E-Directory instead of using this great hashing function that NDS has been using for years, let's store them just encrypted with triple des. Okay, we're also storing the key to decrypt it on the server, so that's a problem if you're storing the encrypted version along with the key, is that really encrypted? We can debate that. But for the most part it works pretty well. Basically access into those passwords is controlled by a couple factors. You have to be an admin, you have to have rights to that directory object which is by default only given to admin. You have to access it over SSL, if you're accessing it in clear text it won't give it back to you. And there's password policies that govern whether or not admin can read the password. Of course admin owns these policies and admin can change the policies whenever they want. That was kind of my reason for thinking this was kind of a bad scheme. So as a result, yes it is possible. You have your password policies, so as admin I can just go in them and change them to say hey, let me read them. I can then basically just go and ask the directory say hey, give me everyone's password and I have them. Yes, this is what you're forced to use if you're using password synchronization with Identity Manager 3.0 or later. You have to use universal password and universal passwords are just encrypted passwords and encrypted passwords can be decrypted. I think I've hammered it enough. So that's, yeah, speaking of passwords, I mentioned designer which is that nice little management tool has a encoded password and they do warn you about this if you read the documentation it says do not store your admin password in here. I was gonna put up, I couldn't find my pearl script to decrypt it, but basically it's base64 encoded and you take the, it's basically base64 encoded string and you take the first character of every three characters and move it to the last one and then you just base64 decoded and you have the password back. It's those kind of things that just annoy me because I'm a crypto guy and you can encrypt things so easily why would you base64 encode something and then do some transposition to encrypt it? I don't know, but those are types of things I get mad about. So coming up on the end of this talk I just wanted to say River wrote the art of fuzzing. Your proxy fuzzer is my friend. I found that tool, I love it.  Does great for fuzzing network connections. While I was in here doing this presentation I wrote a little pro script to go in and bulk update every attribute to a fixed string on your user, test user and your database. I recommend people doing this with things like common SQL injections, cross-site scripting. I didn't get to it because I don't have time but yeah, do some cross-site scripting and then try and view your user with iManager. You get some great results. But I'll have that script up on the website as well or link to it somewhere so they figure out how to do that. And fake remote loader, yeah, I'll throw that up there as well. Just go for it, use it. Like I mentioned, yeah, there's several places where iManager just chokes when you rewrite people's attributes. It's pretty ugly. Novell audit, another good application in theory, bad in implementation. Sorry, there's just been lots of bugs in implementation. And fortunately it's been replaced by Novell Sentinel. So, cool stuff. Really easy to fingerprint your drivers. User application, this is a huge application. It's your Linux server running in JBoss with this entire portal application on top of it. It's kind of the new iManager 3 if you're familiar with Novell. It was one of the big selling features of IDM3. And basically, again, it was just not the highest code quality. Lots of ugly things in there in the whole implementation and I don't even have time to do all the write-ups on those, but maybe next year. And then designer, I haven't even started looking at designer yet other than the little password thing. So, lots of work to be done. Basically, I can't find anyone else there out here who is doing research on the security of identity management systems. So, if you're out there, please talk to me because I'd love to thank some ideas off you. But yeah, I'm out there doing it by myself at the moment and would love some help. We don't really have much time, but basically, whatever background you come from, you guys might think I'm crazy. Maybe I am crazy. Maybe I just think this is bad design is some personal flaw, but I don't know. Maybe I'm way off. Maybe I'm right. Who knows? You guys can make up your own mind. So, conclusion. Hopefully, all you guys can raise your hands saying you're running Novell Identity Manager. You're aware of some ways people might exploit your system. Maybe you can go back and, you know, just put on your security hat, take a look at your system and remember it's not invulnerable. Yeah, I hope Novell takes a look at their own stuff and I really do hope someone out there sees it and I have one minute and someone else comes, joins me, doing some fun stuff. That's it. I have some more stuff I skipped, but that's what the Q&A room is for afterwards. So, thank you.