So let's go ahead and we'll introduce ourselves. I'm Tom Holt. I'm an assistant professor at Michigan State University. I run an open-source laboratory there for internet-based research on computer hackers in various communities online, be they deviant, criminal, or otherwise. And I'm a criminologist by training. So my interest is in understanding social dynamics of behavior, what drives individuals, what motivates them, and sort of subcultural forces and dynamics. I've been looking at hacking for the last five or so years, and just find it fascinating. I'm Max Kilger, and I'm a profiler for the Honey Net Project. I'm a social psychologist by training. So my main interests are motivations, future threats, counterterrorism, stuff like that. And I also run the Spartan Devils Honey Net Project, which Max is a partner of as well. Our other two colleagues who are not able to make it today, Dr. Stromsky and Dr. Smirnova, played a critical role in developing this research. In fact, some of the slides that we'll go through are directly generated by them. I am not a social network analyst by training, so a lot of what we'll go through should do have very, very, very specific questions if you are a network person. We can certainly try to get as good an answer as possible. So with all of that in mind, let's go ahead and jump into the presentation itself. First and foremost, you go to DEF CON, you're going to hear a lot about the new hotness. You're going to hear about Conficker, you're going to hear about Storm, you're going to hear about the zero-day exploits, all the really funky, sexy, brand-new stuff. But at the same time, there's a wide range of old stuff that's circulating around. If you look at the hacker community regularly, you've probably seen pinch. There's all kinds of free bots out there. There's crypters. You can still find sub-7 versions floating around, script kitties downloading anything and everything. So you've got this dynamic of old and new. Why do we have so much old stuff circulating around? Why do we have new stuff that only a handful of people know about? What really accounts for this relationship? So there's a variety of explanations that can be given. The old stuff works, and some of it works really well, if you download the program and know how to compile it, if it's not totally broken. If you buy it, you can get a lot of that stuff cheap. You can find a cryptor for $2 or $3. You can buy various iterations of pinch in some sort of working format for $20, $30. You can buy all kinds of stuff. Another potential explanation is that the really, really high-skill individuals who have the capacity to program and develop a root kit that works really, really well are not possibly as high in number as the semi-skilled individuals out there who can run the programs, who can kind of understand how they work technically. Do you have any color to add, Max? No, it's okay. And we're just going to sort of tag-team this throughout, so you're probably going to hear a couple different voices as we go. So it might be a reflection of audience and ability. There's a smaller potential triangle at the top of the great guys and then a wider audience of the folks who can kind of use these tools and maybe can't. So with that in mind, the question that we're asking is how do we find those really, really high-skill folks? How do we identify the individuals who can create the new tools, who can find the zero-days, who are writing the code to really make this stuff happen? And then where do they sit relative to the population of semi-skilled users, of the individuals who sort of have the ability, maybe they don't? There's not a lot of social science research looking at the hacker community in this kind of context. We've looked at social ties qualitatively with interviews and things like that, but there's not a lot of predictive research. Attempts to model and proactively find individuals and figure out what they're up to. And so that's our goal. What we've done is to go to all the resources that are out there. There's forums, there's IRC, there's blogs, there's all kinds of stuff that you can go to to mine for information. And so what we've done is focus on the Russian and Eastern European hacker community using data generated from social networking sites, from individual resources where they're blogging, where they're talking about themselves through live journal. And the blogs themselves give you a great deal of information. They do talk to a certain degree about what they're up to, what they're coding. They talk about their social relationships, who their friends are, who they're talking to. And you can get information about where they live, what they feel is important, how they spend their time. So you can understand a lot of the social world of these individuals looking at this kind of data. And this slide gives you a sense of all the various bits of information that are immediately provided in a profile on live journal. They can list their location, their education, be it through affiliation where they're directly attending. They give biographical information. They do talk to a certain degree about their social state at any given point in time, whether they're angry or they're happy or they're sad, what are they listening to. There are interests, and these can be geographically bound, and that speaks again a little bit to location. We also get a lot of information about friends. And on live journal, there are some distinctive points about social relationships. Friends are the individuals that one person looks at. So a friend is someone that you look to for information. Also friends are people who read your journal but don't have access to your really sensitive stuff. Mutual friends, and this is the most important point, are the individuals who recognize and read each other. So it's an immediate recognition of we are peers, we are clubs. And then there's communities, the different groups that people belong to. And actually what it sort of brings up is an interesting point is what you do this information really takes you in several different directions. If you're interested in prosecution and pursuit, this is one way that you're going to go after that. But you also take an entirely different direction, which is basically more from an maybe an intel community standpoint, which is where is the community going, what groups are going to coalesce together, what new threats are going to emerge. And so really depending on what you're going to do, this information can serve several purposes. And so for our purposes, we're going to try to understand what the networks look like, what's the community look like in terms of demographics, in terms of peer relationships, etc. So to give you a nice robust example of what a profile looks like, this is an individual named Crash. Crash is a person who was extremely successful and skilled. In 2006 and 2007, he released a bot called SuicideDDoS. It was deemed in the top three by some different communities in terms of active functioning bots. He sold it, he's released a lot of different code. And so to look at his site, you can gather a lot of information. We can see his physical location in the Russian Federation. We can see his email address. We get the teams that he belongs to.  We get his interests, and you can see there are some good interests here. They're in alphabetical order, so he talks about alternative, he's into beer, botnets, bugs. Cyberterrorism, which is kind of an interesting one to throw out there as important. Drugs, exploits, hacking, programming, ring zero. So he's got a ton of interests, even traditional terrorism. He doesn't give us anything about school, but we see his friends. So we have a sense of his associations. And to read Crash's blog and to look through the content, you can get some good surprises. Crash isn't here, is he? Oh, good. Okay, hopefully not. Crash, during a marathon programming session of about 48 hours, decided that he was fed up and needed to take a break, and so he took a screenshot of his home setup. Mid-process of writing a root kit for Linux systems. He says, I've been smoking, I've been drinking, I've been up for 48 hours. I need to take some sleeping pills and just pass out. So he takes this picture beforehand. You can see he's got two laptops. I don't know how clear this might be from the back, but he's got two different boxes running and a tower. He's got Ali Debug on one. He's got another sort of assembler program running. So he's mid-progress. Plus, you can see this giant empty bottle of vodka in the center. So he's drinking, he's coding, he's trying to have fun. 427 a.m. and just plowing along, and that was the title of that post. So you can find some really sexy stuff in these blogs. So with that in mind, what we're going to do is walk you through the social dynamics of this community using 364 individuals from eight different groups. These are all individuals with live journal profiles. The groups that we selected are actively involved in the distribution, creation and sale of malicious software, and to a lesser extent stolen data. So this group includes Haxona, Mazafaka, if you're familiar with any of these communities, some of these might be ones that you've heard of, the Hell Nights crew, so lots of different stuff. Dr. Smirnov, who's part of the project, is a native Russian, and she was instrumental in helping to translate a lot of this information given that I understand Russian a little bit, but it's not as good as her ability. We also did a lot of open-source searching, going to Google and looking at each specific handle, plus a variety of keywords to try to assess the individual and their level of risk. Are they heavily involved? Are they producing malware? Are they selling it? Are they just posting in hacker sites? Do they just run a security blog? Are we not finding them at all? And so using that as a metric of risk. And one of the things I want to make clear here is this is not a random sample, and so sort of basic inferential statistics that you might otherwise use aren't really applicable here. Understandably, you can't really get a random sample out of the hacker community. It's very difficult. The numbers that you see and the trends that you see, they're trends, but you have to take them with just a tiny bit of green assault just because it's not a true random sample. And that's an excellent point. And that kind of ties into some of the threat level assessment as well. What is important for you all to know is that we look at this data. We coded the individuals from zero to three or from one to four.   You're just talking about what's going on in vulnerabilities and hacks, et cetera. You're not actually participating in attacks yourself. The twos are the moderate individuals who are posting in hacker sites who have maybe some status or they're registered in a variety of places. And then our high-risk individuals, our threes or our fours, are the people who are actively releasing malware, who are administrators of sites who have specific roles and functions within each group. So crash would be an example of our three or our four assessment. So with that in mind, here's the data. Here's the different groups that we have. You can actually see that there's a lot of variation in terms of how many members there are in each of the group. Two of them have over 100 members and there are a couple of them in single digits. So there is a lot of variance. And that's a very important point. Given when you talk about a community like Damage Lab or Mazafaka or Zloy, these are groups that have very wide and high numbers of registered members. So the reflection that we're getting here might just be a slice of that community. The account variation here as well speaks to the type of live journal that they have.  You can set up a donation program where people can pay you or give you something in particular. So there's some variation here in terms of how these individuals have accounts as well. So beyond accounts, now let's talk about risk or threat levels. So we talked already about this. We searched on individual handles, we gathered the information, we compile it, and we put it together to see what's happening. So this is again the detail. We have zero, one, two, and three. Threes are high risk, two are low risk, one's a security blogger, zero is no risk. Let's see what this population looks like in a nice pie chart here. So actually what you can see is that if you believe the numbers, you really have to worry about 6.3% of the folks, plus or minus something. So that's actually kind of an interesting thing. Almost 70% basically don't have to worry about. It's the top six where you're going to have problems. And so the twos, which comprise about 19%, are those semi-skilled or at least somewhat active individuals who are looking at the community. And they're larger in terms of percentile than our threes. So again, if you believe the notion that there's more high skill than low, pardon me, there's less high skill than low skill, this is reflected out here. In terms of communities and risk, this again speaks to some variation. And there is actually a lot of variation here as you can sort of see. You can look at the mean risk number and look at the standard deviation. And this is kind of interesting because in the past the hacking communities have been a lot more, what you'd call sort of, Stadious Homogeneous. That is, they had skills that were sort of very similar in nature and that's how they sort of formed as a group. But what we're finding now, sort of years later, is that there's a lot more variety in the skill levels that we see in the hacking groups, which is a very interesting sort of phenomena. Yeah, there's some clustering almost where the high skilled people are hanging together. So the Hell Nights group, the two guys who are in it are very, very good. Hackzona, there's a lot more variants. Ruhack, and Ruhack is interesting here because they are a live journal only group. They don't have any other metric outside of this community. There's only nine members, but their risk level is somewhat higher than Damage Lab and Mazafucka. And that's actually really interesting because you might normally just sort of roll over, are you hack and just say, oh, well, we have to worry about those guys. But when you look at this sort of mean risk level, what it says is there's probably something else here. Maybe you better take a closer look at these guys. So in terms of risk and group membership, one thing that we noted is that the higher an individual's level of risk, the more groups they belong to. And that, again, might be an important indicator. They're not just in one community, they're participating across community. Yeah, I think that's actually probably a pretty key indicator. So in terms of journal entries, what we wanted to do is just look at some of the basic relationships in terms of their use in live journal and their threat and group assessments. So this is the average number of journal entries by group. And you can see Ruhack has a very high level of journal entries. But the Hell Nights crew don't have as many, neither does Kupzu, neither does Lloy. So journal entries point to some kind of relationship. In terms of risk, however, the relationships are somewhat more distributed. It's not a really good indicator of anything other than just posting. You can see that our risk three folks, the more dangerous folks, are as active in terms of talking as the risk zero. So moving from posting, let's look at friends, at the individuals these people look at. And so the distribution here, we see BH crew. And for those of you who don't know, the BH crew or Bugger Hucker crew goes out, finds active members who can identify zero-day exploits, has them work on code, and they put out sort of little manuals online about, here's the latest stuff. And so the BH crew have a lot of friends. Damage lab, the Hell Nights crew, Lloy don't have nearly as many. So this is somewhat interesting. When we look at friends by risk class, we see that the level three folks are looking at a lot of people. The risk ones are looking at a lot more. The distribution is again somewhat different. So friends, let's get to the really sexy meaty part of this. The mutual friends, the people who know each other and talk to each other frequently. We see Mazafaka and Ruhack have a lot of mutual friends. The Hell Nights crew, Damage Lab, Kupzu, not as many. And this is a somewhat significant point in terms of group variation. Thinking about mutual friends, where do the risk levels come into play? And here you can see that the high-risk folks, risk group three, have more than risk two and more than risk one. And that actually makes pretty good sense because they're a much more sort of cohesive group. They're more highly skilled. They need to trust each other more. They're more center-focused. They're a much more sort of dense social network and less likely to sort of want to associate with outsiders. And as we have lots of people coming in, there's open seats all over the place. I can see a lot of them. So if you've got a seat next to you, raise your hand real quick so we can get everybody into a chair and not have all the aisles blocked. Cool. So yeah, if you've got an open space, let's get everybody in here. So we've got mutual friends. I'm going to try to continue talking, and I'm going to talk really loud for a few minutes just so I can be heard over the den. So we've got mutual friends. Now let's look at group membership. Group membership, again, is an important indicator because what we're seeing is how many different communities these people belong to. It could be that they're with another forum group like Mazafaka. It could be that they're just tied to a heavy metal group, something like that. So here we've got Haxona belongs to a lot of different groups. Ruhack, again, there's pretty high distribution. Given that Ruhack has a small number of people, but higher risk, that's pretty relevant. Yeah, and in fact, what you see is the groups with the higher risk individuals in them, they tend to have more narrow interest. They tend to have less group membership interest as well. In terms of group membership and risk, we see an interesting pattern here. Our high-risk folks don't belong to as many communities. And that might be an indicator, again, of a certain degree of isolation. They don't want to be as heavily involved perhaps. They want to stay sort of under the radar and so they're going to stay sort of much less visible by not joining as many groups and having as many sort of casual associations. So the final metric that we wanted to consider is how long have these groups actually been around? How long are they established within LiveJournal? Bugger Hooker Crew, Cup Zoo, Zloy, Mazafaka have had a pretty long-term relationship in terms of months, so 50 months or more. That's a pointer about these community establishments. When we think about the hacker community, largely it's going to be forums, IRC, but it seems like they're active in some other outlets as well. In terms of month and risk, everything seems to be relatively the same. There's not an easy point of identification or attribution in terms of time. That's kind of interesting because you would think that the people that were highly skilled and have been around for a while, they'd be around longer on LiveJournal, but that's really not the case. It's not a very good predictor. So now that we've covered the basics of LiveJournal, let's look at the population itself. Let's get into some of the demographic relationships. When we think about the hacker community, there's some ideas that are always permutated about gender, about education, age, etc. So using the data that we have, we wanted to try to parse out as many demographic relationships as possible. Given that individuals might not necessarily admit certain bits of information about themselves, we went through an extrapolated information based on posts. So in terms of age, someone might get a shout saying happy birthday on a specific date, and so we can use that as a metric to understand age. I see Q profiles which individuals give their numbers. When you look at the profile, sometimes they say how old they are. In terms of education, assuming that there's a standard trajectory of age within the Russian educational system, we can give some idea about how old an individual might be. And actually they may just stop for a second. One of the questions we're often asked is, well, if these guys are really skilled and they're trying to stay under the radar and they're under pressure from law enforcement or Intel sources, why are they putting this stuff up on the web? We get that question a lot. And a lot of that has to do with social identity.  You can't really communicate with other people. They don't know who you are. You don't really have an identity. So in order for them to basically exist in that virtual world, they really have to put some information out there so that they have some sort of online identity. That's sort of like issue one. The second issue has to do sort of with commerce. If you're creating malware and trying to sell it or you're carting and trying to sell it, you're like any other business. If you don't advertise, you don't get customers. And so even though you may be very skilled and you're collecting this information you're going to sell, you have to advertise in order to be able to sell it. So you've got to be out there. So just to give you an example of how we would determine age. One of the users, ZDoZ, or I believe that's a correct pronunciation, said he graduated from high school in 2002. He gave us when he studied at a university. So from 2003 to 2008, he says he's been studying somewhere preparing for studies. So taking that as a baseline, if we assume most Russians graduate by 16, if he's in university for five years, then he might be around 22 years of age. So this is an estimate. This is one potential. This might not be concrete, but it's at least a rough guess. And so looking at age distribution, this chart points to something extremely relevant. We see in terms of age patterns, they're somewhat similar across group with one exception.  And that, given that these are individuals who are extremely high risk, we don't have any idea about age. And that actually might be a useful clue is when there's not very much information, that suggests that maybe you better hunt harder. These guys might be adept at hiding themselves, but very instrumental. Pardon? Two members of the Hell Nights crew have this. So again, it's, but these both individuals are high risk individuals. So in terms of age and risk, the distribution is somewhat the same. But there is kind of an interesting note, which is sort of, you see there's sort of a slight decline with each class, which sort of makes sense because some of the younger hackers tend to be a bit more adept, pick things up a little faster. And so that's why you see that, except for risk class three, we're the highly skilled folks. And which basically sort of says, if you're going to be highly skilled, that takes some time, experience, and education. And so that you have to sort of earn your chops. That's where you're a bit older. So looking at age, we also look at education. About 14% of this sample were actively enrolled in some kind of educational institution. But again, we see some unusual information provided by these folks. So some of them say that they're at the Scientific Research Institute of Sorcery and Wizardry. I want to join that one. So clearly a fake place that's actually tied to a popular book within the Russian community. Fake location. We get information that five individuals are attending Lomonoso Moscow State University, which is a very prestigious institution there. The National Research Nuclear University, Moscow State Technical University, and some of these places do in fact have supercomputing capabilities and very strong engineering and computer programs. So the few people who give us these bits of information appear to be, to some degree, in very high class educational places. We get a little bit of a typology here as well. Some are studying engineering, mathematics, physics, and then some are at technical institutions. So distributed somewhat interestingly. Then we looked at location. And location can be termed based on the place where they talk about going to school, could be determined by ICQ profiles, could be from the communities that they belong to, as there are some geographically bound communities. So it might be pertaining to Ukraine or to St. Petersburg or something else. Interests, same way. And we can corroborate the information that they provide in terms of a location with some of the descriptions of places and locations that they visit in the journal entries themselves. And this is really actually pretty important for the pursuit of prosecution, guys, is there's really a tendency for people to eventually disclose where they live or where they're going to school. It will eventually come out. So if you look hard enough, you will find it. And also this points to some trouble with extradition. If we're thinking about how some of these places might work with law enforcement here. Looking just at a general distribution, we have 64% where we can't determine anything about locations. So a healthy percentage is hidden. But of the individuals where we can gather information, many of them appear to reside within the Russian Federation and to a lesser extent to Ukraine. And so breaking that out a little bit further, looking at Russian Federation, we see Moscow, St. Petersburg, and Novosibirsk as the most prominent locations noted. And then the 10% in Ukraine and 2% in Belarus. But location talks about something very, very interesting. The Buggerhucker crew talks about meetups in person in certain places in Moscow. The Buggerhucker crew, guys, know each other really, really well. And if you read the information on the right here, B.H. Crew CQ, the command, reminds that we are seeking the information on unfaithful follower EPXOFF, his nicknames, he gives a variety of nicknames, his approximate IP address, his probable place of work, all resources, a common pool of money, something. The Buggerhucker crew is putting on the table. All comments are screened, we won't forget you. That actually really points out a very interesting component of the hacking community, which is there are very strong norms within the community and within the groups.  So it's much stronger than you might find, say, in a traditional mainstream society if someone breaks a norm. So looking at locational variation, we just wanted to compare what we have in terms of Russian Federation against other metrics. So the graph here at the top is information from a variety of studies on runet use, or in other words, internet usage across Russia. And though our data is not perfect, it does sync up to a degree with their metrics.  It's not a random sample, but looking at the data, it doesn't seem to skew bizarrely, and so at least you feel a bit better about the results. And we also wanted to compare it against known information in the hacker community. So we took data from the Chaos Construction's Hackaround Conference of 2007 where individual attendees could list their name, their group affiliation, and where they were coming from. And so taking that data, we broke it down into various categories, and you can see the graph here at the top is the chaos part. The bottom is our information, and again, it syncs up pretty well. So just using hacker metrics, it appears to match up to a degree. So in terms of location, much like education, we get some people giving extreme details, or at least some unusual examples. So Nate N8 says that he lives in Value Towns in Bobway, and that he only speaks Albanian. Kind of unusual. We get another guy, Arcanoid, who talks about going to St. Petersburg. He says that he lives in Moscow. To read his blog, he appears to reside in both places, or at least travel regularly. But just like education, we get some fake places, the Scientific Research Institute of Sorcery and Wizardry. Also, a handful listed Hogwarts. They're sometimes going to block that information. We also got some conflicting information in terms of gender. Given that there are male or female attributions for words and phrases in Russian, we were able to determine something about gender. And with Mr. Buggers, he's one of the founders of the BH crew. He says that he's female in his ICQ profiles, but everywhere else, he's male. His pronouns appear to be tied to male gender. So it's somewhat mixed. It's hard to really fair it out. Usually with a bit of sort of semantic analysis, you can sort of figure out pretty clearly and pretty quickly whether they're male or female. But this was very blurred. In terms of gender overall, the women in this sample were about 5% of the total community. So a very small slice of this group. The distribution and individual use of the blogs. Bob Nielken, for example, just talks about her day-to-day life. Coyote the One talks a lot about programming, but nothing in terms of hacking. And then Thirteen Ya just talks about her friends and what she's up to. One of the interesting side notes is, in discussing it with some people, is that females in the hacking community tend to have very strong skills and are very expert. And so actually probably more dangerous than the average male. And so to look at the sample, we only had one high-risk, one very skilled individual female named E7 who was really well connected to the rest of the network. These are some of the programs that she's released on her own site. There's a remote shell program, a couple of root kit task managers and different mechanisms in a network scanner. And she's had some other tools that are out there. This is a shot of her live journal page. She's got 521 people that she's looking at, 403 mutual friends. She's a member of 87 communities. She's very densely connected. And there's a picture of her that appears on a great deal of the individuals in this network site. So her face is very, very well known. In terms of live journal, now that we've done some of the demographic stuff, let's look at interest, because individuals can say exactly what they are fascinated by. And so the average number of interest, just as a sheer volume question, the Hell Nights crew, even though there's two members, they have the largest number of interest overall. Ruhack, that live journal based group only, is pretty high up there. Damage Lab, Mazafucka, not quite as high. In terms of interest by class, just as an average, our risk three guys talk about the whitest number of things. And that generally sort of coincides with the assumption that people in the high risk categories are generally curious about everything. And so they have a lot of interests, it makes sense. And so given the interests that are out there, we tried to see if we could predict individual risk based on the interests that they described. So we set it up initially with four categories. Malware, just grabbing all the various phrases that are used that fit into this category. Same with operating systems, drugs and alcohol, and assembly language. You're like the four essentials of hacking, right? So we figured maybe this would be useful. Given that we have such a small number of active individuals, we broke it down into zero, where we have no history of attacks or malware, passive actors, the bloggers, and then active actors. So zero, one, and two. And we try to predict across these three groups. And to look at the predictive analysis, malware is significant, operating systems are significant, but drugs and alcohol and assembly language are not. Damn. And so the higher threat level individuals in looking at this are very, very detailed about what they're interested in, and perhaps this might be part of it. The low level guys just talk about hacking computer security in these broad terms. The higher threat individuals get into botnets, buffer overflows, ring zero, et cetera. They give a wide range and very specific information. So taking interest in getting relatively little out of that, we thought, well, let's just make a broad computer hacking category. And looking at some interests in computer hacking, the Hell Nights crew, 100% hacker interest. Same with RUHAC, they have a pretty high percentage, but some of the other groups are much lower. And then interest in computer hacking by risk, you can see risk level three guys, that's primary importance. This makes very good sense. The higher the risk, the more interest in hacking, which makes perfect sense. Now, this is a sociogram. We figured we got this information. Let's try to map out what the relationships look like based on individuals who express an interest in hacking. And so the little dots up here are nodes, or specific individuals. The lines or the connections between these individuals. And the size of the dot reflects an individual's interest. And so the higher size dots or the bigger dots, like the two yellow ones at the bottom, are indicative of higher interest in computer hacking. The smaller dots, less interests. And you can sort of see that the groups also seem to cluster around the interest. So they're in clumps in various parts of the sociogram. So it's very good. We use some simple sort of free social networking software available in the Net, Patrick. Yeah, Patrick. It's actually pretty good stuff. You should go snag it. I think it's by something in the Czech Republic.  Polish? Yeah, Poland. So to look at this, what's really important to note when you examine sociograms, it's important to see where individuals lie. And what this graph suggests is that individuals are really, really well connected. There's nobody who's isolated. There's no one just sitting out on the periphery. There's lots of dense connections between individuals. So this suggests computer hacking is an interest that ties the group together in quite a number of ways. When we break it out by risk, again, there's density of connections. The green represents unknown or no risk. The yellow is no risk. The orange is moderate risk. And the red is our high-risk individuals. And you can see that there's sort of two clusters of high-risk individuals. And you can see that again and again. And we'll talk just a little bit more about it in a couple of slides or so. And so because of the density of connections, we can say that despite their level of risk, people know each other. People are examining one another. Now, taking all the different interests that are out there, what we wanted to try to do is refine our categories a little bit further. So we listed everything out into a table and then split it down into 46 different categories. So alcohol, animals and wildlife, art, et cetera. And determined low-risk and high-risk. So if somebody's talking about malware, if they're talking about specific forms of hacking, ring zero, et cetera, that's more of a high-risk indicator than say heavy metal music, drugs and Bluetooth. So low-risk and high-risk interests. So to look at low-risk interests across group, Haxona has a lot of low-risk interests. Ruhack, Hellnites, Mazafucka have much less. When we look at interest by risk level, we see that the risk level two guys have a healthy amount but not as much as our zero risk. And our risk three are much, much lower. The risk three high hazard guys are really interested more in hacking stuff, not necessarily music or other stuff. And then to look at high-risk categories, we see the opposite, which is what we would want to find. We see that the Hellnites crew have a pretty high number of high-risk interests. Mazafucka, Ruhack, a little bit better. But some of the groups do not. When we look at risk by risk, so high-risk interests by risk level, this is a good thing to see. We see a real high trend up on the high-risk individuals. Perfectly sensible. So again, here's another sociogram, size reflecting how many high-risk interests individuals have. And you can see that, again, there's a pretty dense series of connections, but our green guys, our no-risk folks, are very, very tightly coupled, less so with our low-risk and high-risk individuals. Makes sense when we talked about the high-risk people sort of being sort of a very interconnected, interconnected, strong, dense network. And to look at risk based on density of connection. So here we're seeing where individuals are in the network based on the connectivity and who they talk to and how. And what's important to note here is where our red individuals are, our high-risk players, and then the distribution of others throughout this network. And what you can sort of see is, once again, that sort of classic split for these high-risk individuals. There's one set in the center there that's sort of a very core, very tightly packed. They have a lot of connections with each other and also with some of the other hacking groups. And then there's this other set of guys that are sort of off to the left. And they're much more sort of isolated and not quite as connected. And that's very interesting. It's sort of an economy that's come out in our research. And you'll notice that the green or the no-risk are sort of distributed throughout, somewhat evenly. And so they're throughout the network, but our higher-risk individuals are more closely into the center. You've got a couple of clicks, sort of here and here, of individuals connected to one another. So that's risk. What we wanted to do next is just get a general sense of social ties. And so this is the entire network based, color-coded, bi-group membership. And so we have the BH crew here in the middle. We have Ruhack dead center, well-connected to a variety of groups. And then up at the top, we have the Damage Lab guys, some of them very uniquely tying others together. The Hell Nights crew are up here, and they're just, to a degree, they're on the periphery. But we see this clustering of Zloy, of Cup Zoo. So the network is relatively dense. Everybody is tied to a degree to one another. And what you can sort of see is it's very, very dense within each of the groups and much less dense across the groups, which is important. And density is going to tell us something perhaps about how information flows. So because there's a lot of redundant ties here, we might be able to explain why so many things get recycled perhaps. Right, and really important to look at these communications channels because you want to figure out how's an exploit being passed? Where's it going to go to next? How's the information being transmitted? It's going to be transmitted through these social bonds. So, I'm sorry, can you say that a little louder? We are trying to dig down a little bit further. We've just been running through some of the initial predictive and network graphs, so we haven't really dug down into why some of these actors are on to a degree the periphery, but we do want to look at that further. Yeah, like these two individuals here acting as primary connectors, yeah, that is relevant, but we also do see some ties to others. So we might argue in social networking research that these couple of individuals here are hubs that do link some groups together, but there are also outside relationships that do matter, so there might be certain individuals who are more prominent within these networks than others. And sort of key communicators across the groups. But we need to look at that a bit further. Beyond network actors, we wanted to look at the strength of ties across groups, so how often and in what way these groups look at one another. And so what you can sort of see, what's really interesting here is you can sort of see the thick lines indicate stronger ties, the thin lines indicate weaker ties among the different groups. And what you see is damage lab at the top, which is basically reasonably strongly tied to many, most of not all the other groups. And one of the things that's sort of interesting is if you're a pursuit and prosecution guy and you're having a tough time getting into one of the groups, what you do is, well, let's take a look at these ties. Let's see another group.   Everything on the left is a little bit less well connected than those on the right, and that's an important point. Couple of final slides in terms of popularity. These are mutual friends and the number of mutual friends that a person has. So the size of a node is indicative of how many mutual friends they have. So if you're big, you've got a lot of mutual friends. If you're small, you don't have as many. And so here, this is just by network group, and we're seeing in the middle here, the blue, we have Mazafucka, the reds, or the Ruhack individuals who are relatively dead center, and then Zloy over here. So we have some clustering. There are some individuals from the BH crew who are spread throughout the groups. We have the Damage Lab guys up here on the left. So there's some distribution that varies by group membership. And more important than anything else we think is where do these high risk actors sit and what's their popularity like? And so the color coding here, the fours are high risk people. You can see they're relatively centered in the middle of this network. They're tightly clustered, followed by the moderate risk individuals, although we have a couple on the periphery. And then the no risks are spread throughout. And once again, you sort of see that dichotomy for the high risk group, which is there's a central core sitting there, where in the center, and then these folks way sitting out on the side more isolated. So once again, the same evidence that points to persistent characteristic of the group. And so to look at this as a function of connectivity, the community itself appears to be looking toward these high risk people. They're well connected. They have pretty high levels of mutual friends, relatives, some of the others. And that's important because when we look at all of this data as a whole, we can make a couple of conclusions. We can make a few ideas about what this means for the broader hacker community, particularly this Russian community here. First and foremost, looking at blogs and social network sites can give you a massive amount of information, a lot to parse through. The problem, however, is that you've really got to drill down into this to actually find things of value, to find reasonable metrics in terms of risk and otherwise. Next, the Russian hacker community, just in terms of demographics within this population, appears to be in Moscow, St. Petersburg, a few other places, and the gender and education levels that we see, given there's not a lot of data about their higher institutions, et cetera. Perhaps these are individuals who've learned on their own.  Maybe they haven't, and the gender additionally, there's a small number of high threat hackers in this particular network, given that we're looking at a wide variety of communities. Here, perhaps this might give some support to the notion that, relatively speaking, there's a smaller number of high risk actors than there are low risk within this community, within the larger community. And the redundancy that we see, the heavy ties across a lot of these groups, might be an indication of why certain things filter throughout the community more heavily. Why there's multiple iterations of pinch and lots of different releases of the same tool. Recycling happens a lot. Primary innovation might not be as common in tools. So the configures, the storms, et cetera, are going to happen. But secondary innovation, taking the stuff that's older and tweaking it in a variety of different ways, might happen more heavily within this particular network. Finally, given the high centrality of the high risk individuals, the community knows who's important, knows who to look at, and they're getting some information from those high risk folks. The moderate risk people are on the periphery, but are to a degree looking at them as well. Our group membership information might not be as strong a predictor as we initially hoped for, given the significant value of, say, the RUHAC group. Which really comes to sort of an interesting observation, which is, years and years ago, the community was a very strong meritocracy, where your status really was very tightly tied to the skills that you had in networking, hacking, computer operating systems, things like that. And what we're sort of seeing is it's not quite the meritocracy it used to be, and we're seeing a lot less sort of, a lot of more variety of skill sets inside a particular hacking group, which is sort of a shift from years before. And if we're going to take anything away from this in terms of do interests matter, is there any kind of key indicators to help us get at high-risk folks? First and foremost, general interests aren't all that helpful. Talking about drugs, talking about beer, lots and lots of people are doing that. But very specific interests are to a degree helpful. They can give us some indicators of who's going to be significant. Belonging to multiple groups is also going to be an indicator of risk. And centrality, even if you don't know who is a high threat, looking at the networks as a whole might help to indicate who's important. Because that really addresses one of the really big problems, which is there are a lot of people out there to sift through. And if you have to sift through them all, you're not going to be able to do it. You really have to sort of prioritize and concentrate on what look like the highest probability targets. All of this, however, is tempered to a degree by some of the limitations that we have. This is just a portion of the actors that are out there, given the thousands of people who appear to be enrolled in all these different forums and websites that are out there. There might also be some self-selection, some bias here in terms of live journal. Also, the no-risk individuals or the zeros within this network might actually be high-risk players that are operating very, very well below the radar. Right, so that sort of big chunk that you saw, the 70% of these sort of no-risk unknowns, probably one or 2% of them are probably the nastiest folks there. And they're really under the radar, and there's no information. And so if you're operating on a site that's got robots.txt turned off and you're really heavily involved in some nasty stuff, we're not going to find you just through a basic Google search. The other point to make is that we really need to mine through this data even further. We want to try to improve our predictive model if we can at all, and we want to try to gather some more data, if possible. And the last thought I'd like to sort of leave you with is basically every human behavior leaves an evidence trail. You might not have the sensitivity to pick it up, but it's there. And if you look hard enough, you may find it. So with all of that having been said, if you've got any questions, if you've got any comments, you all are awesome for having come to this talk because we really, really appreciate it. We thank you all for showing up. If you have any questions, you can hit me on email.


Hi, I'm Peter Burris, and welcome to another Wikibon action item. Hi, once again, we're broadcasting from our beautiful Palo Alto Studios, theCUBE Studios, and this week we've got another great group, David Floyer in the studio with me, along with George Gilbert, and on the phone we've got Jim Cabellus and Ralph Finos. Hey guys. Hi there. So we're going to talk about something that's going to become a big issue. It's only now starting to emerge, and that is what will be the roadmap to automation. Automation is going to be absolutely crucial for the success of IT in the future and the success of any digital business. At its core, many people have presumed that automation was about reducing labor. So introducing software and other technologies were effectively able to substitute for administrative, operator, and related labor. And while that is absolutely a feature of what we're talking about, the bigger issue is ultimately is that we cannot conceive of more complex workloads that are capable of providing better customer experience, superior operations, all the other things that digital business ultimately wants to achieve if we don't have a capability for simplifying how those underlying resources get put together, configured, or organized, orchestrated, and ultimately sustained delivery of. So the other part of automation is to allow for much more work that can be performed on the same resources much faster. So that's the basis for how we think about plasticity and the ability to reconfigure resources very quickly. Now the challenges this industry, the IT industry, has always used standards as a weapon. We use standards as a basis of creating ecosystems or scale or mass for even something like mainframes where there weren't hundreds of millions of potential users, but IBM was successful at using that as a basis for driving their costs down and improving a superior product. That's clearly what Microsoft and Intel did many years ago, was achieve that kind of scale through the driving more and more and more, ultimately, volume of the technology and they won. Along the way though, each time, each generation has featured a significant amount of competition at how those interfaces came together and how they worked. And this is going to be the mother of all standard oriented competition. How does one automation framework and another automation framework fit together? One, being able to create value in a way that serves another automation framework, but ultimately as a, for many companies, a way of creating more scale onto their platform, more volume onto that platform. So this notion of how automation is going to evolve is going to be crucially important. David Floyer, is API, are APIs going to be enough to solve this problem? No, as a short answer to that. This is a very complex problem and I think it's worthwhile spending a minute just on what are the component parts that need to be brought together. We're going to have a multi-cloud environment, multiple private clouds, multiple public clouds and they've got to work together in some way. And the automation is about, and you've got the edge as well. So you've got a huge amount of data all across all of these different areas. And automation and orchestration across that, as you said, not just about efficiency, they're about making it work, making it able to be to work and to be available. So all of the issues of availability, of security, of compliance, all of these difficult issues are subject to getting this whole environment to be able to work together through a set of APIs, yes, but a lot, lot more than that. And in particular, when you think about it, to me, volume of data is critical, is who has access to that data? And- Now why is that? Because if you're dealing with AI and you're dealing with any form of automation like this, the more data you have, the better your models are. And if you can increase that amount of data as Google show every day, you will maintain that handle on or that control over that. So you said something really important because the implied assumption, and obviously it's a major feature of what's going on, is that we've been talking about doing more automation for a long time. But what's different this time is the availability of AI and machine learning, for example, as a basis for recognizing patterns, taking remedial action or taking predictive action to avoid the need for remedial action. And it's the availability of that data that's going to improve the quality of those models. Now, George, you've done a lot of work around this and the whole notion of ML for ITOM. What are the kind of different approaches? If there's two ways that we're looking at it right now, what are the two ways? So at two ends of the extreme, one is I want to see end to end what's going on across my private cloud or clouds, as well as if I have different applications in different public clouds. But that's very difficult. You get end to end visibility, but you have to relax a lot of assumptions about what's where. And that's called the breadth first. So the pro is end to end visibility. Con is you don't know how all the pieces fit together quite as well. So you get less fidelity in terms of diagnosing root causes. So you're trying to optimize at a macro level while recognizing you can't optimize at a micro level. Right. Now the other approach, the other end of the spectrum is depth first, where you constrain the set of workloads and services that you're building and that you know about and how they fit together. And then the models based on the data you collect there can become so rich that you have very, very high fidelity root cause determination, which allows you to do very precise recommendations or even automated remediation. What we haven't figured out how to do yet is marry the depth first with the breadth first so that you have multiple focus depth first. That's very tricky. Now, if you think about how the industry has evolved, there's, we wrote some stuff about what we call, what I call the iron triangle, which is basically a very tight relationship between specialists in technology. So the people who were responsible for a particular asset, be it storage or the system of the network. The vendors who provided a lot of the knowledge about how that worked and therefore made that specialist more or less successful and competent. And then the automation technology that that vendor ultimately provided. Now that was not automation technology that was associated with AI or anything along those lines. It was kind of out of the box, buy our tool and this is how you're going to automate various workflows or scripts or whatever else it might be. And every effort to try to break that has been met with screaming because, well, you're now breaking my automation routines. But so the depth first approach, even without ML, has been the way that we've done it historically. But David, you're talking about something different. It's the availability of the data that starts to change that. So are we going to start seeing new compacts put in place between users and vendors and OEMs and a lot of these other folks? And it sounds like it's going to be about access to the data. Absolutely. So you're going to start at the bottom. You've got people who have a particular component, whatever that component is. It might be storage, it might be networking, whatever that component is. They have products in that area which will be collecting data and they will need for their particular area to provide a degree of automation like a degree of capability. They need to do two things. They need to do that optimization and also provide data to other people. So they have to have an OEM agreement not just for the equipment that they provide but for the data that they are going to give and the data they're going to give back. The anonymization of the data, for example, going up and the availability of data to help themselves. So contracts effectively mean that you're going to have to negotiate value capture on the data side as well as revenue side. Absolutely. The ability to do contracting historically has been around individual products and so we're pretty good at that. So we can say you will buy this product, I'm delivering you the value and the utility of that product is up to you. When we start going to service contracts we get a little bit different kind of an arrangement. Now it's an ongoing continuous delivery but for the most part a lot of those service contracts have been predicated to known in advanced classes of function like Salesforce for example or the SaaS business where you're able to write a contract that says over time you will have access to the service. When we start talking about some of this automation though now we're talking about ongoing but highly bespoke and potentially highly divergent over a relatively short period of time that you have a hard time writing contracts that will prescribe the range of behaviors and the promise about how those behaviors are actually going to perform. I don't think we're there yet. What do you guys think? No way, I mean. Especially when we think about real time. Yeah, it has to be real time to get to the end point of automating the actual reply, the actual action that you take. That's where you have to get to. You can't, it won't be sufficient in real time. I think it's a very interesting area, this contracts area. If you think about solutions for it I would be going straight towards blockchain type architectures and dynamic blockchain contracts that would have to be put in. They're not real time. The contracts aren't real time. The contracts will never be real time but the access to the data and the understanding of what data is required those will be real time. Yeah, we'll see. I mean, Ethereum's what, every 12 seconds everything gets updated. To me that's good enough. That's real time enough. It's not going to solve the problem of somebody. It's not going to solve the problem of the edge. At the very edge. But it's certainly sufficient to solve the problem of contracts. Okay. And I would add to that and say in addition to having all this data available let's go back like 10, 20 years and look at Cisco.  And they might not expose APIs in a way that would sort of make it common across their competitors. But if you had data from them and a constrained number of other providers for around which you would build, let's say these modern big data applications. It's, if you constrain the problem you can get to the depth first. Yeah, but Cisco is a great example. It's an archetype for what I said earlier that notion of an iron triangle. You had Cisco admins that were certified to run Cisco gear and therefore had a strong incentive to ensure that more Cisco gear was purchased utilizing a Cisco command line interface that did incorporate a fair amount of automation for that Cisco gear. And it was almost impossible for a lot of companies to penetrate that tight arrangement between the Cisco admin that was certified, the Cisco gear and the CLI. And the exact same thing happened with Oracle. The Oracle admin skill set was pervasive within large enterprises. It happened with everybody. But there's a difference. The only reason it didn't happen in the IBM mainframe, David, was because of a, well, but it did happen, but governments stepped in and said, this violates anti-trust. And IBM was forced by law, by court decree, to open up those interfaces. That's true. Are we going to see the same type of thing? I think it's very interesting to see the shape of this market when we look a little bit ahead. You're going to, people like Amazon are going to have IAS, they're going to be running applications. They are going to go for the depth way of doing things across or which way around, is it? The breadth. They're going to be as broad as the end. But they will go depths in individual components. Oh, sure. But they will put together their own type of things for their services. Equally, other players like Dell, for example, have a lot of different products, a lot of different components in a lot of different areas. They have to go piece by piece and put together a consortium of supplies to them, storage suppliers, chip suppliers, and put together that outside. And it's going to have to be a different type of solution that they put together. HP will have the same issue there. And there's people like CA, for example, we will see an opportunity for them to come in again with great products and overlooking the whole of all of this data coming in. Oh, sure. Absolutely. So there's a lot of players who could be in this area, Microsoft, I missed out. Of course, they will have the two ends that they can combine together. They may have an advantage that nobody else has because they're strong in both places. But Jim Cabilis, let me check, are you there now? Do we got Jim back? Can you hear me? I can barely hear you Jim. Can we bring Jim's volume up a little bit? So Jim, I asked a question earlier about, we have the tooling for AI. We know how to build, we know how to get data, how to build the models, and how to apply the models in a broad brush way. And we're certainly starting to see that happen within the IT operations management world, the ITOM world. But we don't yet know how we're going to write these contracts that are capable of better anticipating, putting in place a regime that really describes what are the limits of data sharing, what are the limits of derivative use, et cetera. I argued, and here in the studio, we generally agreed, that we still haven't figured that out and that this is going to be one of the places where the tension between, at least in the B2B world, data availability and derivative use and where you capture value and where those profit pools go, is going to be significant. But I want to get your take. Has the AI community started figuring out how we're going to contractually handle obligations around data, data use, data sharing, data derivative use? Short answer is no, they have not. The longer answer is that, can you hear me first of all? Barely. Okay, can I keep talking? Okay, the short answer is no, that the AI community has not addressed those, those IP protection issues. But there is a growing push in the AI community to leverage blockchain for such requirements in terms of blockchains to store smart contracts were related to downstream utilization of data and derivative models. But that's extraordinarily early on in its development in terms of insight in the AI community and in the blockchain community as well. In other words, in fact, one of the posts that I'm working on right now is looking at a company called 8Base that's actually using blockchain to store all of those assets, those artifacts, for the development life cycle along with the smart contracts to drive those downstream uses. So what I'm saying is that there's lots of smart people like yourselves are thinking about these problems, but there's no consensus definitely in the AI community for how to manage all those rights downstream. All right, so very quickly, Ralph Finos, if you're there, I want to get your perspective on what this means from markets, market leadership. What do you think? What's, how is this going to impact who are the leaders? Who's likely to continue to grow and gain even more strength? What are your thoughts on this? Yeah, I think my perspective on this thing in the near term is to focus on simplification and to focus on debt because you can get return, you can get payback for that kind of work and it simplifies the overall picture. So when you're going broad, you've got less of a problem to deal with to link all these things together. So I'm going to go with the shaker kind of perspective on the world is to make things simple and to focus there. And I think the complexity of what we're talking about for breadth is too difficult to handle at this point in time. I don't see it happening anytime in the near future. Although there are some companies like Splunk, for example, that are doing a decent job of presenting a more of a breadth approach, but they're not going deep into the various elements. So George, really quick, last talk to you. Disagree on that one. They built a platform originally that was breadth first. They built all these essentially forwarders which could understand the formats of the output of all sorts of different devices and services. But then they started building what they call curated experiences which is the equivalent of what we call depth first. They're doing it for IT service management. They're doing it for what's called user behavior analytics which is, it's a way of tracking sort of bad actors or bad devices on a network. And they're going to be pumping out more of those. What's not clear yet is how they're going to integrate those so that like IT service management understands security and vice versa. And I think that's one of the key things, George, is that ultimately the real question will be, or not the real question, but when we think about the roadmap, it's probably that security is going to be early on one of the things that gets addressed here. And again, it's not just security from a perimeter standpoint. Some people are calling it a software based perimeter. Our perspective is the data is going to go everywhere and ultimately how do you sustain a zero trust world where you know your data is going to be out in the clear. So what are you going to do about it? All right, so look, let's wrap this one up. Jim Kubilis, let's give you the first action item. Jim, action item. Action item, well, action item automation is just to follow the sort of the stack of assets that drive automation and figure out your, your overall sharing architecture for sharing out these assets. I think the core asset will remain orchestration models. I don't think predictive models and AI are a huge piece of the overall automation pie in terms of the logic. So just focus on building out and protecting and sharing and reusing your orchestration models. Those are critically important in any domain end to end or in specific automation domains. David Floyer, action item. So my action item is to acknowledge that the world of building your own automation yourself around a whole lot of piece parts that you've put together are over. You won't have access to a sufficient data. So enterprises must take a broad view of getting data, of getting components that have data, be giving them data, make contracts with people to give them data, masking or whatever it is, and become part of a broader scheme that will allow them to meet the automation requirements of the 21st century. Ralph Finos, action item. Yeah, again, I would reiterate the importance of keeping it simple, taking care of the depth questions and moving forward from there. The complexity is enormous. George Gilbert, action item. I say start with what customers always start with a new technology, which is a constrained environment like a pilot. And there's two areas that are potentially high return. One is big data where it's been a multi-vendor or multi-vendor component mix and a mess. And so you take that and you constrain that and make that a depth first approach in the cloud where there is data to manage that. And the second one is security where we have now more and more trained applications just for that. I say don't start with a platform, start with those solutions, and then start adding more solutions around that. All right, great. So here's our overall action item. The question of automation, the roadmap to automation is crucial for multiple reasons, but one of the most important ones is it's inconceivable to us to envision how a business can institute even more complex applications if we don't have a way of improving the degree of automation on the underlying infrastructure. How this is going to play out, we're not exactly sure, but we do think that there are a few principles that are going to be important that users have to focus on. Number one is data. Be very clear that there is value in your data, both to you as well as your suppliers. And as you think about writing contracts, don't write contracts that are focused on a product, now focus on even that product as a service over time where you are sharing data back and forth in addition to getting some return out of whatever assets you've put in place. And make sure that the negotiations specifically acknowledge the value of that data to your suppliers as well. Number two, that there is certainly going to be a scale here. There's certainly going to be a volume question here. And as we think about where a lot of the new approaches to doing these or this notion of automation is going to come out of the cloud vendors. Once again, the cloud vendors are articulating what the overall model is going to look like, what that cloud experience is going to look like. And then it's going to be on a challenge to other suppliers who are providing that on-premises, true private cloud and edge orientation where the data must live sometimes. It's not something that they just want to do because they want to do it because that data requires it to be able to reflect that cloud operating model and expect ultimately that your suppliers also are going to have to have very clear contractual relations with the cloud players and each other for how that data gets shared. Ultimately, however, we think it's crucially important that any CIO recognize that the existing environment that they have right now is not converged. The existing environment today remains operators, suppliers of technology and suppliers of automation capabilities. And breaking that up is going to be crucial not only to achieving automation objectives, but to achieve and converged infrastructure, hyper-converged infrastructure, multi-cloud arrangements, including private cloud, true private cloud and the cloud itself. And this is going to be a management challenge. It goes way beyond just products and technology to actually incorporating how you think about your shop being organized, how you institutionalize the work that the business requires and therefore what you identify as the tasks that will be first to be automated. Our expectation, security is going to be early on. Why? Because your CEO and your board of directors are going to demand it. So think about how automation can be improved and enhanced through a security lens but do so in a way that ensures that over time you can bring new capabilities on when a depth first approached at least to the breadth that you need within your shop and within your business, your digital business to achieve the success and results that you want. Okay, once again I want to thank David Floyer and George Gilbert here in the studio with us on the phone Ralph Finos and Jim Cabellas. Couldn't get Neil Raiden in today, sorry Neil. And this, and I am Peter Burns and this has been an action item. Talk to you again soon.