Hello everyone. Yeah, thanks for, thanks for making it to this talk. So my name is Adrian. I'm the head of Family Serving at Zeldin. What I do there is basically build tools that let people serve and deploy machine learning models easily. However, before that most of my career has been spent on classic software engineering, so more of like your normal, like back end and front end engineering, which is why something that has always called out a lot to my attention is how little time we spend talking about security in machine learning. And up to the point where if you check out most of the, some of the most common frameworks, you see key challenges being recommended as best practices. Today we're going to be talking about some of those and as well as some ways to mitigate it. As you can see, so basically we're going to focus today on some of the risks introduced by machine learning artifacts. So for those of you who may not be familiar with it, pickle is Python's native serialization format. So basically anything in Python can be serialized into a pickle object. Functions in Python are also objects, which means they can also be serialized in a pickle. And basically this makes for a really powerful tool, which is why most frameworks, machine learning frameworks, use that. And when I say most of them is pretty much not all of them, but some of the most common ones, definitely. So here we see three examples of usage of pickle on the wild. So cycle learn, very common framework, maybe not the most powerful of them, but it's one where, for example, pickles are recommended as the way to go. They even acknowledge that this is not great. And we'll see why this is not great in a second. But it's what they recommend. There's no current alternative, official alternative to that. And we even see, like, more, I guess, mature frameworks also doing the same thing. So how we see an example for Keras, showing how it also uses pickles under the hood, and then we also see an example with Torch, seeing how it also uses pickles under the hood. Now, as I said before, pickles can't realize any Python object, functions are also objects in Python, so they can also realize that. Because of how pickles are designed, the way to load that back into the runtime is basically by interpreting that code. And probably now, many alarms will start to sound in your head, and if not, they should. We're talking about serializing anything, including code, and then running it when we unload it. So any kind of arbitrary code executions would fall into this category. And we'll see an example more in detail about how this would apply to machine learning workloads in a second. But yeah, basically this talk is focused on this nascent field of ML set cobs, which is nothing more than the extension of standard Dev set cobs to include machine learning workloads. Last year, we presented a talk here in KubeCon where we basically covered some of the top 10 ML set cobs vulnerabilities. This list of top 10 vulnerabilities is basically something that the LFAI started working on a year ago, so a year and a half ago, as part of a working group that they set up on ML set cobs. And basically what the goal here was to sort of publish something similar to the OWASP top 10 for web vulnerabilities for people to take into account when they design and build machine learning systems. Since last year, well actually just to also contextualize this talk today on this list, today we're just going to focus on number three there, so artifact, exploit, injection. If you want to know more about the other ones, you can check out the talk from last year. But compared to last year, some good news have happened. Now it's not just the LFAI, basically looking into the ML set cobs problem. We also have other major organizations also looking into this. So OWASP, for example, now has already published a top 10 for LMS. And I think they were also publishing like a more general ML ops top 10 security vulnerabilities as well. MITRE, which is also like a massive name in the security world, has also published now a catalog of attacks into machine learning systems, attack vectors. It is true that some of these are basically just arrived from the fact that machine learning systems, MLOP systems are nothing else than software systems. So they have the same sort of vulnerabilities. However, the ones that basically don't have like the red symbol next to it are specific to machine learning workloads. So there is definitely something specific here that we should take into account that maybe at the moment we are not. Alongside that catalog, MITRE has, a few months ago, also published a list of mitigations of how to address some of those attack vectors. So today we're going to focus on the security aspects introduced by pickles and more widely machine learning artifacts. That's not to say that besides the security issues, pickles also suffer from way older issues that should make us reconsider whether they are the right tool for the job. So on one hand, they are super sensitive to the version of Python. An artifact that has been serialized with pickle with Python 3.8 probably won't work with Python with something like a server running Python 3.10. Likewise, frameworks are also very sensitive to versioning. So it's not just the version of Python, it's also the version of the framework. So for example here, like you can see an example of a stack trace, showing what happens when you try to load a pickle generated with a different version of scikit-learn using a different version, like a newer version of scikit-learn. And basically it's just gibberish. It's really hard to travel through these kind of issues. So as you can see, I love pickles. I haven't wasted any time at all working with pickles. And you just love them too. They're great. So what we're going to see now is how pickles, how these problems that pickles expose can impact your machine learning workloads. Before doing that, though, basically to make sure that we're all on the same page. When we talk about an MLOPS system, generally we will have something like this. So basically you have, this is like, I guess, drawn in the axis of the model lifecycle. Generally you start with some training data. With that training data you have either data scientists or like CI pipelines, building artifacts, building models, training models out of that data that they can get serialized into artifacts. These artifacts then make it into an artifact store and eventually make their way into your serving infrastructure, where most likely they will be loaded by a microservice and which will load that model back and then run in front of that. If we go into the microservice, what generally happens is you have some kind of docker container loading that pickle artifact and then exposing some kind of real-time API to run inference. Something worth clarifying as well. In this diagram, so you can see some logos, this is vendor agnostic. Like this same sort of architecture, you would see it with any kind of vendor with any kind of tooling that you choose. So you can ignore those logos. So with that in mind, let's see an example now of how basically a malicious pickle, a poison pickle could impact you when you load it on that microservice. So we now go to this notebook. What we're going to do here is first we're going to train a simple circular model. So we do that. We then save it, sterilize it. In this case, you can see that we are sterilizing it using a library called jublib. Jublib basically uses pickle under the hood. So it's pretty much the same thing. Now, if we look into what jublib, what that pickle artifact has, is you can see some references here to Python code, but it's basically something where it would be very hard to know if something is weird or is bad. What we're going to do next is we're going to serve it. So in our case, we're going to load it with ML server, which is basically an inference server that's very easy to deploy as a microservice. So I've already had this running in the background. So I already have my ML server running in my model. So if we now run inference, you can see that it just returns. So we send some data and we got some data back. Great. Now we're going to poison it. So in this case, what we're going to do is basically tweak some of the Python internals to make sure that when we serialize this, what we do is the code that we generate instead is the equivalent of dumping the environment variables into a file. So we do this. And now if we look at the artifact, it's still giveries. So same as before. In this case, it's quite rough. But you could definitely build something more subtle that would be even harder to detect. So now we're going to try to load it back. So we have modified our artifact. We tried to reload it. Everything seems fine. But now if we take out this file, we can basically say that our malicious artifact has now dumped our whole environment. So pickles may not be great. What we have seen is, on one hand, how easy it is to poison a pickle artifact. On the other hand, how hard it is to detect that it has been poisoned. What do we do instead? So one option is to see if we can generate, let's say, higher quality pickles. So how do we do that? On one hand, we have tools like scopes that basically try to mitigate some of the risk introduced by pickle by removing for the generated artifact all the extra functionality that pickle provides, all the extra power that pickle provides, and that can cause troubles. It doesn't remove all of them, though, because then basically tools like scikit-learn wouldn't work. Well, the option is just, okay, let's not use pickle. Let's use something like Onyx. So Onyx is like a more descriptive format to serialize machine learning models, which don't involve loading and running arbitrary code. If we have a look back at our infrastructure, what we have now is basically the same sort of thing, just that now we have cooler pickles. Great. Is that the end of it, though? There is, so basically the question now is, okay, we have now a, let's assume that scopes is great. Let's assume that it removes all the risks, or let's assume that we're using Onyx. We have now artifacts that are saved from the DevSecOps point of view. They're not going to run arbitrary code. They're not going to dump all of our data. But are they saved also from the MLSecOps point of view? What we see here is an example. It's an example attack that was done by researchers at a company called Mystery Security, where they basically surgically modified an LLM to spread misinformation. I think in this case, what they took was that the answer to the question of who was the first person in the moon was Yuri Gagarin, instead of Neil Armstrong. They pushed that to Hagenphase, to Hagenphase Hub, and as they expected, many people basically just downloaded that model and started using it. Because that model from most angles is correct, it's very hard to detect if it has been tweaked that way. And so in this case, basically this attack depended on people trusting that a model uploaded by someone called Mystery Security was saved. But basically what it says is how easy it is to tweak an artifact to make it insecure, even though it's completely safe from a DepsecOps perspective. So what do we do instead? Basically what we have seen is that it's very hard to detect if an artifact can be harmful. So what we need instead is trust or discard mechanisms throughout the whole machine learning model lifecycle. This is also lined with basically the way that MITRE suggests on how to mitigate this sort of problem. So if we look back at our infrastructure diagram, we want, okay, let's just use cooler people, that's fine, but let's just avoid that they get tampered throughout their model lifecycle. And you may say, well, let's just sign the artifacts and let's just check the signature at deployment time and that's it, right? Problem then comes with who then validates that the signature that you create is for your artifact is correct. And then who validates that the signature for the signature of the artifact is also correct. And so on and so forth. So you basically have a recursion problem. If you check out the like DepsecOps literature, this is basically now in us like always basically refers like turtles all the way down. Because in theory, there's this legend that the world is held by a turtle and a turtle can't just stay in singer. So it probably stays on top of another bigger turtle and so on. How do we solve for that? We basically look back to the DevOps literature. As with many other problems in MLOs, it has already been solved in the DevOps world. The question is how to adapt that solution to our use case. So if we look back at DepsecOps, best practices, we will see a lot of resources around supply and security. If you check out, for example, the CMTF landscape, you will see tons of projects that are related to supply and security. So let's just look back at that and let's see how we can apply it to MLOs. So if we look at any supply and security guide, what we will see is that any process, any supply and process has three main components. It has artifacts, it has metadata about those artifacts and it has attestations that verify and validate that that metadata and those artifacts are to be trusted. And then it has policies that verify that those attestations are correct. If we look at how that applies to our use case, artifacts are obvious. So artifacts will be our machine learning models. Be that pickles, be that onyx, whatever it is. Metadata then generally folds into one of three categories. So on one hand you have provenance data. This is all, these three categories is basically what is defined by classic DepsecOps literature. But if we follow that guide, we have on one hand provenance data. Provenance data, in our case, could be things like who trained this model? Or when did they train it? Or what training pipeline did they use? Software buildup materials, in our case, could be things like what dataset was used? Or what package dependencies does the model have? For the first one, for provenance data, there is no sort of a standard agreed on what that should include in machine learning workloads. For software buildup materials, though, there is good news. There are like a couple of working groups looking into this. One of them is Cyclone DX, the other one is SPDX, that are trying to decide to agree on a standard on how this should be extended to machine learning workloads. And then lastly, we could also have vulnerabilities kind of reports. So maybe our model depends on Cyclone learn 1.0 and we know it has a vulnerability, so we just acknowledge it and we just acknowledge and mention that it may not be harmful to our use case. And then lastly, we have attestations. Attestations would be the signatures that would come, that would guarantee that we can trust this metadata and this artifact. So how do we do these attestations? So we can basically just rely on existing projects. One of them, one of the very popular tool in the DevOps world is Sixter. Sixter lets you generate signatures and validate them on runtime in a secure way. Sixter is actually like a whole suite of projects, of products, sorry, projects. So one of them is Fools here. Fools is basically a free certificate authority, so it would be the one generating your artifact. You would then have record. Record is a ledger where every signature that you generate goes to. And basically this is how Sixter goes around this recurrency problem, this recursiveness problem. Things generated with Fools here, we go into record and then at runtime you can basically check that this signature exists in record, so you can trust it. Every signature that Sixter generates goes validated against an OIDC gateway, against an identity coming from the OIDC gateway. That way you also, you can see this as a sort of metadata that really comes for free telling you who created that model. This is quite complex, quite a lot of moving pieces. Good thing is that Sixter actually provides many hosted versions of this which are good enough for tests and for demos and to just try it out, which is basically what we're going to do now. But before that, just to review how our goal system would look like, we would keep the same sort of infrastructure where we generate our Fools, we just lock them to make sure they don't get modified with the only difference that now the signature gets generated by Sixter and we do Sixter to validate it at runtime at certain time. So let's see how that would look like in practice. So if we extend our previous example, what we're going to do now is we're going to train again an artifact, we can just train that, we're going to save it and what we're going to do now is sign it. It's one of the main changes that we will do to compare to the previous process, to the naive process. So we use Sixter, Sixter also comes with some CLIs, some libraries that we can just use. We sign it and what should happen now, if Wi-Fi works, is that it will take me to the OIDC gateway that Sixter hosts, which is good enough for testing. Generally in production, you would obviously just link this with your internal OIDC gateway. So we just tell it, okay, use my Google Authentic identity, cool, it's all done. And if we look at the files that were generated, on one hand we have the same model that we had before on the naive case, but now we also have some signatures that go along with it. We now try to validate it, to verify it. As you can see here, I don't just want to verify that the model hasn't been tampered, I want to validate that it was created by myself. So we verify it, it's all good. Now let's try to modify it. So we're going to do something very similar to what we did before, and we're going to tweak that, and we're going to now try to verify it. And as we expect, it just fails saying that the signature is invalid because we modified that file so the signature is no longer valid. So that's all working locally. How would this work once we deploy it in our serving infrastructure? So what we're going to do is basically tweak our server, we're going to extend one of the runtimes that it comes with out of the box to verify that. And to do that, we're just going to use the sixter SDK to verify that. And we're not going to dive too much into the code, you can check these resources later, but basically the key thing is that we just want here to verify it's all good. So basically something doing something similar to what the CLI does. So that's already running in our server. So let's try it out. Let's first list the models that are available so we can see three of them. We have first the tamper model that we just modified. Sorry. We also have the naive model which was the case that we saw before without any kind of signatures. And then we had the good model, which is the model that works as expected and has signatures to verify it. So let's try first loading the good model. It all works well, signature passes, so it's all good. Let's now try to load the naive model. This is the same model that we saw before. So as expected, it still dumps this bound txt file with all of our environment. And last, we're just going to try to load the tamper model that includes the sixter signatures. So we try to load it and as we expect, it just fails with this verification error. Signature is now invalid. And we can verify that the bound txt file didn't get created. So this is good enough for a demo. Unfortunately though, if we were to take this to a full production case, there will be more things to work on. So basically, one of the key points from this talk would be that there is still a lot of work to do. So on one hand, there are no many serving vendors that currently have provided the ability to apply policies. There are some, like Mr. Security, which are working on it. So basically, a policy would be verifying this automatically every time we serve a model. There are also very few integrations we saw as projects that generate the signatures or generate this metadata out of the box. So it would be great if people like TensorFlow basically tried to promote these sort of best practices. And we still see frameworks like Cycle Learn, which don't provide the new alternatives to pickle. So definitely, there is still work to do. But yeah, if you are interested in joining the discussion or in learning more, you can always join the ML SecOps working group, which is part of the Linux Foundation for AI and data. It joins monthly. So everyone is welcome to attend. And yeah, I hope you enjoyed this talk. I hope you find the content useful and thank you very much. We have about five minutes for questions. There's a mic in the middle of the room. Feel free to walk up to the mic if you have questions. I can also pass the mic around. Great talk. Just a quick question. If a model were being refined or retrained, would you need a transit set of signatures or, like, attestations? Like, how did that work? Have you thought about that? Sure. If you were to retrain a model, you would end up with a new artifact that you would need to sign again. Generally, like, this retraining would happen in some sort of pipeline. So basically, what you want to verify is that the artifact was created by a component that you trust. And this is sort of like the structure of this process. Like, in general, you want signatures and you want supply chain on everything. So the Kubernetes manifest that are deploying that pipeline running with hardware workflow, those also need, in theory, to be verified with a supply chain process. There are a lot of tools like Kiverna that basically provide you that. But basically, you want everything that happens in your cluster to be verified. Would you need, like, the starting model or, like, the initial model to be verified as well? So, like, you're, like, this new model is derived from this old model and so on, right? So it seems like you might want to transit an attestation. Yeah, sir. If you're fine-tuning, yeah, if you're fine-tuning an existing model, then, yeah, for sure. Yeah, you would need to trust that as well, because otherwise, you couldn't end up with a case, like, the use case that Mitter Security looked at. Like, you're maybe pulling a model from Hagen-Faith Hub and you don't know what is inside the artifact. Like, this problem with pickles seems general to, like, protocol buffers and, like, any other type of, like, serialization format. So I'm curious if you know, like, what other folks do for, like, just, like, storing, like, documents in, like, serialized format and so on. Well, I mean... There's, like, a similar type of thing that's going on. Yeah. Oh, I mean, the problems with pickles are, like, some of those are very specific to pickles. Like, generally with documents, you don't expect that you can run arbitrary code executions. We used to, like, BDF had some vulnerabilities like that in the past. But, yeah, it's something that can affect, in general, to anything you run and you load on the cluster. And we saw that recently as well with, like, the look for vulnerability. Every time you accept input from the outside, you have that risk. And there are different ways to mitigate that. Sometimes it will be, you will be able to do, to have scanning tools in place. Sometimes you will need to have this trust or discard mechanism. Generally, you want all of them. You want to add as many security layers as you want, as you can. Thanks again for the great presentation. Thank you.


And I am author of DataSplite, which is an open source intelligence framework. And I also run a DefCon village, which talks about reconnaissance and open source intelligence again. I've been in the conference circuit for a while, and here to share some of my experience around open source intelligence. Directly jumping to the agenda of the talk, we're gonna discuss, first we will discuss what is open source intelligence, and why security teams should use it. Following that we will discuss how you should do continuous discovery of assets and monitor them, and then use those assets for periodic attack simulation, how you can discover sensitive information, how should you monitor breach passwords of your account as well as your employees, and how can you use ocean proactively including social media, where you can check for reputation of your organization, et cetera, and then ultimately we will discuss what are the countermeasures, which you should think about. Before we start discussing that, a few terms, whenever I say brute force, I basically means using a trial and error method for obtaining information such as username, password, a bucket name, OTP code or anything for that matter. Whenever I use black box, I mean that we are referring to a system where we have no information about the system. When we say white box, we believe that we have all the information about the system. So before we do any kind of engagement or exercise on organization, we have all the access about them, and then gray box, which is kind of a mix. So probably assume you have passwords or maybe some access control policies, but not all the information. Whenever I use the word patch, it basically means phishing security vulnerabilities and other bugs, it could be functional bugs as well. So what is open source intelligence? Basically it is collecting information from internet, analyzing them, applying intelligence on top of it, and then getting actionable intelligence out of it. So if you just want to have a representation of that, find raw information, apply intelligence on top of it, analyze it, correlate that information, and then you have something for which you can use for taking some actions. And why we should do OSINT? Because nowadays when you look, everyone has a smartphone, everyone loves uploading pictures on Facebook, especially your boarding passes when you're boarding a flight, which have a QR code, which can ultimately result in leaking information. Now this is one of my favorite tweets on Twitter, where a government person tweeted just to show off what kind of things he's up to, and the screen clearly shows the ID card which he's holding, using which a physical security person can actually clone an ID card. This is also showing some call lists over here. I don't know if I can zoom here. Yeah, I can. So there are some call lists. You can figure out that they are using a Windows box, and IE Chrome and Skype are already installed on the machine. You can figure out, there are some predictable passwords for the Wi-Fi, and the camera placement, if in case someone is interested in doing something nasty to this organization. And this is just an example, you know what kind of information we upload on internet without even realizing that information is going out. For example, if you have a smart DSLR, you take a pic and upload it on some social media, you don't realize that it actually appends your geolocation information with that, unless the social media is stripping that information, anyone can figure out where you actually clicked that photo, okay? And now if it is a photo from your trips, it's all right, but if you are just clicking a selfie at your home, you are revealing your home address. And that is why OSL is important. Now why security teams, defensive teams should do open source intelligence because hackers do, right? And because hackers can do this and use this information to attack you, as a security team, you should also do open source intelligence and be ahead of hackers in the game. Few more reasons because people leak sensitive information on code aggregators like GitHub, Bitbucket, there are so many untracked assets which are running easy targets. Most of them are legacy boxes like a.abc.com, example.abc.com, and most common backup.example.com. Also, when every organization is moving to cloud, we have seen a lot of cases where people spawn boxes with private IP addresses, but they don't realize there's a public IP attached to that box as well. And eventually when you run, you find out the list of assets, you realize you have so many IP addresses lying out in public, which are not even required, and they're running 3306 port open, inviting attackers to gain access to your database, but again, they don't have passwords, and you can get passwords a lot of time from GitHub, Bitbucket, and such code aggregators, which we just talked about. Also, with open source intelligence, attacks become very targeted. If I know you are more interested in football and cycling, the kind of dictionary lists I will make for launching a brute force attack will have those keywords in them instead of running a one million passwords list for attacking you. Also, employees use breach passwords in corporate accounts. This is in line with the policy of using same password everywhere, and people love that. And that is something hackers always keep an eye on. Also, when you have full patch system, let's say you have all the latest patches installed on your machines, a credential leak can actually defeat all of them and still allow the attacker to get inside your network or your machines. There are a few case studies where people have found information about companies on internet, and then they have made a lot of money because bugs were really critical. They have found Slack tokens, API tokens of organizations. Tesla has a bunch of their AWS keys leaking out, which attackers used for logging into their AWS account, spawning institutions and doing some crypto mining. So things like that are pretty common nowadays, especially when everyone is moving to cloud environments, these kind of issues are coming in. And that's the reason security teams should really, really do open source intelligence in their cycle. Now the question is, how do you do that? First thing which you should do is have a continuous discovery process where you continuously discover your assets and monitor them, have a clear number of how many assets do you actually own. There should be periodic attack simulation using open source intelligence again. For example, if you have found 50 passwords related to your organization and you have 30 domains or subdomains, you should actually spread these credentials on your domains or subdomains so that you know if any of them is working or not. Because if that is working, it's a critical issue. You should also monitor breach passwords. You should have a periodic check on discovering sensitive information. Now this could be automated check or you could have a person sitting in your team doing this every month or every week, but these kind of systems should always be in the place. Now you should also be identifying security incidents using social media intelligence. How we will get into that later on? So the question, another question which comes is what is an asset? Now earlier we have always been talking about assets having monetary values. They could be servers, your hard disk, routers, your organization buildings, et cetera, et cetera. Now these are the kind of assets which have a direct financial value you can calculate. But how about your social media accounts? If you have an account which has a huge number of followers and something wrong goes there, do you count it as an asset? If your password is being leaked on your repository, do you count it as an asset? A lot of people don't. But think about it, if that is not an asset and that password can actually cause a lot of financial as well as reputational cost to your organization, it becomes an asset, right? So ideally these things should also be considered when you are making your asset inventories. They don't have any monetary value, but they can damage, they can cause a huge reputational and financial loss to the organization. So picking up the first item from things which we should do, container discovery and monitoring of assets, these are few of the assets, and of course not limited to these, but these are the kind of things people should be looking on. IP addresses, now they could be dynamic IP addresses, dynamic because in your data centers and elastic IP addresses if you are using any cloud infrastructure. You should monitor your domains, subdomains, cloud storage objects which includes your buckets, blobs, spaces where you actually store your information. Lead credentials, API keys, social media accounts, third party API keys, analytics tags which we'll come to that later. And supply chain especially if your company has acquired a company or maybe you have a merger or you have a vendor which has access to your internal domains. They actually increase your attack surface area and hence you should always do some kind of ocean on that part as well. Coming to IP addresses, few of the sources again not limited to, but common sources from where you can fetch the list of IP addresses. You can use your cloud API if you are using AWS, GCP or Azure. They give you APIs from where you can fetch the list of your IP addresses. If you have a system or a hybrid system where you have a data center as well you should contact your admins who can give you the list of IP addresses. Internet wide scans which is something pretty common these days. You can do the whole internet scanning using tools like Zmas, Zscan, using which you can have the whole attack surface of internet and saying that okay, these are the subdomains, these are the IP addresses which exist and port numbers which are open. So you can use those databases. You can also use ASNID which is autonomous synchronization number. It is assigned to an organization which has a huge range of IP addresses. You can also use who is reverse search. We'll come to that and reverse PDR records. Basically you can generate a list of IP address and then do reverse pointer lookup on that which will give you a list of domains and then you can figure out if those assets are actually related to the organization or not. I have a small demonstration around this. So here we are doing who is lookup. So first we have done digon.index.ru. Dig is basically the command for looking up DNS records for a specific domain or PDR records using which we figured out that there's no IP address they have. That is therefore IP address we can pick any one of them. If you pick that IP address and then do a who is on that and then we grab AS out of it. So we get the ASNID which is 13238 in this case. So we go further and then I'm just going to windows. Yeah, so we got the ASNID which is 13238. And then we pass it to this command, which in which we are doing a reverse way. So we are identifying all the IP addresses which has ASNID 13238 and then I have done some grip and unique and sort on top of it. So we just have a list of IP addresses. We run that. These are the list of IP addresses which are assigned to this organization, which is Yandex. Now, of course, in case of the organization which are using a cloud platform, you will get ASNID of those cloud platforms like GCP or AWS, which will not be relevant, of course. But if you're talking about bigger organization, this is one of the way to find out their IP addresses, especially when you're interested in bug boundaries, doing bug bonding for big organizations. Okay, going back to slide. The other thing which we use is Project Sonar, where Project Sonar is a project which is being carried on by Rapid7 and MIT University in which they do internet-wide scans, not only the domain name records, but also port scans and the release of results that you see, of course, every week. We'll have a demo around it, but this is to summarize what kind of data you can get. You can get forward DNS records of A records, AA records, CNM records, TXT records, SOA records. Now, obviously, these databases are huge. They can range from 17 gigabytes to 300 gigabytes. Well, I have uploaded them on Google BigQuery and then doing search on top of it, but if you want, you can have your own systems. The place you can download these datasets is scans.io. You can also find out domains registered on top of an email address. I've used my own email address, which is quite funny, so don't laugh. Okay, I just said don't laugh. Anyway, so this is the email ID which I'm using and all of these domains are actually, I bought on top of this email ID. So you can't just find out email address out of a domain. You can also find domains out of an email address, but there's a disclaimer. This is not reliable information because after GDPR, who is is not a reliable source anymore, but you always have to try all your luck. Then you have to find your subdomains of the organization and again, I'm still talking from a defensive perspective. It looks like I'm an attacker right now, but to defend your organization from attackers, you have to get into those shoes and do all the things which an attacker do. So you can use search engines. You can run queries like site-colonabc.com. It will give you a lot of results and then you can keep removing the subdomains you have already listed. You can again use internet-wide scans. You can use certificate transparency reports. You can use brute force. For example, if you have the subdomain example.com, you can try all the subdomains like a.example.com, b.example.com, and so on and so forth. You can also use reverse IP lookup. So we just had a look. We can find out IP addresses of an organization using AS and ID. So now we have a list of IP addresses which we can reverse lookup and it might actually give us some subdomains for the organization. Some of, there are a lot of, there are actually a lot of tools around subdomain information. These two tools are my favorite. I use sublister and AMAS because they have a lot of sources from which they fetch the list of subdomains. And I use AIO DNS Brute, which is a really good tool to brute force domain. Domain names, the only catch is it is a threat to network bandwidth because it just sends so much of traffic. We will quickly have a demonstration around how you can find subdomains from Project Sonar and I'm falling in the same problem again. All right. So this is the dataset which I have uploaded, open data, deset. This is the table which I've created and then FDNSC name is the table where I have forward DNS records. If you look, this is just 11 gigabytes of data and then the other one is reverse DNS in which we have PTR records and this is 80 gigabytes of data. Now I am running these queries the first time checking anything.nokay.com. Now nokay.com because they have a bug bounty program and also they have a lot of data so it is a good example to showcase. So when I run it on nokay.com it gives me a huge list of subdomains which belong to them as well as IP addresses which belong to them.  So we can actually find out, of course there could be duplicates in this but a good number of chances that there are more than 3,000 subdomains here as well as IP addresses. Similarly we can run this on reverse data on forward DNS. So when I run it on forward DNS, we have 595 domains. So these are just the CNIM records data set and again we have so many domains like scl.developer.nokay.com, scportal.networks.nokay.com. Now if you look at them, it is really difficult to brute force such domains because they are not just subdomains, they are sub-subdomains, right? This is the easiest way to find such subdomains. So now we talk about cloud storage objects. So a lot of companies store their data in buckets, AWS buckets, Google Cloud buckets, Blobs, Spaces. A lot of times they store sensitive data intentionally or unintentionally because sometimes they store sensitive data and they think the bucket is actually not publicly available and sometimes they are just serving aesthetic sites like JavaScript and images but by mistake they sometimes upload backup.tar files because human, you know? And then if you can list those objects, you can download the whole backup.tar file including config.php kind of files. I mean examples, these are just examples. But how do you do that? You can spider the websites, you can fetch the source code of every page, you can extract the name of the buckets or cloud storage objects based on the rejects patterns and then you can check for permissions. The other approach is you can create possible bucket names. For example, if the company name is example, you can have a common nomenclature of, you know, you can generate patterns like example 01 prod, example stage, those kind of patterns and based on that you can brute force that. Just a screenshot of how this could be done.  And then we are running a brute force against it using bucket finder.rv and then we can figure out one of the buckets which has some sensitive information. Going forward, apart from cloud storage objects, we talked about finding leaked credentials. So these credentials or leaked information could be passwords, API keys, third party access tokens, DB credentials, internal domains or a lot more, right? And where can you find all of this? On GitHub, on Bitbucket, pastebin.anion websites, et cetera. So one of the ways which I always do is if I have a project, I figure out the name of the client, they are GitHub repositories and list all the repositories they have and then check for any sensitive information being leaked in that. Because otherwise any information which you get on GitHub, you have to verify whether it belongs to the organization or not, which you can, but it's a pretty lengthy process. The good way is you just find out the repositories of the organization and then, you know, go about it. You can also use Google Custom Search Engine because Pastebin is not just one website where you have such anonymous paste allowed. There are so many websites, including just.github.com, PSBDump, pasty.org, and so many of them. So you can just compile a list of them and create a Google Search Engine on top of it. So you have one way you can search it. And also Google Search Engine allows you to have API, you can automate that because it gives you API keys. So you can actually include that in your automation suits. You can use automated tools, including GitHub and Trafaloc. Trafaloc is the one which I use. But it is a manual example which I was doing before I was creating this presentation. I had a quick search around the API keys or access keys, which we can access. So I came across one access keys, which of course I have redacted. But when I checked it with a tool called Nimgo Stratus about identifying the privileges of that key, I figured out that this was an AWS root account, which basically means I can do anything on the AWS account in case they have billing set up. So this is one of the way you can do a manual search.  So Trafaloc is a small tool which takes input as a GitHub repository. When you pass it, it says, it basically downloads the source code of the repository as well as the commit histories. And then it uses projects patterns to figure out if there is anything sensitive in there. Now it has highlighted a bunch of things in this repository. It says there's a RSA private key, which was updated. Someone created it, updated it, and deleted it. That basically means there could be some juicy information. Let's copy that commit hash, open this in browser, and you figure out that this is actually a private key of the person who has committed that and then realize that he has committed his private key, so he deleted that. Finding this obviously is possible manually, but when you have tons of repositories, it becomes a troublesome task. So it is good to have tools like this where you can just run a for loop or some kind of automation to figure it out. The next thing is social media monitoring. Now this is a very important aspect, not only in terms of your leaked information, but also in terms of your organization reputation. For example, you are a security team in example.com when someone dropped the blog saying example.com hacked. Now you really would like to know to get notified of such kind of blogs, such kind of posts or Twitter tweets which are going around. So you should really keep an eye on that for which you can use streaming APIs. You can write your own scrapers. Also you can use Google alerts to have your organization name and anything, any new result when there is any new result coming from your organization, you should get an alert or you should have a page change detection as soon as you realize that there is a change in one of your pages, you should get a notification. And this is how you can do some kind of monitoring. I wrote a tool called tweet monitor long back which basically takes a keyword and it keeps looking for that keyword. If someone tweets around that keyword, it sends you an email and it also dumps this to Elasticsearch or any database of your choice. Now once this is dumped on Elasticsearch, of course, apart from the tweet, there are a lot of attributes of the tweet which includes screen name, time, zones, et cetera, et cetera. They are dumped to Elasticsearch on which you can draw your dashboards and figure out what are those users who are actually doing these kind of activities around your organization. It could be used for other purposes as well but I have been using this for representation checks. I have uploaded a video on YouTube which I have not included as a part of this presentation but you can go to this link and have a look on that, how this could be done. Another thing which should be really looked out is analytics tags. So a lot of times, we have administrator accounts which set up analytics for any of the domain and then they set up analytics for multiple domains of the same organization. What they forget is that they are using the same analytics ID across all these domains or subdomains. So there could be confidential domains or subdomains which you don't want to leak out but if you are using the same Google analytics code or any other analytics code for that matter, this can leak out all the assets which are using the same ID because they are using the same tag across multiple assets. So you can actually do reverse lookup on this. I'm using a tool called Built With for that matter in which if you open any website, it tells you all the tags being used and then when you click on the tag, it allows you to search for all the other assets in the world, I mean whatever they have crawled to if they are using the same tags. These are a few of the things in terms of discovering your data. Now once you have discovered all these kind of assets on a regular basis of course, you should do periodic attack simulation. Now what I mean by that is you should classify the assets. You have identified that you have bunch of IP addresses of domains, domains, cloud buckets and of course all of these categories have different kind of attack surface. So you should run custom scans. Now the examples could be if you have a server which is running Nginx, you don't need to run a brute force or dictionary attack for URLs for using all the URLs. You just need to find sensitive URLs for Nginx and check with them. Similarly if someone is using SharePoint, you would like to run, you would like to check services.asmx files instead of checkingconfig.php. So this way you can target your, you know dictionary attacks when you are talking about an asset. Also you can pass these assets to vulnerability scanners like Nessus, Nexpose, Burbsuit or anything for that matter. And then you can review reports. Again reviewing reports is a troublesome task so you might like to do some kind of automation there. So you can set up some centralized dashboards of all these reports. Also whenever you have a new release, new acquisition, new merger or any event which basically means that there could be more number of assets coming in or going out. You should check for these new assets and again check for all the assets you have, check for vulnerability resurfacing and run a complete cycle of these things which we just talked about. So apart from that, what are the countermeasures which you should do? Okay, so first thing is you should do everything on yourself before someone else use it against you. You should have OSINT awareness campaigns especially for your employees where you talk about not using same passwords across their multiple things, making sure when they upload pics from office space they make sure they're not revealing some kind of, you know, password is standing on their sticky notes on their desk, things like that. Also you should consider implementing metadata stripping especially when someone is sending some file on emails or uploading somewhere on the portals. There should be metadata strippers so you strip all the metadata which includes local paths, author names, softwares being used to develop those files. Those should be removed. You can also use Collective Intelligence Framework which basically collects information from third-in-del feeds and checks if IP has some bad malicious rank and then use it along with your SIEM integration support. You have an SIEM in which you are logging all the IP addresses but apart from those IP addresses you are also checking the rank of those IP addresses. Now if there's IP which is having a bad rank in malicious network, you can just check if this IP is sending you any hits and there's a new thing which I recently got to know about Honey Credits, just like Honey Pots. You create some fake credentials and spread it across multiple platforms. Now when a person uses this credential against you, you know that this credential does not exist and it's a clear chance of someone trying to attack you so you can simply block that IP address or that person for that matter. Now one important thing here is you should always try to identify the root cause instead of fixing the issue. Now I'll give you an example for that. There could be a case where your company is having a lot of new IP addresses coming every scan and you must be wondering why these assets are coming. So instead of just getting rid of those assets, you should try to figure out which team is doing this. What is the problem, why this team is doing it? Was there the reason or the reason is lack of induction program or the people are not aware of this or people haven't attended any security trainings, things like that. So try to identify the root cause instead of just fixing the issue. I mean, you should always fix the issue but you should always go about identifying the root cause as well. And whatever I just talked so far, it's not the ultimate way of achieving this thing. There are so many other things which cannot be covered in this duration of time but the talk is just to give you an idea of what kind of attack surface is lying out there and you should really focus upon it. This is how the process looks like if you want to understand this in a nutshell. Have a security team, if you don't have one, you should try making one. It can compromise of just one person but there should be some security department in the organization. You should always keep implementing open source intelligence countermeasures and then you should have a process of identifying assets, asset data sources, implementing asset discovery process, then periodic vulnerability checks and scanning on top of these assets. And once you have this data, this data should be passed back to OSINT countermeasures because now you have more insight of what kind of issues your organization is actually facing around these things. So you should pass it back to the phase where you are implementing countermeasures. So this is the kind of process you can follow. Of course not the exact process, not the bible for this but this is something we can use. So now you have done this, what is next for you? I have listed some resources. Again, there are so many resources in the internet but these are some of my favorite ones. Awesome asset discovery list in which this repository basically talks about all the resources which you can use for identifying assets, using assets, listing assets kind of things. Awesome OSINT resources, this is a kind of a generic OSINT repository where you will find resources, not just asset discovery but also in terms of investigations, cyber investigations, marketing, how you can use OSINT for those matters. This is one tool data spread which I wrote long back. This is kind of an open source intelligence framework which automates the process.  Handpicked weekly OSINT news, this is one chap who keeps writing about open source intelligence so if you are interested in learning OSINT or picking OSINT, understanding OSINT, this is one source which you can relate to this person, publishes weekly news, whatever happens in the OSINT domain, this person published that. And then open data which I talked about earlier in the presentation. This is the link where you can actually go and read more about the project, download the databases, et cetera. And that's it, so I'm up for a question and answer. Please ask any questions if you have, thank you. Shiva here, just wanted to be curious on your intelligence in dark web. How do you do or how do you get any ideas or any thoughts? So identifying information in that area is really difficult but of course it is possible there are a bunch of search engines for onion sites like Ahimia and onion.cab.to websites which allow you to search in that space. But it is always tricky because the kind of activities which happens there, they try to keep it very close and they don't allow you to find more information about it. So it is difficult. I haven't seen a lot of automation around it but manually it can be done. I have a different, there are many open source projects who tries to maintain their own infrastructure and then something happens over the security labs and things. So what kind of suggestions do you have for those smaller projects with less number of people? Like can they use to see a couple of these or ideas to make sure they are interested in secure? What kind of small projects are you talking? I don't understand that. Let's say like where we have only four or five people and they manage to just spend money for one degree of instance or a couple of servers, someone is running their own lab. So can they use the similar ideas to make the open source? But I don't understand the objective here, like. Like the whole, like I understand your talk, like that's a very nice way of explaining things. I'm saying like can a smaller group of people can use the similar ideas to identify issues for open source? Yeah, of course. I mean open source intelligence, I mean this is just one perspective of using open source intelligence for organizations but it could be used for multiple ways. I mean people use it for investigations as well. So the answer is yes, they can. So you talked about assets, you've listed out some other entries which are generally not part of the asset list. Yeah. Is that all or is there any other thing that you think we should also be looking at as an asset? I mean, as I said, it is not limited to those categories and it will always be an evolving thing, you know, as the industry will move forward, the kind of assets, the kind of things which will fall into that category will always keep evolving. So this list has always have to be updated.


People in the back, yes. So my name's Sarah Drasner, I'm here to talk about complex responsive animations. Like Adam just said, I'm a senior UX engineer at Trulia Zillow Group, and here's my Twitter and code pen, but you can look at those later. I figured since we're on a comedy stage, I would start off with some CSS humor. I was a little upset about actually using this in the talk because every time I started working on the talk, I'd rickroll myself, but anyway. So before I talk about what I'm like the ways in which to implement this, I'm going to kind of run through why because I think complex responsive animations have not really had their place in the sun yet, but animations are starting to kind of take over the web and they've become a bigger and bigger issue as people implement them in their UI, UX, and you know, some illustrations on their site, and so that means real life applications and real life also means mobile. I know that that sounds really complicated, but I'm going to show you some ways to kind of make it a lot easier and like really cool actually. So one thing I'm going to talk about a lot is animation in terms of informative UX, like stuff that's not just purely for show and just to do it, but also things that like kind of inform the viewer and you're like purposefully thinking about and it's not as hard as it used to be, so just kind of like getting away from the idea that because you're going to implement animations, you can't do things for responsive like that doesn't have to be the case, and you know, it's really important to give your users the correct information for the context that they're using, and a lot of the times that's mobile as well, so you know, not separating out those concerns. And it's really fun, remember fun, that was nice, because I'm going to be talking about two different kinds of animation, I'm just going to separate the concerns just a little bit so that you know what I'm referring to, what I'm referring to them. UI UX animation is basically, I'm going to reload the page here so you can see it, UI UX animation when I talk about that is like pieces of the UI aid in informative UX choreography, there's a great article that someone else wrote about that. And typically this can be done for mobile with well-placed CSS and media queries and JavaScript to trigger those interactions, so a lot of this stuff that you might think is like really complicated and you need JavaScript for you don't really. However, oh, another thing that UI UX animation really helps with is context shifting, so this is an example from CSS animation rocks, and basically, you know, in typical use cases that list item would just pop into view, this like slight transition in bringing it on the stage kind of helps the user like connects those two states and be like, oh, this appeared slowly, and like, you know, that's where that went. I'm not sure if this will load, we'll see, oh, it is. So like the nav on this site is a good example of context shifting, so if you see that logo is like really big and robust and you have our story and context and rather than it just snapping back into the other view, it transitions very nicely so you, the user has that kind of feedback of like, oh, okay, that's where that went. So, there's also standalone animation, I do a lot of these. You can use these on your site for kind of like more illustrative effects and kind of to illustrate your stories or your, you know, we've been talking a lot about how images are super heavy and very performance heavy, there's a lot you can do with SVG that's like really, really trim and I'll talk about the performance effects of that as well that is very easily accomplishable and actually like pretty cool animation as information. Animation really needs to be designed, I think that is not really something that's happened yet and so I wrote a smashing magazine article on it, that's the link to that if you want to grab the slides later. It really, because it helps with context shifting, it can't just be slapped on on the top of it things and I think a lot of people do that and you know, even in like, if you're building out complex systems, it's very tempting to write a bunch of animations and just have those lingering around that you can put on things but and I'm not, I'm definitely not saying not to reuse those resources but just a purposeful idea of like, oh, this is our company, this is like the type of easing we do for these type of modals and so it kind of also gives the viewer this like consistent look and feel rather than them popping up a modal on one page and it bounces and the other one goes in from the left. It's like kind of disorienting. When I'm talking about animation, I'm not saying to animate everything at all. I think animation is really strong and very powerful. We talk about calls to action being bright orange to get the user's attention but nothing really gets people's attention as much as something in motion. Because of that, it should be considered, you know, a helpful thing and also like potentially harmful, like you should really be thinking about the ways that you're using that in a very purposeful and hopefully creative way and not just making everything on the page move because I think that that's where some of animation's bad rap comes from. The other thing was something that I touched on before. It complies with branding so you can have like an entire set of the, this is the way our easing functions work on this site because we are a friendly site, it's very elastic and bounce or like, you know, if you're like, you know, an investment firm you'd have like more linear eases and things like that. So really thinking about how that works with your company's branding and not just being like, oh, I like this and then adding it. And I think that that's really the strength that Google Material Design had. Like that's like the thing that I think is really important and cool about Google Material Design is that those branding guidelines really went for a deep dive into like how those animations and transitions move. And now when you see those animations and transitions you're like, oh, it's Google Material Design. That's good branding like looking at an animation and being like, oh, that must be that company. That's really, really awesome. And then, you know, if you do work for a really big company like I work for Trulia, we have apps and we have web and they're not implemented the same way at all. But if you have a kind of like consistent idea for how things should look and feel with animation then you can kind of go to the mobile team and say like, we're doing this on the web and then keep those two things kind of coupled and paired. So I'm actually going to do this form separately. So when you're working with a contact form on a website, that is actually like the place where you are going to get the most lead generation. You're going to like, that's where your company is actually going to gather like the people that it needs to make money. If you're not thinking about the performance of your contact form, you're doing your customers a great disservice. So I was just considering the contact form and I'm pretty impatient on the web. And so like sometimes I'll fill out a contact form and that submit button like doesn't do anything and I hit it a bunch of times. Have you guys ever done that?  So one of the ways that you can kind of have animation help out with those things is to give the users feedback for those things. So one thing is that when they're hovering obviously on the submit button that they're hovering and then with that push will actually show them some sort of loading animation so that they know that something's coming and that helps with the anticipation part of animation which is one of the 12 principles of animation. And I personally think that it's better if you have like the initial loader sequence be a little bit like one like initial kickoff and then something that repeats after that so that if it takes a while to load it's like still okay. But you know how the sky's the limit on that one.  So if I like fill out this form. So it's like pretty simple but you know it's still it like gives the user that feedback and also that took a minute like that between those two interactions but you probably didn't really notice it as much because you're entertained by that loading animation. So like one thing that animation can help with is like if things do just sometimes forms take a second to submit it helps keep them engaged while that's happening and let them know like okay cool. So a little bit of tough love here like if animation feels like I get kind of tired of people saying like oh animation is like this like fluff if you treat it like fluff then it will be fluff. If you just slap animation on top at the last minute without thinking about real engagement and that user interaction then it's not going to be that great. So yeah of course I use Clippy because that's the kind of like you know the easy one. Another thing I want to cover is that motion on the web is not a one size fits all thing. So like I'm going to be showing you a couple of different techniques that I think are super cool but that definitely doesn't mean like I think you have to use these. Everybody is different and their implementation is different and you know that is just the way of the web. Some people don't do they do these like crazy canvas sites where everything's interactive and that doesn't necessarily mean that they should be making mobile sites. I'm not telling everybody to make the same thing. I think that for most companies though it's more responsive than you might have imagined. Like just saying that like oh it's animation like we're not going to do it for responsive is kind of like throwing the baby out with the bathwater. So motion design language is for projects. Don't worry I'm doing a lot of theory right now but I'll get to the demos later. Motion design language for projects I know it seems like a lot of work because you know you have to like sit before you actually make the thing and you know kind of talk about those things but consider your tooling like when you set up Grunt for the first time it probably took a little bit of time to do that and it took some consideration.  Same kind of thing. You can reuse resources like easing functions and animation keyframes Mixins extends you know the same reason that you make a design language for any CSS site and again what I was saying before like if mobile is decoupled it's easier to keep it consistent across both and like we all don't want to talk to each other all day long about everything we're making. If you have good animation branding guidelines you don't have to have a big meeting every time you implement something or you know like it's not like why is this doing this thing like it kind of saves that time in your company also. So I just wanted to talk a little bit about like the concept of things I feel like there's like this idea that the concept comes first but I don't think actually the concept comes first not just for animation but for like a lot of things so if we're talking about animation being like an intrinsic part of the build process you have your branding that's first that's immovable usually then you have your performance budget that's probably pretty immovable too then you have your concept which should be tied to those things right and then from that can be born the page structure and animation interaction which should come together. So it's actually kind of great that Chris went before me this was like not an intentional thing but Chris set up a lot of the stuff that I'm going to piggyback off of I'm not going to go into like the nitty gritty of what SVG exactly is I'm going to kind of just show you how to work with it in this context so I'm going to be using SVG for a lot of these demos and there are a few reasons for that that Chris kind of went on so it's crisp on every display which is awesome less HTTP requests to handle which Chris also talked about it's awesome easily scalable for responsive super awesome small file size when you design for performance so the key part of this is probably when you design for performance like it's really important that you actually consider those things it's not like every SVG is going to perform well you have to be thinking about that performance when you're designing it's easy to animate it's easy to make accessible which is really nice and it's fun there's a pattern here Canvas can be made accessible as well it takes a little bit more work like you have to make like React.js and Canvas work really well for another type of DOM but SVG is kind of pretty simple and I'll show you some examples for that so so this animation was actually pretty small and it was pretty easy to make in that way like designing it was just drawing those shapes and in Illustrator and bringing it over and like changing some of the colors and opacities and moving things around with transforms I think Ryan talked a bit about how you should be using transforms and opacity almost all of the demos I'm showing today only use those things you can do tons of things with those things so don't feel too limited with just like opacity and transforms transforms are awesome so that's like you know I'm going to be showing you some performance statistics for these things as well before we do anything though we have to optimize so SVG oh my god is Jake Archibald's kind of web GUI editor so you can just drag and drop it in there and there's like all these toggles because I would say that every time you do a compression of an SVG or an optimization of an SVG you're going to want slightly different controls there's SVGs where like optimizing the path is not a big deal or like you know like you have like a really really detailed path and that's a little bit more complicated so those kind of controls are really important Peter Collin Ridges SVG editor is a little bit older but I still really like it that also lets you toggle those controls SVG oh an SVG oh GUI those are like terminal based if you feel more comfortable in your terminal than in a web GUI so and then of course before we get started animation performance is really vital for some reason people do expect the mobile to be faster than the web and that's kind of counterintuitive if you know how the web works but they don't and so they consider these sites to be like smaller sites on their phone so keeping that into consideration that a lot of your users do expect really fast stuff on mobile a place with really good resources for all that stuff is jankfree.org they have tons and tons of resources done by the Chrome DevTools team on the Chrome DevTools team is Paul Irish he does advanced performance audits with DevTools he does some really nice things with like Chrome DevTools for like larger sites and he kind of digs into the timeline it's really cool I wrote a CSS tricks article called weighing SVG animation techniques with benchmarks I wrote this article in February of this year I think it's still pretty good in terms of like you can kind of get a general idea of where things are at I tested SMIL which had more pertinence then than it does now CSS Velocity JS and Green Sock API and the results were actually pretty interesting I'll get a little bit more into that as we get further along CSS did really really well especially when you know everything was like moved with transforms and things like that but the thing to keep in mind about this article is that it exists in a certain place and time at a certain juncture and it was done with certain versions of those libraries so animation on the web is always going to be a bit of a moving target there's new stuff coming out all the time when Chrome updates DevTools changes it the way it reports so you know don't take that as like you know like the Bible and I only did one type of animation and just benchmark those you should be running tests for yourself so like write an animation in CSS and then write it in JavaScript and see how it performs for your use case basically that was like the idea behind the article was to kick off that kind of stuff so some of you or maybe most of you might be familiar with this SVG technique with using responsive SVG sprites which is really cool you know it shows like a progressively bigger or smaller house depending on the viewport I kind of piggybacked off of that idea and created SVG animation sprites so originally I just created this like kind of map that I used in Illustrator for what I wanted to have happened so you know you see in like the expanded view at the top that there are like all of these stars and buildings and things in the next view the background changes and some of the windows drop out of the buildings and things like that and then at the bottom view it's a very like kind of small version of what's going on then I took that and I made it appropriate for the SVG viewport so rather than having it cascade like this because we're going to always use the same viewport for our view box for this image where it's I found all of the redundant kind of things between one SVG and the other one for the first two and then made a separate one for the bottom sprite I'll explain more as you see the animation so the side of the page here so as I scroll this animation down the animation completely changes but I'm reusing all of those resources and that sprite now the coolest part about that is that whole SVG and animation and actually the text CSS was eight kilobytes gzipped now if you're using images to illustrate content on the web like if you compare that to using a text with a photo it's like pretty good and it's actually kind of interactive and interesting and also that's like that fun part that we were talking about before so the CSS animation is actually not any different than you might expect it to be it's like I wrote a mix in for accelerate and then I just you know that's like hardware acceleration on any object that's moving and so that's a repeated feature I put that in there I also have an animation on like I just kind of grab all of the classes that are stars and have a common animation for all of those I also just designate one keyframe for all of those animations and have them at slightly different delays and timing so I can kind of reuse those and then also you don't have to set the you know I see a lot of people setting the zero and a hundred percent values for the keyframes you don't really have to do that if it's set on the initial element so you can kind of save some code there so there's no width and height for the SVG itself instead define it define it in CSS so right here we're using percentage we could also use flexbox we can someday use CSS grids which will be awesome it's really important and in fact it's vital to make sure that your viewbox is in there but I actually defined it to be smaller in this case because remember we have both of those views and now we're collapsing down to just one view so in order to do that we need to change the viewbox itself so we also need the preserve aspect ratio x y meet but that's the default so it's not strictly necessary to put it in so this is like really the coolest part is you can actually affect animation in your media query some of you know that some of you might not so you can change the animation like the skew for the mountains totally went off when I collapsed it on the mobile view so I had to kind of change that and that's very easy to do with CSS media queries I can also grab multiple objects and reuse the same animation so a lot of this stuff is much more simple and you already know a lot of these techniques that you'd probably maybe not have realized that you could have affected SVG as well yeah so this also applies to JavaScript I use a JavaScript for my animations as well and I can use media queries for my JavaScript even in like you know for CSS like I can like hide and display and do those kind of things while still animating parts or I can do media queries in the JavaScript so it's all those that same logic that you already know this part I did have to do with JavaScript it's the view box shift so remember when you saw those two images basically view boxes like a little window that you're looking into your SVG at so like the whole thing is like the canvas of the image and then you kind of tighten it so you have this like the view boxes like what is going to be shown on the screen so I have a media query in there and an event handler for you know different screen sizes and then that shifts to the lower view and so that you're only exposing that part of the SVG fallbacks there are really good options for fallbacks as well I use Modernizer I think it's a really great thing I think some people used at support too I fall back to a PNG equivalent a lot like if it's a standalone animation like the one I just showed there's no reason not to fall back to an image like it's just kind of there to illustrate the thing you might not necessarily need that if you're just animating those like remember the site with the nav like those those media queries probably aren't SVG so you might not need a fallback because those are fonts Sarah Soydan wrote a really great article about the picture element in polyfill for SVG I think that you should look at your analytics because that is a lot of work and you should kind of consider whether or not you have that type of audience on all of those older browsers to warrant like all of the picture element stuff maybe that time is well spent it really depends on what your analytics are telling you so this is like kind of a poor man version of an SVG in the illustration of text so it kind of does that and it stays a consistent size so I can just like collapse it on mobile and it just does the normal thing that you would think but I said complex animation so we're going to go step further but in order to do it I'm going to break the rules and use G-SAP which is green sock animation API one of the reasons I'm using green sock oh I think it's really important to say like I don't work for them they don't pay me I'm not like a sponsored person I just really like this tool there's lots of reasons for it this might be the number one reason I don't think I've ever made a really complex animation where I didn't rotate something and this is a total nightmare the word on the street meaning like I bugged people on twitter is that it will be fixed in like three versions of firefox but that's like a little why like ways to wait so consider that if you're going to make something like super crazy and you need to rotate something you might need to switch to something like G-SAP and another really big reason for switching to G-SAP is the timeline you can stack tweens you can set them a little bit before and after one another which is really really nice you can change their placement in time you can group them into scenes you can add relative labels so you can say like start and then have like a bunch of things start at the same time or like that gets really interesting when you have a really long animation it's like you know 20 seconds at 15 seconds you need everything to fire but like let's say you change that like then you know in CSS that's like all this recalculation and you don't have to do that you can animate the scenes you can make the whole thing faster with one line of code move the placement of the whole scene and nest I actually have an animation for you so yeah like I'm just like it you know as a visualization of like what you'd have to do for CSS like imagine that you have this like one second to three second animation you define these key frames and then some of that adjust you might have to change some of those percentage bases and you wouldn't have to do that in javascript so so this pen is just kind of to illustrate the power of the GSAP timeline so you have this like that middle part fires a few times the things go around so this is the current position so that middle part actually gets fired a few times and then that first part shows on the timeline where it goes second part I don't need to tell you you can do this thing so now it's really cool I can just say like oh you guys switch places and I can also say like actually I need everything to be faster so if you write CSS animate like you should definitely still be using CSS animations for those UI UX animations that we were talking about before like those that's a really good use and you don't have to load external resources and things like that but if you're going to try to make like a crazy fancy SVG animation which you totally should they're really fun to make switching over to a library like this is pretty powerful and it's really nice other cool things for complex animation so they support motion along a path that's actually the widest support for motion along a path because they have support in IE SMIL still doesn't even offer IE never will that is eventually going to move into the CSS spec but it that's still a little ways away so for now GSAP kind of gives you the biggest birth for motion along a path which is a pretty common use case for a complex animation that you want to tell it to like go around and do things like that they have draggable so you can drag and push and pull CSS properties draw SVG that's like the thing where it makes the SVG so like usually you take the stroke dash offset of like an SVG around the perimeter of it I think you know Leah did a great job explaining some of that and and you just have it animate along those lines but you have to calculate that out you know you have to find out the length of that path this just does this nice thing where you can say zero to true or you can have it like look like a line is going around the thing because you can say like you know zero and 10 percent or 90 and you know to 90 and 100 so that's cool this is really neat for a responsive dev percentage base transforms on SVG that is a hard thing to do and CSS especially with the support that it has right now so that's really cool so I made this for the talk so I'm just going to let it run once and then I'll show you what I was so can like replay this and resize it to any size I think I put between 20 percent and 60 percent or something and it just stays consistent so that's kind of nice it doesn't actually necessarily have to be fully fluid either like most responsive design is not like your site just going it's like things move around and like you know you have to like actually you know think about it and be creative about it so let's let's do some of that so I've been talking a bit about like ways to rethink things with animation or like ways that animations can be a key component to like what you're making and I was thinking about revisiting old approaches like infographics infographics had like a huge impact on conversion I mean that's an infographic about infographics it's very meta so you have like you know a traditional post gets 243 actions like infographic post gets over a thousand like that's a huge amount of conversion and I wrote an article about this if you're interested in checking it out there are three sources I found with like really good information on like crazy performance numbers like this one this is just a sampling it increased traffic to their website by 400% lead increased by 4500% number of new visitors to their site by 78% but all of these posts are two years old and I haven't really seen a lot I don't know about you guys I haven't seen a lot of infographics lately so I was kind of like thinking like okay what would make something that like has that kind of impact on leads just fall off of the face of the earth I think one of them is that it's not responsive if you know if you get an infographic on your desktop and you're like ooh cool there's all this stuff if you get it on mobile you're like it's like this horrible pinch and grab phenomenon and you're it's like too much information and you're like screw this so I don't think that they perform that well and there was a tipping point in like the amount of mobile and you know I'm sure you guys all know about that stuff up there's an article from Tim Cadlec on it if you want to know more so I think it's also not updating to the current context and one of those is design like design didn't really head in the infographics salon style all over the place like too much stuff kind of way it kind of got flatter and more minimal so trying to kind of reimagine all of that stuff so I made this pen that hopefully you can my brightness down so I kind of sped it up so that you could see the animation I'm demoing obviously the animation more than the content here but you can also go back and forth in the timeline to like see what's going on here and the so there's like the effectiveness the millions billions and trillions and then there's like the on the spinning thing there's like likely to unlikely and you can see like the most likely approach to the most unlikely approach so there's all of these kind of like transitions and things like that and then when I scale it for responsive it does that so like I move the controls over and I change that so like the SVG itself becomes fluid but some other parts do not they just move in placement just like regular responsive design so I talked a little bit before about that view box change here's like another example of the view box change I had it like really super big before and I had that like little thing around the dialogue the text is just text it's not part of the SVG and then I have like the replay button is just a replay button it's not part of the SVG and I'm the millions billions things like that so I basically just cut down the like size of the view box so that you're only seeing that part and then scale that instead because it's adjusting to that not like the whole thing and then you know I have media queries for layouts fallbacks with modernizer like all of these things are should be things that you're kind of familiar with and pretty easy to do and also it's a little more accessible like an image that's just shared like that is not accessible at all went away so like if you this is a work in progress I know that there are other like really cool ways of working with accessibility I'm kind of a newbie at accessibility so I'm starting to learn this stuff I got the idea for these things from our resource with support charts and also this article by Dudley Story it's really great so basically I have like a title for the icons and like an RA labeled by title so that I they know that they can go find that title and then also I made sure that the text was like decoupled from the image so that I can give a description of the image but then the text is still loading separately they can still find it social coding sites help you learn I don't work for them they don't pay me I really think I've learned like a lot of things from other developers and I just think that it's so I'm going to give like a little bit of a shout out to code pen here because I think that like you really can learn a lot of these techniques by following people and like looking at their code and reverse engineering it and this of course is well mostly responsible and it's fun remember fun there's also a million other people that you should check out because they're all super smart so yeah if you want to check out my slides later and like you know maybe follow these people on twitter or code pen they have tons of really interesting and cool demos or things to say thank you these slides are