Hello, everyone. I'm very glad to be here. I'm very glad to be in Bilbao. This is my first Euro Python. And I want to talk about, I think, your stack and ReactJS development on the edge. But let me start with a small intro. I'm a Python and ReactJS developer. And my primary task on my works is designing and developing Becant API. So I'm like Becant API master. But let me show how I will get to that stage. Everything starts in 2002 when I first met Microsoft from Page. That was scary times. But that was then I started learning PHP, HTML4. It was like a standard stack for that time. It was a very, very long time ago. And then I started my own web design studio and make some sites for corporate sites, internet shops for Ukrainian companies. And then sometime in 2007, I decided that PHP is bad and started looking around on what else I should use. And there are two options, Ruby or Python. And I decided to start using Python and Django 095 or 096. So I started using Python. And my life getting better, better, better. I started my outsourcing career with Django. Then I switched to Odesco, professional services. So we made services for Google, Facebook, and other big companies. And then in 2011, I met Flask, the next big Wow Python project. So I primarily switched from Django to Flask. That was a good time. And I used Flask maybe for two, three years in a row. But then in my work, I need to go back to Django. And right now, I'm also working with Django as a framework. So from my point of view, I'm a Python developer for eight years. I'm using Django from ancient times. I'm using Flask from ancient times. But I'm not happy. I'm not happy with all these technologies. This is me when I need to figure out why my Django app working slow or why developers around my team use its dot notation for giving ID for foreign key. So it's like mess, mess, mess. And I'm looking around. So I made a decision. I wanted to switch to JavaScript. And you should be looks like what JavaScript? Come on. It's not an answer. And I agree with you because JavaScript was bad. It's like in first big JavaScript library was, I think, its prototype. And it's big and scary. And then jQuery band on stage. But it's also, first it was small. But then, oh, mess, mess, mess. And my motor was like, let's front end tasks make someone else. Because I really don't want to make front end tasks. Because when you start making JavaScript code in, I don't know, three, four years ago, you're like, why? Why I need to do this? Because Python is very clear, very simple language. If we compare it to JavaScript in that case. The biggest JavaScript problems for me, that was no standard models. You know, this script mess in templates. You're like, someone includes a script here, someone there, someone there, there, there, there. And you don't know where that or else library introduce it to your project. And it's very hard to maintain because you need to copy paste all your code across the projects, across the library, across the directory, so it's a mess. But then Node.js happens and it's good. It's good that it happens because Node.js brings NPM. And NPM starts installing dependencies. And when you have a system for installing dependencies, all code resolution starts being easy and easy. And people around JavaScript world start deciding how to share their code across different projects. So CommonJS happens. And then people decided that we don't need so much script inclusions in HTML pages. So Browserify happens. And then people understand that they need to link their code and then various linters happens. And then even some people decided that, OK, we could use Node.js for making backend applications. But, um, it's serious. OK. OK. How it's a very, very simple example on how it was earlier, yeah, it's like somewhere in template.html you need to include all this code. Yeah, you need to import underscore. Then you should use global underscore. And that was mess. But right now, in some iterations, JavaScript makes it to some better. Yeah, you have one bundle. One bundle for all your functions, everything you need. And in template, in HTML templates, all you need to call your functions. It's like in normal programming languages. So and the old code you see is start looking better, better, better. And right now I want to talk how it's possible, which changed in JavaScript world. And I want to welcome ECMAScript 2015. It's previously known as ES6. But right now, it's changed named for ES2015. So let's see what JavaScript has. First of all, it's modulus. Yeah, it's like normal modulus from Python. Yeah, OK, we can use import. We can use import for libraries, for functions. And as you see, in libraries, we can use export some functions. So we can use this like functional, like library. OK, JavaScript has models. It's a good start, but also better starts that JavaScript no more have var for defining variables. So right now, JavaScript have let and const. Constants for constants is obvious. So if you try to change your constant, you will get an error. And let works properly in blocks. So you don't have all the problems that you have with war outside, inside the blocks, because you can define one let only in one block. So it's fixed var for ES2015. Next, you don't need to use a function, a special word, to execute some very simple tasks. So if we have in episodes var some list of star wars episodes, we can filter them with arrow functions. So you don't need to write function, then eat them, and then return eat them. And arrow functions couldn't have the body and could have a body. Next, JavaScript ES2015 have normal classes. So this is a built-in feature. So we can use normal classes here. Classes can have inheritance. Classes have constructor method. Classes is like normal classes from other languages. So it's a good direction. JavaScript also very bad in the past that JavaScript doesn't have multi-line strings. Right now, they have. You see, it's multi-line string started from third line. It works, and multi-line strings have simple formatting. It's not that good like in Python because you need to define the variables to use in templates before the template, but it's something. Next, this feature, one of the best features I used in JavaScript is destructing something. Python also have first type of destruction here. We can destruct a list or a tuple by the variables, but JavaScript also have an object destruction. And when you start using object destruction in JavaScript, all you need is object destruction in Python because it's very powerful and very good thing. Next, JavaScript introduce new types for the standard library. It's map and set. It's very, very funny that in this 2015, and map and set only included in JavaScript when we have in Python as times ago, but slowpox everywhere. But right now, we have map and set in JavaScript. And more. It's not all features of, yes, 2015. There are more features, and I will post my slides so you can check all the features of ES 2015. They have also iterators, generators, and improved Unicode, and proxy symbols, promises, and more and more. And the most brightest part of all of this that JavaScript, ECMAScript, has changed how it uses the standard. So there are no big... Okay, in next year, there would be ES 2016, then ES 2017, and then. And new, new, new features would be shipped to standard language. So in ES 2016, maybe there would be comprehension class properties function being to assign functions like we have in Python 3.5. And then, so JavaScript not staying on the place, and corporations and companies around the world are willing to develop better JavaScript. So not bad, not bad. And maybe if you are asking how to use it, you need only one. You need to use Babel for all good things. So Babel supports your bundle management system, like Browserify or Webpack, and with Babel, there emerges one and only lint tool. So you have three tools for all you need. Babel for transpiling ES 2015 to old ES 5, and Browserify or Webpack to make bundles from all your code and ES lint to make sure that all your code is conform your coding style, guys. So everything is starting easier and easier in JS world, so there are more and more better projects start developing around JS world, and there is how I found ReactJS, because when I started learning about ES 2015, I checked that many, many people are looking on how to use these all cool features, and they decided to use these all cool features with ReactJS. What is ReactJS? ReactJS is a painless UI framework. It's only UI. It's only framework for making user interface, so it's no even handling like in Backbone, so it's no like some data binding like in Angular. It's like only framework for rendering your components to web page. And it uses virtual DOM as like standard browser DOM is very, very heavy. ReactJS introduces to the world virtual DOM concepts that are used right now by many other JavaScript frameworks, and that concept is willing you to use light-wise components and change it and do everything faster, faster, and faster. And the main feature of the ReactJS is one-way reactive data flow. It means that you can send all the components only one way from the top to the bottom. I see this in the slides. And ReactJS has a very strong community around, so there are a lot of already built-in components, so your start is very easy, because there are a lot of tutorials, a lot of docs, a lot of everything. So this is the most easy and most simplest ReactJS component you see is a simple class which you need to define render method. This React component will render hello Python to the React container element on the page. Okay, this is an example with the state. When the state changed, you see a state definition on the second line, and then using states after the render method, when state changed, render method would be re-relating. So you change the state, and component will re-render. So this is the main idea of React. When the state changed, everything re-renders. And you see, yeah, when we click on the button, state changed, and our body will be re-rendered. Next, using components, sorry for not fitting the screen, using components, React make possible to introduce some components and using them in HTML text. You see the line with if preview, and we're using the new, not HTML text, but new component which named markdown. So like fetching data, it's next, next. We use a new fetch API for fetching data, so we don't need jQuery or other libraries for fetching data from the server. So we're using fetch, and then promises to fetching data. And you see, when we have good data, we're setting it to the components, and when data is bad, we're setting it with false status. And then in render method, when component first will be mounted, it will have no status. So then we show loading to the user. Then due to status and data we show, or server error or empty data or valid data to the user. It's very easy to use and very easy to write. And then I will show you what means one-way data-binding. It's like, if we have on the page comments component and comment, we introduce comments, do some fetching data, and then mapping all comments to simple comment components. What it means, it means that comment component doesn't change the state, because comment components have properties, have props. When data changes in a higher component, it would be a render inner comment component. So in React terms, the comments is a higher-order component, and comment is dump component. What it means for us developers is that we need to develop only higher-order components on our project, and all reusable dump components should be shareable between our projects. So it's easy-fi to develop these phrases. And that's not all React features. In 0.4.14, React even splits base React layer and React DOM to different packages. It means that right now you can use React not only with DOM model. You can use React Native, which is used in iPhones, and React Canvas is which is used in Canvas text. So you don't need the document of object model to use the React. And React have a magnificent routing system, so you can develop single-page applications with ease. React have many, many, many reusable components, and React even have many, many, many technologies to fetch data from the server, like Flux or GraphQL. And when you start using React, you're like, wow, it's really cool. It's really cool because it's easy to develop, easy to maintain, easy to introduce your components to other people in your team. And here is the question. And what about backend side? What about Python side? I don't want to use anything in Node.js to make in backend. I don't want to use any Isomorphic or Universal or how it's spelled JavaScript for making backends. So what about Python? And here comes the most intriguing and most new Python, not Python library, but Python stack is I think your stack. I think we all know what means I think your stack. It's like the standard asynchronous input-output, even loop, coroutine and tasks. It's very easy. It's included in Python 3.4 and available in Python 3.3. And I hope that everyone knows what I think you're about, but I think you're a very low-level stack, so you need to introduce libraries for using databases for making web applications. And then how your HTTP makes sense because IOSTP is HTTP client server for I think here. It's very easy. I hope you understand what's going on here. It's IOSTP client and this is IOSTP server. So this is a simple hello world in IOSTP web. All you need to make your view function a coroutine and then returning a response. So everything is easy. And this IOPG is also very easy. All you need is coroutine and then create pool. It's like all our previous sync code but made with yield from. And I think you're ready and many, many, many tools. So when I start looking around the Python, I will look on the Django, Django REST framework, Flask, Flask REST less, Flask if, and many, many, many solutions. But all the solutions doesn't feel like better solutions for me because I want to step up. I want to use something new and here is where IOSTP and I think your stack is fit in my needs. And you see many libraries already developed for I think here. It's like connecting with MySQL, Mongo, CouchDB, Elastic Search and many, many other technologies. And even some web frameworks already available which built on top of IOSTP web, named Muffin Induction and Spanner. It's like Flask on IOSTP web. So if you're interested, you may see here. But I want to introduce IOSTP web to you and this is briefly an introduction with all features that have that framework. It's very easy to understand and very easy to use. And it uses also one way, not like in React, one way that the flow, one way that the binding flow, but also one way flow with request object passing to the view function. You also can enjoy this in your project. Okay, this is a very simple hello world. Yeah, this is how you made your applications. It's very similar to the Flasker. I think it's a very Flask way to making applications.  I recommend to use Gunicorn, but if Gunicorn not fits your needs, you can use, I think, your event loop or other solutions. And Gunicorn have magic reload flag, so you don't need some reload solutions which built into Django Dev Server or Flasker server. Or you have code reloads in Gunicorn too. Okay, and this is how IOSTP web handling getPostData. You see it's handling getData. You have getMultidict, like in Django, like in Flask, like everywhere. And then you have reading from the post data. So as you need to read this, so you need to yield from, not to block EO, to read data from the request. And then, okay, next hand, like you are adding new roads to your application, it's very, very similar to every other solution we have in Python world. Even we have variable roads. It's something like this. It's not very good architectural decision from my point of view, because if we compare this to Django or Flask, in Django and Flask we have project ID, like positional argument to the view function, but here we read it from matching for dict. It's not very good solution for me, but I think Andrey Svetlov is the author and maintainer of this library, and he will have a big class about using IOSTP web on Wednesday, so you can ask him why he decided to do this. And then, then, yeah, you can use name at roads, you can use everything roads, so you can use everything, yeah, you can as one of the greatest abilities of IOSTP web, that application is inherited from the dict, so you can just modify dict like in it. Here you just put settings to the app dict, so every app in your... When you get up, you have your settings, so you don't need to import your settings anymore. So, and next, that app is appended to your request object, so you see you don't need to do from import, Django for the info settings, so from something info settings, you can just use your application with settings in the view, directly in the view. And then how to customize IOSTP web, you should use middlewares, there are a lot of middlewares for user sessions for everything, and this is an example of material middleware and how to use it with your web application. And this is a real code for using the book toolbar middleware factory or session middleware, so you could have your toolbar or session middleware in your IOSTP web application, so it's very, very, very easy and very, very similar to other solutions in Python. And middleware is like handling exceptions, you need to catch an exception and then rendering an error view. So, user sessions once more, once again, so it's like using session, you first introduce middleware, then you reading from the session, not working IOS, and then right to the session. So, everything is clear and everything is easy to use. Even IOSTP web could be used for rendering templates. I'm not sure that we need to use IOSTP web for rendering templates because it's not that solution, but it could be used. So, first you need to set up Jinja2, the Jinja2 example, and then just render to play it. This decorator is obviously coroutine, so you don't need to decorate your view function with coroutine again. Or if you need to do some more, you could use a render template function. So, everything is easy and everything like everywhere. And here you can use, you can send static files, so it is easy file development, so everything is okay on your development server, so everything like everywhere in Python. And next is how you can use your static files in templates. Okay, and more, there are briefly introduction, so it's not every feature that IOSTP web has. It also has web sockets, except header, custom conditions for rows, like a class-based handler, so you can use not view functions, but class-based solutions for handling, for handling the routing. And I say it again, IOSTP web has web sockets built in, so you don't need to look around on how to implement web sockets in my web application. So, just use IOSTP web and you have your web sockets server and client. So, mi me gusta. And real-world usage. I use IOSTP web for back-end API. It's obvious for project, which named Money Tracker, and that project is looking for my task around different issue trackers like JIRA, like Pivotal, like other, and it may possible to track time, which I spent for fixing some issues. And this is a simple structure of my project.  I have application Pi. When I have my application, I have use-Pi when I have my some, some my global views. I have layer storage-Pi layer for working with database and have API and IOS, I don't know, sub-ups for API. It's for providing me API and also providing authentication for that project. And there are several additional, several additional functions I use in my projects. It's like adding rows for the application, simplifying adding rows for the application. It's in the table settings. We use mapping type proxy from Python 3.3, so our settings are in the table. We can use debug mode in the application. So, right now, iOS is very raw. It's only on the beginning, so it's many, many features not introduced or not developed, but there are many that we need to develop. But I think in several next months, the ecosystem around the IOS TPP app would be, would be larger, larger, larger. So we could use it like, like we use Flask Django without any pain. And then the most interesting thing that I discovered when I using IOS TPP for my API back end is that I should use JSON schema for scamming my data. Like, I could use this same schema for validating data on back end and on front end. And here, with a little help of my project, which is called ROWROW, I'm using a schema for adding the project. You see, we have this kind of schema that the project can have name, slack, and description, and the project would respond with its ID. And then how to use it. We should validate requests on the server, and then we should validate response on the client with one similar schema. It's very, very good, but also very ROWROW solution for validating for you, for ensuring that we have proper data in the IOS TPP web application and in React.js application. And then there is a question. Why might need I think your stack? So there are a lot of already developed solutions in Django, in Flask, so why you need to switch on? Come on, because it's cool and trendy. Just switch on, because you just need to check it out, to check where Python world is heading. But honestly, you need it when your database API is... when you begin the API is the database. When you have a lot of requests to the external APIs, so you could use all the power of our sync here, all the power of asynchronous input-output. When you need your code to be executed in a sync style, and that code that needs to be executed in predictable styles, so you don't need any magic which provides seventh-let, or javent, or other solutions. In that cases, you should consider to use I think your stack on the backend. And the summary. I think the future is here. I suggest to use Python 3.4 for all good things, because if you're still using Python 2.0 like me on the work, it's so-so. Python 3.4 is stable. It's a very good version of the Python. You should use it in... in the work, for your pet projects or something, because IOS GP Web is easy to start, easy to use, so you could have a proper I think your Web... Web framework. IOPG have C-scale alchemy, so you should forget about the OREM. You don't need to object relationship management. You should just use IOPG skill alchemy. And if you miss something, just support it to your stack. It's not that easy. It's not that hard, so just don't forget to pay back to open source software. And from other parts of my slides, JavaScript also makes a very great step in the right direction. Many tools around JavaScript is maturing, so right now you should... you could use... you could write your... use the interface code without the pain. Building a bundling is easy. Webpack. LinkedIn is easy. ASLint. Making UI is easy. React. So it's how I see the evolution of making user interfaces around the Python. We should start using more, I think, your stack on the backend and using proper tools like React on the front-end. And this... this relation makes happen... this relation makes able to build very simple, very easy web applications. And this is true story. Okay, so this... I'll answer the questions. Thank you, Igor. We, unfortunately, are out of time for questions, but if you have something to ask, don't hesitate to reach out to Igor in the hallways and ask him. The next session here, we're starting in five minutes. It's... And I have small bonus. It's how it looks in Python 3.5.


So here's a question. Can any of these statements be objectively true? Now, I'm not asking according to Hume. I'm asking according to you. Can any of these be objectively true? That means that they are true or false regardless what anybody believes. In other words, if they are true, they are true not a virtue of belief, but because they're true for some other reason. Now, you've had a chance to look at them, and I've asked you whether you think that they can be objectively true. Well, what are the consequences? Here's something further to ask. What are the consequences of claiming that none of these can be objectively true? All right, so we say you're for any one of them. They can't be objectively true. Well, what are the consequences of this? Well, considering that first one, right, I have the moral right not to be harmed. Well, if this is not objectively true, then, well, that's just merely a subjective belief. Mm-hmm. It's not going to be true. It's certainly not true for some other person, right? Yes, somebody else through their own subjectivity might have some, oh, shall we say, nefarious intentions towards you. They wouldn't necessarily believe this and, you know, your preference to not be harmed is right up there with your preference for chocolate. And if that's all there is to it, then you're, and if they do actually harm you, well, it's just the same thing as not giving you chocolate. Espresso is the best kind of coffee. Yes, this is objectively true. No, I'm kidding. It's not. Yeah, Espresso is certainly my favorite kind of coffee. If you don't like Espresso, I kind of feel bad for you. But, you know, I understand, right? This is not something that's objectively true. That's up to the individual, okay? So if we say this is false, right? If we say it's false, it's objectively true. Well, it probably is false that it's objectively true. But yeah, yeah, this is just something that is up to the individual. It's bad to cheat on your romantic partner. Now, if you say that this is not objectively true, then whether somebody cheats on one's romantic partner is, yeah, it's it's kind of like a preference. And if there is some infidelity involved, well, it's just the same thing as, you know, not fulfilling that preference for chocolate. By the way, if you don't think this can be objectively true, that might explain why you're single. Okay, so murder is killing another person for no good reason. Well, if you're gonna say this is not objectively true, okay, that's fine. But this is just a definition. This is just a definition. If you don't think this is objectively true, then well, yeah, okay, so maybe you have a different definition of murder. One person's murder is another person's justifiable killing, I guess. So the question, you know, if we say this is that this can't be objectively true, then what definition of murder are we dealing with, right? You have to come up with this the sort of definition of murder. And if we say there is no objective truth as to what murder is, well, you know, this is probably similar to that problem with chocolate. It's bad sportsmanship to gloat when the other team loses. Well, again, this is not objectively true. Well, then we're just talking about preferences here. We're not talking about anything that's actually morally wrong with gloating. You kind of turn the tables on this. Now you have the duty not to harm me without good reason. And if we say this can't be objectively true, well, your duties would depend upon your subjective beliefs. Okay. Well, then harming another person is now just sort of the same thing as not satisfying preferences. You lie when you knowingly say something false with the intention of leading the other person to believe something false. Well, as with the early case we just saw, this is just a definition. This is just a definition. So it's not really a statement about whether a particular instance is lying or whether lying is good or bad. This is just a statement about lying.  We're gonna have deep problems about whether somebody's lying or not.  Somebody says, well, you were lying. I said, no, I wasn't. According to my definition, I wasn't. Well, then lying would just merely be a subjective matter. Not even necessarily good or bad, but lying itself, whether something is lying, is a subjective matter. It's wrong to lie for no good reason. Okay. Well, now we're back to a claim about whether lying is good or bad, right? We left the definition behind. Now we're saying whether it's good or bad. And if somebody does lie and somebody else doesn't want that to happen, well, then we just merely have failure to satisfy preferences as opposed to any kind of moral wrong. There's a difference between right and wrong. By the way, Hume thinks this is right. Even Hume's gonna agree with this, but there's a difference between right and wrong. Sure. For him, what is right or wrong is entirely subjective. Okay, that's fine. But at least there's something of a difference. And if we say there is no difference between right or wrong, well, then there's no difference in your decisions. No, it doesn't really matter what you do. You know, even if you say something like, I have to look out for myself. Well, why? Because that's neither right nor wrong. There is no right or wrong. So if we deny that there's any kind of right or wrong, we've left a lot behind. We've left a lot behind, including Hume. Now, we've seen quite a few statements here about whether something can be objectively right or wrong. Okay, now, if you don't like the consequences that follow, say, you know, murder is merely a failure to satisfy preferences as opposed to something really wrong, genuinely wrong. If you don't accept that consequence, well, you left Hume behind. You're saying there is now something about what's right or wrong independent of mere belief. And if you want to say that, well, that's fine. I'm not saying you can't think that there's something more to morality than mere belief. But if you're going to say that, what does determine what's true or false about morality?


The Narcissist is not done with you yet, it may seem like it's over, it may seem like the Narcissist has moved on with their lives, but Narcissists do not move on that easily, they cannot deal with the thought of you moving on without them, with all of the qualities and abilities that you possess, with your mental and emotional health, where you could rebuild everything that they tried to destroy, where you could meet someone who will treat you right, the Narcissist cannot allow that to happen, because that would reflect badly on them, it would make them feel inadequate as though they were not good enough for you, when they want to create this narrative that you were not good enough for them, which is not something that would happen on its own, they have to be the cause of it, they have to create conditions that make it difficult for you to perform, where it seems like you are deficient in something, and that is why the Narcissist is not done with you, they cannot allow you to move on without everything that makes you who you are, because that makes them look bad, it reveals to everyone who the problem is, because if you're so bad, how could you move on and do better without them, it wouldn't make any sense, it would reveal to people that the Narcissist was the problem all along, but the Narcissist image of reputation is very important to them, so they will do whatever it takes to protect it, for the Narcissist to protect their false self, it means that they have to destroy you, because it's clear that there was a problem in the relationship, it's clear that something was wrong, so someone must have been the cause of it, someone has to be responsible, and they want to make it look like something is wrong with you, they want to make you look like you're crazy, but a functioning person who can provide for themselves does not look crazy at all, which is why they have to destroy everything that makes you who you are, they have to reduce it to dust, because then their narrative is more believable, which is why the Narcissist is not done with you yet, because if you were to move on as you are, it would make them look bad, it would make it look like something is wrong with them, it would make it look like they were the problem, the Narcissist is not done with you yet, they're not done with you, because they still want something from you, you have what they want, it could be supply, it could be money or a place to stay, whatever it is they want it, and that is why they cannot leave you alone, Narcissists are sole losers, they hate to lose, they cannot deal with the thought that they might be missing out on something, once they have invested their time and energy into you, they believe that everything you have belongs to them, so they're not going to move on, at least not until they get a piece of the pie, because in their minds they have earned it, in their minds they have worked for it, Narcissists are arrogant and entitled, they really believe that you should just hand it over to them, and if you don't, they're going to be lurking around for a long time, Narcissists come into the relationship with nothing, and often leave with everything, while the victim comes into the relationship with everything, and often leaves with nothing, the Narcissists cannot deal with the thought of leaving a relationship with exactly what they came in with, or even if they only made a small game, they see a relationship as a job, as though they should be paid for any time they spend with you, as though you should see it as a privilege to be around them, even though they make your life hell, in their minds you should see it as an honour, Narcissists do not enter relationships to build anything, they enter relationships to steal your tools and resources, so that they can use it to build something for themselves, they come into relationships only to take, only for their own benefit, and that is why the Narcissist is not done with you, because they're not finished extracting what they want from you, they can see that you still have more to give, the final discard occurs only when you have nothing, or when they've decided that they've got enough, the Narcissist is not done with you yet, they're not finished with you, because they still want something, because they see that you still have something to give, as long as you have something, the relationship can continue, or if you refuse to be with them, they will hold on to you for dear life, because you are their lifeline, that's what you were when they first met you, and that's what you are when the relationship ends, you're just something that they depend over a means of escape from the difficult situation, you're just something that they use to preserve themselves, without any consideration for you, which is why they will step on you to get ahead, they will run you into the ground so that they can get what they want, because that's the only reason why they got involved with you, so that they could get what they want, they knew that you had it, and when you try to move on, they look at it like you're taking something which rightfully belongs to them, because they see you, and everything you have as their property, they see it as though you belong to them, and if you want to leave, you have to pay the price, you have to pay compensation, because they're always the victims, and in their minds, they have suffered, when you reject a narcissist, it causes a narcissistic injury, and after that, they're going to want revenge, when they targeted you, they felt like they had everything, but when you're no longer under their spell and you leave them, they feel like they've lost everything, so now they want to make you feel the same way, they want to make you lose everything, even though you are not responsible for what happened to them, they lied to you, they deceived you, but the narcissist will not take accountability for anything, in their minds, they were treated unfairly, and that is why they're not done with you, that is why they're going to keep coming back, that is why they're going to be a constant reminder in your life of the mistake you made, thank you for watching, I hope this video resonated with you, please like, comment, share and subscribe, if you would like to donate, my PayPal link is in the video description, coaching inquiries, you can email me at coachinganartsurvivor.uk, thank you for watching and I'll talk to you soon.


Things are blowing up in industrial systems here in Germany this year. I had hoped that these things wouldn't happen, that this kind of future wouldn't be one that we are living in, but unfortunately it is. And I hope that we can make that better, partly through the course of this talk, but more, I think, in the future with your help and your work. So I'm sorry to begin this presentation with such a dark thought, but this year's theme is a new dawn, and it's always darkest just before the dawn. So we're going to go through some of that darkness in industrial systems and Scottish systems to get to a better place, right? Now with that said, no hacker really gets to be where they are without the help of others, right? We stand on the shoulders of giants, and part of the key is not stepping on their toes on the way up. So I would like to say thank you to a bunch of people who are here, and also some people who aren't here, particularly the Oslo Hacker space where I hang out, and these people have taught me a lot of things, not just about technology, but about life, and on-apprentice, which is how Goya signed some of his last paintings and sketches, which basically means I'm still learning. Okay, so with that said, I hope that you will enjoy this talk with its darkness and its humor all at the same time. I used to be in circus as you may have guessed from the moustache, so I encourage you not just to view this as a technical vulnerability presentation, but also as kind of live, technical stand-up comedy. Instead of jokes, we have vulnerabilities, and I hope that you will enjoy them. So these vulnerabilities are in switches. I chose to focus on switches, and that will become clear throughout the presentation, why I chose to do that for industrial systems. And we are looking primarily at three different families of switches, because I don't want to pick on any one vendor. In fact, the whole idea of this talk is to continue giving it. I have two other colleagues who couldn't be here with me today who have some vulnerabilities and some other switches, and they look forward to presenting those vulnerabilities as part of this presentation in the future. So every time we give this presentation, we like to give some new vulnerabilities and show that this is systemic and endemic risk. So the three switches we'll be looking at today are the Siemens Scalance family, the GE Multilin family, and the Garrettcom Magnum family. These switches are usually not very big. They might be eight ports, they might be 24 ports, and they're used in a variety of different locations. So this talk is for you, if you work in a utility. If you test industrial Ethernet switches, if you manage industrial Ethernet networking. If you're comfortable at a Linux command line and you play with web apps, but you don't know as much about reverse engineering, don't worry, I'm exactly the same.  But I care about this stuff, and so I'm learning. If you are a developer of firmware, then I think this talk is for you as well. I hope you learned something from it. If you like vulnerabilities, you'll enjoy this quite a lot.  Some people collect stamps or stories or jokes. I collect private keys, and I like to share them with other enthusiasts, such as yourself. If you happen to work for one of the switch manufacturers, you know I've spoken to you before, I've gone with very well, we speak regularly. Some of you not yet, but I hope you'll come and have a chat with me later. Most SCADA or ICS presentations go a bit like this. PON, the PLC, the RTU, the HMI. These are terms that all of us in SCADA know. Maybe most of you know them by now. They're pretty popular. I hope you do. But programmable logic controller, remote terminal unit, or human machine interface. The big idea of the presentation is, if I PON these things, game over. Physical damage I win isn't the world a scary place. And I encourage you to demand better content. I certainly grew up with better content. I used to go and see the presentations and the talks of a guy called Jason Larson, and he has a fantastic example of this. I want all of you to try it right now. Just think about, if you had complete control over a paint factory, what would you do to damage it? No one's going to get hurt. Everything's safe. It's a thought experiment, right? What would you do to damage it? Most people can't answer this question. And on certain types of processes, I can't answer this question. But other types I've worked with before, and I can answer this question, and I encourage you to ask it. But if you like, and you want to learn more, go and see Marmusia's talk. I think it's tomorrow. Think of my talk as a frame for her talk. She's going to be talking about how to damage a chemical process and what you need to do as an engineer to do that. And the reason she's doing that is to build a better process in the future. You have to break a few things to make them work a little bit better. Okay, so what's the point in industrial control systems security? It's not credit card data. It's not privacy. No disrespect to my privacy friends in the room. I have the deepest love and respect for the work that you do. But confidentiality is the lowest priority for us in industrial systems. It would go availability, integrity, confidentiality. And you might even swap integrity and availability in many cases. So you have to protect the sensor data or the control signals. Everything else is maybe a vulnerability on the path to getting this, but it's not the most important thing that we're trying to protect. So that's why I'm attacking switches. That's where the process is, right? Now, these may not be core switches. They're often a little bit further down in the chain. They're field devices, right? So you might find them in any of these locations. And this last example is not necessarily important because oil and gas is important, but it's important because it gives you the general format of all industrial systems. You have sensor network, and sensor data is traveling back and forth, and you have control signal data. That's it, basically. You might have different control signals on different protocols, and you might have different sensors on different protocols, giving you different values like pressure or heat or whatever, but most processes follow basically this format. Okay, I don't do SCADA 101. There are other people who do this. I'm trying to do a little bit to set the reference for this talk, but usually I avoid it. So basically there's not much authentication or integrity in industrial systems protocols. There's not much cryptography. You would expect there to be, maybe. I'm continually surprised that I don't find any, and when I do find it, it's badly implemented and barely works. So once you have compromised a switch or another part of the network, you can perform man-in-the-middle attacks on the process, or you can create malicious firmwares on these different switches. And that's what I'm trying to prevent. I'm trying to find some of the different methods that people can use to produce these firmwares and then get the vendors to fix them. Okay. These are some of the protocols. If you are new to this space, if you want to do some more work in this area, but you don't know what to work on, take a picture of this slide or go and find it later, and choose one of these protocols and go and work on it. We need people to go to these different organizations. Some of them are proprietary, some of them are open, and complain that there's not enough cryptography going on in this space. And yes, you can use VPNs, but believe me, I often don't find them. Okay. These are the switches, the specific versions of the firmware, in case you're here for vulnerabilities instead of just me waffling on about the basics. If you want to go and look these up, if you're a penetration tester working in this space, you can go and find them all online. And you can get a feeling for the kind of coding practices that go into these different devices. Now I've tried to choose the vulnerabilities that I'm presenting very carefully to take you gently from web app vulnerabilities into a little bit deeper into the firmware. So the first one we'll be looking at is Siemens. And again, I'm not picking on any particular vendor. In fact, I'm very proud of Siemens. They're probably here again. They're here many years. And they fixed these vulnerabilities within three months. And I think that was awesome, especially in the space that I work in. The average patch time in SCADA and ICS is 18 months. So I think Siemens deserves a round of applause for getting these fixed. So without further ado, let's have some fun, right? So MD5, you go to this web page for the switch. This is the management page of a switch, right? And you interact with this web page and you have a look at it. And on the client side, they do MD5 of the password. Okay, that's fascinating. I don't think that's particularly secure. But it's done in roughly the same format as that Linux command. So I use the Linux command instead of the JavaScript just to make it easier for everyone. You have the username at the beginning and the password is in the middle. And then you have this nonce that's at the end, a number used once, right? I was surprised to see the nonce. And it's even called a nonce, right? So somebody had done a little bit of homework on their cryptography and they understood that they wanted to use, you know, this number used once to prevent replay of the hash every time. Okay, that's some pretty good work. Unfortunately, this is MD5, and this is protecting your electric utilities and your water and your sewage systems. And you can brute force this in a few seconds if the passwords are less than eight characters. At around 15, it might take you 20 minutes or something, right? You can do this from PCAPs, from network traffic captures, and then you have the clear text password that you can use forever after with that switch. So off to a bad start, in my opinion. So these are the nonces that we're looking at. I'm glad to hear you laughing. It warms the heart, right? So you can see that they are incrementing and that they are hex. Yeah. What else can you say about this? The last half is different than the first half. Not only is it incrementing, it is sequential if you pull them quickly enough. For those of you who also do a bit of reverse engineering, you might recognize the first half as well. Anybody in the room see any patterns in the first half of the nonces? No? Very good IP address. Mac address would have been a good guess as well. I thought it was at first and I got very confused when I went to look for the IP address because I went to the switch itself and the switch's IP address was not this in hex. It's the client side address, which I just couldn't believe, right? It seems sort of, it makes a sort of sense if you're trying to keep session IDs in state and it's like, oh, I want a different session address and then I'll just use time. I'll use uptime in hex as the rest of my session ID, right? You know, the entire IP space and time, that can't be brute-forced. It has a kind of crazy logic to it, right? Unfortunately, it can be. And you can get the uptime from the device using SNMP. And of course, if you don't want to use SNMP, you can get old school and use the TCP sequence ID numbers. So not a lot of entropy there, I guess I would say. And I think their lawyers agreed when they put out the comments on this. All right, not only can you perform session hijacking and if you are attacking switches, I'd like to point out that session hijacking is not necessarily a great attack in this environment. Think about it like you would at home, right? How often do you log into your router? In fact, even more importantly, how often do you upgrade the firmware on your router? Everyone who's upgraded the firmware on their router ever raise your hand just for an experiment. Thank goodness, right? But wait, keep them up just for a minute. Everybody who's updated it this year, keep your hand up, everybody else put them down. Everybody who's updated it in the last six months. Okay, so that gives you a sense of how long these vulnerabilities can be in play on an industrial systems environment if you multiply that by about 10, right? Okay, so you can simply upload a firmware image to a Siemens scalance device with this version number without authentication. You just need to know the URL. Cross-site request forgery, right? I just say CSRF all the time. I don't even remember what it stands for. So you can download a log file, not that useful, but you get a sense of what's going on in the switch. You know what user names might be present or whatever. Incidentally, all of these switches by default, or at least this one, only have two user names, right? So it's admin and operator, I think, on this switch. Or maybe it's not, but anyways, there's two user names. Admin and manager, I don't know. I get them mixed up now. But the configuration includes password hashes. I'm actually not even entirely convinced they're hashes because when you increase the length of your password, it increases, but I'll leave that for future researchers to examine. You can download the firmware image from the device, which is nice. So you just make a request. You just post an HTTP request to this device and it gives you the firmware that it is running back. That's not that big a deal, right? Because you're just viewing data on the switch. But you can upload firmware and configuration to this device, which is an authentication bypass in and of itself. But it's also interesting because I can take a configuration file from one of the devices that I have at home with a known password. I can upload a new configuration file with the password that I know. I can use the device to do whatever I want to do. And later I can re-upload the old configuration file that I got from the device so no one ever even realizes what's been changed, right? So I think that's a disappointing state of affairs. And I wrote a script to do this so that you wouldn't have to when you are doing penetration tests of these devices.  Cambridge is a small town. There's not much to do in the evening. So feel free to go and examine my GitHub repository where I put up some of this stuff. I'm Black Swanburst on GitHub and on Twitter. So, like I say, Siemens are some of my favorite people. So I'm going to finish up with them. This is old day, if you like, all that you have just seen. But I want you to keep in mind that these vulnerabilities will still be present in the wild for another two or three years. And I encourage you to go and have a look at your systems if you have any of these devices and check them out and upgrade the firmware. I also hope this encourages you that if you haven't done much in industrial systems in SCADA you don't have to be intimidated by all of the engineering and the terminology and the verbiage. There is plenty for any of you in this room to do in the industrial systems space. You need to spend a little time speaking to engineers and translating your vulnerabilities into something meaningful for them. But that's just a matter of spending more time with them and getting to know them. And I think that's valuable too because they have a lot of experience. They carry very deeply about safety and I've learned quite a lot of things from engineers. My general point here is I'd like you to stop defending banks and websites and other stuff. We need your help in industrial systems in utilities. We could really do with living in a safer world rather than one where you were just protecting other people's money. So we're going to move on to the GE multi-lin line. I worked on a GE ML 800 but these vulnerabilities affect seven of the nine switches in this family. Seven because one of the other switches is an unmanaged switch. If you're a hardware person maybe you want to go and play around with those but not so much my thing. And the other one uses a different firmware image but seven of the nine switches use a similar firmware image. GE offers a worldwide 10-year warranty. So let's see if that includes fixing vulnerabilities. I think it should. What do you think? No? A couple of no's? A couple of yeses? No? Undecided. All right. CCC is undecided on something. That's novel. Let's start with some new vulnerabilities. Cross-site scripting. Reflected I grant you but still cross-site scripting. And I want you to pay attention to the details. I'm not going to go slow for you. I'm going to ask you to think. I know it's morning. I know it's tough but I am going to ask you to think. See flash up there? Flash PHP in the third one? Yes. It runs flash in your browser. So if you know something about flash come and have a look at the switch sometimes. I didn't go for active script, you know, attacks. There are so many attack surfaces on this device I just I sometimes don't even know how I'm going to finish looking at all of them. So I just worked with the web interface to begin with, right? You have this cross-site scripting times eight. And I want you to notice in the last section there, arbitrarily supplied URL parameters. I don't know about you but I think that's funny, right? You can just make up parameters to stick your cross-site scripting in. It's unbelievable, right? Yeah. Anyways, what does that look like? It looks like that. They have an error data page. And okay, maybe I'm using a browser that they don't approve or something, but, you know, it deserves looking at. And you can do quite a lot of things with JavaScript on the client side these days. Disturbing. Anyways, I'm not a big fan of XSS, so I'm going to move on to things that I think are worth my time. So if you fetch the initial web page of this switch, before you've even logged in, you get this config. So this is pre-authentication. No authentication, right? Now keep in mind that these switches are designed for process data, right? It's not carrying traffic to images of cats. It's supposed to be for engineering. So what happens if I add a no-cache parameter and I make it, say, 500,000 digits long? I should just be able to crash the web server, right? Maybe, maybe. But you would not expect it to reboot the switch. And it takes, you know, a minute or so for the switch to reboot, which is actually really impressive. It comes up pretty quickly. But, you know, obviously you can repeat this. So I wanted to examine that a lot further. I wanted to know more about that crash, what was rebooting the switch. But like I say, I'm not a very good reverse engineer, so you're going to go on a little journey with me where I learned a couple things about reverse engineering. And I had to change my approach from looking at the web app style loans to moving into this other stuff. So why is a DOS even interesting? You'll remember that I mentioned Marmosia's talk. So the reason I mentioned her talk, this is it, right? Denial of service on a website. Who cares? It's tearing posters down as XKCD once famously explained to us. But in the industrial systems environment, it's very different. It can be very serious, right? A simplistic example is you have an application that has a heartbeat. And if you stop that heartbeat, it might go into some sort of safety state. It might, for example, scram a reactor. There is a famous denial of service on PLCs that did scram a reactor in real life. Does anybody know what H2S is? Any oil and gas engineers in the room? Okay. So H2S alerts not reaching their destinations is pretty serious business, right? For those of you who are not aware of H2S, it's a byproduct of producing oil and gas. And inhaled in very, very small amounts, you can go unconscious and in sort of larger amounts, respiratory failure. So if you take safety seriously, if you ever work on these rigs and these environments, you learn to care about the windsock, right? One of these alerts goes out, an alarm goes off. There are many different alarms. You have to memorize how they all sound on a rig. And then react to them. And when you hear the H2S alert, you look up at the windsock to keep an eye on where the wind is and try and avoid being downwind of wherever the leak is. So a simple denial of service that we would not care about in a web application environment in this environment can be very serious. I'm not saying it always is. It just can be, right? So denial of service goes up in our list of problems, especially when we're looking at networking devices. Okay, so that's it for the denial of service. But like I say, we're going to look at some other stuff. In fact, the story with this switch began with a concerned citizen. About three or four years ago, I found 10,000 industrial systems on the internet as part of my master's thesis. And I was pretty uncomfortable with that. So I sent that data to various computer emergency response teams around the world. I believe it was 52 of them, right? Not all of them were critical infrastructure. A lot of them were small stuff, but maybe one in 100, I was told. Or in one particular country when they got back to me, one in 20 were considered critical infrastructure. And after that, you have a sort of reputation among the computer emergency response teams of the world. So people send you stuff. You get anonymous emails from someone called Concern Citizen. Thank you very much. They sent me a firmware upgrade PCAP of this particular device. I suspect that they worked at one of the utilities. And they wanted me to see how upgrading the firmware of this GE switch was performed. So it all began with a PCAP. So I ran TCP trace to carve out all the files and see what was going on. And you could see instantly that there was an FTP session. Later looking at the switch, I see that you can also upgrade them over TFTP. So the management of the switch happens in HTTPS and is encrypted, but the firmware upload goes across FTP. So you can just carve the file out. A little bit of network forensics, I guess. So instantly I could see that this one is complete and the ports on the end of the number is giving me a clue of what's going on. It's a larger stream. This one seems interesting. Let's have a look at it. So I tried running File and VinWalk. I don't know about you, but I believe that hacking is a journey of understanding. In fact, hacking is understanding a system better than it understands itself and nudging it to do what you want, right? And I also feel that I should understand my tools. I don't really understand my tools until I know where they're going to fail me or they have failed me in the past. In this particular case, I think VinWalk is a fantastic tool and File is a fantastic tool, but they didn't tell me anything. And that was a journey of discovery for me. So that was nice. It was like, okay, VinWalk doesn't always give me everything. I was running an older version and I think it would handle it now. But the point is after VinWalk didn't give me anything, just resort to the old school stuff, right? Go Strings. And I found these deflate and inflate copyright strings. And I could tell that a certain portion of the file was compressed. So this is just from the PCAP. Remember this whole story. So I tried to deflate the whole thing. That didn't work. Again, I just did something simple, right? Get a Python script that checks every byte to see which parts of the file don't produce Zlib errors when you try and decompress them. And you figure out what sectors of this file are compressed. So you go to your friend DD and you carve out this section of the file, right? So we have this larger firmware image with this little compressed section. And we have now cut this little compressed section out. I suppose I could have loaded this up into Python and used Zlib to decompress it. But at the time, I was still trying to use command line tools. And someone said, oh, just concatenate the Gzip bytes on it. Gzip inherits from inflate and deflate. So if you just concatenate the bytes, it should still handle it. So I did that. And I got a decompressed binary. When you ran Strings on that, it started to make a lot more sense. And you could find upcodes in it where previously it didn't make any sense at all. So once you've got an image like that, what do you do? Well, if you're me, you just grep for bugs. I think I learned that from Ilya if he's here in the room. Thank you, Ilya. Thank you very much. Ilya said, I asked him a year or two ago, how do you find so many bugs? And he said, oh, I grep for them. I use find. And so I started thinking about firmware images. If I was going to grep for a bug in a firmware image, what would it be? And my answer is hard-coded credentials and default keys, because you find them every single time. So I have this command alias on my machine, and I just grep for it, and I find private keys. And this is how you, too, can end up with a private key collection. So, there you go. Thank you. Yeah, they're hard-coded keys, but what are they for? It doesn't stop there. You know you've got the keys, but what do they do? Right? That was the next step of the journey for me. Two of them. You can see ones encrypted with a password. We'll come back to that one later. Let's start with the one on the left. If you load this key up into Wireshark, and you use it to decrypt the SSL, you have a self-decrypting PCAP. Remember the beginning? It was using HTTPS to manage the device and upload this firmware image. So, if you haven't to have this firmware image, you can decrypt all the traffic. No forward secrecy, right? Now, you don't have to be lucky and have concerned citizens send you an email. You can download this image from the GE website, and you can carve the keys out of the image in the same way that I did, and decrypt the SSL traffic of any PCAP that is sent to you. Now, the passwords underneath that are in clear text. You can see them highlighted down here. Password Manager and User Manager. You can see them up there as well, and you can see that we've decrypted the SSL with that key. So, default keys, right? Is it a big deal? I believe the vendors in this case say, you can upload your own key to the device. For those of you who aren't used to working in embedded, sometimes it's difficult to generate a key on the device because you don't have enough memory, or you don't have enough entropy, or you don't have enough processing power. And they're true. I shouldn't say excuses. Those things are true. But you could, of course, generate it on the client side and upload it to the device. And that's what they allow you to do with this switch, which is great. But where is your encrypted channel in which to upload this key? So you can use the serial device and make sure visually that there's no man in the middle. But if you're doing this remotely, and I'd like you to keep in mind that most substations are remote, if anyone here works in a utility, are you going to drive to every substation, plug in a serial cable to change the keys on all these devices? It's the sort of thing you need to know in advance. So the problem with key management, particularly with SSL and the industrial systems environment, is that you have to manage the keys. And these particular keys are self, well, the certificates are self-signed. So you can't revoke them. And besides, industrial systems are never connected to the internet. So it wouldn't have made any difference. So these are the kind of problems we're dealing with in this space, and that's why I'm trying to encourage you, whether you do crypto or privacy or whatever, spend a little time in the embedded space, just for a bit. There's plenty of easy work. Okay, so what about the second key? It requires a password. I didn't feel like brute forcing it. Maybe you do, I don't know. I tried all the strings in the image, a classic technique, right, just in case someone had a hard code of the password. I mean, the hard code of credentials were there, the hard code of password. So I guess I got to start reversing, and as I previously said, I suck at reversing. That's why I come to CCC, so I can learn, right? But I did find this PowerPC ROM image, and I think it's running ECost and RedBoot. And I haven't even gotten down to doing hardware stuff, taking it apart, having a look at it, but I probably will in the future. So there's the image. I'm slowly starting to learn my way around and figure out what's going on. So I had to look at the image, and I figured out that this key is used for SSH, right? Like, well, it would be the other encrypted thing. But I couldn't enable SSH on the device. I try and enable SSH on the device, and I'm logged in as manager, by the way, which is the highest level user on this particular device.  Like I said, I tried all the strings in the image. So apparently to enable SSH, I need a password for something. Now, maybe I'm just misunderstanding or I'm not so clear on what's going on, but I don't know about you. I kind of feel like if I buy a device that's supposed to be used for a safety critical process, I should be allowed to use SSH without having to call up the vendor and get some special magic password. So, considering I don't like that approach, what if I patch my own key into the image, right? I don't know the password of their key, but I know the password of a key I can generate. So I just need to make sure it's roughly the right size and try and patch it in. Then I've got some problems with compression because I've got to reverse the whole process that I just described to you, patch it into the larger binary. Will there be any CRCs or firmware signing? I don't know, right? So, the uploaded image is not a valid image for this device. That's correct. I messed with it. But I got this error and it gave me a clue. It gave me a clue that I did, indeed, have some of my CRCs wrong. So when I altered the image again, I got to this state. So you're learning all the time by having a real device. Now, some of my friends, they do static analysis and they don't buy these devices. I decided to buy this one. I found one on eBay. It wasn't very expensive. So, I mean, it depends on your range for expensive, but if you're helping defend industrial systems, I thought it was worth the money. So I bought it. And this enables me to try firmware images out, right? And I can slowly start to figure out what I need to patch on these firmware images to do whatever I want. Luckily, I just tried to patch mine to have SSH because I thought people deserve to have SSH. So that's an Adler 32 up there on the left. And the other CRC is on the bottom. So that Adler 32 and some adjustment of file length, all those zeros and that line just above it, eventually got me to the point where it believes it's a corrupted binary. And then we have this CRC on the end that we need to have a look at. Now, I'm a big fan of suspense. I love suspense. So I'm going to leave that one as a cliffhanger and an exercise for you watching. So I said I was going to talk about GE ML800, but I'm also going to talk about Garrettcom. Luckily, it's not very difficult. It's a personal equipment manufacturer for the GE ML800 series. I noticed that because the certificate I found attached to those private keys said Garrettcom in it. And I went and looked at their firmware images and they have similar CRCs, similar file structures, similar everything. So I believe that they are affected by the cross-site scripting, the denial of service, and hard-coded keys. I understand from some people that they have been in contact with GE to try and fix some of this stuff, but their response to GE was mainly, sorry, this is the end of life on this device. That's fine. I understand you're running a business, but you're selling equipment to people who manage utilities that we all depend on. You know, if Sony goes bankrupt because they get hacked, that's one thing, right? But you can't just dissolve utility and start again, as my friend Klaus points out regularly. Fantastic insights into the industrial system world, Klaus and Vanessa. You can't just dissolve the utility and start again. You still have the same infrastructure, you still have the same workers. It doesn't work that way. You can't bail out utilities that we depend on. So, sorry, end of life. I don't even understand why people buy these devices and this code without code escrow, you know? When you buy the code, make sure you have the code in perpetuity for these systems so that you can fix them when something like this or something worse happens. If I'm your worst nightmare, you have real problems because there are very dark people in the world actually damaging furnaces in Germany. So, if me disclosing keys on stage is scary for you, you need to get a grip. So, Garrett, here's your key, too. The strings come from the images. Developers are funny people, really. I like those. I just put them up because they're funny, right? Some people had some hard times, I guess, writing some of this code. And my respect to them, they do great work, but, you know, there's a couple things we can improve on security in these devices. So, I once had the opportunity to stand in front of six different vendors at the same time, their computer emergency response teams at a conference, and I said to them, will any of you commit to an average patch time for vulnerabilities of three months? An average patch time, because it might take eight months, as it so far has taken in the case of GE and Garecon, to work on these issues. It might take a long time, in some cases, but as an average patch time, I think three months for things that we all depend on is reasonable. So, I asked these six different teams in the same room if any of them would commit to this, and I heard silence for 30 seconds. So, my friend decided to call this the silence of the vendors, right? And I think that sums it up. I'd like to see better patch times. I'd like to see computer emergency response teams in each of these vendors, and I'd like to see someone responsible for security in each of these different utilities. I can dream, right? I think that key management, the current practice in industrial systems is to take some insecure protocol and wrap it in SSL or TLS, which is why we need the help of you privacy people, because TLS and SSL are not the be-all and add-all. They often sort of go the wrong way, right? For example, you can use TLS to do integrity without encryption. So, you can verify that every message has reached its destination intact, but it is not encrypted, and this means that you can still do intrusion detection analysis of the packets, right? That's really good, but nobody uses that in SSL in other ways, right? I'm a big fan of showdown, and you showdown for a variety of different things, usually to get a sense of the internet as a whole, right? Let me back up a little bit. When I was at Cambridge, I went to Darwin College, and because you're at Darwin College, you read up a bit on Darwin, and you think about how Darwin thought. And I think the internet is kind of like that. When it was built by the IETF and various people who did fantastic work, they imagined it one way, and then we inherited it, and it grew, and it became an ecosystem, and stuff happens out there that you wouldn't expect. And so that's why I like showdown. It's kind of like being a natural scientist. What's a survey of the world? What kind of machines are out there? What versions are they running? When do people update their certificates? Do they do it before or after the certificate is invalid? Do they always upgrade the algorithm? Do they increase the key size? How do things change, right? You need to sort of study it as a whole. And that's my point when it comes to just taking SSL and slapping it over a protocol. It's not quite that simple. So again, we need your help. Where can we go with these attacks? And you remember at the beginning, I pointed out the underpants gnome. The emperor wears no clothes. Altering switch configurations is a big deal, because you can ex-filtrate process data. That gives you a map of the process, because industrial systems are bespoke. Each one of them is different. It does run different traffic, and we are lucky to work on security in this space because our users are enumerate and literate, and they care about safety. They don't always understand security, but they do care about safety. They care. There are also engineers at many of these utilities who look at the network 24-7. Not all of them, but some of them. Can you imagine a home network or something else with that kind of user base? We're lucky. We should be taking advantage of that user base. So getting back to the point, denial of service attacks to disrupt the process. Go and see Marmush's talk. This will all make a lot more sense when you go and see her talk. Data traffic can disrupt, alter, or drop traffic at this point if you can affect the switches in the substation. And X-Filtrating the data gives you a map of the process, which leads towards further potential damage for the utilities. Now, it's not always that simple.  It's not true. It takes a little bit of thought. It takes a little bit of work. I am certainly not awesome. I am just a quality assurance person from a former vendor. I just decided to get into security and keep going with it, right? So you can't always perform these man-in-the-middle attacks. People will say you can, but the reason you can is real-time system constraints. Some systems will stop receiving traffic five milliseconds or microseconds later and ignore anything. If a value doesn't arrive in this time, it doesn't care. So the idea that you can route the traffic out to some other country and then back in and disrupt the process is bollocks. Sometimes you have to alter the firmware to achieve that. Now, it depends on the process, but I'm just trying to give you a sense of how performing actual attacks give you a sense of what the limits are, what the logistical burdens are for the attacker, and that's important stuff for us to know. All right, a little bit of an overview. Drunk session IDs, brute-forcing MD5 plus nonce. Cross-site request forgery for firmware upload of all things. Reflected cross-site scripting, eight cases of it. Pre-authentication denial of service. Hard-coded keys, times two in a firmware image. SSL without forward secrecy, self-signed certificates. So there's no revoking, there's no managing of the keys on these devices, right? Not to mention utility workers are busy already. They may not have time to manage all of these devices. We might need to rethink that approach, right? Clear text passwords under SSL, because, well, no one can break SSL unless you hard-code the key in the firmware that's downloadable from the Internet. Enable SSH with a password and three-quarter of a year waiting for fixes for some of this stuff. I'm not happy with that. I think that we could live in a much better, much safer world, and to do so we need to talk very seriously about some of these issues. Don't take my opinion for it. Listen to some other people. The best thing about doing industrial systems work is the diversity of approach, you know? I love that there are so many other people doing SCADA and ICS, and I love that they're growing in different directions. So in the future I plan to be on another stage with some friends and show you some more. Thank you for listening, Mustache fans, and as a parting thought, more tax money is spent on surveillance than on defending common utilities. Well, thank you. You made me a scary Sunday morning. They got a utility down the road. Okay. We'll have some questions taken. Please, as the session is recorded and streamed, anything you say, say it into a mic. Any questions up? Wow, it is Sunday morning. Everybody understood everything? You're kidding me. There's a question. Hey, thanks. I enjoyed your talk. And I think it's very important to raise awareness, but I think it's not to raise awareness not much in this community, but within the engineering community. And I see it a lot of times, and many engineers having lots of problems doing that for several reasons. There is maybe the engineer who is thinking about this, but has its managers in the back, has to deal with service personnel, which know how to work a hammer and a screwdriver. On the other side, engineers have to work with customers, which are more of those lazy people. And so that's how these things happen. And I think it's more important to raise awareness of these kinds of things in the engineering community. So just to repeat a little bit for anybody else that couldn't hear it or for the recording. It's very important to work with engineers. Some of the engineers understand the problem, but typically management or lower level service personnel don't always understand the problem. And it's not so important to raise the awareness in the hacker community, but more with the engineers is what you were saying. Okay, absolutely true. Completely agree with you. I don't just come to these conferences and present to you guys. I go and I present to the engineers too. And in fact, a couple engineers have come to this conference because we did work at other conferences to see what the hacker community is about and learn things from the hacker community because this is a place where you can learn if you're just not afraid of getting pwned a couple of times. And it happens to me too, right? I learn a lot from getting compromised on my machine and watching someone do something. Anyways, back to the point. I don't just work with engineers or hackers. I also work with C-level executives. So I'm on a sabbatical from IOACTIV at the moment at the Cambridge Center for Risk Studies. And I'm working with the insurance people, which has its challenges, shall we say. But some of them are very intelligent people and they want to understand what's going on with hacking attacks and they want to approach this from a slightly different angle. My stake in that is to be sure that when the insurance people do get involved that they actually ask for fixes and improve stuff. So yes, I do my best to raise awareness wherever I can. And I'm not alone. You can help me. Thank you. Okay, there's another question over here. Number two. Oh, and up there too. Yes, we saw you. Okay, number two was first, I think. Go ahead. So you mentioned a couple of things or a couple of vulnerabilities and I was wondering what you would think an ideal system would look like. You mentioned the key provisioning and of course putting certificates. I assume that they were different certificates for different devices rather than the same certificate for all devices. Okay, that's a bad thing. And also sort of the way how the software update management works. So how would you, if you could give them some advice how to design a system, how would you do it? Okay, so first of all, I wouldn't hard code the keys as you discussed to be in every device the same. It's one thing to put in your documentation how you should update the keys. But I mean if I can patch a binary file with a key, then there's no reason you couldn't do that on the website where you download the firmware image just as an example, as a thought experiment. So to make that clear, the upgrade path for these devices is download the firmware image from the website to some machine and then carry it because all these systems are airgap to some other location and then upload it onto the switch with hard coded credentials. So first off, whenever you provision a switch initially, you provision all of the credentials for that device. This is standard practice on many routers and other pieces of equipment today. And I would think less about defending and securing the device than being able to regularly check its integrity, the integrity of the firmware that is running and the integrity of the configuration. So I'd focus on that and I'd focus on being able to recover the switch after it's been attacked. So you reverse your thinking. You assume that one day someone's going to crack your firmware signing and crack this and crack that. Can I quickly upload a new firmware image that is known to be good and verify that the one that is uploaded is good to this device? Thank you. There was a question up there on the balcony. Yes, we have two questions here on the net. The first one is, how would you also end of life issue? Sometimes product lines just get really outdated. That's absolutely true and it is slightly unfair of me to put it so hard on the vendors, but it's my job to take the debate a little bit too far the other way, right? So how would I solve the end of life issue was the question from the internet? I don't know. I think that's not a technical problem. It's a societal problem. Like when we buy bridges, they're bridges until they fall down. When we buy roads, they stay there until they go away. I mean there's probably some end of life issues in there, but it's almost more of a contractual legal issue and someone should study that. There are people studying that, but it's not my area of expertise. But I'll try and answer as best I can. I think code to scroll is a good way to go. When you buy some of these devices, you say, I want the code for this device in the future. I want to have access to it. If your company goes bankrupt, I need you to give up the source code for these devices when you go bankrupt or when you disappear or when it's end of life. There are a couple manufacturers out there that are using open source switches. There's a company called Open Gear who are awesome. They gave me a switch to play with that I haven't had time to look at yet. I think that's amazing. Their code is open source and you can go and examine it. So you would have the code anyway. Those are two different approaches. I think there are others. You can solve this problem technically or legally or socially, but as a society, we depend on these utilities. That code should not just vanish and disappear. There was a second question from the internet. Yes. The second one is, what should a non-technical person in respect of the computing side non-technical person that manage a small town utility do as best practice? I think the first and most important thing is to look for a tax. I'm sorry, I should probably repeat that question just to be sure. What should someone in a small town who manages a utility do to defend themselves and protect themselves? So the first thing is, look for a tax. Even if you spend a few hours a week looking for something, you script something up or you hire some college kid to come in and script something and look for things on your network and ask questions.  So the first problem in utilities is it's kind of the mantra. So for a small utility, find someone whose job it is. If you're a very small utility, there's probably some other small utilities near you and you can hire a resource together to come and visit your different utilities and help you out. The second one is watch your relationship with your vendor. When you purchase this equipment, you spend a lot of money on it. Spend a little bit of time doing penetration tests. Yes, I like it when you hire me, but you don't have to hire me. There are plenty of other people you can hire in the simple vulnerabilities. So when you purchase something, make sure you test it for security purposes. And that's very important, because you can even put into your contract, if you fail these security tests, we will pay you less money. And the vendors are not going to react to security until you do that. So that's the second answer. And I wish I had a third to make it very neat, but I don't. Okay, there was one more question at Mike Four, I think. Yes, hi. Talking to the mic, please. Thank you for your talk. I'm kind of a newbie to the C3 community, and I am not sure about the question. I want to ask you probably many people understand in this room, but I don't. And therefore I would like to ask you what exactly do you mean by arbitrary firmware? No problem. So the question was what do you mean by arbitrary firmware? I mean firmware that I have altered that was not manufactured by the vendor to do whatever I want. How do you trust that this switch sends all the packets that it should send? What if it's my handle is BSB, right? What if it drops every packet that has BSB in the packet, right? You can rewrite a firmware image to do whatever the device can do. If you want to do all things, then the device usually does to damage itself, for example. So an arbitrary firmware is one in which anyone writes the firmware and there is no checking to be sure that this is the image that you want on this device. Whether it's provided by the vendor or the community, right? You still want checking that this is the correct code or the code that you wanted anyway, right? Okay, thank you. Is that a question here, Mike one? Okay, go ahead. In your hypothetical question you asked what damage could I do in a paint factory, but you can also reverse it. What kind of company secrets can I obtain? For example, your favorite recipe for your hot chocolate or the recipes of Coca-Cola. They are vulnerable as well, aren't they? Yes. So the question just again for everyone else, you don't just have to talk about damage in a paint factory or any industrial system. You can also talk about intellectual property and protecting the recipes that we use to bake cookies or make beer or whatever, pharmaceuticals, whatever. And that's a fantastic question and I'm glad you brought it up. A couple of years ago when I was doing well, more than a couple of years, like eight years ago when I was doing industrial system security I realized I wasn't getting a lot of traction. It was before Stuxnet. I was a quality assurance guy.  Stuxnet, career. It's wrong. It's really wrong. But the point is I tried to take that approach. I tried to say you have a process in which you manufacture something and you make money by the fact that that process is relatively secret. And if you don't care about your workers from being damaged, then at least care about the intellectual property because I'll get security in by some sort of backdoor, right? I'm a little bit of a security Machiavellian. I'll find a way to get security into these systems somehow. So I tried to say intellectual property, you should be protecting that. And I found that they didn't care so much. I mean, maybe you'll have more luck. Maybe post Stuxnet, that's a better argument. I hope you do. But it is an important question as well. What potential for damage? I think there's a lot more espionage going on on these networks than there is damage and sabotage. Okay. We'll take one more question on Mike Four. Okay. My question concerns the concepts of software defined networking and open flow. So when I first heard about software defined networking, I thought, well, this is a huge security issue and there may be huge vulnerabilities. After your talk, I think this might actually be a good idea to dump down the switches and put the intelligence somewhere locked up in a safe place. What's your opinion on that? Can it actually improve security? Yeah. So the question is what role could software defined networking play in these sorts of environments? And is it a good idea from a security perspective? Any time someone has a revolution in computing, we also have to update our security paradigm. So I think with software defined networking, it's not whether it's good or bad. It's that you defend that network differently than you defend one of these networks. So it's not so much that it's good or bad, it's neutral. If you know how to defend your network, I don't care what it is. As long as someone's looking to defend it and cares about how the flows are working. So I think software defined networking refresh rate on these devices is not that high. So I don't think we'll see it there for a little while, even though it might be a good thing philosophically. It takes 5, 10, 15, 20 years to refresh these networks. So it'll be a little while, right? But it's not good or bad, it's just learn to defend what you got is the problem, right? Okay, thanks a lot. Okay. Okay, let's give a big hand for Aaron and thank you. Thank you.


Flammable gases and oxygen are indispensable in industrial applications, for example for welding and cutting, for heating, forming, hardening or for surface treatment, as industrial and process gases in production and the laboratory or in countless other applications. Fuel gases such as acetylene and ethylene, propane and propylene, butane or methane or hydrogen mixed with oxygen have proven their value and are safe when handled correctly. However, a dangerous situation can arise at any time. Besides technical faults, the cause in most cases is just incorrect operation and human error. In the event of a flashback, the flame travels through the pipeline at twice the speed of sound. It is impossible to intervene manually and prevent the accident. The worst-case scenario, explosion of the compressed gas cylinder or gas tank. The flashback arrestor provides protection against such catastrophic failures, a multi-stage protection system based on the dry flashback arrestor that VIT launched on the market around 70 years ago. Flashback arrestors have revolutionized accident prevention with fuel gases and are used throughout the world today. A modern quality flashback arrestor made by VIT has at least three protection elements, a gas non-return valve, a flame arrestor and a temperature-sensitive cutoff valve. A pressure-sensitive cutoff valve can also be added, ensuring maximum safety. Let's have a look at the principle behind the flashback arrestor in more detail. If a hazardous situation arises and unwanted gases flow through the pipeline, the spring-loaded non-return valve will come into action. It stops the return flow instantaneously, thereby preventing the formation of an explosive gas mixture. If, on the other hand, the flashback occurs, this will be stopped immediately by the key element of the flashback arrestor, the hollow cylinder made from sintered chrome nickel steel. Only with this technology can flames from the flashback be prevented. The outstanding technical features are based on the highly porous structure of the cylinder. Incoming flames break up into an extended labyrinth, thereby losing their energy and cooling. Any flame is extinguished in a fraction of a second. The temperature-sensitive cutoff valve is the third important protection element. The integrated plastic body melts and the spring-loaded valve triggers before the device can heat dangerously. This automatic interruption of the gas supply prevents a critical ignition temperature from forming at the gas inlet. A dangerous back-burning is stopped at source and remains without damage. For maximum safety, VIT also provides a pressure-sensitive cutoff valve for various models. This additional protection element activates in the event of a flashback and interrupts the gas supply permanently. What makes it unique? A red alarm indicator makes every single flashback visible. Gas flow is only opened again after manually resetting the cutoff valve. This method ensures that no dangerous situation remains unnoticed and prevents the safety device from potentially heating. The flashback arrestor is effectively protected against dirt by a filter in the gas inlet. Flashback arrestors from VIT are compact and robust. They can be installed anywhere irrespective of the orientation. For maximum safety, the fuel gases and oxygen supply should be provided with a flashback arrestor. In many countries, this is already required by law. Choose the original for durable, low maintenance and high quality with verified safety. Safety begins with the product label. Thus, for example, flashback arrestors according to EN730 Part 1 must be labelled with the following information. Number of the standard. Manufacturer name. Model name or type designation. Gas types and maximum operating pressures. Flow direction. Safety elements of the device. The certification number of an external test body provides the certainty that compliance with the standard has been inspected by an independent authority.