edition of RCE. Again, this is Brock Palin. You can find us online at RCE-cast.com. You can find the entire bat catalog linked to all of our Twitter's and our blogs and all that sort of thing. Once again, I have Jeff Squires of Cisco Systems and one of the authors of OpenMPI. Jeff, thanks again for helping out. Hey Brock, and you know, it's that time of year again. It's coming up on supercomputing. Yeah, so once again, both Jeff and I will be at supercomputing. We both have booths, Cisco and the University of Michigan. Jeff, I understand your booth is way off into boonies though. Oh, yeah, we have a fairly terrible location this year due to an accounting mistake on our part. So admittedly our mistake, but please come find us. We're kind of in the back. I'll be hanging around my booth. I'll also be doing the normal OpenMPI State of the Union boff and also involved with a libfabric tutorial on Sunday. So be sure to sign up for that too. So we actually share our booth. We split it every year with Michigan State University, and we're recording this one day before the U of M Michigan State football game, which is kind of like one of the many rivalries around here. So this is always kind of fun. Yes, I think I will stay completely neutral and not make any comments whatsoever. I'm heading to Lansing this weekend, so we'll see how that goes. All right, well, what do we got today? So today we are going to be talking about a software project called SPAC. So we have one of the developers with us.  I'm Todd Gamblin, and I'm a computer scientist at Lawrence Livermore National Lab in the Center for Applied Scientific Computing. And I guess I'm also the author of SPAC. And my day job is performance analysis research. And I've recently been getting into build systems or package management systems mostly out of anger. I can appreciate that. So who did application support? So quickly, what is SPAC? So I guess we built SPAC as a package management system, but I guess you could also call it a port system. It's basically a build from source package manager. And it was designed to deal with all the combinatorial issues that you face with software in HPC. So essentially, on a regular machine, in your Linux box, maybe you have a package manager, you install something and installs the latest version of that and you're done. On an HPC machine, you might want to install different versions of that thing compiled with different compilers and different compiler versions. And so the space quickly blows up. So just before we go too much deeper here, I'm just kind of curious, where did the name SPAC come from? It's kind of a fun name, but a lot of things come up for it when you Google for it. Where did you get SPAC from? It's the super computing package manager. That's where we got it. We have since learned that there are other meanings for the word. Oh, okay. Well, we will leave those off of this podcast here. For us, the only definition is the super computing package manager. That's cool. So, all right, now with that white elephant out of here, let's talk about what SPAC is itself. So you say it's a build from source package manager. Why another build system? Why not port something like ports or why isn't YUM enough and things like that? So, I mean, we did look at VSD ports. VSD ports is essentially like a directory treat full of make files as far as I can tell. And I guess we didn't really think that was the interface that people were used to using. I think the real inspiration for SPAC is my experience installing packages on my Mac because that's a system that comes from the vendor without a package manager. And people want to install other software on it and so they use things like Homebrew or Mac ports or Fink to install things on it. So, I mean, I think if you look at SPAC packages, they look an awful lot like Homebrew packages. They're just in a different language. Now, you mentioned also at the beginning too that part of the rationale for doing this was because it was out of anger, I believe was the phrase. And both Brock and I were nodding like, oh, yes, I know exactly what you mean. Can you tell us exactly what you mean? Can you cite some examples? So have you ever tried to build the Rose compiler? I have not. You have not? The last time you installed something on an HPC machine, did you find all the dependencies that you needed? Because the software that we use is becoming a lot more complex. And so when we try to go build things on the supercomputer, I end up having to go down the rabbit hole of installing the software and its, you know, 10 or more dependencies to get it working. And that might be solvers or that might be different mass libraries or maybe like a GUI like QT. So that's really the problem that that's the basic problem that we're trying to solve is get this thing installed with all of its dependencies so that people can actually use libraries without completely annoying their users. So doesn't this end up causing other type of bloat? Like won't every install of a high level package pull in yet another identical install of NetCDF or HDF5 or some common library? So not necessarily. So SPAC lets you essentially reuse the version of that other library that you've already installed assuming that the package you're installing is compatible with it. So if I say, you know, back install some code that requires NetCDF and that code can build with NetCDF 1.3 and you already have it installed, then it'll just link against the existing version. So you may get codes that require some other versions of NetCDF because, you know, another one of the things SPAC was created to address. And in that case, it'll just build you two versions because, you know, you need them. You have a code that's linked against 1.2, you have another code that requires 1.3. Well, you can't reuse the 1.2 version. You have to go and build a new one. But it won't disturb the existing install. So you could do both. And I think that's one of the main differences between SPAC and, you know, what you might use as part of your OS or if you install something with your OS package manager. So we actually do a lot of things like this manually right now. We'll have software installed locations. It'll be like Red Hat Enterprise 6, NetCDF slash 526 Intel, you know, 4.2 slash, and we end up with a million different versions. So this is a system for kind of implementing all of that and figuring that all out. It is and it probably does more than you might think if you're using them, if you're used to making directory hierarchies. So I mean, I guess, if you look at the way that different sites install software, so like if you use the systems at TAC, or do you mostly use the local systems at Michigan? I use and support on both. So I'm familiar with the way some places do things. Yeah. So like Oak Ridge, Livermore, Texas Computing Center, I guess Michigan, they try to make these directory hierarchies and it usually includes things like the compiler, the compiler version, the architecture, the MPI, the MPI version, and so on. And that's actually just, that's only a subset of the dependencies of most of the things that are installed there. So essentially, if you try to build a purely hierarchical versioning scheme like that, you're not covering all the configuration parameters of the software. So where SPAC is a little different is that the version, or I guess the configuration of something that you install is actually the full DAG of dependencies. And the version itself is just a hash of that entire DAG that includes all of that provenance information. So this software, its version, its compiler, the compiler version, its dependencies and all of that for them as well. Wait, so if it's all stored in a SPAC maintaining like a database someplace, so I can then go ask, what does this install have in terms of options for that package? So it does. The database is pretty simple in that it's really just the install tree. And under each install, we have a little .spac directory that contains a YAML file, which is just, you know, it's like a simpler that has all the provenance information. So it tells you what that package was linked with. And then we can go and query that. So if you type SPAC find and then you type an expression like, you know, find me all of the versions of, you know, HDF 5 that were linked with MPH. You can do that. It'll show you all the versions that are installed that linked with that library. And you can get more specific. You can say, show me all the versions that are installed that linked with MPH as greater than version 3. All right. Now I noticed too, I mean, you said earlier that, you know, you can link against dependent libraries of different versions. So let's say I've got, you know, package one links against some dependent package version X and package two requires, you know, that same dependent version or same dependent package, but, you know, version Y instead of version X. How do you handle, I understand how you can do that from a build system and a link system, but how does that work at runtime? Right. So SPAC is actually, it's based on the way that we already install software at Livermore, which is actually probably less, I guess, module intensive. Are the listeners familiar with modules? No, well, probably some, but let's go ahead and, this was kind of a leader question, right, into modules. Yeah. I mean, so, and actually, I mean, SPAC can do both of these things. So a lot of sites will use modules to manage the environment. So environment modules are a thing that came around in, I think the 90s, like 92, there was a paper at Lisa that talked about environment modules and how they were being used at some compute centers. Essentially, this is just a, it's modules runs entirely in your shell. And it's a way of encapsulating all the environment variables that you need to set to get a particular package to run. And usually the main one that you want there is, get this thing in your path.  And also the, it'll set LD library path and all the environment variables that you need to find the libraries that go along with that package. Okay. So then do you, does the SPAC like integrate with module files or does SPAC just make a directory that's very module file friendly? So SPAC actually does generate modules right now.  It'll look and see if it has a bin directory and a lib or a lib64 directory, and it'll spit out a module file that just adds those to the right path, adds man path as well. Okay. And it does. So like if I, if I have package a that has like six different dependency packages, you would make a module file that includes all the right stuff. Yeah. All the right stuff to it, to a certain extent, right? What we don't have right now is some packages really want to set, you know, their own custom module things. And we're not bringing in pre-rex on the modules right now. So for example, like we won't, we won't add all the dependencies to the module file, which is, which is something we ought to fix. But, but we can very easily generate that because we have all the information that we need. Now what someone recently contributed is support for Elmod, which is, which is tax module system. So are you guys familiar with that? No, I was, I was, I was your, your head of the class because I was about to ask you if you integrate with Elmod too. Right. So we had a user, lots of Miliano Colpo from one of the, I think it's highly in compete, super competing centers. He recently sent us a pull request for Elmod support and it will generate the, the hierarchical, you know, directory structure for the modules. And it'll generate the Lua module files that, that tack likes. And that's really nice. So you can, you can easily swap between like, you know, the version of your package for one MPI and the version of your package for another using that. But really, so at Livermore we don't use modules. We tend to install all of our software so that it will run without the user having to know the magic incantation to load the module corresponding to that executable. So if you find something in a directory, you can generally run it. And the way that we do that is we set our paths on the executable. So I mean, and for people who aren't familiar with our path, essentially, if you, if you know what LD library path does, it's the search path that you set in your environment that says, here's all the directories you want to search to find dependency libraries of this executable or this, or this library. And, and our path is essentially the same thing, but it's actually set on the binary itself. So if you, if you add our path arguments, you're basically saying this binary knows how to find all of its dependencies, just use these paths. So we install everything that way.  Okay, that's actually really cool because I need to do that. I mean, I've ran into it before, but I've never completely thought through what that would actually do for us. And as you're right, the packages are getting more complex. Yeah. You know, you need trialinos, trialinos needs this, but they want to use the Python bindings for trial, you know, it's just like you go down this chain of pain of trying to make it all work. And then they want to be able to run that and, you know, use Matlab at the same time. And Matlab brought its own QCC, something that so yeah, yeah, real pain. So I understand what you're doing here. The workflows are really, the workflows get really nasty because I mean, we have big applications here. We have multi-million line applications, like, you know, some of Livermore's applications have upwards of 50 dependencies. And sometimes you want to run two of those in the same environment. And they may have, they may require conflicting versions of the same libraries. And so if you do it this way with the R-path settings, essentially like each process is going to have its own search paths for the environment and they're going to be automatically set on the binary. So you can run those both in the same environment. And we were thinking you could use that, you know, either to set up like, you know, the M subscripts that you run and on the system on a daily basis, or you could use this as like a tool to set up like a Docker environment.  Docker will preserve it after you do that, but you could use this to set something like that up. So question, let's talk about how a SPAC, since the source build system, but obviously most of packages, we're getting like open MPI. Let's just use that as an example. Just a good random example. Good random example. Should I use the version that Jeff just submitted the pull request for this morning? Is that the one you want me to talk about? Yeah, during my research for this, I might have actually submitted a pull request for you, yes. But does the application like build system need to be modified, or is it more like SPAC, you kind of, so like right now, sometimes I'll make like a shell script that has all my auto comp flags and environment variables that I want set for that build. So I can kind of remember what I use. Is it something kind of like that? And it gives me wrappers around it? Or do I actually need to hook it into build system? So I have to port stuff to it. Yeah, you don't really have to port things to SPAC. I mean, you know, package managers were created and RPM kind of sticks to this philosophy that you have this notion of pristine source code. You really want your package to check out the source as it comes from the developer, because that's what most people can get. And then you would have a script that can build that. So I mean, think of SPAC as like it's an archive for all of your shell scripts that you would write to build these packages manually. And it handles things like deciding where the install paths should be, what the versions of your dependencies are, and filling in the blanks if you don't give it some of those things. So when I'm going through and I'm writing this script, does it give me like special variables to fill in? So like, you know, dash dash prefix equals dollar SPAC prefix or something like that. And it fills that in for me. Yeah, I guess I could, I could talk a little bit about what the package files themselves look and so, you know, if we're talking about the open MPI package, it's just it's a Python file. So SPAC is in Python. Inside that Python file is a class called open MPI and it extends the SPAC package class. It's got, you know, some version descriptors in there that say here's version 1.10.0. Here's the checksum for it if you download it and here's the URL for that tarball. And the other, you know, key piece of this is it has a method called install that takes what we call a spec. That's basically a description of how the thing should be built along with all of its dependencies. And the prefix where you should install it. And so essentially, like you're you're given a prefix and you're given a description of all your dependencies and you have to write a script using that to get the thing installed. And that's basically all it takes. Okay, so you said though, like all the features. So okay, let me back up a little bit. So say open MPI is a bad example, because it doesn't really rely on any external libraries. Right. Oh, no, no, no, no, it totally does. Well, yeah, they're all optional though, right? Yeah. All right, so you can still use it as an example, like, Oh, I need to have verbs installed. I need to have libnl installed. I need to have libpsm installed. Right. But it depends on what your hardware is. But you know, there are actually quite a few dependent libraries that open MPI can depend on. Yeah, I guess I was thinking of stuff normally, you don't have more than one version of PSM floating around on a SQL cluster, right? Because that tends to be very fabric and low, low, low hardware specific. Yeah, fair point. You've usually got one set of hardware drivers, so to speak, whether those are user space or kernel. Right. Okay. But even in that, say I needed to have two different versions of PSM floating around inside this SPAC spec file, but they're very nice. Yeah, yeah. SPAC is overloaded.  How do I tell it which SPAC installed PSM to use? Okay, so we've gone to some length to try to abstract the complexity of dependencies from the user. So you get this install method that takes a spec in a prefix. The spec is really your full dependence DAG. So that's the directed asyclic graph of all your dependencies. But for any particular build of a package, you know what your dependencies are and each one has a unique name. So you can just basically say spec sub PSM, assuming open MPI, the package is written to depend on PSM. And it'll give you an object representing the PSM configuration. And then you can just say, you know, dash dash with PSM equals spec sub PSM dot prefix. And that gives you the location of the other dependency. So it's fairly easy to go and extract that kind of stuff. All right. And I'm shamelessly going to keep using the open MPI example here. But open MPI, like we just said, is all about the optional builds, right? The configure script will say, Oh, is this library here? Oh, cool. It's there. I'll build support for it. Oh, is this library here? Oh, it's not there. Well, then I'll just skip it. Unless a human specifically asked for it, like, Oh, build me verb support. Oh, but I can't find lib ib verbs. Well, then that's an error. And let a human figure that out. So does does this back package file have conditional logic like that? Can you do that? Like say, Oh, if they're, you know, this thing exists in the DAG, like I find lib ib verbs in there, then, you know, force that to be built in the open MPI configure script. So the answer is a little complicated. So at the moment, what we have is variance.  So you can add an option to your package. So you can, so what, sorry, what was the example that you used for, for the optional pick any one of them, like verbs, PSM, anything like, you know, I see it's there. So I want to make sure that open MPI build support for that. Right.  And then they can say back install open MPI plus verbs, and they'll get a version with verbs. Now, what we don't have right now, and what we're actually working on, there's a pull request in there for this at the moment, is good support for like external packages. So like, you know, if, if SPAC has verbs installed, then you can very easily say like, okay, turn on verb support, just use that. If it's on your system, and it wasn't installed by SPAC, then, you know, SPAC doesn't know anything about it. So it's kind of hard for it to find where it's located. We're working on ways to detect external packages. And this is actually something that we want to do for MPI specifically, because on a lot of systems, you know, they come with vendor MPI, right, you want to use that.  Okay, so we're talking a lot about source installs. What about like binary installs? What if a vendor gives me something with certain options already in it, it's just like a tar file to just blow out or comes as an RPM or something like that. So you can very easily just copy that into place in your SPAC package assuming you can download it from somewhere. And you can have a local mirror of it if it's not available on the internet. It's not really geared for binary installs right now beyond just, you know, that simple notion. If you can do it in a script, then you can get it installed. But one thing we are actually looking at at Livermore right now, I don't know if you guys know this, but Livermore maintains their own Linux distribution called TOS or Chaos. I guess TOS is the Trilab open source stack, which is Livermore, Lanell and Sandia. And Chaos is the Livermore internal name for it. I think OS there is operating system, and I forget the rest. But those are both RPM driven, or TOS is RPM driven. So if we want to install something as part of the OS distribution here, we really need to make an RPM out of it. So we're looking into ways to generate RPM using SPAC. And that's actually more complicated than you might think. I guess, do you guys want me to talk more about that, or is that getting a little too detailed? Yeah, so actually, I'd be very interested to tell us why is it harder than we would think to do binary package integration? So TOS, Livermore's Linux distribution is based on Red Hat 6. And all these Linux distributions are made to get you sort of a fixed version of something, and to ensure that it's compatible with all the other versions that are installed. But you pretty much get one software stack there. And like I said before, in HPC, you typically want to install these giant combinatorial suites of things, because different codes require different versions of MPI, different compilers. We have several different ecosystems of software in TOS, the way that we actually build it and distribute it to the other labs. So we have the base OS, which is installed via RPM. We have RPMs that go in slash opt. Yeah, so in opt we assume that all the software there is binary compatible with sort of the major versions of different compilers. So if I link something with Intel 15, I'm not going to mix that with something that was linked with Intel 14. So we allow different compilers, but not too fine-grained. And then we have user TCE, which is sort of the TOS computing environment, or at least that's what we're planning on calling it, where we'd have sort of a fully combinatoric stack. And what that means is, you know, if I build a version of Petsy that links against Intel 15.1.2, and then I build another one that links against 15, or that builds with 15.1.3 and a different version of MPI, those are different. And they'll be installed in completely separate directories based on their hash. So what we want to do is we'd like to be able to take the space of SPAC packages, which is essentially anything you could, any DAG you could imagine, any sort of pendant tree that you would want to build, and project that into each of these areas. So user TCE would basically be, you know, hashed install directories for all the versions possible. Opt would probably contain per package directories, but we'd have to project this SPAC space down onto just version per basically only major compiler versions. We wouldn't include all the small ones. We'd assume that those are the same. So we're working on something to do that and to generate RPMs for all of those places so that our developers could write the same package for installing a one-off build in the file system for a user right when they need it. And then they could easily translate that to an RPM to go into the system later on. So can I infer something from your explanation there that you can actually, let's say you're building some package and you want to say, oh, I want this package built against all my open MPI installations or against all the different compilers that I have. Is that something that's, you know, that kind of automation built in the SPAC? Yeah, you can do that. So SPAC has a syntax for all of this stuff to make it easy to install different versions of things just from the command line. So you can say, suppose you, let's go with the Petsy example again. If I want to install Petsy and I want it to build with the Intel compiler, then I would say SPAC install Petsy percent Intel. And I can get more or less specific with that. So if I wanted a specific version of the Intel compiler, I might say SPAC install Petsy percent Intel at 14 and that would get me version 14. And the cool thing about that syntax is that it's recursive. So I can essentially, I can put a carrot on the command line and do the same sort of constraints on the dependencies. So I could say SPAC install Petsy percent Intel at 14, carrot, you know, open MPI at 1.3, if I wanted to build that with that version of open MPI. So what if I need to have like different C flags or something weird like that specific to my machine? Is there a way to kind of like do like a pseudo thing in there to kind of like have dependencies that aren't real? Oh, so like virtual dependencies? Is that, sorry, you're trying to get, oh okay, let me, hold on, maybe we should edit that out. Let me think, what do you mean by the different compiler flags for different versions? I don't know, I mean sometimes on some systems you find that like say on a blue gene such and such it's you know not stable with O3 or something like that. Right, so well you can do compiler flags in a couple different ways. So we just got through implementing a feature that's getting integrated where you can actually add compiler flags arbitrarily to build and those become different versions. So I could actually say on the command line you know plus C flags dash O3 dash G and that'll get added on to the flags used to compile everything in that package. And that that becomes a different version from if I built without it. The other thing you can do is you can just write the compiler flags right in the in the package file. So you know if you were going to do your configure line you would say configure and you'd set the environment variable for C flags to have your special flags right there. So that's the support for compiler flags right now. All right now is that related to I saw in the documentation this concept of virtual dependencies? Yeah so the virtual dependencies is really where your package doesn't depend on a specific implementation something it depends on an interface and so this was added for MPI. Packages don't usually you know require open MPI or MPH right so if I and actually this is one of the problems that you run into with the OS package managers typically they'll install one MPI version and that's all you get and sites want to run with both. So you could have PetSeed and in the PetSeed package file it would say depends on MPI right and that and then the open MPI and MPH package provides MPI and you could link PetSeed with either of those. Now what SPAC does that other things don't in this regime like a lot of package managers support just this basic notion of a virtual deb but SPAC actually versions it. So you can have the open MPI package provide MPI at version three when it's at a particular version and you could have the MPH package provide MPI at version three when it has version three or higher and MPI version one when it's at version one or lower and the package can depend on MPI at one if it only needs MPI one and SPAC will ensure that it links with an appropriate version of the right of the particular MPI implementations and so the cool thing that that does is it lets you compose the packages so if I say you know I only have to have one PetSeed package that depends on MPI with the right version constrained and then I can say SPAC install PetSeed Carat MPH at you know three and it'll link just fine it'll compile with that version of MPH. Okay so who's using SPAC today? So we've gotten a lot of adoption inside Lawrence Livermore it was it was created by me to sort of help students build things faster because we had a long time we spent a long time spending students up for their summer projects we've since gotten a lot of adoption within the compute center here for installing stuff for users and application teams have actually started adopting SPAC because they need to manage all of their dependencies so you know some of our multi physics code teams who depend on lots of different libraries they can now share builds for all their different packages and build the versions they need and that's saved them a lot of time. We've gotten some interest from external compute centers so we're currently collaborating with NERS to get SPAC up and running for Cory and also with Lannell to get it up and running on Trinity those are both I think XC40 Cray systems and we've had some interest from external universities as well so like Inria has used SPAC to package up some of their solvers and we've got contributors to the GitHub project from a lot of different places. I think we had a guy from Stowers Medical Institute in Kansas City who used to set up a cluster and we've had some other people who just used it to manage their own environment so it's getting some traction. I wouldn't say it's super widely adopted right now but we've had some pretty good experiences with users and we're starting to get contributors. So what's the most complicated project that you've got under SPAC right now because the whole point is to make difficult packages easy to install so what's the juiciest one you've got so far? Right now the juiciest one that we have is probably one of the Livermore Multiphysics codes which has I think 70 dependencies or something like that. We're working with Kitware to get a pair of you in there and that may take the title next. QT is in there so QT I think has 40 or 50 dependencies so those are pretty big and complicated and then I guess just you know maybe not for the number of dependencies but for the complexity of the package we've got GCC in there which is interesting because it kind of has it has different dependencies depending on what version of GCC it is so they add you know different external libraries for different loop optimizations and analyses that they do. Something I'd like to ask is there a public repo of maybe contributed packages for things that are kind of basic like GCC the foundation for a lot of things? Yeah so like Homebrew SPAC comes with a package tree so if you if you clone it right now I think it comes with upwards of 200 packages so if you if you just go to github.com slash scalability llnl slash SPAC it's there you can download it we'd really like to and we're implementing a feature to allow external package repos so that sites can maintain their own sort of site specific packages easily and so that we would have a clear migration path from that to sort of integrating your package in the mainline. So what license is SPAC under? It's LGPL. And why did you choose LGPL? We chose LGPL mainly because we've actually had an easier time working with some companies with LGPL because it if they want to contribute to our project it's it's in the license that if they want to release their changes they pretty much have to so we don't have to go through lawyers like we've had bad experiences working with some companies under BSD licenses where you know they have the option to keep the source code private and so they don't want to give up that option and and so they have to go through a whole release process for it but with LGPL we've had a smoother experience. I'm not sure that we have any particular leanings about the license we could look at re-licensing it if someone wants a BSD version. All right and then there's another question that I'd like to ask all software projects just for pure curiosity and we kind of alluded to it earlier already but what version control system do you use and why? We use Git simply because the you know for an open-source project it's way easier to allow external people to contribute on sites like GitHub. I think you know the web tools are just way better and also I like Git for the branch separation and basically just accepting features from other people is tremendously easier than it was with Subversion. Okay so how does someone want to get involved like you know contribute to the public tree of SPAC files for their project or work on SPAC core directly? So I think the easiest way to get involved is just go to the SPAC website or the GitHub page if you Google SPAC-LNL it's the first link and fork the project and start contributing use your own version and and start editing the packages to suit your environment. There's a mailing list so if you look in the readme for SPAC there's a link to the Google group if you join that we have you know I wouldn't say it's a ridiculously high volume list right now but there are there are interesting discussions and we have weekly telecons so those are at Thursday 9 a.m. Pacific time right now but we may be we may be rescheduling those and that'll go out on the mailing list. So where is SPAC in the road of development? I mean are you guys alpha quality beta quality are you production quality what's the current status? I'd say we're around beta quality right now or at the very least with the next release we will be. The packages I wouldn't say are robust across a lot of different systems right now we we test on Linux at Livermore and that's about it. We would need to test on a lot more different sites to guarantee that the packages would be you know a good experience for most everyone who downloads SPAC but we're getting a lot more contributors we like I said we started working with Kitware on on Paraview and we're adding continuous integration locally at Livermore so that the packages will hopefully be more robust very soon and we're working on Cray support so we'll have more platforms very soon. Okay Todd thank you very much for your time can you repeat the location of the SPAC website? It's github.com slash scalability-llnl slash SPAC. Okay thank you very much. Thanks Todd. All right thanks a lot.


So last week we were talking about internet security that is different example protocols that provide security mechanisms in the internet. And there are different ones available, we're focusing on two really. SSL, Secure Socket Slayer, which is what we went through last week. Usually when our applications connect from client to server, they use TCP with SSL, also called TLS, they're about the same SSL and TLS. We use TCP but we encrypt the data to provide security over that connection. So any application that normally uses TCP can also make use of SSL so that the data is encrypted before it's sent across the TCP connection. Encrypted and we can perform authentication as well. And one of the most common places where SSL is used is in secure web browsing, HTTPS. So HTTPS is simply using normal HTTP, the web browsing protocol, on top of SSL. And we went through an example, we showed a wire chart capture last week, we won't go through that detail again, we'll just summarise what we know about HTTPS. When a web server is using or supports HTTPS, then it normally listens on port number 443. A normal non-secure web server listens on port number 80. And the difference is that when we use SSL or HTTPS, we set up a secure connection before you use HTTP. I think we had something like this last week. Normally our client talks to our server using HTTP for web browsing and normally that's on top of TCP. So we create a TCP connection where the server port typically uses port 80 for the server TCP. The client port number is chosen by the operating system and given to the web browser. So that's the normal way to connect across the internet with the web browsing. With HTTPS, we use SSL or TLS in between. Why did I write them differently? From the perspective of this course, SSL and TLS are the same. There are some slight differences among the versions but effectively the same. And the port number used by the server is different. 443 is the default port number. So the server has the same web pages available but it's set up. And I think some of you may remember for your networking lab, there's a configuration file in Apache web server where you can set up the directories and you can set up different permissions on those directories. You use that configuration file to set up the web server to use SSL as well. So you can say this directory requires a user to use SSL. The server listens on a different port number, 443, and the client, the web browser, creates a normal get request, get index.html and some other information, sends it to SSL which then uses TCP to create a TCP connection to the server. And does the SSL exchange of messages to set up a secure connection. And that's what we went through last week, some of the steps of setting up a secure connection with SSL, like this. This is a generic setting up the connection. We have some hello messages. The server identifies itself, provides a certificate. The client, they do some key exchange and then they say let's start encrypting information using this change cypher spec and then after that everything that they send between client and server is encrypted, encrypted and authenticated. So it's encrypted and there's a Mac in there as well. The result is everything that is sent using HTTP, like the get request and the web page that comes back is encrypted. So anyone in here who intercepts the messages, so this is my computer, this is some server in the US, anyone in between who intercepts the messages cannot see what the HTTP request or response is. So if it contains my bank account information or my username and password to log into the bank then it will be encrypted and someone who intercepts will not be able to see that information. So that's the basics of where SSL is used in web browsing and it's also used in other applications. The authentication in HTTPS, there's two forms. First the client, you, the web browser want to make sure that you're connecting to the correct server and similar the server wants to make sure in some cases at least that you are the person who you say you are. So the typical forms of authentication are is that the server uses a certificate. That is when we establish this SSL secure connection the server sends its digital certificate to me, to my web browser. My web browser looks at that certificate that it receives and tries to verify with the set of certificates of the authorities which are preconfigured in the web browser. You've seen in one of the earlier one of the other topics that your web browser usually stores a set of certificates of authorities that you trust. So if the server's certificate is signed by one of those authorities when they send you their certificate you verify it and you check yes this is the correct server it contains the public key of that correct server. So that's how you check your communicating with the server that you think you are. Because what could go wrong if we didn't have that is that we send a request to mybank.com and someone intercepts that and sends a response and they're masquerading as my bank when they're in fact some malicious user. So we need some form of authentication so I know that who I'm communicating is my bank and it's not someone pretending to be my bank and that's using certificates. So it assumes that this malicious user cannot get the certificate of the server because they can get the certificate of the server but they cannot generate it because the certificate contains the domain name. So the certificate identifies mybank.com but if this malicious person is using some other domain name myfakebank.com then this certificate of the server does not match the domain name here so inside the server certificate there will be the actual domain name for that server mybank.com. So using the certificate is a way to stop the someone pretending to be the server and therefore you wrongly sending information to them. Of course often when you contact a server the server wants to know it's a particular person especially with your bank. When you contact a bank's web server and you want to access your account information the bank wants to know it's you that's accessing your that account is not something pretending to be you and the way that they do that is not part of SSL the normal way they do it is using passwords. So the website has some form and in that form the user needs to enter a username or password which has already been registered with the bank when you signed up for an account they registered your username or password. So you send that username and password in a get request some HTTP message to the server and the server checks that username and password. So that's how the server checks who you are using normal HTTP username and passwords and importantly it's encrypted because everything we send using HTTP is encrypted between the client and server if we're using SSL. So that's the basics of secure web browsing and we cover the details that the messaging exchanged last week so we won't go into any more details. Any questions on HTTPS or SSL maybe when you do your summer training then you'll get a position where you'll set up web servers and you may have to configure a secure web server. So I know some people have done that in summer training at some of the companies last summer. So if you're lucky you'll get something in that area. Last topic or last protocol in this topic secure shell. You know secure shell you've used it in a number of cases you've probably used it in the networking lab to connect to other computers. You used it probably to set your username and password on the IT server. Secure shell is a protocol for connecting remotely to another computer or connecting to a remote computer and then performing operations on that computer. In the past there was a protocol called Telnet. The idea is we have some computer on the internet and it may be a shared computer in that many users have accounts on this. So it's set up so that many users can access this and let's say it's in the old days some mainframe or some fast computer and people have their terminals and they want to run some program on this computer from a remote location. So this may be a computer in some office building at work and here's your computer at home. Your computer at home cannot do what you want it to do. You need to log in to this server to run some program, perform some operation. Access some files. So in the past there was a protocol called Telnet and Telnet allowed you to supply your username and password, sends it to the server. The server checks your username and password if it matches what's in the database and once you're authenticated everything that you type on this computer is sent to the server and the server executes that. So on the command line whatever you're typing in the terminal is not executing on your computer it's executing on that remote computer. So it's as if you're sitting at that remote computer. So that's the application that we want to achieve that we can remotely log into a computer and execute some commands. Telnet was popular but provided no encryption and two problems. You send your username and password across the internet from your client to the server unencrypted. If someone's in the middle then they can intercept and see your username and password and then log in as you. The second problem is that even if they do not see your username and password they can see the commands and the responses coming back. So everything that you send to this computer and everything this computer sends back is unencrypted. So that's the problem with Telnet and that's where SecureShell was developed. It was designed to provide the same features as Telnet but to do it securely and secure remote logging. It also supports some other features like file transfer so we can copy a file from the server to your client or even in the other direction from client to server we copy a file. So you have a file on your laptop you want to put it on the server then SecureShell can do that for you in a secure manner in that the login is authenticated and encrypted and the file is encrypted so no one can see the file. Everything will come to a little bit or at the end. There have been different versions of the SecureShell protocol, SecureShell version 2 developed by the Internet Engineering Task Force. Let's have a look at some examples and the details. It uses TCP so we create a TCP connection and we send our SecureShell messages using TCP. There's a protocol for authentication, there's a protocol for setting up the connections and especially doing tunneling, we'll see an example of tunneling shortly and then everything that we send, all the messages we send from client to server are sent using what's called the SSH transport lab protocol. So the functionality that are needed, authentication, setting up the connection and then the encryption of the data separated into different components, different protocols. Similar to what we saw with SSL, even though it's called the SSH transport layer protocol it doesn't do any of the things that TCP does. So they're both in the transport layer, SSH and TCP but TCP does things like retransmissions, sets up a connection with the SIN handshake, many other things like flow control and congestion control. So TCP does many things to make the network work reliably and efficiently. SSH doesn't do any of that. All SSH does, even though it's called the SSH transport lab protocol, it's focusing on securing the data that we send across TCP. So they're both transport lab protocols but they do different things. Let's, what do we look at first? All right, authentication. Let's show an example just to remind everyone what we're talking about. So I'm on my laptop, on a terminal on my laptop. I want to connect and run commands on another computer. So that's what we use SSH for. The program on my computer is conveniently called SSH. On Windows, the application is called what? Anyone used SSH in Windows? PuTTY, PuTTY is one program. So there are different applications that implement this protocol. We're talking about the protocol but there are different applications that implement it and it depends upon the operating system. So one of them, on my computer, there's a program called SSH or in fact OpenSSH. On other systems there are different programs but they use the same protocol. And the normal approach is that we run the protocol and specify the address of the server that we want to connect to. So one server I know is the IT server. Which says I want to, why do I do that? All right, let's just make it a bit bigger. I want to connect from my laptop, the client to the server, the IT server and then I want to run commands on the IT server. So here I specify a domain name which maps to an IP address. And normally when we connect, we also connect to a server port number. What port number does my server normally use? Web server normally uses port 80 or port 443 if we're using secure access. What about a secure shell server? Anyone remember? 21. 22. 22. So secure shell servers listen on port number 22. I do not need to specify that, so this is the client. I specify from the client I want to connect to the IT server. And by default it says connect to port number 22 because that's the default port. And I press enter and then it uses the secure shell protocol to set up a connection and do authentication. Note, first it gives me some warning message. And this is about the authentication. How do I know that the server that I'm connecting with is in fact the IT server? It's not someone pretending to be the IT server. So I just initiated a connection to the IT server. It's possible that someone in the middle intercepted that and respond pretending to be the IT server. How do I know that's not happening? And what this warning message is is I cannot be sure that the computer or the host that you're trying to connect to, this IT or SIT, I cannot be sure who it is. Here's some information that the server has sent. It sent us a fingerprint which is a signature. The server has sent back in response some signature or fingerprint called here. B2, B0, up to 6967. And now it's asking me at the client, are you sure you want to connect to this server? No. Okay, I cannot connect. So now here's some authentication taking place in that the client cannot be sure who the server is. It doesn't know if it's the real server or if it's someone pretending to be the real server. So then it reverts back to the human user to make a decision. Do you really trust this person or this server? And it gives me some information about the server, the fingerprint. What can I do to make sure that it is the real server? I need to know the fingerprint of the server. So somehow I need to get information from the server and find its fingerprint. And then when I try to connect, if the server sends me the fingerprint that I know, then I know it's the server. If it sends me a different fingerprint, I know it's someone pretending to be the server. The fingerprint is effectively the public key. It's just the hash of the public key. So what's happening is the server is sending me their public key and my client is saying, do you really trust this public key? Do you know it from before? That's right. So we need to know that this is the public key. Anyone can have it. So we need to know that it is the server's public key. How do we know that? Well, there's no way to do it here. We need some other way to access the server. I'll show you later the fingerprint. So for now, let's trust it. Let's assume that we know, because we're in the SIT network from here to some server in the other building, that we're confident that there's no one in the middle pretending to be the server. It's possible, but let's assume we trust it. So we say, yes, do we want to continue? And now, so I said yes, I trust it, and it adds that information to some configuration file on my computer so that when I log in again, it will not prompt me this question again. It trusts anything that the IT server with this fingerprint. Now ask me for my password. So my username is S Gordon, and it asks me, well, what's the password for S Gordon on the IT server? So this is the server trying to authenticate the client now. The first part was the client trying to authenticate the server. Now that we trust we are communicating with the right server, the server wants to know that it's the right client, that it's someone who has permission to access. So I need a password that must have been set up in the past. I type in the wrong password. Note the delay here. I type in press enter. Why does it take so long to respond here? I pressed enter, and then there's a delay. It's adding some security. How? You can keep. Yeah, this is it's the server is manually slowing down so that I cannot continuously try many passwords. You see the delay, in fact, I think we've run out of opportunities, a delay of several seconds, which means I can only make one attempt for a password several seconds. So if I want to brute force or guess a password, then with this delay, it's going to take me a very long time if I try many passwords. So this is a password security mechanism of this delay. It doesn't take long to respond. It's just this introduction of an extra delay to slow down attempts on the password. I tried too many passwords, so here's another way. The server says, okay, you've tried too many passwords, permission denied. I can get around that by just trying again. And coming back to passwords before I enter, the server can also log these password attempts, which is another password security mechanism. If the server knows who is making attempts from what IP address they're making attempts, then they can potentially later track them and maybe in the future block them from making attempts if they're entering the wrong password. But let's try to log in. And I entered in the correct password then. It gives me some warning message about SIT, Internet Access. And now, although it doesn't look much different, I'm actually executing commands on the remote server. Everything that I type in is now running on the IT server, not on my laptop. I can see, although it's not, it doesn't display very well when we're zoomed in, but I can see the IP address for this server, <IP_ADDRESS>. I can see every list of files on the server. So I can see some files on the server. So this is the remote login application working. Everything I type now is running on another computer. If I run a program, it's on that other computer. That's what we intend. And importantly, with SSH, when I type something in, when I hit the letters L and S, those letters are sent to the server, LS is sent to the server using SSH. The server executes them when I press Enter. So when I press Enter, the server executes LS. The server produces some output and sends that output, which is this, back to the client and my client shows it on my screen. So everything that I type in and everything that comes back is encrypted. So again, this person in the middle cannot see what's being sent and received in this case when we're using SSH. So that's the idea of SSH, what we're here. So the authentication of the client was done using username and password. You can also, there's alternatives for authenticating the client. You can authenticate using a public and private key. So there are other ways to authenticate the client. Authentication of the server was done by the server sending information to the client, the public key and a signature of that. And it prompted us. We weren't sure whether we trust it or not and it gave us some fingerprint. In fact, on the server, there is a public key stored and we can see that. If you manage or if you're the root user for the server, you can see the public and private key for the server in this case. They're in these files. There are different algorithms used. We know one algorithm for public key encryption is RSA. So there's an RSA private key and public key. But there are other algorithms, there's DSA, the digital signature algorithm. So there's a DSA private and public key and a variant of DSA using elliptic curve cryptography, EC DSA. So different algorithms for public key cryptography. These are the server keys. Just to check. So now I'm accessing the server. If we trust this server, we can check just quickly the signature. If I've got it correct, I hope. What I've done is, and it's not important to command, but I've just listed the signature of the server's public key. The value is this B2B0 all through to 69667. That is the signature of the server's public key. So now when you log into the server, if you know this signature and you log into the server and it gives you this warning, here's the signature of the server. Do you trust it? Then a way for you to be sure that you trust it is to compare it to this known value. If it's the same, then you're actually logging into the IT server. If it's a different value, then maybe it's someone pretending to be the IT server. So the real way to authenticate the server is in advance you need to know this value. So if I knew this value in advance, you remember the first B2B0 6967. If we exit, I'm back on my laptop and try and log in again, see when it prompts me, do you really trust the server? It presents the fingerprint of the server, B2B0 all along 6967. If that matches the actual fingerprint of the server that we saw before, then we trust it. And I think if you can check, you cannot check, you cannot see the other one, but you could check and see if they're the same. So if you see this warning, it's because your client cannot determine if you trust the server or not. It's up to the human user to make some connection there. And in this case, it's not using RSA for public-private keys, it's using elliptic curve cryptography with DSA, just a variant, a different algorithm. Okay, so SSH authentication to authenticate the server. The server has a public and private key. When we try to log into the server, we assume that a client knows the server's public key. Well, if we don't know the public key, if we don't have it on the client, then the client presents the warning like we saw. And then it's up to the human user to determine whether we trust it. So to avoid such a warning, what I could do is configure my computer to have the public key of the server. If that was set up, then when I log in, there would be no such warning. It's automatically trusted. Authenticating the client, password-based. The user provides a username and password. There's also an alternative, which is key-based, where the client has its own public-private key pair. And that's set up and configured at the server so that you can log in without a password. So there are different ways to authenticate the client. This is a generic or a diagram showing a generic packet exchange within SSH. We have a client and server. We establish a TTP connection, and we'll see this in Weishark. So it just quickly goes through. We tell each other about the algorithms we support and who actually we tell each other about what protocol we support, what version of the protocol to make sure we're using the right version. Maybe an old version is found to be insecure. Version 1.8 has a bug in it. Don't trust it. So that's why the client and server indicate what version they're using to make sure that they're using a version that they both agree is secure. Then they exchange information about the algorithms they want to use. So say I want to use this signature algorithm, this encryption algorithm, this MAC algorithm. So we'll see that. And this key exchange algorithm. So they select a set of algorithms. They perform a key exchange. We need to encrypt our information. To encrypt, we're going to use symmetric key cryptography, and we need a secret key. So how do I get a secret key at both the client and server? We need a key exchange algorithm. We'll see the one that we use normally is Diffie-Hellman. We use Diffie-Hellman to exchange a secret. Once we exchange a secret, so both sides know some secret, then we send some special message to generate new keys. We can think the secret is a master key. We generate a session key. And then we, I think this is an optional message, eventually after this we start sending encrypted data back and forth. Let's look in Wireshark to see that. So this is a capture of me logging into another computer. In that case, it was the ICT server that I logged into. So my IP address is 192.1681.5, that's the client. The server is <IP_ADDRESS>, or maybe it's, yeah, I think, .82, .82. That's the server. So the first three messages, these are just the TCP connections, SIN, SINAC. This is a repeat. This is a retransmission. And eventually we get an AC here. So we have SIN, SINAC, but something went wrong in that the message didn't get through. There was a retransmission here and here, and then finally an AC comes back. So that's just TCP connection set up between client and server. Then SSH starts, we'll see some of the messages. First the server sends some identification information to the client, that's this message. Note that the source port number is 22. It's coming from the server to the client, this message. The server is this 203 IP address. Client is 192.168 IP address. What does this message contain? Very simple. Just a string that identifies the version of SSH being used. SSH version 2.0, the implementation is open SSH, 4.7, Debian, Ubuntu, and so on. So just something to identify the exact version of software and protocol being used. This is the server sending first, so it's the server. In this case the server sends this message. And next SSH message, client sends a response. And the client <IP_ADDRESS> sending to the server sends what protocol it's using. And assuming they both are happy to use that version, they continue. So that was easy. The next step is the client sends information to the server saying what algorithms it supports. So if we expand that message, sorry, here we go. A list of algorithms. There are algorithms for different operations. There are, what have we got? Key exchange algorithms. So we have key exchange. We need algorithms for encryption. And in fact we can have algorithms for encrypting each direction to be different. Algorithms for encryption from client to server and from server to client. Client to server and server to client. So when the client sends something to the server, it can encrypt using AES, for example. And the server sends something back to the client, they can encrypt with triple desks. They can choose different algorithms. And what this message shows is the set of algorithms supported by the client. For key exchange, Diffie-Hellman group key exchange or group exchange SHA256. And it has a list of algorithms given in order of preference. Diffie-Hellman group exchange SHA1, Diffie-Hellman group 14 SHA1, four different algorithms. All based upon Diffie-Hellman, just variations. So they are the algorithms supported by the client. And similar for encryption. AES128 with counter mode. AES192 with counter mode, 256 with counter mode. ARC4, which is another symmetric key algorithm and so on. So these are the algorithms that the client supports. Starting with the first preference, second preference and so on. And other algorithms we have, MAC algorithms. For authentication of the data, we use a MAC. The same in SSL. We take the data and calculate a short MAC, a message authentication code based on the data and some input key. So what algorithm for the MAC? And again it can be different in either direction. HMAC MD5, HMAC SHA1 and some others. Remember a MAC and a hash can do similar things except a MAC takes a key as an input. A hash does not. Hash applies just on the data. A MAC applies on the data and with a key as well. HMAC turns a hash like MD5 into a MAC. MD5 is a hash function that takes just data as input. HMAC takes MD5 and turns it into a MAC function so that it takes both data and a key as input. So that's common use for a MAC function. Use MD5 or SHA1 with HMAC. And the last thing is not about encryption but compression. You can also compress the data and none. That's the preferred compression, no compression. Zlib is another algorithm or library for compression. Client tells the server what it supports. Next message, server sends back a response saying what it supports and it's similar. Different set of algorithms. What's the server supports and prefers. And they choose the first matching one that the server supports. So the first one that's common that is the highest in the list. So this is used to choose the set of algorithms for key exchange, encryption, MAC and client to server. Server to client and also compression. Once we do that, we do the key exchange. Because we're going to encrypt our data with a symmetric key algorithm we need a secret key. Both sides need to know it. So we'll use a key exchange algorithm to agree upon a secret key. And they use Diffie-Hellman. I'll just scroll down. So now the next message, the client sends a message to the server. Something about using Diffie-Hellman. Remember with Diffie-Hellman we choose some value. We calculate a public value and send it to the other side. The other side does the same. And from those values we can calculate a shared secret key. It's in your slides where... This is Diffie-Hellman. So this is from an earlier topic. Where user A chooses XA where we start with a public value alpha and Q and calculate YA, send that to B. B does the same, sends back YB. And then they both calculate K. And they both get the same value of K. And that's their shared secret. That's what we'll see in the capture. It's just that in the capture instead of using alpha, Q, Y, A, XA and so on they use different letters for the variables. And the mapping and the notation used in SSH is shown here. Instead of Q they use P. I'll show you in a moment. Instead of alpha there's G. Instead of YA it's E. And YB is F. So we're looking for P, G, E and F. Sorry. This is from the client to the server. And it contains none of those values. It's saying something about what public values should we use or what set of public values should we use. It's talking about the length in the number of bits of the public value of Q. Remember we have two public values, alpha and Q. Alpha needs to be a primitive root of Q. So what is the value of Q? The client says it should be within some size in terms of number of bits. This is in hexadecimal. What's that? Four times 256. It needs to be between 1,024 bits. And what's 2,000 in hexadecimal? I think maybe 8,000 bits. Or you'll work it out if it's not 8,000 bits. But this is the length of G that needs to be chosen between greater than 1,024 bits and less than some value. That's the length. It's not the value of G, it's just the required length. Did I say G? Q in the old terms. P in SSH notation. Here it is, the modulus. The server chooses it. So we have alpha, Q, YA and YB. Alpha, the notation here is G. The value is 5. Q is a large number, the modulus, because we mod by Q. The notation is P in SSH, and here's the value. It's larger than 1,024 bits. It was chosen by the server. So we can see the value of P here. So that are the public values. Anyone can see these. Our user in the middle who intercepts these messages can of course see them, because I've captured, I can see them. So we can see these values. What's next? Where were we? So here we have alpha and Q. Alpha and Q, G and P in SSH notation. And then the next message, the client sends back its public value, called E here. So the client uses the Diffie-Hellman equation to calculate its public value. It chooses a private value, XA, and calculates YA or E from XA and the public values and gets this long value. Sends it to the server, and the server does the same. The server calculates its private value and then calculates a public value, YB, called F in this case, and sends it back. As a result, both sides know each other's public value and then can easily calculate K, the secret. So they both know the secret. And now they use that secret to do the encryption. But in fact, they don't use that secret. They generate another secret from that initial secret. So the result of this, that they get some secret, let's say KS, both sides know it because that's what Diffie-Hellman does. Both sides know KS as a result. Then they use KS to generate... Let's think of this as a master key. They use it to generate a session key and then they'll encrypt with that session key. How do they generate the session key? There's an algorithm or some algorithms that are standard in SSH for doing that. We will not go into the details of how that works. It's on one of our slides, but too much detail. So the idea is that once you have a secret key, you generate a session key using some other inputs. Some session ID, some string, some hash value, and you hash that all together and you get another key. So this is a way to generate a session key. But we don't need to know how that happens. The server also sends back a signature that they'll use in that generation of the session key. Next message is essentially saying, let's use these session keys to encrypt the data. New keys. We've just generated KS. We've just generated a session key each. They should be the same because we start with the same KS. And this message just says, let's start sending our data encrypted with this key we generated. Let's use the new keys. So the next message is encrypted. We don't see what's the contents. We don't see some cipher text. We see the cipher text and a MAC, where the MAC is calculated based upon the data and some secret key. And all the messages from then on are encrypted. We can just see the cipher text and therefore we cannot find out what those messages were. So when I log in now, somewhere in these encrypted packets was my username and password. It was the plain text. It was encrypted and sent from client to server. But of course anyone who captures this intercepts will not see that information. And that's the basics of SSH. We tell each other what protocol we're using. We tell each other what algorithms we support. Select an algorithm. We generate a... We perform a key exchange so that we generate a secret key which is shared amongst the two entities. And then we start encrypting our data using the encryption algorithm we chose and using the MAC functions we chose and optionally using compression. Protocol version, algorithms, perform a key exchange, tell each other let's use the new keys that we just generated. And from then on we will see the encrypted data. No plain text is sent. Or no plain text can be seen by someone in the middle. So that's the basics of SSH. So we have authentication using keys by the server. Usually using them as some passwords but we can optionally use keys for the client authentication as well. And we go through this initial exchange of packets to agree upon algorithms, generate a key and then encrypt. This is too much detail for that. This is just some of the algorithms supported. So defining what all these strings mean. AES we see in there and other algorithms that we may not have heard of. We just saw the wire shark capture. And that's the basics of SSH. We've got one more thing to demonstrate. It won't take long. Any questions on SSH? The last thing will be about tunneling. Before we do that. Everyone can secure shell into another computer. In an exam next week given some diagram about how SSH works you should be able to answer some questions about what's happening. Of course you don't need to remember the detailed packets. You don't need to remember how this works. Given this or given some capture or given some information about what's happening you should be able to answer some questions about what algorithm was used what was the purpose of a particular set of steps. How was the authentication performed? How do you prevent such attacks? So you don't need to remember the details but you need to know how it works if you're given those details. Secure shell lets us to log into another computer and run commands on that computer. It can also be used for other things. We didn't say this is about how we encrypt the packet. The payload is the plain text. The message we want to send. We can press. We may add some padding to make it the right size for encryption. Add some header. Some sequence number. Encrypt. Calculate a Mac. So we saw in the capture there was ciphertext and some Mac value. That's the packet that's sent. Last last topic. What is my IP address? Just refresh. My IP address. So here's my web browser on the laptop and I've just visited a website called whatismyip and that website what the server does is that it records so the server receives a packet from the web browser. Inside that packet is the source IP address. This website simply displays the source IP address on the page. So from the server's perspective the client that contacted it was 203131209.66. So that's a way to find out a client's IP address. Well it's not so simple because the way that it works in many networks is that all of the computers inside SIT including my laptop go through a special device in SIT and from the outside network's perspective from the public internet it looks like they all come from the one IP address. 203131209.66 address. Network address translation and the it can be combined with a firewall and some security mechanisms and in this case even a proxy. But for now this this website has identified my client as this 203131209.66 and found the corresponding domain name for this proxy of proxy.SIT So this website knows that I'm in SIT and they know I'm in Thailand so they know at least my location and in fact you can find more accurate the location based upon an IP address. Let's try something different. We can use SSH to create a tunnel and in different ways. One way is that that's what we'll show is that when I send data out it goes through the SIT proxy and then out to the web server. What I'm going to use is create a tunnel where the data goes from my computer using SSH I'll send it to some intermediate server and that intermediate server will send it to the web server. What we'll do is create a tunnel to this intermediate server. Was this encrypted? This information? Anyone? So the first case my browser using some port number I don't know what it was. Let's make it up. Contact at the server port 80 and the server was in this case what is my IP? And the browser of course is inside SIT and the server identified my IP address and it knows where I am and it's not encrypted. There's no HTTPS it doesn't show in the browser what's used but you'll see an icon in there that's something it's encrypted if HTTPS was used. So what I sent from my computer to this web server let's say in the US was sent in the clear. Let's try a different approach where we're going to send from my browser to the same web server but via some intermediate server and what we'll do what I'll do is redirect what my browser sends to SSH and connect using SSH to this intermediate server and this part from my computer to this intermediate server is called a tunnel. What we're going to do is send our web traffic which is normally sent using TCP but send it to an intermediate server using SSH so it will be encrypted with this portion of the internet from my computer to the intermediate server and then we'll see the end result from the final web server. So what I'm going to do I need to do two things I need to set up a secure connection from a secure shell connection from my computer to this intermediate server so I need an intermediate server turns out I have one my own web server or my own server secure shell connection to it and I'm going to choose a port to listen on it's a special case of secure shell so the intermediate server is sandylands.info it's this middle one what I'm going to do is this minus ND5555 I just chose that as a port number it's going to tell SSH on my computer because this is running on my computer to listen on this port number SSH is going to listen like it's a server anything that it receives on port 5555 it will send to sandylands.info so whatever it receives here will be sent securely to the server here which listens on port 22 and then this server this SSH server will send it and send it out across the internet unencrypted so whatever comes into SSH is then going to be encrypted and sent using secure shell to sandylands.info and whatever that receives it will take the original input and send it across the internet except it will set the source address to be sandylands.info when my in the first case when my browser sent a packet my computer sent a packet the destination was the web server what is my IP the source was my whatever it was 203 address whatever was shown on the website when we use the SSH tunneling I'm going to send it securely to this intermediate server and then the source address will be that of the intermediate server which turns out to be 106.something we'll see in a moment the idea here is that we're now providing some level of security at least for this portion of the network from my computer to this intermediate server and we'll discuss the impact of that in a moment after we do it so I tell my computer my computer to create a secure shell connection to an intermediate server listen on port 5555 yes I trust it provide the password and it does nothing it just sits there and waits whatever an application sends to port 5555 will be sent across this secure connection which application send to port 5555 anyone know which application send to port 80 simpler which applications web browser send to port 80 which application send to port 22 SSH applications which application send to 5555 no idea ok we're going to tell our web browser to send to this port most web browsers have an option I'm going to set up my web browser in the preferences to say in my connection use a proxy instead of sending everything from my web browser direct to the internet port 80 on some server send it to some intermediate address and I've set it up the protocol or the technique for using this is called socks but it's a local proxy and I've specified the port 5555 what that means when I set this whenever my web browser wants to send a message it doesn't send it out to the internet it sends it to port 5555 which is my secure shell program listening to receive information we set that up I've specified the port already here I just chose a port that was unused or was not common to refresh the web page let's hope it works okay now from what I just did I refresh the web page my browser created a request it sent to this proxy port this 5555 port which is in fact my local secure shell client which then takes that encrypts it sends it across the internet to sandylands.info takes what it receives and then sends that the web request in a packet to the real server the real server receives it and the real web server detects my IP address to be<PHONE_NUMBER> which is in fact the IP address for sandylands.info another example you visit google and which country does it normally take you to which country do you think it should be google just google.com I mean Japan okay because the server this intermediate server is in fact in Japan so from google's web service perspective it's received a request from a computer that it identifies as being in Japan and therefore it sends back the Japanese google web page to my computer so this is one example of using ssh for tunneling what we've done is created a tunnel from my computer to some intermediate server and then access the rest of the internet via that intermediate server I always know my own IP address no there's no way for google or this server to know this one's IP address they receive an IP packet where the source address is this computer it's acting as a proxy in that it's sending on behalf of me and as a result we've got some form of hiding our communications the end server that we're communicating with now doesn't know exactly where we are they think I'm in Japan but I'm here in Thailand okay so that's some form of security in that it's hiding the location of the user normally the location can be determined based on the IP address the other form of security we've encrypted the data going between my laptop and my intermediate server not across from my intermediate server to the end server that's unencrypted let's say I don't trust someone inside SIT okay normally when I send something within SIT it goes from my laptop unencrypted through SIT's network and then out onto the internet but let's say someone is listening into my communications inside SIT so I don't trust someone inside this room for example so my computer encrypts that information SIT's network to some outside server but it's encrypted anyone who intercepts this cannot see what the content of that message is so again we've got some form of confidentiality just across this portion of the network so that's an example of using SSH for this purpose of tunneling traffic via an intermediate server there are other ways to do tunneling but that's the only example I wanted to introduce so it's a common way tunneling is a common way to connect to another network and to pretend as if you're somewhere else or someone else from the server end servers perspective it's receiving data from this intermediate server where in fact the original source is my computer any questions on that quite quick to go through that demo you can try it yourself but you need an intermediate server to do it in this case don't worry there won't be an exam question on that there won't be an exam question on tunneling and generically the tunneling is like this where normally our application uses TCP directly that's in the top part with SSH we can redirect our application traffic via SSH there's what's called local forwarding and remote forwarding in this case I did remote forwarding there's an alternative where we can do local forwarding and it's a way for example to bypass firewalls if a firewall and it will be strange but if a firewall blocked port 80 so you couldn't browse the web but allowed port 22 so you could send with SSH then you could use a SSH tunnel to somewhere outside and access the web from there so it's a way to bypass different blocks from a firewall yes yes there's a proxy in fact you can think there's two proxies this is acting as a proxy for my browser it's intercepting and you can think yes this is a proxy for my connection from here to here someone acting on the behalf of some original source there are proxies available on the internet which do a similar thing but usually so they are some intermediate server that you can send your traffic via which then send on to the final destination in this case our proxy is using SSH but a proxy doesn't have to use SSH a proxy has different functionality in this case we're using SSH yes can someone answer that does Windows remote desktop connection use any secure connection I don't know I think with Windows I don't use Windows so I honestly don't know I think that in the past there VNC for example is a generic protocol for securing connection remote desktops and yes that uses encryption most likely it's an option for Windows to use it most likely it does by default in the latest versions of Windows but you'd have to check that to be sure okay finished finished the course