Well good morning everybody Thank you. Glad to see people are awake Given my sleeping patterns I may pass out during the Presentation someplace here, but hopefully you guys can prop me back up and we can make it through it I'm gonna switch over now to to the presentation We've just sort of been scrolling through some of the IBM sessions that you can come and see during the track today We had a nice video for you, but I'm not gonna trust going through the network to actually play it right now So I'm Todd Moore IBM's Board of Director member here at the OpenStack Foundation I've sort of been with it since the beginning here having worked with Jonathan and Jim Curry and others to shape where we were going with the foundation aspects I'm based in Austin, so I did quite a bit of consulting to put together what the foundation would look like and Help write the bylaws those things OpenStack has been just a phenomenal Undertaking with just rapid growth that you we never would have expected the You know last conference In Atlanta was you know extremely well attended, but if you look at what we're doing here in In Paris, obviously it's a great venue people want to come But we you know we had 4,500 online registrations will have over 5,000 people by the end of The summit just incredible growth and only two years and I think it speaks a lot to the quality of the community and The exciting work that's going on here in cloud If you look at the the voice that we have the foundation collects up marketing information It's it's really a discussion now in the media around OpenStack and and Amazon And I think that really attests to how rapidly people have decided to come in and Adopt and use OpenStack and these days our customers come to us and they say hey We've decided to go with OpenStack And therefore can you help us? Can you show us what to go and do so a lot of what we'll do in the IBM track today? We'll be to talk a little bit about how we're doing it helping customers along the way So with me is is Dave Lindquist Dave lunch and stand up and say hi Dave is our CTO for our cloud and smarter infrastructure work and an IBM fellow and He'll be helping me here today with the presentation so Why don't we get get started? Clicker So OpenStack, you know that this phenomenon that we have here right if you saw the the kickoff that The key note Jim Zemlin did from the Linux Foundation, right Jim Jim highlighted the Impact that open source was having but it's not a new phenomenon All right, this is something that you could trace your roots back say to the HTTP server that Really became the fundamental building block that you know businesses like IBM's web sphere application server built around and every Other industry player who built application service, etc around I grew up around and and it was because at one point We had many many different implementations all of us were doing something different and we realized that Gee, there's a piece of plumbing here that we really need to share and use to Get to commonality that as a result of that will allow everybody to interoperate Make the experience much better for the user and as a result of that a big marketplace build up around it And you see that time and time again as we've progressed through time, right? And along that large arrow there. There's many other open source projects that we see having a real impact now on the industry Whether it's mobile social big data or cloud Open source projects are are playing an ever-increasing dominant role of setting essentially the de facto standard We're not out in some de jure standard body sitting arguing over parentheses and dots and you know What we're going to do in structure in the architecture We're in here working on real code that works for people that as a result of that is a Layer that everybody can count on and and then build a business around it's it's it's a different way of getting to the same end point It's very quick It moves, you know at lightning pace with all of us involved and it's it's that change to that model that is has really turned things around So as we look at At open stack We see open stack as a fundamental technology, but we see other technologies around it that are also having You know a major impact on the cloud environment There's essentially a cloud open cloud platform that is coming about As a result of first the work that we see an open stack, but then bringing in technologies like docker Whether it's docker as a service on top of open stack or docker underneath open stack Providing a way of of getting to you know rapid high performance Infrastructure underneath it not having to start up a VM every time you want to take something down and move it Those types of things that that docker can provide That will you know give us different usage patterns that will enable Our customers And all of you to to get to the kind of experience that you need for the application that you're running And of course We're seeing a layering and we'll talk about that a little later In more detail, but at the paths layer above open stack projects like cloud foundry just have Communities that are exciting to work with that have Attracted a large base what we found when we're working with With any open source project. It's a community that matters more than anything and You know, I thank all the folks in open stack for how well they work together And how much energy they put into the community because that's really what makes the difference here And in the paths space if you're looking for an exciting community to go and work with cloud foundry Is that community and they're getting very close to actually standing up their foundation much as as open stack is So it's an exciting time to be using open technologies in cloud So no matter where you're coming into this picture, whether you're off-prem on-prem how you're trying to integrate There's a set of open technologies underneath that that can be used so that you get the flexibility you want You don't get locked into a vendor and you can come and be part of that community and and add to What's happening and I think You know from our point of view that's that's the exciting thing to do so as You know founding sort of platinum sponsors here in in open stack We've had a long history of being involved with you know the variety of projects Some people call us the adults in the room sometimes because we work really in Primarily in core projects at least at first we've expanded that that out but You know our emphasis always comes down to what are the things that are either, you know enterprise customers or service provider customers are asking for Oftentimes that's resilience hardening work that we go and do in security that The result end result of that is people can count on the infrastructure so For instance, you know as we got going and in the release as I'll say Grizzly, right We we worked there to really stabilize apis we work to improve what was going on in the infrastructure for security Add services like olof that people were looking for In You know projects like nova You know we came up with 21 percent of the design enhancements really To help people see the value be able to extend it more quickly Uh And as a result of that Stability started to happen And we continued that through Havana our contributions there again included things like sammel and The basic support for federated identity You know we had to do staging around that. I think people who watch the keynotes Saw cern in in that activity come up and talk about the federated identity. Well back in Havana We were laying the Building blocks of that into there in order to be able to go and do that and We'll talk more about the the project and collaboration that we've had with seren in there And of course as we moved on into ice house now IBM has had a significant role been one of the top contributors into the The code base and of course, you know, we continue to improve the quality of the code improve the test case coverages Do the internationalization work do a lot of the heavy lifting underneath in Building that uh that functional base that everybody can depend on So it as we've gone through this time IBM contributors have continued to grow and uh, I think in the last release we had led a hundred and nine 109 in in the last release cycle of IBMers who were, you know directly contributing Into the projects, but of course we have a much bigger group of folks Who's sitting back at the ranch working on the products that that we go and produce Um in total, I think we've got a community of about 1200 folks who actively are Participating in an open stack in some way shape or form. So it's it's a big deal for us And uh, we're very happy and proud of what we're doing here Okay So on did you know one of the The key use cases that that continue to come back to us as our customers are looking To be able to do hybrid Cloud right and and that's not always easy Especially when it comes to making sure that what you're doing is secure Right, you want to be able to essentially have single sign on between the instance that you have In your enterprise and the instance that you have out in your your cloud say you want to have Something that spans out into the cloud So the work that's been done in security, especially brad tople here and the team see martin elli others To help bring that support into into keystone has is just Enabling that use case that our customers have told us time and time again. They want to have and You know often the cases Enterprises would love to have something on site But then they want to have that ability to either flex out into the cloud Or they want to be able to do some work out in a development environment in the cloud and then be able to easily move that back on Prem and and that Work is is essentially Making that happen So we've been the number one contributor in cinder across all releases I think we were 33 of the cinder work here for this last release We've added of course some Additional work on cinder storage volumes Uh next chart. We'll talk a little bit more about cinder But you know ibm has had a consistent influence in the cinder project moving that along making it Useful for for everybody Again, it goes back to our central team hardening the core making it What our customers are asking for There's been a lot of upgrades into horizon I don't know how many people use the horizon dashboard anybody using horizon Good, okay Having a consistent dashboard and experience is something that's really important And making it perform it is very important And with the work that's been going on in horizon We're allowing you know quite a bit of more work to now be done down in the client So the experience is much faster. We're not doing server side rendering In compute Now there's been a lot of work in nova to Help Enable of the future right to to be able to then quickly add new extensions into the apis And and that work has has really been going on here in juno and that will set the future for us as we go forward and then of course Interoperability is is really key The work that's going on in ref stack and def core if you haven't been paying attention to that Is is just essential for our future because that's what's going to allow customers to be able to pick and choose and move between clouds And having that consistent interface across them is is going to be so so important So in in ref stack IBM has been the top contributor in there 75 of the code that is is in ref stack has come from IBM And uh, you know, it's it's it's something that will allow us as we move forward To be able to test in with everybody's instances that essentially they they will interoperate that they have the apis And they behave the way that you want them to so That Coupled with the def core work with defined sections of code that and capabilities that every vendor needs to to have Will essentially now start to enable that flexibility interoperability Lack of vendor lock-in that people want to have so Those are the types of contributions that ibm is trying to make into the organization I just wanted to there's lots and lots of of things that we've done I want to just sort of hit a couple in a little more more detail as As some of the team was around and and We thought they were kind of important for the use cases our customers have been talking about So, uh, we I said a little bit about keystone and the federation work Steve Martinelli, Steve here. No, Steve's not here Steve, uh, from my team, uh, worked, um Who was it from, uh, CERN? I'm sorry then Merrick Denny They partnered up to do the federation work that you heard about on the on stage That's that's where the code came from and That federation support is of course allowing CERN to then be able to link up instances and they're very excited about it They of course will be going off to the summit in vancouver They got a free trip and i'll make sure that steve makes it to vancouver as well too Because I think he deserves it for the the work that he was doing there Um So we work with a lot of of folks in regulated industries Um, especially uh, banking industries as an example Auditing is something that comes up All the time when you you talk to folks It was one of the first things that that customer set came back to us when we brought open stack out to them And uh, they said we need this. What are you going to do about it? Your IBM help us so We'd been involved out with DMTF working in the cat of standard and Saw the opportunity to take advantage of that work. So here's a example of a standard that was being developed actually coming into open stack And as a result of of that building that into The support that we've been doing in in keystone Our customers are very excited, especially in the regulated industries And I know those guys don't get excited very often That work has also been extended into what we're doing with federation so that You know the logging that's happening on authentication, etc is is now Being picked up as part of keystone as well too And that will again enable that hybrid use case across cloud In the dashboard Our team has been working with Very popular technologies like jquery and angular j s IBM's actually a founding sponsor of jquery And we've now gone and improved the horizon dashboard by including that support And as a result things render much more quickly on the client side And we think the experience has been greatly improved and it's much easier to now code and It make changes and do customizations the way you'd like to So I mentioned a little about cinder so we've got volume replication support now That means when you've got a problem when something fails and crashes you can reconnect into that You know remote backup that you've gone and made Seamlessly right at high speed so that when you do have a problem and you need to reconnect Cinder now is an enabler and doesn't get in your way We've also added NFS support into the elastic storage work that we've been doing there And and that's something if you're you're looking at Elastic storage you can should go and take a look now. There's some posic work Work that we've gone and done there In orchestration heat hot templates The you know We see you know, for instance, we've had a lot of discussions with giga spaces recently, right? brad here in the front and The demand for the sort of common template common model has come across very strongly and as a result of that We're seeing You know sort of an influx of people very very interested in toska Tosca is a standard that's out there in oasis another instance of a Standard being brought into what we're doing And as a result of the work that the team has gone and done with toska We're now able to you know Deploy on a template model And and we think this is is is really going to to take off as we move through time here Couple of things that we did we provided a translator now that will for instance allow us to translate To juju charms and we're doing some interoperability with that mark shuttleworth was in here in the previous Presentation we've been working with mark seam matt rickowski from from our group and and tomas spitzer and others have have been helping to go make that happen and We've also added in cli not very exciting having a command line interface, but for debugging it's exciting So now we can incrementally do debug and and and work with templates and not have to restart everything and And for the guys who have been working in heat and hot. That's that's been a great feature And of course, we've expanded that Metering work and the catf support right into salameter as well too. So You know a long laundry list of things up here continued effort by ourselves to go harden build up the qa Make the the projects better and of course Handle the enterprise use cases that that our customers have been asked for So at this time I I want to bring up dav linguist Dave I'll hand this over to you. You can go and advance it Thank you, Todd and thanks for the overview of The areas that we've been investing in Within within open stack as Todd went through this I hope you saw a lot of core contributions and what we consider The critical areas that do need advancements in an open stack in particular in the security space As well as an audit as well as in some of the things we'll see in compliance This is these are the areas that are continually being pushed on by enterprises And we've also contributed a lot of technology a lot of code with the community And really furthering how we build a highly distributed share nothing I as environment we pretty much look at all our investments And open stack as how can we Participate within the community to continue to drive open stack as the leading I as in the industry and and I think arguably Open stack Is a shining example in the cloud space of driving communities and driving and driving open technologies Um myself in partnership with angel Diaz and Todd have really been the primary sponsors in IBM For years now and driving our participation in open stack our participation in cloud foundry our participation in docker our participation In the oasis standards particularly around Uh topologies in toska as well as Critical advances in w3c and oasis and link data And in apis to support lifecycle federation of data things that'll keep emerging and importance As we start driving higher and higher in these levels of an api economy Now if you look if you step back and you look at what's going on in the industry You look at the way businesses and in particular application developers are starting to use Cloud to accelerate the delivery of new business processes new business models new business applications And then you look at how the it shops and how the clouds public clouds provider clouds are forming To make apis and services available And then you start looking at how it's being hosted on an infrastructure model You will have seen over the last few years a very natural architecture has been emerging An architecture that's been emerging and evolving and maturing at an api layer Often returned as an api economy at the top At an operating environment or a past layer To really accelerate with the composition of services And of course with this community at the infrastructure layer more in a software defined environment It is critical it is critical for the acceleration Of the creation of these applications The portability the ability to support hybrid clouds and for the many advances that we have Core growing vibrant communities at each of these layers So at the bottom obviously open stack is the center of a core vibrant community for the infrastructure as a service The middle layer We've teamed up with a number of companies On cloud foundry at a past layer At a platform as a service layer that community is growing very rapidly In the acceptance from developers the growth we're seeing with offerings like blue mix It's just phenomenal the developers love this environment If you haven't gone out there and used blue mix encourage you to give it a try You'll see what what the excitement is. Why are developers so excited? You'll see how fast and rapid you can compose An environment from run times to persistence data persistence to management to analytics to higher level Social cognitive services can come together in seconds And you can begin to construct your application You can begin to bring it out to market to the customers you want to use it test it iterate in a highly iterative dev ops manner Using dev ops services that are part of this offering that create a very collaborative development environment for you And at the highest layer in the apis we continue to drive towards open apis You're constantly seeing that many of these applications That are being developed are really A composition of services for the core run times, but also they're composing apis to build these applications They're leveraging services that are opened up as apis running all different places around the internet That's how many of these applications are created so rapidly So if you look at some of the core offerings That ibm has brought forward Recognize there are a lot of offerings and a lot of names. So let me see if I can start clarifying The core sets of technologies and how to think about it When you look at open stack open stack itself We deliver open stack as software Through a cloud manager with open stack It is basically how you get going very rapidly with an open stack four-year private environments within your data center it supports All it's basically juno horizon interfaces and it also supports multiple regions. So it's it's Really a great way to get started a great way to start delivering the value of open stack and bringing it into more of a hybrid construct That next layer We have an orchestrator Think of the orchestrator as the ability to deploy workloads Into the different cloud environments supporting open stack deployments So whether it's public shared environments private managed environments Or within your center. That's what the orchestrator is doing in conjunction with that deployment It is managing the policies. It's managing the integration with your it processes It's managing the automation to set up so you're monitoring events backup recovery all the Elements you need to manage the lifecycle as your applications roll through that progression Of early dev through various levels of tests pre-prod and into production. That's what's going on with the orchestrator Recently and we'll talk a little bit more about this in a number of sessions. We've announced a IBM cloud open stack services That is in the software environment a managed Open stack as a service. That's what it is For businesses, what's exciting in addition to having a managed open stack Is it's running in the dedicated A portion of open stack. So in other words, it is a private managed cloud For enterprises to get going very rapidly But also have 24 by 7 and sla's set up for that open stack environment So very exciting very new offering. You'll see a lot of add-ons Capabilities being added on that environment with best practices around workloads Extensions into the past areas extensions into the open api areas But this is this is something we're very excited about and with the pura system Think of that as a integrated system. You're integrating software and hardware This is all about workloads and workload optimizations And one of the core things we continue to evolve that code base with open stack One of the core things we're opening up inside there is support for the heat heat engine So you'll continue to see us push Heat with hot templates as a way to represent these patterns and deploy these patterns where they're deploying them With software with the cloud manager for open stack with the orchestrator and these hybrid environments in Managed environments with software or in integrated systems I know that's a lot a lot of options But those are the types of dynamics we see and the requirements coming in For enterprises to have that type of flexibility and that's how we're leveraging open stack through the breadth of our offerings Okay Looks like something died here Maybe the machine Beeping you'll go At the at the api layer a couple of things I want to want to note a lot of discussion Software economy a lot of discussion api economy When you go into business units and all industries around the world What you'll see in the excitement in these business units is how cloud is accelerating their strategies is accelerating their ability To go from ideas that they've envisioned Around new applications new business models new processes to instantiation. That's what's going on. That's what's creating all that excitement If you look at studies, you will see the companies that have embraced these types of open technologies and that have embraced in particular Cloud as a strategic part of their company. They are leaders in those industries They are disrupting those industries very rapidly. You can go through example after example What's happening To accelerate this instantiation. Of course, it's composition of the infrastructure. That's a core Of course, it's pulling the services together and how I compose the services the run times, etc But what's fundamentally accelerating it is this api economy when you look at just look at any typical Mobile application you will often see at least six if not a dozen apis being used for x from external systems You will see obviously payments. You'll see access into core data systems You'll see social Trending feeds coming into these applications typically you'll see them hooking into monitoring systems You'll see them hooking into security systems. What they're doing is composing Many aspects of that application through apis. That's why it's so critical to continue to keep these apis open And to continue to drive open technologies and open standards in this domain The platform layer The platform layer Is largely about an operating environment. It's largely about composition of services It's largely about creating an open ecosystem So when I say composition of services when you're developing an application, you're going to pick a runtime like node or ruby You're going to pick some data persistence like no sequel like like a mongo or cloudant You're going to pick maybe a relational A store depending on that application. You might need messaging requirements You're going to end up needing monitoring requirements log analytics security identity access control, etc The the platform layer what happens is the platform layer allows you to rapidly compose those services And in conjunction with the composition It integrates those services in the context of your project And with dev op services, you then have a collaborative development environment to manage Your versions or builds or deploys, etc in collaboration with other developers all integrated into this platform So folks can contribute to your application to your project In a collaborative manner all the tracking and governance that you need Into this project and then you can then make these capabilities this application available To the field to your customers subsets Broadset and you can start doing things like ab testing and start iterating towards your business outcome You couple that with an open set of apis and api management and integration with data sources And integration with other applications software as a service applications All of a sudden you're seeing why businesses are so excited and why there's so much momentum Around these open ecosystems in these open cloud environments and how cloud is changing their business how it is a fundamental part of their Of their business strategy excellent At the infrastructure layer preaching to the choir here obviously With the open stack community To me some of the things that are critical here are obviously the software defined environment the core compute storage networking What it's what it's affording us is the ability to rapidly compose The infrastructure in support of the needs of a workload And that will evolve over time over time we will develop more best practices more understanding of performance And workloads and backup recovery and security and you'll start seeing Best practices evolve with policies and patterns to support Certain types of workloads like a Hadoop workload like a transactional system like a mobile system Though that will start manifesting itself in patterns in support of these passes and then in support of this api economy We I think are at a very interesting point In the open stack community as Todd mentioned earlier a few years ago And i'm sure many of you are in this situation You are often asked by your customers your clients to start contrasting the different techniques proprietary stacks Open technology stacks different open technology cloud stacks And a lot of what we were leveraging was the architecture of open stack The design of open stack the community investing in open stack the growth that was occurring there at this point Over the last year the conversation to me has very much shifted to we strategically have chosen open stack Can you help us How and when to apply open stack to the different types of workloads that were that that were that were targeting So it's become very much How do how do we accelerate now the deployment and the use of open stack to these different workloads? And how does it help us in this hybrid emerging hybrid environment that that we're all See occurring with our with our workloads So that's to me why many of these core projects That us in the community are investing in are so important Many of them are very mature Some of them not as mature as it need to be the networking space We do need more focus in the networking space With neutron in particular Apis feel right the investment the community coming together feels right, but it does need maturity There's a few areas that we will find that start popping out and we'll have to nail that as as a community To really start accelerating the use of open stack at scale Okay, the cursor now Hybrid a couple of comments. I wanted to make on hybrid It is real it is happening today. It has been happening And it continues to evolve in a very rapid manner When I discussed these composition of apis or these services, these are running in multiple clouds around the internet When I work with customers my colleagues work with customers and I work with partners with customers What we see are a couple of requirements coming in over and over again. One is I want the ability To figure out where best to run my application. So I need a level of portability I need to support compliance. I maybe I have regulatory requirements on privacy or data Location, so I need the ability to place information and data or my workload in certain places I need to understand performance in slas for that placement I need to understand the finances. Maybe at some points, maybe in early development I'm looking for the for a cheaper less expensive place to host. Maybe as I accelerate this into production I start monetizing and running at scale. I'm zeroing in on slas I'm after the performance the backup the recovery the 24 by 7 So those are the kind of dimensions that you see in placement and the workloads themselves As mark discussed in his keynote. They are distributed heavily distributed You're constantly seeing what the industry likes to call system of engagements these new mobile social big data dashboards business processes being developed in a very rapid it model on cloud with this composition hooked into Core data systems and transactional systems more of a system of system of record model What that drives is not only placement. Where am I going to place this system of engagement? But how am I going to integrate these systems? How am I going to do the application and data integration? So sets of integration services are emerging and that's an area that we're investing heavily in is these integration services in this new model In addition to that you're seeing a push in brokerage brokering in management Not only brokering. How do I understand where is the best place to place my components elements in my workload? But how can I set up the policy? How can I set up what I need in events and monitoring and insecurity and backup and recovery? How can I hook this into my change my problem my incident systems? These are the things I need to run this workload in production in my environment to to have my business counting on it So these are the types of investments that when you go through any of our offerings You will see we are continually pushing the envelope on these hybrid capabilities How do you bring this two-speed it together around rapid iterations on the front ends? and optimizations deep optimizations of security workload management, etc on the back end and bringing analytics in to understand the Optimization so I can predict adages so I can optimize placement And so I can do extensive search and try to debug and figure what's going on That's what's happening in the hybrid space so a couple Couple offerings that I mentioned that I wanted to highlight And you'll see in our subsequent sessions or at our booth These these technologies and offerings The teams can take you through These capabilities can take you through the experience what you'll see the value that you get out of them as a user um, I'd mentioned the open cloud I mean the the cloud open stack services earlier again think of this as Managed open stack The dedicated portion so private of software very exciting offering Many many businesses Very interested in this capability The IBM cloud manager with open stack It's fully embracing open stack software distribution deployment configuration All the support you need with support for multiple regions the hybrid capabilities And one of the technologies I wanted to mention briefly before I introduce Brad Topol Is in the area of object stores a swift has been around for a while Swift is a very good object store And so one of the things we've done is brought the object store into the platform as a service layer So you will see this You will see this service as a composable service within the blue mix environment Within our platform as a service environment Which if I hadn't said earlier is at its core Based on the open community around cloud boundary So this is furthering the open ecosystem of services bringing swift in into a platform as a service layer And when you start thinking about the options that creates for us As an object store in a platform with some of the things I mentioned in software You'll start getting a feel for the types of investments we're making in the hybrid environment How I can begin to set up policies And begin to have my data persist In locations and regions where I'm comfortable or meets my regulatory requirements So you will see continued advances now In these data services and these integration services These compliance as well as management management services So with that I'd like to introduce Brad Topol. We'll take it quickly through a demonstration Of the uh swift service Thank you, Dave I'm gonna get it started. Yeah, just give me a second. No, I need to talk for No pressure Thank you Yep So this was really exciting, you know, we you know and Dave really touched upon this this this environment We call it blue mix. It's based on cloud foundry. Whereas a developer You're able to create an app and just push it out into this world where there's all these available services that you didn't have to Go configure. You didn't have to get set up. They're all born on the cloud ready to scale And um, you know, and we're not the only ones that saw this Catherine Spence from intel a couple days at the summit gave a cloud foundry presentation They're seeing the same things as well And so you you you as an you want to have all these wonderful rich services available And the one that we saw we needed was object storage Now conveniently, uh, we also had an object storage service available from software based on swift So really all we had to do was put the two together Um, tongue leaves here in the front row raise your hand tongue Tongue's the person that that pulled this off and so we're going to show you what this looks like So this is logging into blue mix again, uh, you're going to see an environment here Where I've pushed out my application into my blue mix environment Uh, after I sign in and you're going to also see the services that are available Um, again, the beauty of this is how quickly you can, uh, Dynamically bind your application to whatever services you want and then with blue mix The other thing that we're adding is just the huge list of services that you might need in a variety of Categories So here you can see we're logging into our blue mix dashboard. We've got our demo application to show off swift and object storage Um, you can look at the different instances that are available They are just positive for quicksack Um, so what's interesting is again, you know the application you can control how many instances are there Um, and what services are it using it's using our object storage service And any of these services in blue mix they they they know how to scale themselves So even though there's already one instance here, we're going to go ahead and add another instance And and that's what you're going to see in a second Uh, and one way you can do is we're going to go through the whole catalog And these are all the different services that are available today on blue mix just you know The beauty of this kind of environment is pull out your credit card and get going Compare that to using your local it and how long it's going to take them to get all this stuff up and running Um, we've got documentation available. So we tell you about the swift based object storage service and how you can get started as a developer Um, and here's the magic We're going to bind that object storage service to the application And say hey, we want to make it use this one There's a restaging where it basically restarts that application with a new service instance that was just added as you know as we saw there And so it's going to push on to production again, you know, like kathryn emphasized and dav emphasized You know, you you you push this application and the developer doesn't worry about necessarily where it's going to run It knows it's going to be pushed out into production and and you know, he doesn't he or she doesn't need to worry about all these details And once that that that application is up and running you're given a url to go get the application And I hope folks are familiar with open stack swift. It's object storage put and get objects in containers So we wrote a little application and here's tong longing in Uh, so he logs in uh to the little application that we wrote that's running on node j s And once he's logged into the application It's going to give him the ability to use the software swift that was provisioned for him He can go ahead and create containers So we're going to create a container called files And We can go ahead in the container Go ahead and browse and upload push up an object And so we're going to push that one up right here And once after it's uploaded, you know, you can you know, you know, Swift is a really nice simple API You can pull it back down and not surprisingly tongue. It's a Chinese text that I can't read but he could translate Um, and then of course other basic Swift operations are available deleting containers deleting objects and and what have you So that's uh pretty much the whole demo and if you needed to delete your object storage instance You can go ahead and delete it as well. And that's what we're doing here So so in summary We'll just stop that in summary you a great environment for developers to rapidly just do their small piece of the world Have all kinds of services at a variety of types of services available to hook into And then very quickly go from development environment to push to production and controlling the number of instances And having born on the cloud scalable services Okay Thank you brad So, yeah Swift, you know seeing Swift mature and the things that we can do with it now surfacing up as a service through blue mix Making of course use of all the other blue mix services that are out there and I know it's scrolled through the catalog very quickly It's something you need to just go and look at What we're seeing are you know our users in terms of the feedback through blue mix say is that You know, this is a just a tremendous environment. We can go we can get things rapidly prototyped We don't have to go back and try to get the services provisioned in an rit environment But we can we can take what we create out in blue mix and then deploy it back into what we're doing on prem And we love it. So it's something to really go and and take some time if you've got it Sign on get an account. It's a free 30 day trial and And give it a whirl Okay, so now that we have no keyboard i'm going to try to restart this Come on And we lost the page that we were on Yeah, it's it's not clicking though. There we go. There's our demo screen Brad, thank you So, you know, I wanted to again hit the def core ref stack topic just one more time. So, you know, I I don't think I can talk about this this too much Uh We've we've gone and we've done some definitions of capabilities and user defined sections of code That are out there for review right now The boards voted on uh in a previous board meeting A set of capabilities that we view are the advisory capabilities the ones that Our customers are asking for the most that are in production the most that we want to be dependable Obviously, we want to enable forwards and backwards compatibility as we go through time So, uh It's out there. It's in the public. You can go and pick it up from looking at the the wikis at the orc in the it's in the minutes as well in the governance wiki on open stack dot org And and take a look at it spend some time looking at what the The team has gone and done there because it's it's worthwhile and you can start planning what you're doing around it now And we're giving these advisory notices to the vendors So that when they want to use the trademark in the future and ref stack becomes a testing infrastructure that we use to go And verify that they know what they have to have you'll know what they have to have And and you can now take advantage to that interoperability that we're creating here So, you know, we're all in in terms of def core and ref stack We think this is one of the most important things that we can go and do for the organization and We're happy to have other people come and join us in in this this effort here IBM is also working now to donate some servers to help Stand up the infrastructure and keep it going as well too So you'll see that icos service that we were talking about that's been launched as A set of now servers that will will go in task to help with the ref stack implementation so So thank you We have some sessions that are coming up They were scrolling by quickly as you came in the room Uh starting at 11 50 over in 2 12 and 2 13 You can come and See a little bit about what we're doing Just a little bit beyond the code I'll get those sections as we maybe it's easier just to step through the pitch here So Manuel and Daniel who are here guys wave your hands I see you over there How you doing? Are gonna come in and talk about You know collaborating out in in the meetups Bringing together the community strengthening it helping others understand and learn what's going on there This is something that we do is give back into the community and if you're Interested in figuring out how to go create your own meetup and get started or or help others Or just where to join in please come and and spend some time with the guys Oh come on Here we go Uh then moe abdulla is is going to come in and give a little more in-depth tour of the ibm offerings Uh, whether it's you know on-prem or off-prem Talk about our ibm cloud orchestrator in a little more detail moe is very energetic and and up and He's a great speaker and hopefully you can come and learn a bit a little bit more about what we're doing inside And then at 1440 It it actually won't be andrew andrew had a family emergency and is flown back to Canada He's a canadian good friend of mine enjoy going out with andrew but unfortunately he won't be here But we'll have a substitute and we'll talk about deploying managed clouds and You know how to go and get your applications going in hybrid environments and and andrew Unfortunately won't be with us, but We'll we'll carry on and hopefully things go well for him So Just want to say thank you. Thanks for coming. Please come and join us in our sessions We'd love to collaborate and talk and Obviously open stack is is where it's at when you're in the clouds at the infrastructure layer So all of you have been been helping us and participating. Thank you so much And for those of you who are new and interested in open stack and Want to learn just how to set it up use it do anything Come and join us. IBM sponsors a lot of the education tracks out there as well as been participating In the education prior to the summit that's gone on here. It's something that we really Really think is important and hopefully We've been helping you guys that that needed as well too. So thank you very much


I live out in Sacramento. I'm gonna be out here from Sacramento. So hi, David. How you guys doing? Talk about, it's gentrification, before I kinda step into a piece about womanhood, but I wanted to talk about gentrification and how in a lot of areas we are being pushed out as black people, as communities of color. We are being pushed out and I wanted to speak to that. So, it's funny how the streets start looking different when you creep up on your old black.  My folks, they still reside in, so I still rep where I come from, but the only thing familiar is the road that's made of asphalt and the sky is still blue. And money still runs all the big talks, and all my brothers still are passing, and all the old spots. Just now that I got to pay a little fee, so they be outside. Couldn't afford it then, and I was trippin' rent, it's too high. The old pole with the broke bow, when we used to roast in freestyle. They went and put up new aesthetics, they wasn't feelin' the broke aesthetics. Oh, you try, this ground is mine.  Minutes, five and a half minutes, kneeling down, find someone shaded in total silence. If somebody can help us, if your kids are back there screaming, and they may be mine, please help us keep this quiet in a moment of silence. This time we chose 5.5 because August 6th was the 55th anniversary for the Voters' Rights Act, which actually finally allowed black women to vote. 50 years after white women were allowed to vote. So, we're still fighting for voting rights today, and we wanna make sure we all have those conversations with our kids about what it means that black women weren't even allowed to vote when women were allowed to vote. 50, we're talking about 50 years. We're talking about so many issues of police violence and Davis that are affecting families like mine, all kinds of families that you would take for granted if you haven't had those things happen to you. If you've never been stopped by the police and asked if you have drugs or anything else in your minivan, like I have, then you can really take that for granted. But those freedoms need to be extended to the whole community. So, thank you so much for coming. Thank you for being part of the solution. We're gonna start 5.5 minutes of silence in honor of all the black victims that were... Victims of police brutality, the women. This is for our mothers, for our daughters, for our sisters. Okay, starting now. Go and sign the Yolo Power petition.  And thank you all for coming. Please, please keep us in the loop. Wild Events Davis, we're gonna try and engage our community more and more in different ways. And thank you for practicing safety. Thank you for wearing masks. And thank you all. Good job, baby! As I'm made. As I'm made.


OK, let's do it. So today, we're going to make a small adjustment to our process model that's going to allow us to talk about the rest of the process life cycle. So we've been talking about processes, which are one of the core operating system abstractions. And we're going to continue to talk about processes today. Hopefully, we'll get through this by the end of the week because by next week, I want to be talking about synchronization, which dovetail a little bit with the second half of Assignment 1. I'm going to start doing this quickly in class. I'm trying to move announcements onto the forum, so I'm not going to do announcements in class anymore just to create a little bit more time for us to talk about non-boring stuff. I have been sending out emails with links to the announcements that are on the forum. You are responsible for those announcements, so you need to read them and process that information. I'm not going to talk about it in class. However, I do want to start doing assignment check points, so you guys have some sense of whether or not you are in the right place with respect to completing the assignments. So these are hard to do for everything, but we'll give them a try. So for Assignment 1, if you're not started, you're behind. You're in trouble at this point, so that's not good. If you don't understand semaphores, you're behind. This is kind of step one of trying to figure out how to do this assignment. If you have working locks and that's it, you're like a little bit behind. I would prefer that you were a little bit farther. If you have finished your condition variables and locks, then you're OK at this point. I see people, I see some wide eyes, I see some happy looks, I see some like, if you don't understand anything on this slide, you are way behind. The Assignment 1 is one of the only exceptions to our rule about the assignments in this class being cumulative. There are a couple of parts of Assignment 1 that you are not going to use in the future. We'll probably not use your reader-writer locks and the synchronization problems that we ask you to solve, the stop-by problem and the whale mating or whatever it is. Those don't matter. Those get compiled out once you start working on Assignment 2. So the place to devote your time and energy is locks and condition variables. Those need to be rock solid. And what I've told people who have completed that part of the assignment, and there are people who have completed that part of the assignment, I've told them to, it's a good idea to come to office hours, sit down with a core staff member, and just have them look at your locks and CVs to make sure they are perfect. Because the tests are pretty good at this point. If you're running the Assignment 1 test through Test 161, that suite is pretty comprehensive, but I'm not going to promise that it's correct. And locks and CVs are both a fairly small amount of code. So if you put that in front of someone who's seen a million of these, they will be able to tell you immediately whether it's correct. Well, not immediately, maybe. It might take them a couple of minutes. But that's a good thing to do before you move forward. Because if you have problems with these primitives, which you will use in later assignments, you're going to be very, very frustrated. Because things in Assignment 2, I mean, you'll probably never get to Assignment 3. You'll just give up and just dissolve into a puddle by that point. But Assignment 2, you'll have a problem with, because you need these to work. OK, any questions about this week's announcements? Stuff from yesterday or today? Again, I'm not going to highlight stuff from this. When will submission be done? Soon. Scott is finishing up some changes to Test 161. That said, you can still use Test 161 locally. How many people have used Test 161 locally? Yeah, just start doing that. You don't have to submit anything. You can run all the tests. And it will print a score for the assignment and everything. It's pretty cool. For Assignment 1, it's a little different than what you're going to get on the server, because you have to guys have to write the reader-writer lock tests that we have on the server. But yeah, no, this was in the announcement for today. So I would suggest that you start using the Test 161 submission testing tool as part of your local workflow. Again, you don't have to submit anything. You can run the same test locally and examine the output and have a pretty strong idea of the score you're going to get on the assignment before you ever submit. That's one of the nice things about it. So I would start using that. Other questions about announcements? Just one note. The public forum is something new this year that we haven't tried. So if you guys notice anybody on the forum who's doing random stuff, like posting stuff that's clearly incorrect or calling people names or posting huge chunks of solution code or whatever, please bring that to the attention of one of the core staff. Like, we're on the forum. We're going to be keeping an eye on it. But at this point, anybody can create an account and sign on to the forum and start posting and reading posts. So just be aware of that. OK. So last time we were about halfway through the process of inspecting how a particular program or programs use system resources using a set of fairly cool command line utilities that you guys can use on your own Ubuntu installation. So the last thing we did was we were using PMAP, which shows us the memory mappings of the process. And this gives us some idea of what processes use memory for. And if you remember, we looked at bash. We saw that what are some of the things that bash has in memory that this command reveals? One thing, anything. Yeah, constants that came out of its executable file. So remember, we saw a case here where there's a small, where'd it go? Yeah, there's a small amount of, oh, no. Sorry, right here. There's a small chunk, only 4k, and that's a significant number. We'll talk about that later. The memory that it's using is marked read only. And so that is information that those aren't parts of its code that would be in this section that's read and executable. Those are constants or other things that were loaded out of its binary file. What else? So bash has constants. What else has bash loaded into memory? Global variables, so we have a section here that's read right. These are probably statically defined C global variables that get declared at the top of a file that have global scope. What else? Heap, we saw a fairly large, well, I don't know, 2 meg. Not that big, it's bash, right? What could it be doing? A heap, what else? The executable code itself, so this is the first part of its address space. This 876k, and these, if you looked at it, these are instructions. This is the part of the address space that's used to store the code for bash itself. So as bash is running, the instructions that it's executing are being pulled from this section. And then we saw that it was using some libraries, which got loaded up here. What else? There's one last little kind of significant. Yeah, Ed, what's that? Yeah, it's got the C library, right? So it's loaded the C library in here. And the C library is interesting, right? Because the C library, like bash itself, has a part that's readable and executable. This is the actual code for the C library. It has some of its own constants, and then it has some global variables it uses as well. So the structure of that part of the address space or of the memory that bash is using mirrors the part that bash uses for its own code. Again, last thing here, maybe it's at the bottom of the slide. I guess you can't see it. Stack. So bash has one thread, and you can see that there's one thread here that has a stack. And it looks like that thread stack is using about 100k of memory. All right, questions about this before we go on? Yeah. So each thread gets its own stack space? Yeah, each thread is allocated its own stack. So one of the things that you should probably be doing is you guys, how many people are kind of unfamiliar with C code? OK, what I would do is I would make sure is your writing C code, you understand where the things that you guys are, the variables that you're allocating and the code to the writing, where does it end up? How do I get something in here? How do I get something into this section? How do I get something in here? Again, this is not a C thing. Every program that you use, if I did this for the Python interpreter, you'd see the same thing. If I did this for no.js, you'd see the same thing. If I did this for Java, you'd see the same thing. Those are just programs that interpret programs that are written and sometimes compiled in other languages. But they all have the same structure internally. So I have global variables that get stored on my heap that can be allocated and deallocated. I have local variables that could put on my stack. Those end up down here. I have static global variables that are never out and deallocated. Those end up here. And I have the code for their C code itself. Yeah, the discussion here. What's that? What are you talking about? Native methods. This is all native. Yeah, I mean, that's a great question, though. When you're talking to other languages about native methods, what is a native method in Python or JavaScript or Java? What does that mean? Does anyone know? What's a native method? What does it sound like? Implemented in hardware. OK, so we went too far down. Let's come back a couple levels. Yeah. Let's say built-in. It's a little different. Built-in functions don't have to be native, although they can be. What does it mean? Native. Native to what? The machine, meaning what? Mumble, mumble. Anyone? What's that? No? Well, I mean, that is the environment to which it's native. But how does this? So again, you think about your writing Java code. Well, let's say you're writing Python. The Python code that you write, does the Python code get compiled?  No, it's never compiled. It's interpreted. So it's a program that takes your Python code and interprets it, parses the syntax, and does the things that the Python code says to do. So most of your Python code is written in Python. If you use a native library in Python, what does that mean? Doesn't have to be built into the system or pre-installed. It has to do with, yeah. OK, we're getting warmer. It usually is. Why? It can be or cannot be. I can write libraries that provide native methods. That's an interesting answer, but I think it's not true. Yeah, we're getting warmer, warmer, native. Yeah, you guys are warm. Maybe I'll just, yeah. I think you guys know this. You just don't know how to say it properly. It's compiled code. It's code that's been compiled for that machine architecture. It is so as opposed to your Python code, which is never compiled and is interpreted, there are parts of the Python libraries and parts of the Python interpreter and parts of external third party libraries that are actually compiled for that architecture. So when you make a call into that library, the code that runs is not written in Python. The code was written in C and it's been compiled. And the library that's loaded in run is actually, quote, unquote, written. It's represented on the machine in assembly code. So it's actually using lower level instructions that are accessing hardware directly. Why would you do that? Why would I ship something that has native methods in it? Yeah, it can be a lot faster. Yeah, absolutely. So I have a library. The process of executing Python code and all the nice features that Python gives you can cause a performance degradation. Whereas the whole idea of compiling is I get the compiler a chance to optimize parts of the code and to represent it in a way that the machine can run as fast as possible. It's not always true. It's not categorically true, but it is true a lot. OK, sorry about that little digression. All right, so here's the picture of the process that we have so far. So remember, this bash had one thread. We can verify that only as one stack, but we were also able to see that using PS. We have its process ID, which is the name that the system uses to think about it, refer to it. And now we have this thing called an address space, which is a memory-based abstraction that we will talk about in detail later in the semester. But you can think of the address space as mapping how the process looks at memory to the contents of that memory. So we know some of the things that are in bash are its code, its static data, it is a heap where it dynamically allocates global variables probably using something like malloc. And then it has a stack for the thread. All right, so let's keep going. And these are, again, these are all commands that you guys can run. Not all of these tools may be installed on your machine, but they're very easy to install using apt-get. So you don't apt-get install lsof, and you'll get this command if it's not already there. Lsof, what do you think this does? List of open files. So this command is designed to allow you to peek into the files that a particular process has opened at any given point in time. All of these commands, so let me ask you a question. Do you think you'd be able to log into Timberlake and run lsof, unlike the department web server? Would that be something that you could do or could do or could not do? Yeah, these open a lot of security holes. So typically, these functions are only used to inspect your own processes. If you have pseudo access or some sort of super user access, you'd probably be able to run these on other binaries. But certainly, looking at the memory mappings of a process can expose information about what it's doing, looking at the files that it has open, whoops, can certainly expose information about what it's doing. So this is bash, again, 5257. That's our friendly PID. What files does bash have open? Does anyone know what these are? Dev pts slash 0. It has that open three times. Anyone know what that is? Take a guess. It's bash, OK? So down here, this is this configuration file, right? And maybe it's using that to load in the configuration as it opens. What are these other three files? I want something that's someone other than David, because I think David knows what these are. Yeah, like how do you get data in and out to bash? You guys use the terminal. Well, I don't know. Maybe you don't. Maybe you have some sort of fancy. Maybe you have an assistant that types for you. Maybe you have like a fancy voice thing that happens. But most of us use our fingers. So what bash is doing is it's reading data from the terminal. Where does bash write data? Like when you write type LS, think about what happens. You had to input data. You hit return. Then what happens? Yeah, bash writes information to standard out. And then there's a third one here. Does anyone know what that's for? Yeah, there's this extra file descriptor bash keeps going for something called standard error. Where is standard error usually directed? I mean, these are all pointing in the same place. So bash is directing standard error where? To your eyeballs, to the console. Same places. So it just gets mixed in with standard out. But you can change that when you run programs. You can tell them to direct those things to other places. This is a common UNIX convention where it's just convenient, in many cases, to distinguish between normal output from a program and output that is produced if there's an error. Now, the conventions for doing that are entirely up to you. If you want to write your program, so it generates all of its normal output to standard error and all of its error output to standard out, you can do that.  LSOF should essentially, I think LSOF works by actually accessing the file table for the process. So it should show you everything that's in it. Yeah, and we'll come back to this later. All right, so and then the fourth thing that's open here as we pointed out is Bash OC. So unfortunately, when I built this slide, I had a really hard time finding something that had an interesting file open, so I just lied. I can do that, I guess they're my slides. So actually, great question going back to, what's your name? Alex, yeah, so the file table, you can actually see that this is the file table, right? This is a file descriptor number, right? So that's the index in the file table. We'll come back to this. Anyway, just in full disclosure, this wasn't open, so I just modified the output and fooled you guys, right? I mean, who cares? I'm the professor of everything I have to say has to be true. So yeah, just think of it as an alternative fact. All right, so yeah, but just pretend that I caught Bash when it started up and it was reading. At some point, Bash has to open this file, right? It has to parse your Bash RC file and load in the settings that you use. So this has to be open at some point. The reason it's not open later is once Bash loads its configuration files, it closes that file inside opening, right? Because it doesn't need it anymore. That file's only used during the start. Okay, so here's our process model at this point. One thread, address space with the variety of different components we've discussed, a PID, and then a file handle. And what we're gonna do next, because it's important to be able to talk about the way that we create processes is we're gonna spend a little bit of time talking about how this is modified slightly to enable some common design patterns that allow processes to share data with each other. So this is close to the way things actually work, but it's not completely correct. All right, I'm gonna skip this aside, you can look at it. Essentially the way that these tools pull this information, actually I can't skip the aside because it's too interesting. The way that, so you might ask, somebody asked about top before, PS, all of these tools like PMAP, where did they get this information from? They get this information from a file system called PROC. Has anyone ever poked around in PROC before? You can do this in your Ubuntu virtual machine, you can go into slash PROC, and you can start looking around. So what do you think is inside this directory? Slash PROC slash 7615. What is that number? It's a task ID on Linux. So it could be a thread ID, it could be a process ID. Remember tasks are sort of interchangeable. In this case, I think this is a process ID. And there's all sorts of cool stuff in here, right? So I can see the executable that it loaded. I can see information about its memory, memory usage. I actually don't know. I can see information about its scheduling, the current working directory. So what directory is it inside? So this is pretty cool. Does anyone know what's kind of special about PROC? There's something magical and special about the PROC directory and the PROC file system. So I'm inside here. I'm looking around. Yeah. Is it not actually files? Yeah, it's fake. There's no files here. There is no place on disk where there's a file called slash PROC slash 7615. It's totally fiction. And this is cool, right? Like you can go in there and you can poke around with CD and you can do LS and you can do tab completion and everything works. But there are no files on disk anywhere that correspond to the PROC file system. This is a common way for the kernel to export information about the system. So every time a process is created, the kernel creates a new fake entry in this process file table and it's able to fool you to thinking that this is an actual file. And this is actually super useful, right? But again, this is what's called a pseudo file system. When we come back and talk about file systems, keep this in mind because as long as I, you know, this is kind of a cool lesson in the power of interfaces. All the kernel has to do is make sure that it respects the file system interface that other file systems use. And it can fool you into thinking that there are all these files and directories here that have no physical reality. They don't exist anywhere on disk. All right, cool, PROC, yeah. Yeah, yeah, yeah, that's how TOP works, right? So TOP and PS, all they do is they read stuff out of this. So if you look at TOP, it basically monitors PROC and it looks for new entries and like every second it reads the entire PROC file system and compiles all the information and redraws the display. So this is cool. I could get this information in other ways but this is kind of a neat way for the kernel to export it because it allows all these other tools to work, right? So again, you can CD into PROC and you can start poking around in there and you'll find out all sorts of cool stuff, right? But there are no actual disk blocks that store the information for PROC. All right, I'm just gonna skip through the review unless people have questions. Okay, any questions about processes? So this is sort of where we've come to so far. So we have some idea of what's inside a PROCESS. We remember that the kernel is responsible for isolating processes from each other and making sure that processes can't crash the system or interfere with an operation of another PROCESS. But there are ways for processes to communicate data to each other, obviously. Otherwise, they wouldn't be very useful. Any questions at this point before we go on? Yeah. Basically, the information that it gets, so where is all, so it's not stored on the disk but the information where is the information from? Oh, that's a great question. So how does PROC get populated? Where does all that information come from? Who knows all that stuff? Yeah. Well, FORC modifies this information but who maintains that information? Who knows a list of every process on the system and all of the memory that they're using and all the files they have open? Who knows all this stuff? The kernel, yeah. So the kernel is the ultimate gatekeeper for all this information and the kernel is involved with creating PROC, right? So as you're poking around in PROC, the kernel is kind of on the fly being like, oh, okay, he asked me to list the PROC directory. Let me give a list of every pit on the system, right? Yeah, so the kernel maintains all this information. It's what knows, right? This is an operating system abstraction and illusion. Yeah. Yeah, absolutely, right? What if I obtained any one of those files? Most of those files you can't write data to, right? Some of them, well, okay. Let me make sure. I suspect that PROC is usually marked read only, right? PROC is a file system that's used to export data to user space. So it's the kernel saying, I wanna share information with you about what's going on in the system, right? And if you have the right permissions, you can actually access that information. Now, there is something, does anyone know this? This is sort of like super linear, someone's got to raise their hand. There is kind of an equivalent of PROC that is that you can write into and does magical things. Does anybody know what it is? There's sort of like the PROC is like the kernel saying, here's a file system that allows me to export information. There's another pseudo file system that's mounted at boot where the kernel allows you to provide information and to control the configuration of the system. Does anyone know what it's called? If you run mount in your Ubuntu Virtual Machine, you will see both of these file systems mounted in kind of special ways. So one's called PROC, other is called slash sys. So if you go and start looking around in sys, there's all of these interesting things. So for example, let's say that I want to change how the kernel modifies the CPU frequency at runtime. I can do that by writing data into the SysFS and then the kernel will change its policies. I think on some systems you can actually go to sleep. You can put the system to sleep by writing a value as somewhere into SysFS. Assuming you have the right permissions, right? I mean, you wouldn't want anybody to be able to do that. Yeah, so Sys is kind of the equivalent where rather than, you know, let's say I want to turn the, I want to give user space processes the ability to turn the Wi-Fi on and off or something like that. I can do that by creating a file in Sys and then to turn it on, I write a one into the file, to turn it off, I write a zero, right? Something like that. All right, yeah. I mean, that's one way to think about it, but they don't really need to be stored, right? The pseudo file systems, particularly PROC, I mean, the kernel can create PROC on demand, right? All the information that's in PROC is stored in kernel memory, but it's not stored as a file system, right? So for example, if you say, okay, show me the file descriptors that a particular process has opened like LSOF does. I think what would happen is when you read out of that entry in PROC, the kernel is like, oh, I know what I'm supposed to do. It finds the task structure that it maintains for the process, which is something you guys will work on for assignment two, and it just enumerates the process's file table, writes the data out in a structured way, and that's what you get, right? So the answer is yes and no, right? I mean, all that data has to be in memory somewhere, or otherwise, or on disk, or in some way where the kernel can fetch it, but it's not stored in that format, right? There's not a file somewhere that has the data in that format, the kernel can just produce it on demand. Does that make sense? Okay, good questions, all right. So let's update our process model a little bit. So the current process model we have is process have some references to open files, and we'll talk exactly about how the rest entrances work in a minute. The actual way that this works is a little bit more complicated, but this is kind of, I wanna point out something here, which is maybe this is one of the first cool system design principles that we're gonna learn in this class that comes out of the way operating systems are designed. And so what happened here, other than the process, oops, process moving up a little bit, in this model the process has some sort of reference to a file, and in the actual model what happens is the process has a reference to something called a file handle that itself has a reference to a file. Anyone know what this general design pattern is called? So I took something that had, oops, one link, and now in order to get to the file object that has information about the file, I have to follow two links. Anyone know what this is called? So sometimes referred to as adding a level of indirection. So again, in order for the process to deal with a particular file, I now have to file two pointers. The first pointer gets me to this thing called a file handle. The second pointer gets me to the actual file object itself. Not really symbolic links in the typical way you would think about a file. These are really references. The kernel file, has anyone programmed using read, write, open, close before? Yeah, well actually, let's put it this way. Has anyone ever, you call sys open, or you call open in the C library, which wraps this open. What is open return? Okay, what is it? Have you ever printed it? What is it? You get something back, and yeah, it's this like, okay, they made up a type for it. It's called a file descriptor or whatever, who cares? Like, what is inside that thing? Yeah, it's an integer. Anyone know that? That's all it is, it's just a number. Yeah, you thought, oh, it's this magic structure, like they taught us when we did Java, and it's got like methods, no, it's a number, right? It's a C, we don't do objects. So, when you call open, what do you get back as a number? Then later, if you wanna read to the file or write to the file, you pass the number back to the kernel, and the kernel says, oh okay, I remember what file you wanna use. So when you call open, that associates a number with that particular file, and then later operations, you use that number again. Why do I get returned in, sorry, why do I get returned a number? How is that being used internally by the kernel? There's a huge hint up on the slide. Yeah. It's an index into an array. This is one of the reasons why systems like numbers like this is because it's an index into a static array. So, when you say write, you pass the file descriptor, the kernel says, okay, they want to write to file three. It takes the number three, it looks it up in your file table, which is an array. It translates that into more information than it needs to finish the operation, including, it doesn't store the name of the file, but it stores something that's functionally equivalent to the name. A lower level file object that allows me to figure out what exact file do you wanna use. All right. But the actual information that you need to complete the rest of the operation is split between these two things. One is a file handle and one is a file. So, I don't know, it's like I wrote these slides or something, I find this all the time. I should just hit this button more. So, the file handle that you get back from open is an int. That points into an array. That array gets you to something called a file handle. There are three different things we're talking about that are easy to get confused. File descriptor is an int pointed into an array. The array contains a pointer to a file handle, which is a separate object. That file handle itself contains a pointer to a file object, which is a third, well, second object. There are two structures in here and one int. And then, the file system is responsible for taking that file object, the second object that I was able to reach using this chain of indirection and is actually needs to find the files on disk or over the network or whatever. The file system is responsible for making this work at the end. So, I have three levels of indirection. A file descriptor maps to a file handle. File handle maps to a file object. And the file object really doesn't even have to map to blocks on disk. It just has to map to something that acts like a file. Remember, we just talked about PROC. PROC has no blocks on disk. All PROC has is a file, is it obeys the file system interface. It acts like the other file system. Okay, so why would I do this?  No one is going to be impressed, right? It's like, well, I had one hash array that was storing references, but I could create three and just have things go together. No, I mean, there is an actual reason to do this. And the reason is that file state is one of the ways, or file handles are one of the ways that processes can communicate with each other after a fork. And this is something that allows me to set up pipes and all sorts of other things. So it's important to talk about. All right. So the actual file descriptors, that number, that's private to every process on the system. When I open, let's say I want to open food.text, the kernel hands me back a number. That number is meaningless to any other process. So let's say the kernel hands me back five and that's my reference to food.text. If some other process says, kernel, I want to read from five, the kernel will either say you don't have a file five open or your five points to bar.text or something like that, right? So the file descriptor is entirely private to the process. It's meaningless outside the process context. The file handles are usually private to the process but can be shared in certain cases. And we're gonna talk about the canonical case in which they're shared in a minute, which is after fork. So normally, when I open a file, I get a private file handle. But after fork, which is how processes are created which we'll start talking about today or early on Friday, the file tables are adjusted so that the parent and child are initially sharing file handles. So the parent and child have the same files open in the same locations with the same file descriptors, but they're actually also sharing this file handle object. And this is really important because it allows me to set up pipelines and others. And then the file objects themselves, the lower level file objects are typically only created one per actual file and those are maintained by the file system itself. These you don't even really need to worry about. You will encounter these when you get to assignment too. There will be a part of the structures that you have to create that corresponds to this file object. I'm sorry, it's not called a file object. It's actually called a V node. That's not my name. But that's what it is in this particular case. And it's not something you really have to worry about other than you have to know how to create one and how to manage those pointers properly. All right. And this is a case where, this is one of our design principles, right? So adding a level of indirection allows us to share state and facilitate control. And we'll see this here when we do this with file handles. The probably the most powerful place that adding a level of indirection comes into operating systems is when we talk about virtual memory, which will be about a month. Okay, so let me pause and we're gonna start talking about fork and then we'll come back to this file handle issue. Yeah. Yeah. Yeah, so, yeah. So that, my file table is private beach process, right? Remember that file descriptor is only meaningful to the process that receives it from open. So once I get it back and here's how to get me to help you with those things. I can help you open files. I can help you add memory to your process. I can help you create new processes. I can help you communicate with the process, whatever. Windows is like, I'm also here to help you out but I have a completely different list of things I can do. Now, underlying capabilities are the same. I still need to be able to create new processes, create threads, access files, but the names are all different, the semantics are different, the calling conventions are different, blah, blah, blah, blah, blah. And so now, obviously, and this is, it's cool I can say this today and I'm very proud of the Windows people for doing this because it's gonna cause more people to stop using Windows. Once you realize how cool it is to do some of the stuff using Unix-like commands, the Windows 10 now includes an environment in which Windows can essentially run Unix executables natively. So that's how much similarity there is. It took a while, it took us to Windows 10, but as a Windows 10, there's now a way for Windows to take the things that are, so essentially it's like, I ask Linux and then there's this new Windows, I think it's called the Linux Windows subsystem that translates Linux into Windows and accomplishes the same things internally. So there's a lot of overlap, but the interface that we're gonna talk about and the operating system that we're gonna talk about is Linux, but again, the concepts here are very, very, very simple. All right, I will see you guys on Friday. I'll answer your question if you come up here.


Hi, I'm Trey Leith. I'm the Executive Director of Maker Ed and welcome to the K-12 online conference. Today I would like to talk to you about Maker Ed and Maker Education and how Maker Education can transform the lives of youth and their learning experiences and how Maker Ed can help you as an educator make those changes. So what I'd like to do in the keynote today is explain a little more about the Maker Education in general and Maker Ed. Stephanie Chang will come on later and explain some of our programs in more specifics. We'd like to show you a couple of videos too. So thank you very much for coming and listening to the keynote today. Humans are makers. In fact we have been making since the beginning of our humanity and in many ways it is making that makes us human. Making is a innovative creative social activity where we create things with our hands and our brains. We have been doing this since the beginning of our existence as humans. For example here you see some art that our ancestors tens of thousands of years ago made. This art on caves used innovations in pigments and drawing methods etc. But we started innovating early on with those drawings. So for example this is an animation. Many of the drawings include repetitions of different parts of the body of the animal. So for example here there are three tails drawn. Tails drawn on different levels of the stone next to each other. Many of these drawings are similar. Heads moving up and down. Horses running. Spears being thrown. So very early on we took those drawings that we were doing and making and innovating and creating them. And we know that we were doing this as a social group because they were being drawn by men, women, and children with. And as you know as time went on we innovated on that even further. And here is a early flip book movie for example. And we started making more and more complex animations and art. Till we're today where we actually have computerized animated movies that can often be very realistic. So we have been making and innovating and creating since the very beginning and it is often what makes us human. There have been make movements in the beginning too. Early on with early industrialization and automation there were counter movements to return us to our making roots or do-it-yourself ability at the handcrafted view. This is a view of an arts and crafts movement chair handmade meticulously created. The arts and crafts movement isn't a perfect example or analogy of today's maker movement but it is has a similar impetus that we as humans are makers. And individually and socially we are makers and we need and we return to our roots. Making or do-it-yourself magazines and materials have existed from very early on since we have been printing for example another innovation made by humans. For example here are two from the early 1950s to make maker magazines in sense popular home craft and home repairs how to make your own boat trailer your own sunroom greenhouses etc make your own chimney make your own boats the one on the left is has actually how to make your own baby grand piano and more as it says more than 150 projects you can make. So we were always wanting to make things ourselves whether those be crafts or complex electronics we have always wished to create with our hands and innovate ourselves and learn from that. In fact we have known from the very beginning that from very early on that making and hands-on creating things is the best way we humans learn it's because that's the way we have been doing it since the beginning. There are many terms for this constructivism experiential learning project-based learning etc but it's basically that the interaction as a community and the creating things with our own hands and innovating and working from that creates a learning experience that is both deep and engaged and this is the way we learn. Well about 10 years ago magazine was started to be published by O'Reilly and now make maker media called Make Magazine similar to the many magazines that have and books that have come in the past of doing things yourself and creating things yourself and building on projects and innovating and developing them with your own hands. Make Magazine took another step as technology and computerized materials became so inexpensive that we could start creating our own electronic and computerized projects. So Make Magazine was born and goes from as you see showing soapbox creating your own soapbox derby cars to creating electronic textiles hacking your plants for example. So Make Magazine is the a new iteration of a long-standing desire to make things ourselves and as technology got cheaper we can now actually add technology to that. Well as people were making they actually started wanting to show that off and so two maker fairs got started. The maker fair in the Bay Area of California and the maker fair of New York for example are two very large maker fairs that attract over hundreds of thousands of people to show off the things they have made and innovated and created and to look at what other humans have made, innovated and created as it's often called it's the greatest show and tell on earth. It includes artists and musicians it includes people doing crafts robotics alternative energies it says here hands on science industrial arts and much more and as the maker fair one of the maker fair one of the maker fair model says we are all makers. The maker fair became extremely popular and there have been many maker fairs M-I-N-I many maker fairs throughout the world in fact over a hundred now and cities as far apart as Washington DC and Lagos Nigeria these have been becoming increasingly popular starting in many new cities and growing in each of those cities as people have gone back to making as their activity. The maker education initiative for maker ed got started about three years ago and it's basically to give maker education we know that making or hands-on learning is an engaged and deep way of learning and we want to support educators in creating maker education programs and approaches. Thanks Trey for providing such great historical context into making a maker education it really informs the work that we do today into the future as well. Hi everyone I'm Stephanie Chang I am maker ed's director of programs and I'm excited to tell you a little bit more about our organization and some of the work that we do and hopefully invite you to engage in some of the programs and services that we offer but before I dive into that I wanted to show you a quick video clip of some of our young makers. These are youths close to our heart who participate in our young makers program out here in the Bay Area and it's a story I think you'll resonate with. It provides context again into explaining why we do the work we do and how we can make an impact on youth through making.  One time I was hammering something and then I was like wait this is a good beat and so I started hammering random things in my lab. What's similar between music and making is that they both have the ability of freedom you could do whatever you want. I'm Sasha I'm 14 years old and I'm going to the 10th grade. The first time that I knew I had test anxiety was in eighth grade it was our midterm for English and I studied really hard for and I should remember sitting down at the test my hands were shaking and I just was freaking out and then got a really bad grade. I was going on trip with my family and I remember just walking through a cave and I see a little hole inside the cave so I asked the cave guide what's behind that is there like a another room or something he's like we don't know we don't want to destroy the caves and I was like huh dad why don't we build a robot that can go inside there and scan the cave and map it out so we know if the cave is worth being destroyed for that small hole or not. I saw Sasha building stuff I'm like hey I can build stuff also my name is Alina I'm 10 years old and I'm going to fifth grade. A creative person can be anyone I think no one has to tell you what to do you can just like build anything you want. Right now my mom just gave me a vcr player and it was all broken and right now I'm trying to take it apart and understand the components. My name is Kai I'm 11 years old and I'm going to sixth grade. This is like a big problem to me when it gets overly hard I feel like it's not going to be possible. I don't give up because I always know if you just keep trying you'll get it one time at least. Before I started the project I felt like I wasn't the making type I felt as if I was incapable of building something. After I built the project I felt stronger and that boosted me up and saying hey you know I can actually do something. I feel more free when I'm making. To continue working with educators organizations and communities to bring making into new environments and integrate them into existing ones. So let's get started. Maker Ed's vision is every child a maker. It's a bold statement but we truly believe that every child is born a maker and as such we as educators and mentors have both the honor and the responsibility of fostering that inclination in kids from when they're born throughout their lifetimes. Everyone at some point in their lives has made something. It may have been something minuscule. It may have been something enormous. It might have had a huge impact on a family member or friend or maybe it was just for an individual purpose. But whatever the case that process of making the act of creating the invention the iteration the struggles that anyone goes through that creates the essence behind maker education. And we feel that this act empowers and allows for youth across the board to feel confident and creative to contribute to their own communities and their own lives. Maker Ed's mission is to create more opportunities for all young people to develop confidence creativity and interest in science technology engineering math art and learning as a whole through making. We work with educators organizations and communities at large to do just that whether supporting them behind the scenes or on the front lines. Our work falls into a couple of different buckets. We provide training and support to educators no matter their institution or setting. For some it is to help them start thinking and get started with their programming and figure out what type of making or maker education program works best for their communities. And for others is to help them develop and sustain the work that they've already begun. At Maker Ed we also offer a number of resources whether directly connected to the programs that we offer or as a one off. We maintain and create an online resource library with over a hundred different resources both internal and external. Additionally we've created playbooks released research briefs and written other documentation and collaboration with other organizations that additionally provide stories and cases for how making can impact communities. We also like to think of ourselves as connectors and we are delighted to be able to provide links and connections from one educator to another or from one organization to another. It's an opportunity for those who are starting and developing not maker education programs to share their successes and challenges to learn from one another and really to collaborate. We are also thrilled to be able to highlight the work and stories of educators whether in small communities or large and show off their successes and impact. We feel like as a community of makers and maker educators this is one of the best ways to really bring the movement forward and keep the momentum going as well. We also care a lot about linking research to practice. As maker education is a new and evolving field there are great questions being asked and good research being done. And through this work researchers and practitioners are able to trade stories share wisdom and further the field overall. We touch upon these buckets in all of Maker Ed's programs. Maker Core is a spring and summer season program focused on professional development that combines online training with hands-on practice to create and provide programming within youth serving organizations across the nation. The Maker Vista project is in partnership with the Corporation for National Community Service under their AmeriCorps Vista portfolio and works behind the scenes to ensure eradication of poverty through maker education means. Our Young Makers program is a flexible club model that brings together youth and mentors to create a project of their own choosing to showcase a showcase event. And our open portfolio project focuses on portfolios as an alternative means of assessment. All of these programs and more are the ways in which we work with educators and the organizations and communities within which they work to ultimately impact the growth the development and the opportunities of our youth. Everything is hands-on. It's a very learner driven experience. It's not a competitive environment. Everyone wants to see everybody succeed. It's definitely interdisciplinary because it's real materials, real tools, real problems. There's not just one answer, there's many. Maker Ed is a nonprofit organization that's bringing making opportunities to communities. And they are masters at bringing all of us Maker educators together. They learn something and they teach you something and that in turn teaches everybody else. Maker Ed has had huge impact on my professional development. The philosophy that we've been given as a part of Maker Ed is that it's one of cross-pollination of ideas. The thing that surprised me the most about this experience is that I learned more about how I teach than what I teach. I just see this tremendous energy in the educators when they walk away from an experience with Maker Ed. You're not really looking for the end product. How you get there is more important. If kids can understand the scientific and the engineering and the math principles behind what they're making, they can change the world. I remembered letting a kid know that, you know, she had really solved a hard problem and this is a kid that had very low self-esteem and then as she walked away she sort of did a little dance. That's just like one of those moments where you realize like it's so small, right? And it could disappear if you don't know it, but that might stay with that child for the rest of their life. That's great. My biggest hope for the students is for their imaginations to just run wild. These are kids that don't look at the world as happening to them. They really look at the world as theirs to design. In the Maker Ed community we create, we explore, we learn and we imagine. Thanks again for joining us today on our keynote. We hope you were inspired to bring making into your educational settings and classrooms. We have lots of resources as our vision is every child or maker. We do have lots of resources for you to use, so I would encourage you to check those out, including a newly released Youth Maker Space playbook that helps you think through and design a maker space for your youth and students. We have a robust resource library for you to check out resources. A lot of different things around making education and we have started recruiting for our Maker Core program and partners, so if you have a chance check those out. You will see the links at the end of this. And thank you again.