Live from Las Vegas, Nevada. Extracting the signal from the noise. It's theCUBE, covering Informatica World 2015. Brought to you by Informatica World. Welcome back everyone. We are here live in Las Vegas for Informatica World 2015. This is theCUBE, our flagship program. We go out to the events and extract the signal from the noise. I'm John Furrier, the founder of SiliconANGLE. George McCose, the segment, George Gilbert. Who's getting wired up right now, but our guest here is Ivan Chong. EVP and Chief Strategy Officer for Informatica. Welcome to theCUBE. Oh, thank you for having me, excited to be here. We love talking strategy with the Chief Strategy Officer, EVP, you got the chess board in your office, you got the lay of the land, all the executives around the table, product groups are kicking butt. What's the strategy? I mean, I'll say you guys are going private. We saw that news, and that's still not finalized, but Michael Dell took his company private. He's so happy, smiling. He's a spring in his stab, he's doing great. The big data world is certainly got a lot of growth. Well, I think just one way to think about the strategy is that we love disruption. We love disruption because we think the world is coming to realize the value of data, right? And as you mentioned, data is having a life of its own outside of applications, outside of systems. And it's going so diverse that having an integration platform, having a data management technology is becoming much more valued by organizations. You guys have an engine, explain what's going on at the engine. So you have an engine, what's the core engine for Informatica and then what's wrapped around it? So we like to talk about an intelligent data platform and it starts with the ability to get at data wherever it exists, right? Not just data because it sits in an application system or not just data that's part of a business intelligence system, but anytime you can harness data, we want to get at it. So those are the connectivity feeds that we pump into the engine. The core engine we call Vibe, right? It's got this ability to process data no matter where it came from. It uses metadata, so it retains context. It does a lot of processing with regards to transformations and aggregations. It applies data quality to be able to produce more valuable and trustworthy data. And then on top of that, we add the metadata to draw context and that's where you'll hear about a lot of our exciting new announcements with regards to security, with regards to discovery of data through live data map, with regards to data preparation. And then on top of that, we do provisioning, not just to systems like business intelligence systems and database systems or Hadoop systems, but to individuals, business people that are using data to be able to make decisions and to harness it to be able to gain insights on their business. You said something interesting about all the processing that goes on in Vibe and then the context that gets added from sources. Traditionally, when you're processing, at least near real time, stream processing, adding the context can slow it down. How do you manage that trade off? I think what we usually do is we provide multimodal processing, right? So you can process it inline. You can start to process it in batch or offline. And so we allow the customer to trade off. How critical is it to have that sort of sub-second response? How critical is it to have it after you've closed out your quarter? So that's why you'll see us investing in all sorts of low-latency technology as well as batch processing technology. So you'll see us in stream processing as well as Hadoop. Okay, so the stream could be like a tunable latency ingest into Hadoop where you do the rich context, the historical context, or it could feed something near real time or real time. Yes, yes. We have to create a platform that supports both modes of operation because based on our legacy and heritage in analytics, we see the value of both and we understand the technology requirements needed to support both. So about real time, this is really interesting. And in context, the cloud. So we talked to Ash earlier about the cloud business doing really great and the consumption that customers are saying is I'm going to move to the cloud, eventually hybrid cloud's certainly hot right now. But real time, I want to get my resources in whether it's content, sales people, assets, to my customer in real time at the moment of impact or at the moment of relevance. It's a very hard thing to do. How do you guys look at that real time with the cloud architecture from a software standpoint, from a data standpoint? Because it's a combination of things, you got to have the data. You got to understand the data. You got to have low latency transit to the endpoint. What's the, how do you guys look at that? Well, we look at it in a number of ways. One is through our platform, right? So we talked about having the different modes of ingest, right, so there's stream processing as well as batch processing. The other way to look at the problem that you've talked about is you need to be able to sense information in real time. You can't just wait until transactions like orders or sales are closed. You need to be able to detect customer intent, customer sentiment, right? So we talk about the platform, our intelligent data platform having the ability to be able to infer things through machine learning, to be able to do natural language processing so that you can take any amount of data and then you can glean insights from it that allow you to make those real time decisions. Another interesting thing you said, applying all these different analytic techniques. One of the things that we've seen with the rise of sort of Hadoop based platforms, even though the uses for most enterprises are primitive, they're unbundling what was a uniform kind of data platform or an integrated analytic pipeline. So we have machine learning, we have the stream processing, we have graph processing, all the different modules that are growing up. And it's a little bit hairy. What's, how does, how do you serve up your components to keep them integrated and low cost of ownership? So what we've done with our platform is we've been able to keep a consistent way of tracking the metadata. And the guiding principle behind that is that the metadata should be able to function regardless of where the data came from. So if you started by processing data that came out of a mainframe, and we've had customers that do this, as you adopt newer forms of data storage technology, maybe you go to a relational database or maybe you go to Hadoop, the same rules that you use to process that data apply in the new context. And that's the guiding principle behind our metadata. Spark is coming out from a different sort of corner of the universe where they have this sort of single execution engine and little wrappers around it where each mode of processing can sort of chain into each other mode of processing. They're not discrete engines. How, how do you compare against that? Well we see Spark as a platform that we adapt to. Just like we have customers that are storing data in HDFS and Hadoop, that stack is going to encompass Spark. This morning we talked with some folks that were experts from University of Wisconsin and that are part of Berkeley's AMP Lab that delivered Spark technology. We think it's just another engine that we can harness so that we can make use of additional technology for our customers. So when we talked about, you asked about the stack earlier, we were very careful to separate the metadata in the context from the engine technology. So the mere value add across all these different types of pipelines is the metadata. Yes. All the way through the source. Yes, that's a secret sauce. Yes. And we can do more things to innovate by gleaning more information about the metadata. So just being able to trace the proliferation of data for security purposes, that's one example. That's huge right there. That's one example. Because now you're tracking, it's geo data, geo tracking, geo space, whatever you want to call it, you can look at data out in the hinterlands of wherever, cloud, device, and understand context of data. Yes. So that's a security notification beacon, whatever you want to call it. Yeah, specifically the context in that case is the sensitivity of the data, right? Because people are drowning in data. So you really want to focus in on what is the data that I am obligated to protect and to secure? And so we have the ability to detect what is sensitive data? What is the date of birth? What is the social security number? What's the credit card number? And then we highlight that data as it flows through the various systems that our customers have. And then we identify, here are the things that you need to focus on to be able to protect data. So your competitive advantage then, if I was just pretending I was working with you guys and say, hey, okay, we've got the metadata secret sauce, let's lay around machine learning and NLP and other cool algorithms to surface notifications around what's important in real time. That way that- Metadata has always been the strength of our company because to be Switzerland, right, to be able to allow for customers to have a very heterogeneous compute environment, you need to be able to focus on the metadata. Can you give us, in addition to security, some more examples of flowing that metadata from the source all the way to the target across different engines? So this new product that we announced last year, REV, it's getting huge adoption, and the premise of it is to allow everyday business users to do what we empowered IT developers to do, which is to be able to blend data together, to shape the data, in case it has different cardinalities, to be able to prep the data on their own without having to go back and forth with IT. Now, to do that, we need to have our metadata. We need to understand, through our architecture, what the business user is doing to the data, right? So when they're blending the data together, it's through suggestions on the UI, but underneath the example you asked for is we're capturing metadata, say, we use the join key on a specific set of fields. It's almost like you've got a log of activity across all the transformations of the data. That's right. In the REV data preparation product, we call it a recipe, right? So that's what the business user sees, but underneath its metadata, that then can be understood and ingested by our traditional engines, either in the cloud or with PowerCenter. So this sounds like one of the differentiators that Cladera hangs its hat on is the governance capability. But this sounds like it would be working across a perhaps wider array of analytic engines and data sources and data targets. Yeah, so Cladera's a great partner with, we work with them because we share very common objectives for our customers. And I think they are a great solution for governance. The way we fit in with them is that they focus a lot on the persistence layer, right? So they've got a very, very valuable economic proposition for storing data, large volumes of data. They've got good technology on top to be able to do the ingest and management of the data. And so when we provide our metadata, when we provide our architecture on top to be able to help them do discovery and cataloging of all the data assets that they've got, you get some of the demos that we showed this morning which show you how you can start with lots of data stored within a Hadoop system and then automatic ability to catalog the information search and discover information and then prepare it. Okay, we're getting the hook here but I want to ask you a final question. What M&A's on the horizon? What kind of white spaces are you guys going to be filling in with M&A? So, perhaps taking over Oracle since the data's more important now than the data manager. Well, I think the M&A strategy that we had before our partnership with Permura and CPPIB is still very much in play, right? So we invest heavily in all data related technologies. All right, so MDM is a big area for us. We continue to invest in that area. Data as a service, cloud technologies, all those cloud integration technologies that can increase the value of how you can manage cloud assets from software as a service and infrastructure as a service. All those are things that we actively discuss and pursue. Well, thanks so much for sharing the strategy and the input, George, hitting you some hard questions that you handed them like a pro. Thanks for coming on theCUBE, really appreciate it. Well, it's been my pleasure, thank you. Thank you, we'll be right back. This is theCUBE, we'll be wrapping up in the short, right after the short break.


The addition of inorganic spherical nanoparticles to polymers allows for modification of physical properties and implementation of new features in the polymer matrix while also providing potential applications in energy storage systems. This article was authored by Thomas Hanaman and Dorothea Vingasabo.


Wondering about you are happily ever after? Falling in love. An exhilarating and life-changing experience. When you meet someone who you feel a deep connection with, you're filled with excitement and anticipation for what the future might hold. However, even when you're head over heels in love, it's not always easy to know whether you're with the right person. That's why it's important to pay attention to the signs that can indicate whether you found the one. So here are your lucky signs that you're in love with the right person. Zen. Do you always feel on edge in your relationship? Do you always worry about what they're doing when you're not around? Or do you feel a sense of calmness and inner peace with them? According to Helen Fisher, a love expert at Rutgers University, being calm around your partner could be the sign that you found the right person. Fisher believes that when you're with the right person, your body releases chemicals that promote feelings of comfort and trust, such as oxytocin and vasopressin. Research has also shown that feeling comfortable and authentic with your partner is crucial for the long-term success of a relationship. A study published in the Personality and Social Psychology bulletin found that couples who reported feeling more authentic in their relationships were more satisfied with their partners and had greater commitment to their relationship. So, if you feel completely comfortable with being yourself and sharing your thoughts and feelings with your partner, it could be a positive indication that you're in love with the right person. Without words, have you ever felt so connected with your partner that you could communicate with them without saying a word? According to Dr. John Gottman, a renowned relationship expert and professor emeritus of psychology at the University of Washington, nonverbal communication is a critical component of a couple's emotional connection and overall satisfaction in their relationship. In his book, The Seven Principles of Making Marriage Work, Dr. Gottman explains that couples who can effectively communicate nonverbally have a higher chance of having long-lasting and fulfilling relationships. He highlights that these nonverbal cues can help partners better understand each other's emotional needs and foster a deeper level of intimacy. Research has shown that nonverbal communication is a crucial predictor of a relationship's success. A study published in the Journal of Personal Relationships found that couples who displayed high levels of synchrony in their nonverbal behavior had more satisfying relationships and were less likely to break up. So, have you noticed any nonverbal cues that you and your partner share? Good direction. Imagine you and your partner have been together for a while now. There's always been a nagging question in the back of your mind. Are you and your partner working towards the same goals in life? You've had some great times and some tough times, but overall, you do feel like you have something special. If so, that could be a sign that you've found the right person. Dr. Terry Orbuck, a psychologist and research professor at the University of Michigan, notes that having a shared sense of purpose is a key component of a strong and lasting relationship. According to a research, couples who have a sense of shared meaning in their lives tend to have better communication, more intimacy, and a stronger commitment to each other. This is supported by a study published in the Journal of Social and Personal Relationships, which found that couples who shared a sense of meaning and purpose in life reported higher relationship satisfaction and were more likely to stay together in the long run. Mutual respect. Mutual respect is an essential aspect of a healthy and fulfilling relationship. When you're in love with the right person, there exists a deep sense of respect between both of you. You genuinely value each other's opinions, ideas, and boundaries, fostering an environment of open communication and understanding. There's a beautiful balance of give and take, where compromises are made and common ground is found. This mutual respect creates a strong bond, fostering equality, trust, and a deep sense of connection. It's a lucky sign that you found the right person to love. One respect is at the core of your relationship. Intuition and gut feeling. Last but not least, what does your gut tell you? Sometimes you just have a gut feeling that you're with the right person. Your intuition tells you that this is the real deal and deep down, you know you're in love with the right person. Can you relate to any of these signs? Feel free to share your thoughts in the comments below. And if you found this video helpful, don't forget to like and share it with your friends and family. We appreciate your support and hope you continue to tune in for more insightful content.  Get your very own sigh. The lovable plushie is here to brighten your days. It embodies the spirit of psych to go, and it serves as a reminder to prioritize your mental well-being. Its green leaf symbolizes growth, renewal, and the importance of self-care. Whether it's for yourself or as a thoughtful gift for a loved one, sigh is ready to be your snugly friend through all of life's ups and downs. Buy your sigh plushie today. Link is listed in the description box.


Good afternoon and good evening. Welcome, everyone. I am Michelle DePama, Principal Solutions Architect for Red Hat and the host of the Data Services Office Hour. For those of you that don't remember, last Tuesday was International Women's Day. And in honor of that, I'm so happy to present members of the Red Hat Data Science Women's Panel. And they are going to be talking about the data science model life cycle explained. So before we get into it, I just want to let our viewers know that the format of the show is going to be a little bit different. So since we have so many panel members, we're going to have each person present. We will take questions in chat. We'll also have questions at the end of their section before moving on to the next section. So with that explained, welcome, ladies. How are you this morning? Hi. Okay, so we're good. Who's going to go first? It'll be me. Okay. All right. So I'm going to go ahead and disappear from the screen and let everyone else disappear and you can take over. Awesome. All right. Yeah. So hello, everyone. And thank you for joining in today. We will be going over the data science model life cycle today. And as Michelle mentioned, we are part of the Office of the CTO here at Red Hat. We are part of different teams, but overall, we make sure that we are working on these different use cases for data science and coming up with innovative solutions and leveraging all the open source tools that we have. So with that, I would just like to introduce myself. I'm Hema Viradi. I'm working as a senior software engineer here at the Office of the CTO recently formed open services group. And joined by me today are my colleagues Akanksha, Oindrila and Selby. And each of us will be going over the different pieces of the data science model life cycle. And to start off with what exactly is this entire AI ML life cycle that we're talking about and each of us will touch base upon the different components involved in this data science life cycle. So becoming data powered is first and foremost about learning the basic steps and phases of your data science project and following them from your raw data preparation to building your machine learning model and ultimately to operationalization. And I will mainly be focusing on the data engineering part of the life cycle. So as you can see in this diagram here, we will be setting up different goals for your project when you set off for a different AI ML use case. You will go ahead and gather all of the required necessary data that's needed for solving this problem. And then you will go ahead and develop your machine learning model followed by deploying this model so that it's available somewhere in the cloud as a service that other people can interact with. And finally, you can implement some inference ways to interact with this model as well as a monitoring in place to actually make sure the model is performing accordingly. So now let's start with the actual data engineering phase of this life cycle. So once we know what is the problem that you're exactly trying to solve, you need to start looking at the data that you have available. And data can be available from multiple sources. So aggregating mixing of all these data from as many different data sources as possible is what makes your entire project great because you can look as far as possible as required. Now, once we sort of identify what are these different data sources, you need to start actually fetching and figuring out the best way to store this data. And in our team for this, we use something called as Ceph, which is an SV backend storage where you can sort of split all of your data sets and arrange them into different buckets within different directories and folders so that you have all of your data stored. And we also have something called as Trino. Now, Trino is nothing but an SQL based query engine, which helps you to connect with your data sources, create all of the required tables that you might need out of these data sets. And then it's essentially like an database engine, right? So you can also query on top of it, explore on top of it, and then maybe connect it to required visualization tools if your end goal is to also do some sort of visualization. So once we know that we have all of the data and it's available and it's ready, what do we do next? We need to actually further pre-process your data and especially for a data science project, there can be a lot of noise in your data and hence it's very critical that we filter out the right data so that we have that to support all of our model development that we start working on. And to perform this sort of filtering and pre-processing or feature engineering, if you may call it, we use something called as Jupyter Notebooks. Some of you may be familiar with it, but Jupyter Notebook is essentially a web-hosted service where data scientists can write their code and create all of these interactive notebooks where the code is sort of in an interactive cell-based manner so you can execute and run them as in while you're working on the different pieces of it. And ultimately you can store this notebook as well as inside your repositories, that's where we put all of our notebooks in GitHub repositories so that other data scientists can also collaborate, provide feedback and have these notebooks as references as part of your data model management lifecycle. So once we have all of this data sort of pre-processed and filtered, what do we actually do next? Now it's time to enrich your data further, right? So you need to manipulate your data so that you get the most value out of it, you can start grouping all of the different data sources that you have so that you narrow down on the essential features and also taking into consideration other aspects such as ensuring that you're not incorporating any bias into your data so that your machine learning model is ultimately not affected by this. So that's what we do in this enrichment phase of your data is basically trying to figure out what the model would require and making sure that the data is not compromised and we have the right amount of information there to power your machine learning algorithm. And finally, once we have all of this done, you can start visualizing your data by creating some insightful dashboards so that you can help data scientists further decide what type of machine learning model would be best suited. And to do this visualization exploration, we have a tool called Superset. Now, Superset is very similar to Tableau with different capabilities and different features, of course, but it's an open source tool. And the great part about Superset is it's also interacts with Trino, which is our SQL based engine, and you can create your basic charts, which can be like time series graphs and bar charts and pie charts, all of the essential charts that you need. And you can create some dashboards in there so that they can be shared with not just data scientists, but also other members on your team, maybe the business team, the stakeholders, so that they have some insights into the initial phase of your data, and they can also help you come up with some key insights, which may actually help you figure out what is the right type of model that we need. So this is essentially the different pieces and different phases of your data engineering part of the life cycle. And I will quickly share a quick demo of exactly what we've done here for each of these pieces. So over here we have one of our projects that we started working on recently. In this project, it may not be an ML use case yet, but at least it still has some of the data engineering pieces involved in it. So in this project, what we're trying to do is essentially look at some GitHub data that we have. And we're trying to come up with some metrics to help analyze the health of a repository or analyze the health of an GitHub organization. So we're looking into different GitHub issues and pull requests and so on so that we can have some metrics to share with our team to show how the repository is doing overall. And we have an initial notebook where we have done all of this data exploration, pulling of the data and storing of the data into Ceph and S3, as I mentioned earlier in the slide. So here what we do is we try to create an environment file, which has all of the required variables to connect to that S3 bucket that you want to affect your data from. And then we go ahead and extract all of this GitHub data that we were interested in exploring. And once you have all of this data up and ready, we can actually do some pre processing of the data. We have some raw data, but we would like to look at what are the different columns that we have available inside of this data frame. What are the columns which are most relevant for our analysis. And then finally, we only retain the columns we think are useful for our data analysis. And then we further process this and store it back as a new file into our S3 bucket that we have over here. So once we have this sort of data available and pre processed, we can actually dive into creating those Trino tables that I mentioned. So Trino is what we are using as our database and we would like to connect with Trino into SuperSet as our visualization tool. So hence we go ahead and create some tables to connect with Trino. And we have an operate first Trino instance available, which we can connect to and have those tables created for us. So we would just need to pass the different Trino related environment variables like the host, the port. And once you have that you can check if your connection was successful or not. And then you can go ahead and create your table by passing the location of the bucket where your data is stored. So that pre process data frame that we did in the notebook is what we have stored as a parquet file in this example. And we are basically creating a table out of that parquet file inside of the Trino database. And that's all of this is being done inside your notebook. So this is where the collection of the data is done, the filtering of the data is done, and the enriching of the data is also covered within this notebook as well as the connection pieces to Trino. So here there is something called as Cloud Beaver. So like I said Trino is the database, right, but we also have a UI, which can help you play around with Trino. Let me just log in here. And what it does is it's basically helping you to run some queries are against your database. So we have the database instance here, and we are looking at a particular catalog where our data is stored. So you can actually go and further look into the schema, what are the different tables that we have. So in the notebook, we created a table called PRs to store all the PR data, and we created a table called issues to store all the issue data. So you can see those two tables here. There's a PRs table, there's an issues table. And inside of those inside of the table, you have all those columns that we looked at inside of the notebook. So great. Now that you have this table created and you know that you have all the columns, we are actually ready to go and play with SuperSet and create those dashboards. So again, we have a SuperSet deployed in our operate first cloud environment. So we can log in. Oops, let me make sure I'm in the right location. Yeah, okay. So yeah, so you can log in to SuperSet over here and create those tables and dashboards. So once you're logged in, you should be able to make some charts, make some dashboards based on the tables that we had. And this is a dashboard that we recently created for that exact issues and PR related data. So here we're just trying to look at a certain repo. And I can go ahead and select a given repo, go ahead and apply, and it'll show us some metrics that we're looking at like how many issues are open. What's the mean time to close issues for this repository? What is the trend of issues that we've created over time, as well as who are the main issue contributors for a given repository. And the nice thing about your dashboard is you also have the option to create different tabs. So if you want people to focus on different sections of your insights, then you can also create those tabs accordingly. And you can arrange your charts to reflect those. So here again, I want to look at all the PRs which is in this particular repository. And I have some metrics populating over here like what are the number of open PRs, closed PRs, and then what are the PRs created over time, as well as some PR contributors for this particular repository. So this is where the visualization and insights are all made from inside of this superset dashboard. And once this is shared with all of our stakeholders, as well as our teammates, we can get some more feedback into the data and they can also help us understand if there is some other missing data required. And if there is an AI, if it's an AI machine learning problem, then we can start moving on to the actual model development phase of your problem and figure out what's the best algorithm that would help us achieve the solution to your problem. So that concludes my section of the data engineering piece, but happy to take if there are any questions. Okay, so I don't see any questions in chat, but I have a couple of questions. Sure. First, what needs to be thought through once you identify the types of data desired? Yeah, so once you have figured out your problem, once you figure out what is the right data that you have, I think the next step is thinking about the ways to collect this data, as well as the data itself, right? So you need to think about the reproducibility of your machine learning model because essentially the data that you have can help develop that reproducibility of your data, reproducibility of your model, the automation piece of it. So if are we trying to train our model on a regular basis? If that's the case, the data also needs to be updated on a regular basis. So you should also have some sort of pipeline in place that can help support this automation and make sure that the database is frequently getting updated. Another thing is sometimes the data can be missing. So you might not have all the data at hand. So in that case, what do you do? So you need to figure out is there a way we can generate synthetic data? Is there some crowdsourcing way of fetching all of this data? Can we look at other different data sources out there which are open and public for us to use? And yeah, so I think there are these different challenges that you'll come across based upon the use case and the data at hand. And just thinking about those before would definitely help us through the entire life cycle. Okay, and you may have answered some of this already, but another question is what are your challenges of bringing together and collecting data from all the different sources? You've already mentioned a few, but what are you, in your experience, what are the challenges there? Yeah, I think the biggest challenge is having everything in a certain unified format because sometimes a data source may expect a certain type of format. And then a different data source might have a different type of how the data is being structured. So figuring out what's the best way to combine and aggregate both and have them in a common place where you can easily extract and pull and explore the data. I think it would be the biggest challenge. But with that being said, you also have a lot of tools that can help you do this and you can aggregate even in spite of them being in different formats. There are ways to manipulate your data and make sure that they're the right format. And especially since data scientists are more comfortable using notebooks, using Python, I think that gives more flexibility for us to actually do all of that manipulation inside of notebooks using Python because that will give us that exploration more in depth for figuring out what's the right way the data can be structured and fed into the model ultimately. And another one more small question. Is this where you spend most of your time? Is the data engineering piece and like if you look at the overall project, is this where you spend the largest bulk of time figuring this part out? It really depends. Sometimes, yes, you might be spending weeks here because the data is just not there. It's just not enough. Even if you have a machine learning model that you're ready to explore, it will not be as performing as you would expect if you don't have the right data. So it's always crucial to make sure that you do have the right data before you actually dive into all those complex models because the model might not even give you the right accuracy if the data is not in place. So I think sometimes there is a lot of challenge in figuring out the exact data that we need to get to and how do we get it? For example, we were playing with this GitHub data. So there are multiple ways to get GitHub data. There's an API that you can use. There are some open source tools out there, but everything comes with some challenges and some roadblocks that you'll have to face. So in case of an API, for example, we were getting rate limited because there's only so many requests you can make against the API to fetch those data. So such things will come up depending upon how you want to extract the data. But yeah, sometimes it's as seamless and the data is already ready for you. So you're just ready to dive into the model pieces. So yeah, it really depends on the problem. Thank you. Wonderful. So there are no questions in chat. So if you're all done, I'm happy to move to the next section. Yes, definitely. Thank you. All right. So hang on. Who's presenting next? Who's there? That would be me. Oh, okay. Akanksha. Okay. Hang on a second. Would you like to present your screen? Sure. Okay. Awesome. So hey everyone, this is Akanksha. I'm a data scientist in the office of the CTO with the Emerging Technologies Group. And today I will be talking about the AI ML workflow lifecycle that will consist of the model development and the model deployment. So a typical project lifecycle starting with data engineers working on gathering and preparing the data to make it ready for the data scientists to develop some machine learning models. A data set is a starting point in the journey of building a machine learning model. Simply put, the data set would essentially be an m cross n matrix with m, the columns would represent the features and n, the rows would be the samples. Next, the data scientists would start with exploratory data analysis to gain a preliminary understanding of the data, like descriptive statistics and some data visualizations. Next, this data will be subjected to various checks and cleaning processes to ensure the quality of the data before we start to develop the machine learning models for your intelligent applications. Well, you may ask, what is a machine learning model? A machine learning model is a file that has been trained to recognize certain types of patterns. You can train a model over a set of data, providing it an algorithm that you can use to reason over and learn from this data. And once you have trained this model, you can use it to reason over data that it may not have seen before and make predictions about this new data. A machine learning model could simply solve problems like identifying or distinguishing between cats and dogs, or bigger problems like predicting COVID cases or recognizing voice patterns or text predictions for our emails. We'll go over this in more detail when we go over the demo. And once the model is in place, we have these powered intelligent apps that are deployed and the machine learning models start inferencing, that is, making predictions based on the new data that it sees. And once all of this is done, you may have to monitor and manage your machine learning models in production to make sure that they are making the right predictions. And if not, they have to be retrained as it's needed. Now let's take a look at a demo where we are going to try to predict and recognize some of the car license plates. Just give me one second. Yes, here we go. So this is a Jupyter Hub environment, which consists of various interactive methods to create Jupyter notebooks, deploy models, create pipelines. So starting with this notebook that we've already created, and we have a machine learning model that will recognize and extract license plates just by looking at some car pictures. Starting with this, we have to first move on and install some libraries. If you're already using an Operate First cluster or Red Hat Open Data Science cluster, most of these libraries would be pre-installed for you. If not, you can go ahead and pivot install using the requirements file. And once all these importing is done, let's do the finding the plate. And to find a plate for us humans, it's really easy to understand what is a car plate. It's just a rectangular object in the front and back of the car. But for a machine to understand this, it's slightly more complicated. So what we're going to do is first we are going to find where the plate is located. So for a computer to understand, we will feed in some code for it to understand that it is a rectangular object in the front or back of the car. So we start by writing some helper functions and processing functions that will allow us to load a model that will help us to identify the location of the license plate in a car image. So we start by loading a model. Then try to resize the image of the car on the size that we had eventually trained our model on. So it might happen in cases where your model is trained on a certain size of images and the input that you might want to get an output of would not be the same size. So we'll try to pre-process this new image and fit it to the same size of the image that is more convenient for a model to understand. Moving forward, we will try to find out the image where the license plate lies and finally like to detect the license plate from the entire image. So let me make a detection. We load the model and we run our various functions on it. And let's see for this car image where we see that the number plate was on the backside of the car. Our model was able to identify the license plate area. So one job is done here. Now that we have a new set of image, which is the license plate, we would like to detect the various characters that are written on the license plate. So we first try to grab the contour of each digit starting from left to the right because that's how we need numbers. And one character at a time, we will try to recognize all the characters that are written on this license plate and then put it all together in one function. And finally load this new model to detect the numbers in the license plate. And finally to detect the number plate number. It might not be super accurate sometimes like it might miss one or two numbers here or there, but it does the job fairly okay, I assume. Once this model is completely up and running, we will put all images to this model and see how the detection looks like. So looking at this, it was able to detect some of the numbers in this license plate, then detected M as an N. So these are few technical inaccuracies that we might face when we are trading models. It might not be 100% accurate, but it does solve the purpose. And this is how we try to get a lot of number plates out of these test images. Next up, we have already deployed a Flask application for this model. And you will see how we can test out this application like we just give an input image and our model would be able to detect what is the license plate number, even without knowing what this car or this number plate looks like. So let's go ahead and start testing this application. First, we will start by testing if the server is running or not. Status okay, so that means the Flask application is running fine. Now we will pass our new image which is cars 374. Let me go ahead and show you what this image looks like. So this is the car image and we want to detect this number plate out of this car. And we pass this data file inside the predictions of our Flask application. Let's try to run that. This is like a command line command like call command to use and detect different answers using your up and running model, but we can also do the same using the Python code. We will try to run this. And here we see that the number that was detected was S7 JDV. If you go back and look at the car image, it is absolutely the same. So this is what a typical model life cycle looks like getting the data. We train the model on tons of images of the cars help help the model to understand what is the location of the license plate. We are moving forward get each set of numbers inside the license plate and deploy this as a Flask application. So you don't have to rerun your model again and again on heavy data sets and consume a lot of time. So this is my set on model development and deployment. Please let me know if you all have any questions. Oh, that was wonderful. Okay, so I don't see any questions in the chat, but I have a couple. Choose which model to develop. So this answer would depend on various, various factors like what is the problem statement, what is the kind of input you have and what is the kind of output you're expecting. And what is the type of the data size of the data also depends on various other factors like how much time you are able to invest and what is the set of resources you have but to give you a broad understanding of how we decide a model. So looking at the data set, there are three broad ways to classify this. So if you have a label data set, say you have a set of images of cats and dogs and you already know what is a cat and what is the dog. You have a supervised set of training. It is a supervised machine learning approach in which you have a label data set. And this is what you're going to go ahead with. In cases where you don't have a label data set, for example, you just have a bunch of images of cats and dogs. So you would go ahead with this unsupervised learning approach which would be like classifying these images into just two parts. Like all of the cats might be looking alike and all of the dogs might be looking alike. So this is an unsupervised learning approach. Whereas there is another approach which is reinforcement learning where your agent takes a course of action in its environment and tries to learn by trial and error method in an effort to maximize the reward. For example, if we were to play tic-tac-toe and we make mistakes time to time and we realize after a point that, okay, this is not how I was supposed to play. So using the same mentality of understanding by our mistakes, we try to train models using the reinforcement learning approach. So having said that it might not be a very straightforward approach when you are choosing a model. It might be a mixture of combination of a few different approaches. But this is the basic approach that we try to follow when we try to choose what model we want to use. Okay, so we have a question in chat, which is for you, what is the hardest part of building a model? All right, so there could be scenarios where you don't have enough data. I think that is one of the hardest parts when it comes to building a model. Like sometimes you know what problem you want to solve, but you don't have enough data to train your models on. So according to me, I think that is the hardest part of building a model. Okay, and last question before we move on. So what are the best tools for machine learning models to develop a machine learning model in your opinion? So there are tons of AI and machine learning libraries and packages to help you out when you are building a machine learning solution. But as I said earlier, it would definitely depend on the kind of data and the resources you have. But to name a few, we have Scikit-learn, PyTorch, TensorFlow, Keras, RapidMiner and XGBoost to name a few. But again, you could maybe use a combination of these to come up with the best model for your problem statement. Wonderful. Thank you so much. Thank you. Okay, ladies, I'd like to ask who has not, or Selby or Orindrilla, who's going next? I can go next. Okay, wonderful. Would you like to share your screen? Absolutely. All right, so here we are. My name is Orindrilla Chatterjee. I'm a senior data scientist in the emerging technologies group at Red Hat. So I'm going to be talking a little bit about AI ML pipelines now that we have an idea of how do you engineer your data? How do you train your machine learning models? How do you deploy them? And now let's take a look at how would you put all of these pieces together in an automated pipeline? So your data science project may involve all of these steps like we saw earlier, data collection, feature engineering and model training. And there might be several instances where you want to automate these different steps of your data science workflow and run your notebooks or run your scripts in an automated fashion in series or in parallel. So AI ML pipelines come handy there. So they are basically reproducible workflows that can be used to automate repetitive steps in a machine learning workflow starting from data extraction, pre-processing to model training and deployment. So how do AI ML pipelines make a data scientist's life easier? So first of all, ML pipelines help automate these manual and repetitive ML tasks. So it helps create faster iteration cycles by parallelizing these tasks. We also know that predefined and automated components leads to better reproducibility and more consistent workflows. So that's a great reason why you can choose to use AI ML pipelines. Next, pipelines can also help introduce better version control and they can help monitor your ML code and artifacts and makes it easier to track different executions and runs. So what tools would you actually use to train or create these pipelines? So using Ilyra and Kubeflow pipelines, which are two tools which are available on Open Data Hub, which is a project that's being operated in an open operate-first environment, we can directly use Jupyter notebooks or scripts and automate them in a pipeline without having to translate them into scripts. So Ilyra is basically a notebook pipelines visual editor that is present and available as a part of Jupyter Lab where most data scientists develop and it enables us to trigger Kubeflow pipelines from a very intuitive UI. And Kubeflow pipelines, which is a platform for building and deploying scalable machine learning workflows allows us to actually put all of these pieces together and run these in an automated fashion. So now that we know how AI ML pipelines can help us and make our lives easier, let's take a look at what are some prerequisites or what would you actually need to create these pipelines. So first of all, you need a Jupyter Lab instance to actually interact with Ilyra. Secondly, you would need a Kubeflow pipelines instance deployed within your cluster. You can easily deploy it on a Kubernetes-based environment like OpenShift. You would also need the Jupyter notebooks or scripts to actually automate the code or the environment where you're actually developing these models. And finally, you would need container images consisting of the notebook dependencies, which you actually want to feed into the Kubeflow pipelines environment or instance so that these notebooks can run seamlessly. So I'm going to quickly share a demo of a pipeline running in action. So let me pull that up. Great. So as you can see here, we have an already created pipeline in a Jupyter Lab environment here. You can see two nodes which are connected using the Ilyra UI. So I'm going to quickly go over both of these steps in the pipeline. So the first step is actually a step which collects some raw data from a CI data platform called Test Grid and saves the data onto S3 storage. And the second step actually downloads that data from the S3 storage and calculates certain metrics and stores the calculated metrics back onto S3 storage stuff. So let's see how we can add a new node or a new notebook to this pipeline. So for that, I would simply drag and drop a new notebook and connect those notebooks using the Ilyra UI. So once we have done that, we also want to set the properties for the notebook so we can select the runtime image that the notebook is running in, which we have already created. We can also select the resources needed to run these notebooks like the CPU, the RAM. We'd also select some file dependencies which the notebook or the script can depend on. So I'm selecting the metric template notebook which these notebooks need to actually run. And we can also select any environment variables which the notebooks need to function such as the cloud object storage credentials or any other environment variables that you have defined within the notebook. So once you have done that, I would save this pipeline and I would run this pipeline to actually see it in action. For that, I'm going to just simply hit this run or play button and I would provide it a name. I would also select a pre-existing Kubeflow Pipelines runtime configuration that I have created earlier. And once I have selected that, I would hit OK. And in order to view this pipeline in action and move over to the Kubeflow Pipelines UI, I can click Run Details here. This actually takes us to the Kubeflow Pipelines UI where we can see the notebooks running as nodes. So as you can see here, the first notebook has already started running. Now in order to view and debug any logs during the execution of this notebook, we can go to logs and detect if anything's going wrong. So this is how a pipeline actually looks like after it has completed executing successfully. There should be a green check mark next to each step of the pipeline. Now in order to view any metrics that were captured during the execution of these notebooks, we can also do that by going to the run output. So if I'm capturing certain metrics like the number of tests, number of bill failures, let's say the time taken to run certain cells and so on and so forth, we can also view that here. This can be especially helpful when you're trying to capture metrics during the execution of a model training pipeline. Let's say the model performance metrics like accuracy and you can also use this to track your hyperparameters, etc. So this was a simple demo of two notebooks or two simple three nodes in action. But this can be extended to complicated AIML development workflows. You could automate your data engineering, your data preparation, and you can also include your model development nodes all in an AIML pipeline in order to make your AIML development cycle pretty seamless. So yeah, that was a quick demo. Please feel free to let me know if you have any questions. Sure, I actually do. Okay, so I assume a data scientist doesn't come out of school knowing how to create pipelines. So what kind of skill set would you recommend that they work on? Absolutely. So as you saw earlier, this ML pipeline consists of the sequence of steps which can actually help a data scientist automate these different bits. So when you go from a prototyping phase into a production ready model, that's when you want to actually take these bits and put them together in an automated fashion. So the main goal of these pipelines is actually to make a data scientist life easier. So luckily you do not really need any advanced skills to set these different pipelines up. So the tools that we saw earlier like Ellyra, Kubeflow Pipelines, even MLflow, all of these make it really easy to set up these AIML pipelines from within a Jupyter notebook environment, which a data scientist is already comfortable with using, like you can use a simple drag and drop UI or a command line interface, whatever the data scientist or the developer is comfortable with. So and like you saw, most of these tools like Kubeflow Pipelines also provide a very neat UI for data scientists to compare model runs. So to answer your question in a just actually not a lot of advanced skills, most of these tools that are available luckily make it very easy for data scientists to build AIML pipelines. Okay, so you mentioned there's Ellyra which you showed in Kubeflow and then what were the MLflow was one of the other tools you mentioned? Are there any others? Yeah, actually there are like several tools which you can use. These were just two examples. You can use our Go workflows to create your own pipelines. You can also even use like simple cron jobs to tie together different notebooks or different scripts. A recent tool that I recently learned last week Apache NIFI is also a really cool tool that you can use to build AIML pipelines. It actually helps even build out the smallest of components like data transformation, loading models, exporting results and you can have these individual steps or nodes for the smallest of tasks which a data scientist is familiar with doing. So yeah, there are actually several tools you can use to build out these pipelines. Fantastic. Thank you. Thank you so much. Okay. Selby, I think you're next. Analia. Let me try to show you the demand for more computing power. Should I have started? No worries. But let me just show my presentation slide first. Okay. Is it visible? Is that was it showing it properly? On my left screen it does. Okay, hang on a second. But I see you sharing my window to that works. I can share my screen, I guess, or I can just introduce myself at the time. I think it's okay. Hello everybody. My name is Selby. I am a software engineer in AI services. So I work with AI ML ISVs and trying to bring their technology into Red Hat OpenShift also Red Hat OpenShift data science. Let me just mark our product. But today I'll be showing a prerecorded demo on distributed training across several GPU nodes. This demo was done by me and Diane Padema, who's not present today. But let me start the demo. In our short demo, we'll focus on how we can utilize all the resources available to us and speed up the training of a convolutional neural network model on an MNIST data set. We're going to use OpenShift, container platform, running Kubeflow and PyTorch to distribute machine learning training across multiple GPU enabled nodes. We'll use the distributed data parallel feature in PyTorch to replicate the model across multiple GPUs on multiple nodes and divide the training data so that each of the model replicas processes a subset of the total training data. With DDP, you replicate the model on multiple GPUs, run the forward and backward paths of the model in parallel, and then you synchronize, do a synchronize operation to aggregate all of the gradients. And you choose how to do this synchronization operation based on the back end that you specify. And you can choose from nickel, glue, or MPI. In our case, we're using NVIDIA T4 GPUs and we'll use nickel for our back end. We're going to look at our PyTorch Python code or script. And so let's say you have a convolutional network model that consists of two separate layers. And what you want to do is you want to distribute it on several GPUs across the nodes. So the one important information that you need is the master address, master port, rank, and world size. World size is a total number of your GPUs. Ranking is what's the ranking of those GPUs among all of them.  So this information actually will be fed by Kubeflow, so you don't have to worry about it at all. It will be set up. But the things that we do need to worry about and set up is the disk init process group. So this command will initialize a distribution process. And here's where we choose our back end, which is nickel in our case because of the GPUs. And another important part of this is this distributed data parallel as mentioned earlier. Here we're going to wrap the model using the DDP and split the data across the devices. And here we're going to see which model and the IDs of the device send the output. And then the rest is we just download the data set. We do the sampling and load it. And then we run our machine learning model training. And after that we do the forward and backward paths and optimize. And eventually we're going to spit out our epochs and steps and loss on each step. And here we're going to set the environment variables that I mentioned earlier. We just set the environments and this data would be fed by the GPU flow. And here we're using 20 epochs. Now let's move to the Dockerfile. So now that we have our PyTorch script, let's actually containerize it. And in this Dockerfile, that's what we're doing. What we do is we have the already built image up on which we're going to build the further. It already has the UBI, CUDA and PyTorch and all other related dependencies installed. What we're going to do is we're going to copy that main.by the PyTorch code that we looked at earlier and we'll automatically execute the script inside the container upon deployment. Look at the YAML that we need to run this on OpenShift. And there are four things that you need to specify when you create an object in OpenShift or Kubernetes. So the first thing that you need is the API version which we have here. And this says which version of Kubernetes API you're using when you create the object. Then we have the next field is kind, which says what kind of object you're creating and we're creating a PyTorch job object. And we can do this because we've used the PyTorch job CRD in this cluster. So it's aware of this type of object. Then we specify metadata, which uniquely identifies the object and what namespace it will run in. And then fourth, we will specify with the spec the desired state for the object. So the coomflow operator is going to see this spec desired state and it is going to cause this to happen. It's going to go to the Kubernetes scheduler and say, hey, I need a master with one GPU. So find me a node that has one GPU and I need a worker with one GPU also. So please find me a node with one GPU. And that could happen on the same node or separate nodes. In this case, it's going to happen on separate nodes. So now we'll take a look at what it looks like when you run this YAML. The file that we just looked at, and we're creating a custom resource definition, which is PyTorch jobs. We can see that the pods are being created and initialized. So we can go to the master pod, look at the logs, and I can see that it's now downloading the data sets. Meanwhile, we can also go to our NVIDIA GPUs and see what's happening there. So we can see it's already being used. We can go to the other terminal and check the other one. The other GPU is also being used and you can see it's going through the epochs. All right, now it's on epoch number six. Now we can see that our training has been complete in one minute and 15 seconds. We can also go to the other pod and check how that went. And we can see that the other worker pod has completed also in one minute and 16 seconds. And the important part here that I would like to point out is the environment variables. So as I mentioned earlier for distributed PyTorch job, we need information master pod, master address, world size, and rank. And this information is obtained by Qflow and fed into the PyTorch. Yeah, and that should be all for my demo. Okay, so I have, of course, a couple of questions. So can you talk about how distributed training across GPU nodes fit into machine learning model monitoring and management? Sure, so basically with machine learning model and monitoring and management, what one can do is track performance, model retraining, CICD pipelines, or distribution of input and output data, or even monitor hardware metrics. And in our demo, we essentially like more or less collect the hardware metrics manually. So by testing this various hardware setups for distributed training as GPUs, CPUs, and so on, we can collect those hardware metrics and optimize the setup accordingly for like a more efficient and faster model training and inference. Okay. Okay, and one last question. All right. So in your demo, I saw that the workflow was distributed across two nodes with GPUs. So how does that scale? Like let's scale that bigger to lots of several GPUs on several nodes. Yeah, the beautiful point about the Qflow is that it's actually agnostic to the GPU setup that you have. So one organization may want to scale up by adding more GPUs right on one node. But some of them, some others might want to scale out by adding more machines with GPUs. So but either way, no matter what kind of setup change happens with these GPUs, there's no need to update or change the Qflow settings. It just with the help of no future discovery and NVIDIA GPU operator that we have, it's automatically able to identify how many GPUs are there in total, like be several GPUs on one node, or several GPUs on several nodes. So it doesn't really matter. It just after your request, after the Qflow basically requests the GPU resources, it will set one of the GPUs as a master and the rest will be the workers. And that way we'll be able to orchestrate this whole distributed training. Oh, well, so you don't have to worry about. Okay. So if everyone's still with us, I'd like to bring everyone back just to say a nice thank you and everything. Hang on, hang on, hang on. Okay, here we are. So I really want to thank all of you for joining me this morning. That was wonderful and I certainly would like to do this again and maybe do a deep dive on particular topics and hear more about what you're what you're working on. And I hope you had fun and I hope our viewers had fun as well. So thank you all. I appreciate it. Thanks. Thank you. Thank you for having us. Here we go.


This is the VOA Special English Technology Report. Microsoft has announced its first tablet computer. The device, called the Surface, is designed to compete directly with the popular iPad. The computer software maker announced its new tablet in June at a media event in Los Angeles. Microsoft will offer two versions of the Surface. Each has a different processor. The Surface for Windows RT runs on an ARM processor. It is less powerful than the Surface for Windows 8 Pro, which uses an Intel processor. The RT is also thinner and weighs less than the Pro. Either tablet weighs no more than about 900 grams. Both have screens that are about 27 centimeters in size. That is a little bigger than the iPad screen. Microsoft has not said exactly when the new Surface will appear in stores. But it is expected to be sometime in September or October. This is about the same time that the company plans to release its new Windows 8 computer operating system. Brenda Estrella owns a laptop computer. She has been considering switching to a tablet. She says for her, the iPad is more of an entertainment device. In her words, if Microsoft is coming out with a tablet that actually you can type and actually do different things other than just swipe, then that would be better. Many technology lovers, including Catherine Clinch, are excited about the new device. She points out that the Surface stands by itself, so you don't have to buy a stand. Also, there is a keyboard that can also act as a cover. Those are all wonderful things, she says. But what interests her most is the potential to get rid of the netbook, to get rid of the laptop, to maybe even down the line get rid of the full computer. Fox Van Allen is a technology expert. He agrees the Surface seems to have more to offer than the iPad. He says it's a device that almost serves as a replacement for a PC. The Surface is meant to compete with the iPad, but not everyone is sure it is a better product. John Ayala says he is glad there is a competitor, but he is sticking with his iPad. For years, Microsoft has made software, not computers. A few earlier attempts by the company to make and sell hardware products failed. That includes the Zoom music player. It could not compete against Apple's iPod. Estimates place the value of the tablet computer market at about $79 billion this year. Microsoft hopes to capture a share of that market with its latest device. For VOA Special English, I'm Mario Ritter.


Thank you so much to all of our keynote speakers for preparing and making all of this happen. This was an amazing start to TubeCon CloudNativeCon Paris and before we head to break, we have a few important announcements for you. Tonight, there's going to be a wonderful evening of networking with the community, and we invite you to enjoy all of the entertainment that includes Lego car racing, VR, arcade games, lawn bowling, and so much more. At the CubeCrawl and CloudNative Fest, starting at 6 p.m. right here in the Solutions Showcase. Yes, thanks to our sponsors at Evinta and Upbound. You can join us for an unforgettable night of love, the fun, and experiences, including the clash loop backoff where competitors can compete head-to-head by showcasing the technical creativity and skill. You won't want to miss it. Also, if you ever wondered how to sign in Kubernetes and CloudNative, the CNTF diff and how the hearing working group will be hosting sessions to help you learn how to say hello or coffee, and other basic CloudNative signs. So be sure to check sketch.com for the times of these really cool interactive crash courses. That sounds really cool. Don't forget that the AI Hub is back. So come join us in rooms W06-07 starting at 11.15 today as we explore AI topics that we've all heard today in the keynotes so far, like LLMs, AI policy, data governance, and more. So you can check out sketch.com for a detailed breakdown of the AI Hub agenda today. And lastly, we've got a few folks to thank for, because without them, none of this would have been possible. So let's take a moment to thank all of our sponsors as well as our DanCons Scholarship Fund sponsors. Now to talk more about the scholarship, it's very important because it helps individuals who are either need-based, come from an underrepresented group in technology, or a project maintainer or a student who may not have the opportunity to attend otherwise. So please, please show your support and attend our amazing sponsors in the solutions showcase throughout the event. And finally, but certainly not the last one to thank for, it's like a huge, huge, huge thank you to our amazing program committee and our track trust who worked so hard to help us select all of these amazing sessions for this event. Definitely, and thank you so much all for joining us for today's keynote. Breakout sessions begin at 11.15 and there are some really great tutorials that you don't want to miss. And after each session today, we invite you to leave feedback on sketch.com. Feedback is a gift and we take it so seriously. There's a link that can be found on each session's schedule page and we hope you have a great day and see you back right here tomorrow. Thank you.


My name is Billy Foster. This is my wife Renee Foster. We're from Gary, Indiana. And I was originally diagnosed with a cancerous tumor in my kidney in 1996. And we had the tumor removed at that time. And not until 2007 did I hear from the cancer again. We found that I had a metastasis to my lung and liver and also some lesions found in my brain. And since that time we've been at the University of Chicago. We tried a couple of drugs. One was Sutent. And now I'm in a clinical trial which is, fortunately for me, been able to keep the cancer under control and even reduce some of the tumors. At one time at the beginning of this I had to have a gamma knife surgery. I had three lesions in my brain. And those were eliminated with a gamma knife surgery. So I've been very fortunate, I feel. I think it's important to keep a good attitude about this and do a lot of praying because I feel that we really aren't in control. God's really in control of this whole situation. And you have to have good family support and good support for your friends. I have several friends that have me on their prayer list and so I feel very fortunate in that. And right now I'm feeling pretty good. Renee is my caregiver. Renee do you have? Of course I do. We have sustained the tests of time. And in talking to those of you who are listening and coping with the bout of cancer, it is a battle. And you have to kind of make your mind up as to whether or not you want to be a survivor. And a lot of times you'll find that you'll need people that are around you who offer support but then you have to have those who listen. You have to seek out if possible those who understand about education in the areas that your cancer or certain kidney cancer or whatever it is. You have to then be knowledgeable and if you're not able to do that then you have to seek out somebody who is your advocate who then supports you in the endeavors of surviving with this cancer. And all in all it has made us a stronger couple and we have an understanding that there's a higher power who controls everything. And in the course of learning who you are and whose you are you then develop a sense of communication.  Sometimes you'll have to say, shut up, stop talking, let me do it. But I, as his advocate, as his caregiver have learned to understand the valuable importance of connecting with people and understanding how to communicate to get what you need and to resolve the matter as best you can. And we'd like to thank the Kidney Cancer Association and the University of Chicago for giving us this opportunity. We've had a wonderful conference. And also the various doctors, the nurses, the various individuals who are the ones who take your blood pressure, the ones who monitor your tests that like you have the one test where he found out that you could put crystal light in it. And it would, what's the name of that? CT scan, you can put crystal light in there. So you learn, you live and learn.


I think we've got someone in the room who's going to be in the room and any stragglers are late, so let's get started. Okay, sounds good. So, I'm Khalid Aziz and I have been working with Matthew on a concept to share page tables across processes. And please feel free to ask questions as they come up while we are talking. So with that, I'll hand over to Matthew to introduce the concept. I put up this, I drew this little diagram last night to illustrate the differences between what we already have and what we intend to have going forward. What's the difference between what we do and what we do with M-Share? So on the left there you can see a pair of single threaded processes. There are two M-M structs. Each of them has a task struct which points the M-M struct straightforward. In the middle there we have a multi-threaded process. We all understand what a multi-threaded process is, but at this point I would say. That is, we've got three task structs, each of them pointing to a different part of the process because we're all executing in different parts of the multi-threaded process address space. So the new stuff is on the right. We have two independent M-M structs. Each happens to be single threaded. They don't have to be. They could be multi-threaded. But they have this M-Share region. And this M-Share region points off into a third M-M struct which has no task structs pointing to it. It's not a thread. It has no threads. Okay. So let's look at what happens. If you could start clicking whenever it seems appropriate. So the first single threaded process maps page using MAP-Shared. Hey, so we've got a little red box. Now any modification that makes to it is going to be reflected with anyone else. Like in the second process which also calls M-Map-Shared. Great. Now either of them can modify it and they'll both see the other's modifications. Right. And then the third process, the multi-threaded process also maps it MAP-Shared. And so the difference here is that if the first process calls M-Protect to make it read only, it will still see the modifications from the other processes. But if the multi-threaded process calls M-Protect, all three of those threads will not be able to write to it, right? So the other processes are deciding on their own ideas, but deciding for themselves what their protections are. But the multi-threaded process, one of these threads can decide for all of the others what are the protections on this because they're all sharing the same area. Okay. So now we're going to have the M-Sharing tasks call M-Map. Click. Okay. And they all appear all at once, right? This is mapped into everybody's address space all at the same time because these are literally shared page tables. And so if any of them calls M-Protect, again, that happens to all of them just like with the multi-threaded process except that in the purple areas, things aren't shared, but in the green area, everything is shared. And I apologize if anyone is colorblind. By the green area, I mean the stuff that's labeled M-Share region. So we're not talking about processes that don't know each other. We are talking about processes which have decided to cooperate with each other. We're talking about maybe a web server that has decided it's going to have some shared chunks of memory and some not shared chunks of memory. Or we're talking about a database maybe that has some very special and unusual requirements. But it's not just, you know, for our employer, right? I think this is a generally interesting thing to do. Holly, do you want to take it away and talk a bit more about the implementation? Of course. Okay, so as Matthew talked about, there's already shared page tables. Why don't we have cooperating processes, shared page tables as well? And one of the big benefits we see is that when you have got multiple processes sharing the same pages, especially when you have thousands of processes, we are wasting a lot of space in each process keeping its own copy of page table entry. So in order to solve that problem, what we are proposing is the M share mechanism which is an operating mechanism because the processes have to have certain level of trust between them. So a process says I'm creating an M share region and I want to be able to share it with everyone else. And there will be some authentication mechanism, essentially permissions. So whichever other process has the permission to share that region will be able to share it. So the process that's creating the M share region informs the kernel, then it starts mapping objects in that region. And any other process that has a permission to access that M share region can then map that region into its own address space and get access to all of the objects that are currently mapped in there and then be able to read and write to those regions. So I have sent out a couple of past series and we started out with an API that is a combination of an in-memory file system and system call and based upon the feedback, I'm now working on an API that looks more like this. The implementation adds a new in-memory file system M share FS. So you mount this file system and when you mount that file system, now you can open files into that file system. Opening a file into that file system creates an M share region and the name of the file is what you refer to the M share region as for the other processes to be able to attach to that M share region. So the process creates a new file in there and then it can M map the FTA gets back from the file system. When it does the M map, that's when we know what the extent of that region is like starting address and the size and at that point the region is defined and now another process can come in, open that same file in M share FS and then M map that FD in its address space. So now all of the objects that were in the M share region become visible to this new process and when the process is done with its interest in the M share region it can simply call an unlink on the file and when the last reference to the M share region is removed, the file is removed. So I have started out with an initial implementation where the focus is on getting the core functionality working and then expand it as the need arises. So one of the questions is what is the size of the region, the minimum size of the region you want to share? I have started out with a pager size which is fairly large but it keeps things simple for now but we can potentially look at sharing at part of PMD sizes as well. So what that implies is a process that wants to share an M share region and it wants to map the M share region in its address space and it has to know what is the size and alignment requirement we are looking at for this specific region. So when you mount M share FS it will populate a file M share info that will provide that information. You just open that, read it and you have the size and alignment requirement. So once the process has created a file and in effect created a new M share region what happens is the open causes a new MM structure to be allocated. This MM structure as Matthew pointed out is not assigned to a task. It is stands on its own. This MM structure is primarily used to hold all of the VMAs that represent the M share region and it also holds all of the page tables for that M share region. And then when the first map happens if the process that is doing that MM already has objects mapped in that region will copy over all of the VMAs from the creating process into this new MM structure and then we set a new flag VM share PT on each of the VMAs that now correlate with the M share region. And then currently I also change the VM MM pointer to point to this new VMAs, this new MM structure. So I have a picture, let me bring that up this way. So you have a process. It has its own MM structure. It has got a bunch of VMAs in there. Each VMA will have its VMM pointing back to its own MM structure except for the VMA that maps onto the M share region and that VMA will have its VMM pointing to the MM structure for the M share region as well as the VM shared PT flag set on it. So when page fault occurs, we look up the VMA where the page fault occurred and if the VM shared PT flag is set on that VMA we know it's an M shared region. So we go to the M share MM instructor instead and continue the page fault handling from there. So as a result we will bring in all the page tables from the shared region and that's what we'll operate on all of these VMAs. Okay, so I have a basically working implementation but there are still a lot of loose ends that should be tied up and questions that we have that we are trying to answer as we go through this process. granularity, I already had just said something to look at API as we have today. One of the questions is does that do the job? Is it sufficient? Is it reasonable and maintainable? And then when we look at M mapping these M share regions currently our intent is to allow M mapping only the entire region but should we be looking at allowing M map of partial regions and then the same thing happens even on the M unmapped when a process does an unmapped do we force it to unmap the entire region or can it leave some regions still mapped in? Then also how would a process map in new objects? So we need to look into how to support M remap when the process that created the original M share region it potentially had objects mapped in there but another process that's not sharing that region could potentially unmap the existing object and map in a new one. So we should possibly support that. There's a question of do we allow these M share regions to stack on top of each other? Which makes it very interesting and then we have potential interaction of these regions with user fault FD. How do we support that? Or do we say no, you can't use user fault FD with the M share regions? Any questions at this point? Yes. Can I get a microphone? Okay, I've got a microphone.  It sounds scary a lot because we have that natural kind of one process has a MM struct. So what happens, who is in charge of that kind of other space? So how do we do accounting? If you try to M map fix into a area that it's conflicting in what happens or what happens, does it happen to everybody who is sharing that MM? So there are very, very many questions that I... I don't know where to start. That's entirely fair. I mean, I think to a large point is it's the same as if they were threaded, like fully threaded. I don't think there's been partially threaded, right? These two processes are somewhat threaded with each other. They shared that chunk of their address space and so for that chunk of their address space they behave as if they were threads of the same process but for the rest of it it's independent. They're independent processes. Yeah, right. I'm not really sure that's answering questions. I think it answers your map fixed question. Yes, so let's say that you are map fixing into that a conflicting area. Does that really mean that you are map fixing to everybody sharing that thing? Yes. Okay, I thought that clone VM without clone thread or seek hand or what's that combination when you essentially share your MM without sharing a common understanding of signals was the biggest sin. This seems to be going beyond that. Yeah. Right. So we do have two disjointed processes that are sharing parts of their address space which is why there are more questions and one of the questions on the next slide is security. What kind of security issues this raises? So you are right that there are a number of these issues that we do have to address as we go along. So in my mind the big question is is the concept useful and is the way we are thinking about it reasonable and then can we address the potential issues that come up? Regarding the API, is file system a must? Maybe a system calls it returns file descriptor and then cooperating processes can the same rights share it and it will do... That was my original design. And the original API did not survive first contact with the enemy I mean the user. They did not enjoy the STM rights approach. They want to have a name in a file system. And having a name in a file system it gives you a little bit more... It gives you the enumerability which M share regions actually exist which otherwise with just FDs I mean you can use F user I guess but there's a lot of scanning to do to find out which shared regions exist. Right and FD based API one of the issues that was pointed out with that was now it becomes a client server model so we have to have a server that holds that FD that can pass it to clients using STM rights and now we have built a bottleneck. I just wanted to add about the API that I knew there was evil underneath but the API struck me as a really elegant thing it looked like a version 2 after you already considered the usual approaches so I was shocked and refreshed to see the API so if you do it please my vote is for that one. Thank you. Dan's got a question to him? I see some questions in the chat people are asking about does it work with Gazer pages fast how do you do the accounting for pinning pages with this? So right now I'm focusing on the core functionality and get user pages fast is on my list of things to address but yes the goal is to make it work with that as well. So in general I think this is a pretty interesting concept but I agree that it's scary so I was wondering if we could come up with some kind of whitelist what you can actually do with something like that meaning for example if you want to do an M lock like no that's not going to happen if you want to map private memory that's not going to happen until we fully explored the problem space so you would actually start with a whitelist that is fairly small but gets the job done and as we go we actually unlock features and once we know what we're actually dealing with and that might be like one part of the issue to page pinning we don't support it there for example maybe that could be one approach to move forward because it is scary I mean I'm not going to lie Our customer's requirement is with DAX they didn't want to have terabytes of memory but they were M mapping and using more terabytes of memory for page tables and they were getting in actual storage because if you do the calculations with about 10,000 processes that's where you end up about twice as many bytes used for page tables as you do for actual storage so that wasn't great and it totally understood why so the original request was hey give DAX the same functionality as huge TLBFS and we all kind of got a bit twitchy because everybody hates huge TLBFS and so it was like okay what can we do instead what can we do to make this not awful and this is what we came up with something different awful now I don't know I think this is kind of interesting so we're interested in the concept of we have an MM struct with no threads attached to it but we do have a file descriptor for it so now we have a file descriptor for an MM struct and what can we do with that concept like could we make system faster or better because now you could implement the system libc call by saying okay start create a new MM struct and then start adding things to it and then say okay now you have and then breathe life into it and have it go off and be that new thing rather than forking and copying all your VMAs and copying all your file descriptors if we had a way say okay let's create a process over here it doesn't have any life but we'll inject file descriptors into it and we'll do various things to it and then set it going I think that could be a really interesting concept Sounds to me like Frankenstein's monster It sounds to you like what? Like Frankenstein's monster Well yeah I mean you kind of saw it but clone is like that right you saw a process together from various different things I think Frankenstein would have loved us Right? A misunderstood genius just like us Maybe if you come up with a new model and a name for this instead of just limiting it to there's a new mechanism isn't it fascinating but maybe go one step further since you can't just have random stuff floating around without a model really because then it's too hard to think about security but if you said well this is a new thing let's call it I don't know a lightweight process or a Frankenstein process or come up with something and then here are the rules for that and they just happen to match exactly what you're doing here that might be more acceptable even though it seems bolder I think it would help So my question would be as I said I cannot really comprehend the consequences of this because this is just beyond any imagination but can you explain what's the actual you said that you really hate huge DLBFS but can you explain why this cannot be done through a mapping so that you actually have something that is really specific to that at this range for example so why cannot you really share on the per mapping basis so essentially just abstract what the huge DLBFS is doing in somehow more shareable way I mean that's ugly right but that's level the level of ugliness is somewhere else than an unbound number of MM structs in something that barely resembles a process okay so when we are looking for different semantics when you start to talk about sharing page tables you go back to the different semantics that you have for example M-Protect and the customer does in fact want to do M-Protect and it wants the semantics I described that if you M-Protect it in one task it becomes M-Protected for all tasks that are sharing it that's a feature not a bug they want to literally share the page tables they want it to be as if it were threaded for that chunk of their address space they just don't want to use threads because M-MapsM sucks and obviously we are trying to fix that in a different way but I mean for now M-MapsM sucks so I don't know that yet we definitely don't want to make it so that if you call M-MapsMapShared you magically get these new semantics because that will break existing programs that are not expecting to say oh somebody else called M-Protect and now I can't read this or write to this page that's got to be opt-in semantics which means at least a new flag to M-Map if not a new syscore and so I thought it was clear to be more open about we are going to share this region of the address space than it was to say if you use this magic flag to M-Map then the chunk of your address space which contains this mapping is now shared with everybody else exactly this makes it very explicit that the process is choosing to do that as opposed to a side effect of a system called that they were already making before so it becomes a deliberate decision to opt into this mechanism so how many of these shared regions are supported per process an unlimited, unbounded or is it just the one? there's no particular limit I mean you can create as many files in M-SharedFS as you want that wouldn't be typically what I would expect to see a program do what I would expect to see a program do is say I want this very large chunk of my address space to be shared and then they can map several smaller things in it and they can be a mixture of map private, map shared files and on you can put all kinds of things in there but you have to choose to put them in if you just call M-Map and you don't specify anything about the private bits of your address space it doesn't sometimes invisibly push into the shared region because that would be a horribly stupid thing the unbounded question is kind of why I thought of this would it be better to just have one possible M-Share versus an unlimited and maybe not have the VMA pointing to it but some other mechanism to say that I would and a special VMA of some type that just blocks it off as opposed to I'm just thinking of what the VMA M-M struck pointer is used for and how entangled into the system that is right now how much will change or will need to change to support this right so just to amplify what you're talking about here is something like we tried to do a write into a read-only VMA and now we need to send a signal but we stop at the M-M which is shared and now which process do we actually send the SIG bus to that kind of thing yeah and who owns the thing and also find VMA rather than find M-M Jason has some comments Jason would you like to go live and explain what you're saying or should I just read that can you hear me yes yes we can oh wow thank you I was just wondering that you're listening to you talk about the use case for DAX I was thinking maybe there's some way to to like make the DAX files hold the page tables basically you're shaking your head I can see you on video okay so that that's kind of what we're considering doing for huge TLBFS right now huge TLBFS goes around and looks at other processes see if anybody else has it mapped and shares their page table with them rather than storing the page table pointers in the iNode so I meant to suggest this to Mike but I only thought of this this morning when I was out for my run and haven't had the chance to chat to Mike yet so this is probably quite a surprise to him but but it seems to have sort of a conceptual appeal like if you could say when I instantiated VMA and I go to install all the pages like maybe instead of installing a bunch of pages maybe I just get one like a p4d or something like that something big and and it's just owned by the VMA or the iNode or whatever mechanism is underneath and all the reference counting kind of follows that so you just install the one thing and you know it follows along naturally you get the file system access that you wanted from your customer yeah so essentially the huge TLBFS concept but somehow obstructed right yeah I think we could do that but but again I think we still need some opt-in from the process to this because they need to understand the the new MProtek semantics it can be the sorry Jason go ahead I'm sorry I was going to say you have to be like a new MAP option like MAP shared protections goodness we have so many MAP flags left but I think the the opt-in and the API is not really necessary tied to the necessity to have additional MM struct so you can have the M share API like Khalid suggested and then create some different kind of object of full-blown MM to just link it from VMAs that are shared and to signify that there is the sharing mechanism going on there that is potentially possible keeping creating a different separate MM structure does simplify being able to use that existing mechanism for managing all of the page tables it's an abbreviated MM struct that we are using just as a container because we have a simple way to manipulate it using existing functions I mean we did add the MAP MAP shared validate flag to allow for MAP sync so like we actually get submit we have maybe I think we still have room for there to have a MAP flag that will fail if it's not supported and work so if you want to do a new MAP it's not I don't think it's impossible okay that's something to consider your thoughts Matthew you know I'm if people are just too scared of this feature then we can go back to doing something else I just thought it was a really interesting idea to say you know we will take these two processes and be partially threaded between them I thought that was worth exploring but you know if it turns out to be in search of a user then you know we can do something simpler okay yeah how about this I'm getting ready to send out the next version of the patch in the next two weeks or so so feel free to comment give us more feedback and we'll tear it apart from this Jan you wanted to say something yeah I just wanted so Matthew said the reason by the application doesn't fully go through it is the MAPSM but then we have actually very similar problems with contention on MAPSM of the like shared mm yeah because basically where we have this M shared region then basically we have to have some synchronization again of changes which are presumably going to happen under MAPSM of the like share mm struct and then we have basically the same MAPSM contention issues there as we would have like in the threaded case or thank you Jan I knew somebody was going to bring that up so thank you for doing it to a certain extent yes to a certain extent no I mean these processes don't just have the big M shared region I mean if all of the M protect M lock M unlock whatever activity was happening on that region then yes you'd be right but they also have that the other stuff going on which actually is private to that thread and so effectively split the MAPSM in half half and if it's like 80-20 in one direction or the other then you know you're going to see some amount of improvement or dis improvement but the key is that they get to choose they get to ask for it yeah and so to that is actually second question so instead of like providing this M shared region wouldn't it be better to like provide the way to a thread to set aside part of other space where you know it would be really thread private and then we would we could do simple locking there like without all the MAPSM denses essentially because we know it's thread local man it's like talking to a version of myself from 12 months ago yeah we absolutely consider that and we talked about that but we ended up deciding that well if you let a thread have its own MM struct isn't that essentially the same thing are we just doing the same thing but in reverse well I think conceptually it might be easier to grasp the situation that you know this range of address space is thread private to a particle thread and we don't do MAPSM locking there essentially then it is to grasp the concept of shared MM struct yeah I guess I kind of I'm sorry but I have to cut it here because we are overflowing to the next slot and this seems like more questions we ask the more questions we get so yeah it's definitely interesting but that's fine we can always continue this discussion on the next slot right right thank you all thank you