Hello and welcome. My name is Shannon Kemp and I'm the Executive Editor for Data Diversity. We would like to thank you for joining this month's installment of the Monthly Game International Webinar Series. This Webinar Series is designed to give our Enterprise Data World Conference attendees education year-round. And we are actually presenting a live from Enterprise Data World 2016, having a great event so far. Just a couple of points to get us started. Due to the large number of people that attend these sessions, you will be muted during the webinar. For questions, we've been collecting them by the Q&A in the bottom right-hand corner of your screen. Or if you'd like to tweet, we encourage you to share your questions via Twitter using hashtag GAMA. If you would like to chat with us or with each other, we certainly encourage you to do so. Just click the chat icon in the top right for that feature. As always, you will send a follow-up email within two business days, containing links to the slides, the recording of the session, and additional information requests throughout the webinar. Now let me formally introduce today's speaker, Glenn Bell. Glenn is the Director at Visual Explanations. He has worked in data management for almost 30 years. He is the President of GAMA in Australia and delivers training in CDMP and data modeling. He is an independent consultant and has worked for a variety of organizations in Canada, Europe, Malaysia, Singapore, New Zealand, and throughout Australia. He holds the Master of Business Information Technology Management from the University of Technology Sydney, and a Bachelor of Science Computing in Mathematics from the University of Queensland. He also has a CDMP and CBIP certification, both at mastery level. Fantastic. And with that, I will turn this presentation over to Glenn to get us started. Hello and welcome. Thank you, Shannon. So today we're going to be drawing on my experiences with a federal government client in Australia where they needed to integrate data from different sources and needed to do that in very controlled fashion. So I have a series of slides that I'll be working through. The first seven slides will set the context and the drivers and the reasons why we have to be so careful when consolidating data. And then slide number eight is actually my favorite slide. It's got a knockout diagram that I'm sure you will all love, and that's the one where we will really focus a fair bit of attention. So if you get a little bit bored with that during the slides up to number seven, don't leave. It will get very, very exciting on slide number eight. So the dangers of consolidating data is that if a citizen would consolidate data from their education, from their employment, maybe some health records, then that becomes very powerful and could be misused and represents reputational risk for the agency that's consolidating the data and a breach of privacy for the citizen that's involved. So you might say, well, if there are these dangers involved, why consolidate data at all? Keep away from the danger and just don't do it. However, for government to be able to ensure that it is producing the outcomes for citizens that it wants to achieve, frequently it needs to be able to get data that's located in more than one government agency. If you could imagine as a hypothetical example, a citizen with disabilities and the government puts in a program to provide support for that person and they want to do that so that they get improved outcomes for that citizen in terms of their improved employment, improved educational outcomes, if possible, better health outcomes and just general increase in integration with the community. No one agency has that data so to be able to measure the efficiency and the effectiveness of that policy, you need to consolidate the data. And that then allows us to make policy decisions that are informed and make the best use of public money. So we're really needing to engage in the dangerous activity of consolidating data from different sources to be able to ensure that we are getting the best outcomes for citizens in a cost-effective manner. And what I'm going to explore in this presentation is how to manage the dangers that are involved, the risks that are involved. So the Australian government has recognised, as has many governments around the world, these dangers and the fact that they still need to embark on this activity. And so within Australia they have established what's called the Integration Authority Accreditation. This is not law, there is no active parliament. What has occurred is that the heads of government departments in Australia known as secretaries for the major government departments have come together in a committee recognised that they need to consolidate data but acknowledged the dangers and then developed the accreditation program to give comfort to those who are leading their agencies that all the controls are in place when consolidating data to protect citizen privacy. So I'm going to be stepping you through an example of a Integration Authority Accreditation application submission. And in essence, key elements include having people who are doing the consolidating work, performing different roles and those roles then give you different access to the data at various stages along the integration path. Also the implementation of this can be done in either a cheap and manual way and my experience with this government agency was a cheaper manual approach. So we would have paper based logs with managers physically sitting in the room when people were performing various roles and ensuring as a check that nothing wrong, bad happened. However you can also go down the expensive and automated route so more mature agencies use automated monitoring and logging of what is occurring during the integration process and sophisticated mechanisms for controlling the access for people's roles based on using virtual machines and more advanced concepts like that. So when I step you through the fantastic diagram that will be appearing on page on slide nine, eight, it's technology independent but you can do it at different levels of expense. So we've discussed the dangers that are involved but the need to still consolidate data and so how do you go about it? Well essentially there's a whole set of policies and procedures that surround the consolidating of data if you were to get the integration accreditation to give comfort to this agency that things are being done properly and audibility. There will be different roles for loading, separating, linking and analysing the data and my diagram on page on slide eight will highlight that. There's also costs that are involved. There's increased audit resources both internal and external for ensuring that data is secure and the policies are being followed and even in the cheap and cheerful approach you still have to do work around software. In the example that I participated in rather than a virtual machine they did folder level locking to be able to control who could write and who could read to a particular folder but that still requires effort to do that. So I suppose the tension that comes in is that the easiest thing in the world is if an agency wanted to consolidate data is to get some extracts from their other government agencies and they give it to a person and they just work with it and put it all together. But unfortunately that would expose all sorts of dangers if that person went rogue. And I might just as an aside talk about how important these controls are as an example in the Australian marketplace and Australian governments. This is a publicly available example. It's been in the newspaper so I'm not giving away any secrets. The Australian Bureau of Statistics which is probably the most sophisticated gold standard for integrating data is very strict controls and procedures and culture around protecting data. Unfortunately one person went rogue and they were involved in releasing quarterly economic data and they had gone to a university with a friend who wound up working for one of Australia's major banks, the National Australia Bank. And prior to the quarterly results being released this person would sms the figures down to his trader friend at NAB and then this trader would take advantage of that and put in trades that would make lots of money. Now the National Australia Bank's risk controls quickly highlighted that this particular person was off the scale and alerted the Australian Federal Police and the Australian Bureau of Statistics to this. And it didn't take too long for them to figure out what happened in this scenario. Both of the individuals that were involved have gone to jail for years. The longer term is for the National Australia Bank person who was also found guilty of bribing a public officer. And for me one of the more important things is that the Australian Federal Police also reviewed what the ABS had done in terms of their procedures and controls and they were found to be fine. What I mean to say is that there was no systemic risk. There were all the controls that you could possibly want to have in place were in place and all the culture and training and everything around it. But we had a person who went rogue, possibly a mental health issue. But for the Australian community, they could take satisfaction that, okay, if something bad happens then the individuals are penalised and jailed for a significant period of time and that the controls are in place to be able to ensure that this is not a systemic problem and that it should never happen again. And the sorts of things that I'm going to be showing you on the infamous slide 8, look at how you can ensure that the systems are in place to prevent as far as humanly possible a breach occurring. So the integration authority procedures come in when there's at least two data sets, well, fair enough. Can't consolidate one data set with itself. It's for statistical research purposes. So there are other powers that control consolidating data for investigating fraud or other operational matters. It's where the data is subject to the Privacy Act, so it's personal information, potential to compromise commonwealth outcomes, and it occurs at a unit record level. So it's where it's individual people data that's happening. On slide 7, one more to go before slide 8 and the integration authority has a set of criteria to evaluate the suitability of the agency's controls to see if they were worthy of the integration accreditation. So criteria one, which is really super important, is the ability to ensure secure data management. So linkage is not performed directly. That is, it's not just one person who's in control of these files and consolidating, but it's separated out. And that the data separation principle is being adhered to, and I'll talk about that in the next slide, and that when you finally get to the point where you do the analysis, the analysis files do not contain identifying data, such as name, date of birth, address, or the identifier from the original data set. So you will now see the much-heralded slide 8, which gives you the end-to-end process for being able to control the consolidation process. So you will see that there are three large boxes, the shaded boxes that form the background, and they are labelled the librarian, the linker, and the analytic. These are the roles that are performed, and a person, they could be different people doing these roles, or if it's the same person, they can only be in the role at one point in time. So we'll start off with the librarian role. Actually, I'll tell you what, I'll just give you a brief overview, and then I will drill down, as John Zuckman would say, in excruciating levels of detail to make sure everything's perfectly clear. So the librarian is the one who receives the data set that needs to be consolidated, the data sets, and they will then put that data set through an extract and standardisation process, and they will then separate the data into the analysis data. You'll see the analysis file 1 on the right-hand side, and the match file data, the data that will be used to match the individuals involved. The match file data, so I'm now looking at the linker box, helps to link the data together, enables the linking of the data to a link file, and then eventually the two analysis files in the analytic box get matched, and you get a consolidated analysis file that has no identifying information. So it is completely de-identified. So I'm just going to use my pinch feature on my Surface Pro 3, and I'm also checking on Shannon's screen, and that looks like you can view what's going on here. So let's look at the initial steps that impact on the librarian. So the librarian receives the original data set, but before they can do that, there's a set of approvals that need to occur. So the external agency that is providing the data set needs to approve, but also the request from the agency doing the consolidation needs to go through a set of approvals as well to ensure that the... In the Commonwealth Government, we have called the senior executive service level, the leaders of the organisation, to understand that this linkage is going to occur and approve receiving the data from the other external agency. So there is full visibility that this is happening, and I would also have, as a data management person, also have the involvement of the Data Management Committee or Data Management Board, or whatever the government structure that is in place, to also provide that SES officer with the guidance that yes, this is required, this consolidation work is required. The trigger for this consolidation work may have been some analysis of the performance of the organisation or in government. It could be something such as a ministerial question. Step number two, so before we even get started with the librarian and the original data set number one, step number two is to ensure that all the people who are involved in this project, whatever the role they're doing, librarian, linker, or analytic, that they have no conflicts of interest. So in my scenario, we developed a no conflicts of interest form and the person had to sign that, and we would then scan that and store it in the document management system. And that form also made people aware that under the act that guided this agency, that if you were to compromise data, the penalty was two years in jail. So people are very clear that this is a serious business and you will be prosecuted if you misuse this information. Step number three, before this project begins, under the integration accreditation by the Australian government, is to inform the community and the National Statistical Service, NSS, set up a website. You can go to it on nss.gov.au and any integration projects that are happening by any accredited agency are listed there so that there is transparency to the community that this is occurring. So the next step then finally is to receive the data set. But obviously this can't be sent as an attachment to an email. We used a secure FTP product to be able to ensure that the data was encrypted, even though we were behind government firewalls to just make absolutely sure that there could be no tampering with the data. Okay, we're now back to our diagram and we're looking at the librarians role. So you will see the original data set has the record ID number one, a record ID. That's the record ID for this data set as held by the original agency, the agency that provided this data. So that has to be stripped out before we can let it go through to the analysis stage because if you've got a hold of that record ID and you were looking at consolidated data, you could go back to the original data set and you would know who the person was. There will be the other things that are in the original data set are the data linkage items. So there are the things that are used to try to determine if two people are the same person. In our example, we used what are known as statistical linkage keys and SLK, which takes various characters out of your first name and surname and your date of birth. And it's a pretty good guide. It's not perfect. Nothing's perfect when you're trying to match data, but it's a fairly good guide for being able to see that two people are the same person. So we need those data linkage items. If you're interested in statistical linkage keys, there's an excellent explanation on Wikipedia. So that gets extracted into a landing area. You'll see the blue box. It gets standardized. And by that, I mean things like Rob, Robert, Robbie and Bob all get standardized to some standard name to help give the statistical linkage key its best chance matching people. Also things like sex for male, female, maybe in the source system. It was called one, two. So you would have some translation standardization to be able to put it into a standard form to again help with the matching of the people. And then the third step that occurs by the librarian is to separate the data. And you will see that they separate data that goes into the linker area. I'm just moving my finger along and looking at Shannon's thing. Yes. So you see match file number one. Oh, sorry. I should have said back at the librarian, source file number one over here. Yes, you can see my mouse. So we've put in a project ID so that we've replaced the original identifier with a identifier that we've created uniquely for this project, the data linkage items. Linkage checked items are things such as country of origin or other things that could help with the matching, but are necessarily part of a SLK, a statistical linkage key. Demographic items are the things that we want to and analyze, which might be things about, you know, where this person lives or could be things about their salary or whatever, whatever democratic, demographic items, not democratic. And some other analysis items. Now in the match file, so this is this part here, so when the person performing the role of the librarian separates the data and creates the match file, in our example we used folder level locking so that librarian could only write to the match file but they could not read the match file. Read anything that's in that folder where that match file goes into. So only the person performing the role of linker, that is that they had signed on to the system with various user IDs and passwords that gave them the authority to go into the linker folder, could see the match file number one. And more importantly, the link file that gets created. And I'll just show you the other thing that comes out of the separate, is that we go over to the analytic area and you'll see analysis file number one, which just has the demographic items, the analysis items. So the match file has stuff that's for matching and the analysis file has stuff for performing analysis, demographic, analytical type items. Again, that analysis file, the person who is the librarian, can only write to that folder, that's the analytic folder, but they can't read from it because if they could, they'd be able to see the consolidated analysis file, which you'll see down here, and that would then compromise things because they've got the librarian has access to the original file. So match file, so we've got the analysis person. So match file number one has been created. It's got a project ID. It's got data linkage items. It's got linkage check items to help try to link people together. Then we receive a second dataset and the librarian follows the same process of extracting, standardizing, getting rid of the record identifier, putting in a project identifier, this is project ID number two, linkage items and demographic items, and they do the same thing. They separate the data into data that will be used for matching and for analysis. So you can see that we've got a match file here and then over on the purple bit, over down here, we've got a demographic items. All right. So then the person who is performing the role of the linker comes in. They access their folder. They've been provided with match file one and match file two. They don't know who these people, well, they don't know any demographic sort of information or analysis information about these people. They only have the information that will be used for linking people for identifying who's who. And so they run the link process, generating statistical linkage keys and mapping people up and create what's called the link file. So now we have project ID number one, project ID number two, and we created a link ID, but we're saying we've allocated a random number to this person here and a random number to this person over here from two different datasets, the same person, and they're linked together. That then takes us through to the analysis portion. So the person doing the linker role can then write that link file into the analytics section, but they can't read anything in the analytics section. The person who is then doing the analytic part then runs a match program based on the link file to match the demographic stuff to produce a consolidated analysis file that is de-identified but has rich information about people, but it has been confidentialised but it is de-identified. That's the thrust of what I'm talking about, but I will just, and I'm expecting a few questions from slide number eight, but I'll just finish off the slide presentation and then open it up for questions. Shannon, are we getting a few questions through? Go ahead. Okay. Page down. So Criterion 1 also goes on to having order programs in to ensure that data security is informed, that the people who are performing the roles of the librarian, the linker and the analytics have been subjected to police checks, that the physical security on the premises where this data is being held is in place with key cards and those sorts of mechanisms. So it's not much good having all these sign-in, sign-out and all that sort of stuff to come in and steal the computer that the data is on, that the internet gateway security is this data accessible from the internet, if so, what sort of security is there in place and that there's an information security policy in place in the Australian Federal Government called the Information Security Manual, the ISM. And then there was a set of other criteria about controls of external data access. Not only does it need to be de-identified, which is the natural process of this activity, but it also needs to be confidentialised, which means that even though we don't know who it is, if the data related to a small country town and the person earns a million dollars a year and they had HIV AIDS, then people in that community could pretty much work out who it is. So there's a process that statisticians use to perturb the data to avoid that from happening. Criteria number three, availability of appropriate skills for the people who are doing this work need to have the experience and the expertise to do that. Criteria four is about the technical capability so that there is the software and hardware and other things in place to make it happen. Number five is the lack of conflict of interest and again the reforms that people would fill in. Number six, that the cultural values, so when you were onboarded that there was training in place or records that you'd been through the training. And other evidence that there was cultural values about confidentialising and protecting information. Criteria seven is transparency of operations which was things like publishing the details about this project, this integration project on the internet and the existence of public governance and institutional frameworks. In our example, not surprisingly we would have data management committees and we would follow as an institutional framework the DMVoc wheel. And that was it. Number 1127. So that was the essence of the presentation. It is about managing the dangers about data being consolidating making sure that there are controls in place to protect that data and I wanted to give you a specific example about how you could go about doing that. Sure. And we have questions coming in already certainly about slide number, the infamous slide number eight. Yes. I expected that to happen. So just a reminder to everyone before we get started on those questions I will be sending a follow-up email to the slides so you can have a copy of slide number eight as well as a recording of the webinar. So one questioner has three related questions let me read all three of them at once and then we can kind of question out a little bit here. So you showed flowing in one direction left to right and do you also implement a right to left process so that an authorized person can go back up the flow to see where that process may occur. And I can read it again too. The diagram implies a large number. The second question is the diagram implies large number of changes to data during the workflow. Do you attach metadata to data to document the province of the data? The thought being that some chain of custody of data needs to be documented in some manner. You mentioned that. As you mentioned, the folder level is locking. Is metadata attached to the folder? Is that the lowest level of tracking that your control process implements? Yes. Great questions. So yes, you're right. So the diagram goes left to right and the questioner has pointed out what it gets to the analysis stage and clearly something is wrong. So we looked so there are trade-offs, something. So we did consider that as things move from the librarian to the linker that we would delete the intermediary files and similarly when it went from the linker to the analytics stage that we would delete those files. But as the questioner has pointed out if something is bound to be wrong at the analysis stage then you would have to recreate the whole process again. So our procedures were to leave as a final step to delete the files. So the files are protected if you like because of the folder level security but still and all at the end of the project those intermediary files need to be deleted. Otherwise, if they're just sitting there hanging around they're vulnerable and a breach could occur. So I hope I've answered that question sufficiently. The next question was about changes to the data. So we don't actually change any of the information about the person in terms of their demographics and that. We do generate an SLK a statistical linkage key. This is all done as part of one project. So you sort of run it through. We have metadata in the sense that we have a paper form that is filled in by each person as they do each step as well as the signature of the manager who's watching each of these people doing these steps and the start time and the end time and signatures as declarations that this is what occurred. So that was our sort of cheap, manual approach if you like. Obviously you can do something much more sophisticated with monitoring and logging and keeping data lineage metadata in place. But we did it as a paper based form that went step one, step two, step three with signatures and dates and times. Still metadata.  The question was about folder level. Was that our main security mechanism? And the answer is yes. However, I would emphasize that this space that this work is carried out on is only accessible to the three roles here. The librarian, the linker and the analytics person. So the only people can get to that space and then at a folder level the librarian could only write to that, read and write to their space. But they could only write to the linker and write to the analytic area. And similarly the linker could read the linker folder but only write to the analytic and then the analytics person could only read in the analytics folder. So obviously you can spend more money. You can do sophisticated things which other agencies do around the area of virtual machines and that. We've been in this project, we were subject to external audits and we're deemed to be satisfactory. So what's my answer to that question? Fabulous. Metadata has become such a hot topic. We get questions in every Web Spinar no matter what the topic is related to metadata. Anything you want to expand on that and just the importance of it in the digital area to play a role in this project? It's absolutely crucial to this type of work because we're going to be subject to continual audits. We are undertaking activities that are very dangerous. We are consolidating data about citizens and so we are continually subjected to audits and the only way that the auditor can be satisfied is if we have the metadata in place to be able to support that yes, we did what was here and it's an approved process. Otherwise, what have you got? You've got this lovely diagram slide number 8 but if nobody's following it then it's complete pass. It's a waste of time. Sure. Everyone's been quiet today no additional questions coming in anything else coming in that you want to know to put it in the Q&A section we'll just hang out here for a couple of minutes. A lot of our attendees are here at the conference anything that you're excited about and seeing or you've learned so far at EDW? Yeah, so there's been some well, as always, great sessions. Happy to meet you at EDW. I'm particularly interested in creating data models for NoSQL databases and Dave Wells gave a great tutorial yesterday on that. He had some very clear examples about the syntax that you would use for document data stores and how that could be modeled using a particular modeling technique and I found that really quite useful. Especially somebody who I teach Graham Simpson's data modeling essentials course in Australia people increasingly in that course is very rich and does a great job around the relational space but I'm always keen to hear about extensions or ways that we can do stuff for NoSQL databases. Oh yes, that's marvelous. I also went to Donna Burbank's session on setting up a data management practice and that was perfect for me just at this point in time because I'm currently at a different client setting up a data management practice so I drew a lot of comfort from the fact that I was doing the sort of things that Donna was saying but I also like the fact that I can go back to Sydney Australia and sit down with my clients and step through some of the slides in particular that I thought were pertinent to them and be able to say well look that's international best thinking so that sort of stuff is just gold. I love it, we do have a couple of questions coming in, more questions too on your topic and we love Donna too and how long did it take to set up the library linker and analytic LDCs? To be honest that was kind of the easier part of it so I've just clicked slide number nine with all the stuff about physical security so I had to demonstrate two auditors that the physical security logs were all in place which meant I had to talk to the property security people and I said I need a list of all the people who have access to this area and of course the property people were not giving you that and I need to be able to show the auditors that these are in place so there were things that were very time consuming like the internet gateway security I had to find out what that was and get the proper documentation to be able to support the auditors to say yes this is in place or this particular display is not being accessed and you know I had to demonstrate that our people had the appropriate skills so I had to get their resumes put it into a format so the submission itself might have been 40 pages but it then referred to all these other supporting documents so I don't know why I'm using my hands here because people can't see but I'm showing Shannon two hands that are about three feet apart because I just had this wide of material to be able to substantiate that we had all the checks and controls in place and that was that took time yeah if there is additional documentation available about the various criteria for example written policy and procedure documents well I do have those things but they're sort of client specific so I don't they're general I've sort of generalised as much as I can here but if you go to the nss.gov.au website you'll see you'll see you can find submissions that have happened from other agencies and you can look at their submissions and you can look at the criteria that's used however they don't actually publish the direct policies and procedures that are used by these agencies I suppose there is a bit of security stuff around this as well sure anything else you want to add I bet all the questions that we've got come in so far anything else you want to add before we end the session no I think I've performed excruciating detail on slide number eight and look even if you didn't grasp everything that was on slide number eight I suppose my take home message is to firstly if you're in the workplace and somebody says we need to consolidate data from different organisations that you immediately raise the warning bell and say what are the privacy implications and what are our controls and procedures around that and if you're getting hassled by a boss just do it then perhaps you can take a copy of these slides and sit them down and show them slide number eight and show them the level of sophistication that's required if you're going to do this in a controlled manner that protects people's privacy and if I've achieved that in this webinar then I'm really happy well you're certainly getting some accolades here well I just want to thank you for this great presentation and thanks to our attendees as always for being engaged in everything that we do and asking such great questions and we will sign off here from Enterprise Data Roll 2016 and we will see you next month it's the next webinar and I hope everyone has a great day cheers


from a company called Whitehawk, and my company specializes in open data consulting all over the world. At the beginning of the year, end of last year, I started doing FOI requests to every single local authority and asking them please provide me with a complete list of all the commercial properties as well as the vacancies, so let me know exactly which ones are empty and which ones are not. And local authorities quite a lot, a significant number, maybe about 15 to 20 percent turned around and said we will refuse, we refuse to release data because it causes crime if anybody knows the properties are empty. And I was like well how, how can an empty property cause crime? So I started a second parallel FOI series request which is to the local police to ask, tell me about crime and empty commercial properties. And that parallel resulted in police authorities do not collect such information. The Thames Valley here in Oxford they do and essentially it's none, one or two incidents in empty properties a year. And when you compare that to the number of crimes reported in actively traded business properties, we're talking, you know, 30, 40,000 crimes in particular local authorities relative to 1 to 0 crimes in empty properties. So we went to the Information Commissioner's office and I submitted something like 60 appeals to them and we're currently going through a process with the Information Commissioner's office. They're likely to issue a finding in the next month or so, which basically the Information Commissioner says, you have to prove that there is a legitimate connection between publication and not. So we're likely to see something like that coming through in the next month or two.  So either present the evidence that tells me that you're unable to release because of crime and I know your local police authority do not collect this because we hold every single police authority in the UK to ask whether or not they collect this data and we know only to do. So you're going to have to give us legitimate reasons or publish. I think every single person who has an interest in data and has a particular, you know, whatever it might be, an edge to scratch or a commercial interest should be pursuing the FI process. If the data and government hands, we need to find a way to get it out. I believe that if we had the FI process before the address database was sold off and we'd gotten that into public hands, that would have been, you know, instead now we're back in a situation where the government has to spend money to recreate this database that they had all along. So definitely use the FI process and, you know, you've just got to recognize that in some cases, particularly if that data are politically sensitive, you're going to go through a slog but you're just going to be persistent and you're just going to be very clear about what it is you want. I'd love to get to the point where I don't have to do it through FI requests and the local authorities just go cool we see the value of this and instead of waiting for you to come and ask me we just publish it because we know we're going to get the request and, you know, maybe a few will in the short term or we'll just keep plugging away with their FI's.


Okay, in this segment what we're going to do, we're going to continue looking at the flat plate. However, what we're going to be doing is looking at that for a turbulent boundary layer. So if you recall from the end of the last lecture we talked about a fluid friction analogy or the Colburn analogy, that is the best thing to use if you're trying to determine the convective heat transfer coefficient for a turbulent boundary layer over a flat plate. And if you recall the relationship looks something like this. And what it enabled us to do was rely on a wealth of information that currently exists in terms of the friction coefficient for flat plate turbulent boundary layer flows. Okay, so what we're going to do, let's take a look at using the Colburn analogy for a turbulent boundary layer on a flat plate and this will be the case of an isothermal flat plate. Now for the turbulent boundary layer we have no analytical solution for the velocity profile. The only thing we have are basically curve fits for different types of turbulent boundary layer flows. You can take a look at my fluid mechanics course. I do have some information on how to try to determine the velocity profile for the turbulent boundary layer but we run into what is called the closure problem with turbulence and it has to do with terms that arise in the Navier-Stokes equations that we are not able to solve for the velocity profile when we have turbulence. And so this is an approximation that is sometimes used. It's a 1-7 power law and you can take that and you can couple it with a momentum, integral momentum analysis and use it to predict the boundary layer thickness. And there are a number of different ways that you can do this. In my fluid mechanic course I do this using a functional fit that von Karman came up with and I can't remember if it was a square or a cubic but anyways if you look there I go through a process of coming up with the boundary layer thickness but doing a similar sort of thing but using the 1-7 power law delta can be approximated to be. And remember this is a case where what we're assuming here is that we have a turbulent boundary layer starting right from the beginning. So in order to do that you would have to trip the boundary layers. Maybe you put a sandpaper or a wire on the front that would be X measured from that location there. So that gives us an expression for the boundary layer thickness but we're really after here we want to know the convective heat transfer coefficient and so for that what you do is you go and you use cold burn analogy and you find relationships that have been collected experimentally that characterize the friction coefficient. And so I'm just going to put one here and we'll work with that with the cold burn analogy but here is one for a skin friction coefficient and this is for a turbulent boundary layer and so CFX that denotes that it's the friction coefficient at a spatial location from the leading edge and we have a minus one fifth and so you can see they would use things we have similar sort of functional form that we had here but anyways that that is experimental data and this particular data set it turns out that it applies from the transition point up to a Reynolds number of 10 to the 8 so if you recall the boundary layer it's going to begin with the laminar boundary layer we go through transition and then it becomes turbulent and the turbulent boundary layer grows at a faster rate than the laminar boundary layer so this is our laminar boundary layer turbulent boundary layer and in this case what we're saying is that it only applies from the transitional for a flat plate typically we use 5 times 10 to the 5 for being the critical Reynolds number where we go through the transition process from laminar to turbulent but anyways that's not critically important what we're going to do here though we're going to take this skin friction coefficient and we're going to plug it into the cold burn analogy and we're trying to determine the new salt numbers so let's take a look at that so writing out the cold burn analogy that this is an expression that we had at the end of the last lecture and this was equal to it was equal to CFX divided by 2 so taking our friction coefficient and dividing by 2 we get 0.0296 so what I'm going to do now is I'm going to take these terms here and this term and bring them over to this side of the equation and what we then obtain is we get a new salt number that looks like this so that would then be an expression that we could use to determine heat transfer when we have a turbulent boundary layer and an isothermal plate and obviously isothermal the plate is going to be higher than the fluid temperature or lower but we're going to have heat transfer so there's some temperature differential driving the convective heat transfer process so that is an example of what you would do if you want to determine the convective heat transfer coefficient for a turbulent boundary layer you basically use cold burn analogy and you use the friction coefficient values plug it into cold burn analogy and then you obtain your expression you could do it experimentally as well and you curve it but why do it if you can easily get the values of CFX so that is the boundary layer what we're going to be doing in the next lecture is we're going to be looking at external flows on bluff bodies so cylinders spheres things like that and then after that we'll be looking at two bundles so multiple cylinders arranged in various patterns so that's where we're going for external force connection flows


How to find good gamer tags hey guys in this video I will show you eight tips to come up with a game attack on your own and examples How to make them fit your needs? number one exploring your passions Take some time to really delve into what makes you tech think about your favorite games movies both hobbies and even personal experiences What themes or characters resonate with you the most by understanding what you're passionate about You'll be better equipped with choosing a game attack that feels authentic and meaningful to you For example, if you'd like to Smith Smith machine or the killing man stick the charming man could work Try thinking outside the box to incorporate with you like into it number two unleashing creativity When brainstorming game attack ideas, don't be afraid to let your imagination run wild Experiment with different combinations of words phrases and concepts try mixing and matching elements from your interests to create something Unique and memorable consider wordplay puns and even creative spellings to add an extra layer of personality to your game attack For example, if you hate sniping you using an operator and valiant ops or ops Could be something that you can come up with number three reflecting your gaming style Consider how your game attack can reflect your gaming persona or play style Are you known for your strategic prowess or your quick reflexes or your fearless attitude? Try incorporating elements of your gaming identity into the game attack So you can help others get a sense of who you are as a player For example, if you'd love to be the teammate that juice the killer in Dead by Daylight You could put something like Master Juggler or if you're a toxic player you could literally put quote You died or quote lost to and the feed it'll say killed by quote you died So if that's not toxic, I don't know what is Number four embracing creativity in humor don't hesitate to inject some humor or personality into your game attack Puns jokes and clever wordplay kind of make your username seem more memorable and entertaining Just make sure it's in good taste and reflect the vibe that you want to convey to other players Maybe in four night you're going to maybe in four night you're going in for the meme so you put quote someone's girlfriend in case you get boxed up and coked and And hey, you're someone's girlfriend. So at least they don't know you at least they know you're not like a try hard Number six collaborating with friends You could make up something funny with your friends like for example five people named but one bought two bought three Etc and just go in a lobby and play together Super funny this way as like the enemy team They would just get wiped out for example by like five people with the same exact name like what the heck Some final things I want to add are finding the balance while creativity is key It's also important to strike a balance between uniqueness and simplicity Even for a game attack that is easy to pronounce and remember when we're still standing out from the crowd void overly complicated and obscure names that may be difficult for other players to understand or remember for example, I like to use dr. Lupo as a good example or Cipher BK because someone like ninja although like he may be popular and he can get that name in every game It's just way too broad in my opinion If you're not like a big streamer or something also before you get to attach to a particular game attack It's essential to check its availability on your gaming platform Most platforms have tools and features that allow you to search for available user names And you can also add special characters for example Like I think it's still Russian alphabet or something that the E is also like looks exactly like the English E So you just replace that and the username would not be taken and you just use it By taking the time to explore your patterns unleash your creativity and consider the impact of your game attack You'll be well on your way to finding a good username that reflects you as a person and brings a smile to everyone's face Thanks for watching and have a good day


Sailors aboard USS Bonhomme-Rachard gathered for a health fair where they learned about a range of topics from stress management to diet and exercise, even education and family planning. This was all a way to help make sailors aware of what it means to be well in the Navy. Wellness is a holistic effect and the approach therefore is holistic. So for wellness we thought that we could take all these themes and put a face to a name in terms of who's leading or championing these specific areas. So again, a great opportunity and a great reminder for sailors that whether we're in port or underway, we have a means of taking care of one another and that we're able to support the ongoing mission. Petty Officer Andrew Frisbee, Fleet Activities, Sausage, Japan.


Chi energy is not a myth anymore, you have scientific data that can show Chi energy can affect how your brain works, affect how your body works and even help you to unlock very specific special states of mind. Allow you to manifest things in your life, allow you to achieve things, accomplish things, create things in your life that normally you wouldn't be able to do.


Okay, Bolo Buddies, I just left the Goodwill Bin's outlet. I am going to head over to the Goodwill Retail Store, and we are gonna see if we can find any treasures. They are right next door, so I always hit them both up. Hey, Bolo Buddies, thanks for watching. It's from my little pony. Amazing, right? All right, let's get started. So when you walk in the door, it's typically like movies, and then Christmas, and then it goes on to hard goods, and the last place I stop is the toys. My favorite, oh, and books over here, books and movies. My favorite section is the toy section. If you've been following my channel, you guys all know that I love to sell toys. I am gonna look through the breakables today. I am gonna be honest with you, I don't pick anything up to buy. I don't like breakables. I don't like to ship them. I'm going to try to branch out and do more breakables in the future. So as I'm going through this section, please let me know in the comments if you see anything that I should have picked up, because I will do Christmas stuff. Like when I say breakables, I mean like glass and plates and stuff like that. I have done it from time to time, and I do well with it when I do, when I do do it, I do well, but it's just not my favorite thing. So that's what's great about eBay. I always say this, sell what you love, what makes your job not feel like a job.  I love listing the stuff I list. I get excited to list my new items, and I think that's what it's all about. Now, if I had somebody that would ship breakables for me, I would buy them all day long. It's not that I don't like the breakables, I just don't like to package them up. There were some staples. I don't know, I thought about picking those up for my stapler, but I'm thinking, how often do I actually use staples? So I ended up putting those back. I'm not against buying stuff for personal use at the thrift store, I definitely do that. So we're in the pots and pans. I know that people make tons of money in this category, but I am not well-versed in it. The homeschooling picker, pots and pans are like her thing. She just knows so much about it. So if you wanna learn about pots and pans, definitely check out her channel. She also sells stuff in tons of other categories also. She is a major Bolo Finder, so not just pots and pans. All right, let's see what else we have here. And again, if you guys see anything that pops out or stands out to you as a Bolo item or something that would sell quickly, please put it in the comments. That will help everybody learn. I am picking some things up. I'm looking at the bottoms. I'm seeing if things are signed. If I see something that's signed, I, from time to time, we'll look it up.  Those glasses, I think that was one of the 12 days of Christmas glasses, or not 12 days. Is it the, what is that, the first day? Yeah, yeah, that's why I was about to start singing. You don't want me to sing. But I have sold those in the past and I sold those individually. So they were easier to package and ship. Putting 12 glasses in one box is terrifying. That was a cute little vintage item, a little teacup and saucer. Some of those can do really well, but you really gotta know what you're looking for. Most of those are just gonna be bread and butter items. Some electronics, another thing I kinda stay away from. Now, I'll tell you who would be amazing at selling electronics. If I would ship them, is my husband. He is super knowledgeable about that stuff. But I don't wanna ship it. He had tons of turntables. And I'm like, I don't wanna ship them. I don't know how to ship them. I'm afraid they won't get there safe. So he ended up selling those on the marketplace. And somebody got a really good deal. Coffee mugs, you really gotta look those up because they're expensive to ship. With the shipping rates the way they are now, most of them, by the time you box them and put your bubble wrap and packaging paper, they're over a pound. So you're paying that priority mail rate. So unless it's a mug that I know is gonna sell quickly or for a good, great profit, I usually skip the mugs. There is one mug brand that I do pick up. It is Heartstone. I don't know why, but I find them all the time. And you know how there's just some items that you'll always pick up? I almost always pick them up if they're a dollar or less. Especially if they're Christmas and Halloween. Those are the ones that typically do better. And they're clearly marked on the bottom. If you've been at my channel for a while, you see them in my what sold videos. And if you guys are new to the channel, definitely check out my what sold videos. I have a series of bread and butter, which are my $35 or less items. And then I have my big money Bolo videos, what sold videos. That is kind of what I'm known for is what solds. I don't do a whole lot of thrift with me videos, but I'm starting, I'm gonna do more. Let's just say that. I'm going to start doing more footage like this. So let me know in the comments if you like this type of video. And if you want me to do more. And maybe any feedback that you have for me, things that you like that people do, or I don't know, stuff like that. So again, plates, I have sold them. I usually sell them individually. So that right there would be one plate quantity of eight or however many were there. That's just easier for me than shipping them in a lot. I will tell you that Toledo Antique has a video where they show you how they ship fine China. They will ship like 200 pieces at a time. It's just unbelievable. So make sure you check out their channel for shipping tips and they are also Bolo finders. They do estate sales and they are super, super knowledgeable. And they also do live events, live selling events. So check them out. All right, I'm looking at the toys. I promise you I buy some stuff. Oh my goodness, I feel like I've just been shopping and putting hardly anything in my cart. I did pick up some movies for my husband and I. So that's always a win. All right, let's see. When I'm not working, from time to time we will watch a movie. All right, does anybody feel like as an eBay seller, like you work in the morning, you work at night, you work in the evening, you just work all day.  I thought about picking this up, but decided it was pretty heavy and bulky and I just decided to leave it behind. But I probably could have sold it. If it was under five pounds it may have been a good Poshmark item. So if you go over five pounds on Poshmark you have to pay additional shipping fees for the overage. So on Poshmark I try to keep everything under five pounds unless it's a big money Bolo and I'm willing to take that hit on shipping because the shipping cost that's the overage is paid by the seller.  And they did have the music playing in the background. I wonder what that little RV, was that an RV? That was kind of cute. I probably should have looked at the bottom of that to look for a brand. When you're at the Goodwill toys are tough because sometimes they're overpriced. I really like to get my toys at the Goodwill. Not the Goodwill. Oh my goodness, at the yard sales and garage sales I like to go and I like to buy in bulk. I like to bundle. All right, here we are. Let's find some plush to take home. Let's find some plush to take home and sell. Alrighty, this little cat, his little hynees in the air. I don't know if that's a doorstop or what that is. I always look at tags, I'm looking for vintage tags, I'm looking for name brands, things that I know that'll sell. That's a Dr. Seuss. I left that one behind. There was a Dr. Seuss plush that I sold a while back. I believe I sold it on Mercari back when they were banning the books and certain books were not allowed. I'm pretty sure it sold around that time because everybody was searching for Dr. Seuss stuff. So this little baby doll, some of these dolls that have bodies like this can be a bolo. I did, I actually messaged the rebel reseller and I'm like, do you know anything about this doll? And she messaged me back and she told me to check the back of the neck, which I know to check the back of the neck of like porcelain dolls, but for some reason I didn't check the back of the neck of that doll. So I was able to look it up. I decided to leave it behind. The condition wasn't that great, but if you guys are not following the rebel reseller, definitely check out her channel. She is a bolo finder. This little guy got it home, did not work unfortunately. It had this little bracelet on its ankle. So what I did is I took that off and I made an eBay listing for the bracelet. I figured that's something that gets lost or that people just throw away and they might want it for their elephant. I think it's meant for someone to wear not the elephant, but they had it on the elephant. So that was good for me. Hopefully I'll make my money back with that bracelet. Unfortunately, that was a womp womp. This little turtle, I probably could have sold this. Now, if the plush is not marked, it is 99 cents. So a lot of the items in this bin that you're seeing are 99 cents, but a lot of them also have a sticker and they are higher. Like the elephant, I paid more than 99 cents for. Unfortunately, that happens sometimes. I was hoping to get the elephant and be able to test that on my reseller testing bolo products and then attach a video to my eBay listing to show how it works. And I was just really excited about it because I had never seen it. And unfortunately, it just wasn't gonna happen. And the battery compartment looked fine. I don't know why it didn't work. I probably should have had my husband look at it because he is like amazing at fixing things. He just can fix almost anything. All right, let's see. Okay, I did pick up this little Boyd's. It was only 99 cents. So Boyd's bear plush, if I can get them for a dollar or less, I will pick them up. Sometimes I will look them up and if they comp out higher, I will pay more for them. They are more of a long tail item but they do eventually sell. This little guy, I've got price tie. My 25% off sale has not kicked in on that yet but will soon. And what was it called? Something nod. But anyway, little clown. If you guys were at my reseller testing bolo products, I was a listing a lot of these items during a live show, a live working hangout. So some of these you guys have seen. This little bear did work. So I've got a video in my ebay description showing how that works. Super cute at like snores. Zombie bargain hunter was there watching that day and he was helping me with some of the pricing and stuff like that during the live show. So the live shows are super fun and educational. So definitely come check out that channel if you have not already. And I'll link it down in the description below. Plus I've also mentioned multiple channels during this haul. So please go check out those other YouTube channels. They are fantastic. All of them just great knowledge. Zombie bargain hunter and the rebel reseller sell tons and tons of plush. They are my go-to for plush. Just super, super knowledgeable. So check them out for sure. All right, we're getting towards the end of the tote here. Tote bin, we'll call it a crate bin. I'm not sure. This right here, I did look this up. It was super cute. It actually holds curtains. So in a kid's room, it's hands hook and loop together. The reason I say hook and loop is because you're not allowed to use the word Velcro on eBay. That is a Vero. They will pull your listing if you use the word Velcro. So do not use it, use hook and loop. I did recently do a series on Vero items not to look for. I typically talk about Bolo items. But what I did is in my Bolo Buddies Facebook group, I posted a post and I asked everybody to share items they have had removed from eBay, or items that have gotten them suspended. And I made a video because the Vero list is crazy long. You can definitely go read through it. But to me, personal experiences, things that have happened to actual people, actual resellers is the way to really learn what not to list. So I will pop up little screen share so you guys can look at those thumbnails and you can go find those on this channel. Definitely, definitely do that. I've had so many people message me and be like, oh my goodness, I had one of those listed. I took it down. Somebody else said, I wish I would have watched this video when you posted it because I just had one of those items removed by eBay. So throw it on while you're working and just play it. Play it over and over and get those items in your head so you do not buy them. This little flower was a singing flower. I probably would have picked it up, but the condition was not that great. I love those animated items, super, super fun. So did pass on that one due to condition. This also is an animated item. I put that one back also. I have one that has been listed for a while, but I think maybe mine doesn't work. I can't remember the details of that, but it's been sitting for a while. I probably should have checked. That's some sort of speaker, I think. I don't know, it connects into something. This dragon, little puppet, super, super cute. A lot of times I will pick up puppets, but again, I've got so much inventory. I have a money mountain that needs to be listed. I've been really, really chipping away at it. I've been doing really good, trying not to source as much, but I wanted to get some thrift with me content, and I wanted to go to the bends, and if I'm at the bends, I might as well come over here because they're next door, right? There's the other monkey, and the comparables on those were not great, so I just went ahead and left those behind. This little ballerina, is that a puppy dog? You guys, is it a dog? What is it? I think it's a dog, but really cute. Could not find another one like that specifically, but I did find a couple in that brand. So that's what I based my price on. I don't know what that was. Thank you guys all so much for being here. If you're new to my channel, I would appreciate it if you subscribed. Let me know in the comments how you found the channel. If you're new, if you're old, if you've been here forever, let me know how you found the channel. Did you see one of my videos on Facebook? Did somebody, did you hear about it on someone else's channel? I love hearing how you guys found the channel. So let me know down there in the comments. All right, I'm trying to remember. I don't think I got anything else. These right here, I left behind. Maybe that was a mistake. I remember having these as a kid by sister and I each had one. I think I might still have it, but just one of those things that I've hung on to that I probably just need to sell. I remember looking them up in the past and I don't think most of them are a bolo, but like with anything else, there are certain ones that are rare and hard to find that will bring more money. So I probably should have looked it up. Little puppet that was like, I was that supposed to be Santa maybe? I mean, there's a lot of money that I left behind. It's easy to do, you know, it's easy to do. So this mug over here, I'm gonna pick it up here in a minute. I bought this for my husband. He likes those big, like pottery type style mugs. And this is actually the second one that I've picked up from this same store on two different occasions. It's so strange, but I did end up getting both of those for him. So in the comments, let me know if you saw anything that you would have picked up that I passed by that helps me learn and it helps everyone learn. And I think I already said that now that I'm thinking about it. I just did a voiceover on a Goodwill Benz video. So that might be up. Not sure which one I'm gonna post first. So definitely go check out my channel for those Benz videos. Those are some of my favorite to make. I love digging through the Benz. I don't always find like the best treasures, but I do find some good stuff. It's definitely worth it. This is the carts that they bring out that are just kind of sitting by the cash register. A lot of times I will find some decent stuff here, but sometimes not. That had like stuff on the bottom. So I'm not sure which brand that was.  A lot of times they put shoes down on the bottom. This store I should probably look at the shoes. You guys, I just skip the shoes. Not a big shoe person looking here at the baskets. I know Rachel Strickland does baskets on her channel, specifically those Thanksgiving baskets.  All right, you guys, I don't normally look at the clothes, but I saw this beauty right here. This is beautiful. Just hanging out. Oh, it's Hawaiian. It's made in Hawaii. I love it. It's got this really cute bottom to it. It did have a stain on it. It wasn't a stain. It was just something. Like, I don't know what. Somebody went by with a stroller and their kid like spit on it. I don't know what it was, but I did wash this. It turned out great. And I'm super excited about this one. I just love the colors. It's red, white, and blue. It's a beauty. So really excited to find that vintage item. Thank you so much for being here and thanks for watching.


Hi, folks. Welcome to the last topic of the content account today. I'm Bo Chen from Alibaba Cloud, and I'm one of the maintainers of the Dadi open source project. Hello, everyone. I'm Haibin from Intel. I'm an on-app commit and a Dadi commit, and now I'm focused on Cloud Media to support Intel speakers. It is our pleasure to show you our written work in the open source summit. This talk is focused on how to accelerate remote container image combined with Intel technology. And Haibin and I will do the presentation in turn. So let's get started. The first thing I'd like to introduce is Dadi. Dadi means Earth in Chinese, and it has been widely deployed in Alibaba Group and Alibaba Cloud. The overlay BD of Dadi is a remote image format based on block layering. The concept of overlay BD is similar to overlay FS, but it managed to do things in a different way. In later slides, we'll explain why we choose block instead of fire system. In the year 2021, we published two papers in Eugenics Annual Technical Conference. The first one introduced the basic theory of Dadi, and the second one demonstrated a large-scale function compute scenario that enabled Dadi. Next, please. In terms of open source, we used to incubate Dadi projects by ourselves, but since last year, container D has adopted our two core projects, accelerated container image and overlay BD. The first one is basically a snapshot of plug-in with an image converter and build kit. The second one specifies a remote image format and includes implementation of storage backend. You're welcome to check out these two projects, especially if you're a cloud provider and willing to accelerate image distribution for your customers. Next, please. A wide block. I think the first and most significant reason is that block is simple, and simplicity almost always brings high reliability. Implementing a remote, bug-free and POSIX-compliant fire system is challenging, while the industry has already had decades of experience on block. The second reason is that block requires less kernel dependencies. Some fancy FS implementations require high-version kernel, and for overlay FS, on the old kernels, it sometimes encounters the cross-layer reference problem, which is to say modifying a large fire is slow, because the kernel needs to copy the entire fire to the upper layer. Another advantage is from the perspective of prefetch, as we know the prefetch mechanism is to warm local cache, including page cache and fire cache, to accelerate the container startup. Google has a star-gz format, and we know that its prefetch is based on fires, but for this reason, we know that its prefetch is based on fires, but for the block, because block IOs are much easier to capture and understand, and relatively easier to replay, and block has better granularity compared to the FS. Besides, block also has many interesting features. For example, native writable support. You don't need to resolve any disk space or host anymore for overlay FS upper layer, and multiple fire systems for means that you can choose your own FS type now. I think some of the programmers may have encountered a problem where your code is working on EX-4, but caught up on XFS or other fire systems. By specifying your desired FS type in your own image, you are now able to eliminate this situation. What's more, Windows containers might become feasible, considering there is still no good replacement of overlay FS for Microsoft. At last, block could be integrated into EBS, where we can sync the IO into the underlay world to get better performance. After all these talks, I just want to say I'm not here to judge and deny the importance of overlay FS. It's absolutely a perfect solution to host local image service, but when things come to the remote image, I have to say you are suggested to consider a better alternative such as overlay BD. Next please. As you can see here, the dark boxes are the four major components of DAVD. The first one, overlay BD is actually a log-structured algorithm for merging image layers into sequential blocks. And TCMU is responsible for exporting a virtual block device. The third part is caching library. It can cache remote image blocks in registry or P2P system, with small granularity. The last one is a high-performance streaming decompression module called Z-File, and we also need to calculate checksum in this module to verify the integrity of the file. Next please. Our cooperation with Intel is just on this Z-File. Why we want to accelerate Z-File? Because for a traditional tar image, which is basically OCI-V1 tar file, the snapshot is supposed to download all the blocks and verify and decompress them together, and the container runtime will do random access on those local files. However, for a remote image like overlay BD, the runtime is actually doing p-reads on a virtual block device. After the transformation of overlay BD, a bunch of HTTP render requests will be simultaneously sent to multiple blocks in registry. And finally, merge all reads into a sequential block. Of course, the last step, which is here, merging to the sequential block, it doesn't belong to Z-File, but belongs to overlay BD. In a word, it means that all the IOs are taken in flight, and it's not hard to imagine that DA-D is very sensitive to IO latency. So we definitely want a faster Z-File. On the other hand, CPU workloads have changed. During the container startup time of a remote image, the workloads now consist of both checksum and decompression and user applications. So CPU uploading becomes meaningful. Next, please. Let's take a look at how we use to implement Z-File. By design, Z-File has to be signable so that the decompression or CRC calculation can be done in any position of the file. We choose the chunk size to be equal to the sector size of the block. We use LZ4 instead of GZIP as the compression algorithm because it's relatively faster than GZIP. And the CRC function is got from GCC built-in. We did this by enabling SSE 4.2 instructions. Then we see the opportunities to work with Intel Fellows on the new incoming SPR platforms, where we've been told that we can use hardware to accelerate CRC and decompression. And thus the CPU workload will be reduced. So next, I have been to give us more details about how to accelerate Z-File by using Intel technology. Happy, please. Okay. Thanks, Timber. And now I will introduce about GSE, how to optimize the CRC. GSE, first of all, we need to know is Intel Data Streaming Accelerate, which is an accelerated built-in SPR that performs the common data moving operation. The GSE and the CPU are connected by accelerators and encapsulates inside the socket.  When we use the GSE, we can copy like a DMA to copy like a DMA from network to memory. But GSE have an additional function is we can do CRC calculation. We can do some checks for checks done. Is CRC checking the right or not? If not right, it can be dropped directly and reduce the memory occupation. If the CRC checks them is right, we are copying data from network card to memory. And when we transport data from network to memory, we have other functions like give, do or cast. Okay. GSE have four operations, move, fill and compare and flash. Any move operation have four types. The first is more memory move. Transfer data from source address to destination address. Source and destination region can be either in main memory or MML. The second is CRC generation. Generate CRC checks them only transfer data. The third is DIF, DIF means the data integrated field check. If insert, strip or update, we are transferred it. The fourth is DRCAST, which is a copy data simulator to the destination location. The fill is fill memory rich with a fixed pattern. When compared operation have four types in memory, compare with the fill memory rich with a fixed pattern, and have data decoder create an emerging function. The pattern zero detected in the special case of compare where instead of a second input bar, an eight-batch pattern is specified, that may be zero or anything. The last is cache, flash. Cache flash is very useful to cache, to even evict outlines in a given address range from all levels of the city of cache. Now let us to see the DSA Internet how it works inside the DSA. It consists of several work queues and engines, where the composition can be configured with a global configuration engine. Work queues are on-device storage to contain descriptors that have been submitted to the device. An engine is an operational unit. Any engine in a group may be used to process a described postage to any work queue in a group. Below the next slide have a example of a combination. Each descriptor, each part of a work queue is available to be dispatched by the group at a beaten to an available engine in a group. They are beaten for each group. Each group is dispatched to describe the group for the primary work queue in a group according to their priority. Now this slide we have given one example about how to accomplish the work queue on an engine. For example, this group 0 is comprised of work 0 and work 1 to engine 0 and engine 1. So engine 0 can process the work 0 and work 1. Engine 1 also can process the work 0 and work 1. Now let us see the software architecture. The software has two parts. One is config DSA. One is usually DSA. First, if we want to configure groups, work queue and engine, we need to use the access the configure application. And after config DSA, we need to, we can use the label access the configure or DMRO to submit tasks to DSA. But if we use the label access the configure, just one, just one task to DSA. If we use the DMRO, we can have two, have many, so the two that meet task to DSA. Now, after we use the DSA to accelerate the CSA calculation, we have do some performance tests. This is the performance data. First, as we know, the CRC calculations are 10% of inter-duty calculation. And we put in CRC computing performance from LISCQPS to LISCQPS about improved five times. And this is one SPL code. And this is one DSA. And after uploading CRC calculation, the CPU will not only have to calculate CRC. The problem to project status is we have completely enabled DSA by use the label access the configure library and to push it to community in GitHub. And also, we have enabled DSA in our BD for CRC calculation. The code has been merged in our BD, and can be our BD. We use the guide. We can to find it in label access configure and the DMRO and our BD data mean. Well, and future work in Intel, we are doing optimization for data compression. Performance using QAT, which also accelerates hardware for SPR means the server lepids, which is overloading decryption from CPU to QAT. Again, we improve the QAT part and they fail to, they fail programming. They fail to access the layer on the line, access the Java framework to access the QAT device. Okay. Next. Thanks, Haibing. And having just said, the depression acceleration is still not in Dadi and we're waiting for his excellent work. And in terms of Dadi's future work, Dadi first, the first thing is to try to be adopted by OCI as one of the official remote image formats. And the second thing is to try to be merged into kernel as a kernel module as a equivalent of overly FS. Okay. Thank you very much for participating in today's meeting. Any low-right kernel is my in-contact information under this simple email. We are coming to actively contact us. We have 10 minutes to answer questions. We are coming to actively ask questions. Thank you. Thank you very much to attend this meeting. Thank you.


Hi, I'm Mark Velker with VMware and the Deafcore Committee and I'm here today with Chris Hodge who's the OpenStack Foundation's Interoperability Engineer and we're going to talk a little bit today about Deafcore. So Chris, why don't you start off by giving us a little introduction to Deafcore and why the Foundation cares about it. So Deafcore is the board-appointed committee that is defining the interoperability standard for OpenStack and what the interoperability standard is supposed to do is give vendors a guideline of that is backed by tests of you know, a guideline that's backed by tests of the products that they can sell and put the OpenStack logo onto with the intent of giving users a guarantee that they'll be able to port applications written on one OpenStack cloud to another OpenStack cloud. Great, and we should probably specify that what Deafcore is primarily focused on today are capabilities that are exposed to end users. So not necessarily to administrators. The things that end users can actually write code against. Fair enough? Yeah, yeah, I think that'd be fair enough. So Deafcore obviously became kind of the enforcing part of the standard earlier this year and we've got a couple guidelines that are out there now. Can you give us maybe a quick overview of some of the things that people can expect to see in those guidelines now? Yeah, so so right now when you look at the guidelines we are targeting two different sets of what we would call components and these are essentially projects that can be licensed. One is for compute and one is for storage. We're particularly object storage and you know, so with the 2015-05 and 2015-07 guidelines we have a basic set of capabilities that define how your OpenStack compute or how your OpenStack's object storage or the combination which we call OpenStack platform should run. Currently we're working on a new guideline which is expanding the capabilities across those to not only include Nova and Swift as the projects we're testing against but also the rest of the kind of core OpenStack projects including Keystone, Glance and Neutron. So Neutron, while we're talking about Neutron, we should probably talk about some of the new capabilities that are being proposed there. So I think that's one that's been of interest to the community for a long time. So yesterday at the board meeting here in Tokyo the next guidelines draft went up before the board so we could actually start soliciting feedback on that. We'll go up for a final approval until January. So there's a few months here where we have some built-in feedback time. It does include new Neutron capabilities, which I know you and I have worked on quite a bit. So among other things we'll see floating IPs, security groups and L2 and L3 CRUD operations. So you know kind of four buckets of basic capabilities for OpenStack networking for the first time. Hopefully we'll be able to count on. So that brings up the question of how people can supply feedback because not all those things are things they're implemented widely across a lot of clubs today. It's a little bit controversial in some cases. So you want to talk a little bit about how people can bring in feedback? Yeah, so there are a few ways that you can get involved with the Devcore process. The first is just coming to the meetings. We meet every week on Wednesdays at 1500 UTC and it's a great place to you know, you can take a look at the proposed guideline and offer feedback and contribute to the process. The guidelines are stored on the OpenStack GitHub repository. And we also follow the kind of the standard OpenStack development process. So you can offer feedback directly on a guideline through patches and code reviews. Great. So three months until that guideline is up for the board. What do you think is going to be the thing we discuss the most? So definitely networking is going to be at the top of everybody's lists, you know as we try to balance existing network models with network models that we would like to see happen in the future. But I also think that we're also going to be looking at some block and image storage capabilities too and trying to decide, you know exactly what features that we want in there to guarantee interoperability. Okay. So, you know, I think networking is probably on the list as well. Obviously, we spent a good deal of time with that. And the board actually had told us from previous meetings that they wanted that to be one of the big priorities for this next set of guidelines. Because it is something that OpenStack has sort of mutually exclusive options for. So it's kind of a tricky area. But something you said earlier is something I want to go back to. We talked a little bit about how OpenStack is changing as a result of DeafCore. So I think maybe we should talk a little bit about how that's influencing some of the choices that projects are making. Obviously, we've seen a lot of bugs going to OpenStack and into tests as well. And seeing those get fixed has been pretty satisfying as an outcome. But I know we just came from a session earlier today where we had a cross-project meeting talking about things like how to deal with images in OpenStack clouds and how to deal with API transitions in OpenStack clouds. And that's something that we're starting to roll out more of that feedback to the TC, to the PTLs directly, and to the user committee as well. Which is something pretty exciting to see. Any other things that along those lines that you'd like to talk about? Yeah, I mean I think that what you bring up speaks to the intentions of the DeafCore committee and the interoperability process. It's not just a branding and marketing program, but it's a way to give vendors guidance on what they need to provide for their users. It gives users assurance that applications they write can be ported from cloud to cloud. But it also actively solicits feedback from the developers and the vendors and the users to help shape the standard and truly be a community-defined standard. Not anything that is defined by one group. And so in that way, it really captures the essence of open collaborative development. Yeah, that's a good point. So DeafCore as it exists today is really powered by a fairly small set of people that are doing the legwork of things. But we've actually started to see a pretty big expansion in the number of people that get consulted or contribute to reviews and patches going in and those sorts of things. So that's maybe a good note to end on, is talking a little bit about how people can get involved in the active decisions that we're making throughout the course of the next three months, and also for the next guideline as well. Yeah, three great ways to get involved. Come to the meetings, share your voice. Contribute code reviews to the current specs and patch sets we have up for the DeafCore repository. But also if you're running a cloud, downloading the ref stack client and running the tests and submitting your test results is also going to give us great feedback on what capabilities are out there and what we should really be focusing on for the next guidelines. Yeah, I think we've kind of talked earlier about how of the 12 criteria that DeafCore has, about eight of them are trailing indicators of market acceptance for capability. So getting those results in really helps us get some visibility into what folks are actually deploying and what actually works in the clouds that people are operating today. As well as the user survey, we should say that last user survey got a big bump in the number of people that have responded. And that's been really good for us to be able to have a little more feedback from the community. Well, thanks for talking with us today. Have a great summit and we'll see you soon.