Hello everyone. Good morning, good afternoon, etc. Good morning. So, please. Hi, Elena. How are you? You're ready? Yes, thank you so much. I'm really good. Yeah, ready. So, I'm going to wait two minutes because people are normally late, late in the sense of a couple of minutes. The other thing is that we are going to record this session like every other session. The recording is already started. Two things before we start. One, the Hyperledger Code of Conduct, we are meant to be going by the Hyperledger Code of Conduct, which says that we will treat each other with respect, even when we are disagreeing with people or even when we are agreeing with people. Please be respectful of each other. The other one is the antitrust policy. We go by the Hyperledger antitrust policy, which means that we do not engage in any antitrust activities that is the, these are the only two requirements for participating in this call. And of course, we also know that all the material that is developed and put into play here are completely open source under the Apache 2.0 for code and the other open source, open source licensing for everything else. So if you can, please add your name to the end of the, to the list. Let's help each other develop the meeting minutes. And we are going to start off with Elena, who's going to describe the challenges of testing the LT software for financial market infrastructure. And exact pro has been in this business for a while. She is a researcher and business development manager in exact pro. She'll take it from here now. Thanks. Yeah, thank you. Thank you. Yes, so thank you so much for having me today. And this is really exciting for me to be a part of this group. Just a couple of words about who I am. As we've been said, I represent exact pro and we are a firm who is focused on software testing for financial sector as a business development manager and responsible for partnerships and events here in the U.S. And I'm also a part of the exact pro research team, which means that I talk in our research activities, the research paper submissions or maybe speaking at the conferences, or maybe making the overviews for some internal use. And this is exactly the reason that oh, why I am also a part of this group because the discussions that you give here are really high quality and they are really important for us because they give us help us to understand what's going on in the capital markets sector related to the DOT domain. So let me share my screen. Just a couple of seconds. And I will start with a presentation. Okay, can you hear, can you see it okay? Yes, yes. Yes. Okay. And here we go. All right. So just maybe a couple of words about the company itself. So exact pro provides quality assurance services for exchanges, clearing houses, investment banks, brokers, and some other financial institutions performing both functional and non-functional testing. And we've been a part of the London Stock Exchange Group between 2015 and 2018. In January 2018, exact pro completed a management buyout from LSAG. We are headquartered in London. We also have operations in the U.S. in Georgia, Georgia, the country. And we have several offices in Eastern Europe. Exact pro currently employs almost 600 specialists. And as I mentioned, our focus is mainly in financial market infrastructures. And our current clients are quite diverse in terms of their geography. We have clients over 20 locations on all six continents. And from the beginning, the main focus was on traditional technology. So our first engagements here were in the trading and post trade areas. And then maybe a couple of years ago, we got interested in the distributed ladder technology. And we tried ourselves in a couple of POC projects. But at that time, most of DOT was that early day, early day attempts to build an application and to prove that it actually works. The whole idea of testing is exactly the opposite to stress this system to its limits, to expose it to some stress and to expose its weakest points and to see what happens in the worst case scenario. So back then the industry was not exactly ready for really intensive testing. But surprisingly, quite quickly, it came up with the solutions that needed real testing. First, it happened in the area of DOT per se. And then we were also contacted by some digital exchanges for whom testing turned out to be extremely important given the need to be established as part of the capital market community. And most of them are still struggling with the regulatory challenges. And just for you to understand the context of our work, I'd like to describe our general approach to testing of traditional technology, what is usually involved here from software testing. As everyone here knows, I think they underline technology and the financial sector is extremely complex. First, the complexity is caused by the structure of a typical platform within a systemically important infrastructure. The most vivid case study for testing will be probably a post-trade system which can have about, like, 80 inward and outward connections, dozens of components, sophisticated system dependencies, like upstream and downstream. Such systems also have complex participant structures. They work with lots of asset classes. And they are accessed via diverse APIs and protocols. And the second aspect of this complexity is the life cycle of such a system because the system's daily activities involves, again, for post-trade, for example, it's trade uploads, different calculations, computation, margin runs, settlement sessions and so on and so forth. And all of those activities are performed every day. They happen in a predefined order and they happen at predefined times. And they depend on each other. And actually what does it mean for us in terms of software testing? That means that to test such a system properly, we will need to test all these repeatedly happening processes and to test them with all the multitude of parameters. So in other words, this leads to hundreds of millions of tests, which is apparently a problem that can be approached only with the automation. So as you can see, it's already enough complexity to deal with from the software testing perspective, even before DLT is brought into the picture. So for that, even for pre-DLT, I mean technology, we used to address this complexity with our test automation tools. And here on these slides, there are just two examples of those. So one of the tools is aimed for testing the complex scenarios and business life cycles. And it supports concurrent work with multiple gateways with diverse protocols and APIs. And on top of this tool, there was an enhancement allowing testing at the conflicts of functional and non-functional testing. So that was the pre-DLT era. And then we noticed that financial institutions started gradually adopting DLT, which of course required significant transformation and replacement of the core parts of the platforms. So to address those changes, we started to think about what the test approach for new transport platforms should be like. And we decided to accommodate the test approach. And we needed to understand the new challenges, the challenges introduced by the DLT. And at the very beginning, it was absolutely clear that all the challenges around the complexity are in place. So the traditional challenges didn't go away. And DLT did not eliminate them, of course. In fact, they became even more pronounced now because of the hybrid nature of the new platforms. So compared to traditional systems, DLT platforms pose additional challenges in terms of quality assurance. And these stem from the necessity to verify the interoperability of nodes in the network because there are a large number of connection permutations between them. And you need to understand the differences between varying permissioning logic. There are also challenges stemming from the non-functional requirements to DLT platforms. They have to be really high performing, having considerable transaction throughput. Another requirement here is very high availability, especially in those components which are aimed to ensure the correctness around the logic of the ledger states and their correct smart contracts and the correct propagation of the changes across the network. And maybe another weak point here is, of course, continuous upgrades where there is a risk that something will go wrong. And again, the error could be propagated to the entire network. Apparently, to address those challenges, we need a two-pronged approach with both functional and non-functional testing, be it testing of the network plumbing itself or the distributed apps. So we found here that there are a number of test tools properties that we needed for testing DLT. And as for the platform itself, our case study that you can see on this slide covers the experience with the R3 coordinate technology from the functional point of view. The existing testing tools needed to be improved with the following capabilities. So as you can see on the diagram, it represents main DLT system components. It is a network of nodes. And our tool was used for extensive anti-un-test automation of this scenario. And it was adapted so that we could simulate main node functions on the corded ledger in order to send a transaction with certain parameters. We predefined them in the automated script. And such a script we internally call a matrix, but actually it is a file in a CSV format. During testing, a whole set of matrices is run concurrently through multiple connections. And for different connections, we can trigger here the interaction between the nodes and we can check the results in volts. And some of the other enhancements are also listed on the slide. So this is around nodes deployment, ability of port apps deployment, nodes administration, and so on and so forth. So this was a case study for testing the network itself and from the functional point of view. And the next slide demonstrate the approach to the very same corded network, but from the non-functional point of view. So as you again, you may know, a non-functional testing of any platform is extremely complex and it requires robust configuration and may even involve some preliminary development activities around it. So to run automated non-functional tests, it's required to monitor various metrics of the platform under test and its hardware usage. It is also very important to have the option to start, to stop, to adjust load injection or some other actions based on the current system stage. So the exact automation framework allows the initiation of flows across a large scale network of nodes and we can do it against a remote network. We use open source tools here to gather metrics, to visualize the results, and we are now able to cover various non-functional testing types. As you can see here in the slide, it's load and stress testing, failover testing, soak testing and the measurement of latency and performance. The last two case studies are more examples of functional testing of distributed apps as in contrast to testing the network plumbing itself. So in this case, our test tool was adapted to simulation of nodes for the trade reconciliation and position update business flow. As you can see, it's for two parties and a CCP. So for this scenario, of course, this is a much simpler case. And in this example, testing was orchestrated using just exact test tools. Some of the framework components amulated the nodes, some of them triggered some actions through RPC calls and some performed checks on the ledger. And another case study is again around the business layer and it is the one with the digital asset modeling language demo. And here we used digital assets swaps lifecycle module application which was based on the Easter CDM model. In this case, the simulation that we did was performed in one of the parties node as well as in the market data provider node plus testing framework and related time shifts to facilitate the checks which were associated with a particular lifecycle events in a swap. So this is basically it outlined the pains that we had before with testing traditional platforms within large scale market infrastructures and shared some examples of how one can adjust the testing to beat the new needs which are emerging with DLT. So thank you. Thank you so much. Elena, congratulations. It's a very comprehensive look at the complex landscape. Maybe we went through it a little fast but I'm sure people have questions especially since I noticed Jim on the call he's already asked questions on the zoom. It says, did you use jmeter for loads? Did you test CI CD and change management? How do you define bad test data cases? Anyway, you can read just like I can obviously but I'm just drawing your attention to the questions on the... Right now I'm looking at this at my presentation so maybe I'm stopping. No, it's okay. You can... I can read it out to you. We can answer that. I'm looking at it right now. So regarding the first question about the jmeter we use an internal tool called that to generate the load we use our own load injector which in some way is similar to jmeter but it is more adjusted to our needs to our specific knowledge domain area so it's more focused on the trading software so for non-functional testing we mainly use this one and regarding the CI CD I... Yes, we used open source tools here and of course we use it extensively in our experience. So... How do you define bad data test cases? Well, our overall approach is an extensive MTM testing so that means that we look at everything so that's why I started my presentation with this maybe not really interesting for you but for the overview of the complexity of our typical platform so as you can see in our testing so many things are involved so we look at the data, we look at data permutations we look at how data is transmitted from one point to another because through different protocols and APIs data can be changed for example we receive some data from market data feed we receive it in one format and if it is received by some custom application or component in the platform and then it's transmitted to some downstream system the data structure can be changed a little bit it will be tested for everything for all the permutation I hope I answered the questions because I think that here I... you can have several understanding of that but data test cases so if it doesn't answer your question you can always contact me and I will try to offer you another reply here so pre and post assertion in your test framework I'm not sure how to answer this so certain languages so some languages provide capability for pre and post assertions much like a J unit kind of a thing right so you can say that the entering state is X the exiting state is Y and you can assert that those things are and then you look for what I call the transformation in the middle if that makes sense yes it does and I would say that we do not focus a lot on the formal verification here so it's more of the overall business functioning and we specialize more on the backtesting for APIs and protocols so if we approach testing from a more black box perspective thank you the next one is about whether you use some kind of identity open ID connect authentication I'm afraid I'm not familiar with this basically what kind of identity model do you use in testing or you do not you just go with one in other words like in some frameworks like caliper for example you have a way to spin up hundreds of clients with multiple identities interacting in complex ways because that will simulate the real life here of course Jim is referring to a specific identity framework called open ID connect but you know you can use any various different ones depending on the identity model inside the framework I understand it's just I'm not so deeply involved in testing so that I cannot answer this one but I can get back to you later I think we have a dedicated page on hyperledger portal questions can be asked and answered there this is the one I just have to go to my team I mean testing is a vast landscape so obviously he's got some questions hi Elena thanks for a great presentation can you elaborate more on the digital asset and the CDM testing with respect to IRS and CDS considering the fact that CDM is continuing to make a lot of changes in this testing how and when did you do the testing you could elaborate a bit more on the CDM testing itself first of all I wouldn't call that the testing of CDM so this wasn't a testing of the underlying model because as you know CDM is a framework for the description of business scenarios which was used by the dental SDK and we focused more on the application level so we simulated the business flow we tested the business logic how everything is working on the level of this very application it was a proof of concept project that wasn't a production case so yes and maybe some more interesting detail on CDM we were not only testing this we actually participated in hackathon and we developed some even developed some application but on core the framework using CDM so if you heard about this Jerry hack we participated as well we were participants in Singapore and we took price for best CDM adoption something like that but from the development point of view it wasn't from active thank you anybody else have questions for Elena no but I did want to say that was an excellent presentation obviously you put more than a couple months of effort into building out this extensive test environment which is really robust and I can agree having been on the other side of the fence at fidelity having responsibility for those systems that a much higher level of testing is needed so the fact that in a sense your company provides that that's a big big value service and there's very few companies honestly that have a high focus on what I call bad data which is more important than big data I can almost separate them and I say all of the broker dealers that you're familiar with don't have a super high threshold or a quality pain point whereas when I flipped it around to the JPMorgan data services those guys were way up there on the food chain in terms of the priority for accurate data so there are some differences there and there's still the gap with the client where they don't have that same standard I'm sure okay thank you thank you so much thanks anybody else yeah I have a question related to the most common errors right or mistakes as you work with many clients if you were to maybe highlight some of the the main issues that you encounter on the test design or if you could share with us some of the most common occurrences yeah but I think I'm not about speaking about the details here but maybe the main pain point that we experience is connected with the underestimation of the complexity of the platform so everybody is talking about digital transformation about AI and DLT and everything and they think that those new technologies pose huge risk but in fact the most risk the most of the risk stems from the non determinism of the current platforms because they are really complex and no one really understands how they work I mean in depth so there is no in-depth understanding of the exact workflow what if we change this data and what it will what will happen especially this is this makes sense in the face of the crisis that we can now see because before we had lots of problems with stakeholders of the system that we tested because sometimes we came there and we tried to perform some tests but they came back and said okay guys this scenario is really unrealistic so why are you doing it at all and especially this can be seen in some functional testing performance testing because we tested insane levels of loads and the clients think that we just want to break the system actually this is true we try to break the system because it is important to see the systems reaction we have to be sure that it has failure capabilities but the spike in the volatility showed that some scenarios which we were told were unrealistic they proved to be realistic in current situations so this is definitely was a mistake of some of our clients not to understand this well what else yes most of the problems that we uncover they are not in some particular place of an application or in some particular component so I cannot say that some components are innately prone to these errors but it is rather at the confluence it can be either at the confluence of like non-DLT connection it can be at the confluence of functional and non-functional testing because if we create just some load we cannot see an error we can like miss it or vice versa during functional testing we can again miss it but if we do like a smart simulation or smart model based testing which involves generation of lots of scenarios and loading the system under test with these scenarios using this approach we can find some errors and again one more thing I can think about is the errors which occur when we create systems which we when we upgrade some components of the nodes so something that is related to this okay thank you so much for sharing it's really insightful Elena can I ask you one more question a major challenge for you I assume you guys have a it's not a project approach it's an environment for a given company right even their testing service correct normally I would assume what I mean what I'm trying to say is a company doesn't hire your company to come in and say just test temporarily they're hiring your company I assume to do a continuous testing over a long term we have different engagements you're multi-year service agreements but your multi-year service agreements where you're considered the primary testing facility for a company how are you integrated into their change management system because every time they make a change it would impact your testing so how does that integration process work for you the answer is simple our testing framework is usually deployed into the client's framework so it's in their network it's on their premises where it's in their cloud so the testing tools are part of this framework they have internal access and so on and so forth and I think it's a really small percentage of our testing that is performed on our own frameworks mainly infrastructures anyone else has any questions if not I have some okay you mentioned you know you brought up the question of black swans basically nobody is aware of the extent of the stress scenario until it actually happens which is the true definition of a black swan because it is not within people's minds to think about these things I myself when I was running a mortgage back security system between 2008 and 2016 we created tests like pandemic we did create tests like nuclear attack in time square we did create scenarios like California earthquake all of this of course resulted in a model based approach saying the spreads were widened by this much amount but the problem is you know the spread even those metrics are difficult to project because it's all based on somebody saying okay if there is a global pandemic this is how much the spread will increase by but it may be not so to bring all this back into the picture I had written something about chaos testing inside blockchain because it is now available on medium but we had started thinking about this because the whole practice of chaos engineering is based on the premise that nobody can have the entire system in their head I mean it's impossible for this complex system which is the point that you just made just a few minutes ago I think this is definitely what do you call it so chaos engineering theory basically tries to inject faults in a random and chaotic fashion yeah we are great what's that yes we really are into the theory and we know about this and we use some of its principles in our testing and speaking on the black swans we prefer not to think about what happened as a black swan because apparently it is more of a white swan because it was predicted long ago and what we now observe in financial market infrastructures we are responding to the crisis it was actually tested because again we always we do not hope for the best or something so some firms have certain thresholds for example there are rules within the company to test for example double of historical maximum so this is not our case this is not how we work we do not have such thresholds we do not have any presuppositions what is the bad scenario we just test for the very very good results we try to kill the system components so we just go for a kill and then we just see what happens how the system reacts what's happening what is its behavior and based on that the system and our ideology here is that we have to focus on the white swan events that we have and it's enough of those and we have to be always prepared for the worst because you don't think that these are black swans exactly for our testing we always anticipate the highest volatility the highest loads the highest outages so this is something that we test that means you live in Australia because black swans were found in Australia and when the English landed there they saw that these black swans obviously they were not black swans they were not anything special for the people who lived there and were used to seeing them every day in that sense yeah anyway so testing is a very comprehensive practice and it's a very specialist practice and it has to be continuous like Jim was saying obviously whenever there's change either in the software or in the infrastructure the deployment itself you change the number of nodes if you're going to multi-cloud or if you're doing any other kind of things those are all changes that affect the software one of the other questions I have is your practice your slides are they available or are they not yes I can share them with you or I think I can publish them myself yeah you can go to the to the agenda website or meeting website which I have set up it is a meeting minutes for this call and you can just download your slides over there and link them if you have difficulty doing this just send them to me I mean either way it's fine maybe you can go okay I'm going to put the link in again because I had it earlier because I want people to help me crowdsource not just the meeting minutes but you know reactions their own presence in the you know so I have the link in the chat if you want to go there but we can have exchange later and then we can do this there okay and we also have this case study that I partially ran through on our website so it is open for download so you can go ahead and do this but of course I will share some slides as well that's great because you know I think it's important that when we come to deployment we should be thinking about these you know testing even before we get it out into the open now any of these pieces of software anything there that is available on a demo basis or on an open source basis from your site or is it all paid stuff we do not charge for our tools we have actually some of our tools in open source on our github web page we also have several videos on the topic so one of them is here on this very page which I referred to when I spoke about the case study and maybe there is and there is also a video covering this case study video I recorded from home during the pandemic already if you want to see me again go ahead to our YouTube channel and explore some videos there it's more the material that is very compelling and you are a very sympathetic presenter so that is a very good thing anybody else has any more questions on this topic no but it was an awesome presentation Elena thank you so much if I may one additional question Elena I don't know if you heard about the token taxonomy and the joint initiative that many companies are gathering together to define standards that are almost agnostic to the platform and when you think about testing I know that you may go from a higher level of abstraction right until maybe to the very fine grain a detailed unit test so in that sense talking about token standards have you come up with a more like a business level if you will and how to design comprehensive tests for different token standards whether they are smart contracts right specifically to blockchain for instance yeah I'm sorry I missed the name of the framework was it TTF? yeah the token taxonomy framework it's a joint initiative that they are trying to come up with a token templates that would define behaviors and also properties so whenever somebody wants to implement those templates they could be reused and extended to their specific business needs I'm afraid that the only place that I've heard about this framework is in this group this very group so thank you so much for sharing this knowledge and I'm afraid we haven't looked at it yet in terms of testing yeah sure thank you well you know that we have implemented that ETALAR which is based on this framework and we are adding more we may add more security definitions on that framework but recently they have migrated to another group called IWA we talked about it in this this group before so anyway I think it is going to have the similar effect let's say as CDM on the contract side so it is always good to incorporate some standard methods I see that you have in your slides the reference to SWIFT ISO 20 or 22 messages probably so these are all standards but the problem with ISO 20 or 22 for example is it seems more like a union of existing messaging standards and they are not cryptographically sealed or cryptographically integrated properly with the lineage and everything else like CDM is so we have that now we are coming to the new age in terms of those messages so I don't know how far this I mean I know that ISO 20 or 22 is a recent standard and heavily used in ASX where DML was very active but I wonder about the future of these kind of frameworks which are not which do not have this internal integrity in fact some of the most famous hacks against SWIFT came because of this lack of lineage or cross reference because the Bangladesh hack system hacks that have been perpetrated against SWIFT have come because you can enter the message stream at different points the weakest link for example and create messages that are not integral to the chain to the flow and hence create problems so I don't know whether SWIFT themselves have thought about this that's another topic for another time but still I think the testing could incorporate these kind of things into the framework what were you saying yes absolutely though if it is more about the functional side so we do not do the security testing per se but from the business perspective I think this is a very good idea to incorporate this into the testing the ISO 22 or 22 we can already see it incorporated in our business scenarios within our clients something that definitely has future any other questions for Elena before we go to the next two items okay thank you Elena and we will be looking forward to getting a copy of your slides and I will put the recording up which shows you in your glory talking about all this stuff it's very very good because this is one of the topics that is often missed thank you for your attention thanks for having me two things one is I would like people to volunteer to do other presentations we want a diverse group diverse presentations on different topics all dealing with capital markets so the first thing is I will send out the biannual report that I'm supposed to be doing I will send out a draft copy to people and you can respond to me right away because it is due by the 15th of July the second topic is that we have introduced a new lab proposal for cross-chain settlement instructions on an interoperability situation as a solution to interoperability but it goes away from the regular interoperability conversation because it is in the creation of cross-chain messages which are cryptographically bound and integral with lots of different things that attest to the source of the message and also the ways to also ways to respond to this in a very non reputable way I have already sent out the proposal but more details will be prepared once the lab is approved anybody who wants to participate in that lab can contact me but we do need people to commit some time to it otherwise you know we will get nowhere either in the future I don't want to get into any other stuff there can I ask the use cases you are describing do they fall in the same domain that the cactus project is looking at or are they different? when you say the same domain there is a settlement leg of an asset transfer we are starting small in other words we are coming from the other end of the spectrum we are not proposing a general solution for all cross-chain interoperability which seems to be the same by proposing a series of plugins and a plugin architecture that can then take any solution and you can plug it in it is possible that we can create this and then plug that into cactus but we are not doing it we are almost at time it has been a delightful hour as always next week we will have a close reading of the digital dollar project which Karen had proposed we are going to talk about this in case you are interested there will also be deeper discussion on CBDC maybe we can bring in the R3 paper into the picture and discuss what it means in terms of implementation where do we have to concentrate we can give our feedback because they are looking for alternate views so please you can get back to me or to the group especially after the next week's presentation we can talk about sending a formal reply to the DDP what do you guys think about that thank you thank you I am going to end the recording and the presentation until July 15th then thanks Pipin thanks Elena thank you


Go out mate. Awesome. I just wanted to talk about the squad at the moment and how much deadwood there is in my opinion Yeah, and just see how long you guys think it might take to get out that deadwood and bring in something We can actually use what I tell you what right name is your deadwood and that'll make it a little bit easier for us I think he's got him go up. I think he's got it No, no problems if I start from the back I think in defense you got players like Mason whole gay Michael Keen they need to go And then moving into midfield the likes of Andre Gomez the likes of gobbam in and up front I mean, I we could be here all night talking about Rondon But I don't know what the point was in giving that man any sort of contract So I want to see him out the door as quickly as possible and I could go on but I'll let you start with Five players like that and you know every single one of them is is definitely definitely People I would be trying to get rid of but in the summer. I think Mason Hallgate is Not going anywhere in his career, and I think he had ever Well, that's his career, isn't it? I think he'll know that he's 24. He should be Starting whatever where you get to 24 25 you should be starting whatever you are and if you're not then you have to ask His question why you know and move so I think he's definitely Michael Keen 100% and I think if everything in the summer could get one really really good center back Then I think those two plus yet he mean it could leave the club May maybe a Michael Keen might stay because of just because he's got experience and he might become the number three The number four center back But definitely Mason Hallgate boys Yeah, it's sad because I actually thought he was gonna develop into something really good whole guy, but he's been He hasn't really kicked on whatsoever since he come back from that injury the summer when anti-lots he's like faced Well to start a last season He'd finished the season strongly the one before got an injury Q games before the end of name But he come back pre-season and everyone including anti-lots he was saying he's like the face choice center back Remarkably and then he done Stupid challenge in that Preston friendly didn't he and put himself out for three months and since he's come back He's played a right back. He's had to go with sense. But he's just not got any any form whatsoever. It's the point now He's almost a liability When he's playing now because he takes chances when he shouldn't and he's rash So yeah for me it's for him his own career and for us a partner the way I don't see Michael Keane leaving I could probably yeah, I could probably live with Michael Keane being a squad player But but the problem Michael Keane is and this is not his fault is that he's the only player who ever stays fit He's never injured. Yeah, he's like he's in the team center back.  So that's why pretty much starts every game So I'd keep him around to be a to at the waist because I think we're gonna have to sell me this birthday tomorrow Michael Keane 29 tomorrow some yet. He mean it. He's got one year left in the summer. Yeah, he wouldn't no point given a new contact So we're gonna have to sell him get rid of him get rid of Mason Holgate That leaves but I'm through it Keane and Godfrey get get your number one center back in then to play up to play and I honestly I I honestly believe that if we were to get a real top-class center back in I Mean when I say top-class, I'm not talking Champions League level. I'm talking, you know someone who's just on the on the on the ladder below and on the rumba low on the rumba yeah, I Think he would make all those players better and then you say to my then Michael Keane becomes a squad player Moving into midfield Gomez. I Agree, I think he's he's a lovely player in possession and he'd suit It suits a lovely Iberian team You know Portugal or Spain playing his final days in the Sun just lovely pace full It's it suits Italy as well but somewhere because he's a lovely player in possession but out of possession and Any any time the opposition have got a press on him It's game over so that's I'm with you there. He's another one. He said he's at the right age to just get something out of him financially I do feel sorry for him because I think the injury has just took that that little bit out of him that We were benefiting from but I think the injured you what you watching physically He's he doesn't move like he used to move for us You remember the face remember that old Trafford and then I'm field People like who's this like it big strong help people off And I know I know he wasn't like that in every game. It wasn't just the injury, but when he got the injury it just Physically and conference wires. It just took everything. He's 29 in the summer as well So he's how long you got left in contact two years. So I'd look to shift him definitely who's the other one in midfield Baman right before I I would try and get a bad man out on loan now If you're not gonna use them and let's get 20 games under his belt And let's just see what the lad is like after 20 games Don't get me wrong. It's someone was to come in for a with addition often now I would probably take it if it meant we could replace him with Again someone who was a starter watch you because you you think I think you think he's got a bit more in him No, I'm a bit like you I just it must be a pity to see him go off the toilet not knowing right But if I was given an offer today, I Would probably take it but I thought he don't know right Saturday I still think I think he's got the attributes we want for him in field a big strong can't before his injury was all over the pitch But I Understand why Josh is late as like mentioning about him because we're getting off and I haven't three games or something this season And the league is appeared and started one he's on for three minutes in one ten minutes in another It's not as if the manager's going. I've got a plan for him if the manager's letting people Away that he can go out then he's not in his plans as he he's taking money. He's 26. We've seen nothing of him So I understand with that, but I mean there is others the Delf Well delves going anyway, isn't it delph on a shelf get him out now cancel his contract today And like throw something out of him on the way out. It's like We're the French fan gaiters I reckon he could hit the bramble bushes Possibly possibly get him out the club and the build Nasus on the other side of that Okay, so you get a job He'd be injured he'd like three bricks. He wouldn't see him again for about six months now delft to go Rondon and Rondon. Let's be honest Yeah, he was bought as a sub But I think when we said actually it makes sense He's a sub and not because we were thinking what he was like in Newcastle and West Brom Yeah, we've seen none of that I mean on Saturday was probably one of his better games for everything for a bit And that basically consisted of flicking a couple of others on and holding the ball up every now and again You make two big chances apparently apparently But there you go So yeah Josh up there wouldn't be any questions from what I wouldn't be questioning you that's all over them either But there's others isn't that you know what we've just mentioned Delf What would you do with Tom Davis Josh because he's another one. It's just Rolling on and he's not making his mark is he No, he's not but I think what what Tom Davis has that the other players don't is he's that homegrown talent So in some ways you can you can pull back on that and there is some affinity from the fan base towards him.  No, and I don't like It's it's not like I don't he's not like one of his plays. He's lazy at it. He just can't move Yeah, yeah, yeah, it's not like he's not I do believe I just it's just he literally is me in nightmares kind of run away from people We know what we said this on Saturday, but it was like because I just said to me mate that a couple of hours before I had that dream where ever now she's a play and then you're playing you can't run and I was like It's my dream. Look the book I play and he's like Scooby-Doo Run like you're running Maybe someone will take his mask off and he's Cucamartina No, no, it's good


city and in the territory on west there's just one way to handle the killers and the spoilers and that's with the u.s. Marshall and the smell of gun smoke I did states Marshall the first man they look for and the last they want to meet it's a chance a job that it makes a man watchful and a little lonely blood out of a turnip you sure did try three four weeks now free load none dodge and you know it they came clear from Ohio dodging up their supplies somewhere is that what they call it you've got a real burn I'm tired man tired of all this heat this dust these nights that are hot and days crying babies have never seen before and never see again I'm glad that mob's gone I'm darn glad you live on it too if you're asking me am I fed up the answers yes whiskey Sam and none of that trade whiskey you've been thrown into our recent get see there I'll be sitting over here kind of got the dog hasn't all of us kitty places pretty empty that sure wasn't two hours ago the whole town was in here to say goodbye to that crowd yeah I know come on let's step out back too hot in here maybe gravy was right start not at night at least they'll miss one day a son about what their chances I'm all naked so I'm warm and always bothers me to see him why I don't know it's a fresh and eager when they hit dodge they think they've already been through it they've put it rivers they've seen a few blanket Indians the kids have had a fever but they don't know what's waiting for them out there on the high plane that's what's biting the doctor they think it's going to be a higher the Kansas all the way and you can't tell them any different get it yeah you and doctors nothing you can do for them I'm sorry I'm restless I think I'll go back inside I'll get it yeah Matt take good care of the awkwardly yeah I won't most gimmicks hey camp it was kind of a party out there everything all right in the wagon tree you seem to be good night good night Chester what is it dark what's the matter up there come up here Matt somebody's not dark a baby but he's still insisted on checking that baby before I could get him to lie down imagine Matt it's a girl she's off a little kitty how much you know about taking care of a baby less than you'd think you know it's going to be your responsibility my I mean until we find the mother oh you must be in that wagon train Chester now right after him tonight we catch him by morning you better wait until Doc wakes up and finds that baby in his room you'll never take another drink the gravy said he was going to keep pushing till the sun got overhead it ain't but seven or so now and it's uncommon or maybe two cornered whoever you are because California is a long way from Kansas it's too long at it too late didn't bring in charges against anybody so why don't you think it over before it it's too late huh now you know that stuff's habit form and don't you for your information I need sheets torn bed sheets she's not gonna show up Matt what makes you think so any woman who'd abandon her baby isn't gonna be swayed by your brand of sentimentalism maybe not dog but I'm gonna be here just in case all right so she's out just up what are you doing down here oh now don't worry doctor I'm going right back but I just remember something I already tell Mr. Jones did you just remember to tell me something just sure I did see there was a fella come by here and he said he was a soldier oh yes red seems he just got his discharge what did he want well he was asking about the wagon train see he thinks maybe his wife was with it his wife yes he was supposed to meet her farther out at similar on crossing but he got the dog sooner and what he expected now what did you tell it I told him they was long gone but he could catch up with him easy so he rode off I guess ain't too important but I feel bad I didn't tell you before see what was that new baby and all my nerves Mr. Don't I better get back up there for some all right just a go ahead out of sheets but it hardly speak to Sam maybe he's blaming Sam for what happened well Doc's got quite a homey problem with that baby I know what are you gonna do math let nature take its course I guess you always were an optimist my business I have to be say you can eat could you get me something to eat sure yeah what is the chest she's here that little baby's mama is waiting for you over at the office Marshall Dylan ma'am I'm Mrs. Bale you were with the wagon train weren't you miss Bailey yes I've come for my baby oh where's your husband didn't he come with you well Marshall my my husband's been away in the army for two years I see that's why I I left my baby yes Matt this Dr. Adams Miss Bailey oh oh word with you all right now right now I'm Clint Bailey I wrote all the way out to that wagon for Jenny and hello Marshall this is my husband but what are you doing here you know something wrong Jenny what is it Marshall you tell him Mr. Bailey your wife came for her baby came for her baby what do you Jenny she could have left it but she didn't is it true Jen well is it yes but Jen we never had kids all the years we've been married I know Clint it wasn't for me not wanting them well what is this baby it's a girl that's good that part's good can we keep her clean I've been waiting two years to come back to you Jenny and I can't lose you now are you sleeping no no no Matt I I was just thinking no good for you you know it's a thing like that tonight that gives you all the hope you need to keep growing you'd have been better off sleeping back it wasn't her baby you know well are you talking about she came to me three weeks ago I told her then that she could never have a baby never well why did she lie to her husband because you she was afraid you wouldn't let her take it but then I'm sure she's told him the truth but not doc you should have told me it this isn't right what if the real mother shows up do you think she will no no doctor I don't can I buy a drink man just one of course just one think that'd be fine a weeks ago we asked you to let us know which gun smoke stories you would like to hear again and your letters and cards have enabled us to select the five most popular but more important however to those of us here on the show with the wonderful expressions of warmth and interest we're deeply grateful or next week the first of the five most popular shows it's one we have called simply home surgery William Conrad as Matt Dillon U.S. Marshall for parley bear as chester or jealous as kitty other specially transcribed story on is the united states armed forces radio and television service


Item number SCP-2153 Object Class Euclid Special Containment Procedures Between 25 and 50 live SCP-2153-1 specimens are to be kept in Site 101's greenhouse, and are to be cared for in an identical fashion to non-anomalous clovers. Infestations of SCP-2153 outside of containment are to be destroyed through herbicide application. And webcrawlers are programmed to censor references to SCP-2153-A. Research is ongoing on methods to block emails sent by SCP-2153-A. Members of the public who receive emails from SCP-2153-A may be administered Class-A amnestics on a case-by-case basis. A Foundation Front Organization will assume credit for humanitarian airdrops by SCP-2153-A. Description SCP-2153 is a virus of unknown genus, which infects members of the genus Trifolium, classification Clovers, and spreads via pollination. Infected clovers, designated SCP-2153-1, invariably have four or more leaves. Humans attempting to pick or otherwise damage SCP-2153-1 instances will fail in some way unless they are carrying a method of monetary transaction capable of transferring a value of at least four US dollars. If a subject carrying the appropriate amount picks or otherwise damages an SCP-2153-1 instance, they will lose a value of four US dollars, at which point the instance will become physically identical to a non-anomalous clover. SCP-2153 is believed to have been engineered by an organization called Trifolium Charities, hereafter SCP-2153-A. SCP-2153-A is responsible for over ● humanitarian airdrops in various locations since 2005, containing both various humanitarian supplies and hundreds of preserved instances of SCP-2153-1 sealed in plastic sheets. SCP-2153-A will email subjects who have picked SCP-2153-1 instances within 12 hours of either picking the instance or of an airdrop occurring. There is currently no known way to prevent these emails from being sent, and they will also be sent to email addresses created by subjects after picking SCP-2153-1 instances. However, individual emails may be deleted normally and will not reappear afterwards. Addendum The following are sample emails from SCP-2153-A. The first was sent after picking an SCP-2153-1 instance, and the second was sent following an airdrop. Thus far, all emails received from SCP-2153-A are identical. Websites linked in the emails lead to 404 pages. Dear Recipient's First Name On behalf of everyone at Trifolium Charities, I would like to thank you for your recent purchase. Your support allows us to perform life-saving work all across the world, helping the people who need it most. To learn more about our organization and our mission, please visit our website at www.tropholium4good. While you're there, consider signing up for our monthly newsletter. Enjoy your clover, and best of luck to you. Reipens T. Prytent's CEO Dear Recipient's First Name Trifolium Charities has just successfully delivered humanitarian supplies in Location of Airdrops To learn more about our recent relief efforts, visit www.tropholium4good.com Slash Latest Remember, your donations are what makes these life-saving activities possible. Reipens T. Prytent's CEO


So, welcome to the Open Group webinar on Secure Global IT and Supply Chain. The Open Group, in particular the Open Group Trusted Technology Forum, has been addressing the problem of product integrity and supply chain security for commercial off-the-shelf information and communication technology, which we refer to as COPS-ICT, for the past three to four years. We published the Open Trusted Technology Provider Standards, specifically focused on mitigating the risk of maliciously tainted encounters at product in 2013. And last week at the Open Group Conference in San Francisco, we announced the launch of the OTTPS accreditation program, which credits organizations who conform to the best practice requirements and the standards. So this webinar is going to provide insight into our work that got us to this point, the resultant accreditation program that now is available to all and the outreach efforts for increasing awareness of these efforts. So later we'll hear from some of our members who had a significant impact on the creation of the standard and the accreditation program as they each present a portion of the presentation, and at the end we will participate in questioning and answers for that section. So I'll provide a brief overview of the global challenge we're facing, how the Open Group is positioned to address it, and some insight into how we got started and where we are today. So the problem, COPS-ICT products are developed globally and used globally. They're built from hardware and software components developed and manufactured around the world, and they're integrated into critical infrastructure and government systems and commercial solutions. And at every point in the product's life cycle, they're vulnerable to the threats listed here. Today the world relies on global supply chains, and therefore constituents from around the world must work together to protect them. And the global nature of this problem is one of the main reasons the Open Group was chosen as the place to work through and address this very, very significant problem. So a little bit about the Open Group. It is a global organization with over 40,000 participants, from over 95 continents with over 400 members, headquarters in over 37 countries. The Open Group provides a vendor-neutral environment where customers, both governments and commercial customers, and technology providers can come together to create standards and accreditation programs that are consensus-based and meet the needs of all the participants. So how the Open Group trusted technology forum gets started and become part of the Open Group? It began with a series of roundtable discussions with the Open Group and DOD, AT&L, and CIO, and it included many of the most mature IT providers in the industry. During those meetings, the following questions were posed and discussed. Now the governments and commercial entities around the world are moving away from high-assurance customized solutions to COTS-ICT. How can commercial customers and government confidently identify good trustworthy COTS-ICT products in their providers? So the recommendation that came out of these early meetings were these. If all the major IT providers are building quality products and for the most part secure products, then doesn't it make sense for them to get together, pool their practices, and come up with a set of best-to-breed best practices that could be established as a standard for the rest of the industry? And doesn't it make sense to then build a brand around the standard to identify those trusted technology providers who are conforming to standards? So that's how it all got started. These informal series of meetings found a lot of traction and the constituents decided to form under the Open Group as the Open Group Trusted Technology Forum and began some serious work on responding to those recommendations. So who are those constituents? Here they are. As you can see, from the member chart, there's a broad representation of COTS-ICT providers, integrators, and third-party information assurance laboratories. There's product distributors and there's government-related constituents. In my humble opinion, three of the most important takeaways from this slide are these. When we say we defined and published the standard and accreditation program, we mean the membership. It's the way that we're talking about and it was all done by consensus, which is a huge feat. Secondly, we meet and have met over the past three or four years, at least twice a week, meeting not only with the technical participants from some of these companies, but with the chief security officers and the chief technology officers, the VPs and program directors of the companies. That indicates an amazing amount of talent and commitment. And finally, it's a true example of industry working together with government to create something that's reasonable, open, and practical. And by practical, I mean that it's actually based on inputs from members who understand how product development and supply chain operations really work in the field and therefore can identify the best practices that are needed to mitigate the risks of the tanks and counterfeit products. So one last slide for me, just a brief overview of our milestones. As mentioned, we started with early collaboration. We formed the OTTF in 2011. We began working on a standard immediately called the Snapshot in version 1.0 of the standard in 2013. While we were finalizing the standard, we were also developing conformance criteria, the accreditation policy, and the assessment procedures that would be used in the pilot program, all of which we started at the end of 2012. The accreditation program was finalized and approved in October 2014. We implemented the program and launched officially last week, also announcing that one of our pilot participants had successfully made it through the accreditation program. And we're pleased to announce that, again, that successful candidate with IBM was accredited as an open trusted technology provider for their application, infrastructure, and middleware aimed at the software business division. So congratulations, again, to IBM. So with that, I look forward over to Ed McConway, the Chief Security Officer of Global Supply Chain Francisco Systems and Vice Chair of the OTTF, who will talk about the challenges of Global Supply Chain and how the standards address them. So Sally laid the foundation for what we're really all about. And after the initial conversations that we had, what we came to the conclusion as a public-private partnership was that we did need to identify this key challenge. Let me remind you of what that is. Sally, why don't you build this out? The challenge really is that we who make commercial off-the-shelf information and communication technology leverage a supply chain that reaches around the world. As we look to those from whom we would source, those who partner with us in the fabrication and the methodologies by which we deliver in between the supply chain and ultimately to the end customers of the supply chain, we wanted to have a program that would enable the acquirers to look at the product, yes. But rather on top of that, to look at the organization and to say we understand that from these organizations we can buy products that are trusted. We're buying with confidence and the products are trusted because of something somewhat unique. We actually, in this accreditation program and standard, encompass the full product lifecycle across commercial information and communication technology. And what we're doing is saying we want to look at this end-to-end lifecycle because we are aware of the fact that risks can occur at any part of this supply chain or product lifecycle.  So the takeaway there is public-private partnership looking comprehensively around the globe at the supply chain itself and as well looking comprehensively at the end-to-end product lifecycle. In order to focus ourselves, oops, sorry, go back there just for one moment, if you would, Sally, in order to focus ourselves in this first version of the standard, we tackled two not insignificant, quite substantial threats, but these two. And these two have been on the minds of our customers and the community of acquirers of information and communications technology. And we'll talk a little bit about the risks associated with them. They include maliciously-tainted products or counterfeit products. So, Sally, if you'd go to the next slide, that would be great. So you're all familiar with what the risks are. I won't drill into them, but I want you to take your mindset to why this standard is somewhat unique. By addressing these two particular threats across this spectrum that you can see below on this slide, what we've done is tried to take a comprehensive approach, thinking through the niche and stage of when information and communication technology is begun as an idea, how it's developed and designed, and taken through each step. A set of practices, whether they're technology practices, whether they're physical security practices or logical processes, that together establish the right steps in the right stage of this product lifecycle and supply chain to give that higher degree of trusted products and allow our acquirers to buy with confidence. Sally, if you can go to the next slide. So let me give you a little bit of insight. This is a very high level into the best practice areas that we address. And we've broken this down into two major categories across this comprehensive product lifecycle or supply chain, specifically product development and engineering requirements, secure development and engineering requirements, and the third one is supply chain. So you can see some of the critical focus areas that the requirements drill down on. And I'm just going to highlight a few that I think are unique. We really talked a lot in product development and engineering requirements about not just the traditional development and design practices, but product sustainment, recognizing that given the ubiquitous reliance on information and communications technology, that those who have it within their networks and how they use it rely upon, we must inevitably ensure the integrity of the product portfolio and the organization supporting it throughout its life. For secure development and engineering requirements, what we've done is embrace all of the traditional best practices, but also remind it ourselves that we have to have an outward facing view as well. So I want to highlight that what we are also embedding within the standard is a mandate to regularly monitor and assess the impact of the never-ending changes that we see in the threat landscape for information and communications technology. So you can see we've addressed full product life cycle, sustainment through utilization, and awareness of the external environment that affects our information and communication technology community. Sally, if you could go to the next slide. From the perspective of supply chain, we really again took a unique comprehensive approach, thinking with some of the best partners that we had here as members of the standard body itself, and thinking about what we wanted to drill down into. And clearly, you can see what they are, but I wanted to highlight just a few. So risk management and security go hand in hand. Understanding the risk landscape and implementing security in a comprehensive fashion must be aligned and are in fact embedded throughout the supply chain requirements in the Open Trusted Technology Partner standard. Additionally, again focusing on the end-to-end product life cycle, we have embraced raw materials, work in process, finished goods, and products that are at their end of life. And those products include hardware and software. They are somewhat unique, but as you know, in information and communications technology, work hand in hand. So that's a quick overview of the in-depth requirements embodied within this version of the TTPF. Thank you. So now I'm going to turn it over to the chair of the standard body, Andrash Sekal. Thank you very much, Edna. That was an awesome overview. So I am the chair and my name is Andrash Sekal and my segment is intended to help viewers understand the purpose, process, and value of the recently announced OTTPS accreditation program. And while the typical cyber threat is very well understood by most folks, you know, maybe you're not as familiar with that of technology supply chain threats. So we've tried to kind of boil this down into a table as a reference to help you understand the landscape and which threats the OTTPS standard and accreditation program is intended to mitigate. Now, of course, for more information, you can download the freely available OTTPS standard or shameless plug. You can get a copy of the most recent issue of the IBM Journal of Research, which I published an article on this particular subject, and endeavor to learn a little more. But version 1.0 of the OTTPS standard and the accreditation program are focused on helping to mitigate what we believe are the two most significant threats, as Edna pointed out, counterfeit components and maliciously tainted products or components. From a taint point of view, we also think our standard goes a long way to mitigating those other risks that are outlined here. However, we do not explicitly focus on the insider or negligent cases. We believe that the OTTPS standard and now the accreditation program is exemplary of a holistic approach to securing the global supply chain. The trusted technology forum was founded by our customers and includes representatives from each of the major stakeholders who participated in the development of the standard and the accreditation program. This ecosystem is essential to effectively addressing the risk in establishing a normative standard and representative trustworthy accreditation program. Remember, the OTTPS accreditation is an organizational assessment against the industry best practices described in the OTTPS, the standard. And this industry ecosystem is depicted here and includes providers, in other words, the technology producers, their suppliers, component suppliers, integrators, and in partnership with standard organizations like the Open Group and the labs that are responsible for third party accreditation to the standard. Next chart. Last week, the OTTPS officially announced the launch of the OTTPS accreditation program and, of course, IBM's successful pilot and accreditation for our Aim Product Service Division. I'm sorry, Aim Product Division. I used to be part of the Aim Services Division, so I got that on my brain. While the development of the standard requires Open Group membership, both the standard and the accreditation program are open and freely available to the public. The standard can be downloaded and used by the organization, any organization, free of charge. And the accreditation program, while operated by the Open Group, is available to the public as well. And the accreditation are conducted by third party labs on behalf of the Open Group, following the accreditation processes, practices, and policies that were established within the OTTPS and by the industry. So an open OTTPS accreditation, I'm sorry, an OTTPS accreditation lasts for three years before an organization must re-accredit. In the meantime, an accredited organization warrants and represents that they continue to maintain the practices that meet the accreditation conformance criteria. And they can associate the OTTPS accreditation logo with the products produced by the accredited organization. We believe this approach will significantly increase the trust across vendors, suppliers, and acquires to build with integrity and buy with confidence. So an organization seeking accreditation may only produce one product or many products. It doesn't matter. The accreditation process works for providers of all sizes. In most cases, providers will accredit an organization that's responsible for producing many products, in which case we have developed an approach that uses established techniques for identifying and sample size that will be used to assess the candidate organization and make the accreditation practical and affordable. This is called the scope of accreditation. It may consist of a product line, business unit, or maybe an entire software company. It really depends on the provider and how they want to approach the accreditation. As I said previously, the accreditation lasts for three years in which the organization warrants that their practices represent those defined in the OTPPS standard, the best practices. Should any nonconformances be identified, the organization must remediate, i.e. come back into compliance in a timely manner, or they'll be removed by the accreditation authority, in this case the open group, from the accreditation registry. The OTPPS trademark represents to the industry a trusted technology provider. So why the open group? Why did the customer and the industry select the open group? The reason is really very simple. It's a very significant history of effective public-private partnership, standards development, and a track record of successful certification conformance programs, which includes really well-known profession certifications, the UNIX specification standard conformance to it and POSIX, and even lottery and WAP conformance to best practices, as well as core technology standards. Now I'm going to hand it over to Fiona Pattinson of ATSEC to talk about how we are managing third-party labs and assessors to implement accreditation. Okay, thank you very much. I'm Fiona Pattinson, and I'm working with one of the already recognized assessor companies that were part of the launch last week. So I'd like to talk a little bit about how the assessment and the becoming a recognized assessor works in this program. First of all, some of the key points about the assessment is that the open group and the OTTF forum have produced some publicly available assessment procedures. This will help achieve objectivity, repeatability, and consistency as we're performing the different assessments. When we look at the standard and the assessment procedures, we find there are two types of requirements that need to be assessed. We look for evidence that the processes that are recommended and part of the standard are there in place and that they are documented by the organization. We are not just performing a process assessment. We have a second set of checks that we make to the implementation evidence. So we look to see that those processes are indeed implemented in the organization. The open group program has established formal recognition of the OTTPS third-party assessors and they have established criteria and examination, which I'm going to explain in a little bit more depth in the next few slides. It receives certificates and there is a public registry of the recognized assessor companies so that this can be checked. The requirements to become a recognized assessor we have divided into two components. First of all is the component of belonging to a recognized assessor company. So here we're looking for an organization that is mature and professional in the way that they handle such assessments. We have a variety of management system standards that are related to this in the world of assessor companies and I'll go through those on the next slide. But the key point is that the open group will accept organizations that are already certified management systems and they're looking at things like documentation management, record control, personnel, training, resource management, their own internal auditing and how they handle preventive and corrective actions. This reliance on existing industry standards is a very exciting development. It allows for the recognized assessor program to take advantage of the established pool of IA assessors and their companies that are already in professional practice. So we talked a little bit there about the company that could be a recognized assessor. We also talked about the individuals who are the actual assessors and their competency. So the requirements for that is that these people have already been trained as assessors and have a minimum of two years experience in performing process assessments and in looking at people's documentation, organization's documentation. Thank you. Again, here I'm going to detail some of the standards or the standards that are currently accepted in order to become a recognized assessor. So they're on the left. There are three standards that are very often used in the information assurance industry to qualify assessment companies. They're ISO standards, and there are three of them listed there today. ISO IEC 17020. ISO IEC 17021. And ISO IEC 17025. So if the company is already accredited to these standards, then they can reuse that accreditation to become an open group assessor company. For the actual individual assessors, then the auditing or assessing qualifications that we commonly see in the information security assurance field can be accepted. The current list includes lead auditors for some of the ISO standards, the 2701, the information security management systems, the 9001, the CMMI appraisers, common criteria evaluators, and FIPS 140-2 testers. If you're interested in applying, all the information is on the accreditation program website. Okay. So finally, here we talk a little bit about the specific requirements that are required to join the open group program. So in addition to an existing management system, the company must also show and prove the open group accreditation authority that they do have established processes for performing OTTPS accreditation. And for the individual assessors, then we work very hard to make sure that the skill level of these people in the subject matter of OTTPS is sufficient. So we ask for skills in supply chain management, the terminology and techniques. We ask for technical knowledge of all the OTTPS attributes and the OTTPS assessment program. And we also have in place an OTTPS assessor examination, which the individuals must successfully complete. So that's a brief overview of the accreditation program requirement to be recognized assessors. And with that, I'll hand over to Dan. I'm Dan Reddy from EMC Corporation, and I am co-chair of an internal work stream of the forum that is focused on a combined effort to both reach out to a number of global constituencies to inform them about our work, but also to harmonize their work in building the standard and the accreditation program with other initiatives where there may be some synergy. Our approach in this effort is to remain fact-based so that we can be a credible partner in this broad community. When we're referenced by other entities, such as the ones mentioned here, like the GAO report and the NIS Special Publication draft on supply chain risk and NASA RFP that has a reference to our standard, we think if it is based on facts and the word gets out that it helps everyone understand how our work ties in with other work that is being done in other spheres. On a regular basis, we are asked to speak at conferences where there may be a focus on maliciously tainted or counterfeit products, and there could be some synergy, and if you know of any such opportunities, you might bring that to our attention. In addition to focusing on facts for our outreach, we focus on the details of our technical content because we think that that's important for the community. We've done several internal efforts where we have taken our detailed requirements and those in other standards, and we've looked for alignment or congruity or to identify possible differences and gaps. One strategy that we're following in order to build momentum whereby others can leverage our work is to make it clear to the outside world what we're about and to talk about the value that we have in having someone voluntarily adopt our standard and possibly the accreditation program. We think that it is a more solid foundation to take that approach rather than a process that would rely on mandatory regulation. As people learn about our work, we hope that this effort will grow and there will be more cross-references and that the demand will grow. In terms of our priorities, obviously with the launch, there are activities like this podcast that are ongoing in support of the launch. We think it's a big deal that we have a standard that focuses on supply chain risk management and offers a measurable method to gauge that conformance. Anything that we can do to get the word out and is good, we hope to continuously look at our own content, especially as we compare it with content that may be out in the world at large. Let me just highlight some of our key priorities in this area of harmonization. Again, we've combined harmonization and outreach. Although this is a global standard built through content, we realize that there is further value in linking our work to that of ISO and IEC. We are actively exploring a potential, stronger relationship with ISO regarding their related work concerning supplier relationships, and we have a formal liaison in place. The open group is already a recognized submitter to ISO and IEC, so therefore we think we're in a good position to consider submitting our standard as a relevant part of the work for ICT COPS providers. From the very outset of this initiative, we have made our relationship with the common criteria a high priority. Many of the larger technology providers, such as my company, are actively going through common criteria certifications for our products on a regular basis. We see a great deal of synergy. Some of the same companies and individuals who are working on the common criteria of supply chain technical working group are the same people and companies that are working on this initiative. We think that the threats that are being addressed here are also aligned between the groups, and we are moving in a direction of formalizing our working relationship with the Common Criteria Development Board, hopefully creating a formal liaison between the programs. As I had mentioned previously, we feel that we can add value by mapping our work to those of others, and we have done that internally with much of the work embedded in the Common Criteria, which looks at products, not at organizations and the practices the way we do. But it's important to note that the members of this forum have adopted a measure once approach, whereby during the trusted technology provider accreditation process that has been discussed today for a particular product organization, they may encounter a situation where the assessor is looking for evidence that a product team has met a particular OTTPS requirement. That evidence may be related to the same product that has been certified under Common Criteria for a matching requirement. We expect to be a leader in recognizing the value of those forms of evidence so that we can measure once as we go forward with our accreditation. We believe that this approach of focusing on the facts and the content and making continuous improvements is very important for the ongoing credibility and adoption of our work. So I think we're nearing the end. The next slide gives you an overview of a summary of the accreditation process that Andrash and Fiona have outlined. So this can be a good reference for you as you take a look through the slides. And then the last slide in the deck has a series of resources that we would encourage you to tap talking about some of the work that we have done in written form or other testimonials and information that can be valuable. So on behalf of the entire team, thank you very much for joining us. We would like to hear from you in terms of other ways that we can inform people about our work and those opportunities will be something that we can capitalize in with your help. Thank you. Good. Well, thank you, panel. We've now got to the stage where we can go through some of the questions that we've had submitted on the QA facility. We've got a few, so if anyone's got any more, then please do send them in now. Start with the first question from John. John is asking, so if a specific product is accredited, does that mean the entire supply chain for that product has been reviewed and accredited? Or does each vendor stroke element in the supply chain need to be accredited? Who would like to pick that one up? Well, it's Andrasa. So the organization as a whole becomes credited against the best practices. So you don't have to become accredited for each component or each element of the standard. It's a holistic approach. So the question, I think, really was do we look at the entire supply chain in a product? And I think our best practices, our requirements, as Edna pointed out, do cover the full life cycle of a product. So from design disposal, including supplier relationships and how you interact with your suppliers and things like that, we do believe that it would be in the best interest of everyone if the component suppliers got accredited on their own. And we hope that because that will be assurance that they're following the best practices through all of their development cycle and ensure that they're not contributing to stated and counterfeit products. So the goal really is to have component suppliers also be accredited on their own. Did Dad answer the questions between Andra and I? Yeah, and we also have accreditation requirements, best practices around kind of holding your suppliers to the same, you know, level of integrity as those best practices defined in the standard. Right, that's true. Okay, thanks. John's just put in another question asking, has there been any thought to how a requirement for accreditation might be included in government contracts competed under FAR Part 12 commercial items? Well, so if you look at NIST SBA 100161, they've done something very similar to what we have done here, and that is that they are trying to be descriptive and outcome-based and not pejorative. And they do cite the OTTPS standard in that piece of work. So I believe that if there is a defined standard and there is that of policies that may get established, you know, per the NDAA that was just passed for 2014, that we may be one of probably maybe the preeminent program in which to measure compliance. But there might be many different ways a company can use both this program and other techniques to show compliance. So we're hoping that this doesn't, like, get written into some sort of stone, you know, from a policy perspective because there may be many exceptions as well as rules that could be established and that might make it more difficult for the industry to evolve over the period of the next few years. You know, if I could just emphasize that, you know, I mentioned that our approach is to get people to pay attention to this on a voluntary basis and not to try and actively try and get it embedded in regulation. And as Andrush said, there are, especially in the area of supply chain, this is evolving and it is unclear to many in the U.S. exactly what constitutes the right kind of best practices and how you measure it. And as he said, we do offer something that is clear about building global consensus and is measurable. And when you compare some of the baseline controls in 853, for example, I think there is good alignment between many of the fundamental practices that we have in our standard and the accreditation program that line up suitably with some of those expectations. Okay, so let's go on to the next question. This is a question from Richard. It's a question and a statement. So first he's saying congratulations to IBM for being the first certified. He's asking, how quickly can we expect other vendors achieving certification? Do we have any visibility on that stage yet? So this is Edna from Cisco. Let me take a crack at that. I think, Richard, that we have membership that is looking at it. So those vendors who are part of the TTF, as well as those outside. And I don't think we can give you a specific anticipated commitment on when folks might be accredited. I think everybody's in the stage of evaluating the excellent work that has been done by the forum and the standards alignment with other standards, which are utilized across our various organizations. So that's a way of answering without specifically answering. Lots of folks thinking about it. Not sure yet. We do have one other company that's participating in the pilot at the moment. So I think we're all weighing the many benefits that we see and trying to fit it into our schedules over the next fiscal year or so. Thanks, Edna. So question here from Jerry. This is a long one, so bear with me. Jerry is asking, how does accreditation requirements impact small businesses that are engineering services and ID product integrators for the larger OEMs? He goes on. Will small businesses be expected to be accredited if they are in an authorized reseller or IT product? Or IT product, sorry. Yeah, I believe our goal as industry leaders and I believe that our government partners would also agree is that over a period of time, you would see the industry adopt the best practices and that would include small companies as well as large companies. And we really painstakingly took a lot of effort to focus on making these best practices consumable for small one, two, three, four, five, 50-person companies. So I believe that it's very much attainable for smaller companies, even a one-man company. It is certainly probably even a little bit more work for larger organizations, I think, in some ways because you have separate lines of responsibility. But these are best practices and they're very similar to ISO 27,000 security controls based on outcome-based best practices for supply chain security and integrity. If I could just add to that answer. We were also very cognizant as we were building the accreditation program that there was enough flexibility that we could accommodate a number of different situations where we would have different kinds of organizations. It could be an integrator. It could be a component supplier. And the way they describe what their products are, that same flexibility because we felt that it was not possible to build something that was one-size-fits-all. And yet we had to have enough standardization to have real conformance that you could bank on. Yeah. And that's a good question, Dan. I mean, a point, Dan, because we used the word provider consciously. We didn't use the word supplier. We use provider because a provider might be an integrator or they might be an actual producer of a product that gets consumed upstream. Right. And they could be producing that at any level within the supply chain. Right. Yeah, so this is Sally from the helping group. I think one of the beautiful things about this program is this holistic approach, as one of Andras' slides pointed out, so that if we can really get all constituents involved, which really is the point of securing the whole all of the global supply chains, then that would be a wonderful thing. Your component suppliers are practicing the best practices and they're accredited. The technology providers who are utilizing the component suppliers and integrating their components get accredited. The integrators get accredited. And then you ask, then you partner with those open trusted technology providers who are accredited and are reducing the risk of counterfeit and tainted products. So yeah, this is a global standard and it affects, it does affect all constituents. And the point, one of our markers was to raise all votes so to make it attainable for all constituents around the world. Thanks, Sally. We've got one final question. This is from Srikan. And I think he's really asking for a little bit of information about the upside of getting involved. So he's asking, what is in it for organizations that have to fund the implementation for accreditation? And he's given example, so accreditation to OTTPS will bring an organization to any compliance requirements such as PCI, DSS, or FISMA, or HIPAA. And he says, if not organizations may not really fund it. So does someone want to just talk to that point? So, you know, first off, I would say, you know, my position from his company's point of view would be reviewing and assessing the standard itself, the impact on your business with respect to the standard. And are you actually conducting those best practices? And then you have to make a business decision about the customers you're serving or the supplier or the provider who's using your components and the business decision they're making to prioritize over those component suppliers who are providing not only a well-formed, you know, valuable service or product, but also one that they can trust. And so really, this is going to be, you know, business-driven requirement ultimately. But I would, you know, begin assessing and using the standard today. This is Edna. Let me join in as well. And Dan may want to speak to this. There's a corollary to that as well, which is, in addition to assessing yourself, as Andras just articulated, we have gone out of our way and will continue to do so to identify where we align with other standards, some of which you referenced. And the reality is that there may be some overlap, there may be some positive recognition where you can, when I say overlap, not in a negative way, but a benefit from doing one that you will automatically get credit for or be able to leverage and utilize effectively. So, you know, this isn't a standalone standard that sits on a mountain and does not recognize that is part of a global standards environment. And I think that's very important to note that most of us who will be looking at this will recognize that it fits nicely and dovetails with some of the other activities in which you're engaged. But again, you are correct. All of us need to make business decisions based on cost and efficiencies. And that is the very fact why we tried to align and we had a harmonization organization or subtract that Dan was heavily involved with. Dan, do you want to elaborate on that? Sure. I do agree with Andras in terms of the business decision around the value. Do you and your customers see value in being listed as a trusted technology provider in a way that has been measured by an independent third party? That's the fundamental decision. And when you look across the landscape of requirements, the whole area of supply chain is relatively new to some of the other areas of security. And this is something that is concrete has value and is being measured today. How you translate this particular conformance through the accreditation program to the various groups that you work with, I think it represents solid evidence that you can point to. We are focused on the threats of maliciously tainted encounters, but some of our practices that are foundational include regular things like vulnerability response and secure coding. And these are things that you can point to in a variety of settings where you've had a third party assess your practices. Okay, team, thank you for that. Well, we've come to the end of the question, so I think this will probably be a good time to bring today's event to an end. But Sally, would you like to say anything just to finish off with? Sure. Just thank you for everyone who participated, and please do feel free to contact me, s.long, at opengroup.org if you have any further questions. And also, the links on the slides are very important. Please go visit the forum to see what that's all about. But the accreditation link will take you right to the live launched accreditation program. It has all the information there that you would need to prepare for accreditation, including the standards and the policy and the assessment procedures. So please do explore that site. Thank you, everybody. Sally, one last thing. I would just like to clarify that you do not have to be an open group member to take advantage of the free standard or to go through the accreditation program. We certainly welcome people to join as members if they want to shape the future of the standard in the accreditation program, but it is not a requirement for being accredited. Absolutely. Thank you, Dan. All right, Sally. Well, thank you for that. Thank you, team. And thank you, everyone, for joining us today. And I'll bring today's event to a halt now. Thank you very much.