And so from Lagos, as Nigeria moves into the rainy season, residents of Ijegun area in Ikotun, Ijegun local council development area of the state have raised an alarm over the poor stage of their roads. They called on the state government a hastened road rehabilitation and construction in the area to avoid being caught up in the flooding that may come with heavy rains. Our correspondent Love Ikoko Yedokun visited the area and now reports. For many years, Ijegun Ikotun road remained in a bad shape with portholes and blacked drainage system. This took a toll on vehicles flying the area and on fares. Residents have to pay higher to move in and out of the area. To let the car or motor or motor to come from Ikotun to Ijegun or from Ijegun to Ijagemo. In fact, it is very terrible when rain is falling. The whole place will be flooded with water. Just as reason that just a grady road now is more or less a little bit mutarebuna. I'm very worried because when it starts raining now or when it stopped raining, so to go out you will not be able to see all these bikers, you will not be able to see this Marua and most of them will be calling price. So if you don't have enough transport you can't go anywhere from this place. Though the road looks partially rehabilitated, the worries of residents have not abated with the raining season here again. They fear they might just be rained for another round of nightmare while the season lasts. You know, never reach almost two months. When they level this place now, better look out this road now. It's not good, it's good anyway. Secondly, if the June light come now, we see all this, the level that we are doing, the erosion we carry, all the water, the sound go to Gota. We want government to come to our head, at least to come and help us to do this road from Mijegu down to Ujagemo. So even the lander is going to buzz out about that, that is the last road. If it is done in that way it will be better. Weather forecast has it that Lagos State will witness about 250 to 300 days of rainfall across the 20 local governments this year. It is also expected that the recent increased frequency of extreme weather events will continue with days of extremely high rainfall, which may result in flooding according to the Nigeria hydrological services. From Lagos, lab could be weird. Last evening.


Thank you for coming through this afternoon, my first time in Madrid and first time at Big Data Spain, so really excited to be here today. Today we'll be talking about productionising machine learning pipelines with the portable format for analytics. So I'm Nick Pantryth, MLNIC on Twitter and GitHub, I'm a principal engineer at IBM where I work for the Centre for Open Source Data and AI Technologies, or CODE, I'll talk a little bit about that team in a moment. I focus on machine learning, data science and AI applications, and I have a long history in the Apache Spark project where I'm a committer and PMC member of that project, I've written a book on machine learning with Spark, a little bit out of date now, and travelled around the world speaking at various conferences and meetups on machine learning related topics. So I mentioned the CODE team within IBM and when I joined this team it was known as the Spark Technology Centre, and was founded by IBM to focus on the Apache Spark project and the kind of surrounding ecosystem, and it's a purely open source team working out in open source in the community, everything we do is in the open, and more recently the focus of the team really evolved to encompass not just Apache Spark, but the entire end-to-end enterprise AI life cycle and all the technologies that are part of that. So we aim to make it easier to create, deploy and manage AI applications in the enterprise, and that involves focusing on still on projects like Apache Spark, but increasingly the Python data science stack, including pandas like it learn, the deep learning libraries, TensorFlow, QRS, PyTorch, CAFE and so on, and a couple of other projects, the model asset exchange and the fabric for deep learning, which I'll mention just at the end, but today we'll talk about one of the other important aspects of this life cycle that we focus on, and that's model deployments, and in particular open standards for model deployment. So the machine learning workflow is, as we all know, really simple. You start with data, you do some machine learning, and you profit, right? You can all go home and we've done our jobs, but in reality this is a really complex workflow, and it spans teams, so you do start with data, that's true, but that data can be sitting around in various data stores across formats, some of it is historical, some of it is arriving in real time in a sort of streaming fashion, and that's normally the domain of your data engineers, who are managing various data stores, metadata, cleansing, all of that kind of thing. You then need to ingest that data in various formats into the core data science and machine learning and research workflow, which is all about starting with data, pre-processing, visualization, exploratory analysis, trying to find out what are the properties of this data and what can we use it for, how can we join different disparate datasets together, then moving to the feature engineering and feature extraction, and then into the part of the workflow that everybody obviously enjoys the most, which is your model training and development and model selection, and that's a sort of workflow within a workflow, constantly tuning different types of models and pipelines, and selecting the one that is going to perform the best on the particular business problem or use case that we're dealing with. And then we hit the end of that piece of the workflow, and that's where a lot of the typical discourse on this topic stops, okay, once you've got a model you're done, but really you need to deploy that model into some production environment, into some application to do anything useful with it, and that is normally the domain of your machine learning and production engineers who are running live systems 24-7, high availability, high throughput, they care about performance and uptime, so as we'll discuss a little bit later, this is not just about deploying machine learning models, but actually machine learning pipelines, you have to worry about versioning, lineage, tracking, all the input features and parameters that went into that model that's deployed, and then once it's in the live system, predict on new data, monitor the performance of the model, and gather feedback, which then goes back into your data store effectively, that feedback is used to retrain models or update models, and that is again arriving in real time and stored in your batch systems. So this workflow has a lot of moving parts, it's really much more complex than the common perception, and it involves all these disparate teams working together with very different goals in mind, and that workflow, and in particular the data science kind of sub workflow spans tools, so every data scientist, machine learning engineer, researcher and so on has their own favorite tool, whether it's R, Python, deep learning, Spark, they're all dealing with a wide variety of data formats, so the data is sitting around all kinds of formats and needs to be ingested into these different frameworks, the different ways of approaching model selection, cross-validation, even building pipelines, and all of these tools are typically in play and in use in any given scenario and any given organization and team, and finally that entire workflow is only a small piece of the puzzle, so I really like this image, which is from a Google paper, the hidden technical depth of machine learning, and it really illustrates that machine learning code, while critical to the intelligent application and to the business use case and adding a lot of value, is really just a small percentage of the overall picture, and in particular, we need to worry about how is that code integrating to the serving infrastructure, how are we getting it from that training to production and deployment, which is what we're going to talk about today, machine learning deployment. So machine learning deployment covers a few questions, three important ones, which is what are you deploying, what is the model that we all talk about all the time, where are you deploying it, what is that target environment, the runtime, is it for batch inference, streaming inference, real time, a combination, and how are you deploying it, this is the DevOps deployment mechanism, exactly what is the details about how you taking that model and putting into production and serving frameworks. So all of these are important questions, but we're really going to focus more on the what today, and perhaps a little bit about the how. So what is a model? Everyone talks about machine learning models, and when we say a model, typically what we think about is okay, I've trained a logistic regression, and that's my model, right? That's my algorithm, or it's my deep learning architecture, my neural net, my image classifier. But really, almost no machine learning algorithm actually takes raw input and spits out a result, they need to operate effectively on vectors or tensors, so they need to operate on a numerical representation. And even though the common perception of deep learning is that it can operate on raw data and there's no feature engineering involved, that's also not true, because even in the simple case of image classification, you typically need to do some pre-processing to that image, resizing, cropping, and so on. So there's always this pre-processing step, which has to take some form of raw data and raw features in the form of categorical variables, text, raw images, and pre-process it and transform it into a form that is actually usable to train the model. But before we've even done that, we've actually got to start right back at the beginning from that raw data and joining those disparate data sets together, transforming the raw data and extracting features from the data, using feature extraction techniques to combine features and create new features. So there's a whole set of steps that come before that model training. So at training phase, we need to apply all of those steps to the raw data to get to the model. We then make predictions and update the model weights using some optimization technique until it fits the data really well. Now, if we don't apply those exact same steps at prediction time, then we're not going to have any model that is useful.  And if we're not following that exact same set of steps and transformations that come before prediction, we're not going to be successful. So deploying a model is all about deploying a pipeline and not just a machine learning model or a trained machine learning algorithm. So deploying the model part is just simply not enough. That entire pipeline must be deployed. So the data transformations that came right at the beginning on raw data, all the feature extraction, feature engineering and combinations, the pre-processing steps, of course, the machine learning model itself, which is important. And then often overlooked is actually the inference result transformation or prediction transformation. So typically a business user doesn't really care about tensors that don't know what a vector of numbers may or may not mean. That vector of, let's say, class probabilities and predictions in a classification problem needs to be applied. You need an inverse transform applied to get that into that kind of domain knowledge or that raw data space so that you can associate, for example, the actual class labels with those numbers and do something useful. So that entire pipeline needs to be deployed as one unit. And it needs to be consistent between training and test time or training and inference time. And even ETL pieces of that process and ETL steps in that pipeline are actually a part of it. So fortunately for us, as practitioners, we have pipelines available in many machine learning frameworks, you know, Cyclo and SparkML are prominent, also TensorFlow and R. And these frameworks make it really much easier for us to chain these pieces together, train them on data, and then get back a kind of unit that we can then deploy. So all of those frameworks make it a little bit easier to deploy pipelines. But we still have many challenges, right? So we need to bridge all of these gaps between languages and frameworks. The data scientists and machine learning researchers may be using Python R. They do a lot of things in notebooks. Maybe they're using a bit of, you know, Scala with Spark. But your production environment often wants to use a compiled language for performance, so Scala, Java, C, Go, Rust, perhaps. So frameworks, I mean, there are too many to count, and you typically, in any medium to large organization, going to see all of them. And everyone wants to use their latest and greatest version and their latest and greatest framework. You're going to have to manage all the dependencies of these different frameworks and components and pieces and versions of both your frameworks and your models. And the performance characteristics can really vary widely across these dimensions. So a model in R, or Python, raw Python, kind of maybe not that performance. But it may be extremely performant if it's wrapping C++ or TensorFlow code or PyTorch or something like that on GPUs. So it's really difficult to reason about what is the performance of each framework and each machine learning pipeline going to be upfront. So then in the same way that you have to bridge all these languages and frameworks, you have to bridge these teams that we saw earlier. And each team has different objectives and different needs and goals. So data scientists and researchers always want the latest and greatest. They may want to be deploying models off the TensorFlow master branch because it's got some new version of an RNN or it's got some new transformation or some performance improvement. Production engineers really don't like that. They care about having control, stability, minimizing changes they don't like to be working up to in the morning with a pager because the server's on fire. And the business user doesn't care about the details when it comes to the technical details. So they don't care that you're using TensorFlow, PyTorch Keras, whether it's running on Kubernetes, all they care about is that it's up, it's working, and what is the impact on the business? What are the metrics that we care about? So in the same way that we have to have this proliferation of languages and frameworks, where that comes formats. Each framework has its own format. So they are sort of open source but non-standard. You can go and look at what they are, MLEAP, Spark, TensorFlow, Keras, PyTorch. They all have their own internal format for representing models for saving them, for saving pipelines for exporting them. You, of course, have proprietary formats, which may be very performant, but they completely lock you into one particular product. And then we have open source, open standard, which is what I'll be talking about today, including PFA. So this lack of standardization leads inevitably to custom solutions. There's always custom code. And where some standards do exist, often the limitations of those standards lead to custom extensions. And the minute you extend a standard with some custom component or plug-in that is non-standard, you lose all the benefits of standardization. And there's no point in doing it in the first place. So this is a general talk on ML pipelines, but Spark is important here because it sort of sparked the work, so to speak, that we did in PFA export. So Spark solves many problems. And this can, to some extent, apply to other pipeline frameworks, like CyclicLearn, for example. In that each component within Spark, Spark data frames, Spark ML, streaming components, allow you to use one holistic system to manage all of these different parts of the pipeline. And all of the different teams can use one API, effectively, one set of APIs, which is really great. But it introduces additional challenges. And again, these are not completely unique to Spark, but certainly exacerbated. So the main problem with trying to score models built with Spark ML is that there's a tight coupling to the Spark runtime. So that introduces two problems. The first is you have to manage a lot of complex dependencies. So in order to score a model, you have to actually go and deploy a full Spark deployment or instance, which, yes, it may be running on one instance locally as Spark local. But you still have to worry about a lot of compatibility, versioning issues, and a lot of dependencies that are potential dependency conflicts that are pulled in for no particularly good reason. And probably more important, that scoring models for Spark in real time is really slow. So it's optimized for batch scoring and for scoring over data frames or streaming. I mean, this is changing a little bit with the new streaming optimizations in Spark. But certainly the overhead of data frames with respect to task scheduling and query planning are just not suitable to real time. So if you need a few milliseconds to up to maybe a few hundred milliseconds of latency, you're just not going to achieve that. And then in addition to that, you have to do a lot of custom work just to get Spark models out of Spark. So if you want to avoid these, you have to export those models into some sort of custom format, using custom export formats or readers. And you then have to either export that saved model into a custom machine learning framework for scoring. Or you have to write some sort of wrapper or converter into something like scikit-learn if you can. So everything ends up being custom, everything ends up being slow, and there's a lot of complexity. So before we talk about open standards, I'll mention one solution to this, which is becoming very popular. That's containers. And containers have seen a lot of success and wide adoption in software deployment in general and certainly for machine learning software. So it provides you a lot of benefits. You get this repeatability and ease of configuration in theory that if something works in Docker on your laptop, it'll work in production. And it allows you to have the separation of concerns between data scientists and production engineers. So the data scientists can focus on their favorite framework, what is the model that they're training, what is the what of the model. And the production engineers, the DevOps environment running Q-clusters and all of that, can focus on how, how it is deployed and managed. And each one doesn't really have to worry about the other. But that's not entirely true, because what goes in the container is still the most important thing. So as I mentioned before, performance can be really variable across these different dimensions of language framework, version, et cetera. And to do this correctly still requires good DevOps knowledge, deployment pipelines, continuous integration and testing, good practices. And most importantly, it doesn't solve this issue of standardization. So if you can throw anything into a container and throw it over the wall, that's all very well for easing deployment from a technical perspective, OK, you just push a container onto your Q-cluster. But it doesn't tell you how to consume that, you know, that new microservice in your business application. There's no standardization that makes that easy. So it ends up, you know, you end up having to go and inspect what's running in that Docker container to figure out how do I even call this model, what format, input format does it expect, what does it give me back. So there's no standardization with respect to the formats and what's in the container or the API is exposed. So you still have to write or use some complex serving framework on the top of that. So on the formats issue and problem and the aspect of standardizing model formats, I'll talk about the portable format for analytics as a solution to this. So PFA is being championed by the daily mining group, of which IBM is a founding member. And the DMG previously created PMML, which is the predictive model markup language. That's an XML based standard, which is arguably the only real kind of viable sort of open standard in use today. And it has wider adoption. It's used for many traditional machine learning applications. But it does have many, many limitations. And the key of the limitation of PMML is that if something is not in the spec, if a certain model or transformation that you want to use is not baked into the spec, you simply can't use it. It's very difficult to extend things in PMML without having a custom extension. And the minute you have a custom extension, again, you lose all that benefit of having a standard. So PFA was created by the members of the DMG specifically to address these shortcomings. And it's aimed to be a much sort of more flexible version of PMML in many ways. So it consists of JSON as a serialization format. So instead of XML, probably a little bit more modern JSON formats, it uses Avro schemers for all type definitions. So to define the input and output data types that you want to work with, and that your model works with, as well as the inputs to the input types to various function calls, it's all done with Avro, so it's all standardized, very flexible and extensible. And what PFA does is it codes the functions called actions that are applied to each input to create the output. So that's using a set of built-in functions and language constructs. So it acts like a sort of mini language, as well as the ability to combine various built-in functions together in user-defined functions. So essentially, you can think of PFA as a mini functional math language with a schema specification attached to it, a type system. So because you have this type system, and it's effectively sort of strongly typed, it means that any valid PFA document can be loaded by a compliance scoring engine. And it can be verified at load time that it will execute correctly and give you the correct result. So you may have certain runtime errors, but you eliminate a lot of the potential for runtime errors using this approach. So what this gives you is true portability, true separation of concerns from the model producer that is exporting that model from the pipeline, and the model consumer, which is your production environment or your API, your business application. So it's portable across languages, frameworks, runtimes, versions. If you have a compliance scoring engine in PFA, you don't care where the PFA model or document, as you call it, came from, as long as it meets the spec you can execute it. So I'll give you a simple example of multi-class logistic regression. So the key components here is we want to specify the schema, so the input and output that this model operates on, and then the action, which is the core model logic. So as you can see here on the top left there, you have an input and an output. And in this case, the input is a double array, so a feature vector. And the output is just a number, a double, and this would be the predicted class. And on the right, we have the action. And the JSON representation is really meant to be human, operated on by machines. But it is human readable. It's a little bit verbose. But essentially what it's doing is kind of very similar to what you might do in Python code. You at first do a vector dot product and add a bias, which is exactly what the model dot reg dot linear function actually does. It takes that input vector, which is referenced as an array, and the other argument to that function is a cell, which is the model coefficients. Now in the cell, we'll talk about it in the next slide, the cell is a form of data storage. So the cell there is storing the coefficient vector and the bias. So it performs that dot product. It's a syntactic sugar, and a little helper function to do that for us. We could actually break it down into individual functions if we wanted. And once you've got that, we do an argmax on that vector, and we end up with the predicted class. So while a little bit more verbose, it looks very similar to what we would actually do in a language like Python. So I mentioned cells, and what are they? So in PFA, the data storage is specified by cells and pools. Cells are a lot more common. So a cell is effectively a named value. You can think of it as a key value pair. And it acts as a sort of global variable during the execution of each PFA document. So each time you do an execution or run of that PFA document, it'll perform that set of actions that we saw before on the input, and it'll return an output. And at the start of each run, that cell value is effectively immutable between runs. So that cell value is available as a sort of fixed value at the beginning of the run. During the course of execution of that action, you can update the cell value. So it is immutable within an action, within a specific execution of a PFA document. So in this case, here's an example of, let's say we're doing text processing, and we want to convert text into a bag of words or something like that. What we need is a vocabulary mapping. So we need to map the actual each term or word to an index in a vector. And for that, we want to use a map. And we can see here that we've got a vocab mapping cell, and it has a type definition, which in this case is going to be an Avro record. And the record has a field, which is then a map. And it's a map from string to integer. It also has an initialization value, and that init value at the top there, which is circled, would be the state. So cells are a way to store the model state, whether it's coefficients or vocabulary mappings or vector normalization values. Anything that we want to typically use in a machine learning pipeline or analytic pipeline that has states that we need to apply is stored in cells. Pools are a very similar concept, but they are actually mutable across action execution. So you can think of them as very similar to cells, but they're like a database field that can be updated in a kind of atomic fashion, and they're shared across all models and all executions of that model. So some other features of PFA are the special forms. And that is effectively making PFA act much more like a little mini-language. So you have control structure, loops and conditionals. You can create local variables and update them. You can have user defined functions that you can create your own arbitrary functions using any of the built-in functions effectively, casts, null checks, some very basic error handling, but extremely basic. And it has a comprehensive built-in library. So on the right there, you see the built-in function library.  So it has comprehensive string processing functions, much like you would find in any language, like Python or R or Java. So what is the current status of PFA? The spec is sort of at the 1.0 level or status. It's still evolving, it's still young, but it is fairly well-rounded in terms of specification. A member of the DMG who is very involved in the initial creation of PFA, called Open Data Group, has created reference implementations in the Hadrian project. So that covers Python, R and Java. So it provides export and domain-specific language in Python and R and covers scoring in Python, R and the JVM. So what does PFA do well? It really has strong support for traditional machine learning. So as you saw in the previous slide, we've got a whole bunch of built-in functions that are really suited to your traditional analytic and machine learning applications. It allows control flow, flexibility and composability using a functional approach and user defined functions. And this type system allows strong verification at runtime. But there are limitations. So at the moment, there's no built-in support for generic vectors, so mixed, dense, and sparse, which I'll mention a little bit later. No support for generic tensors or any of the typical kind of deep learning operators that one needs in that space. And there's some open questions around industry usage and adoption, although that is growing, and as well as performance. So I mentioned Spark ML in particular earlier. And some of the shortcomings and issues around exporting Spark ML pipelines for production scoring. And it was this gap and this need that led us to create the ARTFARC project. So I'm from South Africa. And ARTFARC translates to an Earth pig. So it's a wild animal, kind of like an anteater. And ARTFARC is a bit of a play on words there. So this is PFA export for Spark ML pipelines. Now the Hadrian project has a JVM engine for scoring PFA, but not much support for actually exporting models or creating PFA documents. So we created in ARTFARC a Scarlet DSL for exporting PFA documents. It uses Avra for S and JSON for S. And then ARTFARC Spark ML component for using that DSL to export models from Spark to PFA, sorry. So on the right we see a recreation of that example that we saw before for multi-class logistic regression. And that's how it looks in the Scarlet DSL. So we define cells, we define what the input and output data types are, and we define what the action is using the DSL. So it starts to look a lot more like native Scarlet code and a lot more fluid. So some of the challenges that we ran into with Spark ML in particular, but that highlights some of the issues in ARTFARC, one of them is that we, I mentioned this generic vector support. So in Spark and similarly in kind of scikit-learn using NumPy scipy arrays, you can operate on dense and sparse vectors without worrying what they are. And it'll do the right thing if you have mixed vectors in operation like dot product or addition or something like that. PFA doesn't allow you to do that. So it results in really kind of complex and quite hairy workarounds and code and doesn't fully work. So that's a shortcoming that needs to be addressed. The question of combining the components into a pipeline is quite tricky. So we had to do a bit of work around that, trying to kind of match Spark's data frame operation as much as possible in the sense that it transforms rows of data effectively in a data frame. And it typically updates or appends columns. So we try to match that by taking Avro records, which is a very, very analogous to a kind of row with fields or columns and effectively copying that record and adding a field each time. So it becomes a bit verbose, but so far it's the best way we've found to do that. And then another issue is that Spark ML, once you've trained a model, that model itself has no knowledge of schema. So it has some functions that where you can pass in the input data frame, and you get out an output data frame, you can then analyze that data frame schema to figure out what's going on, what are the input and output types that you need to work on, and what are the columns involved. But that's fairly cumbersome and adds a few extra steps to export. That's not that convenient. So it would be nice if a Spark ML model was a kind of once trained, had a set type or had a set type or a way to introspect itself and know what are the input and output types that I've been trained on and provide that as an API. So ArtVarq is open source. We released it in June. And you can go and find it at github.com. Ford slash Coday Ford slash ArtVarq. The coverage at the moment covers almost all predictors in Spark ML, many feature transformers, pipeline support, equivalence tests between Spark and PFA, and some tests for the core DSL. We obviously need more help to round it out and to look beyond just Spark and looking at scikit-learn and other light GBM, XGBoost, and other popular frameworks and libraries. But it's interesting to see that it's starting to have increased adoption in the open source community. So Salesforce released Transmogrify, which is an auto ML toolkit for Spark ML. And for their local scoring and model export, they're using ArtVarq to do export to PFA and Hadrian to score those models. So it's interesting to see other projects starting to pick this up. And if you're interested, please go and check it out and give us some feedback. So I'll very briefly mention other open standards out there. PMML I mentioned before was the precursor in many ways to PFA by the data mining group. There's a lot of support out there for frameworks, scikit-learn, R, XGBoost, Light GBM, Spark ML, but it has the shortcoming that anything that cannot be represented in PMML can only be done by custom plugins. And then you lose the benefit of standardization. There's no point, really. MLEEP is a project which was developed by Combust ML, a startup to export Spark ML pipelines to their own format for model scoring. So it has really good performance and it has really good coverage, but it's really not a standard. So it's an open source format, sorry, but it's not a standard. So it forces this tight coupling between versions of the model producer and the model consumer. So every time that you will Spark framework, version has to be upgraded. So you have to upgrade your scoring engine in MLEEP, which is not the case for something like PFA. And finally, the open neural network exchange, or ONIX, is really interesting. It's a protocol, but buffer based format for exporting neural networks and it's focused on deep learning. So it has really good support for deep learning. In many ways, similar to PFA in the sense that it serializes the machine learning model and the state as well as the functions or operators that happen on that input to create the output. But at the moment, it has very poor support for traditional machine learning. So there is some support in the standard, but there's actually no support in the actual scoring engines out there. So this is something that may change, but at the moment for traditional machine learning or let's say non-deep learning machine learning, the support is limited. So finally, a quick note about scoring performance. We compared PFA with Spark and MLEEP and we did a comparison on a test dataset of 80,000 records with string indexing, effectively categorical feature indexing on 47 columns and then assembled those vectors together with some numerical columns and did a linear regression. So the Spark time per record was actually in 1.9 seconds, which is off the chart literally, so we didn't put it on. And we can see there that MLEEP is still a lot faster than PFA, so it's about almost three times as fast, but they're both still real time, less one minute, second or less. So this highlights that there is some work to be done in the PFA, which I'll mention briefly in the next slide. So in summary, PFA provides an open standard for model deployment, for serialization of analytics workflows, true portability across languages, run times, frameworks, and decoupling the model producer from the model consumer. And as such, it solves a significant pain point in the machine learning community. Certainly starting, we started with Spark ML, but this applies to everything out there, our scikit-learn, XGBoost, IGBM and all the others, but there are still risks. PFA is a young standard, it's still sort of gaining adoption as we saw production at scale. It has not really been tested, and there's certainly some open questions around performance. Can that scoring engine be optimized for PFA? And what about deep learning? It doesn't support the deep learning operators, so there are limitations there. There's no technical reason it can't be extended to support deep learning, it's just that it's not in the spec at the moment, and it's a significant amount of work to do and it hasn't been done yet, but that is still an open question. And finally, a standard has many, many positives, but one of the negatives is that it can move really slowly and it's kind of designed by committee. So once it's released, it can be quite difficult to change it. So finally, the future directions we've done in ArtVox, our initial work focusing on Spark ML, as I mentioned, we'd like to look at scikit-learn pipelines, XGBoost, IGBM, performance testing and adding more comprehensive performance tests, proposing generic implementations to the standard, so generic vector tensor support, some improvements to performance and schema definitions. And then finally, looking at PFA for deep learning, so can the spec be extended to encompass deep learning at GPU support, deep learning operators, tensor support? What is required there and is it feasible? And can it actually become a holistic standard to cover both traditional machine learning as well as deep learning? So thanks very much, I encourage you to go and check out codea.org to see all the other projects that our team has worked on, including the model asset exchange and the fabric for deep learning, a couple of links there. Don't really have time to talk about them now, but please go and check it out. Thanks very much. I think we might have time for one or two questions or... Is how do you import actually these models to some other languages or... And I didn't see any solutions for that. The question is, we're talking about export here, model export to PFA or some other format, but what about importing models? Well, the PFA itself can be used as an interchange format. It's not really designed for that, much like PMML, ONX and PFA are all, at the moment, really focused on the export phase. So the objective is to try and bridge the gap between all the different frameworks and languages and allow you to export them all to a standard format, which allows you to use one scoring engine to score them all. So at the moment, you could, hypothetically, and I mean, it's not that challenging, but you could export a model, let's say, from Spark into PFA and then load that PFA model into, let's say, Scikit-learn. It can be used in that manner, but it's not really designed to be used in that way. It's a short answer. So, you know, for data interchange or model interchange, I think that's not an open question. Even ONX has some goals in the future to allow, you know, to have that as an intermediate representation where, let's say, you could train a model in TensorFlow, export it into ONX, and then load it into PyTorch and do transfer learning, right? That's the goal, ultimately, but that stand is not there at that point.  PFA could be used for it. There's not necessarily a technical challenge, but it's not really designed for that. It's designed for exporting from, you know, it's kind of a many-to-one relationship, exporting from many frameworks into one standard format and then loading that into a standardized scoring engine and executing inference. Anything else? Great. Thank you very much.


10 not-so-sweet facts about sugar. Number one, on average Americans consume 130 pounds of sugar every year. 130 pounds of sugar equals about 1,767,900 skittles. Number two, the average American consumes 675 grams of sugar every five days. To put that in perspective, in 1822 Americans consumed on average 45 grams every five days. Now we consume 17 times that. Number three, the only taste humans are born craving is sugar. Number four, on average people consume 500 extra calories a day from sugar, which is roughly the amount of calories needed to gain a pound a week. Number five, sugar changes the muscle protein of the heart and its pumping mechanism. Consuming too much sugar can increase the risk of heart disease. Number six, sugar and alcohol have similar toxic liver effects. Liver damage can occur even without excess calories or weight gain. Number seven, sugar is found in unlikely places such as tonic water, marinades, crackers, bread, fat-free dressing, and tomato sauce. Number eight, Americans eat 10 times more sugar than all other food additives, except for salt. Number nine, eating high sugar foods lights up the brain on an MRI in the same places as it does with cocaine or heroin. According to research by Dr. David Ludwig, director of the New Balance Foundation Obesity Prevention Center at Boston Children's Hospital.  A recent study showed that eating a diet high in fructose over a long period of time can impair your brain's ability to learn and then remember information. So there you have them, 10 not-so-sweet facts about sugar. For more information for your health, you can contact us at Professional Supplement Center. Either send us an email or give us a call.


The study found that T6L4V on UHMWP is better than SS316L on UHMWP and Co-CRM on UHMWP at reducing the amount of contact pressure during gate cycles. This suggests that T6L4V may be a good choice for hip implants due to its ability to reduce the amount of contact pressure. This article was authored by J. Jamari, Muhammad and Mom Amaralla, Gadot Santoso, and others.


Hi everybody, it's Sam at Mixed Up Graph. Thank you for watching my tutorial today. I'm going to be showing you how to make this really lovely what I'm calling a bokeh card. It's using the supplies from the May Hobby Base Kit which has a wedding theme. So I've used this lovely paper which was in the kit. These are the paper flowers which I highlighted last week in the tutorial. I finished it with this ribbon and basically you just undo this. It's using just that whole piece of 6x6 and then it opens up to reveal this little sentiment but all those flowers on the top and I just think it's really nice. I just I love it. It's very easy to make. It doesn't take too long at all. I've made the envelope as well which I'll talk through because it is a little bit of an odd size but once you know what you're doing you could easily obviously change that up but I do think it works really well with the 6x6. So once I finish doing this bow we can crack on and make it. Okay so you're going to need I've got the hobby base rose die which was what I said was in the kit. Then we've got all these paper flowers which I used in last week's project that I shared and I've done them sorry I'm just sorting those out with the gold over which was the gold color that came in the kit. This time I'm inking them up so I've done some here sorting them out these and they just take to the the inks so nicely and you can see there and I've popped the gold fasted gems in the middle of the yellow ones and this is just from my own stash just to kind of blend in but I'm kind of just working through those until I know kind of how many I need but basically what you need to do is get a piece of this is 6x6 and this was one of the sheets that came in the kit and I just thought that this worked really well as a nice kind of pattern that looks like it is kind of like you know a bouquet wrap so this is I'm going to keep this on the inside because this would be where I can write my sentiment and basically the easiest way to do to explain this is you want to loosely fold them over on top of each other so it's forming like a cone and just make sure you get a really nice point okay and then with your bone tool when you're happy your bone tool your bone folder and you're happy with where you want it I'm going to put that one over the top actually just make sure you get it right to the the point there so you get a nice edge and then bring the other one over they won't you don't want it to line up there because that's not you want it to look like a bouquet so when you buy them in the shop and they wrap them that's kind of how they will be and we'll have that ribbon around it so like that so you can see there so I've got a little bit it's not gone right up to there because that's not how I want it to look and now when it opens up we'll be able to write our sentiment in here or here so you can stamp and so on and so forth but that is the card so that's really really easy to do so now it's all down to the decoration and again these colors they just work really nicely against this so you just want to basically just go in die cut or if you have these paper ones like I have just get a variety of different sizes different shapes with the rose die here I'm going to these are the colors I die cut but I think I'm going to probably keep that that one in here maybe I'm not sure but I am going to be using I die cut the leaves again on green and I'm going to nestle these in just to create some foliage and I might raid my my own stash of like leaves and stuff just to maybe add a few of those in but I'm just going to build this up now get lots of stuff together and start putting it in place and show you what I come up with okay so I've had a play around and I changed it I've added in the gold that I had from last week I've die cut the leaves from the die in like a minty green and then I've actually added in just plain white ones because I think it just kind of just brings it down a little bit and it actually matches in better with this paper so what you want to do is when you stick them down because you want to be able to open this without it really catching on the flowers so it's you need it to kind of sit in line so if I kind of pull that away now because I'll be working on it again but where you've got your the top of your folds here imagine there's an imaginary line there or even put a faint pencil line along there and make sure you keep all your flowers stuck above the top okay you don't want to go out too far because it means you're going to have to make your envelope bigger I mean I'm going to talk you through that but you do want to kind of keep it together but I'm going to start sticking this down and I'm just going to speed up the video just so you can kind of see how I'm working on it I've put together really quite simple and may add some more as we go I'm not too sure but I just use my hot glue for the big flowers and then I just use some of my wet glue just what's on my hand just to pop in the others there and then when you open it up I've got a few of the leaves coming down here as well into this area now this one here I've got I want to use I might put it there I don't know what my sentiment will be yet or it could look nice with the ribbon kind of bringing in the tie around that bit so I'm not entirely sure okay so I've just got this bit of ribbon here which matches really well and I'm just going to before I stick it down just need to decide whether I think I'm going to have it tying slightly off to the right hand side here so let's just roughly see how this is going to look so I want to stick it on the back so that the recipient does actually have to undo the bow I just think that's a nice little feature okay so it's going to be roughly like that don't worry if it slides off because obviously you're just going to push it up but now all I need to do I think that looks really really pretty is just pop some double-sided tape inside this piece here so let me grab my I'm just going to put a tiny little strip of this red tape just inside the ribbon literally just a little bit because it's so light you don't need a lot so and then just peel the backing off the next side and then just get into stuffing it pop it back in but this time you know lie it against the bows because you don't want it to stick too soon and then once you're happy that it's nice and lined up keeping it all nice and straight and just stick that down there we go so I just need to clean up the the edges there but basically now when they receive it they will then open up that it doesn't fall off and it will reveal your message and the sentiment and everything inside so I think this would look lovely as a nice way to do an invite as well so if you're having a party a very girly party you could or if you've got the patience to do your wedding invites or something like that you could you could sit down and put these together the actual card is easy you saw how quickly you just got to make a cone shape and if you die cut a load of flowers they actually aren't that time consuming to do so again they're just like I said tidy up the ends of the ribbon but I quite like that now possibly that could go above maybe no I think I'm going to stick with that inside okay so next now we need to make our envelope okay so to make the envelope there are two ways so if you are following exactly what I've done here um and you haven't gone too far over the sides here the width of my card ends up being four inches okay and then the length is nine and that's what you want to kind of stick to all I've done is I've got a piece of nine by twelve sit the envelope here and you can see and I've given myself a bit of room either side so along the nine inch side I've scored at four and a quarter and eight and a half giving me a half an inch tab so that means it will wrap around um and still have a little bit of room but if yours is a bit bigger then you would cut this to maybe ten inches and you could score at four and a half and then at nine and then have you know or nine and a half inches for example you know what I mean you can extend this quite easily and again along this length I've scored at three quarters of an inch and at ten inches and that's then the card sits in perfectly it's given me a two inch flap to flap over the top and then a three quarter inch bottom but again if yours is longer you could easily just have a one inch um flap or even a half an inch if you've gone really tall so by doing it this way it's very easy to adapt it to fit what you're going to do but like I said I scored here at three quarters of an inch and at ten inches so take away the scoreboard and we just need to burnish all of these score lines okay and then with the half inch tab on your left hand side here we're going to remove this piece along the top so just some scissors and I'm just cutting so I'm removing the score line all the way along to that score line there so you've got to pass that score line to the second score line and then cut down okay so I've just removed all of that then again tabs on the left hand side down the bottom here you will have this very small little rectangle in the corner just remove that and then directly below here so this piece here we're going to take that out so just cut up like so and then remove that whole piece okay then I just want to tidy up my corners here so I've just got my corner punch and I'm just going to round those off and then what you want to do is flip it over so this is now on the left hand side and you'll have your two tabs here and you just want to pop some tape on the tops of those so this is one and half an inch tape so it fits perfectly along that tab and just take a little wedge off of them as well just so you kind of you won't have any of this overhanging take it off the bottom one there as well and again okay so that's what you should have and then you want to remove the backing from both of these pieces because we're going to stick it all down together flip it over fold that one in and fold that one up as you fold that one up if you sit that one over the top it will stick to it already like so so it's already creating a little pocket there and then bring this over and if you start sticking it on the end here first and then you can just work along and get the rest all stuck down try not to stick your fingers onto it there we go like so and then that will fold over and then that will sit nicely inside on the bottom so now you can just run some double-sided tape on there and stick it over write the address on it on this side is you can post that it would obviously cost a little bit more but that's ready to go but just before we finish I want to add a little sentiment inside now we did get happily ever after in the kit which I used last week but I just I don't know I'm going to basically I've just got this little celebrate one and I'm just using some of my VersaFine and I'm going to ink it up now if you want to do this first or put it in your stamp positioner then do so but I'm just going to go for it so fingers crossed this is going to work but I'm putting it in the center that ribbons a bit bulky but it should still work just going to rock it slightly it's okay right quickly remove the VersaFine because this stuff is lethal I've dropped it on projects before and there we go so I've just put little celebrates now I can write to and from I can put some stuff on here as well if I want to and that has dried already so that can all be tied back up okay and there you have it I think this is a gorgeous card I think it's really pretty it's really different and you can really kind of customize it and you know make it match a theme or anything really but I think they're gorgeous so I hope you liked it please give me a thumbs up if you did and as always if you're interested in joining the HobbyBase Club all of the links are shared in the video description below thanks for watching bye


All history is a collection of stories or what we call narratives But in history not all stories get an equal place Some get promoted others get pushed on the side Because of people in power who can decide what goes out stories of victims Like a blank slide often go unrecognized and unacknowledged My research is about a branch of victim narratives Stories emerging from a revolution that took place in Sri Lanka when I was just three years old In July 1987 a political movement in Sri Lanka the people's liberation front Rows up in arms against the state in the conflict that ensued over 60,000 people lost their lives a vast majority of these killings were carried out by the state military who enforced abductions Incarcerations torture rape public display of bodies enforced disappearances The sons and the daughters of some of these victims Were my friends people known to me The trauma of losing a near and dear one haunts them to this very day When the conflict was over its history was written by powerful people in places of privilege The victims they had a story, but that story was pushed into the margins. It was suppressed Now my research is all about getting that story back into the center and Setting it side by side with what the dominant authors claim is history I'm a student of literature Using conceptual tools that are available in literary theory. I Analyze the memoirs of writers writers who have gone through the camps who have survived with Mental and physical scars to bear witness to what exactly took place behind closed doors Analyzing the writing I look at how they challenge and dismiss Claims made by narratives of people in power My research is all about giving a voice to a stolen and murdered generation of people Based on my findings. I propose to write a biography a biography that encapsulates that victimization one that will put long overdue color and detail on What is otherwise a historical blank slide?


Welcome to eslgoal.com where you will find quick answers to your language questions. Ask us anything about learning English and we'll be happy to help you. Question of the day. Today's question is probably the number one question we get. How can I improve my English speaking skills? So the title for today's video is How to Improve Your English Speaking Skills. Are you all ready to speak? Wait a minute. I don't know what to talk about. I don't know what to say. I don't have anyone to talk to. What am I going to do? Well, in order to speak a language, you will need three things. Number one, something to talk about, a topic. What? A way to communicate. Language. How? Someone to talk to. A partner or partners? Who? eslgoal.com and our related sites are here to help you with these things. Let's look at each item individually. Number one, something to talk about. Where can you find topics to discuss? Look around you. There are plenty of interesting things to talk about. Let us help. ESL Gold has hundreds of topics for you to choose from, depending on your level. And we provide you with vocabulary, grammar, questions, pictures, and conversation strategies to get you started. Are you new to English? Check out our beginning topics. Do you have some experience in the language already? Look at our intermediate topics. You want to challenge? Try our advanced topics. Let's jump right in and see what we can find. We'll start by going to the home page. And there's a link to that right up here in the right hand corner. So eslgoal.com has what you're looking for. We have videos, free lessons, web pages, just about everything you need. I would suggest waiting until the end of the video and then going back to click on the individual links. Then you can keep up with us as we go through this site. All right. The first place to start is at eslgoal.com. Go to the top center and it says speaking. This is our number one section in the site. And then go to the pull down menu where it says conversation topics. And here is where you'll find all kinds of topics for you to talk about. All right. Let's look at the list here and it says easy topics. We'll start with those because they're for beginners. All right. So click on easy topics. And you'll come to a page like this. It says here are some easy topics to talk about. Now we have them numbered. The 100 topics are the easiest ones, then 200, then 300, 400, and so on. So let's look at one of these vocabulary topics for today. It's topic number 124, transportation. Again, there's a link up in the right hand corner if you want to come back and go to that page directly. But let's do it right now. Click on that and we'll see a sample lesson. Now you've heard the expression stop, look, and listen. At eslgoal we're going to say start, look, and listen. So we'll start with what you see and we'll help you with vocabulary and questions and things visually to help you get started. OK, so it says look at the picture, read the questions, and think about the topic. So this topic for today is talking about transportation. So we'll start with some pictures and it says look at the pictures here. And if you click on the link, you'll see pictures of things like airplanes, cars, bicycles, buses, et cetera. And you'll see the vocabulary over there on the right. But if you click on the vocabulary where it says look on the pictures, then you will see pictures and you will hear the pronunciation of the words like boat or motorcycle. And you can repeat those for pronunciation. So you see them and you hear the words being spoken by a native speaker. Then we're going to use or practice the language. And down at the bottom of the page they'll have additional questions for you to talk about transportation. Here are some sample questions for you. How do you get to work? I usually drive my car. How long does it take? It takes a half hour. How often do you ride the bus? Once in a while. Do you ever walk to work? No, that would take forever. So as you continue on with this lesson, you can learn new words and put those words in context where you can practice them with a partner. Let's go on to the intermediate level topics. These are a little bit higher for people who have some experience with the language already. Again, we're at eslgold.com and we're going to the center where it says speaking and going down to the conversation topics. Now instead of easy topics here, we're going to go to something a little bit more difficult. Now if you look down here it says conversation prompts. A prompt is something that helps you get started in your conversation. So let's click on that. And you'll see a page like this. So it has several different types of things you can talk about. We have narration, opinion, description, hypothetical situations, situations for conversation, giving instructions in English, things like that. Let's click on the opinion section. And we'll come to something like this. Again, we're using the method start, look, and listen. So we're going to start with a video which will enable you to see and hear the language. So it will give you phrases and pronunciation for asking for and giving opinions in English. And this will take you right to the YouTube video. Then after you've watched the video and you've listened to the way that you can ask for and give opinions in English, then we'll let you put it in context. We'll give you some situations in which you'll have to give your opinion like what you learned in the video. That's the intermediate level. There are lots of topics like that with video to help you get started and then situations in which you can practice the conversation you learned. For more topics, go back to the homepage and scroll down to where it says Free English Lessons. Now these are organized according to level. So let's click on that and you will see beginning, intermediate, and advanced topics. What about advanced topics? Well, you'll get a list of topics here and these are usually articles or videos that help you get started thinking about topics so you can talk about them in English with your partner or teacher or tutor or whoever you want to talk to. All right, let's go to this article here that says Things to Quit Doing. And when we click on this, this is a sample advanced lesson. Again, we'll start with something that you can see. So you look at the pictures, read the question, think about the topic. Now this is in preparation for you to read an article. And the article is called Six Things to Quit Doing in order to be successful. And we'll have a link to the article so you can read it and then you can discuss it. Now we also have vocabulary preview. You can see that in the middle and you can also see that on the right-hand side where it has the words and you can practice these for pronunciation, avoidance, burnout, constraints, criteria, et cetera. And down at the bottom of this section, you'll have some of the expressions they use in the article as it turns out, falls into the camp, free up, get stuck, play a role in something, plugging away at. So these are expressions that you will see and read about in the article for today's topic. If you go down to the bottom of the page, you keep scrolling down, you'll see some questions that ask you to discuss some of the things that you learned from the article. And if you've got partners or classmates or tutors to talk to, this is an excellent way to get some practice using the vocabulary that you've learned from the lesson. Again, if you want more of these, go to the section called Free English Lessons. There's a link from the home page and here's a direct link on this page as well. And it starts with basic, then intermediate, then as you scroll down, you can get to the more advanced lessons. Okay, now that you have something to talk about, the next thing you need is a way to talk. This is where language comes in. You will need expressions, grammar, vocabulary, and pronunciation. ESLGold.com can provide these things as well. Let's take a tour of these pages. Probably the most popular section of our site is called Phrases for Conversation. Here you will find conversation strategies including text, audio, and video. For example, want to know how to make suggestions in English? Well, you will go to the section with expressions called Phrases for Conversation. It's part of the speaking section. If you go to the middle, click on speaking and you'll see a pull-down menu that says Phrases for Conversation. It's organized by difficulty. So it starts with low beginning, then goes to high beginning, intermediate, and advanced. So let's click on the topic that says Making Suggestions. And here's what you'll see. When you click on that page, you'll see at the top of the page of video. And it says, what do you say when you want to make suggestions in English? And this video will give you several tips and phrases to help you get started. So click on the video. It's a very short video and there's a direct link there. Then after that, you can practice some of these phrases on your own. So look down at the bottom and it says, Repeat for Pronunciation. So we have a lot of phrases there. And all you have to do is click and you'll hear a native speaker say the phrase and you can repeat and practice your pronunciation. But that's not all. You need more than just pronunciation practice. We will give you situations for conversation. In other words, we'll give you context in which you can use these phrases for making suggestions. And then you can talk with a partner or a tutor about these kind of things. What about grammar? Well, what about it? We provide explanations, examples, and exercises for you to get the knowledge and practice you need. Web pages come with text, audio, and video. Check them out. Where do you find grammar? If you go to the top of the website, you will see the second tab just to the right of speaking. This is our second most popular section of the website, grammar. And you click on that and you'll see a pull-down menu that says Explanations, Examples, and Exercises. Now this has them organized according to level. So if you're just starting out, you might want to look at some of the easier grammar topics. So look at the list and click on the things that you would like some help with. Now a lot of people say, well, I don't know what grammar is all about. I don't know what these concepts are called. I like to give you an analogy of fixing your car. If you want to fix your car, you kind of need to know what the parts of your engines are and what you call them and how they work. So grammar is kind of like that. If you know what the part of language is that you need to fix, and you know how it functions, and you can label it, then you can find the information you need to fix your grammar. All right, these are organized by level, but we also have a way that you can just go directly to the grammar point. In most cases, we'll have a direct link from the speaking page that you're on. But if you want to learn grammar and practice grammar on its own, then you can go to our sister site, which is freeenglishstudy.com, and we have grammar all outlined for you. Some people like to do it that way. And just choose the topic. These are organized according to topic, not according to level. And again, you will see videos, explanation, examples, and exercises. So on freeenglishstudy.com, you'll see this menu of grammar topics. So you can find exactly what you're looking for. Here's a sample. This is one of our most popular pages. We have a video, and then we have text. This is on freeenglishstudy.com. This one's called irregular verbs in English, finally explained. And what it does is it explains the irregular verbs in English and puts them together in categories. For example, you'll see categories like sing, sing, sung, ring, ring, rung, and we'll put them together so you can see the relationship and the patterns among these words. But we also have an alphabetical list if you prefer to learn it that way. Now let's go on to vocabulary words and phrases. We'll start with the lower level. For the lower levels, we have what we call picture dictionary. And this is a list of items with a picture and sound or pronunciation so you can practice and repeat the words after seeing the picture. Now for this, you go to the low beginning or the high beginning level. And here's what you will see. Here's a sample picture page. This again is in vocabulary. It's in the third section from the left, vocabulary, and go to words and phrases. The pages include pictures, audio, pronunciation, and quizzes. So for example, this sample page is about body parts.  And if you click on the link next to the picture, they will say it for you so you can listen to it as many times as you want. A neighbor speaker will pronounce the word for you. And then you can repeat it for pronunciation. Now in addition to this, we have quizzes. So on these pages, you click on the quiz section and it will give you a quiz by showing you a picture and then you have to say what that picture is of in English. This is so much better than translating it from your language.  And you remember it a lot better that way. Let's go on to the intermediate level for vocabulary. At the intermediate level, we focus on word parts and vocabulary in context. So we'll give you a short reading article and let you see these words being used in a natural context or situation. We also explain some of the common mistakes people make at this level. So at the intermediate level, you can get more practice with vocabulary as well. Finally, at the advanced level, we have academic vocabulary lists and quizzes. So we have a lot of matching exercises which let you look for the definition and put the words together with them. We also have these videos which will test you on your vocabulary. Now we're talking about general definitions here and you'll see the list at the bottom. Of course, there are more than one definition for a lot of these words. So we'll start with the general definition and you can look at other places on the website and through the dictionary to find out more definitions for these words, but these are just general meanings. What about pronunciation? Well, we have different pages designed for pronunciation. We have most of the difficult sounds in English outlined for you, the L sound, the P and F sound, the R sound, the V sound, the TH sounds. Did you know that there are two TH sounds, the voiced TH, the and the voiceless TH, the. Then we have other sounds as well. Then we have pages for you to practice these and you can read and listen and practice these words in. Phrases, sentences and oral paragraphs. Now in addition to that we have something unique on eslgold.com and it's called speech patterns and we have special videos for fluency and helping you talk faster in English.  We have different kinds of reductions or ways to speak English faster here. Another video that's quite popular is did ya, did you do your homework and it shows you how to reduce certain words in English in order to speak faster. Check out these videos, they're in the pronunciation section. Finally, number three, you need someone to talk to. This is something you may have to find on your own for now but we will have some things like this provided for you in the future. You need to talk to someone, you're not just going to talk to yourself or a textbook. So speak to a teacher, speak to a tutor or a classmate or a friend or a family member. A lot of people say well what if they're not a native speaker of English. That's okay because you're getting practice by using the language yourself and if they're patient with you they can help you and maybe you can help them as well. For the future we're going to bring together some of the best teachers, classes and tutors in fact we're doing that right now in programs to help you learn English. Wait and see what we have to offer. If you want some help in this area contact us at the link below for more information. Wow, that was a lot of information about learning English. To review, in order to improve your speaking skills you will need what? A topic. How? Language. Who? Partners. ESOgoal.com can provide many of these things. Jump right in and get started. Please refer to the links on the website and in the comments section below will provide all the links for you. If you want to go back to sections of the video and click on the link directly you can do that or you can look at the links in the comments section below. If that was helpful for more information please subscribe to our channel and our website. It's all free. And please give us a thumbs up in the comments section below. Thanks for watching. If you have any questions or would like us to make a video on a particular topic please let us know at the link below ESOgoal.com-submit-question Well what are you waiting for? Jump right in. Go back and look at the sections of this video and start improving your English today.


and welcome back to Scrum Mastery Challenge. This is episode five of the Agile Coach game show. And so far, we've had a new leader at the end of every episode. And as you can see from the leaderboard here, Paul is currently sitting at the top, albeit by just half a point.  This time I gave them 20 pounds each and they had to spend it with one condition, that whatever they bought would significantly increase the agility of a team. Let's start by seeing how Paul went about this challenge. It was tough. It's tougher than I thought, actually. So I considered stationary. So I thought I could just buy some interesting post-its and cork boards. In my opinion, it has an immediate effect on the agility of a team if you give them something more physical, the task boards or that to actually use. But I thought everyone would be doing that. I wanted to do something different. So then I went to a toy shop. I looked at some kind of desktop games for teams to play, to relax their, then mind a bit and look to concentrate on something different. So I looked at there's some air hockey games and there was a speaking Simon with the colors. If they pick the colors and memorize the color of a memory game, I thought I liked that idea. Bear with me. This is called Creepy Hand. Right. This is a game. And what it is, is it, well, I'll show you what it is. Look at here. Ready? Ah! And it points. It does. So I thought that was, I looked at that, I thought, I can see how you could apply that. Okay, I'm listening. Okay. I thought, so if you want a fun way to start a daily stand-up, that's an interesting way. Put that on the table, get people standing up and they have people, it selects the first person at random, it's a bit of fun. And then it comes with some cards as well.  So the first one here says, the question is, when did your last go to bed without cleaning your teeth? Yeah.  And the forfeit is to mime a nursery rhyme. The cards, I thought were just as interesting as the hand, because that made me think of some kind of, either a nice way to start a daily stand-up was something a bit more fun. And you could change, obviously, you could write your own truth, dare and forfeit cards. So that's where I spent my money, Jeff. All right. Well, there you go. A little creepy perhaps, but creative. That's what Paul was going for. So his aim there really was to break the monotony of something that could potentially become a little bit boring and stale for a team and spice it up a little bit. And the cards that went along with it, I think were even more useful. Helping teams get to know each other a little bit more, which for me is a big contributor to real successful, honest, open teamwork. And open collaborative teamwork is the basis of agility. So yeah, pretty good for me, but again, I'm not going to be the one judging it. I wonder whether our judges will see the same things. Next, let's see what Helen spent her 20 pounds on. It baffled me a bit at first, but then as I sat down and thought about it a little bit more, I thought it was quite obvious and pretty much the things that I bought with me came to my mind straight away. So the first thing that I looked at was bearing a minor 20 pound budget. The first thing I got was straight onto a well-known website and I looked at cork boards. I also looked at white boards, but I realized quite quickly that A, they're expensive and they are kind of like still quite small. You'd given us a 20 pound budget, but actually you shouldn't really need to spend any money if you want to enhance agility within your teams. So I didn't actually spend any money. I had all of these things at home already. And I'd like to think that teams in their general office would have these things. So my first item, masking tape. So with masking tape, I believe that you can create any visualization you want, put it onto a wall, and teams will be able to put their work up there. My second object, post-it notes. So once you've got your board up and running, you can map out what your work is. And to do that, you're gonna need my third object, which is a Sharpie. I've actually written something on the top of my first post-it note. This is one of my favorite websites, and that's RetroMap. And that's some place that I always recommend to Scrum Masters or anyone, if they want to get good inspiration on how to run fun and creative retros, they can just go to this website. So visualization is absolutely key in ensuring that the teams understand what's going on. It also shows you lots of patterns. It shows you lots of dysfunctions. And it shows you where the things are that you need to fix first. Visualize the work, and then you can see the problems from there. So Helen nailed her colors to the master visualization there, trying to keep things simple, just like she did in the food challenge last time. It served her well then, so why not now? Every agile team will get some benefit from being able to visualize what they're doing. And with tape, post-it notes and Sharpies, arguably the staple diet of any agile team, that's what they'll be able to do. So no matter where they are, visualize the progress, be able to make some changes, make some improvements, become more agile. I can see the thinking. Will our judges see it that way though? Can't wait to see what crazy idea Sam's got for how to spend his 20 pounds. It's quite difficult to tell, because I think improving the agility of a team is actually quite contextual. So what would be good for one team, might not, you know, what solves one team's problem might not even be a problem for the other team. So kind of picking warm thing is quite difficult. So initially I actually thought, well, what books I think actually embody the ideas, high level principles or kind of philosophies about how to function well. And there were three books, Who Moved My Cheese, which is obviously a great book about how to deal with change positively, right? And that's a fantastic book. The second one would be Zen and the Art of Multi-Cycle Maintenance. I've read that book about 10 times and every time I've read it, I've been able to apply it to different circumstances. And then the final one for, to be a good team, and if you want an example of how to function well in a team, I don't think you can do better than Chris Hadfield's and Astronaut's guide to life on Earth, is pretty much a de facto guide on how the right way, the right mindset to be effective in a team. Yeah, the other thing I considered is I'm a big fan of oblique problem solving and to do that, you need a space away from your working space. And I found sort of that water cooler environment. So I actually found a mini water cooler. That was only like 10 pounds. So what I realized is I could create an entire space for this. So I thought I'd then get like a little whiteboard for five pounds. Just so that any ideas, any ideas that come from these oblique conversations can get kind of recorded just to make sure they don't get forgotten. I could also then afford a packet of Jaffa cakes. And the final thing was a poster of logical fallacies. Just so you can, when you have an idea, you can just make sure you're not committing the logical, the sin of, you know, thinking about it rationally. So, and then finally, Jaffa, I got what I got. Okay. Right. Which is, and I've decided it's not my job to pick what the team should buy to improve their agility. It's not my job to do that. So I've got 20 pound coins. I'll tell you why I've got 20 pound coins. So first of all, they can take this and buy whatever they want, right? The buy whatever they want to improve their agility with that remit, which means they can fail and learn from it. Or they can buy something that will allow them to succeed. And then finally, before they buy it, I think they could do the penny game just to show the value of small batch sizes. And additionally, because I don't have any receipts to give you, you've not got any comprehensive documentation you have to deal with. So I think we find that is perfect. Self-compartment, ladies and gentlemen. Trust Sam. Well, that could be seen as ingenious or I suppose it could be seen as lazy. He knows, and you probably know as well that I'm a big fan of asking the team. But I wonder how our judges will view that. We'll find out soon. But what we can't argue about really is his argument about the contextual nature of this. What one team would see as an improvement in their journey towards agility, another team might not even have us a problem. So addressing each situation on its merits, I think, can't be argued with. Next up, it's Christina. I wonder how she has spent her 20 pounds. Let's find out. I found it quite hard because for me, the word significantly is a pretty important word, but it means different things to different people. So yeah, I found it quite tough. One of the things that I've realized is food is usually a really good way to get teams to just talk, collaborate, just have a conversation and just breaks the ice every now and then. But I couldn't bring a box of donuts. So I thought they might go off by the time I came here. So yeah, but food was my first thought. So the first thing I thought about buying was, it's like these little booklets that say funky things on it like I love lists or yay. You know, they learn something new and they want to write it down. Often they get to write it on the poster and I thought, oh, buying each of them maybe like a book that they can have, they can write something down and could be kind of cool. But I thought that's one more thing they have to write. And then, you know, when you go to construction sites and you've got signs which say do not enter or men at work, there was a really cool one which basically said, do not disturb. And I thought that could be kind of cool because when the teams really trying to get their head in the game and you know, they're really sort of up against it.  But then I thought that would that really help? And I considered buying a box of story cubes because I really think they are a fun way to just engage the team, especially when it comes to retrospectives. But that was too predictable. So I spent my money on a small little trivia box which is the ultimate science quiz. Okay. Really random. So one of the things that I've come to learn is getting the team to step out of their current environment really helps them just think differently. And usually when I know I've got writer's block or I can't think of an answer to a question, doing something that's completely random and not connected to my work really helps me suddenly just go, ah, there's the answer or just gets my brain, the juices in my brain flowing. So a team can use them in many different ways. I like to kind of leave things on the table because I think people's curiosity gets the better of them. But we could potentially use this in a retro literally do nothing that's related to software development or talking about the sprint, just using it for the team to just have complete and utter downtime. Well, a wide range of ideas there from food to signage to notebooks to prompts for storytelling. Just like Paul, she was a little bit worried about being predictable. So went for something abstract to try and stimulate some creative thought process within the team and break them out of their paradigm. Quite random, I suppose. But an interesting perspective. Wonder whether her gamble will pay off with the judges. Finally, we have Freya. I wonder what a teenager who's never actually worked in an office before will think of to make a team more agile. Fun. Yeah. I went for lots of little things instead of one big. I got this big pink pig. Big pink pig. Because for lots of reasons, actually, because if I walked into work and I saw a big pink pig, I'm going to be happier and think this is going to be a fun day.  And when the team has an idea, that anyone on the team has an idea, they can use my giant pencil to write their idea on a post-it note.  And if they put an idea in the paper, they can have a chocolate bite at my door thing. And then there's a calendar so that they can write what needs doing and how long it's going to take for each thing that they need to get done. Post-it note, because it was a bit simple and you can put your ideas in Mr. Pig, however you fit best. And I was going to buy a sort of a statue of gold trophy so that whoever ideas get used can have the trophy. But then the point of having the pig is that you don't know whose ideas, so when they get discussed with the group, there's no name and shame if it's not a great idea so that there wasn't favourites when I didn't get that prize. Oh, focusing on making the boring fun, increasing participation and engagement, rewarding ideas, more than productivity. This, coupled with a calendar to help teams cute themselves accountable, is an interesting way of spending the money. I think I'd like to see a big pink pig when I came into work as well. Well, all I can say is I'm glad I'm not judging these. We've got a spread of judges from around the world again and they didn't know anything about this challenge. They certainly didn't know who was involved or who bought what. So let's see what they make of what our contestants chose to spend their 20 pounds on. The entry that I picked was the 20-point coins and then let the team decide because I think it really helps the team to take the ownership of what they're doing. So by giving them the money, they're gonna know best what they want to do to help improve their agility. It depends on the challenges that they're having at that time, what's the right thing for them to be using. The second option in that, I chose the piggy bank. Giving people just that space to go, okay, if you come up with an idea, don't hold it off or don't try to remember just until the retrospective. Here somewhere that you can write it down and capture it. I really do like the idea of the piggy bank and the chocolates just to reward people because that's a nice thing to do. The third one that I picked was the brainiac game. I like things that sort of put the team outside of their normal day-to-day. So sparking the creativity, you know, I'm sure could lead you up out of camaraderie. So that would be my top three choices in this. Hi everyone, this is Jo from South Africa and I'm here to judge the Agile Purchase Challenge. In first place, I have the 20 pound coins. I thought it was a really cool idea, take it to the team, let the team decide what is gonna help them to be more agile or increase their agility. Number two for me was the creepy hand. I have come to realize that the quality of the interactions between the team members is directly proportional to the quality of whatever the team outputs. And so figuring out how to learn to be a little bit vulnerable with each other will significantly increase the level of interaction and the ability to conflict in a constructive way. The green tape and the sticky notes got my third vote. I thought that this would be a totally awesome way to help a team be more agile if they didn't have a stand up order, if they weren't doing anything of that sort already. My fourth vote went to the Brainiac card.  Thanks very much to everybody and hope you have a great day. So I've been asked to judge and rate the entries for the Agile Purchase Challenge. My number five in this round is the creepy hand because let's face it, the hand is creepy. I'm not sure how that would help with Agile. Number four would be the green tape and post-its. I'm hoping that most every Agile team already has this. That's like giving a team air. My number three is the 20 pounds of coin. Although this is unique and it lets them self-organize, it seems like this team kind of phoned it in and said, we don't have an original idea, let the team decide. So this would be my number three, okay? My number two would be the game Brainiac trivia. I love the idea of inspiring ideas and letting people kind of learn more about each other. So it seems like this might be a good choice in there. Which leaves my number one, which would be the Ideabank. So not only is this helping teams to think about what the next idea might be, but it encourages them to do that and track that. So it would be a really, really nice thing to have. Hello, this is Adam Weisbart, an Agile coach base just off of Seattle, coming in at number five. I am going to say that giving your team 20 pounds and letting them spend it as they will might seem Agile, but also seems to me to be frankly a bit lazy. So I'm going to give that the lowest ranking of five. For number four, I am going to go with tape, post-its and a Sharpie. Well, those are definitely things all good Agilists should have. I also think all good Agilists should have them. Coming in at number three is the Ideabank. This one, I liked well enough. I think the reason it got a bronze in this competition is that I did quite like that the team was thinking about how to get more shy people to participate with the team. Coming in for the silver, I am going to go with the creepy hand. The creepy hand game, while I thought was something different and useful, it was also creepy and it scared me. Coming in with the gold is the Brainiac trivia. I loved how much thought was given to this and how it could be used in many different ways, either over a cup of tea in a retrospective, used to start a daily scrum. I thought scattering them about for the team was a fun way to get interaction with the team in a way that wasn't specifically about work but could certainly help you in your work and that it would break you out of normal patterns. So for the gold, Brainiac trivia. Hi, this is Karen Greaves here. When I first saw the question, I thought, oh, what would I do? And I decided that what I would probably pick is some kind of game to help people kind of build relationships and kind of connect. And so I was really excited to see that there were two games that were chosen. The one was Brainiac, which I really liked because it kind of allows creative thinking and it can just kind of spark ideas and leaving cards like that around might mean that the team picked them up whenever they kind of need them. The creepy hand truth or dare game, I actually don't like so much because it feels a little bit like it, it forces people to speak and truth or dare might be a little bit risky for some people. So yeah, so my picks are first pick, first place would be for Brainiac. Second, I think it would be the stationary, especially because it's got a link to RetroMath and I think all good teams start with having really good retrospectives and I highly recommend RetroMath. Third, I think I really like the idea of letting the team decide what they would do with the money. That's kind of an agile principle. So letting the team make their own choice, I thought that was cool. Fourth place that gives the idea, Piggy Bank, I quite like that as an idea, but I didn't really like the chocolate as an incentive. And finally, as I mentioned, creepy hand is just a little too creepy for the workplace. So that is fifth place for me. A huge thank you there to Ashlyn, to Joe, to Howard, to Adam and to Karen, especially as they didn't have a lot of context for what they were judging. Those views of our judges have now been translated into points and as you can see, our judges didn't really appreciate the simplicity of Helen's idea and I guess Paul's hand was just a little too creepy. But Christina wins her second challenge of the series. And that means we have another change at the top of the leaderboard with Christina and Sam sharing the lead. We're now over halfway through scrum mastery challenge and we're in a situation where all of our contestants are still in touch and the good or a bad week for any of them can change everything. Well, I'm afraid that brings us to the end of another episode. I hope you enjoy it and I'm sure you'll appreciate all the suggestions that our contestants came up with there from toys, to books, to games, to even asking the team, lots of ways that you could potentially increase the agility of your team. If you've got any ideas, some cheap, easy ways of increasing the agility of your team, please share them in the comments. I hope to see you next time on scrum mastery challenge for episode six.