Hello, and thanks, everyone. Thanks for being here. And for those of you who are watching live, good morning, good evening. Good afternoon, depending on which part of the globe you are in. We are at the open source summit, North America here in Seattle, and today we are going to talk about machine learning pipelines and lineage, right? And specifically on top of Kubernetes, right? We all know Kubernetes as the container orchestration platform, but off late, you know, there has been a lot of great work happening in terms of establishing machine learning and ML Ops platform on top of Kubernetes, and this talk is going to go into some of the aspects of it. Myself is animationing. I'm the CTO for Watson AI and Data Open Technology, and leading a lot of engagements in open source from IBM. With me I have my colleague Tommy Lee. Tommy, you want to introduce yourself? Sure. Yes. Tommy is part of the team and leading a lot of the initiatives here, so let's move on. Now, one of the things which we are noticing definitely is, you know, enterprises are still struggling to scale AI beyond experimentation. I mean, a lot of times the news come that, you know, I've developed my model. It has more than 11 months, the model is still not deployed in production. Even if it is deployed, it's hard to measure the business impact of those deployed models. In some cases, right, there is not trust. The organizations or the end users who are supposed to take action on these deployed models, they don't have trust in these deployed models. So now, one of the things is like, you know, the best practices for building accurate models are well understood, like how do you go through different parts of the life cycle in terms of producing these models. What is not is essentially, you know, how to create these production data science platforms at scale. Right, you need a holistic architecture. You need model management. You need integration with the rest of the enterprise silo systems. You need logging tracing. Then, you know, you need more effective engineering, which means, you know, a lot of automation, a lot of standards and a lot of pipelines, right? Unless you standardize and automate this whole field, you're not going to be able to scale to the extent you desire. And last but not the least, you need smooth operations, right? It's one thing to deploy. It's one thing to monitor and manage. The day two, which is much harder, right? So you need a lot of monitoring around models, around your maintenance strategy, et cetera. So I think we understand clearly that, you know, the three pillars of AI life cycle, right? We have data sets. You use data to build models, which then automate decisions, right? So data and models are the two pillars. And when we look at what is the third one, that essentially is the pipelines, right? You cannot do this breadth-oriented, very manual intensive procedure unless you automate and the way to automate is using pipelines, right? Starting from data cleansing, feature engineering, data ingestion, all the different fields you go through. And then when you come to the machine learning part of it, when you're running distributed training, you're running hyperparameter optimization, then you are running validation on top of your trained models, finally deploying them. But then, you know, your journey is not stopping there while deploying, right? You need to be monitoring for drift and normally other things, right? So all these things need to be automated. And the way to automate this end-to-end life cycle is through pipelines.  So for that, one of the projects which became very popular in the industry, Open Source Project is called Q-Flow Pipelines. Essentially, you know, it's, think of it as a CI CD pipeline as a very basic level, something which can run on Kubernetes.  At the basic level, that's what it is, but then you start working towards, you know, machine learning capabilities. What are the machine learning capabilities in it? So essentially, you know, there is a Python DSL so that data scientists can actually program this whole pipeline end-to-end using Python directly. No need to deal with Kubernetes, YAML files, dog and containers. If you just want to use Python, program pipelines, launch it, use it, right? The other part is, you know, there is a lot of capabilities built around lineage and metadata which I will, you know, go into some of these details which are very, very important in the machine learning world. So for example, in this case, like when you see this definition here, this is like a 15 lines of Python code. What it is doing is, you know, it's doing TensorFlow data validation, doing preprocessing, then, you know, training, running distributed training, using TensorFlow under the covers. Then finally, you know, once your model is produced after training, it's running model analysis and then deploying your model in production. All end-to-end data and machine learning lifecycle done through these 15 lines of code.  That's the power which Kubeflow pipeline brings using the Python SDK and makes it very friendly for data scientists. Now, when we decided to productise this within Watson and within IBM, one of the things which we did, so Kubeflow pipeline by default, the architecture essentially is that, you know, it uses Argo, which is a CNCF project, right, as the underlying container orchestration CICD engine. We replaced that whole thing and re-rode the Kubeflow pipelines to actually run on top of Tecton. Now, Tecton is another project. It's part of the CD Foundation under Linux Foundation. Initially started by Google, IBM and Red Hat are prolific contributors and we have standardised our whole cloud pipeline story around Tecton. That means Red Hat OpenShift pipelines, they are based on Tecton. IBM cloud pipelines, they are based on Tecton, so it made all the sense for us to build our machine learning and AI strategy and the pipelines on top of, you know, Tecton. So we leveraged, you know, the Tecton substrate under the covers and re-rode Kubeflow pipelines to run on top of it. And now, you know, you get all the machine learning capabilities from Kubeflow but running on top of Tecton, right.  I wouldn't go into the details of it, but it suffices to say think of it like, you know, task is your unit of work and task is essentially made up of different steps, right. And each task is essentially, you know, running in its own pod and the combination of all these makes up a pipeline, right. And that's how very modally architected Tecton is. So now let's talk a bit about machine learning capabilities, right. What are the machine learning capabilities beyond the Python DSL? Is it just the Python DSL? No, there is a lot of metadata logging which Kubeflow pipeline provides. Now, why do you need metadata logging, right. Hey, first of all, you need to find out, right, trace back what dataset my model was trained on, what hyperparameter was used, what TensorFlow version was used, because tomorrow if your model is making decisions like denying someone a loan or admitting someone to a university, you will need to have that auditability and tracing, right. And metadata logging provides you that, right. So you can trace back all the way to the data to the hyperparameters to the framework version. And then you can compare various different experiments which you are running, right, to train your models, which one is performing better on different parameters or which one is running faster, right. Not only this, when you are looking at the field of transfer learning, where you get actually some base models and you need to produce more customized industry-specific models, metadata logging can help you in that as well. And last but not the least, right, in terms of being able to cache. So if you're running these very, very long pipelines which are running distributed training, and you probably, you know, at some step, 30 minutes or two hours down the line there is a mistake, you don't want to go back all the way and start again. You want to be able to cache and move forward, right, in the next run. So metadata logging also provides that capability. Then there is artifact tracking, right. So as I was talking about the artifact tracking, essentially here is all the artifacts which are produced, whether it's the next version of the dataset or the model which is produced. All these things are logged as different artifacts under Q-Flow pipelines and you can go in their all version nicely and you can look at the history. And then combine this, all these different artifact tracking and the pipeline runs and you get linear tracking which essentially allows you to go all the way back from your produced model. If you can see the model is here but you can trace back all the way from the model. How was the dataset transformed? What was the initial dataset? So you can trace back everything all the way to the back using this linear tracking capability. And to enable this metadata and linear tracking, one of the things which Q-Flow pipeline on Tecton uses is essentially what is called metadata writer which is watching all these different pods which are producing these different outputs onto these empty directory volumes and then it's watching and copying them back to machine learning MLMD which is an open source metadata tracking system which was launched by Google. So this is the solution which helps us track all the metadata as part of the Q-Flow pipelines. And once it is running, it's all seamless. What you see is as the pipelines are running, you see all the logs getting produced in real time. If anything is going right or wrong, you can see all this and then you can look at the MLMD data from this tab and you can launch the different artifacts and from those artifacts then you can do the linear tracing where you can trace back the lineage from the model deployed from the scoring endpoint all the way back to the data on which the model was trained. Now this is all open source, right? And on top of this open source, we launched our product called Watson Studio Pipelines. So if you're not aware of Watson, Watson is the IBM's machine learning and AI umbrella. All the platform products which we launch under the AI space come under the Watson branding. So Watson Studio Pipelines essentially was just launched in open beta and it's part of this ecosystem of MLops portfolio of offerings which we are launching under the Watson brand. And that essentially gives you a lot more capabilities on top of that open source solution, right? Obviously provides the capabilities which are there, but then gives you a drag and drop interface, right? So you can drag and drop different components and construct your pipelines if you don't want to write Python, right? For example, so this skaters also towards business users. And then there are components, right? For all the different IBM products like, you know, data refinery, which is our tool for data manipulation, AutoAI which launches AutoAI experiments. So a lot of these pre-built components are provided for you to be able to drag and drop onto this palette, construct a nice pipeline and launch it. Now as you can see on this particular screen, right? This one is using AutoAI. It's using DataStage. Now DataStage is an IBM product which you'll be surprised it has been deployed at more than 2,000 plus customers for the last two decades. It's very, very popular product in the ETL space. Now with the power of Q-Flow Pipelines and Watson Studio Pipelines, you can just drag and drop that particular product capability onto this canvas and string it with other capabilities from Watson and Breva. Okay, so what did we talk about so far, right? We talked about that we need pipelines to heavily automate machine learning in AI and move machine learning models in production. For that, the project which we talked about is Q-Flow Pipelines and we have a special flavor of Q-Flow Pipelines which is built on top of Tecton. Q-Flow Pipelines on Tecton is our machine learning and AI pipeline engine. That's an open source and on top of that we have launched Watson Studio Pipelines which is the product version. Now there is one more thing, right, which still doesn't exist, right? What doesn't exist? So one of the things which we are seeing across enterprises is that there are different groups within companies which are getting formed which are focused on different parts of the AI lifecycle, right? Some teams are doing feature engineering, some teams are producing data sets and different versions of the data sets. Some teams are producing models and then there are teams which are producing pipelines and individual steps of these pipelines. What is happening is a lot of duplication. Sometimes an organization within the same company, even smaller companies, they are producing very similar versions of the data sets which they have modified or very similar versions of the models. Obviously a lot of the pipelines and the pipeline steps we are seeing, you know, getting duplicated a lot, right? So there is a need for this one central place where you can actually essentially go share what you are building within your organization so that it can be shared across team and organizational boundaries, not within your organization only, right? It can probably be shared across, right? And then the other part is, you know, when you are picking up, for example, you want to pick up a data set or a model from the public repositories, you want to be assured of the licensing, right? You don't want to pick up something and train your model on a particular data set where some and some ensues you, hey, this is not the right data set or you don't know the lineage of that data set, where did it originate? What was the licensing? What is the auditability trail? So you want to be able to go to that one single certified place where you can, you know, download assets where you have the right set of metadata and lineage. Now to handle this problem, we are joining hands with Linux Foundation AI and Data and we are announcing Machine Learning Exchange. So Machine Learning Exchange essentially is built around the concept of sharing and reusing pipelines, datasets, models, notebooks, pipeline components and this is a project which we are launching jointly with Linux Foundation AI and Data on their infrastructure and we are hoping, you know, more and more partners will join on this mission, right? So essentially, as I mentioned, right, you can have, this is one single central place to go.  Now, under the covers, why is it relevant in a pipeline's talk? Okay, because under the covers, you know, the components which it is using apart from, you know, sharing a lot of the Kubeflow pipelines and components, it's using the Kubeflow pipelines on Tecton engine for executing all these artifacts. So it's not only an engine where you can go share, download or upload, you can essentially also execute. So if you deploy your own version of machine learning exchange on your Kubernetes cluster, then, you know, you can upload your own assets and you can actually launch them. So the pipelines which are there, hey, you should be able to try it out. Yes, you will be able to launch. The models which are there, you should be able to deploy it. Yes, you would be able to deploy it right there on that particular engine, right? So that's the inbuilt and the integrated capability which machine learning exchange comes with, with Kubeflow pipelines on Tecton as a pipeline technology as well as, you know, another project called K-Serving or K-Serving, right, for serving your models in production, right? So it's pre-integrated and pre-built into this. So as I mentioned, like one single place to go, look at your pipelines. If you're running your own version, you can register, search for pipelines or you can, or upload your own pipelines. And not only that, you should be able to launch your own pipelines as well, right, with the pre-integrated engine in this. So as you can see, when you click the launch button, it comes with all the different input parameters which you can enter and run it. So once it launches, as you can see, you know, it's using the underlying Kubeflow pipelines on Tecton engine. Similarly with components, right? So components themselves are the individual building blocks of the pipelines, right? This gives you one central place where you can upload and share all the components and you can try them out before you integrate them in a pipeline, right? A component can be as simple as, hey, echo a hello world, right? But then, you know, once it is there, nobody else in your organization should be creating echo hello world, right? I mean, that's the idea. And then you should be able to test it out as well before you start using it as part of the pipelines. Models, definitely, right? They are at the heart. And at the center of this, you can go and a lot of the models of a system project called model asset exchange. A lot of the models initially come from there. We are also integrating a lot of the ecosystem models, but models around, you know, object detection, text sentiment classifier, you name it, they're all present there and you should be able to download and use them, right? And if you're running your own version, you should be able to upload your own models and register and expose. And datasets, right? One of, you know, the integration of datasets within the industry and they are at the very beginning of the machine learning life cycle. You need different datasets to be able to, you know, train and create, you know, more refined models so you can actually go look at a dataset, look at the metadata surrounding the dataset, the licensing and then, you know, you can essentially use the pre-built capability within machine learning exchange to download that dataset onto your Kubernetes cluster or OpenShift cluster or, you know, you can take that dataset so now, so that's the machine learning exchange framework and in the catalog as I was talking about, you know, there is a lot of high quality curated content, right? And just to give you a sample of the kind of content which we have in machine learning exchange, for example, you know, one of the datasets which we have, it's the data for, you know, code, essentially think of it as it's, you know, AI for codes, ImageNet equivalent, right? Which allows you to do different kinds of code analysis, it allows you to create code, it gives you suggestions how to make your code performance. So a huge dataset, in fact the largest AI for code open source dataset which is right now available extends as part of the machine learning exchange, right? And if you want to see the breadth of it, it's like 14 million code samples with around half a billion lines of code, right? That's the amount of code which is available in the AI for code dataset and this particular dataset is available as part of machine learning exchange, right? And not only this dataset, right? You have this dataset but you also have, you know, subset of these datasets for a specific task because as we talked about, right, the dataset is humongous. You also have associated notebooks, right? For example, in this case, you see a notebook here for project coordinate language classification, right? These notebooks essentially allow you to take that largest AI for code dataset and then, you know, create more data. So, all in single place, all these related assets. Please give it a try, ml-exchange.org, very easy to hit and once you go there, it will redirect you, I mean not redirect but you have a link there to the GitHub as well. Please go to the GitHub repository, try it out and give us your feedback. Okay, with that, I'll pass on to Tommy to actually talk about some of the advanced Qflow pipelines and Tecton capability and also show you some demos. Tommy? Hi, I'm Tommy. So now I'm going to talk about some of the advanced Qflow pipeline and Tecton capability. So, as we first implement Qflow pipeline on top of the Tecton engine, we realize a lot of the advanced Qflow pipeline features such as parallel loops and recursion cannot be done with the native Tecton capability right now. So, this is why we talk to the Tecton communities and we have come up with this new concept called Tecton Custom Task. So, this is a Tecton feature that you could use to plug in alternative execution engine into existing Tecton pipelines. And they are actually processed in a separate controller. So, the user could actually define their own weekend style logic that's much differently than the regular Tecton task. Just for some background regular Tecton task is created with like a new part to run this code. So, it's actually always containerized whereas in Custom Task, it's actually inside the same cluster. And further forward, you could also add more complex execution flow such as loop and recursion on top of your weekend style logic. And in Qflow pipeline with Tecton, we actually make Custom Task a plugable pipeline component just as other Qflow pipeline components you could use in the Python DSL. So, now I want to go over some of the feature timeline within Tecton Custom Task it's been a long journey on this feature in Tecton. So, when we actually first talked to the community, they introduced this Custom Task feature back in June 2020. That's when they asked the proposal and introduced the new run API in back in July 2020. However, at the time like the run API cannot be run inside a Tecton pipeline API. So, this is why our IBM team kind of stepped in and try to contribute and make it closer as like the regular Tecton Task API. So, we introduced the Custom Task API inside the pipelines back in December 2020 and add like additional capability such as like supporting workspace service account part templates early this year and Custom Task result in February 2021. And recently in the past few months, we also like try to fill in like more gaps in like the TaskSpec feature. So, we actually introduced like embedded TaskSpec for Custom Task back in June. And also, we have supported Timeout API just last month. And at the time, we actually try to fulfill like the rest of the API gap from Custom Task at the moment with the communities. And we have the proposal accepted as well last month. So, now I want to introduce one of the main Custom Task were created for QPOP pipeline in Tecton to fulfill some of the core missing feature in QPOP pipelines. So, one of the core feature we want to fulfill in QPOP pipeline is able to loop and recursively run a same subset of the pipeline over and over. So, this is why we introduced this new Custom Task in Tecton called pipeline runs where the main feature is able to like typically loop over a set of parameters. And additionally, it could also loop into multiple arguments inside a list of like JSON requirements as well. And last we also handle like conditions so it could actually run loops recursionally. And in this Custom Task, our main goal is first be able to like support loops on a subset of pipelines so it could handle like the QPOP pipeline in Tecton DSL such as parallel loops. And second of all, we want to able to create sub pipelines which is equivalent to QPOP pipelines graph components in this case. And the third thing, we want to support a recursive reference where we could actually create like wild loops using like recursion loop inside DSL. And we also have support for parallelism which is also like a feature existing in the QPOP pipelines as well where you could limit the number of parallel blending graph within the same parallel loops. And last, we also have the capabilities to optimize like some of the retry and timeout strategies. So let's say if one of the parameter in the loop is failed, you don't have to retry every parameter in the loop over when you are able to optimize retry strategies. And furthermore more than the looping components, we actually have introduced all the Tecton Custom Task to optimize our workloads in QPOP pipeline as well. And one of the example we have introduced is Custom Expression Language which you could actually process much complex condition just inside the controller without creating any new parts. So you actually optimize a lot for the container run times. And then another example is able to for commonly used tasks such as just sending notifications could be also just done inside the controller without having created new tasks for every pipeline when you just want to send a notification to one of the user, one of your organizations. And then lastly we also introduced some of the new components in QPOP Tecton. One of the components we have introduced is called any sequencer. So any sequencer in short is a dependent task that actually starts when any of the tasks or condition dependency completely successfully. So that pilot actually doesn't have to wait for all the task dependence to be complete before moving to the next steps. And this can also be done in some conditions. So let's say in this case you have Job1, Job2, and Job3. When Job1 completes any sequencer could actually just trigger Job4 directly. Or when you have Job3 complete you could actually verify if Job3 complete in a certain way before you trigger the condition for running Job4. So it's actually a way flexible dependencies to run on QP in Decton. So let's look at the demos on how this actually works in action right now. So I will go over the demo for running the any sequencer on key flows pipelines. So let me bring up the sample over here. So we have uploaded a pipeline that is running any sequencer. So any sequencer in the base form is just a regular key for pipeline components where you could actually just depend on three other different tasks. And then when one of those tasks actually completes, then the any test sequencer task could actually just see it and actually trigger the next components. So let me start to run these examples so you can see in actions. Yes. Oh, sorry, yeah. Give me one second. Thank you very much. Yes, so we just execute like the any sequencer pipeline. So let's take a look. So to begin with, the any sequencer pipeline will have three components that these are just sleep components as you can see. They actually sleep for different section of the time. So one of them actually sleep for 15 seconds, another one sleep for 200, and this one sleep for 300. So this is a task called flip coin. So we flip a coin to say whether the result is head or tails. So let's take a look at the result of this. So because this actually says it's tails, so the any test component I just see is not expected because the any test component expect a head condition before it actually trigger the next drops. So we actually look in the logs for the any sequencer. You could see, okay, the flip coin however the condition didn't met as they expect the outcome. So it actually not going to start the next components. It will just continue to wait for either the sleep component to a component free to be complete in order to proceed to the next task. I also have a previous run where it actually returns head. So let me go on this one. Oh sorry, actually this one. So in this one because the component actually returns head as input. So you could see it. So that's why it says the task run of flip coin succeeds and continues the next task where it runs the drop for sleep components. So in short, this is how like any sequencer work in actions. Because so now we're going to like talk about some of the other demo we have prepared. One of them is actually what animation have mentioned about. We have a managed service on IBM cloud to run give a pipeline in tectons. So what we have called is the Watson Studio Pipelines. So for Watson Studio Pipelines you could actually create this on any IBM cloud Watson Studio service. So when you go to Watson Studio projects you could click on add the projects. We will see like a pipeline managed service. When you add this and you actually will see one of the pipeline work with it here for the demo. This is what we call AutoAI Pipeline to actually pool like data sets and automatically turn the hyper parameters and then find the best result of the pipelines and deploy them on Watson Studio so you could actually use it in actions. So once you get into the Watson Studio Pipelines you will see it's a little bit different from DQ4 pipeline dashboard where in Watson Studio Pipeline you actually have the ability to drag and drop different components as you can see over here and like create new dependencies and then you could also simply just remove them if you don't want to and when you double click it you can always modify the different input output parameters and one of the cool feature we found is like having the ability to able to add annotation and comments actually helps you understand what these components actually try to do like when they see in the UX of a dashboard. So this is what we find very helpful and once we like prepare the pipeline well with all the parameters then we could actually click on run and start running these pipelines to select the datasets and then we will select a Watson Studio space to deploy our models so at the end we could actually go on the Watson Studio space to run the predictions and then we also fill an API key for your relevant services that you use in these pipelines. So once we click on runs behind the scene you create like a pipeline for you on a Kubernetes cluster and then we will run this step by step just like the key for pipelines for open source. So because this pipeline will take like several minutes to actually like get the dataset and train the models so I will just show you what the problem we have previously run. So in this case you will see all the pipelines has been complete and you can also check the logs once the pipeline is finished and you can also click on here with the pipeline outputs where you can see all the output for each components and on one page wheels which is very nice. You can actually keep track of it without actually click on each component and once the pipeline is complete you can still like add the annotation part whereas if this pipeline is very good let's say modify the annotation a little bit just let the datasets know what is going wrong and when the next datasets take a look at this pipeline they have some idea how to keep track of stuff for each component inside this pipeline. So this is an overview of what was the pipeline looks like and last and not least we are going to just show you a brief view of the new open source project we will introduce called machine learning exchange. So in this case this machine learning exchange is actually running one of our clusters that could execute pipelines on this. So actually let me enlarge so people could see it better and in this case we have several different kind of assets so one of them is datasets as Anna mentioned one of the cool dataset we have is called project code net where we actually have 40 million code samples and we click on this dataset it actually shows you all the data set what kind of license you have used and you want to just have a more short view of the dataset metadata itself you could see in the YAMLs and of course you could also launch this dataset on the same cluster so it actually create a volume and provide a dataset that is ready to use for your other assets such as pipeline and notebooks. So once the dataset is ready you actually could run a notebook to use this dataset so let's say we pick a dataset for the project code net let's say for language classification this notebook uses the subset of the code net dataset where of course you could actually launch it directly over here but you could also preview this notebook code to see what this notebook is actually trying to do so this notebook will first just try to import a list of libraries to process the dataset and then this is a classification model so you could see this is the list of classes we're going to classify so because we're only running a subset of the code net so we only have this 10 different classification class in this model so because we don't have that much time to run over this whole notebook we actually pre-train the models beforehand and register as a model asset already so in this case we pre-train the language classification models on machine learning exchange and register as a model assets over here so inside the model assets you could also provide a readme to describe how this model is being trained what kind of data is being used and of course you could also generate samples in the background or run on your own Q4 pipeline instance so because it actually integrates with the Q4 pipeline we have built inside machine learning exchange so we're actually going to just launch it directly over here so in this case we're just going to launch this model inside a Kubernetes cluster with a simple deployment and services to launch it is very simple so you just have to provide what kind of action we're going to do in this case just create a platform and we'll pre-define the running name for you so you just simply click on submit behind the scene we will create a Q4 pipeline with the integrated platforms and you could see they actually create a pipeline behind the scene first extract the config from the machine learning exchange model and translate it into something that you could understand and once you see this is done it will create a separate component that actually take the deployment from Kubernetes and actually deploy on the same cluster so the user can actually navigate and try out these models so I believe the Wi-Fi is a little bit slow but once the model is complete you should be able to see inside the logs the model is being deployed and this is like the endpoint that this model will host it could just directly click on this endpoint and because this model is already wrapped up with a swagger API so you could actually just directly using a simple post with the built-in swagger and in this case we will just do a simple model prediction so we need to actually try out these models you could just upload any coding files I will just pick a Python file let's say in this case and then we just execute then you could see okay this deploy the models actually give me a result okay this file is written in Python with 0.73 confident rate of course this is a training with a subset of the code data sets so that's why the confident rate might not be that appealing but of course as you train with more models and more sophisticated algorithm like this confident rate could be improved and more suited for your use case and this is the end of our demos thank you very much and please check out our other talks we have over the open source summit as well and one of the talk we want to highlight is tomorrow morning we have the keynote with Tom Moore our EP's to talk about more in details on how advanced data AI quantum using open source and we will also introduce versioning exchange in more detail over there thank you very much for watching is there any question online or in person is there any question in person or do you see any question online just one there yes go ahead so oh yeah yeah yeah so for the tacton custom test the question is about is the custom test is outside of Q-Flow right so right now these two tacton custom tests we actually bring in as public Q-Flow problem in tacton one is the problem which is actually integrated as part of it another one is the common expression language that's also integrated as part of the Q-Flow library we have however like for other custom tests we actually make a plugable so you actually could go to the type of communities if you build any new custom test you could just plug in just like how you build other Q-Flow problem components you just find that custom test what Q-Flow problem component you understand and you can just run it on top of Q-Flow problem with tactons that's how we actually working right now thank you very much is there other question if not thanks everyone thanks for being here thank you thank you very much


So over the last week we've been talking a lot about the Nintendo Switch 2 because frankly we have some legit reports or at least reports from reliable places in video game chronicle, Eurogamer, Go Nintendo, Nate The Hate all coming together to start opening the floodgates of the Nintendo Switch 2. And they're not the only ones opening the floodgates for Nintendo Switch 2. There's one part of those reports we haven't really dove into and I wanna dive into that today, but not only that, we also potentially have Nintendo themselves giving us some information through patents on a potential new feature for how this system might control. Now quickly before we get into this news, I just wanna say we're on our road to 150,000 subscribers and if you're really enjoying these updates and you wanna say it's as up to date on Switch 2 and Mario Wonder and the new 3D Mario game and Future Zelda games and pretty much all things Nintendo, why don't you go ahead and just drop a like, subscribe to the channel and ring-a-ling that ding-a-ling so you can get notified of all of our videos. I wanna get into something that was brought up but I personally kinda glossed over it because I really wasn't sure what it meant but it actually could be something quite important both to the internal memory and storage on Switch which we talked about, you know, the 512 potential gigabytes of that storage and also for the cartridges. And that's because we need to go ahead and go back to this Go Nintendo article we referenced yesterday and there's this part here where it says the device will use a new cartridge format and this is that one thing that was really glossed over by myself because I really wasn't sure what it meant but then as I listened to people like Nate the Hate and Modern Mintage Gamer and RGT85 and then did some digging into some tech forums, I realized a few things are happening here that are really important to the future of this platform. So let's dive into what this really means by talking about NAND flash memory. Now look, the Nintendo Switch itself uses some form of ECMM memory and it could potentially switch over to NAND flash. That's obviously a discussion that is hard to know because when we hear about the internal storage of the Switch being 512 gigabytes, we really don't fully grasp what that is or what that means or what it potentially could mean in the future, it's just not something that we have a deep enough knowledge of at this point to see if they're going away from that to go with NAND flash but there is some reasons for that and in particular one thing we do know about the current Switch today is that the physical cartridges of the Nintendo Switch today use what's known as 2D NAND flash and the prices on 2D NAND flash haven't really gone down over the years. They're pretty expensive to make and obviously are limited in some ways. You guys see those super long load screens in some games that are slightly faster when you have a digital version in set installed on the internal hard drive? Yeah, that's one of the limitations is that while they're faster than disks, they're not so fast as to reducing load times in a meaningful way. But when we hear about the new cartridge format, some people might just go, oh, they're gonna reshape the cartridge so it doesn't fit in the old Switch. Sure, that's probably gonna happen. We saw that with 3DS. But what if I were to tell you that not only could we see a reshaping of the cartridge a little bit, maybe even just a notch, that they might change the entire internals of their cartridges moving forward to 3D NAND. So what does this mean? Well, we have this excellent article here that we're about to go over that will show you about 2D versus 3D NAND and it's an educational article. This is purely educational and here's the difference. Here's what we need to know about it and why 3D NAND should probably already be being used but maybe they needed to wait for a whole new platform to do it. So 3D NAND is the latest version of NAND technology. So it's usually preferred over 2D. NAND flash storage technology uses cells stacked within the smaller chassis to give the users a small storage component with better performance. 2D NAND devices play storage cells side by side. 3D NAND actually just adds another layer in stacked cells vertically so you don't necessarily have to have modules next to each other. Now, yeah, you can still have modules next to each other of course, but they would be much larger modules due to the verticality with that extra layer. And by storing cells within the drives chassis, 3D NAND provides more storage at faster speeds. Now that all sounds good. More storage, faster speeds. This is gonna be more expensive, right? A 3D NAND flash storage device also provides more storage at a lower price than a 2D NAND device. In addition, the newer 3D NAND technology reduces power consumption and increases the speed at which a device can write data to cells. Now, write is one thing we don't know about read speeds. That's not really covered in this article. And for the purposes of loading times, read speeds are really what matters. But if the read speeds aren't any worse, but this storage can have more of it for cheaper with less power needed. That sounds like a go-to. And that's why I said maybe this is also a technology they might consider for the internals. If it happens to have lower power consumption than say an ECMM memory or whatever the heck they're using inside Switch right now. I don't remember those exact details and I'm sorry I don't have them on hand at the moment. But yeah, this is clearly going to affect their technology. This article basically concludes, you should basically just be using 3D NAND whenever possible. You really shouldn't worry about using anything else other than 3D NAND. And that makes a lot of sense for future cartridges. Now, are they gonna need different PIN readouts? Is this gonna mean backwards compatibility is going to end up being a problem because the way they design this means they can't accept the older cartridges. That is something that we're not sure about but is a distinct possibility and is something starting to come up in a lot of conversations where if there is backwards compatibility it might only be for digital libraries and that would probably upset some people. But also if they're using 3D NAND then it also makes cartridge sizes, not only able to get much bigger, right? Like right now we max out at 32, you could probably get a 64, maybe even a 128 out there with a new cartridge design although the 128 is gonna be quite expensive, still would be feasible and maybe closer to the price of what a 32 gigabyte cartridge costs now because again, 3D NAND is cheaper. It is interesting to see that this could cause them not to have cartridge compatibility. They could just put a second slot on there specifically for Switch games and maybe they do that at launch and eliminate that with future models. We've seen this, you know, where the Wii launch could take GameCube disks but later Wii models couldn't. So it is entirely possible that they include physical support at launch and then get rid of it with new models on the line. But at this point, look, this 3D NAND feels like a shoe-in. It just makes too much sense. Nintendo's gonna make cartridges cheaper. So your 4, 8, 16 and 32 gigabyte cartridges are all gonna be significantly cheaper than they were this generation and now you'll be able to get 64 and higher for maybe similar pricing to what we got for say the 16 and the 32. So that is really, really exciting and obviously the fact that their lower power consumption means, hey, you know what? It does take some power to get the data off but these will take even less power than the current disk which will enable longer battery life when you're playing physically. So cool, that's really, really neat but that's not the only thing we're making this video for today because we have some information from patents from Nintendo. Now this first one that we're about to go over isn't necessarily, I believe, for the next switch. It could be for a future Pokemon game, however. And I figured with the Pokemon presents happening in a couple of days, maybe it's just worth glancing at this. So on this patent here, you'll see that it is an applied for Nintendo and you see the inventors here, both in Kyoto. But as we scroll down, you'll notice that this kind of starts to look like a Pokeball. Now you guys might remember the Pokeball controller they released with Pokemon. Let's go Pikachu and let's go Eevee. This feels like a more compact version. Now again, this might not even be for the Nintendo Switch for a future Pokemon game but it also might be, maybe it's something that's supposed to be compatible with their new Pokemon sleep app. I don't know, you have a spot here to put a wrist strap. So to me, it feels like something, it's not something you hold occasionally. It's something that you would probably just use throughout playing Pokemon.  You have your USB-C charging. You have on off switch here. And yeah, this might be a reset switch or a syncing button so you can sync it to your switch. And as you see, as we go down here, this kind of looks like it's just going to be a controller. You see how it's held. You see how you're meant to press the one button. You have your thumb on a center. Not really a traditional control stick as you're seeing here.  So it could be a D-pad replacement or something like that. It's hard to really tell but it's not like a traditional control stick. Maybe it's meant to be a touch direction pad but it's obviously some sort of directional control there. And again, it looks very similar to how D-pads are usually set up as we're looking at all of these full breakdowns. So while this is probably not even related to Nintendo Switch 2, with it coming up, this upcoming Tuesday a Pokemon presents in the early AM, I just wanted to make sure that we at least show this off because this potentially could be something that's about to be announced. And again, you can clearly see this whole D-pad like module that they're working on here. So again, probably for a future Pokemon game if I had to guess. But yeah, and you can see the actuation and how that works or do you be pressing in different directions and it'll push down on the various touch points there on the pad. So yeah, that's cool but that's not what I really wanted to talk about today. Now this is the patent that I actually wanted to spend time talking about because this is the one that could be related to Switch 2. Now, let's back up a bit. Let's back out of the patent for a moment and just briefly talk about the efficacy of what patents really represent. So Nintendo often releases patents well before things come out. And these patents don't necessarily mean anything, okay? Nintendo's patents often come out for things that either already exist or are ideas that they scrapped. So I'm gonna warn this could be a scrapped idea but there are caveats to this. You remember the patents we got for Tears of the Kingdom? Well, those patents ended up showing us new features that were not shown off in Tears of the Kingdom yet and most of those features ended up coming true. Now there are specific parts of the patent that you can point to and go, oh, we can't quite do that in Tears of the Kingdom. So even those patents presented some ideas that didn't make it into the final game but other aspects of that patent were true. And this also technically happened with the Nintendo Switch. So if you guys remember the Nintendo Switch patent back in the day, back when we were still calling it NX and you had that screen that went around the control sticks and we had that fake leak out there where someone 3D printed a very convincing model of one. What was interesting about that patent when you dug through it is it included many things. Not only obviously that it's a handheld with a screen and controls but it was a USB-C. It could be technically plugged into your TV via that USB-C port. So it could be taken on the go or played on your TV. Naturally we didn't have physical buttons. We didn't have detachable controllers. So all of that required a separate control unit to do it but the point is there were some basic ideas there that showed you Nintendo was leaning towards making a hybrid system that was both handheld and could be used on your TV and used USB-C which Nintendo had never used at that point and obviously had a system with a screen on it. And while a lot of the design terminology completely went away and that was totally just a prototype they still ended up releasing a device that you could hold in your hand and this time through a dock use USB-C to use in your TV with detachable controllers. So it was one of those designs that was probably made on the way to getting to that final design of what the Nintendo Switch ultimately became. And so that is why when I say we're about to look at this patent remember everything in this patent might not happen but there are aspects of it that could and again this does affect how you would control the system. Now I wanna know there's gonna be a lot of speculation because we aren't exactly sure all this works. It's a very interesting design. So you can see that the United States patent application was filed on April 3rd of 2023 with a published date. Can't remember where the published date was. The published date was something like a few days ago I think it was August 3rd. Now way down deeper in this I don't know if we'll get to it today but deeper on it notes that they originally potentially filed this like way back in 2020. And again, Nintendo's always working on their next system. So this is an idea they've had around for a while. Just on April 3rd is where they actually wanted to publish it and like lock it in. They wanted just to get fully approved as of April which could be a sign that this is something that's going into the final design.  Yeah, okay. So what are we talking about? We got this little button like device here that looks very interesting. It shows that there are multiple parts to it and it can move in six different directions and can spin. You see this where, oh, it can spin on the Z axis. It can spin on the Y axis. It can spin on the X axis. So you can move it in a circular motion while moving it in a direction. Now that is, it feels a little similar to the slide pad we had on the 3DS except obviously when you push the slide pad up to go forward or left, right, et cetera you couldn't then also spin it in that direction. So that is a very interesting design and I do see how this has the potential to become a possibly a new type of control stick and an innovation in a way that I don't think any of us were thinking about. And so keep that spinning motion in mind as we go down through the figures. So here's where you get your first look at this sort of looks like a little bit of a traditional control stick. Now, this is obviously pressed up and you're gonna see them over this whole thing. Here is what moves. But I do find this interesting in how this component works because if it just works like a traditional control stick that's totally fine. But now adding extra functionality to that including pressing down button click, right? We've been able to click in our sticks for a long time. But keep this in mind when we get to this really interesting upcoming image. And this is where we see what the top sort of looks like it's probably a bit exaggerated because if you see in this picture here it doesn't quite look that wide of a circle around it. So this is probably an exaggeration just to make a point that this can move independent of this as well in four directions. And this is the component that spins. This is what spins. That's why you see the arrow here. So keep this in mind as we look at this right here. So this looks like to me mostly a traditional control stick. Now you're seeing that it comes off of this, right? Off of this. I'm not so sure that the coming off of that is how the mechanic would work. But the way that I figure is that when you're using a control stick, you know, you press forward and it leans forward. You press back and it leans back, side and it leans side, right? So that's sort of like, hey, we're leaning forward, we're leaning backwards but also we could slide forward and we still have this device on top that we know can turn.  That to me would be a total Nintendo innovation and if they ever brought this out, that would just be insane. Now it's also entirely possible in looking at this because if you remember up here it showed this top part here can move in four directions. It doesn't have the six direction or eight directions. It's got two accesses here and it can spin. What if in the final design Nintendo does away with the spin because maybe it's just too mechanical and maybe too flimsy and breaks but they keep the four directional slide. So you tilt the control stick forward and then there's that additional thing in the middle that you could just sort of nudge in four directions for an extra for essentially inputs, button inputs, directional inputs in addition to a single control stick. That would be quite curious if they would do something like this. Also it's possible that they only put this on one of the control sticks. So maybe like the left stick, might be your traditional movement stick and the right stick that we're usually pointing cameras and guns and stuff with maybe has this additional fine tune functionality to it. But we're just speculating because we don't really know what this crazy controller stick design is going to be. Is it a sliding stick? What are we talking about? We don't really know. All of this is just explaining the six access data storage and just basically the basic principles of how all of the directions and stuff and get into operation right here, the screen generation section. Like, hey, all your controls and then here's what happens on the screen. And again, you're just seeing sort of a side cut view of how it would be put into a controller. Which by the way, one thing I like noting about this part here is look at this curve. You see this curve? Well, let's say this was a vertical slice of the new controller. This curve suggests better ergonomics for the new controllers. If this, nothing else, you assume back here you're gonna have some triggers, right? You'll have your triggers back here. But when you see this curve, Nintendo might have actually made more ergonomic joy-cons or controllers or whatever they're working on for this thing. So in the end, what are all of my thoughts on this? I do think that obviously Nintendo's gonna go with 3D now. I don't even, I don't think it's a rumor at this point. It's cheaper. So you save money. It uses less power consumption. You get more in a very similar size package. Yeah, duh. They're clearly gonna use 3D NAND for their new cartridges, which unfortunately could mean, could, I'm not 100% sure, that they need a different pin layout. And with a different pin layout, that potentially might mean you can't just do a 3DS to DS situation where you plug in your DS cartridges in the same slot. Now maybe that's a non-story. Maybe they don't have to change the pin layout because you're just changing the chip that goes on the board and you can use the same pin layout and then they'll just have a little notch or something like that to make it so you can't put it into your Nintendo Switch and you move on with your day. So that is obviously entirely possible that you could still have that backwards compatibility. But unfortunately, I don't know enough about if you need to change pin layouts for this new style of NAND flash. And if you do, that to me puts some doubt at least in Nintendo supporting physical backwards compatibility. But that's obviously a conversation for another day. I feel like we're gonna hear a lot more about the physical backwards compatibility and just backwards compatibility in general after Gamescom. I think once developers get together and they cooperate their information, we're gonna start hearing if backwards compatibility is truly a thing physically and digitally. But then when we talk about this new patent on the controls, again, it could be nothing and this could be an entirely scrapped idea, but it also could be something or there could be aspects of this that make it into the final design, even if the entire patent doesn't. And again, I really do love that image. It shows we might have some more ergonomics to this. Like, you know, look, this is a very sharp cutoff. That's not what this one looked like. This one had more of a curve to it, a more of a curve, which feels a lot more ergonomic than what we have here, where right now, you know, my hand right here doesn't even feel the curve, you know, because it's too separated from it. So if there's a nice little curve here that you can sort of feel with the bottom of your hand, it just makes holding things more comfortable. So while they might not still be the most comfortable controls, it would represent a redesign that would show, hey, we do have a new type of Joy-Con. I wish the update I had for you is that Nintendo was gonna use Hall sensors for the control sticks. That would have been really the update I'd like to see, but anyways, that's not what that patent says. I actually read all of that billion, the bunch of text you saw. I read all of it just to see if Hall sensing was in there anywhere, and it's not. So anyways, guys, you guys, let me know what you think about this patent, what you think about the 3D NAND storage, if it's gonna impact, maybe some of you guys are more knowledgeable and you wanna educate me on if they'd have to change pin layouts because that would affect the ability to use old cartridges. I don't know, all I know is I wanna thank you guys for being here, and I'll catch you in the next video.