everybody first of all I'd like to apologize to not being able to come to Detroit to give this talking person I hope you're all having a great time at the conference I wish I was there but unfortunately it was not possible for me this year so today I will talk about Calix which is a new product that Light Bell has been building for I believe two years now and it's now available for use my name is Renat Kavokant and I'm principal engineer at LightBend work on the Calix team I guess you already have heard about Calix during your day at the reactive summit this year but nonetheless I'll give a short introduction to it explain our motivations to build such a product and then give a quick demo so you get a good understanding of how it feels to build a Calix application but before let's talk about ACA if you have been following the work of LightBend you know that we have been providing solutions to help developers to build reactive applications that's basic what ACA is ACA is the most powerful and successful actor system implementation of the JVM it has enabled developers all around the world to build reactive distributed systems without having to build everything from scratch it's our passion to make distributed system affordable and accessible to a broader audience distributed systems are hard to build and therefore we want to provide the tooling to make it accessible to any developer out there you can consider ACA as a runtime where you can implement your business logic and let ACA do the heavy lifting for you we provide very powerful tools to help you at this task the most powerful of them in my opinion is ACA cluster so think about that distributed databases providers like Cassandra or CochranDB or message broker providers think about Kafka they all provide cluster solutions in the cloud here with application service serving millions of users data needs to be spread sharded over many nodes and someone needs to ensure that data stays consistent but what about your application code can you also distribute your application well the easy path here is to build stateless applications and stateless means that you deploy as many instances as you need and you let the database layer take care of the consistency the common technique used in such a scenario is the classical optimistic look you read that from the DB and that data is a version and when you have to save it back you check if the server the version you have and the version that is in the DB is still the same if that's the case you can save it if not the process fails and needs to be retried or automatically or re-triggered by the user this technique works until it doesn't work anymore that's pretty much fine at small scale but but not when you need to serve millions of users this is when libraries like ACA becomes relevant you want to be able to build application that can take that kind of load without having to rely on a mystic lock and that's what ACA cluster does it will manage data over the cluster for you including persistence in a in a way that you don't need to resort to the mystic login so ACA is great it allows the valves to build extremely performant applications we love it however the story doesn't stop here while ACA gives you the tooling to build your own clusters applications you are still responsible for its deployment and to maintain the whole cluster infrastructure by that I mean you need to manage your database you need to manage your Kubernetes class you need to take care of security just to mention a few for those reasons we have built Kaliqs think about Kaliqs as a cloud runtime for reactive and distributed applications Kaliqs is building the idea with the idea of providing building blocks that you can use to implement your business logic without having to even think about how data will be persisted how that will be retrieved how your application will scale out how to manage your application cluster and so forth the philosophy is bring your code Kaliqs does the rest but how does that really work so in reality under the hoods we manage a Kubernetes cluster as you could imagine and in that cluster we run what we call execution cluster and there when you create a project it will live in that execution cluster you come and you deploy your service and on that slide the user function on the right side is your code the code that you're gonna write on the left side is a Kaliqs proxy and they are all together when you deploy your user function we put together with that your container your user function container we put the Kaliqs proxy and they will find each other they work together when a request comes in it goes to the proxy and the proxy delivers data to your user function that has the logic your user function will process it and send back to the proxy and the proxy saves the data so if we look a little bit closer what we have here is like your project that you're creating Kaliqs you deploy your code and they all go together with the proxy then as you go and you may need more resource power more computer power we scale out your service to a few other nodes but those are not stateless nodes they form an acupuncture so your code in one side they go always together with a proxy but on the proxy side so at some point you have let's say three pods each of them run in two containers on your code and what we call a sidecar but also the sidecar in Kubernetes terms for us we call it it's our proxy running there so your code does not make a cluster it's your service one side but the proxy that goes together with your code that are deployed together they do form a cluster now whenever we go back to the previous slides I have a line showing request coming in it goes to the proxy and if there is data that needs to be brought to memory it will take that from the database put in memory take the commands that just got in and send it to your code the state of your model and the commands your code will do what needs to be done and return to the proxy the new state if the state was updated if it's an event source entity it will return events as well that needs to be perceived so this dance between the proxy and the code happens on each time that there is a request keep in mind that the proxy is an aqua closer and it's managing the persistence and your code is just providing the logic to be applied on the data so you don't have to deal with persistence yourself but how how do you do that so you start by define your APIs using grpc definitions everything in calix today is about grpc then the calix tooling will generate class for you to implement so you define an entity an event source entity and we generate the skeleton for you you just have to implement it you can test it locally using unit test integration tasks or you can even run the application on your computer for that we're gonna you're gonna start your user function and also a proxy in a container in the docker container so the proxy and user functions are now running on your machine and you can just use them like to test manually if everything's work as expected the same happens integration tasks we also start a proxy and but then it's automated of course and once you are ready you'll deploy to calix but when you deploy to calix you don't deploy the proxy you only deploy your user function we provide a proxy it's included in the in the cloud infrastructure so calix has different components that you can use to build your system and the first one are event source entities they are backed by a journal and use commands and events to mutate its states basically commands get in are validated and events are emitted the emitted events are persisted by calix and then apply to the model updating its state event source involves capturing change to data as opposed to overriding existing values then we have also other kind value entities another kind of entities also persisted the difference here they are very similar to event source entities except that they don't emit events they are not event sourced they only persist the current state and that's why we call them value entities they are entities but the whole state is persisted the whole value that are in memory is persisted on each incoming command the model decides if the command must be rejected or accepted if accepted typically the command will mutate the states and the state in its whole is persisted next we have replicated entities they are also entities but they are not persistent they are uniquely identified by the given identified identifier but they don't go to the storage they stay they are distributor of the class so if you have your application has three nodes they will be in all of three of them in memory and you mutate in one node the change will be propagated the others eventually so they are eventually consistent but in memory data structures think about CRDT's and distributed eventually consistent distributed cache now those are entities and the two first entities they are persisted and but in all three case you cannot access them if you know their ID so you can always read the data if you know the ID but sometimes you need to do to do other kinds of queries you want to build a model that you can search that's not only based by ID for example you have many users and now you want to find to collect all users that has an email address that ends with gmail.com for instance for that you need to create an index and that's what the views gives to you so views are means to expose different representations of your model we can generate views based on the events of an event source entity or from the states change of value entity every time that the change of value entity the whole state is propagated to your view in the view implementation you get the value entity state and then you decide what to do and it allows you to query for the entities using different fields like using the email address instead of the unique ID and you are free to to to persist or to index the way you want you can say that you have a user and at the view side you also use the same model the user that you can just search by email or you can create another type you can say on the entity it's about the user but in the view it's a little bit different it's my user view for instance and there I don't store the password for the for example the last the last type of component we have are actions they are the most simple one they are stateless whenever you call an action it respond a new action to to receive the request and when it's finished the action just disappears so they are stateless if you put this stated in there it just get discarded it's each time and why you would use actions while there are few use case the first the most common one is the pure functions you get some inputs you produce you process data and you return that's just pure function computations you transform from A to B you can also use actions as a facade in front of your entities so the action receive the command from the exterior from from the outside and then you do some transformation and you forward it to the entity and the entity then responds to the to the action that then respond to the color color and the other options are you can also use actions to subscribe and to publish events so you can subscribe to to a topic Kafka or Google pops up and push data to Kafka Google pops up but you can also subscribe to event source entities events for instance and push into Kafka into on a on a topic and those are what we call the Felix components and there are a few more features we have first I already mentioned about the topics we have broken integration if you choose to deploy in Google clouds you get the integration with PubSub Google PubSub otherwise you can bring your own Kafka you configure it your Kafka's usually people use Kafka in the cloud as well they just configure Kafka in in calyx and then your calyx application can start to read and and push that into Kafka as well we have timers timers is a functionality that you can only use from an action and and timer you can say okay I want to make this call but not now in five minutes so you scan a call to be done in five minutes and this persists so if you shut down your machine your your notes at that moment and bring them back they will be executed we have ACL so access control is to you can annotate your methods say who who is allowed to to call it who I mean anyone on internet it's open to the internet it's only available for the internal service and so on and we have JWT support okay so I'd like to to write a little small application give a small demo and that's what we're gonna do to them right now I will use this color template for that it will create a value entity for us I call it calyx demo reactive summits I just skip the default options for the versions library versions and then the package I call it calyx demo okay we have here a project I would just quickly open that on visual studio because I want to show what is inside before I start to compile it it has just some protocols with the definition of my entity here is the state of my entity it's a counter well the template comes already with a model so you can play around with that the goal is that you delete that afterwards and use the templates to start your own model so we just keep going with the counter for convenience and it has commands message that can be sent to it increase decrease like increase the counter and the important bit here is that we have this information telling calyx that we need to we want to generate a value entity to manage that state the state in question is the counter state here I will add one extra field here I will say that this guy here has a name and what is that so we have the counter state is what is persisted and the current counter is what I returned when someone asked to get the counter so that's my external representation and just to make it a little bit different I'm adding here the name so next I will compile that but I will open it in IntelliJ because then it's more convenient to to work on it so let's let's have a look what got generated I have a main function which registered the counter and I have the counter that I have to implement so let's do it so the counter the first thing is that what is the empty state when there is no counter there when I have nothing my database and I want to send my first command what from where I start so I need an empty state or initial state and I think the very sensible one for a counter would be something that started before a counter that start with starts with zero then when an increased command comes in what I will do is I will create a new value that is the current state plus plus my incoming request so increase value by so so much then I have a new value and I tell now Klex what I want I want Klex to persist that update that state that's an integer or long I don't remember yeah an integer and here's the state I will just send a new state to Klex and Klex will persist it for me and then I confirm to my users that the command was accepted by send this empty message with kind of acknowledgement I will just copy this code here because it's almost the same except that this is the decreased one and I will do a subtraction and when the user sends a reset I will update the state back to zero the empty state and I reply and and finally the current state when I someone requires request the current state I will return this other type here which is then reply I will reply with demo current counter but the thing here is I have two options well first first option is to has the states the state that is persisted is the one that is getting returned here in pass to the to the method and it's counter state and the next one is the name and the name I can use in the command here in the incoming command there is the counter ID that's one option but this also available on the context is the entity ID I will use the entity ID is the identifier of this entity to fill in the name that I added here this one and the context is always available to your entity and the entity ID will be whatever we use to to call this counter not a view I create a new package here in the protobuf folder call it views and counter view photo and I will copy here some code that I already prepared and first some basic protobuf configuration import I would say then I want to import the types that I have already defined it here in my Klex demo project and I will define a service and if you remember the previous one here it was saying that to the we said to the code gen that's a value entity in this one we're gonna say to the code gen that's a view so first I'll start with this message here and then the updated methods the query methods let's let me start with the update method and I explain what it is so the updated method is saying there is a method that we receive counter state that is the state of my entity and we'll return current counter which is that representation that I have to that I want to expose to the external world then inside it I have Klex annotate say well I want to subscribe to any change that happened on entities identified by this entity type call it counter and then I want those updates to go to the counters table and now I'm gonna define the counters index or table which I do by create another method that receives a request and return a stream of count current counters and it will say give me everything in the counters index in the counters table whose value is greater or equal the value that I get from my request and you see here value it's repeated here so that this field here you need to match this one and that's what we get from the outside and is used it to execute the query when we generate our code we generate this main function and we have one counter here but now one component but now we have two components we have the counter and we have the view so what I will do I will delete this one and you let I will let Klex generate it generate it again I compile close this one this one as well and now I have the view and the view receives counter state and return current counter for the empty state in this view we are fine to just say no and for updates we just for every time that we get an update I will just up to return a new counter current counter effects update state and the first thing is the counter state is the value the second thing is the name but where I get the name here well the name I will get from the update context and the event subject here is the entity ID for the state that is being passed here inside and that's my view we have our view and now I want to deploy it in production so I will start SBT again but then now passing my Docker Hub username to it because I want to publish that image that project on my Docker Hub account to publish that Docker image into Klex normally you have to install the Klex clients I have already done it but for Mac you have a brief install for Linux and Windows there are the tooling you can have a look at all our documentation but let's say I have it already installed so the first thing that I do and I want to show you of course I already signed up but I want to show you that you can sign up from the command line yeah and you get this page where you can fill in or you can connect with Google I'm using here my Google account my my private one I just continue and and I don't have any project yet okay I sign up I think I still need to authenticate let's see yeah I want to continue it will ask me to give authorization to the command line to have access to my Klex project great now I will create a Klex project this one I will create a new Klex project Klex demo reactive submit and on this region you can choose the region I'm on Google Cloud and so far so good and let's see the image got published it's octonato Klex demo reactive submit and then I can call this command in Klex which will create a new service deploy something with this name Klex demo reactive submit and that's the image that we are looking for that it should look for to deploy okay Klex service lists yep unavailable and now my service is ready the next thing so I want to be able to call it from outside so I will ask Klex to expose this service and it will generate a URI for me just one and I will go back to my Joppy SQL and the port is 443 okay and everything that is written that is there is nothing there so that's it so what we saw here is we can develop an application with very minimal code just the basic just explaining to Klex what he wants for the views explaining to Klex what is the business logic and all the rest is being taken care by Klex that's basically the magic source of Klex is giving to you a knuckle cluster with doubts asking you to manage an knuckle cluster you just write your business logic now we can build a Klex application I want to go back to this slide here and what we saw it's a very simple application of course for demo we should keep it simple and small but imagine imagine that you have a very big system lots of counters and at some point you need to scale out you bring your code your code is just the counter and the view and they are deployed there but the state is managed by Klex and as I was saying we do that by using the knuckle cluster so now your application is a clustered application and stateful because the knuckle cluster take care of the state at any single time even if you have 20 nodes deployed there is only one instance of each single counter in memory doesn't matter where they will one instance over across the whole cluster so that we manage the state for you we managed stateful aka cluster application for you you want to bring your code and that's the magic source of Klex is that it's allowing now you everybody to build an application on top of aka cluster without even dealing with clustering database Kubernetes and everything you just bring in your code we do the rest okay before I wrap up a small announcement or teaser for a near future we have worked on the springers decay which is based on spring boots no protocol for encoding just Java you come write your code in Java you use jacos Jackson serialization and you can use the rest end points define the rest and points using spring annotations thank very much I love you to try out Klex the demo code it's on my github account at github.com octonato slash Klex demo reactive submit stay tuned for the upcoming springers decay that will be very it's really shaping very well I'm have very happy with what we are doing there and if you have questions I think we still have some time for questions I will be around otherwise you can contact me at renaughtatlightband.com and my Twitter account is just octonato like my github one thank very much


All right. Thank you, Candace. So we're going to be talking about how Wasm simplifies streaming data pipelines. First, a little bit about myself. My name is Tyler Rockwood. As Candace said, I'm a senior software engineer at Red Panda. I love all things distributed systems, databases, and programming languages. I'm the current technical lead for web-assembly-powered data transformations at Red Panda. There's a few of my socials will be at the bottom. They'll be posted in the chat and then also at the end of they'll be posted again at the end. If you want to get ahold of me for any reason, I'm happy to talk all things Red Panda and web-assembly. So briefly go over our agenda. We're going to start out by just giving you an overview of what is web-assembly, how a little bit of details of the VM, how it works, and how you can interact with it. Why would you want to use web-assembly for stream processing? And then we'll talk about the integration challenges of putting a web-assembly runtime into Red Panda itself, and then talk about how we overcame some of those challenges and some of the cool technical things that we got to do. And then we'll end with some Q&A. First, we're going to start with just what is web-assembly? Maybe you've heard of it. Maybe you haven't. We'd like to start with just a little poll to gauge your level of experience with web-assembly. Yeah, maybe you've never heard of web-assembly and you're here to find out some more. Maybe you have heard of it, know it's some cool, interesting technology. But yeah, have it interacted with it. Maybe you're somebody who uses this all the time in production. We'd love to hear what your level of experience is in the poll. Cool, it looks like answers are coming in. So I'm going to go ahead and get moving forward here. Thanks for your responses. So jumping into what is web-assembly? So web-assembly, also known as WASM, is a binary instruction format for a stack-based virtual machine. So maybe you're familiar with virtual machines or not, but it's designed to be a virtual machine that's a compilation target for many different languages. So you can take all of your favorite languages, such as Go, or Kotlin, or Rust, and compile them down to web-assembly. And then you can run them really anywhere. Initially, it was built for the web and for the browser. But today, as you'll see, they're being used in message brokers and server applications as a function as a service platforms. It's really, really taking off and being used all over. So the main things that make web-assembly appealing to use in these situations are that it's efficient, it has very good compatibility guarantees, and it's safe. We'll touch more on all of these, and you'll get a better idea of them as we progress through. But efficient in terms of the byte code itself for the VM often maps directly to common hardware instructions. So for things that are doing compilation of web-assembly into native code for speed-up, which we'll talk more about, it can be a fairly easy mapping. It's also open and backwards compatible and versionless. And there's new standards that can be incrementally added into web-assembly, which is key for if these things are running in browsers for a long time or in hosted services. And then safety is also key pillar of web-assembly. So it happens in a sandbox execution environment. You have lots of controls over almost every aspect of memory and CPU limits that you can do. And we'll talk additionally about a standard that gives you the web-assembly the ability to access OS-level primitives. Cool. So we'll talk about the main major pieces of a web-assembly VM. So the main one is probably the compiled module, which is this blue box here. And is the compiled version of that binary instruction format loaded into a VM. So that represents things like what functions are available and all the instructions required to execute those functions. And then specification for how much memory it needs and something called a table, which eventually effectively gives you the ability to do pointers to functions so that that's defined in a sort of portable format. And then once you have that compiled format, you also need to take the actual host memory, give it to the VM so that the web-assembly guest code can interact with it. And then also you need an instance table to store all of those different function pointers and things like that. And when you take those three things and combine it together, you can get an actual live running instance of a web-assembly VM. So next we're going to dive a little bit more into functions and the execution of functions. So earlier I mentioned that web assembly VMs are a web assembly is a stack-based model for the VM. So we're going to go through an example of just adding two numbers just to sort of give you a sort of idea if you're not familiar with VMs or what a stack-based VM looks like, just sort of how the execution works and how these different instructions. So these are three actual instructions from web assembly and we're just going to walk through the execution sort of like you're stepping through it with a debugger one of the time. So this first instruction is representing a constant for a 32-bit integer. So there are four main types in web assembly. There's a 32-bit integer, a 64-bit integer, and then the same for floating point, there's a 32-bit floating point and a 64-bit floating point. Types, there's another standard. Other standards have come along and added more types such as SIMD for sort of vectorized instructions and also references to functions and things like that. But those are the first four, the four main core types. So this first instruction, this constant instruction just pushes a constant value onto the stack. So we're pushing one onto the stack and then the next instruction pushes two onto the stack. And now we have two items on our stack and we get to our add instruction. And basically all these instructions consume and produced onto the stack. So these constant instructions, all they did is push things onto the stack, they didn't pop things off. Add pops two things off the stack, adds the result of them and pushes them back onto the stack. So if you see we pop one and two, add them together and we'll push three back on. So that's sort of the general mental model of the stack-based machine and how it works. So generally when a VM runs, it needs to convert this sort of stack-based machine into actual hardware and instructions that are ran. So there's a variety of ways of doing this and they all have their own trade-offs. So I have sort of here a spectrum of compilation time. So how long it takes to take the WebAssembly bytecode format and turn it into something that's executing on a CPU. So on one end, you have really fast. So an interpreter has very little startup time needed. But on the other hand, there's execution speed. So the more time you spend optimizing and converting the bytecode into actual machine code, you can push performance faster and faster. So the sort of spectrum I'll talk about is there's an interpreter on one end, which just sort of natively is a big loop that walks over every bytecode instruction and evaluates it as we walk through in our example. And then on the other end of the spectrum, you have a compiler, something like the Clang compiler, LLVM compiler suite can let you build something that would take all those bytecode instructions and actually turn them into hardware instructions directly and doing all the different optimization passes that a compiler framework like Clang can do. And then sort of, and then in the middle is something known as a just-in-time compiler. If you're familiar with JVMs or anything like that, it's very popular, the most popular JIT compiler. Basically, you do the sort of compiler step but you do it incrementally and on the fly. So instead of doing everything at once and doing a big optimization passes, you'll just do a single function. And there's sort of a spectrum, even within just-in-time compilation, where you do sort of what's known as a baseline or non-optimizing compiler that will just basically directly map individual instructions to individual bytecode instructions to hardware instructions to sort of the goal being to do compilation really fast. Whereas the other end of the spectrum you have an optimizing JIT compiler that can do things like profile code as it's running to sort of figure out the right optimizations to apply and can do a lot more expensive and spend more time doing compilation for your hot code paths. So that sort of gives you the spectrum. There are WebAssembly runtimes on this full spectrum and some individual runtimes implement multiple tiers of this. So they have an interpreter that runs and then you can sort of upgrade functions to using the baseline JIT or even the optimizing JIT. And there are some VMs that just use ahead of time compilation to really squeeze out all the performance that you can during runtime. So this is sort of similar model to like, for example, the JVM I mentioned, it actually has a similar model. This is a initial code runs as an interpreter then there's the C1 and C2 compilers and C1 compiler sort of represents the baseline JIT and the C2 compiler is sort of the more optimizing JIT. And this is sort of a standard framework for that. So when you start to think about integrating WebAssembly into a live system, you got to kind of work through, what's my use case? What are my trade-offs here that I want to take on this in the spectrum and help you can pick it? There's a variety of different runtimes out there can help you pick between them. So that leads us into the next point, which is why I use Wasm to do lightweight stream processing. So you may not be familiar with stream processing. So I just wanted to throw up a poll and sort of gauge your level of experience here. Have you heard of stream processing before? Are you somebody that does this all the time? Is your day job? Yeah, I'd love to hear, get your thoughts in the poll. And we'll give it a couple of seconds for everyone to put in your answers. Awesome. And we're sort of jumping to talking about stream processing. So stream processing allows you to, as events get processed through like a message broker, such as Red Panda, you can sort of transform those on the fly. So one very common use case is sort of the stateless transformation. So we'll give an example here of there's a stream of JSON click events coming in from your website or whatever it is. And you want to run a transformation because other parts of your software stack use Avro instead of JSON, which is Apache Avro is another common format used in data streaming applications that has schemas and things like that. So you want to do this transformation for downstream consumers of this stream of events, these stream of clicks to consume them in Avro. So today what you do, if you do this, you have these events coming into Red Panda, your message broker, and then what you do is you set up this whole other distributed system. Apache Flink is a very common system for this where you basically, it's a great framework for setting up these big streaming pipelines where you can read in the data from the broker, do your transformations and you can write it back into the broker. Now, Flink can do a lot more than that for this, but for this simple use case, there's a lot of extra overhead of additional VMs to provision management. You got to learn the separate programming model of Flink and manage it production and monitor it. So sort of the idea with WebAssembly within Red Panda is that we can just integrate these sort of stateless transformations directly into the message broker. So the broker itself as it receives these JSON events can run your WebAssembly module to perform these different transformations and write it back out to a separate topic within the data streams for other, these other consumers. So this eliminates a lot of overhead. We do not ping ponging data all over between these different distributed systems. There's less moving pieces to manage and it's just a lot simpler to get started in going when you go from just message broker to okay, now I want to do some stream processing and do some small transformations of my data. So that's sort of the main idea here. And the benefits of using Wasm over some other sorts of programming language or plugin system is, I mean, some of the advantages that I talked about from the very end, you can really use any language. A lot of languages, most languages can compile down to Wasm. And I think that supports only growing. So if freeze users, you can meet your users where they're at and let them kind of program in their favorite language. And then also, especially in Red Panda, you can strictly control the sort of runtime of the VM. So you can strictly impose limits on CPU usage, on memory usage, and things like that. And then also we're gonna talk a little bit about the Wasi standard, the WebAssembly system interface, which basically gives a sandbox POSIX environment for people to give sort of, you can expose common programming patterns to people. So it doesn't feel like they're writing in some weird alien environment. So yeah, these are the lists of some of the languages that you can use. There's some languages that are even only compiled WebAssembly, Grain and Assembly script, or examples of that. These are both languages that were built directly to write, compile down to WebAssembly. And then a lot of other popular languages can run in WebAssembly VM as well. Awesome. So some of those runtime limits, like I said, very important for Red Panda, and we'll talk a little bit more about that later in the presentation. But here's just some example of the CPU and memory limits that you can give. So for CPU, there's a concept known as gas metering, which basically lets you stop execution after some number of instruction counts. So for example, if somebody accidentally has some weird edge case that triggers an infinite loop, that's not just gonna hog up CPU in our broker. We can say, okay, you can put some time limit on, say every transformation can run for X amount of milliseconds. And if it runs longer, then abort it. And there's other techniques as well for supporting, for like other threads, aborting the runtime and things like that. So lots of different controls you can give. Also for memory usage, you can put bounds on how big, how much memory you can use. It's important in Red Panda, because we have a lot of strict requirements on how much memory we have. You can put bounds on those table sizes and those are the two big usages of memory. And then also a lot of the VMs themselves have pluggable allocators. So if you wanna have different allocation techniques, this is common if you're in like C or C++, you have different allocators that you wanna use. Maybe you wanna use a pre-reserved pool of memory for your VMs. A lot of them support being able to plug in different allocators. So we're gonna switch gears and talk about WASI. I mentioned WASI is great. It's a system interface. All these common things that you're used to interacting when you're writing code or things like accessing a clock, accessing environment variables, having access to random data, standard in and out in a file system, or sockets or things like this. These are all things that WebAssembly core doesn't have a concept of. But again, there's a standard for sort of, it's important for upstream languages and tool chains to be able to compile it down to some standard. And then basically it happens it's up to the host that the VM is running in to implement all these methods. So you can do things like only grant access to parts of the file system or only certain file system commands you can do or disallow socket traffic or things like that. So there's a lot of nice things about having this sort of sandboxed, sort of like over the OS level of primitives that you sort of take for granted, you can sort of have full control over within WebAssembly. So as somebody who's embedding WebAssembly into your software, how do you interact with the VM? So basically you end up defining an application binary interface. So this specifies essentially there's some functions that both the program that's running expects to be exposed. So the guests that runs within the VM can have specific methods that it knows it can call into and it's gonna expose certain methods that it knows will be called. So for example, we can do this for every time there's a new record that comes through on our data stream we can call a specific callback. And there's a limited set of types that you can use for this, which is just basically the types that are available within WebAssembly. You can have integers and pointers. And there's also a standard that lets you pass arbitrary host data that's sort of like an opaque pointer but not all runtimes have support for that quite yet. So, but if you wanna pass more complicated data, generally the pattern that you use here is I32 values can be interpreted as pointers into an offset within the module's memory. So if you wanna have like a more complicated structure some binary encoding that you pass data through like JSON or whatever it may be you can write that into the VM's memory and then pass a pointer to the start offset of that data and that's sort of the mechanism you can use to pass back and forth. So we're gonna walk through a very specific example within the WASI spec of just reading some random data. So that top signature is the signature that says if I am going to run in a WASI environment I expected to be a function called random underscore get that is exported that I can use. And if we translate that sort of that signature into what a C function would look like you can see you will return some int 16 that's an error code of yes no if they're getting reading from the random source works and then there's a buffer and then how long that buffer is which in the WebAssembly land just are a couple of integers that get passed around but as you see within to the guests those are actually pointers into the its own memory and then how much into how long of a buffer within that memory you should write random data into. So hopefully that gives you a good idea of sort of what that interface between the two guests in the host running look like. So we're gonna talk a little bit about downsides to WebAssembly it's not a perfect thing it's getting better all the time but there's still some downsides. One is there's an upstream tooling that's still maturing a great example of this is Golang there's a separate compiler for Golang called TinyGo that compiles down to WebAssembly using LLVM and that can use wazzy but upstream wazzy upstream go the main go compiler can only compile it cannot compile to go that runs in that wazzy environment that I was just talking about. Although it's coming in the next release that I believe is maybe already out or coming this month and also things like interpreted languages. So if you wanna run JavaScript or Python or Ruby inside of wazm you need to package the whole interpreter for that language which ends up getting quite beefy just to run your interpreted languages. All right and then yeah so feature support like I said there's WebAssembly has been built to be modular and backwards compatible so all these new features that have been added to it are sort of like added on to the side and what support is there is like constantly evolving very rapidly even. For example there's like GC and this is needed for languages like Kotlin and Java to be able to do garbage collection very efficiently inside the VM. It's right now only supported in V8 and I believe it's behind a flag although maybe it's an origin trial now and then there's also a big change coming through called the component model. You can read about that but there's it basically changed how you link in the interface between different WebAssembly modules because you can have multiple of these that you upload sort of like shared libraries and access and talk to each other and then the current wazm support is based on a preview snapshot from 2020. So that has changed a lot and made a few different iterations and yeah it's continually evolving. So yeah, if maybe those downsides don't work for you or what are the other alternatives? So there are some languages that have better support for like plug-in experience. One that comes to mind for me is I've worked in a lot in JVM systems is Java has support for sort of loading jars dynamically and other interpreted languages can usually dynamically load code as well. So that can be a good option for some people although usually with those options there's like little to no sandboxing. So essentially when you load that jar it can do sort of anything you want. So as long as it's highly trusted in highly trusted environments you can also kind of push performance a little better because you get all the same performance. There's no like extra VM wrapper in there excuse me, within the JVM. So and that can be an alternative. And then there's also some more specific or niche solutions such as Google's common expression language. So I used to work at Google and on the common expression language itself and it's a great language for things like writing policy or very small snippets that specifically built not to be Turing complete and execute very fast. So if you have certain use cases that don't need the full overhead of Wasm although Wasm VMs are very lightweight generally compared to like something embedding a full JVM into a system you may even want something more on the lightweight version of the spectrum which something like cell is a great use case for that. Cool, so we're gonna switch gears here and talk about the challenges about integrating a WebAssembly VM into Red Panda. Sort of we've been talking about generally what is WebAssembly and sort of how to integrate it into a system and I'll get into sort of the background and lessons we've learned from Red Panda. Before we jump into that I wanna talk about Red Panda. And before we do that I wanna just briefly ask a poll of how familiar are you with Red Panda? Have you heard of it at all? Is this the first time you've maybe heard about Red Panda? Are you using it in development or are you running a production cluster of Red Panda? I'd love to get your answers in the poll and yeah, we'll wait for those answers to kind of come through. Wonderful. And I see there are a few QA questions I will answer those towards the end. Keep those coming in. Great, all right. I think we're good, we'll move on. So now I wanna introduce, yeah for those of you who are not familiar with Red Panda just sort of in 60 seconds give you a sort of idea of what Red Panda is and then we'll talk about sort of the interesting constraints that it has as a system is somebody building and hacking on Red Panda itself. So Red Panda is a high-performance streaming data engine. If you're familiar with Apache Kafka we are compatible with their API. So you can produce and consume records but we are a more efficient and faster and lower cost of ownership version. So we'll talk through some sort of the key tenets that make Red Panda very fast and efficient but if you're unfamiliar with Kafka and sort of the paradigms there Kafka basically breaks out into there are partitions that you read and write records to and each of these partitions are logs essentially and there can be several thousands of these per cluster and each log or partition is a raft group itself. So we run raft usually three members is sort of the de facto number there that replicates each of these logs. So there's a single leader elected all the reads and writes go through that leader and then the leader then will replicate that data to the two followers. And then one of the other key things is we're super easy to get going play with manage and run there's not a bunch of individual microservices to run everything's packages a single binary which really helps with getting running in production. So that's Red Panda in 60 seconds we're gonna walk through sort of some of the more well and also data transfers data transforms of what the scene. So we talked about those two different logs or those partitions. So the main thing about these web assembly data transforms I highlighted this very briefly earlier on when we talked about Flink but you have these individual squares or records within the log they get written and written by clients and then basically web assembly is something that will pull from one log do some transformation and then write different records into a separate log sort of as like a background process running so this gives you the ability to really easily sort of like write data in a different format for example or you can do things like strip out PII or do data scrubbing for certain other parts of your application there's a bunch of very common use case within these big streaming distributed systems is doing these very simple straightforward tasks. Cool. So we're gonna talk a little bit about Red Panda itself sort of and what makes it so fast what makes it so efficient what are sort of the things that Red Panda does how is the system sort of work at a high level? So one thing is that the system's built to be entirely asynchronous. So there's no blocking system calls and we run a thread per core architecture. So this is something that's common in a common sort of architecture in high performance computing environments but basically we run this reactor loop on each individual on a thread that's pinned to a core so that thread never executes on a different core there's no context switching or anything like that always runs on that core we can in sort of Numa architectures we can co-locate our memory and our core together to sort of squeeze out all the performance we can from the hardware. We use a shared nothing model so no memory is shared between each of these threads on startup we'll divide the system memory equally between all these threads and then the threads communicate themselves using single producer single consumer message queues and then it's built using this a lot of this functionality comes in this framework called C star which is a really great framework that we use that handles a lot of this basics for us. So I talked a little bit about this reactor loop that runs so one thing I said everything's asynchronous so everything that runs needs to run for a very short time it uses cooperative multitasking you don't have these big threads that are running that you context switch between and you rely on the kernel to sort of make sure all these threads are making progress is if a single task within a thread runs for 30 seconds that's known what we call as a reactor stall and because during that 30 seconds that runs or even 30 milliseconds really for us is doing this 30 milliseconds no other IO can happen no client request can be served so all of these stalls impact the P99 latencies because there's nothing else that can happen on that core during that time. So what happens as a result of this any long running computations that run need to be broken up into smaller chunks of a millisecond or two to prevent blocking really important IO work that is happening so such as servicing client traffic or reading from disk and things like that. We use a shared nothing allocator so again we talked about we split up the memory for each of these different cores on startup and then we use a buddy allocator on each shard and there's no synchronization between different threads so a lot of the very advanced if you've studied allocators a lot of the advanced allocators do all these interesting thread local and try to reduce synchronization and we sort of do that by design by just saying no thread share any memory at all they all get their own portion of it and then there's no pagecast so we use direct DMA controller writes direct IO to write directly to disks and don't use the kernels page cache at all so it allows us to have full control over the best ways to cache and manage memory ourselves but it's on us to do that, right? The kernel has a lot of great optimizations that have been put on it by many people but we get full control to sort of eke out those last couple of bits of performance. Another interesting note especially for within the context of WebAssembly is that generally when you're writing a C program you can malloc and ask for memory from the OS and it will never really directly fail. It'll sort of slot like soft fail by as you ask for memory in typical situations it'll just performance will start to degrade as you use that memory and page tops can happen and things like that and eventually as you use more and more memory the umkiller can run in but C stars allocators different in that if you all that memory that we divided up on startup that's it that's all you got. So if you, if we have too much fragmentation or exhaust memory we'll throw exceptions and it's sort of on us to either handle those or abort the process or whatever it should be. There's a lot more constraints and each of those generally we run we usually say like at least two gigabytes of RAM per core or per thread that's running so it's generally the sort of model that you run with so it's not a ton of memory that you may normally be used to running with. So we're gonna talk next about sort of how we address these challenges so there's all these interesting constraints about that make red panda really fast but it also becomes a challenge when you're integrating a VM that may not be specifically designed to meet all of these or designed under these sort of constraints, right? So one is like one thing that we did very early on is we built a software development kit to sort of hide the complexities of some of the techniques that we did to users. So for example, one thing is like I said memory usage is very limited and we try not to do large allocations that are contiguous because those can cause lots of fragmentation. So what we do is we do we'll do a lot of things within the VM for users to manage memory well such as pooling buffers and reusing memory when possible and things like that. And we handle that sort of transparently inside the software development kit and give users a sort of idiomatic experience. So this is an example of our tiny go SDK the sort of functions, the snippet of code that you would write to transform or something says you get a record that comes in as XML and you wanna transform that in the JSON. This is sort of a snippet of how you would do that. But yeah, the main thing is like all the complexity stuff of our ABI specification that's all handled and done by the SDK. So the fact that we're, you know writing into some place in memory and you need to call this very specific function that the runtime expects to be there that's all handled within SDK. So users just get this nice single function to write their logic in. One thing for memory like I said because like I talked about fragmentation and sometimes as a long running process goes memory may be fragmented enough that if a VM needs to get 10 megabytes of memory we may not have a contiguous range of those 10 megabytes of memory and all the VMs need the sort of large contiguous chunks of memory. So what we do is we pre-allocate all this on startup for each cores will basically, you know each core has two gigabytes and will reserve some configurable amount of memory that then the VMs get to use themselves and run. And then the other thing that we do that is really neat is there's, well, there's two versions of this. One is some functionality that we expose. So one example of that is we have a registry of what schema your data is currently at and how we implement that within Red Panda is the data for that is sharded across all the cores. So we don't have all these requests going to a single core because again we're doing message passing if you need data and we're not using up all the memory on one core to service all these schemas. So they're in memory and what we'll do is we'll is if a wasm transformation needs to go look up one of these schemas is we'll, we run each of these VMs on a separate stack and what we'll do is we'll I'm using sort of if you're familiar with stack full co-routines is sort of how they work but we'll pause the VM by when it calls into our host function to grab that if we don't have the data locally on our shard we'll pause the VM, suspend it and do other sorts of IO work, client facing traffic, whatever needs to happen within Red Panda for that core we'll hop over to another core, request the data and when it comes back we'll resume the VM with the data now and that's all transparent within the VM there's no async stuff that needs to happen within the VM itself and sort of if we had to do things like network calls and things like that other sorts of async work because Red Panda is an async first system we would use this sort of technique to do that but again this can all be transparent you don't have to write highly asynchronous code so if you're writing something like C or TinyGo you don't have to, it's a very familiar to like a co-routine sort of programming model it's you don't have to burden yourself with understanding like when things sink and have all manage all these callbacks so we do this with stack switching it's really quite fun so we'll talk about, so it's sort of like the challenges that we face with Red Panda and sort of some of the ways we solve them is like how are we going to do some of these asynchronous work without blocking a reactor and causing all these stalls and hide latencies so we're going to talk about what other system other systems that use Walsum and what they use them for so one example is the Envoy proxy so it's sort of if you're familiar with IngenX or Apache these sorts of different network proxies is Envoy is sort of a cloud native first one and it allows you to upload web assembly to do request filters or change requests or route them different ways or do all sorts of custom logic within you can upload a Walsum module to do that another example of Walsum being used is in the open policy agent so this is a, there's a language here called Rego which allows you to write policies of authorization for example of what things are allowed to happen and those can get compiled into web assembly modules so then they can run sort of anywhere within a web assembly VM so you can have custom, these custom policies that are ran without needing to build an SDK and an interpreter for Rego and every individual language Cool, all right and the key takeaways here is yeah, so web assembly can be used to simplify data streaming and you don't have to set up another distributed system within Red Panda and it's lightweight and efficient and then the Wasi standard is a great way if you're not familiar with that to expose as familiar sort of POSIX environments to developers, you wanna hide details that users don't wanna worry about such as what your ABI contract is and how you call into the VM and manage that and then the Walsum ecosystem is constantly evolving but there's already enough to like really build production ready systems in it and integrate this on the server side platforms. Awesome, thanks for joining, we have some Q&A now there's already a few questions at all that are already posted, I'll read through here and answer if you have others, feel free to type them into the Q&A box and happy to answer them. So what is your language of choice to use with web assembly? So one of the languages that we use a lot is Go, we think it's got a great TinyGo specifically right now but we hope to switch to the upstream compiler soon. TinyGo in general is a great language because it's very simple to use and it's efficient because it's compiled but it doesn't force all of the things that you need to do such as manage memory and things and if you're using C++ or Rust so it's like a nice middle ground of like developer ergonomics and also performance that has pretty good support for web assembly. So that's what we started with our SDKs is using Go so you can develop these different streaming plugins and transformations using Go. So next question, are there any projects using WebAssembly extensively that you might know of? So I mentioned two there, those are the two that come to mind besides Red Panda. The next question, can you use Lua and JavaScript with Wasm? So yes you can but again, like I mentioned very earlier is you need to, when you're using these more interpreted languages you have to build an interpreter into WebAssembly and then within WebAssembly VM that's running there's a interpreter that's running that runs your JavaScript code. There may be ways to take your JavaScript in Lua and ahead of time compile it and then run that directly as Wasm but I don't know any efforts for JavaScript specifically and not for Lua. All right, next question is Red Panda SDK takes care of the Wasm API presumably means you have to maintain SDK in each language that you want users to write transform in, how toilsome is this and how hard it is to change your ABI? Yeah, so this is a great question. So it is fairly toilsome, right now we just support Go so it's not that toilsome, we're just trying to hammer out the experience there and we'll support more as time goes on and yeah, there'll be a porting way. I think the main ways that I've seen to reduce the toil and maintaining these multiple SDKs is to try to mirror them as much as possible outside of like language barriers is you basically write them all very much the same so that you can kind of look at your JavaScript and look at your Go and the code looks almost exactly the same is one way. Another technique you could use because again, Wasm everything sort of compiles into Wasm you could potentially write all these in something like C and then integrate them all that way as sort of in Go you see Go well it ends up just all being WebAssembly at the end of the day and then changing our ABI is a great question. We sort of initially or from the start have built in a protocol to sort of manage versions within our ABI so one of the functions we export and assume that a guest function is going to tell us is what version of the ABI you are so we can sort of boot up, run the VM and sort of understand what version they're running at and then we can make sure the right support is there for that and use different host modules to sort of version our protocol as we evolve it so we have not yet had to make a change but it's built in such a way that should be straightforward we will have to support the old ABI's for maybe forever but at least we have a way to make progress and encourage people to upgrade. Okay, next question is Wasm strict runtime limits only apply to Wasm instructions are there any host environment function call have to do their own gas metering? How do you make this work with Wasi? It's a great question so host environment calls do have to do their own gas metering some WebAssembly VMs support attaching a instruction cost to a given module or to a given host function so after you run X number of instructions then you call into some host code and then it will automatically decrement the host code count is like 20 instructions or whatever it is or whatever your gas cost is which is pretty cool. So that's one way to do it. You can also a lot of VMs end up letting you directly access the sort of place where the gas is stored so you can also just in your host function if you have like a dynamic cost you wanna associate you can increment or decrement the gas as it runs but again this is sort of specific to what VM you end up using. Cool and then let's see next question how does Red Panda store and retrieve Wasm code? Do you have pre-execution validation or similar checks before modules are loaded into the database? Yeah, that's a great question. So we store them internally. We have sort of a mechanism for storing things metadata that we store and then we also store the code itself on disk and replicate it so that it's all safe to use and you can access it from any node in the cluster and then yeah pre-execution validation so every time somebody uploads a WebAssembly module we compile it and run it through a few of these checks the ABI checks, make sure we can specify the ABI version and yeah, things like that as they're loaded in that's a great question. And then last one here is what is the compilation model you're using for Wasm? So today we are using a JIT compiler in Red Panda for a variety of reasons. One is the ahead of time compilation would be great for our use case because we sort of expect you to upload something once and we're gonna run that a ton and we want that to be really fast but there's developer experience to sort of consider here when you're also picking a compilation model because if you're uploading this already ahead of time compiled machine code you can have a very easy attack vector there of like you need to make sure you can validate that machine code is only executing within the sandbox constraints and things like that. So that's the reason we're using just in time compilation you can get some of the performance benefits of compiling to machine code but you're doing that on the server and just in time as you need it so there's less like you need to compile this for your right host and architecture and some of all these concerns you can just deploy some raw Wasm from your M1 MacBook and then your Linux x8664 machine can just compile that to the right thing so that's one of our concerns there. Cool, what Wasm runtime are you using in production? So currently we're able to use a couple of different ones we've experimented with VA, Wasm time, Wasm edge of the main ones we sort of like took a little bit VA for a number of reasons doesn't work super well in the constraints that Red Panda has in terms of memory and thread per core architecture. So we've been working with both Wasm time and Wasm edge and are sort of like working between the two. All right, and then another question here. Great, I love all these questions. Does a data transform explicitly materialize the transform log or does it support fuse change transformations? So yeah, that's another great question. So transformations right now are written to that separate log. We don't have support for like ephemeral change ephemerally changing data as it's written or read at the moment although this is something we're actively exploring and looking into. But yeah, so currently the model today is you write into one partition within a topic and then that partition gets transformed into an output topic. And you can also chain these into make sort of like a DAG of transformation. So you could take one, you could write things into one topic from your producer and WebAssembly could change that to another and then you could have another one that changes that back although we don't support cycles because obviously then you'd just be looping data all around through the system. All right, I think we're running close to time here. These are great questions. Thanks everyone for attending. Feel free to reach out here. Walk through all my socials and contact. Feel free to reach out to me and ask some questions on Twitter or LinkedIn or you can email me. Happy to talk more about Red Panda or WebAssembly at all. So again, thank you very much to Linux Foundation and I'll hand it back over to Candice. Thank you so much Tyler for your time today and thank you everyone for joining us. As a reminder, this recording will be on the Linux Foundation's YouTube page later today. We hope you will join us for future webinars and have a wonderful day.


To apply mock objects, we're gonna use Moquito. Moquito is the most popular Java framework for mock objects and I really recommend you to go to the Moquito's website as the documentation is very complete and Moquito is a very powerful framework. So if you want to master it, you will have to read the documentation. So what we're gonna do in our test is to get rid of the real database and mock it. So let's just start by completely deleting the database access part from the test. Then, now that our test doesn't use the database class, we introduce Moquito. And the first thing we do is to tell Moquito which class it needs to mock and in this case it's the invoice database, the one we want to mock. So the mock method from Moquito receives the class we want to mock and you see the dot class because we are getting the definition of the class from the JVM. Moquito returns the same type of the class, in this case invoice data access object. And this is very useful for us because it means we don't have to change our code that much because the mocked class has the same interface as the real class. Of course for Moquito to do it, there's lots of magic going on behind the scenes. But for us consumers of the framework, we can just take advantage of it. The second part, as discussed in the previous video, is about setting the behavior we expect. And for this mock, what we wanted to do is to return a list of invoices when the method all is involved. And this list of invoices can now be just an in-memory list, way simpler than a database. So that's what we do here. We create an array using the arrays.slist method from Java and we add the tooling voices I1 and I2. And then we use moquito.when method and when the all method happens in the mock object then returns this list that we just created in memory. With these two lines of code from Moquito, we now have a class that simulates the database object. And this mock object is able to return this in-memory list when the all method is involved. That's nice. We're almost there, but now we need to do changes in the production code as well. And I'm going to discuss more about it in future videos, but changing the production code to ease the stability is something that we will do quite often in practice. So if we go to the production code, we need to get rid of the instantiation of the database object class. We cannot do this anymore because if we do this new there, this means we are always going to use the real database and that's not what we want. We want our code to use the mocks when the program is being tested and we want the program to use the real class, the real data access object when the program executes for real. A common way to do so is to receive the class we will mock as a parameter of the class. And we usually do it in the constructor. So as you can see in this code, the invoice filter class now has a constructor receiving the database object. And this is nice because due to the polymorphism of the Java language, we can pass any class that is or inherits from invoice data access object in this constructor. In other words, I can pass the real one, the one that I want to be executed in production, but I can also pass the mocked one just for the tests. So our production class is now more flexible. This pattern has a name and it is called the dependency injection principle, which we'll discuss more about it later. We are almost there. So if we go back to our tests, we just need to then pass the mock via the constructor. And in terms of implementation, that said, what happens is as soon as we execute a test, the filter method invokes the all method in the data access object, which is not a mock. And this means that this all will return the least we created in the tests with two invoices. And now we completely got rid of the database and we are able to test our invoice filter class without being bothered by the complexity of having a database in our tests. This is again a unit test. And because of the mock object simulating the database, we can now explore bad weather, corn cases, and write several test cases for this class in a much more easier way. This is what mock objects are about. They help us simulate the behavior of classes so that as testers, we can really focus our energy on the class we want to test and not on its dependencies.