Good for me to start. Cool. Thank you. Thanks everyone for joining me today, my first FluentCon. Happy to be here talking about a topic that I've spent a lot of time with in the last couple of years, but it's not really my background. But I wanted to share with you some of the findings and work that I've seen happening out there in security and how FluentE, FluentBit can be used to build use cases around security and SIM. My background, more of an operations type person, practitioner for a long time. I've run various types of organizations. I switched in my career and became an analyst at Gartner for a few years covering what's called observability today. Back then it was monitoring IT operations. I'm currently the CTO at Logs.io. We're an open source SaaS company. We build a lot of different technologies for operations and security based on an open source platform. We run a lot of different teams there. I'm based in Florida. It's good to be in the West Coast. It's been far too long. So today I'm going to talk about all the great food that we're going to eat in LA over the next few days. I'm sure all of you have very big agendas of Asian food and Chinese food and of course Mexican food that we're all going to be eating. Lots of tacos on my plan because in the East Coast we don't understand Mexican food. So it's great to be back here where the best Mexican food is. But besides that, we're going to talk about SIM. We're going to talk about data collection, parsing and enrichment, things that are really critical to a SIM strategy. Talking about ingestion in storage and then correlation and use cases and users and what I hear and what we hear from security practitioners. How to build your own if you want to do this. Talk to lots of companies that are doing that, trying to do that, various parts of the journey. And I'm going to talk about food a little bit too because that's kind of the theme of what I'm doing in LA this week besides talking tech. So what SIM is, is it's this generally legacy way of trying to collect all the signals from your different security technologies and bringing it together so that you can understand the threat level in your organization. And you try to aggregate it and correlate it because when someone's attacking you or you're trying to understand an attack that's occurring, there's going to be different fingerprints left across different devices. One could call these logs. Events depends on what you want to call them, but generally they're the same thing. You want to store them centrally, analyze them, alert people that bad things are happening, and then folks investigate them. So the tool can help people do threat hunting, forensics, other types of investigation. Some SIMs, especially the newer style, we store raw data. We store all the logs in information. Older systems would correlate it and store an event. Like this was something bad that happened, but I'm not going to keep all the data that goes into it. So SIM doesn't necessarily mean you're storing everything all the time. And we're always trying to figure out ways to make this more efficient because obviously it's super expensive to store all of this security data. So there's a lot of challenges in the market. So there's a lot of different types of food. There's a lot of use cases. I've outlined many of the ones that I hear about regularly. And then a lot of users. These are different users. I know most folks here have different types of use cases and operations, but totally different users. Some companies will tell you DevSecOps is a thing. I can tell you out of roughly 200 of our customers using our SIM. And over 1,800 users using us for DevOps. We have about two customers, one of which is a really large company and one of which is a company and one of which is a company and one of which is a company and one of which is a company owner. So by and large, this is not a thing that people do, even though it would be way more efficient for them to store this data once and use it for two use cases. It's not really what happens because the buyers are different. So like these users down here, they have a separate budget. They have a separate set of needs. And the things that they're doing are also quite different. When you look at some of these and I'll just pick out one of the use cases to give you an example. Let's say threat hunting. One can say how is threat hunting any different than doing analysis of an incident or an outage. The issue is that when I do threat hunting, I want other information to be brought into the tool that gives me more context from a security perspective. What was the attacker's IP address? What network did they come from? What's the AS number? Do I know anything else about when that same attacker did anything to my network? These are totally different things than we think about in ops. So these are some of the challenges. Different use cases, different users. And the market dynamics for those that are interested in SIM, this is the latest magic quadrant from Gartner. You can take a look at almost all of the vendors on here. I think there's one and a half of these vendors that actually build cloud products. Most of them are on-prem, legacy, appliances. This is the market, but yet people spend $3 billion a year trying to solve this problem. It's crazy how old it is in terms of the technology. Proprietary agents, no open source really to speak aside from one company, maybe one and a half that do a little bit of open source. So it's a really legacy market, obviously, right for disruption. And you say, well, you could build this yourself, right? I can use ELK. I can use some of the other open source tools out there. The challenge is that there isn't a lot out there. There's probably two really interesting projects. Elastic's obviously gone a proprietary route over the last year. They don't really have an open source option. LogStash is commonly used. And so we always recommend people use FluentD and FluentBit for these types of use cases. There's also a new elastic search replacement called OpenSearch. A lot of people are starting to use that if they want to keep things in the open source community. There's also a project I'll talk about called The Hive that's a really cool tool for helping do incident management workflow and some enrichment that I'll talk about. But it's not really a sim. So you have to do a lot of things to give you some ideas of how you could do something like that. So similar to a lot of the discussions today about enrichment, some really interesting talks and some great insights that I picked up. Most of the data types for sim are still syslog. We're talking UDP 1980s technology here. And then there's log files. But there's a lot of syslog out there still, especially from network devices.  So this stuff is very old technology in general, what people want you to support in your system. And then there's metadata, which I'll talk about. So there's insecurity threat feeds are critical. There's all these companies that create feeds that say these domain names, these IP addresses, these subnets or AS numbers depending on how you want to look at it constitute botnets and people that are doing attacks on the networks. And with sim, you want to take this data and stitch it in real time. So threat feeds are really critical to sim because they provide great context. And then there's a lot of look up lists, whether it's GOIP that one's easy, we do that. But you can get into some pretty tricky ones around data center management, CMDB. There's a lot of different ways that you want to do look up tables in real time. And Honorog and I have talked about this a lot. He was on a podcast that I sometimes host and look up lists are an interesting topic. And I'll touch upon that. And where I think FluentD and FluentBit can help a lot at helping federate look up lists because right now it's really difficult to do this at scale. And so there's a lot of different data sources in sim. They're usually security or infrastructure focused. And a lot of these are traditional infrastructure. And when you move to the cloud, a lot of this changes quite a bit in terms of the technology itself. But you're talking about the perimeter, meaning the firewalls, the IDS, load balancer is various things like that, proxies, reverse proxies, infrastructure itself and then the host. So you want to make sure that hosts are not compromised. You have to understand the integrity of the host. These are all critical things in compliance. They're critical in security so that you understand when changes occur. And then there's lots of cloud data sources. Everything from our typical cloud providers, to the cloud providers, to the cloud providers, to the cloud providers. And then there's all these different things that are really listed up there to really specific ones like I want to get my Salesforce data and understand how that's being changed. I want to understand my office 365. And then there's so many federated identity systems that people use today. It's a big challenge. And there's more and more of these things. We have a huge amount of plugins and technologies to make this easier. There are sometimes some weird ones where we have to still go to Logstash because they have plugins that do things like office 365 where there aren't good options in the open source world today, unfortunately. And many of these get pulled via API. So if you want to get audit data from Salesforce, you've got to hit it, grab the data. It gets a bit tricky. Some of the newer systems out there, and we do this on our platform, you can subscribe to a stream. And I know AWS pushes this a lot. It's way cheaper for you to subscribe to a topic and get information than querying an API over and over again where they charge you every time you hit it. So stream processing is very new today. We have a lot of people who are using it. Almost no one's doing it today. But I think that that's the way of the future because of the volume of data. And the fact that when we're polling APIs, it's extremely inefficient and expensive. And the cloud providers are making us pay for that. So anytime we can stream data in, we're looking at all the ingredients that go into our guacamole over here on the right, all the important stuff, you have to understand schema. So a big challenge in SIM and in logging in general is understanding what a message is. If I get the same type of message from a firewall, a switch, and a host, how do I know that this is going to work? The answer is it's completely inconsistent, and there's been a lot of attempts at creating schemas. ECS, which is the elastic common schema, is open source. It's one way that people try to normalize the data. Open telemetry, we just incorporated, and I do some work on hotel as well. We just incorporated flexible telemetry. So we're going to be a way to do the same type of thing, normalize the data so that I can say this is a user that's authenticating. And we can start to understand different types of telemetry without necessarily needing to build custom parsers. In security, there's also a ton of frameworks for compliance. Here's a few examples. There's a lot of compliance needs that my organization have, and there's many other ones that are specific to an industry like PCI for payments, HIPAA for healthcare, and it goes on and on and on. When customers come to us and they're under a compliance requirement, they want out of the box reporting, they want best practices in the SIM. It's really hard to implement that. It's really hard to implement that. Building of rules and parsing and views, it's a big challenge when you start to get into compliance. But everyone's under more and more regulation, and that's just the state of security today. So I mentioned enrichment. So the most common one that everyone here deals with is this data. And something that is definitely a big focus for open telemetry. We're just starting to make recommendations of using eBPF and open telemetry to provide enrichment. So there's a lot of great work happening where logs are soon going to be enriched with additional data automatically, and we're going to start seeing more and more information come in. So we're going to start looking at who a user is, what group are they in, what type of employee are they, full-time contractor, external. These are all things that when we enrich the logs with this data, the logs become way more descriptive and useful, and it shortcuts the forensics that we do. And so we start to inject additional data that is in question about the user in question. And these all increase the size of the data, but they make it way more valuable. Similar to spices or herbs as the picture shows here. We put herbs into our food because it gives us a great set of flavor, it gives us way more depth in what we're eating, and this is the same type of thing, so very similar. The other way, the challenge with enrichment is some of this data you want to centralize. So when the data comes in, I want to enrich it as it's before it's stored right on the edge of that. But in many other use cases, it's better to do this at the edge because it's more scalable. So for example, based on the scale of the environment or how much data you're bringing in, how do I update these lists? And a lot of the challenges when I'm looking at threat intelligence or I'm looking at asset information, how do I update that data and keep it relatively real-time? It's a hard problem to solve, either doing it centralized in one point or doing it on the edge. And I think we're going to need to do more and more of this on the edge, which is that we're going to need to do more and more of this on the edge. So I'm going to go ahead and log data, which is something that is not well done today in the industry. And I'll give you an example. If a user authenticates and it's okay, should I keep every authentication? Maybe not. Should I keep maybe 1% of the authentications or should I keep it? Maybe I want to keep it when one user authenticates several times fails and then authenticates. That could show maybe they figured out something with a password or they figured out some other type of two-factor authentication. So we have to think about these use cases. If we do that on the edge, we can actually reduce the data volumes that we're dealing with. So there's a lot of things we have to think about as we move more computing to the edge and how we deal with some of these challenges with enrichment. So lots of problems for the community to figure out and lots of cool features that we can build in open source to make unique ways of dealing with the data. So enrichment I think is going to be a key for them in open source. I think it's an interesting area for sure, especially with cost reduction, cost control. We all deal with these challenges. So a little bit about ingestion and storing. I had to pick like someone eating a taco for ingestion. So the filtering is important. The pipeline is very simple. A lot of folks, us included, use a lot of Kafka to help manage pressure on the back end to do various things in our pipeline. We use it for building machine learning models. We use it for doing real-time alerting to circumvent the fact that we have to store it and then analyze it. We try to analyze things in the stream as much as possible. We have to do a lot of research. We have to search the data in. How do I archive data, restore data? Because in compliance, some of this data like PCI, you have to keep seven years of transactional data. That's expensive. I want to put that on glacier. Keep it really inexpensive. But I need to be able to reconstitute it when I have to. I want to be able to do that when I have to. Providing a lot of archive capabilities in the ingestion pipelines now. I hope that's going to advance a little bit and be more useful for these use cases. But hotel is very early in logging today. We're going to keep working on it and hopefully it will improve over the next year as well. We're going to keep working on it. Some most organizations keep 90 days of log data. That's kind of the standard. If you were to ask me of those customers, what do they keep? It's usually 90 days. They want to be index, searchable, available. That doesn't mean they're going to search it all the time. They mostly are searching five terabytes of data a day. That's the challenge. It gets really bloated. It's a lot of data. Someone sending you 5 terabytes of data a day for 90 days, you're talking about a huge amount of storage. A lot of cost. Some of the things that we're thinking about and some of the capabilities that are going into the open search project which is open source is how do we filter the data for incidents going back a long time? How do we filter this? How do we make it more intelligent in the way that we store? Then, of course, moving things down to lower cost. SSD to spinning disk to S3 to Glacier as an Amazon example because that's where I do the most of my work. Then do I have to actually restore it? How can I make things searchable? We're working with Amazon on UltraWarm capabilities in the open source. This lets you store data in low cost but yet keep the indices in hot storage so you can do some basic searching. It's a big challenge. I think open-source tooling is going to get a lot better here because everyone complains about cost. It's the big challenge correlation. Let's talk about that for a couple of minutes here. So kind of the way that you have to think about data coming in and you can substitute the word event for log is you basically have an event and then you have a set of rules that help you classify whether the event is something that I care about or that I want to be alerted on. There's usually some type of mapping into a framework and then is it going to generate an alert and the challenge is if we just do those first three steps we're going to end up with a lot of alerts. So we have to start doing correlation. Is this alert that I'm about to send related to another alert that I've already sent or that I'm detecting?  And so it starts to get really tricky because how do you normalize all of that data? And today this is usually done centrally but I think it needs to be done in a more distributed manner. I think we have to start thinking about correlation at the edge because it is a big challenge and everyone does it centrally today and I don't think it's very efficient. And it can also take a little while. When you think about you being under attack if the delay is one or two minutes before you know about it they could already have your information. They could have those critical assets already completely controlled. So that's the challenge. It's a scale problem here with correlation and we've been doing it the same way for 20 years in the industry and I haven't really seen it change a lot. So most of the correlation today is rule-based and you can go in the exhibit hall tomorrow and talk to all these vendors that tell you they have some magic AI correlation system and they may have a little bit of more advanced correlation but there's always a rule set underneath and the rule set is generally regular expressions or some type of parsing of strings to figure this out. In open source there is an alerting system that's part of OpenDistro and it does some pretty nice anomaly detection some interesting pattern matching so if you're looking at building something in open source it's a pretty cool way to go but it's still missing a lot of things like multi-query correlation. We've actually built some of this in our product but I don't see anything in open source that does a lot of the more complex alerting today. So there's still a lot of things that you have to build yourself if you want to build it. I think the open source is going to continue evolving. There's a lot of folks working on OpenSearch right now so that's definitely a good thing to keep an eye on. So how does the user use the data? I do want to touch on a couple of use cases here. So how do you simplify the problem which is billions of log messages down to log messages or events down to hundreds of alerts down to dozens of incidents. So that's kind of the goal is we want to simplify the problem, make it digestible to the security operations team and a good example here which is what we call user behavior anomaly detection over on the right. So we have all the authentications. There could be billions of those. We maybe have hundreds of authentication failures and maybe we've got a dozen times where users logged in from two different geographies. And maybe the users connected to a VPN and they're on the wrong one, that's fine. Or maybe the user the credentials have been stolen and there's some nefarious organization in some other country trying to compromise the account. So this is kind of an example of how you distill the problem down and there's lots of ways to try to identify those incidents. So the idea here and this is zooming out so there's SIM and then on the bottom is SOAR. That's basically the operation, that's the orchestration and automation platform for security. So you'll see companies today, many vendors that build SIMs, they are also building or buying SOARs to try to own not only the incident investigation but the automation to try to remediate the incident and that's what SOAR is about. So the idea is the SIM is the intelligence, the brain, it takes all the data, distills it down, creates an incident, opens a ticket or an incident in the tool. And they often have some basic incident management in the tool or you could use an external tool, maybe a service now or JIRA or security specific tools, there are many of them out there that do ticketing and workflow like the Hive, which does that. And then you can create automation. So a lot of our more advanced customers, they want to change a firewall rule, lock down a port, do something on an end user's machine, you would do that with the SOAR in security and that's really where a lot of the automation comes in and how these link together. So if you want to build your own with open source tools, you can kind of try to, and I've deleted a lot of stuff that's secret stuff that we do, but you could try to build your own and this is kind of some of the architecture of our back end systems. It's not easy to do, it takes a long time, how do you scale it and manage it, do all the alerting, everything else in between, never easy. Or you could try to use some open source technologies and do it yourself. Lots of good ways to ingest data, lots of ways that you can put stream processing in place, trying to aggregate it. So most folks that are using open search, elastic search usually run Kafka if they're at scale. So all of this kind of comes together. It's really exciting to see that Kafka is simplifying. I'm sure all of you hate ZooKeeper and it's finally almost going away from Kafka. They just released non-production version of Kafka 3 that has no ZooKeeper. I really want to upgrade our stuff to it because ZooKeeper is a bit archaic to say the least. So I think Kafka is making some awesome improvements thanks to the community and the folks at, who was at Confluent, right? Yeah, Confluent.


Welcome to the OpenStack Summit, Boston. Can you please tell us your name and what you do? Okay, so hi, my name is Adrian Lebr. I am a new year researcher in France. So in AI is the French Grand Nantan Institution in computer science And actually I have been sharing a new initiative which is called the discovery initiative That helps to investigate how you can operate what we call the fog and the edge infrastructures Okay, so what is fog edge and massively distributed computing and why is it important right now? So basically during The last couple of years the main trends what to build what we call mega data centers type a actor like Google, Amazon and Microsoft so the idea is that you will mega data center where you put thousands of servers in order to cope with the demand of cloud computing Unfortunately with the trends of new usage such as internet of things Industrial internet tactile internet this mega data center cannot satisfy the needs in term of latency for such applications So we need to propose a new model that will be able to satisfy all these latency critical requirements And the idea is to deploy smaller data centers, but closer to the end users So the main idea will be to leverage the backbone the internet backbone and to deploy inside each network point of presence a Few couple of servers and those servers will be able to satisfy the needs in term of computations in term of storage for this new kind of applications Okay, can you give us one or two examples of how it would be used in layman's terms? Yes So so the typical Application you can think about is that you have your smartphone and actually your smartphone as you know as some limitation in terms of capabilities So the idea is that now with this new HDTV format all that stuff all this new 3D around the ring What you would like to have is that you would like to have enough capabilities to for example watch your TV your Show in 3D and in that case the idea will be to use actually the computation that will be provided by this edge Infrastructure so instead of running the application your smartphone You will run this application just close to the edge cloud you can have so that's the first Application that we can illustrate the second one will be something related to the medicals So you can think about tele-surgery applications where you have a doctor that want to Do some surgery remotely and in the same in this case the latency is very critical So you also have to provide a cloud computing computing capabilities much closer to this hand users Okay, so what role does open stack play in fog edge and massively distributed computing? So when we create this discovery initiative has a researcher we start from scratch That is we took a white paper and we start to think what can be such a system in charge of operating such a massively distributed infrastructure That is deployed close to the end user usage so we do some paperwork, let's say as we are used to a mansion in research and when we start to think about developing a poke a proof of concept We discovered that actually it will require a lot of engineering efforts and here comes open stack Actually instead of preventing the wheel we choose to join the open stack community in order to leverage How the rich ecosystem and to see how we can extend it in order to cope with our needs So what we decide to do is that we created a decade dedicated working group Which is called the fog edge and massively distributed working group within the open stack ecosystem And actually we try to gather and fed it hold the developers in order to create this new kind Operating system to operate fog and edge infrastructures. Okay, and and and where can our folks join this community? You mean sorry or get more more information. So Right right now as we saw during the opening keynote by Jonathan the fog and edge computing something that's really become trendy In the sense that now we are a lot of actors that are really interested by this new usage. So as I said, you have all this Internet industry all all this Tactile internet all these things around the high o.t. Movements and actually we Look that a lot of company now are really coming and are expecting From the open stack community a solution to operate such Massively distributed infrastructures. So our goal here within the the open stack community is only to try to see how we can combine our effort and how we can bring Such a new open stack vision during the next year Well, thank you for taking the time to chat with us. Enjoy the rest of the summit. Thanks. Thanks for the interview


The slides are available through a link on the conference website. And as well, there's a set of exercises which you can get at by clicking on this link when you load a front page. Okay. First of all, thank you for coming. I have to admit to at least three conceits about this workshop. This first one is the words in the parentheses. The second conceit is that I actually think that I can give a good accounting of APL in 16 expressions. And the third conceit is that I think I can do it in hour and a half. And the fourth conceit is that in the abstract of the workshop, I said that it would be approachable for beginners and yet thought-provoking for experts. There's no use in not setting high goals, eh? So here it goes. The first expression is one of the first, is probably the first APL one-liners ever written. This one here. In Iverson, the inventor of APL had it in his 1962 book, section 1.4. So here is an example of using it. X is a four-element vector. And the expression applied to X gives a one minus one or zero result for each element. And stated conventionally, it would be if X is greater than zero than one, L is if X is less than zero than negative one, L is zero. So it computes what's called a signum of X, the sign of X, for real numbers X. And then shortly thereafter in the book, Iverson used the second expression, which is X times the first expression. And stated conventionally, if X is greater than or equal to zero than X, L is minus X. In other words, the absolute value of X with no branching involved. And what makes this work in APL is that Boolean functions like less than, greater than, equal to, and so forth have values that are zero or one rather than true and false. And the second thing in APL that makes this work is that functions apply to entire arrays. Sean Hughes in his talk referred to this as whole something, whole value. So APL had this 50 years ago. You know, Ken Iverson, the inventor of APL was kind of an impish guy, he goes around causing mischief. So in the year 2000, at the annual meeting of the American Mathematical Society, we had a booth there and Maple also had a booth there. Maple is an algebraic computer algebra system. So we went to their booth and said, how do you find a number of elements of a vector greater than 100? Now he knew very well, they didn't have zero or one, he knew very well they are true and false. So in APL it's immediate, right? How many elements of X are greater than 100? Reduce or plus insert X greater than 100. Now he left the Maple booth a few minutes later and they were still scratching their heads. And then the last thing in APL that makes this work well is a very simple function precedence. Right to left. The second expression is one for computing the average. If you were here this morning at Morton's talk, he also used this example. We call this a train but you may think of it as a functional form, it has three functions. So what it does is it applies the left function and the right function to the argument or arguments getting two results and then it applies the middle function. So average is the sum divided by the number of items. So for example, if X is a vector of 20 random integers between zero and nine inclusive then average of X is 4.4. At this point I'll go to an APL session and do some examples. So X is one E7 for zero. So that X is a vector of 10 million random numbers between zero and one and the average of X. So I'm executing this to give you an idea of the speed of it. It's entirely interpreted, there's no compilation involved as yet. Okay, in this further examples, 0.1 plus 345 reshape of iota 60. So iota 60 are the integers from zero to 59 inclusive. I reshape it into a 3 by 4 by 5 three-dimensional array, you have a question. And then I add 0.1 to it and that's the result. So now I'm going to find the average of various averages to do with. So by default, average applies to the leading axis or the leading dimension of X. So you get a 4 by 5 result but we have a thing called a rank operator which allows you to apply a function to sub arrays. So here I'm saying apply average to the matrix season X. That's what the rank 2 is and getting a 3 by 5 result. So rank is a generalization of map because map, I don't believe allows you to control the size of the sub array that it applies to but here I can specify a number which indicates the rank of the sub arrays that I want to apply the function to. And it works for monadic as well as dyadic functions. For dyadic functions you can specify two ranks, the rank on the left and the rank on the right or just one rank which will apply to both arguments. And you can say average rank 1 which means apply average to the vectors in X and what do you think average rank 0 does? So the average of one number is just that number itself. And this is true for every number in that three dimensional rank. So that's one of the characteristics of APL is that we try hard to get it to work consistently on edge cases. So even though it doesn't make too much sense to find the average of a single number, it still falls out from taking care of the edge cases properly. Okay, I call this the index of selfie. What's a selfie to the man on the street? You take a photo of yourself, right? So here I'm applying index off to an array itself. So this symbol, which Morton also presented in this morning's presentation, we call that index off. It can apply to two different arguments. It doesn't have to be the same arguments on the left and the right. What it does is it finds the index of the first occurrence of an item in the right argument in the left argument. So let's try an example. Oh, I'll just use quad A. So quad A is the letters of the alphabet. If I say quad A, index of B. So it finds the index of F in the alphabet. No, what quad A, V, quad A, sorry. Yeah, and if it can't find it, then it's the length of the left argument. So F is the letter five, O is letter 14 and so forth. And the first letter's first index is zero. Yes, I'm sorry? Yes, yeah. And since space doesn't occur in the letters of the alphabet, it gives you the length of the left argument. I'm sorry? Yes. Not just strings, but numbers, arrays, and boxed arrays. OK, so what I'm saying here is that x index of x is a very useful combination, I mean, a calculation. Because the analogy is that it's like an ID number. So for each item in x, it computes an ID number. And questions of identity on x are very often answered very much more efficiently on ID numbers than on x itself. OK, so here are a few uses of xi over x or x index of x. Finding the unique elements. This second expression is an efficient computation for an inner product of two matrices. And the same expression is an efficient computation for an outer product of two non-simple vectors. And it's also, and the last one is an efficient computation for using the key operator. I don't have the time to go into details of each of these. But we can write these as equivalences, an algebra of functions, because you can write this on the left-hand side, and then a match, and then this on the right-hand side, and with the appropriate arguments, it will tell you that it's the same. OK, x and ty are relational tables, names, sex, country code, and age. And index of works on this representation of relational tables, just like we saw for strings and letters of the alphabet. So for example, the first row of ty is rows. So this is row 0, 1, 2, 3. And then this is row 1, and so on. And the last two rows of y don't occur in Tx, so they're indices of 5. And these are selfies, Tx index of on itself, and Ty index of on itself. Now this way of representing relational tables is quite inefficient in APL, because each of these guys in the box is a proper array. And arrays in APL are self-describing, so each of the guys in a box has a 32-byte overhead of information about the array itself. So you can imagine it's quite inefficient for a scalar number like 29 to have a 32-byte overhead. So a more efficient representation of a relational table is what's known as an inverted table. So what you do is you take all the values of column 1 and collect them together as one single, in this case, one single character matrix. All the letters in column 1 collected together as a character vector and so on. And they're very short functions for converting from one to the other. So invert will convert the nice-looking table into an inverted table. And to get it back, you apply the invert function or the uninvertion. So this kind of thing is very common. You check it out by using match. So x is this, and this expression just shows that an inverted table is the same number of items in each column. And this comma bar, each of x, displays the inverted table x in a very nice form. And this slide just shows the space comparison between an inverted table and the ordinary table. So tx is the one with all the boxes. It takes 920 bytes. And x, the inverted table, takes 272 bytes. And I do 1,000 times replicator of the table with all the boxes and invert it. You see the uninverted form takes 880k bytes. And the inverted form takes 55,000 bytes. So it's a factor of 16 in this case. So not only that, but you get hit in time for the uninverted form. So this is showing that in the inverted form, if I do a comparison on the column, it's 14.77 times faster than the form with all the boxes. And in this example, the size factor is factor of 40. So now, how do you find the index of each record? Well, in the ordinary case, you just say tx index of ty. Here, we can't do that, because here I'm looking for this is an item. So it's looking for this item of y and x. And it's not going to find it and so on. But you can get the index off on tables by first uninverting it. I'm a lazy type, so I don't want to write uninvert. So uninvert, to me, is vert. So if you vert x and vert y, you can find the index of the records. But suppose you have a huge table, like maybe one with 100 million or even a billion rows.  So here's a derivation of an efficient index off for an inverted table. So we saw in the previous slide that you can vert the inverted tables and then do the index off, and it works. So the second line just substitutes the definition of vert into the expression. Substitute is a word I've heard quite a few times in these couple of days. So I'm just substituting definitions. Now the next line is the realization that this funny symbol n closed rank, negative 1, is just a scalar encoding. And I said that for questions of identity, quite often the questions are more efficiently answered on the ID number, which you get by index off, than on x itself. So I'm using, in other words, I can use x index off to get a scalar encoding, which works just as well for questions of identity than on the data itself, which is n closed rank, negative 1. So here's an illustration of how it works on column 0. So I pick out column 0 assigned to x0, and closing rank in it, negative 1, I get John, Mary, Monica, Min, and Max. And index of x0, I get those numbers. And then for y, same thing. If I pick out column 0 of y, and then closing the rows, I get that. And then if I do x0 index of y0, I get those numbers. These integers, when you're talking about identity, work just as well as these guys. And then this, the other stuff, you have to learn a bit more of APL to know what they're doing. What happens is that if you do x index off each, you get a four, in this case, you get a four element vector of integer vectors. And the upward pointing arrow mixes them, forms a four row integer matrix. And then I transpose it to get a four column integer matrix. And then index off works the way we want it for integer matrices. So here's an illustration of this intermediate result. So transpose of mix of x index of each y, I get that. Likewise, for x index of each y, I get this four column integer matrix. And on the left-hand side, that's how you can compute index off if you have enough space. I invert each of the inverted tables and apply index off. And here, you don't have to un-inverted table. You compute these indices and then use index off. So this is one of the advantages of functional programming that John Hughes talked about this morning. You can do formal manipulations on programs. They give the same result. But this one is very much more efficient. Next one is bar chart. So the argument x, in this case, has six elements, which are integers. And I want to plot a bar chart of those numbers. So 3, 1, 4, 1, 5, 9. The way I did it was the intermediate steps on the intermediate results on the right-hand side. I find the maximum of x, which is 9 in this case. And then the integers from 0 to 9 minus 1. And then I do a commonly called outer product. But I like to refer to this as a table, function table. Because everyone knows function table. In school, you learn addition. And they give you an addition table, and multiplication, and so forth. So here, we're computing a function table using greater than as a function, getting 1s and 0s. Function tables are a good way of looking at even functions that you know, and especially on functions that are new to you. For example, how do you know what does a function table of a commutative function look like? Yes, it's symmetric. When you transpose it, it will be the same. So you can describe that to kids, and they would get it. Because you flip it, and it's the same function table. That's what commutative means. So you can try it. You know greater than and greater than, equal to, equal to, and so forth. Try function table even on those functions. I think you may learn something. I learned this expression for computing a bar chart when I was first learning APL, too many decades, than what I wanted to say. And at that time, sorry? Yeah, yeah, yeah. But it's one line. And it was already in the APL milieu at that time. It's something everybody knows, right? And this expression drove home to me the point that in APL, the index or the subscript can be an entire array. That wasn't obvious to me. But after looking at this, you say, ah, yeah, yeah. Next expression is one for computing all the permutations of the first-end integers. Now in TDD or test-driven development, the methodology recommends that you start with the simplest case you can think of and then go for it from there. You know, but I like to start in the middle. The reason is that in the very simple case, you may have particular properties that don't hold in a more general case. And you have to kind of backtrack and go for it again. So here are all the permutations of order 3. You get it however you get it. And then try to generate all the permutations of order 4. So you kind of sit down and look at it. And here's something that comes close. What you do is you add 1 to all these permutations and then can't lay a 0 column, assign it to Q, and then use Q to index into 0, 1, 2, 3, 1, 0, 2, 3, 2, 0, 1, 3, and so forth. Now if you collapse these together, then you have the answer. Those are all the permutations of order 4. This is one of the what I call benchmark problems that I have, a personal set of benchmark problems. It's the benchmark not in the sense of timing, not only in the sense of timing, but it's a benchmark to test my own understanding. And more importantly, test on advances in the APL language. So the thing on top is the best I could do in 1981. And it's pretty close to the best that you can do in APL at that time. This thing is as bad as it looks because it means go to. So go to, if 1 is greater than or equal to n, then go to line 0 or exit and so forth. And in these two lines, it's looping around. But the idea is all there. It's using t, which is the permutation of order 1 less, and then indexing into a vector. So this year, the best I can do is the function on the bottom. Now, a little bit about notation. It's a recursive function. And these are what's called guards. For a guard, if the expression to the left of the colon is true, then you return the result as a result, the expression to the right of the colon before the next diamond. So if n is 0, then you return 1, 0, reshape of 0. Otherwise, you do this thing on the right. And the notation is that this upside down triangle means recursion. And having that way of specifying recursion means you can have recursion on anonymous functions. So whatever name you assign to the function doesn't matter because this thing already indicates recursion. So go back to the previous slide. If we collect together these vectors that I'm indexing into, 1, 0, 1, 2, 3, 1, 0, 2, 3, 2, 0, 1, 3, and 3, 0, 1, 2, is what I'm trying to generate on the bottom here. So the intermediate results are compute a function table using equal as the function. In other words, the identity matrix. And to that, I apply gray down rank 1. So gray down are the indices that you need to sort the thing in descending order, right? Yeah, because if you're sorting down, the first index should be 0, and then the rest. And then the second row, if you apply gray down to this, the first index should be 1 because you want to pick the one first, and then the rest. And then 2, and then the rest, and then 3, and the rest. Of course, there are other ways of generating a matrix like this, but I live for expressions like this. And then finally, you use q. What q and q slide to index into this matrix, rank 1. Again, as I said before, rank is a generalization of map because I specify this indexing function. I can specify that it applies to vectors on the left and vectors on the right. And then this stuff is to collapsing the three-dimensional result into a matrix result. And it's not the last word. This is a version in 2016. Some of these sub-expressions can be improved. But the language, the APL language, cannot yet express the thought that I have. OK, next slide is the next expression is the parentheses nesting. In other words, the def or nesting by parentheses. So yeah, so x is this character string representing an expression. You can see that when you see a left parentheses, you increase the count by one, another one, and then remains the same. And then when you see a right parentheses, right parentheses, that decrease by one and so forth. Here it increases again, stays the same, decrease by one, decrease by one, and then so on. And these subsequent lines are intimate results that shows what's going on. You find the index of x in this two element vector of the left and the right parentheses. So you can have a 0, 1, or 2 result. Use that to index into a three element numeric vector, 1 minus 1, or 0. And then do a plus scan on that. And you get exactly the result that you calculated. Now, if you remember from the first slide, there's another way to compute this expression. If x is equal to left parentheses, you get a 0, 1 result. Minus if x is equal to the right parentheses, you get another Boolean expression. If you find a difference, you get the 1 or minus 1. And then if you do the plus scan, you also get the same answer. So you write another function, depth 1 or something, and then you can compare these two programs. And they should be the same. But I prefer this form with the indexing because it generalizes to when you have more than two possible choices. So you can use index off to get a result which can be greater than 2. And then index into possible, index into a longer vector than just three elements. There's actually an illustration of computing a function because this pattern of indexing into something and then using the indices to index into something else is another way of computing a function. You're probably not old enough to remember this. But when the new math curriculum came in, they introduced functions as a set of order pairs. And there was a lot of criticism. What is this new math? But I remember that lesson because I know that a function can also be considered as a set. And sometimes it's more efficient to do table lookup. This is what it is, this table lookup than to actually compute it using other ways. Quick sort. It was mentioned in one of this morning's presentations. So I quickly looked it up. And Haskell also has a very short quick sort. It'd be interesting to compare the two. Anyway, here it is. And this version of quick sort is what we call an operator. What we call an API operator. Or higher order function. Because it takes a function. First of all, it takes a function argument to produce a second function. And then you apply that function to arguments. So what the function argument is, in this case, minus, is the thing that tells me the ordering when I compare elements. So what Q requires is that the result of the operand function has to be less than 0, equal to 0, or greater than 0. Conventional. So here I have random integer vector. If I specify minus as a comparison function, that's it. So in APL, you wouldn't actually sort this way, because there's a more primitive means, which are highly optimized and much faster than this. But the purpose of this one line function is to, well, notation of the two of thought. It helps you to understand what is this quick sort thing. OK. Carrying on with Q. So here's a function, operand function, for comparing and closed strings. So if they are equal, if the two strings are equal, I return 0. Otherwise, I compute something whose details I will not go into. But it'll be negative if alpha is less than omega. And it'll be positive if alpha is greater than omega. And it can sort and close character strings. Again, Q is a recursive function. So again, there is a guard here. If the number of elements in omega is 1 or 0, then return omega itself. Otherwise, recursively, that's the upside down delta triangle again, bell, apply to the items of omega, which are less than 0, catnated with the items that are equal to 0 in the comparison, catnated with the items that are greater than 0 in the comparison. And the comparison is done on a randomly chosen item, what's known as a pivot in this algorithm. OK. Now the application of this, which is called order statistics, suppose you want to find the 47,000th element in the sorted order. How do you do that? Well, one way to do it is to sort the whole thing and then pick out the element that you want. So here, S1 is an expression or a function for computing that, sort the whole thing and then pick out the gray the whole thing to be more precise and then pick out the appropriate element of the gray and then use that to index into the argument. So in other words, or for example, the zero order statistics would be the minimum. If you use index, which is the index of the last element, then it will give you the maximum. And if X is an odd number of elements and you specify the middle, it will be the median. And here, S is an operator using the quick sort partition idea to compute the order statistics. Same answer. So how about timing? So S1 is the one that sorts the whole thing and then picking out the right one. This one is using the quick sort idea. And this one, hardwires the comparison to make the timing fair, because this is still using an operand function to do the comparison. So here's X is a million random integers between 0 and 1 billion. And then finding the 4E5 or 4E5. Anyway, 4E5 element of the sorted vector and doing a timing on it. CMPX is a utility for doing timings as well as comparison. So it will give you the timings in the form of a bar chart. And as well, if one of these results are not the same as the first one, it will put an asterisk on it. So it is both a tool for comparing equality or matching as well as on timings. So you see that SN, this one with a fixed comparison, beats the one that sorts everything, hands down. Golden ratio. We saw scan before, the plus scan. And it's a higher order function or operator. So if I give it the function plus composer divide or reciprocal, plus chart, reciprocal just means x plus reciprocal of y. So of course, that computes a continue fraction. If I apply it to the 16 element vector of all 1s, I get those numbers. It happens to converge to the golden ratio. Compute it exactly as that. Of course, it's not exact, because girl root is not exact. I've set up the default in my session to display six figures. But you can get it to display all the figures that it has available, which is 16 or 17 or 18, depending on the number or the computation. So it's IEEE 64-bit flows. And then if I compute the LCM of these flowing numbers and one, surprise. Or maybe you're not surprised. But what you get are the Fibonacci numbers. And the last one is the equivalent expression in J. J has rational numbers with infinite position. So you apply plus composer reciprocal scan on these rational numbers, you get exact results. Again, with the Fibonacci numbers showing up, I have 203.45, right? Yeah, OK. Good shape. I'm sorry. OK. That's one of the goals of this workshop. Inner product. John mentioned that it is workshop this morning. And other people have mentioned it. Now, the classical way of doing inner product is you take a row. You mentioned that in your workshop too, Martin. The classical way and the way they teach you in school to do inner product, you take a row in the left argument, apply to each column the right argument, and so on. So every row against every column. But this expression also computes inner product. Again, it's like comparing equivalent functions. So in this case, for those arguments, they're the same. But this comparison is actually more powerful than application on any old argument. Because it's applying it to random arguments. So unless you're unlucky or the algorithm is unlucky, you're not likely going to get that truth on random arguments. So what it is is using rank, this generalization of map again, applying two vectors on the left and matrices on the right. So what it's doing is what I call row at a time. It takes a row of the left argument, apply it to the entire matrix on the right, and then pick the next row, apply it to the entire thing on the right. And the benefits are manifold. It has very beneficial cache effects. As you know nowadays, in modern computers, cache is king. So in the row by column way of doing inner product, the matrix doesn't have to be very large. And to compute the result, you can get a cache miss on every element of the result. And I'm saying that for row at a time, this way of doing things. It doesn't have that undesirable effect. Also, you can take advantage of zeros in the left argument because the sort of checking for zero is amortized over the entire row of the right argument. So you can check for zero too also in the row by column way of doing things. But there you're checking on every element of the result. So it's not really cost effective, aside from other detrimental effects in the execution because you're doing checking and then probably branching in the machine code. And lastly, if the left argument is Boolean, in other words, zero or one, or even has a small number of possibilities, then you can do this table lookup thing that I mentioned before because the number of possible answers is small when the left argument has only a small number of possibilities. OK, so I'll illustrate the workings of this algorithm. First of all, row by column. Now, dot, the operator that's using plus dot times is a proper operator in the APL. So I can give it any old function. And quite often, it's quite illustrative to use n close of alpha omega as a function. So it's taking every row in the left argument and applying it to every column in the right argument. So it's 20 against 0, 21 against 5, 22 against 10. And then the next column, the same row, 20 against 1, 21 against 6, 22 against 11, and so on. And here, I'm picking a second row of x, 23 and 0, 24 and 5, 25 and 10. And then the next column, 23 and 1, 24 and 6, 25 and 11. In contrast, for a row at a time, so here I'm using alpha length of omega for the function. Just looking at what rank 1, 2 means. So it's taking the first row of x, applying it to the entire of y, and then taking the second row of x, applying it to the entire of y. So rank 1, rank 2, rank 1, rank 2. And then if you look, you focus in on it, giving it a more complicated function. So inside the function, there's a rank negative 1 there. So this part has the same structure, but now I'm looking inside each of the rank 1, 2 application to see what it does if I give it a rank negative 1 thing. And so it's using, in each row, it's using each scalar, because rank negative 1 means the subarray having one rank, one less than the rank of the array. So it's 20 against the first row of x, y. 21 against the second row of y. 22 third row, y, and so on. Morton. Right, this is, yeah, I mean, no, no. Yeah, good point. The interpreter, the interpreter, the implementer, having understood what this thing is doing, then wrote his C accordingly. And then, of course, in the QA, you're going to compare that expression against the plus dot times. And if you had a row by column implementation, you'll also compare it against the row by column implementation.  Because if you haven't understood that and you start writing a C, you get all mixed up, get it in the details. There's one final interesting story about this. Like, we've coded this row at a time method for several years. And then in the current release, we switched to Visual Studio 2015. Having used Studio 2005 for all those years. And we finally switched over to the new for us, Visual Studio 2015 compiler. And inner products sped up by a factor of 3.3 with no changes to the C code. And when we looked at the machine code that it generated, it's using vector instructions. Which, for comparison, we also coded explicitly the row by column implementation. So for that one, it's not able to use the vector instruction, but for this row at a time implementation written in C, there's no way for us to tell the compiler that, hey, we're doing row at a time. But it recognized that that's the situation. It can use the vector registers more efficiently. And we got a factor of 3.3 with no changes to the C source. Some days, you earn your pay, right? That was one of the days. Katie's theorem. So I still have a lot to go, so I'll just quickly go through this. T are all the non-singular Boolean matrices. I'm using them because I want to do groups. And I don't want a commutative group. So these are the non-singular Boolean matrices. And if you form the group table, again, here's the table again. Using Boolean matrix multiplication, you get the group table. And then if you take this group table and find the indices of each element in the ravel of the table, you get this bunch of numbers. And the point is that when I relabel, I'm not changing the group theoretic properties. It's like I'm using a different font to display each of these guys. And if I'm using a different font, surely it doesn't change the group theoretic properties, right? So you look at this table. And for a group table, necessarily, the rows are permutations. Otherwise, it's not a group. And then if you look at each row, i.e. if you split the matrix into its rows, you get all the permutations. And what are some natural things you do with permutations? You index. So you find a table using indexing on these permutations. You get another table. So R of G is this. And then you compute the table using indexing. And then you relabel it again. Lo and behold, these are the same. So that is Katie's theorem. Every group, finite group in this case, is isomorphic to a subgroup of the group of permutations. How many characters? That's a very succinct way. You can see yourself discovering Katie's theorem if you have this available to play with on your computer. I discovered this myself in 1987 because I had APL to play with. And I thought I'd discover something, but actually it already has a name, Katie's theorem. Discover or proven in the mid-1800s. Symmetries of the square. If you have a permutation, you can form a Boolean matrix from it and get the permutation back. And the significance of the Boolean matrix is that if you index using the permutation, that's the same as matrix multiplication using the corresponding Boolean matrix. So actually there's an isomorphism between permutations and Boolean matrices. And what are some things you can do with matrices? You can rotate them, flip them, doing mirror images. So in this chart, those are the eight possible versions of the square that you can get using rotations and reflections. And in each on the edge of display, the operation that I had to use to apply to the matrix in the upper right-hand corner to get that image. And if you apply that operation on Boolean matrix, getting another Boolean matrix and then converting that result back into a permutation, that's the thing that you can use to get the corresponding permutation. So this is a quick way of proving lots of identities on these functions on permutations. So this is saying that if you look up the table, this row and this column, and the claim is that this composed of that, you get the identity. So this composed of that, you get the identity.  And apply grade down three times, you get grade up, composed of reverse. There's a quick APL expression, one line, that I can use to check what I just said. But in the interest of time, I won't execute it. But it's in the exercises. You can try it in the privacy of your own home and see that it does work. Interval index. Now we're thinking of adding this as a primitive in mixed version of APL. And of course, we want to model it. And that's the model, or simplified version of the model, and get our users to try it to see whether they like it, to find out what applications they have. Typically, we like to work that way because you can write down the specs in words till you're blue in the face and give it to people. They can read it, whatever. But until they can try it, it's not very effective. So what interval index is, the left argument is a sorted list. And being sorted, it divides the universe into particular partitions, the universe into non-contiguous intervals. And the interval index finds the interval of the index that contained an item of Y. Yeah, and it works for strings and closed character vectors and so forth. Anything that's ordered. Now, we get to this funny number. This past summer, I had the opportunity to have an intern. She's a high school senior. She's an aspiring mathematician. So what better book to recommend to her than Hardy's, a mathematician's apology? So I thought I would read the book again before recommending it to her. And I came across this statement. The number of primes less than 1 billion is that number. No, what happened to the utility confused exactly that. Negative 1 to function encoding. PCO of 1E9, I get the number of primes less than 1 billion is 50,847,534. Both of those numbers cannot be correct. They could be one is right, the other is wrong, or they can both be wrong. So I did a Google search. It turns out this number is right. And there's a pictures description of the wrong number. It's called, it has a name, even though it's wrong. It's called Bertelsund's number, which Mathwell says is an erroneous name, erroneously given the erroneous perhaps he was funny. So even though the internet says this is right, how do you know that's right? I mean, for me, I also have the additional dilemma of already giving the standard spiel to the summer intern that you've got to prove everything. So how do you prove a number like that is right? Well, one thing you can do is to prove, correct the program that computes it. So here's a very short program for computing it. I won't prove it here, but it's very easy to prove. Compute bit vector having length n. You can use the bit vector to compute the actual primes. You can add it up to get the number of primes less than n. But if you sum it, it doesn't work. Work space full or not enough space. It's likely that it will remain work space full for a long time yet because it generates a Boolean matrix of size 1E9 by 1E9. So not only do you get killed with space, but it will take a while to compute the answer. So here is kind of funny. Here's a program you can prove correct, but it can't compute the result. Isn't that funny? So what I did was I, oh, PCO itself, I can try proving it correct. But it would be a daunting task. It has about 20 lines and there's a couple of long tables in it. So I wrote from scratch an efficient program likely to be correct, likely to be amenable for proving. And it also says that. And the optimizations that it uses are instead of doing the marking for each number, it only does the marking for non-prime multiples of each prime less than the square root of n. And multiples of 2, 3, 5 are initialized by a more efficient method because it's an optimization because if you get rid of all the primes that I mean all the numbers are multiples of 2, 3, 5, you already got rid of a high proportion of the numbers. And then subsequently, you only need to do the odd multiples of primes greater than 5. And multiples of a prime, to be marked, can start at a square. Looking at this function, the hardest line, I think the hardest line to prove correct would be this line, the m line. Because it's the number of times you have to mark each number. Why is it the hardest? Because in computer science, the two hardest problems are naming, the two hardest problems are naming, garbage collection, and off by one error. So I've traveled all this way. And I'm only 200 miles from Kumbakuna. Is that how you pronounce it? Kumbakuna. So of course, I have to talk about Ramanujan. So this little expression started when Jay, who was here last year, sent out an email saying, who is to the development group? Who is interested in going to see this movie? And Fiona says, ah, Mr. 1729. I'm in. And John is in, and so am I. And shortly thereafter, Nick Nikolov, another of our colleagues, found a block entry by Wolfen talking about saying that in Wolfen language, the language formerly known as Mathematica, in Wolfen language, you can compute partition P of 200 instantaneously. I'll explain what partitions are in a minute. But that sounds like a challenge. So can I compute partition P of 200 instantaneously in APL? So here are the partitions of the integers from 0 to 6. Notice the 0 case. Again, APL tries to handle edge conditions. So me too. So these are all the partitions of integers from 0 to 6. So for example, 4, how many ways can you write 4 as a sum of vectors of positive integers? So it's 4 itself. This is a vector, by the way, it's not a scalar. 3, 1, 2, 2, 2, 1, 1, and 4 ones. And the order does not matter. So how many ways can you do it? Well, I solved this problem before using Euler's pet pentagonal number theorem. This thing, you can find out on the internet. And you notice it's recursive, but not only that, it's highly, multiply recursive. Because P occurs in it, in itself. So here, the guard is 1 is greater than n, or omega. Then the answer is 0 less than or equal to omega. Otherwise, you recurse. And REC computes the numbers that you need to recurse on. So Pn of 0 to 6 are the number of partitions for 0, 1, 2, 3, up to 6. And Pn 30 is 5, 6, or 4. So I'll just go into my APL session. I already have Pn defined. So Pn is here, then REC is here, and Pn of 30. Ticks a while. Now, the trick with live presentations is that when something ticks a while on your system, you keep talking to distract the audience to make it seem not as long. But it's going to come back. Trust me. But the time it takes shows you that you shouldn't try this on 200. And I'll show later on why Pn of 200 is not a good idea to compute. I'm waiting. No, no, it's plugged in. It does. There it is. OK, so why is it not a good idea to apply it to 200? Well, I'll show you why. Because it's going to use REC compute. Argument that it's going to apply recursively to getting those. And for each of these guys, it's going to recurse. So on 199, it's going to recurse on those guys. And then after it's done that, it's going to recurse on 195, and then 188, so forth. So what to do? Well, the memo operator was mentioned yesterday. We call it an operator because it's a higher order function. So the key thing with the memo operator is that f of m is the same as f. That's no compromise with that. But it keeps a record of arguments and results for reuse. It is commonly used for multibuy recursive functions. Sounds like just the ticket. So here's an implementation of a memo operator in APL. I won't try to explain the details, but the key thing is that it takes your definition, in this case, pn, and then inserts some code in it to do the memorization. OK? So what's next? Yeah, so pn memo of 30, pn memo of 200, let's do that. Yeah. So that's the answer for pn of 200. You can check it against the previous slide, see that's the same answer. Yeah, how long would pn of 200 have taken? Well, there's a way to estimate that by counting the number of function calls. So pnc is the associated number of function calls. It has the same form, same recursive form as pn itself. So I can apply memo to it and get the number of function calls. And so pn of pnm using memo tells me that's the number of function calls for 30 and for 200. I timed the pnc of 30 before, it was 35 seconds. It's not too far off. You saw it taking however long it took. And then extrapolate. So with the extrapolation, it would have taken 1.31e42 number of seconds. Or 4.15e34 number of years. In that number of years, not only would your computer have crumbled to death, but the universe would be well on its way to entropy death. I'm so happy because I have time to do. Oh, yes. Oh, no, this is still part of it. In J, a dialect of APL, it has extended position numbers. And it has the memo operator built in. So pn of memo of 1,000 takes 52 milliseconds. And it gives you the exact answer there. So the answer to the challenge is that pn of 200 can be computed instantaneously in APL for some suitable definition of instantaneous. And the memo operator, even the model that I have, works on anonymous functions as well as functions with non-scaling results. If you go into the exercises, there's couple of exercises on that. Finally, so glad I can do this expression. It's Euler's identity in the opinion of many mathematicians the most beautiful equation in all mathematics. 0 is equal to 1 plus the exponential, exponentiation, you know, e to the pi times 0j1. So in one expression, it combines the fundamental quantities of 0, 1, and 0j1, or i, the square root of minus 1, and the operations plus times and exponentiation. So I'm going to prove this by first proving Euler's formula on sine and cosine. Now this is not yet executable in APL because of this infinity. There's a 30, more than 35-year-old proposal for adding infinity to APL, but it's not yet implemented. Because for some strange reason, there's no commercial call for having infinity in APL. So what these are are power series coefficients on the three functions, e to the 0j1 times omega. So it's 1, 0j1, minus 1, 0j minus 1. Repeat it, weight it by factorial i whole to infinity. Similarly, for sine, it's 1, 0, minus 1, 0, 1, 0, minus 1, 0, and so on. And sine is 0, 1, 0, minus 1, and so on. So in other words, if you get rid of weight, which is factorial i whole to infinity, you get those numbers. And you sit there for a while and it doesn't take too long. So you'll notice that if you multiply s by 0j1 and then you add it to C, you get identical vectors. And since those series are absolutely convergent, you can reorder the way you add them instead of add across, which is what the power series is saying, you're adding down before comparing. You see that exponential is cos plus 0j1 times sine. In other words, order is formula. So order is formula again. Pi, cos of pi is minus 1. Cos of sine of pi is almost 0. And then on the right-hand side, if you add it up, you get minus 1, almost minus 1. And on the right-hand side, you have order's identity, 1 plus exponential pi times 0j1. If you don't know, here I'm showing off, because that's 0. It's quite unusual, because often what you get is a very tiny number, but it's not exactly 0. The vicissitudes of falling point numbers. So if these 16 expressions are not enough for you, I'm working on a book called A History of APL in 50 Functions to appear on November 27, 3 o'clock in the afternoon, because that's when the first workspace was saved in APL. And when it's the 100th anniversary of APL, I'll write a history of APL in 100 Functions. Any questions? Yes. Yeah, that is one of the things. But you notice in this APL session, we have these checkered things.  You can just click on the things on top and get their character. Yes. And that's multi-metric? Yeah.  Anyway, the keystroke you need and a brief explanation of what the symbol is to jog your memory. And so this is matrix divide, so forth. OK? Yes. No, we don't have dictionaries as a primitive type, but you can easily implement it by using index off and then indexing. So you would have an array of the words of interest through index off into the words and then index using subscript or indexing to get the value that you want. So that would be a dictionary. Yes. The question is, one of my examples has using indexing to subscript into a table to get speed up. And the question was, can I do that on, I guess, an expression somebody enters on the keyboard? Yeah. Yeah. That's a relation. Yes. No, it doesn't do that on the fly, but it has that in some primitives. So if you're forming one by integers, for example, there are only 256 possible answers. So it's not going to laboriously format all those numbers. It's going to index into a table. Yes. What would the motivation for using J, a dialogue, a API? Oh, at the time, many of the things I could do here were not possible in APL yet. And for some strange reason, after I joined dialogue, these things started to show up in APL. Yes, in the back. Yes. The question is, is there a version of APL that replaces these funny characters using normal characters available on other keyboards? I guess you can use J, which is ASCII only, but using digraphs, or sometimes even trigraphs, but the ideas are the same. I tell you. Yeah. Yes. OK. Sorry.


Now, the third topic is abdominal mass and role of ultrasound. As I said before, for many topics, there is no protocol put forth. So, my aim is to put forth a protocol for this abdominal mass. So, mass can be due to an antenatal scan. So, an antenatal scan, they could have picked up a mass in the abdomen, so it may come for a scan. Or clinician may have felt a mass or abdominal distention or a previous scan report would have said a mass or when you are doing scan for some clinical symptom, you pick up a mass. Now, this is a case of an antenatal scan reported as abnormal mass and now you see a mass in the right adrenal area and color Doppler there is flow and that was removed turned out to be a neuroblastoma and more common one is a no variances antenatal scan and now postnatally you see that it is complicated and there is lot of debris inside. So, that is no variances which is complicated. Now, role of ultrasound in a mass is confirmation of the mass and to know the organ of origin or if not possible a plane of origin and characterization of the mass whether it is benign or malignant and if it is malignant extension of the mass and then complications due to the mass. Now, to know the organ of or plane of origin these are the various signs described will go one by one. Now, first one is the pattern recognition. So, pattern recognition is once you are seeing any number of scans and moment you do the scan by the pattern you are able to tell the diagnosis. So, one example is the hydranophenosis. You see cystic mass in the area of the kidney you see dilated calluses and the dilated pelvis. So, you know that it is hydranophenosis as opposed to a multi cystic kidney where you see multiple cysts of varying size non-communicating and you do not see any parenchyma.  So, these are some examples of pattern recognition the moment you see that you know the diagnosis.  So it does not produce any effect on the adjacent organs it goes around and with acute angles that means it does not respect tissue planes and here you see the colon and all around the colon you see a cyst. So, this is typical appearance of retropaltonic lymph angioma. So, this is again pattern recognition and now this is a 55 year old man there is a painful mass in the right iliac force this is the appearance you get. So, this is the convex probe so many times you may not get features on the convex scan but if you shift to the high frequencies scan you get the pattern recognition what do you see here you see multiple layers so this is called the cut on in appearance. So, cut on in appearance in the right iliac force it is a pattern recognition agnostic of musinus stadenoma of appendix because of temporal deposit of musin. Next sign is called the intra-organ sign that is the mass you see but you see that it is well within the confines of an organ. So, there is no doubt about the origin of the mass because it is within the organ this is some examples assist within the liver, multi-cepted assist within the liver and hypoallergenic mass in the liver angiomyelipoma in the kidney a calculus in the urinary bladder and a mass within the urinary bladder. So, this is intra-organ sign next is phantom organ sign where you see a large mass arises from a small organ and as a result the organ is not seen. So, here you see right kidney in the right adrenal area you see a mass and you do not see the adrenal separately. So, that is we diagnose adrenal mass now coming to the beak side. So, how is it produced? Now, this is the organ you imagine that it is ovary that is a mass arising from the ovary it keeps on enlarging. So, when it is becomes much enlarged than the organ the rest of the organ gets compressed and forms a beak over the mass. So, this is called the beak sign. So, if it is present we know that the mass is arising from the organ. So, that is the example the ovary and you see a large ecopoam mass and you see the compressed ovary with the beak sign on either side. So, this is the beak sign confirming that the mass is arising from the organ. Some other examples of beak sign in the kidney are multiple masses in the kidney and you see the beak sign in all of them. So, that is the beak sign showing that the mass is arising from the kidney. Now, claw sign. So, claw sign is multiple beak signs. So, here you see the kidney and the mass and I will convert the picture to see better similar to the claw. So, that is the kidney and that is the mass and you see the claw sign the beak on both sides. So, that is the claw sign confirming that the mass is arising from the kidney. Now, this is the reverse beak sign. Now, you see the organ and the mass, but in between you see the groove that is the reverse beak. So, when you see this that is see that suggests that the mass is not arising from the organ. They are close together, but not arising from the organ. So, this is the ganglion neuroma of the retroperitonium anterior to the kidney and you see the reverse beak sign. Now, here to stress the reverse beak sign, I will show in this example, the two month old female infant, previous ultrasound focal mass of the left clove of liver, mangyondotheliuma. Marscap reported as a tichal cystic lesion, segment 4, kori mesangkaimang hamatoma. So, referred for repeat ultrasound, you see the liver and you see the mass. So, this is looks like a necrotic mass and you see the reverse beak sign. So, that means it is not arising from the liver. Again another section you see the reverse beak, there is a mass and the liver which is compressed. So, that is reverse beak sign, so it is extra hepatic. So, what is it? Again you see is high frequency, you see the surface of the liver, there is a crater. This is the mass, this is the necrotic substance and so that is an abscess and you see the crater. So, what has happened is a ruptured liver abscess with an extra hepatic collection. You see the wide gap in the diagnosis by using signs and using the high frequency scan particularly in children. So, it is not at all a tumor, it is just ruptured liver abscess. Now, the another sign is embedded organ sign, here what happens is the an organ is embedded within the mass. So, here there is a bowel loop which is embedded in a large mass, a ecopower mass. So, this is a eccentric lymph node mass of lymphoma, embedding the loop of bowel. Here there is a mass around the kidney and embedding the kidney, this is the case of perinephoric hematoma. Now, this is central abdomen, you see a large septated cystic mass, but in the center you see gas filled bowel. And in the perpendicular to this section, you see the multi-septated cystic mass and you see a loop of bowel going through that. So, you know now you know the diagnosis, it is an eccentric lymphoma. You can see the real time, you see the whole mass moves indicating that it is intraabdominal and multi-septated and you see the peristalsis in the bowel loop, in the center. So, this is embedded organ sign, this is eccentric lymphoma. Next sign is floating iota sign, what is it? It is displacement of the abdominal iota away from the vertebral column. So, that is the iota, it is displaced from the vertebral column. This happens only when there is a later paternal lymph node mass lifting up the iota. So, it is a sign of retropytonic lymph node mass. Then you diagnose mass based on the location and shape. So, here you see a cystic mass in the region of the gallbladder in the real gallbladder force and you do not see the gallbladder separately and it is piriform taking the shape of gallbladder. So, it is dilated gallbladder. Now, this is a mass in the renal force, it has got the shape of the kidney and so it is an enlarged kidney. Then we come to the sign of continuity.  So, continuity confirms the diagnosis. Now, here you see a large cyst in the right flank and you see the kidney and you see a mild dilated renal pelvis and there is a communication of the renal pelvis with its large cyst. So, this is what is the appearance. You see the kidney, you see the dilated chelicis pelvis which is communicating with the huge dilated, grossly dilated upper ureter. This is happens in a ureteric valve with cystic dilation of the upper ureter. Differential diagnosis will be a retropytonial cyst. Now, to stress on the continuity sign, I will show you two examples identical two children presenting Spain abdomen, pre both the cases previous scan reports said pseudo cyst. Case 1 and case 2, you see the cyst.  But from where you see the transverse colon here. So, the transverse colon and the wall of the transverse colon and wall of the cyst are continuous. So, continuity, sign of continuity says that it is a duplication cyst of the transverse colon. Whereas, in the case 2, in the high frequency scan, you see the cyst, again gut signature, duplication cyst, but this is the stomach and you see the wall of the stomach and wall of the cyst are continuity. So, it is a gastric duplication cyst. You can see how this has helped us to differentiate. Then tissue planes, so parietal masses will be seen in the abdominal wall as seen here a lipoma and since this of course is clearly within the abdominal wall and when you have a doubt, a real time will show that the mass does not move with respiration whereas intraabdominal structures will move. Parietal structures will fail to move confirming that it is a parietal mass whereas here there is a mass which is intraabdominal as evidenced by the respiratory movement. You see the nice movement shows that it is intraabdominal, then you can further characterize and then another sign is sign of exclusion. So, suppose see here, this is a child with a coronal scan of the left flank, you see a mass here, so this mass in the region of the left adrenal, so what are all possible? It is it can be mass in the left adrenal area and this is a transverse scan of the left flank and this is a transverse scan of the again the transverse scan of the left flank at a little higher level. So, it can be arising from the spleen, but you see the spleen normal, so it is not arising from the spleen. It is it can arise from the kidney, but again you see the left kidney, so you see the mass is not arising from the left kidney. It can arise from the adrenal, but you see again the adrenal is normal and it can arise from the tail of pancreas, again in this section you see the tail of pancreas which is normal and stomach again looks normal in both the sections. So, all the possible organs are excluded that narrows down to the mass arising from the retropyrtonin. So, this is by exclusion. Then effect of the mass, so here you see a mass in the mid-dabdomen and you see a dilated loop proximal to that and you see the loop entering the mass. So, by this you know that it is a mobile mass with obstruction with a dilated loop proximal. Here similarly hydronephrosis and you see the dilated ureter and the mass, so this is a ureteric tumor producing obstruction and hydronephrosis. The effect on adjacent organs, so this is a young girl with a mass in the supra-pubic region and the sagittal scan shows the bladder normal and post it to the bladder, you see a large cyst and the high frequencies can a little above shows that the cyst and the uterus anterior to it and you see the service within the cyst confirming that it is a hematoculpus. Another case you see the hydrometroculpus, so this effect confirms that it is hematoculpus. Now here you see in the mid-dabdomen, you have a multiceptated cyst which is a left kidney showing hydronephrosis and the cyst is compressing the ureter and producing hydronephrosis, so that confirms that it is a retropotone lymphoma compressing the ureter. Now next sign is feeding vessel sign or the fruit on the branch sign. Now here patient presenting with melena, you see mass above the urinary bladder and high frequency shows again the mass, so whether it is a mass or a fecal mass, you put on the color, you see a large vessel supplying the mass. So single vessel supplying the mass and branching. So this is the feeding vessel sign confirming that it is a polyp in the colonic polyp and this is also called the fruit on the branch, so that is the fruit on the branch. So that vessel is the stalk of the fruit, so that helps us to diagnose the polyp. Then origin of the mass started out by color Doppler, you will see some examples. Now this is the kidney and you see a mass and you see the reverse beak, so suggesting that the mass is not arising from the kidney, again here reverse beak. But when you put on the color, you see the feeding vessel is from the renal artery, so it is arising from the kidney, but it is almost like a adunculated exophytic mass from the kidney. Another mass in the left iliac fossa anterior to the sauvus muscle that is again longitudinal scan, but on color you see it is supplied by a branch from the external iliac artery. So that confirms it cannot be from the intraabdominal structure, so that narrows down to a retropyrtonal mass. Now here in the transverse scanner you see a large irregular mass and in color shows that the celiac artery is within the mass, diagnosis of celiac lymph node mass. Then another example, a mass in the region of the pancreas, the supramacentric vein is anterior. So if it is a pancreatic mass, then it should be, the supramacentric vein should be compressed, but in transverse scan you see by the vessels, this is IVC posted to the mass supramacentric vein anterior to the mass. So in between the IVC and supramacentric vein is the uncenate process of the pancreas. So the mass is arising from the uncenate process of the pancreas. There is a large mass in the mid-abdominal, colored opera shows supramacentric artery causing through it, making a diagnosis of mercentric lymph node mass, similarly an echogenic mass in the mercenary as evidenced by supramacentric artery into mercentric fibromidosis. Now there is a large mass in the right iliac flow cell and if you put color, you see the externally of vessels are pushed anteriorly by the mass making a diagnosis of a retropyrtonal mass, most likely lymph nodes. Now here again kidney with a mass, the reverse big sign, but on colored opera the flow is from the kidneys, so it is a renal mass. Now here it is a newborn with a large mass in the region of the pancreas. So how to know the origin of the mass? First is the deodorant. You see confirmed by the fluid that it is deodorant, second the iota and the IVC. So the mass is anterior to the iota and IVC and you see the supramacentric vein. So the mass is surrounded by deodorant IVC, iota and supramacentric vein. So it has to be in the head of pancreas and it is heterogeneous and in the newborn so it is a pancreatical blastome. Another case of mass and high frequency source that it is related to a small bowel loop and it is continuous with the small bowel loop involved and colored opera shows flow from the bowel loop making it as a gist from the bowel. Now some maneuvers you can use to diagnose. Now this is the left loop of where you see a mass. So it can be from the left loop, it can be from the stomach, it can be from the spleen and so if you see the stomach there is a beak of the stomach so it could be from the stomach so you give oral fluid. You see the fluid moving going all around the mass instead of the mass making a diagnosis of with the beak that it is a gliomaoma of the stomach. Similar mass but here you see the stomach being posterior so it can be from the left clove of liver but you see the beak here.  And this is a mass in the left hyperchondrium stomach was seen separately but you see gas inside the mass so when you give oral fluid the fluid goes into the mass confirming that it is an exomorphotic mass of gist from the stomach. Now here post uretroscopy there is a large cystic mass with hydranophrosis and you see that there may be communication so you press release with the color Doppler you see flow away and flow towards across the breach so in the ureter so this is a trauma to the ureter with a urenoma. You have to seek a mass in these situations now here in a patient with hematuria you see a clot so close look at the kidney shows that there is a mass and bleeding PR you see a mass here and you see gas all around so whether it is a fecal mass or polyp you put on the color you see flow inside making it a polyp. Bleeding PR you see a mass here with gut signature whether it is mass or duplication or meccals and for normal loop it is a peristaltic so you repeat the scan after one hour it remains the same making that it is abnormal and it is a meccals diverticulum. How Mach bilius vomiting you have to look for the mass of hypertrophic pylorextinosis in bilius vomiting you have to look for the volvulus as seen here the whirlpool sign of midget malrotation and volvulus and in red current jelly with cry you must suspect intercess option so you see intercess now clinically you suspect a mass but you do not see a mass and ultrasound now here this is the kidney and you do not see a mass in the right flank but now this is normal you see the distance posted to the kidney this is normal here it is increased so maybe you are missing and you see gas there so this gas is preventing visualization of the mass so little scan with the tilt of the patient you see that there is a large abscess with gas which is prevented because of the gas shadowy and coronals can make out the abscess so this is a posterior perinephric gas with abscess there is a child with constipation and a mass in the suprapibic reason but in ultrasound you see this can be gas in the dilated rectum but in transverse scan there I mean there was resistance felt for the transducer and transverse scan shows a huge dilated rectum with resistance so that is a fecalith this is a 40 year old woman fullness and mass felt clinically but you fail to see a mass this is a very important situation but there is resistance for the transducer so when you observe you see a rotatory movement indicating that there is a mass and mimicking bubbles so this is a cystic teratoma and imagine if you have not reported this and the surgeon operates now mimicker for a mass so here you see the urinary bladder posted to that you see a cystic mass but this is due to reverberation artifact because of gas in the rectum how to avoid you repeat the scan after some time or go to the laterally and do the section and you can another pointer is you can see the distance it will be equidistant and the pelvis will end there but you see the mass extending beyond the contentment of the pelvis you must suspect a reverberation artifact now then we come to characterization that is a cyst solid ecogenic heterogeneous and high frequency is very useful as seen here you see the kidney which was reported as solid and cystic mass in retroperitonium but here you see the big sign showing that it is intra-renal and the reverse big sign that it is also extra-renal it looks like abscess so what has happened is there is an intra-renal abscess which is ruptured and produced a perinephric abscess you see how much detail you get and this is a mass in the left globe of liver with high frequency you see that it is typically hemangioma with enhancement renal mass bilateral high frequency shows that it is classically heart somal recessive polycystic kidney it is a patient with acute pain right iliac closer there is a large mass we see that there is a tender mass of momentum fat ilium and appendix showing that it is an inflammatory mass of appendix now here there is a mass in the adrenal area and you see the broken diaphragm sign because of the difference in speed of ultrasound and that tells us that the mass contains fat so the diagnosis becomes myelolipoma here there is a so necrotic mass with color flow you so know that it is necrotic tumor now this is an ecopo mass and with enhancement so you think of cyst but when put on color there is flow inside making it a tumor and there is a cyst in the tail of pancreas which put on color if there is flow that indicates aneurysm of the splinic artery now here there is a mass in the liver in a newborn there is dilated hepatic vein and the portal vein indicating that it is a hemangiopendothelium of the liver now some maneuvers can characterize the mass here you see a mass in a patient with hematuria whether it is a mass or a clot to change the position of the patient shifts so it is clot now here a woman with lower abdominal mass there is this solid mass anteriorly with shift of position it also shifts indicated only mass which can float on fluid is fat so this is fat that brings us to a diagnosis of cystic tartaroma you see the shift in real time now characteristics may be useful if you know the age history and echo pattern now this is a child with urinary bladder you see a mass inside and with the change in position it shifts so you diagnose calculus but the history is ureteric re-implantation done three months back so within three months you cannot get such a big calculus so that brings us to a diagnosis of foreign body depth over in the urinary bladder this is assist in the pancreas with the history of acute pancreatitis so it is pseudosis and once you are diagnosed malignant mass you have to look for extension now this is renal carcinoma the breach of the perinephric fat line perinephric infiltration and when there is no perinephric infiltration the mass moves well with respiration whereas here you see that the liver moves but the kidney is fixed indicating that there is perinephric infiltration renal carcinoma there is infiltration and thrombosis of the renal wing no flow on color Doppler and extends into the IVC also so by this you know the extension here again a renal cell carcinoma extending into the renal pelvis along the ureter up to the bladder and again renal cell carcinoma with a distant hepatic metastasis now a case of hepatocellular carcinoma infiltrating the portal vein with thrombosis and a metastatic lymph node in the portaepatitis and lymphoma with involving the bowel as well as lymph nodes and we come to the complications of mass now this is the renal cell carcinoma with haemorrhage you see the perinephric hematoma and this is the hepatocellular carcinoma on the surface which is bled with a local hematoma and this is the angioma lipoma which is bled with a retroperitone hematoma now this is a movement with the distention of abdomen and pain you see asitis and in the lower abdomen you see this collapsed ovarian cyst so this is ovarian cyst which is ruptured and this is a renever abscess which is ruptured with asitis and hemoperitoneum this is a hemoperitoneum with a clot and you see a spleenic artery aneurysm which is leaked and complicated now this is a case of pain abdomen and vomiting there is a mercentric cyst as evidenced by a bowel loop but patient has presented with acute pain so that this is not explained by this diagnosis but you see this area so a close look shows that there is nemoperitoneum and asitis so what has happened is look a little more medial to that you see the whirlpool sign indicating that the mercentric cyst has resulted in volvulus producing the perforation of bowel now this is a child with ovarian cyst and with hairs indicating cystic teratoma but the patient has presented with acute pain and vomiting so you have to look for any complication and you see this area with high frequency that shows the snail shell sign of whirlpool sign and in real time you see the whirlpool sign indicating that it is ovarian cyst torsion ultrasound is also useful in follow-up after surgery for melignancy is a case of carcinoma of left kidney and follow-up shows local recurrence and it can be lymph node metastasis or it can be recurrence at the urethral cycle junction or there may be recurrent hematoma in the IVC so to stress the role of ultrasound in a mass I will show this example as the concluding case the four-year-old boy with pain in the left flank for 15 days his earlier ultrasound has been reported as iodide exist which will unusual for four-year-old and CT scan was reported as large peririnal complex cystic mass with a few mildly enlarged parietic lymph nodes so missing camel melignancy has to be ruled out this is the coronal scan transverse scan you see a large cystic mass in and the kidney is compressed here so whether it is renal or extra renal this is the real time showing that the mass moves with the kidney so maybe arising from the kidney or infiltrating high frequency transverse scan of upper pole shows the big sign showing that the mass is arising from the kidney sagittal scan shows the reverse big sign so it may be misleading sometimes and there are solid areas showing that it is a tumor and an high frequency I mean color shows flow within the septa showing that it is not a hydrat it says and it shows low resistance flow making it more melignancy and high frequency show medial to the mass shows enlarged metastatic lymph node so the final diagnosis will stir up with lymph node metastasis so this is how the various modification of your technique and various signs will help to arrive at a final diagnosis in an abdominal mass thank you very much for your patient listening